A method and device for achieving distributed transaction consistency

Abstract

The present invention discloses a method and device for achieving distributed transaction consistency, which belongs to distributed, wherein the method comprises: when a first system receives a first remote call request to the first system sent by a second system, caching the first remote call request; if the first system performs a corresponding transaction operation according to the first remote call request and an abnormal state occurs, then the cached identifier of the first remote call request is placed in a preset retry queue; according to a preset reading time, the identifier of the first remote call request is read from the retry queue, and the first remote call request is obtained according to the identifier; and the first remote call request is resent to the first system. The present invention ensures the consistency of the distributed transaction processing results by retrying the first remote call request with an abnormal transaction operation state.

Description

Technical Field

[0001] The present invention relates to the technical field of computer systems, and in particular to a method and device for achieving distributed transaction consistency. Background Art

[0002] In a microservice environment, a transaction request may require the coordinated operation of many systems to obtain the final result. These systems need to be accessed through remote procedure calls (RPC), which involves network consumption and may cause the client and server to have different processing states for the transaction. Inconsistent states will cause the final request result to deviate from the actual result.

[0003] Monolithic applications can achieve transaction consistency by using database-based transactions or spring-encapsulated declarative transactions. However, in a distributed application system in a microservice environment, due to the isolation of applications, applications communicate with each other through the network, so database transactions and spring declarative transactions cannot work. In addition, due to the complexity of network interactions in a microservice environment, it is difficult to achieve complete data consistency like in a monolithic application environment. Therefore, transaction final consistency becomes a better choice in a microservice environment.

[0004] In order to achieve the final consistency of transactions, the prior art provides a two-phase or three-phase transaction submission mode. The two-phase submission is divided into a submission application phase and a submission execution phase. In the application phase, the transaction coordinator asks each transaction participant whether the submission can be made. If all participants respond in agreement, the coordinator performs the second-phase submission execution, otherwise a rollback operation request is initiated to each participant. If a participant fails to respond successfully in the second phase, a rollback operation request is also initiated. Based on the two-phase submission, the three-phase submission adds an inquiry phase to the application phase and handles the timeout. The coordinator initiates an abort request to the participant for all timeouts. This method splits a transaction into multiple phases to complete, which improves the success rate of transaction execution. However, because the coordinator is involved, network nodes are added, and the uncertainty risk of network communication between the transaction coordinator and other transaction participants is also increased accordingly. In addition, the mechanism of inquiry and waiting for response also causes loss of overall timeliness and low efficiency.

[0005] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section. Summary of the invention

[0006] An embodiment of the present invention provides a method for achieving distributed transaction consistency, the method comprising:

[0007] When the first system receives a first remote call request to the first system sent by the second system, the first remote call request is cached;

[0008] If an abnormal state occurs when the first system executes the corresponding transaction operation according to the first remote call request, the cached identifier of the first remote call request is placed in a preset retry queue;

[0009] According to a preset reading timing, reading the identifier of the first remote call request from the retry queue, and acquiring the first remote call request according to the identifier;

[0010] Resend the first remote call request to the first system.

[0011] In one embodiment, after the step of resending the first remote call request to the first system, the method further includes:

[0012] If the first system executes the corresponding transaction operation according to the resent first remote call request and an abnormal state occurs again, the transaction consistency failure information is sent to the operation management platform; or

[0013] If the first system again encounters an abnormal state when executing the corresponding transaction operation based on the resent first remote call request, repeat the following steps until the number of times the first remote call request is resent reaches a preset threshold: put the identifier of the cached first remote call request into the preset retry queue again, read the identifier of the first remote call request from the retry queue again according to the preset reading timing, obtain the first remote call request based on the identifier and resend the first remote call request to the first system again.

[0014] In one embodiment, after placing the cached identifier of the first remote call request into a preset retry queue, the method further includes:

[0015] In the retry queue, marking the delay time corresponding to the first remote call request identifier and timing it;

[0016] The step of reading the identifier of the first remote call request from the retry queue according to a preset reading timing includes:

[0017] The first remote call request whose delay time is 0 is read from the retry queue by periodically reading the call request whose delay time is 0 in the retry queue.

[0018] In one embodiment, the first system executes a corresponding transaction operation according to the first remote call request, including: a transaction operation executed within the first system or a transaction operation implemented by remotely calling a third system;

[0019] The delay time corresponding to the first remote call request identifier is determined according to the duration and / or frequency of the transaction operation processing performed by the first system or the transaction processing performed by the third system according to the remote call.

[0020] In one embodiment, resending the first remote call request to the first system includes:

[0021] The first remote call request is sent to the interface corresponding to the first system by remote calling.

[0022] The embodiment of the present invention also provides a device for implementing distributed transaction consistency, comprising: a retry component module, a retry service module, a request cache queue module and a retry queue module; wherein:

[0023] The retry component module is used for caching the first remote call request sent by the second system to the first system in the request cache queue module when the first system receives the first remote call request sent by the second system to the first system; if the first system performs a corresponding transaction operation according to the first remote call request and an abnormal state occurs, then putting the identifier of the cached first remote call request into the retry queue of the retry queue module;

[0024] The retry service module is used to read the identifier of the first remote call request from the retry queue according to a preset reading timing, and obtain the first remote call request according to the identifier; and resend the first remote call request to the first system.

[0025] In one embodiment, the retry service module is also used to, after the step of resending the first remote call request to the first system, if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, send information about transaction consistency failure to the operation management platform; or if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, repeat the following steps until the number of times the first remote call request is resent reaches a preset threshold: put the identifier of the cached first remote call request into a preset retry queue again, read the identifier of the first remote call request from the retry queue again according to a preset reading timing, obtain the first remote call request according to the identifier and resend the first remote call request to the first system again.

[0026] In one embodiment, the retry service module is specifically used to, after placing the cached identifier of the first remote call request into a preset retry queue, mark the delay time corresponding to the first remote call request identifier in the retry queue and count the time; and read the first remote call request whose delay time is 0 from the retry queue by periodically reading the call request with a delay time of 0 in the retry queue.

[0027] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for achieving distributed transaction consistency when executing the computer program.

[0028] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for achieving distributed transaction consistency is implemented.

[0029] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for achieving distributed transaction consistency is implemented.

[0030] The above method and device for achieving distributed transaction consistency provided by the embodiment of the present invention caches the first remote call request when the first system receives the remote call request of the second system, and delays and retries the pre-stored first remote call request when an abnormal state occurs in the subsequent transaction operation, such as when the transaction status is unclear. On the one hand, pre-storing the first remote call request can prevent the loss of request information caused by interruptions in the transaction processing process, and facilitate the subsequent retry of the first remote call request. On the other hand, the retry of the first remote call request also ensures the consistency of the distributed transaction processing results. The above method can be directly implemented in the application system (such as the first system, etc.) without relying on an independent network entity. Compared with the prior art, the modification to the original application system is relatively small, the compatibility is better, and the processing efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0032] Figure 1 One of the flow charts of the method for achieving distributed transaction consistency in an embodiment of the present invention;

[0033] Figure 2 This is a second flowchart of a method for achieving distributed transaction consistency in an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of the interaction between various systems in an example of an embodiment of the present invention;

[0035] Figure 4 A flowchart corresponding to an example in an embodiment of the present invention;

[0036] Figure 5 It is a structural block diagram of a device for implementing distributed transaction consistency in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0038] The present invention belongs to the field of distributed technology. In order to solve the problem of transaction consistency of distributed systems in the prior art, an embodiment of the present invention provides a method for achieving distributed transaction consistency. In order to facilitate the description of the method, different application systems involved in the same transaction processing in a microservice environment are distinguished by using "first system", "second system" and "third system", etc. The above names do not represent the order of precedence, primary and secondary relationships, etc., but are only for distinguishing different systems.

[0039] Reference Figure 1 As shown, the method for achieving distributed transaction consistency provided by an embodiment of the present invention includes the following steps:

[0040] S11. When the first system receives a first remote call request to the first system sent by the second system, the first remote call request is cached;

[0041] S12: if the first system executes the corresponding transaction operation according to the first remote call request and an abnormal state occurs, the cached identifier of the first remote call request is placed in a preset retry queue;

[0042] S13, according to a preset reading timing, reading the identifier of the first remote call request from the retry queue, and obtaining the first remote call request according to the identifier;

[0043] S14. Resend the first remote call request to the first system.

[0044] In this step S14, for example, the first remote call request is sent to the interface corresponding to the first system by remote calling, that is, the second system is simulated and the first remote call request in S11 is sent again.

[0045] The above method for achieving distributed transaction consistency provided by an embodiment of the present invention caches the first remote call request when the first system receives the remote call request of the second system, and when an abnormal state occurs in the subsequent execution of the transaction operation, such as when the execution state of the transaction is unclear, the pre-stored first remote call request is retried with a delay. On the one hand, pre-storing the first remote call request can prevent the loss of request information caused by interruptions in the transaction processing process, and facilitate the subsequent retry of the first remote call request. On the other hand, the retry of the first remote call request also ensures the consistency of the distributed transaction processing results. The above method can be directly implemented in the application system (such as the first system, etc.) without relying on an independent network entity. Compared with the prior art, the changes to the original application system are relatively small, the compatibility is better, and the processing efficiency is higher.

[0046] In one embodiment, after the step S14 of resending the first remote call request to the first system, the method for distributed transaction consistency provided by the embodiment of the present invention refers to Figure 2 As shown, the following steps may also be performed: S15. After the retry fails or the number of retries reaches a specified number, the transaction consistency failure information is sent to the operation management platform. Specifically, that is:

[0047] 1. If the first system executes the corresponding transaction operation according to the resent first remote call request and an abnormal state occurs again, information about the transaction consistency failure is sent to the operation management platform;

[0048] 2. If the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, repeat steps S12-S14 until the number of times the first remote call request is resent reaches a preset number threshold. For example: put the identifier of the cached first remote call request into the preset retry queue again, read the identifier of the first remote call request from the retry queue again according to the preset reading timing, obtain the first remote call request according to the identifier and resend the first remote call request to the first system again, determine whether the number of times the first remote call request is currently sent reaches the preset number threshold, and if so, directly send the transaction consistency failure information to the operation management platform.

[0049] Sending the information of transaction consistency failure to the operation management platform can facilitate the business management personnel of the operation management platform to further locate the problem and handle subsequent problems.

[0050] In one embodiment, after the above S11 puts the cached identifier of the first remote call request into a preset retry queue, the following steps may also be performed: in the retry queue, the delay time corresponding to the identifier of the first remote call request is marked and counted.

[0051] For example, the delay time corresponding to the first remote call request identifier is marked as 10s, and the timing starts after the marking is completed until the delay time reaches 0.

[0052] Correspondingly, in the above step S13, the first remote call request with a delay time of 0 can be read from the retry queue by periodically reading the call request with a delay time of 0 in the retry queue.

[0053] In the specific implementation of the embodiment of the present invention, in the above S11, the cached first remote call request can be placed in a cache queue; when it is determined that a first remote call request needs to be retried, it is placed in the retry queue. In this way, in the retry queue, there may be more than one first remote call request identifier that needs to be retried, and the delay time corresponding to each first remote call request identifier may be different (the time of being placed in the retry queue may be different, and the delay time may also be different). Each time it is read, only the first remote call request identifier whose delay time is 0 is read, and the content of the first remote call request is read from the cached queue, and then retried again.

[0054] The above-mentioned first system executes the corresponding transaction operation according to the first remote call request. In the specific implementation, there may be multiple situations, such as the transaction operation executed within the first system or the transaction operation realized by remotely calling the third system. The transaction operation executed within the first system includes the reading operation of its own database, etc. If the status is abnormal or unclear, it will also trigger the retry process of the first remote call request. For example, the first system needs to remotely call the third system according to the first remote call request. The remote call to the third system may cause network abnormalities or the status returned by the third system is unclear. This situation will also trigger the retry of the first remote call request.

[0055] The delay duration of each first remote call request that needs to be retried may also be determined based on the duration and / or frequency of the transaction operation processing performed internally by the first system or the transaction processing performed by the third system based on the remote call. A longer processing duration and a lower frequency may allow a longer delay time to avoid unnecessary resource loss caused by retrying the first remote call request too frequently, and is also beneficial for providing different retry strategies for different services during the retry process.

[0056] The method for achieving distributed transaction consistency provided by the embodiment of the present invention is described with a specific example. Figure 3 As shown, in this example, there are three application systems, namely system A, system B and system C. System B has been improved by adding retry components, redis request cache queue, redis retry queue and retry service, and operation platform.

[0057] The retry component is mainly responsible for pre-storing request information and putting the request information into the retry queue after communication exceptions; the redis request cache queue is mainly used to cache pre-stored request information, and the retry queue is mainly used to store the request information ID that needs to be retried; the retry service is mainly responsible for periodically obtaining the request information to be retried from the retry queue and sending the request information through RPC.

[0058] Reference Figure 4 As shown, the overall process is as follows:

[0059] S41. System A calls the interface of system B through RPC;

[0060] S42. After receiving A's request, system B first caches the request information in the redis request cache queue;

[0061] S43. System B then processes the normal business logic of the request and calls the RPC interface provided by system C;

[0062] S44. If the RPC network of system B calling system C is abnormal or the status returned by system C is unclear, the retry component puts the request information ID in step 2 into the redis retry queue and marks a certain delay time in the queue;

[0063] S45. The retry service obtains data with a delay time of 0 from the redis retry queue at regular intervals, that is, the ID of the request information, and obtains specific request information from the redis request cache queue based on the ID;

[0064] S46. The retry service calls the interface provided by system B through RPC, which is equivalent to simulating system A to resend the request in step 1;

[0065] S47. If the number of retries reaches a certain number or an error occurs in step 4, the retry failure information needs to be sent to the operation management platform, and the operation management platform will display the failure information to relevant business personnel.

[0066] The present invention also provides a device for achieving distributed transaction consistency, as described in the following embodiments. Since the principle of solving the problem by the device is similar to the method for achieving distributed transaction consistency, the implementation of the device can refer to the implementation of the method for achieving distributed transaction consistency, and the repeated parts will not be repeated.

[0067] Reference Figure 5 As shown, an embodiment of the present invention provides a device for implementing distributed transaction consistency, including: a retry component module 51, a retry service module 52, a request cache queue module 53 and a retry queue module 54; wherein:

[0068] The retry component module 51 is used for caching the first remote call request sent by the second system to the first system in the request cache queue module when the first system receives the first remote call request sent by the second system to the first system; if the first system performs a corresponding transaction operation according to the first remote call request and an abnormal state occurs, then putting the identifier of the cached first remote call request into the retry queue of the retry queue module;

[0069] The retry service module 52 is used to read the identifier of the first remote call request from the retry queue according to a preset reading timing, and obtain the first remote call request according to the identifier; and resend the first remote call request to the first system.

[0070] In one embodiment, the retry service module 52 is also used for, after the step of resending the first remote call request to the first system, if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, sending information about transaction consistency failure to the operation management platform; or if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, repeating the following steps until the number of times the first remote call request is resent reaches a preset threshold: putting the identifier of the cached first remote call request into the preset retry queue again, reading the identifier of the first remote call request from the retry queue again according to the preset reading timing, obtaining the first remote call request according to the identifier and resending the first remote call request to the first system again.

[0071] In one embodiment, the retry service module 52 is specifically used to, after placing the cached identifier of the first remote call request into a preset retry queue, mark the delay time corresponding to the first remote call request identifier in the retry queue and count the time; and read the first remote call request whose delay time is 0 from the retry queue by periodically reading the call request with a delay time of 0 in the retry queue.

[0072] In the above-mentioned device for implementing distributed transaction consistency provided by an embodiment of the present invention, the retry component module and the retry service module can exist independently of the application of the first system and be decoupled from the application, which can reduce the cost of modifying the existing application to support the retry mechanism to solve the transaction consistency. It is only necessary to introduce the component and module into the existing first system and modify the interface to support idempotence; the retry component module pre-stores transaction information before transaction processing, which can prevent the loss of transaction information due to program processing interruption; the retry component module can also configure different retries and retry intervals for different requests, which can maximize the use of retries to solve transaction consistency problems according to the application processing rate, which is conducive to reducing the number of retries; the retry component module and the retry service module exist independently of the existing application, which can also reduce the resource consumption of the application initiating a retry request, and is also conducive to maintaining different retry strategies for different services; after the number of retries exceeds the limit or the retry fails, the transaction information is sent to the operation management platform to prevent the loss of transaction information and facilitate business personnel to track problems in real time.

[0073] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for achieving distributed transaction consistency when executing the computer program.

[0074] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for achieving distributed transaction consistency is implemented.

[0075] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for achieving distributed transaction consistency is implemented.

[0076] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0080] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for achieving distributed transaction consistency, characterized in that: include: When the first system receives a first remote call request to the first system sent by the second system, the first remote call request is cached; If an abnormal state occurs when the first system executes the corresponding transaction operation according to the first remote call request, the cached identifier of the first remote call request is placed in a preset retry queue; According to a preset reading timing, reading the identifier of the first remote call request from the retry queue, and acquiring the first remote call request according to the identifier; Resending the first remote call request to the first system; The method further comprises: after placing the cached identifier of the first remote call request into a preset retry queue, marking a delay time corresponding to the identifier of the first remote call request in the retry queue and timing the delay time; The step of reading the identifier of the first remote call request from the retry queue according to a preset reading timing includes: The first remote call request whose delay time is 0 is read from the retry queue by periodically reading the call request whose delay time is 0 in the retry queue.

2. The method according to claim 1, characterized in that After the step of resending the first remote call request to the first system, the method further includes: If the first system executes the corresponding transaction operation according to the resent first remote call request and an abnormal state occurs again, the transaction consistency failure information is sent to the operation management platform; or If the first system again encounters an abnormal state when executing the corresponding transaction operation based on the resent first remote call request, repeat the following steps until the number of times the first remote call request is resent reaches a preset threshold: put the identifier of the cached first remote call request into the preset retry queue again, read the identifier of the first remote call request from the retry queue again according to the preset reading timing, obtain the first remote call request based on the identifier and resend the first remote call request to the first system again.

3. The method according to claim 1, characterized in that The first system executes a corresponding transaction operation according to the first remote call request, including: a transaction operation executed within the first system or a transaction operation implemented by remotely calling a third system; The delay time corresponding to the first remote call request identifier is determined according to the duration and / or frequency of the transaction operation processing performed by the first system or the transaction processing performed by the third system according to the remote call.

4. The method according to any one of claims 1 to 3, characterized in that: Resending the first remote call request to the first system includes: The first remote call request is sent to the interface corresponding to the first system by remote calling.

5. A device for achieving distributed transaction consistency, characterized in that: include: Retry component module, retry service module, request cache queue module and retry queue module; wherein: The retry component module is used for caching the first remote call request sent by the second system to the first system in the request cache queue module when the first system receives the first remote call request sent by the second system to the first system; if the first system performs a corresponding transaction operation according to the first remote call request and an abnormal state occurs, then putting the identifier of the cached first remote call request into the retry queue of the retry queue module; The retry service module is used to read the identifier of the first remote call request from the retry queue according to a preset reading timing, and obtain the first remote call request according to the identifier; and resend the first remote call request to the first system; The retry service module, in the retry queue, identifies the delay time corresponding to the mark for the first remote call request and counts the time; and reads the first remote call request whose delay time is 0 from the retry queue by periodically reading the call request whose delay time is 0 in the retry queue.

6. The device according to claim 5, characterized in that The retry service module is also used for, after the step of resending the first remote call request to the first system, if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, then sending information about transaction consistency failure to the operation management platform; or if the first system again encounters an abnormal state when executing the corresponding transaction operation according to the resent first remote call request, then repeating the following steps until the number of times the first remote call request is resent reaches a preset threshold: putting the identifier of the cached first remote call request into a preset retry queue again, reading the identifier of the first remote call request from the retry queue again according to a preset reading timing, obtaining the first remote call request according to the identifier and resending the first remote call request to the first system again.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for achieving distributed transaction consistency described in any one of claims 1 to 4 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for achieving distributed transaction consistency according to any one of claims 1 to 4 is implemented.

9. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for achieving distributed transaction consistency as claimed in any one of claims 1 to 4 is implemented.

Images