Method and system for guaranteeing consistency of transaction data among multiple systems

Abstract

The invention discloses a method and system for guaranteeing the consistency of transaction data among multiple systems, and the method applied to a transaction provider comprises the steps: generating an interface call ID according to request information, and extracting a cache value from a distributed cache according to the interface call ID; if the cache value exists in the distributed cache and is not null, performing service request processing according to the request information; and updating or deleting the records in the distributed cache according to a request processing result. Through the technical scheme of the invention, the problem of transaction consistency generated when the micro-service is combined and arranged into the combined service is solved, the supporting capability and fault-tolerant capability of the system are improved, and the labor cost for processing error and abnormal data is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method for ensuring transaction data consistency among multiple systems and a system for ensuring transaction data consistency among multiple systems. Background Art

[0002] With the implementation of the specifications for the intelligent middle platform of communication operators, the requirements for components and capabilities have been further improved, and a new software architecture, microservices, has been introduced. In a microservices architecture, an application can be decomposed into multiple smaller-grained services, and each service can be developed and deployed independently in parallel by different teams.

[0003] When a system adopts a microservices architecture, the original business may not change, but the system has been split into many new microservices. Compared with the traditional architecture, the microservices architecture is more dependent on the collaboration between microservices to achieve a complete business process. Since each microservice corresponds to a separate data transaction, the issue of transaction consistency inevitably arises in this way of collaboration between different microservices.

[0004] For example, to process a user request, two tasks A and B need to be completed successively in the Caller system, which are completed by different systems respectively (task A is completed by system ProviderA, and task B is completed by system ProviderB). Task A is processed first, and after task A is successfully processed, task B is processed. However, in practice, due to various reasons such as network and business, it is very likely that task A is successfully processed but task B fails, resulting in data inconsistency in the ProviderA and ProviderB systems.

[0005] Although the asynchronous message processing method can be adopted, this method also has several drawbacks:

[0006] 1. Asynchronous processing is not suitable for scenarios with high real-time requirements;

[0007] 2. There is no effective measure to roll back the successfully committed transaction after the asynchronous message processing fails. Summary of the Invention

[0008] In view of the above problems, the present invention provides a method and system for ensuring the consistency of transaction data between multiple systems, which is ensured through the coordination between the transaction provider and the transaction invoker. The transaction provider needs to have the ability to ensure that the results of one or multiple requests initiated for the same task are consistent, and there will be no side effects due to multiple initiations. The transaction invoker needs to have the ability to automatically retry after a task request fails, solving the problem of transaction consistency generated when microservices are combined and orchestrated into composite services, improving the support ability and fault tolerance of the system, and reducing the labor cost of processing error and abnormal data.

[0009] To achieve the above object, the present invention provides a method for ensuring the consistency of transaction data between multiple systems, which is applied to the transaction provider and includes:

[0010] Generate an interface call ID according to the request information, and extract a cache value from the distributed cache according to the interface call ID;

[0011] If the cache value exists and is not empty in the distributed cache, perform business request processing according to the request information;

[0012] Update or delete the record in the distributed cache according to the request processing result.

[0013] In the above technical solution, preferably, if the cache value exists and is not empty in the distributed cache, perform business request processing according to the request information, and the specific process includes:

[0014] If the cache value exists and is not empty in the distributed cache, directly return the cache value;

[0015] If the cache value exists but is empty in the distributed cache, return a request in-processing flag;

[0016] If the cache value does not exist in the distributed cache, record the request information in the distributed cache.

[0017] In the above technical solution, preferably, update or delete the record in the distributed cache according to the request processing result, and the specific process includes:

[0018] If the business request processing is successful, update the request processing result to the distributed cache;

[0019] If the business request processing fails, delete the record corresponding to the business request in the distributed cache.

[0020] The present invention also proposes a method for ensuring the consistency of transaction data between multiple systems, which is applied to the transaction invoker and includes:

[0021] Define and initialize shared variables and counter variables, and create a task sub-thread;

[0022] Call the target task interface through the task sub-thread, and update the shared state according to the call result;

[0023] Read the shared variables in the memory, and determine the task execution result according to the shared variables;

[0024] According to the task execution result, when the task fails, handle it according to the retry policy or rollback policy.

[0025] In the above technical solution, preferably, the process of defining and initializing the shared variables and counter variables, and creating a task sub-thread specifically includes:

[0026] Define the shared variable FLAG and initialize it to undef;

[0027] Define the counter variable CNT and initialize it to 0, and define the threshold X for the number of retries;

[0028] Create a task sub-thread.

[0029] In the above technical solution, preferably, the process of calling the target task interface through the task sub-thread, and updating the shared state according to the call result specifically includes:

[0030] Call the api interface of the actual target task through the task sub-thread;

[0031] If the api interface call is successful, modify the shared variable FLAG to success, otherwise modify the shared variable FLAG to fail.

[0032] In the above technical solution, preferably, the process of reading the shared variables in the memory, and determining the task execution result according to the shared variables specifically includes:

[0033] Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the task execution is successful and the execution step ends;

[0034] If the shared variable FLAG is undef, the task execution has no response, and if the counter CNT has not reached the preset retry times threshold at this time, loop to recreate the sub-thread;

[0035] If the shared variable FLAG is fail, the task execution fails.

[0036] In the above technical solution, preferably, according to the task execution result, when the task fails, handle it according to the retry policy or rollback policy, and the specific process includes:

[0037] When the task fails, according to the rollback policy, reset the shared variable FLAG to undef;

[0038] Create a task rollback sub-thread, call the api interface of the actual rollback task, and modify the shared variable FLAG to success when the call is successful, otherwise modify the shared variable FLAG to fail;

[0039] Reset the counter CNT to 0;

[0040] If the main thread is blocked, wait for the response of the task rollback sub-thread;

[0041] Increment the value of the counter CNT by 1 to record the execution times of the rollback sub-thread;

[0042] Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the rollback task is executed successfully and the execution steps end;

[0043] If the shared variable FLAG is still undef, the rollback task is not completed or there is no response. And if the counter CNT does not reach the preset retry times threshold at this time, loop and wait for the response of the task rollback sub-thread. If it reaches, record the failure log and the execution steps end;

[0044] If the shared variable FLAG is fail, record the failure log and the execution steps end.

[0045] The present invention also provides a transaction data consistency guarantee system between multiple systems, which is applied to a transaction provider and includes:

[0046] An interface call module, configured to generate an interface call ID according to the request information, and extract a cache value from the distributed cache according to the interface call ID;

[0047] A service request module, configured to perform service request processing according to the request information when the cache value exists and is not empty in the distributed cache;

[0048] A record update module, configured to update or delete the record in the distributed cache according to the request processing result.

[0049] In the above technical solution, preferably, the transaction data consistency guarantee system between multiple systems, which is applied to a transaction invoker, includes:

[0050] A thread creation module, configured to define and initialize shared variables and counter variables, and create task sub-threads;

[0051] An interface call module, configured to call a target task interface through the task sub-thread and update the shared state according to the call result;

[0052] A result determination module, configured to read a shared variable in the memory and determine a task execution result according to the shared variable;

[0053] A failure handling module, configured to perform processing according to a retry policy or a rollback policy when the task fails according to the task execution result.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows: guaranteed by the coordination between the transaction provider and the transaction invoker, the transaction provider needs to have the ability to ensure that the results of one request or multiple requests initiated for the same task are consistent, and there will be no side effects due to multiple initiations. The transaction invoker needs to have the ability to automatically retry after a task request fails, solving the transaction consistency problem generated when microservices are combined and orchestrated into composite services, improving the system's support ability and fault tolerance ability, and reducing the labor cost of processing error and abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic flowchart of the method for guaranteeing transaction data consistency between multiple systems disclosed in an embodiment of the present invention in the transaction provider;

[0056] Figure 2 It is a schematic flowchart of the method for guaranteeing transaction data consistency between multiple systems disclosed in an embodiment of the present invention in the transaction invoker. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0058] The present invention will be further described in detail below with reference to the accompanying drawings:

[0059] As Figure 1 shown, according to a method for guaranteeing transaction data consistency between multiple systems provided by the present invention, which is applied to a transaction provider and includes:

[0060] Generate an interface call ID according to the request information, and extract a cache value from the distributed cache according to the interface call ID;

[0061] If there is a cache value in the distributed cache and it is not empty, the business request is processed according to the request information;

[0062] Update or delete the records in the distributed cache according to the request processing result.

[0063] In this embodiment, it is guaranteed through the coordination between the transaction provider and the transaction invoker. The transaction provider needs to have the ability to ensure that the results of one request or multiple requests initiated for the same task are consistent, without side effects due to multiple initiations. The transaction invoker needs to have the ability to automatically retry after a task request fails, solving the transaction consistency problem that occurs when microservices are combined and orchestrated into composite services, improving the system's support ability and fault tolerance, and reducing the labor cost of handling error and abnormal data.

[0064] Specifically, the provider (Provider) adds preprocessing and supplementary processing to the native business interface before and after the task processing logic respectively through the AOP feature of Spring, and introduces a high-performance distributed cache for recording business interface calls.

[0065] In the above embodiment, preferably, if there is a cache value in the distributed cache and it is not empty, the business request is processed according to the request information. The specific process includes:

[0066] Generate a unique ID for the interface call through the combination of information such as the interface name, request message, and request serial number, which is used to uniquely distinguish different interface requests. Extract the cache value from the distributed cache according to this ID. At this time, there are three situations:

[0067] If there is a cache value in the distributed cache and it is not empty, directly return the cache value;

[0068] If there is a cache value in the distributed cache but it is empty, return a request in-progress flag;

[0069] If there is no cache value in the distributed cache, record the request information in the distributed cache.

[0070] In the above embodiment, preferably, update or delete the records in the distributed cache according to the request processing result. The specific process includes:

[0071] If the business request is processed successfully, update the request processing result to the distributed cache;

[0072] If the business request is processed failed, delete the record corresponding to this business request in the distributed cache.

[0073] The above logical steps need to be encapsulated based on the AOP feature of Spring to achieve the decoupling of the idempotency verification and business processing logical units.

[0074] As Figure 2 shown, the present invention also proposes a method for ensuring the consistency of transaction data between multiple systems, which is applied to a transaction invoker and includes:

[0075] Define and initialize shared variables and counter variables, and create a task sub-thread;

[0076] Call the target task interface through the task sub-thread, and update the shared state according to the call result;

[0077] Read the shared variables in the memory, and determine the task execution result according to the shared variables;

[0078] According to the task execution result, when the task fails, handle it according to the retry policy or rollback policy.

[0079] In the above embodiment, preferably, the process of defining and initializing the shared variables and counter variables and creating the task sub-thread specifically includes:

[0080] Define the shared variable FLAG and initialize it to undef;

[0081] Define the counter variable CNT and initialize it to 0, and define the threshold X of the number of retries;

[0082] Create a task sub-thread.

[0083] In the above embodiment, preferably, the process of calling the target task interface through the task sub-thread and updating the shared state according to the call result specifically includes:

[0084] Call the api interface of the actual target task through the task sub-thread;

[0085] If the api interface call is successful, modify the shared variable FLAG to success, otherwise modify the shared variable FLAG to fail.

[0086] In the above embodiment, preferably, the process of reading the shared variables in the memory and determining the task execution result according to the shared variables specifically includes:

[0087] Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the task execution is successful and the execution step ends;

[0088] If the shared variable FLAG is undef, the task execution has no response, and if the counter CNT has not reached the preset retry threshold X at this time, loop to recreate the sub-thread;

[0089] If the shared variable FLAG is fail, the task execution fails.

[0090] In the above embodiments, preferably, according to the task execution result, when the task fails, it is processed according to the retry policy or the rollback policy. The specific process includes:

[0091] When the task fails, according to the rollback policy, reset the shared variable FLAG to undef;

[0092] Create a task rollback sub-thread, call the api interface of the actual rollback task, and modify the shared variable FLAG to success when the call is successful, otherwise modify the shared variable FLAG to fail;

[0093] Reset the counter CNT to 0;

[0094] If the main thread is blocked, wait for the response of the task rollback sub-thread, and the blocking duration is T2;

[0095] Increment the value of the counter CNT by 1 to record the execution times of the rollback sub-thread;

[0096] Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the rollback task is executed successfully and the execution step ends;

[0097] If the shared variable FLAG is still undef, the rollback task execution is not completed or there is no response. And if the counter CNT does not reach the preset retry times threshold at this time, loop to wait for the response of the task rollback sub-thread. If it reaches, record the failure log and the execution step ends;

[0098] If the shared variable FLAG is fail, record the failure log and the execution step ends.

[0099] The present invention also proposes a transaction data consistency guarantee system between multiple systems, which is applied to the transaction provider and includes:

[0100] An interface call module, which is used to generate an interface call ID according to the request information and extract a cache value from the distributed cache according to the interface call ID;

[0101] A service request module, which is used to perform service request processing according to the request information when there is a cache value and it is not empty in the distributed cache;

[0102] A record update module, which is used to update or delete the records in the distributed cache according to the request processing result.

[0103] In the above embodiments, preferably, the transaction data consistency guarantee system between multiple systems, which is applied to the transaction invoker, includes:

[0104] A thread creation module, which is used to define and initialize the shared variable and the counter variable, and create a task sub-thread;

[0105] An interface call module, configured to call a target task interface through a task sub-thread and update a shared state according to a call result;

[0106] A result determination module, configured to read a shared variable in a memory and determine a task execution result according to the shared variable;

[0107] A failure handling module, configured to perform processing according to a retry policy or a rollback policy when a task fails according to a task execution result.

[0108] According to the transaction data consistency guarantee system between multiple systems disclosed in the above embodiments, the functions to be implemented by each module respectively correspond to the steps of the method for guaranteeing transaction data consistency between multiple systems disclosed in the above embodiments. During implementation, refer to the above embodiments for operation, which will not be elaborated here.

[0109] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for ensuring transaction data consistency among multiple systems, characterized in that: Applicable to transaction providers, including: Generate an interface call ID according to the request information, and extract a cache value from the distributed cache according to the interface call ID; If the cache value exists in the distributed cache and is not empty, processing the service request according to the request information; The records in the distributed cache are updated or deleted according to the request processing result.

2. The transaction data consistency guarantee method among multiple systems according to claim 1, characterized in that: If the cache value exists in the distributed cache and is not empty, the service request is processed according to the request information, and the specific process includes: If the cache value exists in the distributed cache and is not empty, directly return the cache value; If the cache value exists in the distributed cache but is empty, a request processing flag is returned; If the cache value does not exist in the distributed cache, the request information is recorded in the distributed cache.

3. The method for ensuring transaction data consistency among multiple systems according to claim 1, characterized in that: The records in the distributed cache are updated or deleted according to the request processing result. The specific process includes: If the business request is processed successfully, the request processing result is updated to the distributed cache; If the service request processing fails, the record corresponding to the service request in the distributed cache is deleted.

4. A method for ensuring transaction data consistency among multiple systems, characterized in that: Applicable to the transaction caller, including: Define and initialize shared variables and counter variables, and create task subthreads; Calling the target task interface through the task subthread, and updating the shared state according to the calling result; Read the shared variables in the memory, and determine the task execution result according to the shared variables; According to the task execution result, when the task fails, it is processed according to the retry strategy or the rollback strategy.

5. The method for ensuring transaction data consistency among multiple systems according to claim 4, characterized in that: The shared variables and counter variables are defined and initialized, and a task subthread is created. The specific process includes: Define the shared variable FLAG and initialize it to undef; Define a counter variable CNT and initialize it to 0, and define the threshold X for the number of retries; Create a task child thread.

6. The method for ensuring transaction data consistency among multiple systems according to claim 4, characterized in that: The target task interface is called by the task subthread, and the shared state is updated according to the calling result. The specific process includes: Call the API interface of the actual target task through the task subthread; If the API call is successful, change the shared variable FLAG to success, otherwise change the shared variable FLAG to fail.

7. The method for ensuring transaction data consistency among multiple systems according to claim 4, characterized in that: The process of reading the shared variables in the memory and determining the task execution result according to the shared variables includes: Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the task is successfully executed and the execution step ends. If the shared variable FLAG is undef, the task execution is unresponsive, and if the counter CNT does not reach the preset retry count threshold at this time, the child thread is recreated in a loop; If the shared variable FLAG is fail, the task execution fails.

8. The method for ensuring transaction data consistency among multiple systems according to claim 4, characterized in that: According to the task execution result, when the task fails, it is processed according to the retry strategy or rollback strategy. The specific process includes: When the task fails, according to the rollback strategy, the shared variable FLAG is reset to undef; Create a task rollback subthread, call the API interface of the actual rollback task, and modify the shared variable FLAG to success if the call is successful, otherwise modify the shared variable FLAG to fail; Reset counter CNT to 0; If the main thread is blocked, wait for the task rollback sub-thread to respond; The counter CNT value is increased by 1 to record the number of times the rollback sub-thread is executed; Read the shared variable FLAG in the memory. If the shared variable FLAG is success, the rollback task is executed successfully and the execution step ends. If the shared variable FLAG is still not undef, the rollback task execution is not completed or there is no response, and if the counter CNT does not reach the preset retry count threshold at this time, then the task rollback sub-thread is looped to wait for a response, and if it is reached, a failure log is recorded and the execution step ends; If the shared variable FLAG is fail, a failure log is recorded and the execution steps are terminated.

9. A transaction data consistency assurance system among multiple systems, characterized in that: Applicable to transaction providers, including: An interface calling module, used to generate an interface calling ID according to the request information, and extract a cache value from the distributed cache according to the interface calling ID; A service request module, configured to process a service request according to the request information when the cache value exists in the distributed cache and is not empty; The record updating module is used to update or delete the records in the distributed cache according to the request processing result.

10. A transaction data consistency assurance system among multiple systems, characterized in that: Applicable to the transaction caller, including: The thread creation module is used to define and initialize shared variables and counter variables, and create task subthreads; An interface calling module, used to call the target task interface through the task subthread, and update the shared state according to the calling result; A result determination module, used for reading the shared variables in the memory and determining the task execution result according to the shared variables; The failure processing module is used to process the task according to the task execution result and according to the retry strategy or rollback strategy when the task fails.

Images