Unit-based distributed transaction processing system and method

Abstract

The application provides a kind of distributed transaction processing system and method based on unit, comprising: the public processing module obtains transaction processing information and distributes to transaction processing module processing, then through basic service module in each transaction scheduling, call downstream service interface, while transaction information module records this call and is persisted;The call record includes the basic information and execution progress of the called transaction, and data is shared to each module.The application tracks the execution process of transaction through the distributor of distributed transaction consistency, meets the distributed transaction including unit application architecture, and the individualized processing of transaction for different business scenarios.The application is executed through the connection of each module, so that the transaction of unit service can meet the eventual consistency of database level, and can also guarantee the eventual consistency of application level business implementation.

Description

Technical Field

[0001] This invention relates to the field of data access technology, and more specifically, to a unit-based distributed transaction processing system and method. Background Technology

[0002] Typically, after a distributed deployment, multiple services will work together to complete a transaction, and these services will exist on different servers or network environments. The industry mostly uses methods such as TCC or SAGA to solve this problem.

[0003] Patent document CN105892957A discloses a distributed transaction execution method based on dynamic sharding. In each data center, a dynamic transaction sharding unit is deployed in the database management server to control and manage transactions. The dynamic transaction sharding unit includes a dynamic sharding module and a dynamic conflict analysis module. The dynamic transaction sharding unit processes transaction requests from the application server, performs dynamic sharding and dynamic conflict analysis on the transaction request, and then executes the sharded transactions using the sharding execution module, and performs specific data access operations on the database storage.

[0004] However, existing distributed physical execution methods suffer from drawbacks and shortcomings such as low performance, high maintenance costs, and limited applicability. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a unitized distributed transaction processing system and method.

[0006] According to the present invention, a unit-based distributed transaction processing system includes: a common processing module, a transaction processing module, a basic service module, and a transaction information module.

[0007] The common processing module obtains transaction processing information and distributes it to the transaction processing module for processing. Then, it schedules the transactions through the basic service module and calls the downstream service interface. At the same time, the transaction information module persists the call record.

[0008] The call record includes basic information about the called transaction and its execution progress, and the data is shared with each module.

[0009] Preferably, it further includes: a synchronous exception handling module, an asynchronous exception handling module, and an interrupt recovery module;

[0010] When an unresolved sub-transaction occurs, the synchronous exception handling module will synchronize the results through a scheduled task. If the result is a failure, the asynchronous exception handling module will handle the exception. If the result is a success, the interrupt recovery module will continue to process the subsequent sub-processes.

[0011] Preferably, the common processing module includes: a transaction dispatcher module, a transaction processor acquisition module, and a transaction lifecycle management module;

[0012] The transaction dispatcher module is used throughout the entire link;

[0013] The transaction processor acquisition module is used to acquire the corresponding transaction processor;

[0014] The transaction lifecycle management module is used to monitor the entire lifecycle of a transaction.

[0015] Preferably, the transaction processing module includes: a transaction loading module, a transaction dynamic scheduling module, and a processor automatic configuration module;

[0016] When the project starts, the transaction loading module loads all transaction processors and transaction request templates into the listener;

[0017] The transaction dynamic scheduling module uses an internal unitization algorithm to load balance transaction requests to downstream related services, with the unit as the basic unit of service deployment.

[0018] The processor automated configuration module is used for configuring the basic process and automatically configuring and generating the corresponding processor.

[0019] Preferably, the synchronous exception handling module is called by the transaction dispatcher module when an exception occurs in the transaction, and performs corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module.

[0020] The asynchronous exception handling module queries and processes pending transactions periodically via asynchronous scheduled tasks for exceptions where the request result cannot be obtained in a timely manner; and it rolls back and reverses transactions periodically via exception tasks for rollback operations with low real-time requirements.

[0021] The interruption recovery module recovers transactions that failed to update their results properly during system failures by using the interruption recovery mechanism and based on recorded operation logs, or performs corresponding exception handling.

[0022] A unit-based distributed transaction processing method provided by the present invention includes:

[0023] Step S1: Obtain transaction information and distribute it to the transaction processor;

[0024] Step S2: The transaction processor receives the transaction information, processes it, and matches it with the corresponding basic service;

[0025] Step S3: Call and execute the basic service, and record the current execution step to the transaction information module.

[0026] Preferably, it further includes: a synchronous exception handling step, an asynchronous exception handling step, and an interrupt recovery step;

[0027] When an unresolved sub-transaction occurs, the result is synchronized through a timed task in the synchronous exception handling step, and the result is determined to be successful. If not, the asynchronous exception handling step is triggered to handle the exception; if yes, the interruption recovery step is triggered to continue processing the subsequent sub-process.

[0028] Preferably, the transaction information is obtained by initiating a specific transaction scenario;

[0029] While acquiring the transaction information, the transaction lifecycle management steps are triggered to monitor the entire transaction lifecycle.

[0030] Preferably, the process includes:

[0031] Transaction loading steps: When the project starts, all transaction processors and transaction request templates are loaded into the listener;

[0032] Transaction dynamic scheduling steps: Using an internal unitization algorithm, transaction requests are load-balanced to downstream related services, with the unit as the basic unit of service deployment;

[0033] Processor automated configuration steps: Configure the basic process and automatically generate the corresponding processor.

[0034] Preferably, the synchronization exception handling step is as follows: when a transaction exception occurs, the transaction dispatcher module calls and executes the corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module;

[0035] Asynchronous exception handling steps: For exceptions where the request result cannot be obtained in a timely manner, pending transactions are queried and processed periodically through an asynchronous scheduled task; for rollback operations with low real-time requirements, reverse transactions are rolled back periodically through an exception task.

[0036] The interruption recovery step involves recovering transactions that failed to update their results properly during a system failure, or performing appropriate exception handling based on the interruption recovery mechanism and recorded operation logs.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This invention tracks the execution process of transactions across the entire chain through a distributed transaction consistency distributor, which satisfies the personalized processing of transactions for different business scenarios, including unitized application architectures.

[0039] 2. This invention enables transactions in unitized services to satisfy both the eventual consistency at the database level and the eventual consistency of business implementation at the application level through the connection and execution of various modules.

[0040] 3. This invention allows for customizable operation granularity, reducing data-level lock conflicts and improving business concurrency. At the same time, it is not highly intrusive to applications and can effectively address performance issues. Attached Figure Description

[0041] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0042] Figure 1 This is a schematic diagram of the workflow of the present invention.

[0043] Figure 2 This is a schematic diagram of the process under normal circumstances of the present invention.

[0044] Figure 3 This is a flowchart illustrating the abnormal conditions of the present invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0046] This invention utilizes the distributed transaction consistency dispatcher OpenApiDispatcher to satisfy distributed transactions, including those in unitized application architectures. This dispatcher tracks the entire transaction execution process. For transactions in different business scenarios, the IOpenApiProcessor interface is implemented to provide customized transaction handling. The solution includes the following modules: a transaction loading module, a transaction dispatcher module, a transaction processor acquisition module, a transaction information module, a basic service module, a synchronous exception handling module, a dynamic transaction scheduling module, an asynchronous exception handling module, an interrupt recovery mechanism, automated processor configuration, and transaction lifecycle management. This ensures that transactions in unitized services satisfy both database-level eventual consistency and application-level business implementation eventual consistency.

[0047] This invention, while satisfying the requirements of a modular architecture, incorporates a processor automated configuration module to configure distributed transactions according to a template for the order in which sub-transactions are called. Upon project startup, the transaction loading module loads all configured distributed transaction scenarios into memory. The project then uses the transaction dispatcher module to initiate specific transaction scenarios and obtain transaction processors, simultaneously triggering the transaction lifecycle management module to monitor the entire transaction lifecycle. Then, the transaction dynamic scheduling module calls each sub-transaction, each of which is a basic service module. When a basic service module is called, the transaction information module persists the call record. When an unresolved sub-transaction occurs, the synchronous exception handling module synchronizes the results via a scheduled task. If the result is a failure, the asynchronous exception handling module handles the exception; if the result is a success, the interrupt recovery mechanism continues processing subsequent sub-processes.

[0048] Example 1

[0049] According to the present invention, a unit-based distributed transaction processing system is provided, such as... Figures 1 to 3 As shown, it includes: a common processing module, a transaction processing module, a basic service module, and a transaction information module. The common processing module obtains transaction processing information and distributes it to the transaction processing module for processing. Then, the basic service module schedules the transactions and calls downstream service interfaces. At the same time, the transaction information module persists the call record. The call record includes the basic information and execution progress of the called transaction and shares this data with each module.

[0050] The common processing module includes a transaction dispatcher module, a transaction processor acquisition module, and a transaction lifecycle management module. The transaction dispatcher module is used throughout the entire process. The transaction processor acquisition module is used to acquire the corresponding transaction processor. The transaction lifecycle management module is used to monitor the entire transaction lifecycle.

[0051] The transaction processing module includes a transaction loading module, a transaction dynamic scheduling module, and a processor automatic configuration module. The transaction loading module loads all transaction processors and transaction request templates into the listener when the project starts. The transaction dynamic scheduling module uses an internal unitization algorithm, with units as the basic unit of service deployment, to load balance transaction requests to downstream related services. The processor automatic configuration module is used for configuring the basic process and automatically generates the corresponding processors.

[0052] The system of the present invention also includes: a synchronous exception handling module, an asynchronous exception handling module, and an interrupt recovery module.

[0053] Specifically, such as Figure 2 and Figure 3As shown, when an unresolved sub-transaction occurs, the synchronous exception handling module will synchronize the results through a scheduled task. If the result is a failure, the asynchronous exception handling module will handle the exception. If the result is a success, the interrupt recovery module will continue to process the subsequent sub-processes.

[0054] Furthermore, the synchronous exception handling module, when a transaction exception occurs, is invoked by the transaction dispatcher module and executes corresponding rollback or compensation steps based on the operation and exception information recorded in the transaction information module. The asynchronous exception handling module, for exceptions where the request result cannot be obtained in a timely manner, periodically queries and processes pending transactions through asynchronous scheduled tasks; for rollback operations with low real-time requirements, it periodically rolls back and reverses transactions through exception tasks. The interrupt recovery module, for transactions whose results were not updated normally during system failures, recovers the transactions or performs corresponding exception handling through the interrupt recovery mechanism and based on the recorded operation records.

[0055] Furthermore, the various modules of the system in this invention are further described and explained:

[0056] Transaction Loading Module: When the project starts, this module loads all transaction processors and transaction request templates into the listener using the @ApiProcessorTag and @FormatTag tags.

[0057] Transaction dispatcher module: The main function of this module is to run through the entire chain, including initialization, transaction template parsing, idempotency verification, transaction persistence, response construction, and exception handling.

[0058] The module for obtaining transaction processors retrieves the corresponding transaction processor based on the processor ID after parsing the transaction request template.

[0059] Transaction Information Module: Records basic information about transactions and their execution progress to enable data sharing between different modules.

[0060] Basic service module: Schedules transactions within each transaction and calls downstream service interfaces to implement basic operations. Using the `@BasicServiceTag` and `@EtradeTransactionTag` tags as markers, it records the steps of the current operation to the transaction information module through aspects.

[0061] Synchronization exception handling module: When an exception occurs in a transaction, it is called by the transaction dispatcher module to perform corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module.

[0062] The transaction dynamic scheduling module uses an internal unitized algorithm, with units as the basic unit of service deployment, to load balance transaction requests to downstream related services. This involves database unitization and downstream interfacing service unitization. While solving storage and performance bottlenecks, the advantages of divide and conquer, fault isolation, and non-interference also greatly reduce the impact of failures.

[0063] Asynchronous exception handling module: For exceptions where the request result cannot be obtained in a timely manner, an asynchronous scheduled task is used to periodically query and process pending transactions, improving the system's completion rate. For rollback operations with low real-time requirements, an exception task is used to periodically roll back corrective transactions, improving system performance.

[0064] Interruption Recovery Module: Based on the transaction information module, it records operation logs before each request to basic services. In the event of a system failure and transactions whose results were not updated correctly, the interruption recovery mechanism restores the transactions or performs appropriate exception handling based on the recorded operation logs.

[0065] Processor Automated Configuration Module: The basic process is configured via Excel, automatically generating the corresponding processor without the need for manual coding.

[0066] Transaction Lifecycle Management Module: Defines and manages the transaction lifecycle. Each transaction strictly follows the predetermined lifecycle state process, and every step involved is recorded and persisted through aspects, ensuring that each step of the transaction is locatable and traceable. For transactions that fail, the recorded lifecycle state can also be used for anomaly repair and compensation.

[0067] Example 2

[0068] A unit-based distributed transaction processing method provided by the present invention includes:

[0069] Step S1: Obtain transaction information and distribute it to the transaction processor. Specifically, the transaction information is obtained by initiating a specific transaction scenario; simultaneously, the transaction lifecycle management step is triggered to monitor the entire transaction lifecycle.

[0070] Step S2: The transaction processor receives and processes the transaction information, and matches it with the corresponding basic service. Specifically, this processing step includes: Transaction loading step: When the project starts, all transaction processors and transaction request templates are loaded into the listener. Transaction dynamic scheduling step: Using an internal unitization algorithm, transaction requests are load-balanced to downstream related services, with the unit as the basic unit of service deployment. Processor automated configuration step: The basic configuration process is automated to generate the corresponding processor.

[0071] Step S3: Call and execute the basic service, and record the current execution step to the transaction information module.

[0072] In addition, the method of the present invention also includes: a synchronous exception handling step, an asynchronous exception handling step, and an interruption recovery step; when an unresolved sub-transaction occurs, the synchronous exception handling step uses a timed task to synchronize the result and determine if the result is successful. If not, the asynchronous exception handling step is triggered to handle the exception; if so, the interruption recovery step is triggered to continue processing subsequent sub-processes. Specifically, the synchronous exception handling step: when a transaction exception occurs, the transaction dispatcher module calls and executes corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module.

[0073] Asynchronous exception handling steps: For exceptions where the request result cannot be obtained in a timely manner, pending transactions are queried and processed periodically through an asynchronous scheduled task; for rollback operations with low real-time requirements, reverse transactions are rolled back periodically through an exception task.

[0074] The interruption recovery step involves recovering transactions that failed to update their results properly during a system failure, or performing appropriate exception handling based on the interruption recovery mechanism and recorded operation logs.

[0075] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0076] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A unit-based distributed transaction processing system, characterized in that, include: The module comprises a public processing module, a transaction processing module, a basic service module, and a transaction information module. The common processing module obtains transaction processing information and distributes it to the transaction processing module for processing. Then, it schedules the transactions through the basic service module and calls the downstream service interface. At the same time, the transaction information module persists the call record. The call record includes basic information and execution progress of the called transaction, and the data is shared with each module; The transaction processing module includes: a transaction loading module, a transaction dynamic scheduling module, and a processor automatic configuration module; When the project starts, the transaction loading module loads all transaction processors and transaction request templates into the listener; The transaction dynamic scheduling module uses an internal unitization algorithm to load balance transaction requests to downstream related services, with the unit as the basic unit of service deployment. The processor automated configuration module is used for configuring the basic process and automatically configuring and generating the corresponding processor.

2. The unit-based distributed transaction processing system according to claim 1, characterized in that, It also includes: a synchronous exception handling module, an asynchronous exception handling module, and an interrupt recovery module; When an unresolved sub-transaction occurs, the synchronous exception handling module will synchronize the results through a scheduled task. If the result is a failure, the asynchronous exception handling module will handle the exception. If the result is a success, the interrupt recovery module will continue to process the subsequent sub-processes.

3. The unit-based distributed transaction processing system according to claim 1, characterized in that, The common processing module includes: a transaction dispatcher module, a transaction processor acquisition module, and a transaction lifecycle management module; The transaction dispatcher module is used throughout the entire link; The transaction processor acquisition module is used to acquire the corresponding transaction processor; The transaction lifecycle management module is used to monitor the entire lifecycle of a transaction.

4. The unit-based distributed transaction processing system according to claim 2, characterized in that, The synchronous exception handling module is called by the transaction dispatcher module when an exception occurs in a transaction, and performs corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module. The asynchronous exception handling module queries and processes pending transactions periodically via asynchronous scheduled tasks for exceptions where the request result cannot be obtained in a timely manner; and it rolls back and reverses transactions periodically via exception tasks for rollback operations with low real-time requirements. The interruption recovery module recovers transactions that failed to update their results properly during system failures by using the interruption recovery mechanism and based on recorded operation logs, or performs corresponding exception handling.

5. A distributed transaction processing method based on unitization, characterized in that, include: Step S1: Obtain transaction information and distribute it to the transaction processor; Step S2: The transaction processor receives the transaction information, processes it, and matches it with the corresponding basic service; Step S3: Invoke and execute the basic service, and record the current execution step to the transaction information module; The process includes: Transaction loading steps: When the project starts, all transaction processors and transaction request templates are loaded into the listener; Transaction dynamic scheduling steps: Using an internal unitization algorithm, transaction requests are load-balanced to downstream related services, with the unit as the basic unit of service deployment; Processor automated configuration steps: Configure the basic process and automatically generate the corresponding processor.

6. The unit-based distributed transaction processing method according to claim 5, characterized in that, It also includes: synchronous exception handling steps, asynchronous exception handling steps, and interrupt recovery steps; When an unresolved sub-transaction occurs, the result is synchronized through a timed task in the synchronous exception handling step, and the result is determined to be successful. If not, the asynchronous exception handling step is triggered to handle the exception; if yes, the interruption recovery step is triggered to continue processing the subsequent sub-process.

7. The unit-based distributed transaction processing method according to claim 5, characterized in that, The transaction information is obtained by initiating a specific transaction scenario; While acquiring the transaction information, the transaction lifecycle management steps are triggered to monitor the entire transaction lifecycle.

8. The unit-based distributed transaction processing method according to claim 6, characterized in that, The synchronization exception handling steps are as follows: when a transaction exception occurs, the transaction dispatcher module calls and executes the corresponding rollback or compensation steps based on the operation information and exception information recorded in the transaction information module. Asynchronous exception handling steps: For exceptions where the request result cannot be obtained in a timely manner, pending transactions are queried and processed periodically through an asynchronous scheduled task; for rollback operations with low real-time requirements, reverse transactions are rolled back periodically through an exception task. The interruption recovery step involves recovering transactions that failed to update their results properly during a system failure, or performing appropriate exception handling based on the interruption recovery mechanism and recorded operation logs.

Images