Global serializable multi-database transaction control method and system

Abstract

The invention provides a globally serializable multi-database transaction control method and system. The globally serializable multi-database transaction control method comprises the steps that an application service registers a multi-database transaction processing logic to a gasket layer; a multi-database transaction request initiated by a client is received, and a gasket layer intercepts and starts a global transaction of a multi-database transaction; the gasket layer executes the sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic; and submitting the global transaction to the two-stage submission coordinator, and returning an execution result of the multi-database transaction request to the client. According to the method, atomicity, durability and global serializability guarantee is provided for multi-database transactions; concurrency control of an existing database system is reused based on a gasket layer, and seamless integration to existing applications can be achieved; transaction coordination overhead is low, and the method adapts to a high-concurrency environment; the consistency distributed transaction control of the grid has a good ACID (Access Control Identifier) characteristic; the method does not depend on a specific service implementation platform; and low invasiveness is kept for the application, and the transaction control does not influence the writing of normal business codes.

Description

Technical Field

[0001] The present invention belongs to the technical field of database transaction management, and in particular, relates to a globally serializable multi-database transaction control method and system. Background Art

[0002] In recent years, with the widespread application of microservice architecture and serverless architecture, distributed service-oriented applications have become more and more common. In these applications, various functions are currently divided into independent, loosely coupled application services, and each service is equipped with a database system to manage its proprietary data. This distributed structure makes it inevitable that complex workflows span multiple services and generate multi-database global transactions. The emergence of multi-database global transactions aims to coordinate data access between multiple database systems to ensure data consistency while accessing different services.

[0003] In the prior art, a multi-database global transaction is usually composed of multiple sub-transactions, each of which is executed on a different database system. Take a banking application as an example, in which different database systems manage the user's savings account and checking account respectively. In order to complete a withdrawal operation, one service must read the account balance of another service, thus forming a multi-database transaction across databases. However, although each service can ensure serializability when executing its sub-transaction locally, the overall transaction may still not be globally serializable. For example, when multiple withdrawal requests are made simultaneously, the existing system is prone to concurrency anomalies, that is, each service may mistakenly approve multiple duplicate withdrawal requests when judging that the balance is sufficient, thereby violating business constraints. This situation exposes the limitations of the existing system in dealing with global data consistency.

[0004] To solve the above problems, some existing technologies have proposed multi-database transaction protocols to ensure global serialization. However, these protocols usually rely on conservative assumptions, resulting in significant performance degradation. For example, the ticket method and altruistic locking strategies do not allow sub-transactions to be executed in parallel on the same database system, and avoid errors by globally sorting multi-database transactions, even if there is actually no conflict between these transactions. This global sorting method usually leads to a significant performance degradation, limiting the overall efficiency of the system.

[0005] Another existing technology attempts to improve performance by performing more concurrency control at the application layer. These solutions usually give up the guarantee of global serialization and only provide snapshot isolation. For example, the Epoxy protocol and the Cherry Garcia protocol implement snapshot isolation between multiple databases, but they require the application to coordinate sub-transactions at the application layer and use the underlying database system as a version store. Such methods not only limit the use of advanced database features such as unique constraints, foreign key constraints, and predicate-based updates, but also force the application's data to be mixed with metadata, making the underlying database inaccessible directly. At the same time, these solutions only guarantee snapshot isolation, so they are still prone to concurrency anomalies.

[0006] The concurrency control mechanisms commonly used by existing database systems, such as serializable snapshot isolation or strict two-phase locking, cannot achieve global serializability in a multi-database environment. This is because these systems only control the execution of local transactions, while the commit order of global transactions is usually controlled by the two-phase commit (2PC) coordinator based on the order in which messages are received, which cannot ensure global data consistency. At the same time, no system in the current prior art can achieve high performance and global serializability at the same time without modifying existing applications and database systems.

[0007] Patent document "Global Transaction Manager and Transaction Processing Method for Distributed Systems" (CN201410214371) discloses that the global transaction manager includes application devices, transaction management devices, resource management devices, transaction rollback devices, fault recovery devices and information coordination devices, providing distributed transaction control with strong isolation and strict consistency, and has good ACID characteristics. However, it still cannot guarantee the global serializability of transactions, and does not utilize the concurrency control mechanism of the database, and cannot achieve low-code modification and high concurrency performance.

[0008] Therefore, a new solution is needed to ensure global serialization of multi-database transactions while avoiding major modifications to existing systems and applications, providing more efficient performance. Summary of the invention

[0009] In view of the defects in the prior art, an object of the present invention is to provide a globally serializable multi-database transaction control method and system.

[0010] A globally serializable multi-database transaction control method provided by the present invention comprises:

[0011] Step S1: Instruct the application service to register the multi-database transaction processing logic with the shim layer;

[0012] Step S2: The application service receives a multi-database transaction request initiated by the client, and instructs the shim layer to intercept and start a global transaction of the multi-database transaction;

[0013] Step S3: instructing the shim layer to execute the sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic;

[0014] Step S4: Instruct the shim layer to submit the global transaction to the two-phase commit coordinator and return the execution result of the multi-database transaction request to the client.

[0015] Preferably, in step S1, the shim layer provides an interface for registering multi-database transactions for the application service. Before the application service is started, a processing function for the multi-database transactions is registered through the system interface. The processing function defines the global transaction logic including local sub-transaction requests or remote sub-transaction requests when the application service is running.

[0016] In step S2, the shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in the thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return.

[0017] If a confirmation is received from the two-phase commit coordinator, no operation is performed. If no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended.

[0018] After all sub-transaction requests in step S4 are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0019] After receiving the reply, the thread local variables are eliminated and the execution result is returned to the client.

[0020] Preferably, step S3 comprises:

[0021] Local transaction steps: The application service executes the local subtransaction request, the shim layer intercepts the local subtransaction request, generates a subtransaction ID, and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. If the thread-local global transaction ID has not been set, the application service continues to execute directly.

[0022] Execute subtransactions and auxiliary operations in the preparation phase based on the shim layer corresponding to the database that processes the subtransactions;

[0023] The shim layer notifies the two-phase commit coordinator that the subtransaction is ready.

[0024] Remote transaction steps: The application service executes the remote sub-transaction request. The shim layer intercepts the remote sub-transaction request and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the global transaction ID is appended to the request header, and the remote sub-transaction request is sent to the remote application service and waits for the result to be returned. If the thread-local global transaction ID has not been set, execution continues directly.

[0025] After receiving the remote sub-transaction request, the remote application service determines whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as the thread-local global transaction ID. If not, the execution continues directly.

[0026] The remote application service executes the local transaction steps, clears the local variables and returns the results to the application service.

[0027] Preferably, the execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, including:

[0028] When the database is an SSI protocol-based database, the database starts a local subtransaction and performs operations, reads a random row from the database's Dummy table, updates the same row with a random value, and determines the thread local variable;

[0029] If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread local variable is set, the database local subtransaction and auxiliary transaction are prepared and the two-phase commit coordinator is notified that the subtransaction is ready and waits for the result to be returned.

[0030] When the database is based on the S2PL protocol, the database starts a local subtransaction and performs operations to determine thread local variables;

[0031] If the thread-local variable is not set, the database transaction is committed. If the thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready.

[0032] When the database is based on other protocols, before the subtransaction is executed, the subtransaction global lock is acquired, the database starts the local subtransaction and performs operations to determine the thread local variables;

[0033] If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

[0034] Preferably, the step S4 comprises:

[0035] Step S4.1: After receiving the request to commit a global transaction, the two-phase commit coordinator determines whether the transaction is committed in the current state according to the standard two-phase commit protocol;

[0036] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or a global transaction that has not been committed when the system is restarted, a sub-transaction rollback request is sent to the application service that registered the sub-transaction;

[0037] Otherwise, a sub-transaction commit request is sent to the application service that registered the sub-transaction.

[0038] Step S4.2: After the application service of the registered subtransaction receives the request to commit / rollback the database subtransaction, the shim layer intercepts the request to commit / rollback the database subtransaction, commits / rolls back the database subtransaction, determines the database type, and performs auxiliary operations in the commit phase through the shim layer corresponding to the database;

[0039] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back. If it is a database based on the S2PL protocol, no operation is performed. If it is a database based on other protocols, the global lock of the subtransaction is released.

[0040] Step S4.3: After all application services complete the sub-transaction commit / rollback operation request and return, the thread local variables are eliminated and the client is notified of the execution result of the multi-database transaction request.

[0041] A globally serializable multi-database transaction control system provided by the present invention includes: an application service, a database server and a two-phase commit coordinator.

[0042] The application service includes a set number of application services, an integrated shim layer, and is selected based on the concurrency control protocol of the called database system.

[0043] The database server contains the actual application data and auxiliary data tables, supports the execution of multiple database transactions and the maintenance of dependencies, and works in conjunction with the two-phase transaction coordinator.

[0044] It is built in client / server mode, and the application service, database server and two-phase commit coordinator are connected through the network.

[0045] The application service registers the multi-database transaction processing logic with the shim layer;

[0046] The application service receives a multi-database transaction request initiated by the client, and the shim layer intercepts and starts a global transaction for the multi-database transaction;

[0047] The shim layer executes sub-transaction requests of multi-database transactions according to the multi-database transaction processing logic;

[0048] The shim layer submits the global transaction to the two-phase commit coordinator and returns the execution results of the multi-database transaction request to the client.

[0049] Preferably, the shim layer provides an interface for registering multi-database transactions for the application service. Before the application service is started, a processing function for the multi-database transactions is registered through the system interface. The processing function defines the global transaction logic including local sub-transaction requests or remote sub-transaction requests when the application service is running.

[0050] The shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in the thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return;

[0051] If a confirmation is received from the two-phase commit coordinator, no operation is performed. If no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended.

[0052] After all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0053] After receiving the reply, the thread local variables are eliminated and the execution result is returned to the client.

[0054] Preferably, the sub-transaction request for executing a multi-database transaction includes:

[0055] Local transaction module: The application service executes the local subtransaction request. The shim layer intercepts the local subtransaction request, generates a subtransaction ID, and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. If the thread-local global transaction ID has not been set, the application service continues to execute directly.

[0056] Execute subtransactions and auxiliary operations in the preparation phase based on the shim layer corresponding to the database that processes the subtransactions;

[0057] The shim layer notifies the two-phase commit coordinator that the subtransaction is ready.

[0058] Remote transaction module: The application service executes the remote sub-transaction request. The shim layer intercepts the remote sub-transaction request and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the global transaction ID is appended to the request header, and the remote sub-transaction request is sent to the remote application service and waits for the result to be returned. If the thread-local global transaction ID has not been set, execution continues directly.

[0059] After receiving the remote sub-transaction request, the remote application service determines whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as the thread-local global transaction ID. If not, the execution continues directly.

[0060] The remote application service executes the local transaction module, clears the local variables and returns the results to the application service.

[0061] Preferably, the execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, including:

[0062] When the database is an SSI protocol-based database, the database starts a local subtransaction and performs operations, reads a random row from the database's Dummy table, updates the same row with a random value, and determines the thread local variable;

[0063] If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread local variable is set, the database local subtransaction and auxiliary transaction are prepared and the two-phase commit coordinator is notified that the subtransaction is ready and waits for the result to be returned.

[0064] When the database is based on the S2PL protocol, the database starts a local subtransaction and performs operations to determine thread local variables;

[0065] If the thread-local variable is not set, the database transaction is committed. If the thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready.

[0066] When the database is based on other protocols, before the subtransaction is executed, the subtransaction global lock is acquired, the database starts the local subtransaction and performs operations to determine the thread local variables;

[0067] If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

[0068] Preferably, the shim layer submitting a global transaction to the two-phase commit coordinator includes:

[0069] After receiving the request to commit a global transaction, the two-phase commit coordinator determines whether to commit the transaction in the current state according to the standard two-phase commit protocol;

[0070] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or a global transaction that has not been committed when the system is restarted, a sub-transaction rollback request is sent to the application service that registered the sub-transaction;

[0071] Otherwise, a sub-transaction commit request is sent to the application service that registered the sub-transaction.

[0072] After the application service that registers the subtransaction receives the request to commit / rollback the database subtransaction, the shim layer intercepts the request to commit / rollback the database subtransaction, commits / rolls back the database subtransaction, determines the database type, and performs auxiliary operations in the commit phase through the shim layer corresponding to the database.

[0073] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back. If it is a database based on the S2PL protocol, no operation is performed. If it is a database based on other protocols, the global lock of the subtransaction is released.

[0074] After all application services complete the subtransaction commit / rollback operation request and return, the thread local variables are eliminated.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] 1. The present invention ensures the global serializability of multi-database transactions and uses two-phase commit to achieve the atomicity and persistence of transactions. It can ensure the serializability of global transactions in complex distributed scenarios, effectively reduce concurrency conflicts, and enhance the reliability, stability and consistency of the system.

[0077] 2. The shim layer used in the present invention reuses the concurrency control of the existing database system and can be seamlessly integrated into a variety of existing multi-database application systems without modifying the application service logic, database code and database internal structure. It is easy to integrate into the existing application system, enhance scalability and maintain system flexibility.

[0078] 3. The present invention reduces system coordination overhead and delay in complex distributed scenarios by performing non-blocking coordination in the preparation phase, greatly improves system parallelism, significantly improves overall performance, and adapts to high-concurrency environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0080] Figure 1 It is a schematic diagram of the overall process of the globally serializable multi-database transaction control method;

[0081] Figure 2 Request a flow chart for executing a local subtransaction;

[0082] Figure 3 Request a flow chart for executing a remote subtransaction;

[0083] Figure 4 This is a flowchart of calling a remote subtransaction method;

[0084] Figure 5 Schematic diagram of the global transaction process submitted by the two-phase commit coordinator;

[0085] Figure 6 This is a schematic diagram of the process of submitting a database subtransaction;

[0086] Figure 7 This is a schematic diagram of the architecture of a globally serializable multi-database transaction control system. DETAILED DESCRIPTION

[0087] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0088] According to a globally serializable multi-database transaction control method provided by the present invention, Figure 1 For example, it includes:

[0089] Step S1: Before the application service runs, register the multi-database transaction processing logic with the shim layer.

[0090] Specifically, the shim layer provides an interface for registering multi-database transactions for application services. For example, in Java code, you can use the annotation method to mark multi-database transaction functions. Before the application server is started, register the multi-database transaction processing function through the system interface. The processing function defines the global transaction logic when the application service is running, which includes local sub-transactions or remote sub-transaction requests.

[0091] Step S2: When the application service is running, it receives a multi-database transaction request initiated by the client. The shim layer intercepts the multi-database transaction request, generates a global transaction ID and stores it in the thread local variable to mark whether the application service is in the global transaction state, notifies the two-phase commit coordinator to start a global transaction, and waits for the two-phase commit coordinator to return a confirmation result.

[0092] Specifically, during the operation of the application server, the shim layer intercepts the call to the global transaction function in the client request and stores the global transaction ID in the thread local variable, avoiding the problem of needing to modify the underlying database or application code in traditional technologies.

[0093] When the request is a global transaction processing request from the client, the application service that receives the request is defined as the global transaction initiation service for this request.

[0094] If a two-phase commit coordinator is received, that is, the two-phase commit coordinator returns "confirmation", no operation is performed; if no "confirmation" is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended.

[0095] Step S3: After receiving the reply from the two-phase commit coordinator, the shim layer executes the sub-transaction request according to the multi-database transaction processing logic.

[0096] The application service executes the operations included in the global transaction processing request function logic, including local subtransactions and remote subtransactions. For local subtransaction operations, the application service registers the subtransaction with the two-phase commit coordinator, executes the subtransaction and auxiliary operations of the preparation phase in the database, and notifies the two-phase commit coordinator that the subtransaction enters the preparation phase after completion; for remote subtransaction operations, additional information is added to the request header and the corresponding remote service interface is called according to whether the current application service is in a global transaction.

[0097] Specifically, step S3.1: Figure 2 For example, the application service executes a local subtransaction request. The shim layer intercepts the local subtransaction request and generates a subtransaction ID. Whether it is a subtransaction of the global transaction is determined based on whether the thread-local global transaction ID variable has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. Otherwise, continue with subsequent operations. The shim layer corresponding to the database that processes the subtransaction executes the subtransaction and preparation phase auxiliary operations. After completion, the two-phase commit coordinator is notified that the subtransaction is ready.

[0098] The execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, specifically including:

[0099] For databases based on the SSI protocol, the shim layer maintains dependencies between transactions by creating auxiliary transactions to ensure global serializability of transactions. The database starts (Begin Transaction) a local transaction and executes local transaction operations. After the execution is completed, the auxiliary transaction begins, that is, a random row is read from the database's Dummy table (a table with two columns, created before the application service is started), and then the same row is updated with a random value. Finally, the thread local variable is used to determine whether it is in a global transaction. If the thread local variable, that is, the local global transaction ID variable, has been set, the database local transaction and auxiliary transaction are prepared (PrepareTransaction) and the two-phase commit coordinator is notified that the subtransaction has been successfully prepared and the operation is completed after the result is returned; if the thread local variable is not set, the database transaction is committed (Commit Transaction) and the auxiliary transaction is rolled back.

[0100] For databases based on the S2PL protocol, the shim layer maintains the dependencies between transactions by inserting auxiliary write operations during the execution of read-only transactions to ensure the global serializability of transactions. Specifically, the database starts a local transaction and executes operations. After the execution is completed, it determines whether it is in a global transaction through thread local variables. If the local global transaction ID variable has been set and it is a read-only transaction, a local auxiliary write operation is performed (the original subtransaction uses a random value to update a random row in the database's Dummy table), and then the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction has been successfully prepared; if the thread local variable is not set, the database transaction is committed.

[0101] For databases based on other protocols, the shim layer executes subtransactions after acquiring the subtransaction's global lock in the database to ensure global serializability of transactions. Specifically, before the subtransaction is executed, after acquiring the subtransaction global lock, the database starts a local transaction and executes it. After the execution is completed, the thread local variable is used to determine whether it is in the global transaction. If the local global transaction ID variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction has been successfully prepared; if the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

[0102] By performing non-blocking coordination in the preparation phase, the system coordination overhead and latency can be reduced in complex distributed scenarios, the system parallelism can be greatly improved, the overall performance can be significantly improved, and it can adapt to high-concurrency environments.

[0103] Step S3.2: Figure 3For example, the application service executes a remote sub-transaction request, and the shim layer intercepts the remote sub-transaction request. If the local global transaction ID variable has been set, the local application service appends the global transaction ID to the request header, then sends a remote transaction operation request to the remote application service and waits for the result to be returned. Otherwise, no operation is performed.

[0104] By appending the global transaction ID to the request header, modifications to the business code are avoided.

[0105] by Figure 4 For example, after receiving a remote sub-transaction request, the application service needs to check whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as a thread local variable; if not, the application service continues to execute. The application service executes the local sub-transaction request. After completion, the local variable is cleared and the result is returned to the caller.

[0106] The global transaction ID in the request header indicates whether the current subtransaction belongs to a global transaction.

[0107] Step S4: After all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply. After receiving the reply, the thread local variables are eliminated and the request execution result is returned to the client.

[0108] Specifically, step S4.1: Figure 5 For example, after all operations contained in the global transaction processing request function logic are executed, the global transaction initiation service notifies the two-phase commit coordinator to commit the global transaction. After receiving the global transaction commit request, the two-phase commit coordinator needs to determine whether to commit the transaction based on the current status to achieve the atomicity of the global transaction. If committed, a sub-transaction commit request is sent to the application service that registered the sub-transaction; otherwise, a sub-transaction rollback request is sent to the application service that registered the sub-transaction.

[0109] The two-phase commit coordinator operates according to the standard two-phase commit protocol. If any subtransaction fails to prepare (including downtime and restart during subtransaction execution), or the two-phase commit coordinator fails to successfully persist the commit decision or the global transaction has not been committed when it is restarted, its subtransaction will be rolled back. Otherwise, all subtransactions will be committed.

[0110] Step S4.2: Figure 6 For example, after the application service receives a request to commit / rollback a database subtransaction, the shim layer intercepts the request and commits / rolls back the database subtransaction. The shim layer corresponding to the database performs auxiliary operations in the commit phase.

[0111] For databases based on the SSI protocol, roll back auxiliary transactions;

[0112] For databases based on the S2PL protocol, no additional operations are required;

[0113] For databases based on other protocols, the global lock of the subtransaction needs to be released.

[0114] Step S4.3: After all application services complete the sub-transaction commit / rollback operation request and return, the thread local variables are eliminated and the client is notified of the execution result.

[0115] The present invention can be seamlessly integrated into a variety of existing multi-database application systems without modifying the application service logic, database code, and database internal structure, making it easy to integrate into existing application systems, enhancing scalability and maintaining system flexibility. In complex distributed scenarios, it ensures the serializability of global transactions, effectively reduces concurrency conflicts, and enhances system reliability, stability, and consistency.

[0116] According to a globally serializable multi-database transaction control system provided by the present invention, Figure 7 For example, it specifically includes: application service, database server and two-phase commit coordinator.

[0117] Application service: contains multiple application services, responsible for receiving client requests and initiating global transactions. It runs the application service logic, acts as the initiator of transactions, connects to the database server through the network, integrates the shim layer, and can select based on the concurrency control protocol of the database system called by the application service.

[0118] The application server executes local transaction operations on the database server or initiates remote transaction operations to the remote service through the shim layer. The application server feeds back the local transaction execution status to the two-phase commit coordinator.

[0119] Database server: contains a variety of database systems, including actual application data and auxiliary data tables, with its own concurrency control protocol, used to store and manage application data and auxiliary data, support transaction execution and dependency maintenance. The database server determines the content of auxiliary operations in the preparation phase and interacts with the application server, and works with the two-phase transaction coordinator.

[0120] Two-phase commit coordinator: An independent service node that is responsible for coordinating operations of global transactions, namely, managing, preparing, and committing operations, and communicating with application servers to ensure the consistency and durability of global transactions.

[0121] The two-phase commit coordinator performs global transaction control based on feedback and eventually passes the commit or rollback command to each application server. Each application server completes the transaction commit or rollback operation based on the two-phase commit coordinator command.

[0122] The system is constructed in client / server (C / S) mode, and the application server, database server and two-phase commit coordinator are connected through the network.

[0123] The application service registers the multi-database transaction processing logic with the shim layer;

[0124] The application service receives a multi-database transaction request initiated by the client, and the shim layer intercepts and starts a global transaction for the multi-database transaction;

[0125] The shim layer executes sub-transaction requests of multi-database transactions according to the multi-database transaction processing logic;

[0126] The shim layer submits the global transaction to the two-phase commit coordinator and returns the execution results of the multi-database transaction request to the client.

[0127] In more preferred examples, the shim layer provides an interface for registering multi-database transactions for the application service. Before the application service is started, a processing function for the multi-database transactions is registered through the system interface. The processing function defines the global transaction logic including local sub-transaction requests or remote sub-transaction requests when the application service is running.

[0128] The shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in the thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return;

[0129] If a confirmation is received from the two-phase commit coordinator, no operation is performed. If no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended.

[0130] After all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0131] After receiving the reply, the thread local variables are eliminated and the execution result is returned to the client.

[0132] In more preferred embodiments, the sub-transaction request for executing a multi-database transaction includes:

[0133] Local transaction module: The application service executes the local subtransaction request. The shim layer intercepts the local subtransaction request, generates a subtransaction ID, and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. If the thread-local global transaction ID has not been set, the application service continues to execute directly.

[0134] Execute subtransactions and auxiliary operations in the preparation phase based on the shim layer corresponding to the database that processes the subtransactions;

[0135] The shim layer notifies the two-phase commit coordinator that the subtransaction is ready.

[0136] Remote transaction module: The application service executes the remote sub-transaction request. The shim layer intercepts the remote sub-transaction request and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the global transaction ID is appended to the request header, and the remote sub-transaction request is sent to the remote application service and waits for the result to be returned. If the thread-local global transaction ID has not been set, execution continues directly.

[0137] After receiving the remote sub-transaction request, the remote application service determines whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as the thread-local global transaction ID. If not, the execution continues directly.

[0138] The remote application service executes the local transaction module, clears the local variables and returns the results to the application service.

[0139] In more preferred embodiments, the execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, including:

[0140] When the database is an SSI protocol-based database, the database starts a local subtransaction and performs operations, reads a random row from the database's Dummy table, updates the same row with a random value, and determines the thread local variable;

[0141] If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread local variable is set, the database local subtransaction and auxiliary transaction are prepared and the two-phase commit coordinator is notified that the subtransaction is ready and waits for the result to be returned.

[0142] When the database is based on the S2PL protocol, the database starts a local subtransaction and performs operations to determine thread local variables;

[0143] If the thread-local variable is not set, the database transaction is committed. If the thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready.

[0144] When the database is based on other protocols, before the subtransaction is executed, the subtransaction global lock is acquired, the database starts the local subtransaction and performs operations to determine the thread local variables;

[0145] If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

[0146] In more preferred embodiments, the shim layer submits a global transaction to the two-phase commit coordinator including:

[0147] After receiving the request to commit a global transaction, the two-phase commit coordinator determines whether to commit the transaction in the current state according to the standard two-phase commit protocol;

[0148] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or a global transaction that has not been committed when the system is restarted, a sub-transaction rollback request is sent to the application service that registered the sub-transaction;

[0149] Otherwise, a sub-transaction commit request is sent to the application service that registered the sub-transaction.

[0150] After the application service that registers the subtransaction receives the request to commit / rollback the database subtransaction, the shim layer intercepts the request to commit / rollback the database subtransaction, commits / rolls back the database subtransaction, determines the database type, and performs auxiliary operations in the commit phase through the shim layer corresponding to the database.

[0151] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back. If it is a database based on the S2PL protocol, no operation is performed. If it is a database based on other protocols, the global lock of the subtransaction is released.

[0152] After all application services complete the subtransaction commit / rollback operation request and return, the thread local variables are eliminated.

[0153] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A globally serializable multi-database transaction control method, characterized in that: include: Step S1: Instruct the application service to register the multi-database transaction processing logic with the shim layer; Step S2: The application service receives a multi-database transaction request initiated by the client, and instructs the shim layer to intercept and start a global transaction of the multi-database transaction; Step S3: instructing the shim layer to execute the sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic; Step S4: Instruct the shim layer to submit the global transaction to the two-phase commit coordinator and return the execution result of the multi-database transaction request to the client.

2. The globally serializable multi-database transaction control method according to claim 1, characterized in that: In step S1, the shim layer provides an interface for registering multi-database transactions for the application service. Before the application service is started, a processing function for the multi-database transactions is registered through the system interface. The processing function defines the global transaction logic including local sub-transaction requests or remote sub-transaction requests when the application service is running; In step S2, the shim layer intercepts the call of the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in the thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return; If a confirmation is received from the two-phase commit coordinator, no operation is performed. If no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended. After all sub-transaction requests in step S4 are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply; After receiving the reply, the thread local variables are eliminated and the execution result is returned to the client.

3. The globally serializable multi-database transaction control method according to claim 1, characterized in that: The step S3 comprises: Local transaction steps: The application service executes the local subtransaction request, the shim layer intercepts the local subtransaction request, generates a subtransaction ID, and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. If the thread-local global transaction ID has not been set, the application service continues to execute directly. Execute subtransactions and auxiliary operations in the preparation phase based on the shim layer corresponding to the database that processes the subtransactions; The shim layer notifies the two-phase commit coordinator that the subtransaction is ready; Remote transaction steps: The application service executes the remote sub-transaction request. The shim layer intercepts the remote sub-transaction request and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the global transaction ID is appended to the request header, and the remote sub-transaction request is sent to the remote application service and waits for the result to be returned. If the thread-local global transaction ID has not been set, the execution continues directly. After receiving the remote sub-transaction request, the remote application service determines whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as the thread-local global transaction ID. If not, the execution continues directly. The remote application service executes the local transaction steps, clears the local variables and returns the results to the application service.

4. The globally serializable multi-database transaction control method according to claim 3, characterized in that: The execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, including: When the database is an SSI protocol-based database, the database starts a local subtransaction and performs operations, reads a random row from the database's Dummy table, updates the same row with a random value, and determines the thread local variable; If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread local variable is set, the database local subtransaction and auxiliary transaction are prepared and the two-phase commit coordinator is notified that the subtransaction is ready and waits for the result to be returned. When the database is based on the S2PL protocol, the database starts a local subtransaction and performs operations to determine thread local variables; If the thread-local variable is not set, the database transaction is committed. If the thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready. When the database is based on other protocols, before the subtransaction is executed, the subtransaction global lock is acquired, the database starts the local subtransaction and performs operations to determine the thread local variables; If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

5. The globally serializable multi-database transaction control method according to claim 1, characterized in that: The step S4 comprises: Step S4.1: After receiving the request to commit a global transaction, the two-phase commit coordinator determines whether the transaction is committed in the current state according to the standard two-phase commit protocol; If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or a global transaction that has not been committed when the system is restarted, a sub-transaction rollback request is sent to the application service that registered the sub-transaction; Otherwise, a sub-transaction commit request is sent to the application service that registered the sub-transaction; Step S4.2: After the application service of the registered subtransaction receives the request to commit / rollback the database subtransaction, the shim layer intercepts the request to commit / rollback the database subtransaction, commits / rolls back the database subtransaction, determines the database type, and performs auxiliary operations in the commit phase through the shim layer corresponding to the database; If the database type is based on the SSI protocol, the auxiliary transaction is rolled back. If it is based on the S2PL protocol, no operation is performed. If it is based on other protocols, the global lock of the subtransaction is released. Step S4.3: After all application services complete the sub-transaction commit / rollback operation request and return, the thread local variables are eliminated and the client is notified of the execution result of the multi-database transaction request.

6. A globally serializable multi-database transaction control system, characterized in that: include: Application servers, database servers, and two-phase commit coordinators; The application service includes a set number of application services, an integrated shim layer, and is selected based on the concurrency control protocol of the database system being called; The database server contains the actual application data and auxiliary data tables, supports the execution of multiple database transactions and the maintenance of dependencies, and works with the two-phase transaction coordinator; It is built in client / server mode, and the application service, database server and two-phase commit coordinator are connected through the network; The application service registers the multi-database transaction processing logic with the shim layer; The application service receives a multi-database transaction request initiated by the client, and the shim layer intercepts and starts a global transaction for the multi-database transaction; The shim layer executes sub-transaction requests of multi-database transactions according to the multi-database transaction processing logic; The shim layer submits the global transaction to the two-phase commit coordinator and returns the execution results of the multi-database transaction request to the client.

7. The globally serializable multi-database transaction control system according to claim 6, characterized in that: The shim layer provides an interface for registering multi-database transactions for the application service. Before the application service is started, a processing function for multi-database transactions is registered through the system interface. The processing function defines the global transaction logic including local sub-transaction requests or remote sub-transaction requests when the application service is running; The shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in the thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return; If a confirmation is received from the two-phase commit coordinator, no operation is performed. If no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call has failed and ended. After all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply; After receiving the reply, the thread local variables are eliminated and the execution result is returned to the client.

8. The globally serializable multi-database transaction control system according to claim 6, characterized in that: The sub-transaction request for executing a multi-database transaction includes: Local transaction module: The application service executes the local subtransaction request. The shim layer intercepts the local subtransaction request, generates a subtransaction ID, and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the application service sends a subtransaction registration notification containing the global transaction ID and the subtransaction ID to the two-phase commit coordinator. If the thread-local global transaction ID has not been set, the application service continues to execute directly. Execute subtransactions and auxiliary operations in the preparation phase based on the shim layer corresponding to the database that processes the subtransactions; The shim layer notifies the two-phase commit coordinator that the subtransaction is ready; Remote transaction module: The application service executes the remote sub-transaction request. The shim layer intercepts the remote sub-transaction request and determines whether the thread-local global transaction ID has been set. If the thread-local global transaction ID has been set, the global transaction ID is appended to the request header, and the remote sub-transaction request is sent to the remote application service and waits for the result to be returned. If the thread-local global transaction ID has not been set, the execution continues directly. After receiving the remote sub-transaction request, the remote application service determines whether the request header contains the global transaction ID. If so, the global transaction ID in the request header is set as the thread-local global transaction ID. If not, the execution continues directly. The remote application service executes the local transaction module, clears the local variables and returns the results to the application service.

9. The globally serializable multi-database transaction control system according to claim 8, characterized in that: The execution sub-transaction and the auxiliary operations in the preparation phase are differentiated according to different database concurrency control mechanisms, including: When the database is an SSI protocol-based database, the database starts a local subtransaction and performs operations, reads a random row from the database's Dummy table, updates the same row with a random value, and determines the thread local variable; If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread local variable is set, the database local subtransaction and auxiliary transaction are prepared and the two-phase commit coordinator is notified that the subtransaction is ready and waits for the result to be returned. When the database is based on the S2PL protocol, the database starts a local subtransaction and performs operations to determine thread local variables; If the thread-local variable is not set, the database transaction is committed. If the thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready. When the database is based on other protocols, before the subtransaction is executed, the subtransaction global lock is acquired, the database starts the local subtransaction and performs operations to determine the thread local variables; If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread local variable has not been set, the database local transaction is committed and the subtransaction global lock is released.

10. The globally serializable multi-database transaction control system according to claim 6, characterized in that: The shim layer submits a global transaction to the two-phase commit coordinator including: After receiving the request to commit a global transaction, the two-phase commit coordinator determines whether to commit the transaction in the current state according to the standard two-phase commit protocol; If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or a global transaction that has not been committed when the system is restarted, a sub-transaction rollback request is sent to the application service that registered the sub-transaction; Otherwise, a sub-transaction commit request is sent to the application service that registered the sub-transaction; After the application service that registered the subtransaction receives the request to commit / rollback the database subtransaction, the shim layer intercepts the request to commit / rollback the database subtransaction, commits / rolls back the database subtransaction, determines the database type, and performs auxiliary operations in the commit phase through the shim layer corresponding to the database; If the database type is based on the SSI protocol, the auxiliary transaction is rolled back. If it is based on the S2PL protocol, no operation is performed. If it is based on other protocols, the global lock of the subtransaction is released. After all application services complete the subtransaction commit / rollback operation request and return, the thread local variables are eliminated.

Images