Global serializable multi-database transaction control method and system

Abstract

The application provides a globally serializable multi-database transaction control method and system, comprising the following steps: an application service registers multi-database transaction processing logic to a shim layer; a multi-database transaction request initiated by a client is received, and the shim layer intercepts and starts a global transaction of the multi-database transaction; the shim layer executes a sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic; the global transaction is submitted to a two-phase commit coordinator, and an execution result of the multi-database transaction request is returned to the client. The application provides atomicity, durability and global serializability guarantee for the multi-database transaction; based on the shim layer, the concurrent control of the existing database system is reused, and the application can be seamlessly integrated into the existing application; the transaction coordination overhead is low, and the application is suitable for a high concurrency environment; the distributed transaction control has good ACID characteristics; the application does not depend on a specific service implementation platform; the application has low invasiveness, and the transaction control does not affect the writing of normal business code.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of database transaction management, and specifically relates to a globally serializable multi-database transaction control method and system. BACKGROUND

[0002] In recent years, with the widespread application of microservice architecture and serverless architecture, distributed service-oriented applications have become increasingly common. Currently, in these applications, various functions are divided into independent, loosely coupled application services, and each service is equipped with a database system to manage its proprietary data. This distributed structure inevitably causes complex workflows to span multiple services and generates multi-database global transactions. The emergence of multi-database global transactions aims to coordinate data access between multiple database systems to ensure data consistency while accessing different services.

[0003] In the prior art, a multi-database global transaction is usually composed of multiple sub-transactions, each of which is executed on a different database system. Taking a bank application as an example, in this application, different database systems manage users' savings accounts and checking accounts, respectively. To complete a withdrawal operation, a service must read the balance of another service's account, thus forming a multi-database transaction that spans databases. However, although each service can ensure serializability when executing its sub-transaction locally, the overall transaction may still fail to achieve global serializability. For example, when multiple withdrawal requests are made simultaneously, the existing system is prone to concurrency anomalies, i.e., each service may mistakenly approve multiple duplicate withdrawal requests while judging that the balance is sufficient, thus violating business constraints. This situation exposes the limitations of existing systems in handling global data consistency.

[0004] To solve the above problems, some existing technologies propose multi-database transaction protocols to ensure global serializability. However, these protocols usually rely on conservative assumptions, resulting in significant performance degradation. For example, the ticket method and altruistic locking strategy do not allow parallel execution of sub-transactions on the same database system, avoiding errors by globally ordering multi-database transactions, even if there is no actual conflict between these transactions. This method of global ordering often results in a significant decrease in performance, limiting the overall efficiency of the system.

[0005] Another prior art attempts to improve performance by doing more concurrency control at the application layer. These schemes typically give up the guarantee of global serializability and only provide snapshot isolation. For example, the Epoxy protocol and the Cherry Garcia protocol implement snapshot isolation across multiple databases, but they require the application program to do inter-subtransaction coordination at the application layer and use the underlying database system as version storage. Such approaches not only limit the use of database advanced features such as uniqueness constraints, foreign key constraints, and predicate-based updates, but also force the application program to mix data and metadata, resulting in the underlying database being inaccessible directly. At the same time, these schemes only guarantee snapshot isolation, so concurrent anomalies can still occur.

[0006] The concurrency control mechanisms commonly adopted by existing database systems, such as serializable snapshot isolation or strict two-phase locking, cannot achieve global serializability in a multi-database environment. This is because these systems only control the execution of local transactions, while the commit order of global transactions is usually controlled by a two-phase commit (2PC) coordinator according to the message reception order, which cannot ensure global data consistency. At the same time, the current prior art does not have any system that can achieve both high performance and global serializability without modifying existing application programs and database systems.

[0007] The patent document "Global transaction manager and transaction processing method for distributed system" (CN201410214371) discloses a global transaction manager including an application device, a transaction management device, a resource management device, a transaction rollback device, a fault recovery device, and an information coordination device, providing strong isolation, strict consistency, and distributed transaction control with good ACID characteristics. However, it still cannot guarantee global serializability of transactions, and does not utilize the concurrency control mechanism of the database, making it impossible to achieve low code modification and high concurrency performance.

[0008] Therefore, there is a need for a new solution to ensure global serializability of multi-database transactions while avoiding major modifications to existing systems and application programs, providing more efficient performance. SUMMARY

[0009] In view of the defects in the prior art, the purpose of the present application is to provide a global serializable multi-database transaction control method and system.

[0010] According to the global serializable multi-database transaction control method provided by the present application, the method comprises the following steps:

[0011] Step S1: Register the multi-database transaction processing logic with the shim layer by the application service;

[0012] Step S2: The application service receives the multi-database transaction request initiated by the client, and the shim layer intercepts and starts the global transaction of the multi-database transaction;

[0013] Step S3: The shim layer executes the sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic;

[0014] Step S4: The shim layer submits the global transaction to the two-phase commit coordinator and returns the execution result of the multi-database transaction request to the client.

[0015] Preferably, the shim layer provides an interface for the application service to register the multi-database transaction in step S1. Before the application service starts, the processing function of the multi-database transaction is registered through the system interface, and the global transaction logic of the application service running time containing local sub-transaction request or remote sub-transaction request is defined in the processing function.

[0016] In step S2, the shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in a thread local variable, notifies the two-phase commit coordinator to start a global transaction, and waits for the return of the two-phase commit coordinator.

[0017] If the return confirmation of the two-phase commit coordinator is received, no operation is performed, and if the return confirmation of the two-phase commit coordinator is not received, the client is notified that the multi-database transaction request call fails and ends.

[0018] In step S4, after all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0019] After receiving the reply, the thread local variable is eliminated and the execution result is returned to the client.

[0020] Preferably, step S3 includes:

[0021] Local transaction step: The application service executes the local sub-transaction request, the shim layer intercepts the local sub-transaction request, generates a sub-transaction ID, judges whether the thread local global transaction ID has been set, if the thread local global transaction ID has been set, the application service sends a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator, and if the thread local global transaction ID has not been set, it directly continues to execute;

[0022] Execute the sub-transaction and prepare phase auxiliary operation based on the shim layer corresponding to the database processing the sub-transaction;

[0023] The shim layer notifies the two-phase commit coordinator that the sub-transaction is ready.

[0024] Remote transaction step: the application service executes the remote sub-transaction request, the shim layer intercepts the remote sub-transaction request, judges whether the thread local global transaction ID has been set, if the thread local global transaction ID has been set, the global transaction ID is attached to the request header, the remote sub-transaction request is sent to the remote application service and the result is waited to be returned, if the thread local global transaction ID has not been set, the execution is directly continued.

[0025] After the remote application service receives the remote sub-transaction request, it is judged whether the global transaction ID is contained in the request header, if the global transaction ID is contained, the global transaction ID in the request header is set as the thread local global transaction ID, if the global transaction ID is not contained, the execution is directly continued.

[0026] The remote application service executes the local transaction step, clears the local variable and returns the result to the application service.

[0027] Preferably, the execution sub-transaction and preparation phase auxiliary operation are distinguished according to different database concurrency control mechanisms, including:

[0028] When the database is a database based on the SSI protocol, the database starts the local sub-transaction and executes the operation, reads a random row from the Dummy table of the database, updates the same row using a random value, and judges the thread local variable.

[0029] If the thread local variable has not been set, the database transaction is committed and the auxiliary transaction is rolled back, if the thread local variable has been set, the database local sub-transaction and the auxiliary transaction are prepared and the two-phase commit coordinator is notified that the sub-transaction is ready and the result is waited to be returned.

[0030] When the database is a database based on the S2PL protocol, the database starts the local sub-transaction and executes the operation, judges the thread local variable.

[0031] If the thread local variable has not been set, the database transaction is committed, if the thread local variable has been set and is a read-only transaction, the atomic transaction updates a random row in the Dummy table of the database using a random value, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction is ready.

[0032] When the database is a database based on other protocols, before the sub-transaction is executed, the sub-transaction global lock is acquired, the database starts the local sub-transaction and executes the operation, judges the thread local variable.

[0033] If the thread local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction is ready, if the thread local variable has not been set, the database local transaction is committed and the sub-transaction global lock is released.

[0034] Preferably, the step S4 includes:

[0035] Step S4.1: After receiving the request of global transaction commit, the two-phase commit coordinator determines whether the current state is to commit the transaction according to the protocol of the standard two-phase commit;

[0036] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or the global transaction that has not been committed when the server is restarted, the application service that registers the sub-transaction is sent a sub-transaction rollback request;

[0037] Otherwise, the application service that registers the sub-transaction is sent a sub-transaction commit request.

[0038] Step S4.2: After receiving the request of committing / rolling back the database sub-transaction, the shim layer intercepts the request of committing / rolling back the database sub-transaction, commits / rolls back the database sub-transaction, and determines the database type to perform the commit phase auxiliary operation through the shim layer corresponding to the database;

[0039] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back, if the database type is a database based on the S2PL protocol, no operation is performed, and if the database type is a database based on other protocols, the global lock of the sub-transaction is released.

[0040] Step S4.3: After all the application services complete the sub-transaction commit / rollback operation request and return, the thread local variable is eliminated, and the client is notified of the execution result of the multi-database transaction request.

[0041] According to the multi-database transaction control system provided by the application, the application service, the database server, and the two-phase commit coordinator are included.

[0042] The application service includes a set number of application services, integrates a shim layer, and is selected based on the concurrency control protocol of the called database system.

[0043] The database server includes actual application data and auxiliary data tables, supports the execution of multi-database transactions and the maintenance of dependency relationships, and cooperates with the two-phase transaction coordinator.

[0044] The client / server mode is adopted for construction, and the application service and the database server, and the two-phase commit coordinator are connected through a network.

[0045] The application service registers the multi-database transaction processing logic with the shim layer;

[0046] The application service receives the multi-database transaction request initiated by the client, the shim layer intercepts and starts the global transaction of the multi-database transaction;

[0047] The shim layer executes the sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic;

[0048] The shim layer submits the global transaction to the two-phase commit coordinator and returns the execution result of the multi-database transaction request to the client.

[0049] Preferably, the shim layer provides an interface for the application service to register the multi-database transaction, and before the application service is started, the processing function of the multi-database transaction is registered through the system interface, and the global transaction logic of the application service running time containing local sub-transaction requests or remote sub-transaction requests is defined in the processing function.

[0050] The shim layer intercepts the call to the global transaction function in the multi-database transaction request initiated by the client received by the application service, generates a global transaction ID and stores it in a thread-local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the return of the two-phase commit coordinator;

[0051] If a confirmation is received from the two-phase commit coordinator, no operation is performed, and if no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call fails and ends.

[0052] After all sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0053] After receiving the reply, the thread-local variable is eliminated and the execution result is returned to the client.

[0054] Preferably, the sub-transaction request for executing the multi-database transaction includes:

[0055] Local transaction module: the application service executes the local sub-transaction request, the shim layer intercepts the local sub-transaction request, generates a sub-transaction ID, judges whether the thread-local global transaction ID has been set, if the thread-local global transaction ID has been set, the application service will send a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator, if the thread-local global transaction ID has not been set, it will continue to execute directly;

[0056] Perform sub-transaction and preparation phase auxiliary operations based on the shim layer corresponding to the database processing the sub-transaction;

[0057] The shim layer notifies the two-phase commit coordinator that the sub-transaction is ready.

[0058] Remote transaction module: the application service executes the remote sub-transaction request, the shim layer intercepts the remote sub-transaction request, judges whether the thread-local global transaction ID has been set, if the thread-local global transaction ID has been set, the global transaction ID is attached to the request header, the remote sub-transaction request is sent to the remote application service and waits for the result to be returned, if the thread-local global transaction ID has not been set, it will continue to execute directly.

[0059] The remote application service receives the remote sub-transaction request, judges whether the global transaction ID is contained in the request header, sets the global transaction ID in the request header as the thread local global transaction ID if the global transaction ID is contained, and directly continues to execute if the global transaction ID is not contained.

[0060] The remote application service executes the local transaction module, clears the local variable, and returns the result to the application service.

[0061] Preferably, the execution sub-transaction and preparation phase auxiliary operation are distinguished according to different database concurrency control mechanisms, including:

[0062] When the database is a database based on the SSI protocol, the database starts a local sub-transaction and executes an operation, reads a random row from the Dummy table of the database, updates the same row using a random value, and judges the thread local variable.

[0063] If the thread local variable is not set, the database transaction is committed and the auxiliary transaction is rolled back, and if the thread local variable is set, the database local sub-transaction and the auxiliary transaction are prepared and the two-phase commit coordinator is notified that the sub-transaction is ready and waits for the result to be returned.

[0064] When the database is a database based on the S2PL protocol, the database starts a local sub-transaction and executes an operation, judges the thread local variable.

[0065] If the thread local variable is not set, the database transaction is committed, and if the thread local variable is set and is a read-only transaction, the atomic transaction updates a random row in the Dummy table of the database using a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the sub-transaction is ready.

[0066] When the database is a database based on other protocols, before the sub-transaction is executed, the sub-transaction global lock is acquired, the database starts a local sub-transaction and executes an operation, and judges the thread local variable.

[0067] If the thread local variable is set, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction is ready, and if the thread local variable is not set, the database local transaction is committed and the sub-transaction global lock is released.

[0068] Preferably, the pad layer submits the global transaction to the two-phase commit coordinator, including:

[0069] After the two-phase commit coordinator receives the request for submitting the global transaction, it judges whether the current state is a transaction according to the standard two-phase commit protocol.

[0070] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or the global transaction that has not been committed when the machine is restarted, the application service that registers the sub-transaction is sent a sub-transaction rollback request.

[0071] Conversely, a sub-transaction commit request is sent to the application service registered with the sub-transaction.

[0072] After the application service registered with the sub-transaction receives the request for committing / rolling back the database sub-transaction, the shim layer intercepts the request for committing / rolling back the database sub-transaction, commits / rolls back the database sub-transaction, and judges the database type to perform the commit phase auxiliary operation through the shim layer corresponding to the database.

[0073] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back; if the database type is a database based on the S2PL protocol, no operation is performed; and if the database type is a database based on other protocols, the global lock of the sub-transaction is released.

[0074] After all the application services complete the sub-transaction commit / rollback operation request and return, the thread local variable is eliminated.

[0075] Compared with the prior art, the present application has the following beneficial effects:

[0076] 1. The present application ensures the global serializability of multi-database transactions and uses two-phase commit to realize the atomicity and durability of transactions, can ensure the global serializability of global transactions in complex distributed scenarios, effectively reduces concurrent conflicts, and enhances the reliability, stability and consistency of the system.

[0077] 2. The shim layer used in the present application reuses the concurrency control of the existing database system, can be seamlessly integrated into various existing multi-database application systems, does not need to modify the application service logic, database code and database internal structure, is convenient for integration into existing application systems, enhances the scalability, and maintains the flexibility of the system.

[0078] 3. The present application reduces system coordination overhead and delay, greatly improves system parallelism, significantly improves overall performance, and adapts to a high-concurrency environment by performing non-blocking coordination in the preparation phase in complex distributed scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0079] Other features, objects and advantages of the present application will become more apparent after reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0080] Figure 1 It is a global serializable multi-database transaction control method overall flow schematic diagram;

[0081] Figure 2 It is an execution local sub-transaction request flow schematic diagram;

[0082] Figure 3 It is an execution remote sub-transaction request flow schematic diagram;

[0083] Figure 4 Flowchart for calling remote sub-transaction method;

[0084] Figure 5 Flowchart for two-phase commit coordinator to commit global transaction;

[0085] Figure 6 Flowchart for committing database sub-transaction;

[0086] Figure 7 Architectural diagram of global serializable multi-database transaction control system. DETAILED DESCRIPTION

[0087] The application will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the application. These are within the scope of the application.

[0088] According to the global serializable multi-database transaction control method provided by the application, the application server is registered with the shim layer before running, and the shim layer is registered with the multi-database transaction processing logic. Figure 1 For example, the method specifically includes the following steps:

[0089] Step S1: Before the application service runs, register the multi-database transaction processing logic with the shim layer.

[0090] Specifically, the shim layer provides an interface for the application service to register the multi-database transaction, for example, the multi-database transaction function can be marked by an annotation method in Java code. Before the application server starts, the processing function of the multi-database transaction is registered through the system interface, and the processing function defines the global transaction logic of the application service when running, which includes local sub-transaction or remote sub-transaction request.

[0091] Step S2: When the application service runs, receive the multi-database transaction request initiated by the client. The shim layer intercepts the multi-database transaction request, generates a global transaction ID and stores it in a thread local variable to mark whether the application service is in a global transaction state, informs the two-phase commit coordinator to start a global transaction, and waits for the two-phase commit coordinator to return the confirmation result.

[0092] Specifically, during the running of the application server, the shim layer intercepts the call to the global transaction function in the client request and stores the global transaction ID in the thread local variable, avoiding the problem of modifying the underlying database or application code in the traditional technology.

[0093] When the request is a global transaction processing request from the client, the application service that accepts the request is defined as the global transaction initiation service for the current request.

[0094] If the two-phase commit coordinator is received, i.e., the two-phase commit coordinator returns "confirmation", no operation is performed; if no "confirmation" is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request call fails and ends.

[0095] Step S3: After receiving the reply of the two-phase commit coordinator, the shim layer executes the sub-transaction request according to the multi-database transaction processing logic.

[0096] The application service performs the operations included in the global transaction processing request function logic, including local sub-transactions and remote sub-transactions. For local sub-transaction operations, the application service registers the sub-transaction with the two-phase commit coordinator, performs the sub-transaction and the preparation phase auxiliary operation in the database, and after the end, notifies the two-phase commit coordinator that the sub-transaction enters the preparation phase; for remote sub-transaction operations, additional information is added in the request header according to whether the current application service is in the global transaction, and the corresponding remote service interface is called.

[0097] Specifically, step S3.1: taking Figure 2 as an example, the application service executes the local sub-transaction request. The shim layer intercepts the local sub-transaction request and generates a sub-transaction ID. According to whether the thread local global transaction ID variable has been set, it is judged whether it belongs to the sub-transaction of the global transaction. If the thread local global transaction ID has been set, the application service will send a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator. Otherwise, the subsequent operation is continued. The sub-transaction and the preparation phase auxiliary operation are performed based on the shim layer corresponding to the database processing the sub-transaction, and after completion, the two-phase commit coordinator is notified that the sub-transaction is ready.

[0098] The execution of the sub-transaction and the preparation phase auxiliary operation is distinguished according to different database concurrency control mechanisms, specifically including:

[0099] For the database based on SSI protocol, the shim layer maintains the dependency between transactions by creating auxiliary transactions to ensure the global serializability of transactions. The database starts a local transaction and executes local transaction operations. After the execution ends, an auxiliary transaction is started, that is, a random row is read from the Dummy table (a table containing two columns, created before the application service starts) of the database, and then the same row is updated using a random value. Finally, it is judged whether it is in the global transaction through the thread local variable. If the thread local variable, that is, the local global transaction ID variable, has been set, the database local transaction and the auxiliary transaction are prepared (Prepare Transaction) and the two-phase commit coordinator is notified that the sub-transaction has been successfully prepared and waits for the result to be returned after the operation is completed. If the thread local variable has not been set, the database transaction is committed (Commit Transaction) and the auxiliary transaction is rolled back.

[0100] For the database based on S2PL protocol, the shim layer maintains the dependency between transactions by inserting auxiliary write operations during the execution of read-only transactions to ensure the global serializability of transactions. Specifically, the database starts a local transaction and executes operations. After the execution ends, it is judged whether it is in the global transaction through the thread local variable. If the local global transaction ID variable has been set and it is a read-only transaction, the local auxiliary write operation (the original sub-transaction updates a random row in the Dummy table of the database using a random value) is executed, and then the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction has been successfully prepared. If the thread local variable has not been set, the database transaction is committed.

[0101] For the database based on other protocols, the shim layer ensures the global serializability of transactions by executing the sub-transaction after acquiring the global lock of the sub-transaction in the database. Specifically, before the execution of the sub-transaction, after acquiring the global lock of the sub-transaction, the database starts a local transaction and executes. After the execution ends, it is judged whether it is in the global transaction through the thread local variable. If the local global transaction ID variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction has been successfully prepared. If the thread local variable has not been set, the database local transaction is committed and the global lock of the sub-transaction is released.

[0102] By performing non-blocking coordination in the preparation phase, the system coordination overhead and delay are reduced in complex distributed scenarios, the system parallelism is greatly improved, the overall performance is significantly improved, and the high-concurrency environment is adapted.

[0103] Step S3.2: with Figure 3For example, when an application service executes a remote sub-transaction request, the padding layer intercepts the request. If a local global transaction ID variable is set, the local application service appends the global transaction ID to the request header, then sends the remote transaction operation request to the remote application service and waits for the result. Otherwise, no operation is performed.

[0104] By appending the global transaction ID to the request header, modifications to the business logic code were avoided.

[0105] by Figure 4 For example, after receiving a remote sub-transaction request, the remote application service checks if the request header contains a global transaction ID. If it does, the global transaction ID in the request header is set as a thread-local variable; otherwise, execution continues. The application service executes the local sub-transaction request. After completion, it clears the local variables and returns the result to the caller.

[0106] The global transaction ID in the request header indicates whether the current subtransaction belongs to a certain global transaction.

[0107] Step S4: After all sub-transaction requests have been executed, the padding layer submits the global transaction to the two-phase commit coordinator and awaits a response. Upon receiving the response, it clears thread-local variables and returns the request execution result to the client.

[0108] Specifically, step S4.1: with Figure 5 For example, after all operations in the global transaction processing request function are completed, the global transaction initiation service notifies the two-phase commit coordinator to commit the global transaction. Upon receiving the global transaction commit request, the two-phase commit coordinator needs to determine whether to commit the transaction based on the current state to achieve the atomicity of the global transaction. If committed, a sub-transaction commit request is sent to the application service that registered the sub-transaction; otherwise, a sub-transaction rollback request is sent to the application service that registered the sub-transaction.

[0109] The two-phase commit coordinator operates according to the standard two-phase commit protocol. If any sub-transaction fails to prepare (including during sub-transaction execution and restart), or if the two-phase commit coordinator fails to persist the commit decision or if a global transaction that has not yet been committed at the time of a crash and restart fails, its sub-transactions will be rolled back. Conversely, if the coordinator succeeds, all sub-transactions will be committed.

[0110] Step S4.2: with Figure 6 For example, after the application service receives a request to commit / rollback a database sub-transaction, the padding layer intercepts the request and commits / rolls back the database sub-transaction. The commit phase auxiliary operations are performed through the corresponding padding layer for the database.

[0111] For databases based on the SSI protocol, roll back auxiliary transactions;

[0112] For the database based on S2PL protocol, no additional operation is needed.

[0113] For the database based on other protocols, the global lock of the sub-transaction needs to be released.

[0114] Step S4.3: after all application services complete the sub-transaction commit / rollback operation request return, the thread local variable is eliminated and the client is informed of the execution result.

[0115] The application can be seamlessly integrated into various existing multi-database application systems without modifying the application service logic, database code and database internal structure, facilitating integration into existing application systems, enhancing scalability, maintaining system flexibility. In a complex distributed scenario, the serializability of global transactions is ensured, effectively reducing concurrency conflicts, enhancing system reliability, stability and consistency.

[0116] According to the global serializable multi-database transaction control system provided by the application, for example, specifically includes: application service, database server and two-phase commit coordinator. Figure 7

[0117] Application service: contains multiple application services, responsible for receiving client requests and initiating global transactions. Run the application service logic as the initiator of the transaction, connect the database server through the network, integrate the shim layer, and select the concurrency control protocol of the database system based on the application service call.

[0118] The application server executes local transaction operations or initiates remote transaction operations to the remote service through the shim layer to the database server, and the application server feeds back the local transaction execution state to the two-phase commit coordinator.

[0119] Database server: contains multiple database systems, contains actual application data and auxiliary data table, has its own concurrency control protocol, is used for storing and managing application data and auxiliary data, supports transaction execution and dependency maintenance. The database server determines the content of the preparation phase auxiliary operation and the application server interaction, and cooperates with the two-phase transaction coordinator.

[0120] Two-phase commit coordinator: an independent service node responsible for coordinating global transaction operations, i.e. management, preparation and submission operations, and communicating with the application server to ensure the consistency and persistence of global transactions.

[0121] The two-phase commit coordinator controls the global transaction according to the feedback, and finally delivers the commit or rollback command to each application server, and each application server completes the transaction commit or rollback operation based on the two-phase commit coordinator command.

[0122] ​The system adopts a client / server (C / S) mode to construct, and the application server and the database server are connected through a network with the two-phase commit coordinator.

[0123] The application service registers the multi-database transaction processing logic with the shim layer;

[0124] The application service receives a multi-database transaction request initiated by a client, and the shim layer intercepts and starts a global transaction of the multi-database transaction;

[0125] The shim layer executes a sub-transaction request of the multi-database transaction according to the multi-database transaction processing logic;

[0126] The shim layer submits the global transaction to the two-phase commit coordinator and returns an execution result of the multi-database transaction request to the client.

[0127] In more preferred examples, the shim layer provides an interface for the application service to register the multi-database transaction, and before the application service is started, a processing function of the multi-database transaction is registered through a system interface, and the processing function defines a global transaction logic of the application service running time containing a local sub-transaction request or a remote sub-transaction request.

[0128] The shim layer intercepts a call to the global transaction function in the multi-database transaction request initiated by the client and received by the application service, generates a global transaction ID and stores it in a thread local variable, notifies the two-phase commit coordinator to start a global transaction and waits for a return from the two-phase commit coordinator;

[0129] If a return confirmation from the two-phase commit coordinator is received, no operation is performed, and if no return confirmation from the two-phase commit coordinator is received, the client is notified that the multi-database transaction request call fails and ends.

[0130] After all the sub-transaction requests are executed, the shim layer submits the global transaction to the two-phase commit coordinator and waits for a reply;

[0131] After the reply is received, the thread local variable is eliminated and an execution result is returned to the client.

[0132] In more preferred examples, the execution of the sub-transaction request of the multi-database transaction includes:

[0133] The local transaction module: the application service executes a local sub-transaction request, the shim layer intercepts the local sub-transaction request, generates a sub-transaction ID, judges whether a thread local global transaction ID has been set, if the thread local global transaction ID has been set, the application service sends a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator, and if the thread local global transaction ID has not been set, the execution is directly continued;

[0134] performing the sub-transaction and the prepare phase auxiliary operation on the corresponding shim layer of the database processing the sub-transaction;

[0135] The shim layer notifies the two-phase commit coordinator that the sub-transaction is ready.

[0136] Remote transaction module: the application service executes the remote sub-transaction request, the shim layer intercepts the remote sub-transaction request, judges whether the thread local global transaction ID has been set, if the thread local global transaction ID has been set, the global transaction ID is attached to the request header, the remote sub-transaction request is sent to the remote application service and the result is waited to be returned, if the thread local global transaction ID has not been set, it is directly continued to be executed.

[0137] After the remote application service receives the remote sub-transaction request, it judges whether the global transaction ID is contained in the request header, if it is contained, the global transaction ID in the request header is set as the thread local global transaction ID, if it is not contained, it is directly continued to be executed;

[0138] The remote application service executes the local transaction module, clears the local variable and returns the result to the application service.

[0139] In more preferred examples, the performing the sub-transaction and the prepare phase auxiliary operation is distinguished according to different database concurrency control mechanisms, including:

[0140] When the database is a database based on the SSI protocol, the database starts a local sub-transaction and executes an operation, reads a random row from the Dummy table of the database, updates the same row using a random value, and judges the thread local variable;

[0141] If the thread local variable has not been set, the database transaction is committed and the auxiliary transaction is rolled back, if the thread local variable has been set, the database local sub-transaction and the auxiliary transaction are prepared and the two-phase commit coordinator is notified that the sub-transaction is ready and the result is waited to be returned.

[0142] When the database is a database based on the S2PL protocol, the database starts a local sub-transaction and executes an operation, judges the thread local variable;

[0143] If the thread local variable has not been set, the database transaction is committed, if the thread local variable has been set and is a read-only transaction, the atomic transaction updates a random row in the Dummy table of the database using a random value, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction is ready.

[0144] When the database is a database based on other protocols, before the sub-transaction is executed, a sub-transaction global lock is acquired, the database starts a local sub-transaction and executes an operation, judges the thread local variable;

[0145] If the thread local variable is set, the database transaction is prepared and the two-phase commit coordinator is notified that the sub-transaction is ready, and if the thread local variable is not set, the database local transaction is committed and the sub-transaction global lock is released.

[0146] In more preferred examples, the pad layer submitting the global transaction to the two-phase commit coordinator comprises:

[0147] After the two-phase commit coordinator receives the request for global transaction submission, it determines whether the current state is to commit the transaction according to the standard two-phase commit protocol.

[0148] If there is a sub-transaction preparation failure or the two-phase commit coordinator fails to successfully persist the commit decision or the global transaction has not been committed when the system is restarted, the application service registering the sub-transaction is sent a sub-transaction rollback request.

[0149] Otherwise, the application service registering the sub-transaction is sent a sub-transaction commit request.

[0150] After the application service registering the sub-transaction receives the request for committing / rolling back the database sub-transaction, the pad layer intercepts the request for committing / rolling back the database sub-transaction, commits / rolls back the database sub-transaction, and determines the database type to perform the commit phase auxiliary operation through the pad layer corresponding to the database.

[0151] If the database type is a database based on the SSI protocol, the auxiliary transaction is rolled back, if the database type is a database based on the S2PL protocol, no operation is performed, and if the database type is a database based on other protocols, the global lock of the sub-transaction is released.

[0152] After all the application services complete the sub-transaction commit / rollback operation request and return, the thread local variable is eliminated.

[0153] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the specific embodiments described above, and various changes or modifications can be made by those skilled in the art within the scope of the claims, which do not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A globally serializable multi-database transaction control method, characterized in that, include: Step S1: Instruct the application service to register multi-database transaction processing logic with the pad layer; Step S2: When the application service receives a multi-database transaction request from the client, it instructs the padding layer to intercept and start a global transaction for the multi-database transaction. Step S3: Instruct the pad layer to execute sub-transaction requests of the multi-database transaction according to the multi-database transaction processing logic; Step S4: Instruct the pad layer to commit the global transaction to the two-phase commit coordinator, and return the execution result of the multi-database transaction request to the client; The application service includes a set number of application services, an integrated pad layer, and is selected based on the concurrency control protocol of the database system being called; The pad layer provides an interface for the application service to register multiple database transactions. Before the application service starts, the processing function for multiple database transactions is registered through the system interface. The processing function defines the global transaction logic that includes local sub-transaction requests or remote sub-transaction requests when the application service runs. The gasket layer interception application service receives a call to the global transaction function in a multi-database transaction request initiated by the client, generates a global transaction ID and stores it in a thread-local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return.

2. The globally serializable multi-database transaction control method according to claim 1, characterized in that, If a confirmation is received from the two-phase commit coordinator, no action is taken; if no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request has failed and the process ends. After all sub-transaction requests in step S4 are completed, the padding layer submits the global transaction to the two-phase commit coordinator and waits for a response. Upon receiving the response, the thread-local variable is cleared and the execution result is returned to the client.

3. The globally serializable multi-database transaction control method according to claim 1, characterized in that, Step S3 includes: Local transaction steps: The application service executes a local sub-transaction request, the padding layer intercepts the local sub-transaction request, generates a sub-transaction ID, and determines whether a thread-local global transaction ID has been set. If a thread-local global transaction ID has been set, the application service will send a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator. If a thread-local global transaction ID has not been set, execution will continue directly. The sub-transaction and preparation phase auxiliary operations are executed based on the padding layer corresponding to the database that processes the sub-transaction; The gasket layer notifies the two-phase commit coordinator that the subtransaction is ready; Remote transaction steps: The application service executes a remote sub-transaction request. The padding layer intercepts the remote sub-transaction request, determines whether a thread-local global transaction ID has been set, and if a thread-local global transaction ID has been set, it appends the global transaction ID to the request header, sends the remote sub-transaction request to the remote application service, and waits for the result to return. If a thread-local global transaction ID has not been set, it continues execution directly. After receiving a remote sub-transaction request, the remote application service checks whether the request header contains a global transaction ID. If it does, the global transaction ID in the request header is set to the thread-local global transaction ID. If it does not, execution continues directly. The remote application service executes local transaction steps, clears local variables, and returns the results to the application service.

4. The globally serializable multi-database transaction control method according to claim 3, characterized in that, The execution of sub-transactions and auxiliary operations during the preparation phase are differentiated according to different database concurrency control mechanisms, including: When the database is based on the SSI protocol, the database starts a local sub-transaction and executes the operation, reads a random row from the database's Dummy table, updates the same row with the random value, and checks the thread-local variable; If no thread-local variable is set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread-local variable is set, the database local sub-transaction and auxiliary transaction are prepared, and the two-phase commit coordinator is notified that the sub-transaction is ready and the result is awaited. When the database is based on the S2PL protocol, the database starts a local sub-transaction and executes the operation, checking thread-local variables; If no thread-local variable is set, the database transaction is committed. If a thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready. When the database is based on other protocols, before the sub-transaction is executed, the global lock of the sub-transaction is acquired, the database starts the local sub-transaction and executes the operation, and the thread-local variables are checked; If the thread-local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread-local variable has not been set, the database-local transaction is committed and the subtransaction global lock is released.

5. The globally serializable multi-database transaction control method according to claim 1, characterized in that, Step S4 includes: Step S4.1: After receiving the request to commit the global transaction, the two-phase commit coordinator determines whether to commit the transaction based on the standard two-phase commit protocol. If a subtransaction fails to prepare, the two-phase commit coordinator fails to persist the commit decision, or a global transaction has not yet been committed when the system restarts, a subtransaction rollback request is sent to the application service that registered the subtransaction. Conversely, a subtransaction commit request is sent to the application service that registered the subtransaction; Step S4.2: After the application service that registered the sub-transaction receives the request to commit / roll back the database sub-transaction, the padding layer intercepts the request to commit / roll back the database sub-transaction, commits / rolls back the database sub-transaction, and determines the database type by performing auxiliary operations in the commit phase through the padding layer corresponding to the database. If the database type is based on the SSI protocol, roll back the auxiliary transaction; if it is based on the S2PL protocol, do nothing; if it is based on other protocols, release the global lock of the sub-transaction. Step S4.3: After all application services complete the sub-transaction commit / rollback operation requests and return, eliminate thread-local variables and notify the client of the execution result of the multi-database transaction request.

6. A globally serializable multi-database transaction control system, characterized in that, include: Application services, database servers, and two-phase commit coordinator; The application service includes a set number of application services, an integrated pad layer, and is selected based on the concurrency control protocol of the database system being called; The database server contains actual application data and auxiliary data tables, supports the execution of multiple database transactions and the maintenance of dependencies, and works in conjunction with a two-phase transaction coordinator. It is built using a client / server model, with application services, database servers, and two-phase commit coordinators connected via a network. The application service registers multi-database transaction processing logic with the pad layer; When the application service receives a multi-database transaction request from the client, the padding layer intercepts it and starts a global transaction for the multi-database transaction. The pad layer executes sub-transaction requests of the multi-database transaction according to the multi-database transaction processing logic; The pad layer submits the global transaction to the two-phase commit coordinator and returns the execution results of the multi-database transaction request to the client; The pad layer provides an interface for the application service to register multiple database transactions. Before the application service starts, the processing function for multiple database transactions is registered through the system interface. The processing function defines the global transaction logic that includes local sub-transaction requests or remote sub-transaction requests when the application service runs. The gasket layer interception application service receives a call to the global transaction function in a multi-database transaction request initiated by the client, generates a global transaction ID and stores it in a thread-local variable, notifies the two-phase commit coordinator to start a global transaction and waits for the two-phase commit coordinator to return.

7. The globally serializable multi-database transaction control system according to claim 6, characterized in that, If a confirmation is received from the two-phase commit coordinator, no action is taken; if no confirmation is received from the two-phase commit coordinator, the client is notified that the multi-database transaction request has failed and the process ends. After all sub-transaction requests have been executed, the padding layer submits the global transaction to the two-phase commit coordinator and waits for a response. Upon receiving the response, the thread-local variable is cleared and the execution result is returned to the client.

8. The globally serializable multi-database transaction control system according to claim 6, characterized in that, The sub-transaction request for executing multiple database transactions includes: Local Transaction Module: The application service executes local sub-transaction requests, the padding layer intercepts the local sub-transaction requests, generates a sub-transaction ID, and determines whether a thread-local global transaction ID has been set. If a thread-local global transaction ID has been set, the application service will send a sub-transaction registration notification containing the global transaction ID and the sub-transaction ID to the two-phase commit coordinator. If a thread-local global transaction ID has not been set, execution will continue directly. The sub-transaction and preparation phase auxiliary operations are executed based on the padding layer corresponding to the database that processes the sub-transaction; The gasket layer notifies the two-phase commit coordinator that the subtransaction is ready; Remote Transaction Module: The application service executes remote sub-transaction requests. The padding layer intercepts the remote sub-transaction requests, determines whether a thread-local global transaction ID has been set, and if a thread-local global transaction ID has been set, it appends the global transaction ID to the request header, sends the remote sub-transaction request to the remote application service, and waits for the result to return. If a thread-local global transaction ID has not been set, it continues execution directly. After receiving a remote sub-transaction request, the remote application service checks whether the request header contains a global transaction ID. If it does, the global transaction ID in the request header is set to the thread-local global transaction ID. If it does not, execution continues directly. The remote application service executes the local transaction module, clears local variables, and returns the results to the application service.

9. The globally serializable multi-database transaction control system according to claim 8, characterized in that, The execution of sub-transactions and auxiliary operations during the preparation phase are differentiated according to different database concurrency control mechanisms, including: When the database is based on the SSI protocol, the database starts a local sub-transaction and executes the operation, reads a random row from the database's Dummy table, updates the same row with the random value, and checks the thread-local variable; If no thread-local variable is set, the database transaction is committed and the auxiliary transaction is rolled back. If the thread-local variable is set, the database local sub-transaction and auxiliary transaction are prepared, and the two-phase commit coordinator is notified that the sub-transaction is ready and the result is awaited. When the database is based on the S2PL protocol, the database starts a local sub-transaction and executes the operation, checking thread-local variables; If no thread-local variable is set, the database transaction is committed. If a thread-local variable is set and it is a read-only transaction, the atomic transaction updates a random row in the database's Dummy table with a random value, prepares the database transaction, and notifies the two-phase commit coordinator that the subtransaction is ready. When the database is based on other protocols, before the sub-transaction is executed, the global lock of the sub-transaction is acquired, the database starts the local sub-transaction and executes the operation, and the thread-local variables are checked; If the thread-local variable has been set, the database transaction is prepared and the two-phase commit coordinator is notified that the subtransaction is ready. If the thread-local variable has not been set, the database-local transaction is committed and the subtransaction global lock is released.

10. The globally serializable multi-database transaction control system according to claim 6, characterized in that, The padding layer submits global transactions to the two-phase commit coordinator, including: After receiving a global transaction commit request, the two-phase commit coordinator determines whether to commit the transaction based on the standard two-phase commit protocol. If a subtransaction fails to prepare, the two-phase commit coordinator fails to persist the commit decision, or a global transaction has not yet been committed when the system restarts, a subtransaction rollback request is sent to the application service that registered the subtransaction. Conversely, a subtransaction commit request is sent to the application service that registered the subtransaction; After the application service that registers the sub-transaction receives the request to commit / roll back the database sub-transaction, the padding layer intercepts the request to commit / roll back the database sub-transaction, commits / rolls back the database sub-transaction, and performs auxiliary operations in the commit phase by determining the database type and using the corresponding padding layer for the database. If the database type is based on the SSI protocol, roll back the auxiliary transaction; if it is based on the S2PL protocol, do nothing; if it is based on other protocols, release the global lock of the sub-transaction. After all application services complete the subtransaction commit / rollback operation request and return, the thread-local variables are eliminated.

Images