Distributed database-oriented compensating transaction implementation method

Abstract

The present application discloses a distributed database-oriented compensating transaction implementation method. The method specifically comprises: a client initiates a transaction request, and determines, by means of an execution directory, whether a transaction SQL exceeds a limit value (S1); a coordination node allocates a timestamp to a transaction initiated by the client in step S1, and simultaneously sends a related operation of the transaction to a data node (S2); upon receiving the transaction request, the data node determines whether to commit or roll back the transaction, and returns an execution result to the coordination node for confirmation and verification, and if the verification fails, the coordination node notifies the data node and performs a compensation operation (S3); and when the operation is completed, the coordination node commits the transaction, performs regular cleaning, and returns the execution result to the client (S4).

Description

A Compensation Transaction Implementation Method for Distributed Databases

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 202311703142.X, filed on December 12, 2023, entitled “A method for implementing compensation transactions for distributed databases,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application belongs to the field of distributed database technology, and specifically is a method for implementing compensation transactions for distributed databases. Background Art

[0004] Distributed systems have become a central component of modern software architectures, providing high availability, scalability, and fault resilience. These systems require business operations across multiple services, which leads to the need for distributed transactions. Distributed transactions refer to the participants in a transaction and the servers that support it. Currently, distributed transactions primarily include the following modes: AT mode, Saga mode, TCC mode, XA mode, and compensating transaction mode. The AT mode is a non-intrusive distributed transaction solution with zero learning cost, but SQL is managed and executed uniformly by the framework, which can lead to dirty write issues. The Saga mode is a long-term transaction solution that operates on the database in the first phase, which can lead to dirty read issues. The TCC mode is suitable for core systems and other performance-critical scenarios, where prolonged transaction execution can lock rows and prevent dirty reads. The XA mode offers a strong consistency solution, but its performance is low and its use is limited. The core concept of compensating transactions is to undo completed business operations by executing a series of reverse operations, known as "compensating operations." They ensure that the system can be restored to a consistent state in the event of errors or exceptions, offering advantages such as decoupling, fault tolerance, and flexibility.

[0005] For example, a Chinese patent application with publication number CN112182103A discloses a distributed database and a method for achieving strong consistency of cross-node transactions. The distributed database adopts a storage and computing separation architecture, including three types of nodes: coordination nodes, data nodes, and cataloging nodes. Each type of node can be horizontally expanded. The coordination node is responsible for distributing requests to the data nodes that need to participate, the data nodes are responsible for accessing and storing data, and the cataloging nodes store system metadata and partition-related information. A transaction is initiated by the coordination node and sent to one or more data node groups for operation. The distributed database introduces a transaction negotiation and transaction compensation mechanism between the data nodes that are participants. When a local system failure causes the participants to be unable to normally receive the transaction control message from the coordination node that is the coordinator, the mutual status confirmation between the participants is used to determine whether the transaction should be committed or rolled back to avoid data inconsistency problems.

[0006] For example, the Chinese patent with authorization announcement number CN110019469B discloses a distributed database data processing method, device, storage medium and electronic device. This application includes: after receiving an export request for exporting data from a distributed database, the database import and export management node obtains the current active transaction list and export table distribution information; the database import and export management node notifies the corresponding database to perform the data export operation based on the export table distribution information; after determining that the corresponding database has completed the data export operation, the database import and export management node sends the active transaction list to the database proxy node of the corresponding database; after receiving the data consistency reverse compensation statement returned by the database proxy node, the database import and export management node imports the data exported by the corresponding database into a predetermined database, and instructs the predetermined database to execute the data consistency reverse compensation statement.

[0007] The above-mentioned related technologies all have the following problems: 1) The implementation is complex, which has a significant impact on database performance and is not suitable for high-concurrency and high-performance scenarios; 2) The compensation task is lost after the node is restarted. Summary of the Invention

[0008] According to various embodiments of the present application, the present application proposes a method for implementing compensation transactions for a distributed database.

[0009] To achieve the above objectives, this application provides the following technical solutions:

[0010] A method for implementing compensation transactions for a distributed database, comprising:

[0011] Step S1: The client initiates a transaction request and determines whether the transaction SQL exceeds the limit value by executing the directory. If it exceeds the limit value, the backend database returns an error and the transaction information is stored in the transaction SQL mapping module.

[0012] Step S2: The coordinating node assigns a timestamp to the transaction initiated by the client in step S1 and sends the transaction operations to the data nodes.

[0013] Step S3: After receiving the transaction request, the data node determines whether it is a transaction commit or a transaction rollback, and returns the execution result to the coordination node for confirmation and verification. If the verification fails, the coordination node notifies the data node and performs compensation operations; and

[0014] Step S4: After the operation is completed, the coordinating node commits the transaction, performs scheduled cleanup, and returns the execution result to the client.

[0015] Specifically, the predetermined limits of the SQL statements of a transaction include: execution time, resource usage, data size, and number of concurrent accesses.

[0016] Specifically, the steps for the coordinating node to perform scheduled cleanup include:

[0017] The coordinating node receives the query request and broadcasts the request to each data node;

[0018] Each data node's shard processes the query request, performs data query, and places the qualified data in a priority queue. After each shard performs the data query, it returns the document ID, node information, and shard information to the coordination node;

[0019] The coordinating node aggregates all the results and performs global sorting. The coordinating node sends a get request to the shards containing these document IDs, and the corresponding shards return the document data to the coordinating node; and

[0020] The coordinating node returns the document data to the client.

[0021] Specifically, the specific steps of step S1 include:

[0022] Step S101: Determine the directory path for transaction execution and read the included SQL script file;

[0023] Step S102: Analyze the SQL script file content to determine whether there are SQL statements or operations that may cause the transaction to exceed the limit value and output the analysis results. Based on the analysis results, determine whether the SQL statements of the transaction exceed the predetermined limit value;

[0024] Step S103: Start the transaction and monitor its execution. If the transaction exceeds the limit during execution, the backend database responds and immediately returns an error; and

[0025] Step S104: Receive the error and process it. If the transaction is successfully executed, continue to process other transactions or close the connection.

[0026] Specifically, the specific steps of step S2 include:

[0027] Step S201: The client sends a transaction request to the coordinating node. After receiving the transaction request, the coordinating node parses and processes it.

[0028] Step S202: The coordinating node generates a globally unique timestamp according to the compensation rule, and assigns the generated timestamp to the transaction initiated by the client in step S1; and

[0029] Step S203: The coordinating node forwards the generated timestamp together with the transaction request to the relevant data nodes.

[0030] Specifically, step S201 includes:

[0031] The timestamp in step S202 specifically includes: transaction start time, transaction commit time, and transaction rollback time.

[0032] Specifically, step S201 includes:

[0033] The coordinating node parses the transaction request and extracts the operations to be performed and related parameters;

[0034] After parsing the transaction request, the coordinating node verifies the legitimacy of the operation;

[0035] If the operation is verified to be legal, the coordinating node will assign the operation to the appropriate transaction participating node; and

[0036] After assigning operations to participants, the coordinating node monitors the execution of the transaction.

[0037] Specifically, the timestamp in step S202 includes: transaction start time, transaction commit time, and transaction rollback time.

[0038] Specifically, the specific steps of step S3 include:

[0039] Step S301: The data node receives the transaction request forwarded by the coordinating node, sorts and executes it according to the timestamp, and determines whether to perform read, write or other operations;

[0040] Step S302: Based on the judgment result, the data node reads or modifies the local data and determines the status of the transaction;

[0041] Step S303: If the data node determines that the transaction can be committed, the operation result is persisted in the database and a confirmation message is returned to the coordination node. If the transaction needs to be rolled back, the previously executed related operations are undone and the database is restored to the state before the transaction started, and then a confirmation message is returned to the coordination node; and

[0042] Step S304: After receiving the confirmation message from the data node, the coordinating node verifies the operation result using the three-phase commit strategy. If the verification passes, it indicates that the operation of the data node is successfully completed. If the verification fails, a rollback is required.

[0043] Specifically, when sorting is performed based on timestamps, the data to be sorted is passed to the sorting function together with the corresponding timestamps, and the sorting function arranges the data in ascending or descending order based on the timestamps.

[0044] Specifically, the three-phase commit strategy in step S304 is as follows: a pre-commit phase is added between the preparation phase and the commit phase, so that the coordinating node sends a pre-commit request to all participating nodes, and determines the commit or rollback operation based on the responses of the participating nodes.

[0045] Specifically, the specific steps of the compensation operation in step S304 include:

[0046] Step S3041: Determine the transactions that need to be compensated and define compensation rules;

[0047] Step S3042: Mapping the operations in the distributed transaction to compensation operations through transaction compensation mapping according to the compensation rules;

[0048] Step S3043: Record the compensation execution status of each operation through compensation execution flag mapping; and

[0049] Step S3044: If the compensation operation is successfully executed, submit to the database and record the submission status of each operation through submission success mapping.

[0050] Specifically, the compensation rules in step S3041 include: reverse operation, data repair, transaction rollback, state recovery, sequential execution, repeatability and consistency guarantee.

[0051] Specifically, a unique execution flag is generated for each operation;

[0052] Establish a mapping relationship between the execution flag and the operation result;

[0053] During the transaction execution process, the execution flag and operation results are recorded;

[0054] Under the condition that a certain operation fails, the corresponding mapping relationship is found according to the execution flag, and the completed operation is undone or restored according to the mapping relationship; and

[0055] When all operations are completed successfully or compensating operations have been performed, the transaction ends and the client is notified.

[0056] Specifically, the specific steps of step S4 include:

[0057] Step S401: After receiving responses from all participating nodes, the coordinating node confirms that all operations have been completed;

[0058] Step S402: The coordinating node submits the transaction to the database or distributed transaction manager and records the submission status with a submission success mapping;

[0059] Step S403: After confirming that the transaction is successfully submitted, the coordinating node regularly cleans up historical data and releases resource operations; and

[0060] Step S404: Return the transaction processing result to the client.

[0061] A compensation transaction implementation system for distributed databases, comprising: a transaction SQL mapping module, a transaction submission success mapping module, a transaction rollback mapping module, a compensation Tmp mapping module, a compensation mapping module, and a compensation execution flag mapping module;

[0062] The transaction SQL mapping module is used to determine whether the transaction SQL exceeds the upper limit or submission limit value;

[0063] The transaction submission success mapping module is used to record the transaction mapping of the distributed transaction submission success and clean up the data when there is a scheduled task;

[0064] The transaction rollback mapping module is used to record rolled-back distributed transactions and clean up the data therein when there are scheduled tasks;

[0065] The compensation Tmp mapping module is used to record temporary node transactions to be compensated;

[0066] The compensation mapping module is used to record node transactions to be compensated; and

[0067] The compensation execution flag mapping module is used to mark that a node is performing a compensation transaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without creative work.

[0069] FIG1 is a flow chart of a method for implementing compensation transactions for a distributed database according to the present application;

[0070] FIG2 is a system analysis flow chart of a method for implementing compensation transactions for distributed databases in the present application;

[0071] FIG3 is a flowchart of a three-phase commit coordinator state transition of a compensation transaction implementation method for a distributed database in the present application;

[0072] FIG4 is a flowchart of the state transition of participants in a three-phase commit method for implementing a compensation transaction for a distributed database according to the present application;

[0073] FIG5 is a diagram showing the architecture of a system for implementing compensation transactions for distributed databases in the present application. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0075] In order to make the technical means, creative features, objectives and effects achieved by this application easy to understand, it should be noted that in the description of this application, the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to has a specific direction, is constructed and operated in a specific direction, and therefore cannot be understood as a limitation on this application. In addition, the terms "No. 1", "No. 2" and "No. 3" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. The following further describes this application in conjunction with specific implementation methods.

[0076] In one embodiment:

[0077] Referring to FIG. 1 , FIG. 2 , FIG. 3 , and FIG. 4 , an embodiment of the present application provides a method for implementing a compensation transaction for a distributed database, comprising the following steps:

[0078] Step S1: The client initiates a transaction request and determines whether the transaction SQL exceeds the limit value by executing the directory. If it exceeds the limit value, the backend database returns an error and the transaction information is stored in the transaction SQL mapping module.

[0079] The four major characteristics of transactions include: atomicity, consistency, isolation, and durability.

[0080] The predetermined limits of the transaction's SQL statements include: (1) Execution time: Set the maximum execution time of the transaction. If this time is exceeded, the transaction is considered to have failed to complete within the time limit and may return an error or roll back the transaction; (2) Resource usage: You can set the maximum limit on the resources used by the transaction during execution; (3) Data size; (4) Number of concurrent accesses.

[0081] The advantages of setting limits on transaction SQL are: (1) It ensures that transactions will not be interfered with by other transactions during execution, thus maintaining data consistency; (2) When an error occurs during transaction execution, the transaction can be rolled back to undo the executed operations and avoid data inconsistency; (3) By setting the transaction lock level and concurrency control method, the transaction isolation level and lock contention during concurrent access can be controlled, thereby improving system performance and response time; (4) Data consistency can be restored by rolling back and committing transactions; (5) The logical flow of the program can be controlled by setting transactions.

[0082] Step S2: The coordinating node assigns a timestamp to the transaction initiated by the client in step S1 and sends the transaction operations to the data nodes.

[0083] In a distributed database, a coordinator node is a special node used to coordinate the search and aggregation operations of shards. When a client sends a search request, the coordinator node forwards the request to the data node storing the relevant shards and returns the results to the client. The coordinator node does not store data and does not participate in data searches.

[0084] Step S3: After receiving the transaction request, the data node determines whether it is a transaction commit or a transaction rollback, and returns the execution result to the coordination node for confirmation and verification. If the verification fails, the coordination node notifies the data node and performs compensation operations; and

[0085] Transaction rollback means undoing the update operations on the database that have been completed by the transaction. In the transaction, each correct atomic operation will be executed sequentially until an incorrect atomic operation is encountered.

[0086] Step S4: After the operation is completed, the coordinating node commits the transaction, performs scheduled cleanup, and returns the execution result to the client.

[0087] Compared with the related art, the beneficial effects of this application are:

[0088] 1. This application proposes a compensation transaction implementation system for distributed databases, and optimizes and improves the architecture, operating steps and processes. The system has the advantages of simple processes, low investment and operating costs, and low production work costs. While ensuring complete functions, unchanged request methods, and subtle performance losses, it implements a compensation transaction solution that does not affect data.

[0089] 2. This application proposes a compensation transaction implementation method for distributed databases. In terms of data, the transaction status and SQL status of each node are recorded through local files, which can not only ensure the efficiency of SQL compensation, but also ensure that the compensation transaction is not lost when the node is restarted; in terms of function, after entering the compensation transaction, only the directory level is locked to minimize the impact on the business. The timed thread will automatically perform the compensation task, and the lock will be released when the compensation is successful. The business is unaware of this process.

[0090] 3. This application proposes a method for implementing compensation transactions for distributed databases, which uses a combination of memory and local files to record transactions that need to be compensated. This can not only ensure efficiency but also ensure that the compensation tasks are not lost after restart. On the basis of ensuring complete functions, unchanged request methods, and subtle performance losses, a compensation transaction solution that does not affect data is implemented.

[0091] The steps for coordinating nodes to perform scheduled cleanup are as follows:

[0092] (1) The coordinating node first receives the query request and broadcasts the request to each data node. The shard of each data node processes the query request, performs data query, and places the qualified data in a priority queue;

[0093] (2) After each shard performs a data query, it returns the document ID, node information, and shard information to the coordination node. The coordination node aggregates all the results and performs global sorting.

[0094] (3) The coordinating node sends a get request to the shards containing these document IDs, and the corresponding shards return the document data to the coordinating node; and

[0095] (4) Finally, the coordinating node returns the data to the client.

[0096] The specific steps of step S1 include:

[0097] Step S101: Determine the directory path for transaction execution and read the included SQL script file;

[0098] Step S102: Analyze the SQL script file content to determine whether there are SQL statements or operations that may cause the transaction to exceed the limit value and output the analysis results. Based on the analysis results, determine whether the SQL statements of the transaction exceed the predetermined limit value;

[0099] Step S103: Start the transaction and monitor its execution. If the transaction exceeds the limit during execution, the backend database responds and immediately returns an error; and

[0100] Step S104: Receive the error and process it. If the transaction is successfully executed, continue to process other transactions or close the connection.

[0101] The specific steps of step S2 include:

[0102] Step S201: The client sends a transaction request to the coordinating node. After receiving the transaction request, the coordinating node parses and processes it.

[0103] The specific steps include: (1) the coordinating node first needs to parse the transaction request and extract the operations to be performed and related parameters; (2) after parsing the transaction request, the coordinating node needs to verify the legitimacy of the operation; (3) once the operation is verified to be legal, the coordinating node needs to assign the operation to the appropriate transaction participating node; and (4) after assigning the operation to the participants, the coordinating node needs to monitor the execution process of the transaction.

[0104] Step S202: The coordinating node generates a globally unique timestamp according to the compensation rule, and assigns the generated timestamp to the transaction initiated by the client in step S1; and

[0105] Step S203: The coordinating node forwards the generated timestamp together with the transaction request to the relevant data nodes.

[0106] The specific content of the timestamp in step S202 includes: transaction start time, transaction commit time and transaction rollback time.

[0107] In a distributed database, the generation and distribution of timestamps must ensure global uniqueness and consistency to avoid conflicts and data inconsistencies. Common timestamp generation methods include using physical clocks, distributed clock servers, or generating pseudo-clocks through algorithms.

[0108] The specific steps of step S3 include:

[0109] Step S301: The data node receives the transaction request forwarded by the coordinating node, sorts and executes it according to the timestamp, and determines whether to perform read, write or other operations;

[0110] When performing timestamp sorting, you need to pass the data to be sorted along with the corresponding timestamp to the sorting function. The sorting function will sort the data in ascending or descending order based on the timestamp.

[0111] Step S302: Based on the judgment result, the data node reads or modifies the local data and determines the status of the transaction;

[0112] Step S303: If the data node determines that the transaction can be committed, the operation result is persisted in the database and a confirmation message is returned to the coordination node. If the transaction needs to be rolled back, the previously executed related operations are undone and the database is restored to the state before the transaction started, and then a confirmation message is returned to the coordination node; and

[0113] Transaction commit and rollback transactions are stored in the corresponding transaction commit success mapping and transaction rollback mapping modules. Transaction commit success mapping refers to mapping the result of a successful transaction commit to an object or system. The specific steps include: determining the mapping object, defining mapping rules, executing the mapping operation, verifying the mapping result, and recording the mapping log. Transaction rollback mapping is a mechanism in a database management system that restores the database state to the state before the transaction began when an error or exception occurs during transaction execution. The specific steps include: error detection, preparing for rollback, generating a rollback log, executing the rollback, and committing the transaction.

[0114] Step S304: After receiving the confirmation message from the data node, the coordinating node verifies the operation result using the three-phase commit strategy. If the verification passes, it indicates that the operation of the data node is successfully completed. If the verification fails, a rollback is required.

[0115] The three-phase commit strategy in step S304 is specifically as follows: a pre-commit phase is added between the preparation phase and the commit phase, so that the coordinating node sends a pre-commit request to all participating nodes, and determines the commit or rollback operation based on the responses of the participating nodes.

[0116] In distributed databases, common strategies include two-phase commit and three-phase commit. This application chooses the three-phase commit strategy, which can reduce the blocking problem caused by waiting for all participating nodes to respond. The three-phase commit includes:

[0117] (1) Preparation phase: The coordinating node sends a readiness request to all participating nodes and waits for responses from the participating nodes;

[0118] (2) Pre-commit phase: While waiting for responses from all participating nodes, the coordinating node decides whether to perform the pre-commit operation based on the responses of the participating nodes. If all participating nodes return an "agree" response, the coordinating node enters the commit phase; otherwise, the coordinating node enters the rollback phase.

[0119] (3) Commit phase: After the pre-commit phase, if all participating nodes return an "agree" response, the coordinating node sends a commit request to all participating nodes; after receiving the commit request, the participating nodes perform relevant operations and persist the results to the database; the coordinating node waits for all participating nodes to complete the operation and return a confirmation message; when all participating nodes have completed the operation and have been confirmed and verified by the coordinating node, the coordinating node commits the transaction and returns a successful result to the client.

[0120] The specific steps of the compensation operation in step S304 include:

[0121] Step S3041: Determine the transactions that need to be compensated and define compensation rules;

[0122] Transaction compensation rules include: 1) Determining the compensation point: During the transaction execution process, determine the steps and states that need to be compensated, thereby determining the scope that needs to be rolled back; 2) Determining the rollback scope; 3) Generating the corresponding rollback command based on the rollback scope and compensation point; 4) Executing the corresponding rollback operation based on the generated rollback command; 5) Recording the information of the compensation operation in the compensation log; 6) After executing the compensation operation, checking whether the compensation result meets expectations; 7) Notifying the relevant users or systems of the results of the compensation operation.

[0123] Step S3042: Mapping the operations in the distributed transaction to compensation operations through transaction compensation mapping according to the compensation rules;

[0124] Compensation mapping means that when a transaction fails or an exception occurs, the data is restored to the state before the transaction started by performing a compensation operation to ensure data consistency and integrity.

[0125] Step S3043: Record the compensation execution status of each operation through compensation execution flag mapping; and

[0126] The advantage of the compensation execution flag mapping is that it can accurately record the results and status of each operation, so that when the transaction fails, the undo or restore operation can be accurately performed to ensure data consistency. At the same time, by establishing a mapping relationship between the operation result and the execution flag, this application uses the compensation execution flag mapping to record the compensation execution status.

[0127] The implementation of compensation execution flag mapping includes the following steps:

[0128] (1) Generate a unique execution flag for each operation;

[0129] (2) Establishing a mapping relationship between the execution flag and the operation result;

[0130] (3) During the transaction execution process, record the execution flag and operation results;

[0131] (4) If an operation fails, find the corresponding mapping relationship according to the execution flag, and then cancel or restore the completed operation according to the mapping relationship; and

[0132] (5) All operations are completed successfully, or compensation operations have been performed, ending the transaction and notifying the client.

[0133] Step S3044: If the compensation operation is successfully executed, submit to the database and record the submission status of each operation through submission success mapping.

[0134] The compensation rules in step S3041 include: reverse operation, data repair, transaction rollback, state recovery, sequential execution, repeatability and consistency guarantee.

[0135] The submission methods of compensation operations include: direct submission, asynchronous submission, and scheduled submission.

[0136] The specific steps of step S4 include:

[0137] Step S401: After receiving responses from all participating nodes, the coordinating node confirms that all operations have been completed;

[0138] Step S402: The coordinating node submits the transaction to the database or distributed transaction manager and records the submission status with a submission success mapping;

[0139] Step S403: After confirming that the transaction is successfully submitted, the coordinating node regularly cleans up historical data and releases resource operations; and

[0140] Step S404: Return the transaction processing result to the client.

[0141] In another embodiment:

[0142] Referring to FIG5 , another embodiment provided by the present application is a compensation transaction implementation system for a distributed database, comprising:

[0143] Transaction SQL mapping module, transaction submission success mapping module, transaction rollback mapping module, compensation Tmp mapping module, compensation mapping module and compensation execution flag mapping module;

[0144] The transaction SQL mapping module is used to determine whether the transaction SQL exceeds the upper limit or submission limit value;

[0145] The transaction submission success mapping module is used to record the transaction mapping of the distributed transaction submission success and clean up the data when there is a scheduled task;

[0146] The transaction rollback mapping module is used to record rolled-back distributed transactions and clean up the data therein when there are scheduled tasks;

[0147] The compensation Tmp mapping module is used to record temporary node transactions to be compensated;

[0148] The compensation mapping module is used to record node transactions to be compensated; and

[0149] The compensation execution flag mapping module is used to mark that a node is performing a compensation transaction.

[0150] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are illustrative rather than restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are protected by this application.

[0151] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for implementing compensation transactions for distributed databases, characterized in that: include: Step S1: The client initiates a transaction request and determines whether the transaction SQL exceeds the limit value by executing the directory. If it exceeds the limit value, the backend database returns an error and the transaction information is stored in the transaction SQL mapping module. Step S2: The coordinating node assigns a timestamp to the transaction initiated by the client in step S1, and sends the operation of the transaction to the data node; Step S3: After receiving the transaction request, the data node determines whether it is a transaction commit or a transaction rollback, and returns the execution result to the coordination node for confirmation and verification. If the verification fails, the coordination node notifies the data node and performs compensation operations; and Step S4: After the operation is completed, the coordinating node commits the transaction, performs scheduled cleanup, and returns the execution result to the client.

2. A method for implementing compensation transactions for distributed databases according to claim 1, characterized in that: The predetermined limits of the transaction's SQL statements include: execution time, resource usage, data size, and number of concurrent accesses.

3. A method for implementing compensation transactions for distributed databases according to claim 1, characterized in that: The steps for coordinating nodes to perform scheduled cleanup include: The coordinating node receives the query request and broadcasts the request to each data node; Each data node's shard processes the query request, performs data query, and places the qualified data in a priority queue. After each shard performs data query, it returns the document ID, node information, and shard information to the coordination node; and The coordinating node aggregates all the results and performs global sorting. The coordinating node sends a get request to the shards containing these document IDs, and the corresponding shards return the document data to the coordinating node. The coordinating node returns the document data to the client.

4. A method for implementing compensation transactions for distributed databases according to claim 1, characterized in that: The specific steps of step S1 include: Step S101: determine the directory path for transaction execution and read the SQL script file contained therein; Step S102: Analyze the SQL script file content to determine whether there is an SQL statement or operation that may cause the transaction to exceed the limit value and output the analysis result, and determine whether the SQL statement of the transaction exceeds the predetermined limit value according to the analysis result; Step S103: Start a transaction and monitor its execution. If the transaction exceeds a limit value during execution, the backend database responds and immediately returns an error; and Step S104: Receive the error and process it. If the transaction is successfully executed, continue to process other transactions or close the connection.

5. A method for implementing compensation transactions for distributed databases according to claim 4, characterized in that: The specific steps of step S2 include: Step S201: the client sends a transaction request to the coordinating node, and the coordinating node parses and processes the transaction request after receiving it; Step S202: the coordinating node generates a globally unique timestamp according to the compensation rule, and assigns the generated timestamp to the transaction initiated by the client in step S1; and Step S203: The coordinating node forwards the generated timestamp together with the transaction request to the relevant data nodes.

6. A method for implementing compensation transactions for distributed databases according to claim 5, characterized in that: The step S201 includes: The coordinating node parses the transaction request and extracts the operations to be performed and related parameters; After parsing the transaction request, the coordinating node verifies the legitimacy of the operation; If the operation is verified to be legal, the coordinating node distributes the operation to the appropriate transaction participating node; and After assigning operations to participants, the coordinator node monitors the execution of the transaction.

7. A method for implementing compensation transactions for distributed databases according to claim 5, characterized in that: The timestamp in step S202 specifically includes: transaction start time, transaction commit time and transaction rollback time.

8. A method for implementing compensation transactions for distributed databases according to claim 7, characterized in that: The specific steps of step S3 include: Step S301: The data node receives the transaction request forwarded by the coordinating node, sorts and executes it according to the timestamp, and determines whether to perform reading, writing or other operations; Step S302: According to the judgment result, the data node reads or modifies the local data and judges the status of the transaction; Step S303: If the data node determines that the transaction can be committed, the operation result is persisted in the database and a confirmation message is returned to the coordination node. If the transaction needs to be rolled back, the previously executed related operations are undone and the database is restored to the state before the transaction started, and then a confirmation message is returned to the coordination node; and Step S304: After receiving the confirmation message from the data node, the coordinating node verifies the operation result using the three-phase commit strategy. If the verification passes, it indicates that the operation of the data node is successfully completed. If the verification fails, a rollback is required.

9. A method for implementing compensation transactions for distributed databases according to claim 8, characterized in that: The method further comprises: When sorting by timestamp, the data to be sorted is passed to the sorting function together with the corresponding timestamp, and the sorting function arranges the data in ascending or descending order according to the timestamp.

10. A method for implementing compensation transactions for distributed databases according to claim 8, characterized in that: The three-phase commit strategy in step S304 is specifically as follows: a pre-commit phase is added between the preparation phase and the commit phase, so that the coordinating node sends a pre-commit request to all participating nodes, and determines the commit or rollback operation according to the responses of the participating nodes.

11. A method for implementing compensation transactions for distributed databases according to claim 10, characterized in that: The specific steps of the compensation operation in step S304 include: Step S3041: determine the transaction that needs to be compensated and define the compensation rules; Step S3042: According to the compensation rule, the operations in the distributed transaction are mapped to compensation operations through transaction compensation mapping; Step S3043: Record the compensation execution status of each operation through compensation execution flag mapping; and Step S3044: Under the condition that the compensation operation is successfully executed, the database is submitted, and the submission status of each operation is recorded through submission success mapping.

12. A method for implementing compensation transactions for distributed databases according to claim 11, characterized in that: The compensation rules in step S3041 include: reverse operation, data repair, transaction rollback, state recovery, sequential execution, repeatability and consistency guarantee.

13. A method for implementing compensation transactions for distributed databases according to claim 11, characterized in that: The compensation execution flag mapping includes: Generate a unique execution token for each operation; Establish a mapping relationship between the execution flag and the operation result; During the transaction execution process, the execution flag and operation results are recorded; Under the condition that a certain operation fails, the corresponding mapping relationship is found according to the execution flag, and the completed operation is undone or restored according to the mapping relationship; and When all operations are completed successfully or compensating operations have been performed, the transaction ends and the client is notified.

14. A method for implementing compensation transactions for distributed databases according to claim 12, characterized in that: The specific steps of step S4 include: Step S401: After receiving responses from all participating nodes, the coordinating node confirms that all operations have been completed; Step S402: The coordinating node submits the transaction to the database or the distributed transaction manager, and records the submission status with a submission success mapping; Step S403: After confirming that the transaction is successfully submitted, the coordination node regularly cleans up historical data and releases resource operations; and Step S404: Return the transaction processing result to the client.

15. A method for implementing compensation transactions for distributed databases according to claim 14, characterized in that: include: Transaction SQL mapping module, transaction submission success mapping module, transaction rollback mapping module, compensation Tmp mapping module, compensation mapping module and compensation execution flag mapping module; The transaction SQL mapping module is used to determine whether the transaction SQL exceeds the upper limit or submission limit value; The transaction submission success mapping module is used to record the transaction mapping of the distributed transaction submission success and clean up the data when there is a scheduled task; The transaction rollback mapping module is used to record the rolled-back distributed transactions and clean up the data therein when there are scheduled tasks; The compensation Tmp mapping module is used to record temporary node transactions to be compensated; The compensation mapping module is used to record node transactions to be compensated; and The compensation execution mark mapping module is used to mark that a certain node is compensating for a transaction.

Images