Distributed database consistency snapshot optimization generation method and system

Abstract

The invention discloses a distributed database consistency snapshot optimization generation method and system, and aims at a distributed database comprising a plurality of database nodes. The method comprises the following steps: regularly detecting the local consistency snapshot freshness of a database node, notifying the database node serving as a transaction coordinator to initiate transaction timestamp updating operation on all transaction participating nodes when a transaction blocking snapshot propulsion exists on the node, and generating a new consistency snapshot and a corresponding safety timestamp after updating is completed. According to the method, stable updating of the consistent snapshots can be realized, so that the situation that the upper limit of blocking time cannot be ensured when the snapshots are blocked in reading is avoided.

Description

Technical Field

[0001] The present invention relates to a technology in the field of distributed data processing, in particular to a method and system for optimizing and generating consistent snapshots of a distributed database. Background Art

[0002] Distributed databases usually replicate a piece of data to generate multiple copies, thereby improving the availability of the system and supporting the copies to provide reads to increase the database read throughput capacity. Due to the delay in the replication of write operations between the master and the slave, outdated data may be obtained when reading the replica, resulting in data consistency issues. Therefore, replica reading is usually implemented through snapshot reading, that is, when reading, a timestamp is set. When the latest consistent snapshot generated on the corresponding replica meets the read timestamp, the snapshot data corresponding to the timestamp can be returned. In the existing consistent snapshot generation technology, when the distributed transaction coordinator node loses the request to end the transaction on the participant node, and when the transaction is deadlocked and long transactions are executed, some transactions will be in the stage of preparing to complete but not committing for a long time. The security timestamp on the replica involved in the transaction cannot be updated, thereby blocking the consistent snapshot, resulting in the snapshot read with a read timestamp later than the security timestamp being blocked until the corresponding transaction ends. Summary of the invention

[0003] In view of the fact that the prior art cannot solve the problem of distributed transaction deadlock and snapshot reading blocking in long transaction execution scenarios, the present invention proposes a distributed database consistent snapshot optimization generation method and system, which can achieve stable update of consistent snapshots, thereby avoiding the inability to guarantee the upper limit of blocking time when snapshot reading is blocked.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a distributed database consistency snapshot optimization generation method. For a distributed database including multiple database nodes, the freshness of the local consistency snapshot of the database node is regularly detected, and when there is a transaction blocking the snapshot advancement on the node, the database node as the transaction coordinator is notified to initiate a transaction timestamp update operation for all transaction nodes involved, and a new consistency snapshot and a corresponding security timestamp are generated after the update is completed.

[0006] The distributed database includes multiple database nodes, each of which stores part of the data locally. When a distributed transaction involving reading and writing data on multiple database nodes is executed, one of the database nodes where the data is located is selected as the coordinator node of the transaction, and the other nodes are selected as the participant nodes of the transaction.

[0007] The method specifically comprises:

[0008] Step 1: Take any database node in the distributed database as the current processing node, obtain the current timestamp TS and calculate the local security timestamp TS safe , and then calculate the local consistency snapshot freshness T fresh , specifically: T fresh =TS-TS safe ,in: The value is the minimum value of the prepare timestamps of all prepared but uncommitted transactions on the current processing node.

[0009] When there are no prepared but uncommitted transactions is positive infinity.

[0010] Step 2: When the freshness of the consistency snapshot of the current processing node in step 1 does not meet the threshold, execute step 3; otherwise, execute again from step 1 after a delay.

[0011] Step 3: In step 1, the current processing node obtains the abnormal transaction that blocks the freshness of the consistent snapshot from the local transaction manager, that is, the set T of all prepared but uncommitted distributed transactions whose preparation timestamps do not meet the threshold. 1 ,…,txn n}, where: txn is a distributed transaction that has been prepared but not committed on the node, and the local transaction manager contains the preparation timestamp and coordinator node information of the transaction on the current processing node.

[0012] Step 4: For each prepared but uncommitted transaction txn in the transaction set T i , the current processing node notifies the corresponding coordinator node Transaction txn i The prepare timestamp is too old.

[0013] Step 5: For the prepared but uncommitted transaction txn in set T i The coordinator node After receiving the notification, calculate the lower limit of the submission timestamp And send a prepare refresh request to all participating nodes of the transaction.

[0014] The prepare refresh request is a request sent to a transaction participant node to refresh the transaction prepare timestamp when the final state of the transaction has not been determined on the coordinator node of the transaction. When the coordinator node sends the prepare refresh request, it adds the transaction information to be refreshed and the lower limit of the transaction commit timestamp t min-commit When a participant node receives a prepare refresh request, it refreshes the prepare timestamp of the prepared but uncommitted transaction to t min-commit, and then update the secure timestamp T on the participant node safe , promoting the generation of consistent snapshots.

[0015] Step 6. All received txn i The database node that is ready to refresh the request will send txn i The prepare timestamp is updated to After the current processing node receives the prepare refresh request from all coordinator nodes in step 4, it recalculates the security timestamp and provides a consistent snapshot below the security timestamp, and then starts executing from step 1 again after a delay. Technical Effects

[0016] The present invention adds a new preparation refresh step based on the security timestamp scheme, and timely refreshes the transaction preparation timestamp on the participant node by periodically detecting abnormal transactions that block the generation of consistency snapshots and notifying the corresponding coordinator node to send a preparation refresh request, thereby achieving stable update of the security timestamp. Compared with the problem that the freshness of the consistency snapshot in the existing consistency snapshot generation scheme is affected by long transactions and deadlock transactions, the present invention can ensure that the freshness of the consistency snapshot is stable within the configuration time without affecting the normal operation of the transaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the present invention;

[0018] Figure 2 It is a schematic diagram of the participant node flow in the present invention;

[0019] Figure 3 This is a schematic diagram of the coordinator node process in the present invention;

[0020] Figure 4 A schematic diagram of a process executed in a transaction deadlock scenario in an embodiment;

[0021] Figure 5 It is an effect diagram of the embodiment;

[0022] Figure 6 It is a schematic diagram of the system of the present invention;

[0023] Figure 7 A flow chart is provided for an embodiment application. DETAILED DESCRIPTION

[0024] like Figure 1 As shown, a distributed database consistency snapshot optimization generation method involved in this embodiment includes:

[0025] Step 1: For a database node, obtain the current timestamp TS and calculate the local security timestamp TS safe, and then calculate the freshness of the local consistency snapshot, where: security timestamp The value is the minimum value of the preparation timestamp of all prepared but uncommitted transactions on the database node. If there is no prepared but uncommitted transaction, Is positive infinity. Consistency snapshot freshness T fresh =TS-TS safe .

[0026] Step 2: When the freshness of the consistent snapshot of the database node does not meet the threshold, execute step 3; otherwise, execute again from step 1 after a delay.

[0027] Step 3: The database node obtains the abnormal transactions that block the freshness of the consistent snapshot from the local transaction manager, that is, the set of all prepared but uncommitted transactions whose preparation timestamps do not meet the threshold value T = {txn 1 ,…,txn n}, where: txn is a distributed transaction executed on the node, and the local transaction manager contains the preparation timestamp and coordinator node information of the transaction on the node.

[0028] Step 4: For each prepared but uncommitted transaction txn in the transaction set T i , the database node notifies the corresponding coordinator node Transaction txn i The prepare timestamp is too old.

[0029] Step 5: For the prepared but uncommitted transaction txn in set T i The coordinator node After receiving the notification, calculate the lower limit of the submission timestamp And send a prepare refresh request to all participating nodes in the transaction.

[0030] Step 6: The database node receives the data for txn i Prepare refresh request, and change txn i The prepare timestamp is updated to After receiving the prepare refresh request from all coordinator nodes in step 4, the node recalculates the security timestamp and provides a consistent snapshot below the security timestamp, and then starts again from step 1 after a delay.

[0031] like Figure 2As shown, after receiving the prepare request, the participant node locally obtains the resources required for transaction submission, obtains the timestamp as the timestamp of preparation completion, and returns the prepare timestamp and the result of successful preparation to the coordinator node. When the participant node encounters the pending transaction and causes the freshness of the consistency snapshot to exceed the threshold, the coordinator node of the transaction is actively notified. When a prepare refresh request is received from the coordinator node, the prepare timestamp of the corresponding transaction is updated according to the timestamp carried in the request, and the participant node is notified to update the consistency snapshot. When a commit or rollback request is received from the coordinator node, the resources occupied by the corresponding transaction are released and the local records of the transaction are cleaned up. Subsequent prepare refresh requests for the transaction will no longer be processed and will be directly rejected.

[0032] The prepare timestamp ensures that the commit timestamp is greater than or equal to the timestamp when the transaction is finally committed.

[0033] like Figure 3 As shown, after the coordinator node receives the transaction submission request from the client node, it sends a preparation request to the corresponding participant node. Before receiving the preparation results from all participant nodes and after the pending time of the transaction exceeds the threshold, the coordinator node actively sends a preparation refresh request to the corresponding participant node. After receiving the notification from the participant node, a preparation refresh request is sent to the participant node. After receiving the results of the preparation requests from all participant nodes, it is decided whether the transaction is submitted. When submission is required, the latest timestamp is obtained until the timestamp is greater than the lower limit of the submission timestamp on the coordinator node. After the final result of the transaction is determined, the transactions on all participant nodes are cleaned up. After the submission timestamp is determined on the coordinator node, when receiving the notification from the participant node, the preparation refresh request is no longer sent, but a transaction submission or rollback request is sent. After receiving the reply from all participant nodes, the coordinator node notifies the client node of the final result of the transaction.

[0034] The prepare refresh request contains the lower limit of the commit timestamp that can be determined for the transaction, such as the maximum timestamp / current timestamp / maximum prepare timestamp received on the current node, which is used to update the prepare timestamp of the transaction on the participant node. The coordinator node can configure the transaction pending timeout threshold according to actual application requirements, and will periodically send prepare refresh requests until the transaction determines the final result.

[0035] The triggering conditions of the prepare refresh request include:

[0036] 1) When the pending time of a transaction on the coordinator node exceeds the configured threshold, the lower limit of the commit timestamp is periodically calculated and a prepare refresh request is sent to all participant nodes until the transaction is committed or rolled back on the coordinator node;

[0037] 2) When the freshness of the consistent snapshot on the participant node does not meet the configuration, the participant node checks whether all the reasons for the blockage are caused by Too low. Too low, get the maximum value of all transaction read timestamps that exceed the configured time Check all prepared but uncommitted transactions on the node. For a transaction, request the coordinator node of the transaction to send a prepare refresh request to update the prepare timestamp of the transaction on the participant node.

[0038] In the two trigger conditions of the prepare refresh request, the trigger thresholds can be configured independently to adapt to different business needs. For example, for a data range that requires a lower read blocking time, a higher consistency snapshot freshness can be configured on the corresponding database node separately, thereby ensuring a lower snapshot read blocking time.

[0039] The method for calculating the lower limit of the submission timestamp is to first obtain the maximum timestamp that has been received locally, and when it can meet the freshness requirements of the participant node, the timestamp is used as the lower limit of the submission timestamp. When the maximum timestamp received locally does not meet the freshness requirements, the latest timestamp is obtained as the lower limit of the submission timestamp.

[0040] When the coordinator node sends a prepare refresh request to a participant node, it can adopt an aggregation method to aggregate all prepare refresh requests with the same destination address within a period of time into a batch and send them to the corresponding node for unified processing, thereby reducing communication overhead. When the transaction preparation timestamp of the participant node is too old and sends a notification to the coordinator node, the node can aggregate all requests with the same destination address within a period of time into a batch and send them to the corresponding node for unified processing, thereby reducing communication overhead.

[0041] In the distributed database cross-region multi-copy deployment scenario, after the freshness of the consistency snapshot on the participant node exceeds the configured threshold, the participant node can request the replica of the coordinator node of the corresponding transaction in the local area to obtain the known lower limit of the commit timestamp of the corresponding transaction, thereby avoiding the high latency overhead of cross-region communication.

[0042] like Figure 6 As shown, the distributed database system for implementing the above method in this embodiment includes: a cluster management unit, a timestamp service unit and several data service units, wherein: the cluster management unit stores and manages metadata and node information and routes requests, the timestamp service unit calculates and allocates monotonically increasing timestamps, and the data service unit parses SQL requests and implements data storage, data addition, deletion, modification and query, and transaction processing.

[0043] The data service unit comprises: an SQL parsing subunit, a prepare-to-refresh subunit, a data storage subunit and a transaction management subunit, wherein: the SQL parsing subunit analyzes and converts SQL statements into internal representations, ensures grammatical correctness, and generates an execution plan to optimize database operations; the prepare-to-refresh subunit handles the problem of too old consistency snapshots from the transaction management subunit, queries transactions generated by local blocking consistency snapshots and notifies the corresponding data service unit; handles pending timeouts of local transactions and blocking notifications of consistency snapshots from other data service units, calculates the lower limit of transaction commit timestamps according to local transaction status, and sends a prepare-to-refresh request to the relevant data service unit; the data storage subunit receives and processes data changes and query requests from the transaction management subunit, and manages all submitted data version information on the data service unit; the transaction management subunit executes transactions and manages temporary version data and transaction status information of transactions that have not been submitted on the data service unit, maintains security timestamps on the data service unit, and periodically calculates and updates security timestamps, checks the freshness of consistency snapshots and the pending duration of local transaction coordination records.

[0044] The data service unit stores data from one or more data tables, each data table can be divided into one or more logical data partitions, each data partition includes one or more physical data copies, one of which serves as the master copy, receives writes to the data partition and has all the latest data of the data partition, and the other copies serve as slave copies, which do not provide writes but can provide reads. Data copies can be distributed on different data service units as needed.

[0045] When a transaction involves different data partitions, the transaction is considered a distributed transaction. The system runs one or more distributed transactions simultaneously. Each distributed transaction maintains the latest status of the transaction in the data partition on the master copy of the corresponding data partition. The master copy copies the data to the slave copy, and the slave copy maintains the status of the transaction in the data partition at a certain time in the past. The distributed transaction interacts with the data service units where the master copies of the corresponding data partitions are located to implement the standard two-phase commit of the distributed transaction. At the same time, the master and slave copies of the corresponding data partitions can use the prepare refresh subunit to promote the generation of consistent snapshots.

[0046] After specific experiments, this embodiment Figure 4 The simulation is performed in the scenario shown, involving two distributed transactions txn1 and txn2. Both transactions operate on key1 on participant node 1 and key2 on participant node 2. The simulation process is as follows Figure 7 As shown, specifically including:

[0047] Step 1: Client node 1 initiates transaction txn1 execution and submission request at timestamp 10s, and selects coordinator node, participant node 1, and participant node 2 as the execution nodes of the transaction. The coordinator receives the corresponding transaction request at timestamp 10s, and the transaction preparation refresh timer threshold is configured on the coordinator as 10s. The database configuration lock wait timeout is 30s.

[0048] Step 2: The coordinator node sends the corresponding transaction operation and preparation request to participant node 1 and participant node 2. After receiving the request, participant node 2 obtains the write lock wlock2 and obtains the current timestamp 10s as the preparation timestamp.

[0049] Step 3: Before participant node 1 receives the prepare request of transaction txn1, transaction txn2, which is not fully identified in the figure due to space limitations, completes the prepare operation on participant node 1 and obtains the write lock wlock1 of key1 on participant node 1. The transaction also involves participant node 2, which is waiting for the release of a write lock wlock2 locked by transaction txn1.

[0050] Step 4: Participant Node 1 receives the prepare request of transaction txn1, and transaction txn1 starts waiting for the release of write lock wlock1 from timestamp 10s on participant node 1. At this time, transactions txn1 and txn2 wait in a loop.

[0051] Step 5: Client node 2 sends a snapshot read request txn3 to participant node 2, and obtains the current timestamp 13s as the read timestamp. At this time, T safe Blocked by the prepare timestamp of txn1, which remains at 10s.

[0052] Step 6, on the coordinator node of transaction txn1, when the timestamp is 15s, under the solution provided by the present invention, the timer of transaction txn1 times out, and actively sends a prepare refresh request for txn1 to participant node 2, where the lower limit of the commit timestamp is 15s from the current timestamp.

[0053] Step 7: Under this method, participant node 2 receives the prepare refresh request of txn1 and adjusts the prepare timestamp of transaction txn1 to 18s. safe Updated to 18s, txn3 snapshot read ends the blocking state and returns the read result.

[0054] Step 8: At timestamp 40s on participant node 1, txn1 times out waiting for the write lock wlock1 to be released, and returns the result that the preparation of txn1's coordinator node has failed. The coordinator node of txn1 receives the preparation failure information and rolls back txn1 on participant nodes 1 and 2.

[0055] Step 9: Participant Node 2 receives a request to roll back txn1 at timestamp 40s. The resources locked by local txn1 are released, and the T safe Update to the latest write time 40s. And reply to the coordinator node of txn1. After receiving the replies from all the participant nodes, the coordinator node of txn1 replies to the client node that the transaction submission failed and has been rolled back. safe After updating to 40s, the snapshot read transaction txn3 ends blocking and returns the read result.

[0056] After the above simulation, the above method is run with the configuration parameters of consistency snapshot freshness and transaction pending time threshold of 8s and lock timeout of 30s. The experimental data that can be obtained are as follows: Figure 5 As shown, when a long transaction occurs due to deadlock or other reasons, compared with the safe timestamp method, the present invention can provide a consistent snapshot with a freshness of less than 8 seconds on the database node involved in the long transaction without interfering with the normal operation of the transaction, proving that this method can stably promote the generation of consistent snapshots without being blocked by long transactions. The safe timestamp method does not generate a new visible consistent snapshot until the long transaction is committed or rolled back. The freshness of the consistent snapshot provided during execution is linearly related to the running time of the long transaction and the lock timeout time. When the lock timeout time is reduced, some long transactions that can be submitted normally will be interrupted.

[0057] Compared with the existing technology, this method timely identifies the blocking problem of consistent snapshot generation caused by long-pending transactions, and updates the transaction preparation timestamp on the participant node by notifying the corresponding coordinator node to calculate the lower limit of the transaction commit timestamp and send a prepare refresh request, thereby regularly promoting the generation of consistent snapshots.

[0058] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for optimizing and generating a consistent snapshot of a distributed database, characterized in that: For a distributed database including multiple database nodes, the freshness of the local consistency snapshot of the database node is regularly detected, and when there is a transaction blocking the snapshot advancement on the node, the database node as the transaction coordinator is notified to initiate a transaction timestamp update operation for all nodes participating in the transaction, and a new consistency snapshot and the corresponding security timestamp are generated after the update is completed; The distributed database includes multiple database nodes, each of which stores part of the data locally. When a distributed transaction involving reading and writing data on multiple database nodes is executed, one of the database nodes where the data is located is selected as the coordinator node of the transaction, and the other nodes are selected as participant nodes of the transaction.

2. The method for optimizing and generating a consistent snapshot of a distributed database according to claim 1 is characterized in that: include: Step 1: Take any node in the distributed database as the current processing node, obtain the current timestamp TS and calculate the local security timestamp TS safe , and then calculate the local consistency snapshot freshness T fresh , specifically: in: The value is the minimum value of the preparation timestamp of all prepared but uncommitted transactions on the current processing node; when there is no prepared but uncommitted transaction is positive infinity; Step 2: When the freshness of the consistency snapshot of the current processing node in step 1 does not meet the threshold, execute step 3; otherwise, execute again from step 1 after a delay; Step 3: In step 1, the current processing node obtains the abnormal transaction that blocks the freshness of the consistency snapshot from the local transaction manager, that is, the set T of all prepared but uncommitted distributed transactions whose preparation timestamps do not meet the threshold value = {txn1, ..., txn n }, where: txn is a distributed transaction that has been prepared but not committed on the node, and the local transaction manager contains the preparation timestamp and coordinator node information of the transaction on the current processing node; Step 4: For each prepared but uncommitted transaction txn in the transaction set T i , the current processing node notifies the corresponding coordinator node Transaction txn i The prepare timestamp is too old; Step 5: For the prepared but uncommitted transaction txn in set T i The coordinator node After receiving the notification, calculate the lower limit of the submission timestamp And send a prepare refresh request to all participating nodes of the transaction; Step 6. All received txn i The database node that is ready to refresh the request will send txn i The prepare timestamp is updated to After the current processing node receives the prepare refresh request from all coordinator nodes in step 4, it recalculates the security timestamp and provides a consistent snapshot below the security timestamp, and then starts executing from step 1 again after a delay.

3. The method for optimizing and generating a consistent snapshot of a distributed database according to claim 2, characterized in that: The prepare refresh request is a request sent to a transaction participant node for refreshing the transaction prepare timestamp when the final state of the transaction has not been determined on the coordinator node of a transaction. When the coordinator node sends the prepare refresh request, it adds the transaction information to be refreshed and the lower limit of the transaction commit timestamp t min-commit When a participant node receives a prepare refresh request, it refreshes the prepare timestamp of the prepared but uncommitted transaction to t min-commit , and then update the security timestamp TS on the participant node safe , promoting the generation of consistent snapshots.

4. The method for optimizing and generating a consistent snapshot of a distributed database according to claim 2, characterized in that: After receiving the prepare request, the participant node locally obtains the resources required for transaction submission, obtains the timestamp as the timestamp of preparation completion, and returns the prepare timestamp and the result of successful preparation to the coordinator node. When the participant node encounters the pending transaction and causes the freshness of the consistency snapshot to exceed the threshold, it actively notifies the coordinator node of the transaction. When receiving the prepare refresh request from the coordinator node, it updates the prepare timestamp of the corresponding transaction according to the timestamp carried in the request, and notifies the participant node to update the consistency snapshot. When receiving the commit or rollback request from the coordinator node, it releases the resources occupied by the corresponding transaction and cleans up the local record of the transaction. It will no longer process the subsequent prepare refresh request for the transaction and directly reject it. The prepare timestamp ensures that the commit timestamp is greater than or equal to this timestamp when the transaction is finally committed; After receiving the transaction commit request from the client node, the coordinator node sends a prepare request to the corresponding participant node. Before receiving the prepare results from all participant nodes and after the pending time of the transaction exceeds a threshold, the coordinator node actively sends a prepare refresh request to the corresponding participant node. After receiving a notification from the participant node, the coordinator node sends a prepare refresh request to the participant node. After receiving the results of the prepare requests from all participant nodes, the coordinator node decides whether to commit the transaction. When the transaction needs to be committed, the latest timestamp is obtained until the timestamp is greater than the lower limit of the commit timestamp on the coordinator node. After the final result of the transaction is determined, the transactions on all participant nodes are cleared. After the commit timestamp is determined on the coordinator node, when receiving a notification from the participant node, the coordinator node no longer sends a prepare refresh request, but sends a transaction commit or rollback request. After receiving replies from all participant nodes, the coordinator node notifies the client node of the final result of the transaction.

5. The method for optimizing and generating a consistent snapshot of a distributed database according to claim 2, characterized in that: The triggering conditions of the prepare refresh request include: 1) When the pending time of a transaction on the coordinator node exceeds the configured threshold, the lower limit of the commit timestamp is periodically calculated and a prepare refresh request is sent to all participant nodes until the transaction is committed or rolled back on the coordinator node; 2) When the freshness of the consistent snapshot on the participant node does not meet the configuration, the participant node checks whether all the reasons for the blockage are caused by Too low leads to Too low, get the maximum value of all transaction read timestamps that exceed the configured time Check all prepared but uncommitted transactions on the node. The coordinator node of the transaction is requested to send a prepare refresh request to update the prepare timestamp of the transaction on the participant node; In the two trigger conditions of the prepare refresh request, the trigger thresholds can be configured independently to adapt to different business needs. For example, for a data range that requires a lower read blocking time, a higher consistency snapshot freshness can be configured on the corresponding database node separately, thereby ensuring a lower snapshot read blocking time.

6. The method for optimizing and generating a consistent snapshot of a distributed database according to any one of claims 2 to 5, characterized in that: The method for calculating the lower limit of the submission timestamp is to first obtain the maximum timestamp that has been received locally, and when the freshness required by the participant node is met, the timestamp is used as the lower limit of the submission timestamp. When the maximum timestamp received locally does not meet the freshness requirement, the latest timestamp is obtained as the lower limit of the submission timestamp.

7. The method for optimizing and generating a consistent snapshot of a distributed database according to claim 2, characterized in that: When the coordinator node sends a prepare refresh request to the participant node, it adopts an aggregation method to aggregate all prepare refresh requests with the same destination address within a period of time into a batch and then sends them to the corresponding node for unified processing, thereby reducing communication overhead; When the transaction preparation timestamp of a participant node is too old and the participant node sends a notification to the coordinator node, the node aggregates requests with the same destination address within a period of time into a batch and sends them to the corresponding node for unified processing, thus reducing communication overhead. In the distributed database cross-region multi-copy deployment scenario, after the freshness of the consistency snapshot on the participant node exceeds the configured threshold, the participant node requests the coordinator node of the corresponding transaction to obtain the known lower limit of the commit timestamp of the corresponding transaction, thereby avoiding the high latency overhead of cross-region communication.

8. A distributed database system for implementing the method according to any one of claims 1 to 7, characterized in that: include: A cluster management unit, a timestamp service unit and several data service units, wherein: the cluster management unit stores and manages metadata and node information and routes requests; the timestamp service unit calculates and assigns monotonically increasing timestamps; the data service unit parses SQL requests and implements data storage, data addition, deletion, modification and transaction processing.

9. The distributed database system according to claim 8, characterized in that: The data service unit comprises: an SQL parsing subunit, a prepare-to-refresh subunit, a data storage subunit and a transaction management subunit, wherein: the SQL parsing subunit analyzes and converts SQL statements into internal representations, ensures grammatical correctness, and generates an execution plan to optimize database operations; the prepare-to-refresh subunit handles the problem of too old consistency snapshots from the transaction management subunit, queries transactions generated by local blocking consistency snapshots and notifies the corresponding data service unit; handles pending timeouts of local transactions and blocking notifications of consistency snapshots from other data service units, calculates the lower limit of transaction commit timestamps according to local transaction status, and sends a prepare-to-refresh request to the relevant data service unit; the data storage subunit receives and processes data changes and query requests from the transaction management subunit, and manages all submitted data version information on the data service unit; the transaction management subunit executes transactions and manages temporary version data and transaction status information of transactions that have not been submitted on the data service unit, maintains security timestamps on the data service unit, and periodically calculates and updates security timestamps, checks the freshness of consistency snapshots and the pending duration of local transaction coordination records.

10. The distributed database system according to claim 8, characterized in that: The data service unit stores data of one or more data tables, each data table is divided into one or more logical data partitions, each data partition includes one or more physical data copies, one of which is used as a master copy, receives writes to the data partition and has all the latest data of the data partition, and the other copies are used as slave copies, which do not provide writes but provide reads, and the data copies are distributed on different data service units as needed; When a transaction involves different data partitions, the transaction is regarded as a distributed transaction. The system runs one or more distributed transactions simultaneously. Each distributed transaction maintains the latest status of the transaction in the data partition on the master copy of the corresponding data partition. The master copy copies the data to the slave copy, and the slave copy maintains the status of the transaction on the data partition at a certain time in the past. The distributed transaction interacts with the data service units where the master copies of the corresponding data partitions are located to implement the standard two-phase commit of the distributed transaction. At the same time, the master and slave copies of the corresponding data partitions can use the prepare refresh sub-unit to promote the generation of consistent snapshots.

Images