A data backup method, device, system, apparatus and storage medium

Abstract

Embodiments of the present specification disclose a data backup method, device, system, equipment and storage medium. The method comprises: performing physical backup on stored data in a storage node, and determining an operation log site corresponding to a starting moment of the physical backup in the storage node as a first site; constructing a set of commitable transactions, and determining a pruning site of the storage node; wherein the set of commitable transactions comprises two-phase commit transactions with a commit event between the first site and the pruning site; the pruning site of the storage node is the latest preparation event end site recorded in the operation log of the storage node between the two-phase commit transactions of the set of commitable transactions for the storage node; and the set of commitable transactions and the operation log between the first site and the pruning site in the storage node are backed up.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of database technology, and in particular to a data backup method, apparatus, system, device, and storage medium. Background Technology

[0002] For databases where data is distributed across at least two storage nodes, backups need to be performed on multiple storage nodes.

[0003] In related technologies, logical backups of databases can be performed. Specifically, a logical backup can be achieved by retrieving data from all tables in the database using a `SELECT *` query. However, logical backups consume significant database resources, impacting the execution of the database's core business logic. Summary of the Invention

[0004] To address the aforementioned problems, this specification provides a data backup method, apparatus, system, device, and storage medium. The technical solution is shown below.

[0005] A data backup method, applied to a database, the database including at least two storage nodes; the method includes:

[0006] Perform physical backups on the data stored in the storage node, and determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point;

[0007] Construct a set of committable transactions and determine the pruning point for the storage nodes;

[0008] The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point.

[0009] The pruning point of the storage node includes the latest recorded preparation event end point in the operation log of the storage node between the two-phase commit transactions for that storage node in the set of committable transactions;

[0010] Backup the set of committable transactions, and the operation logs between the first point and the pruning point in the storage node.

[0011] A data backup device is applied to a database, the database including at least two storage nodes; the device includes:

[0012] The physical backup unit is used to perform physical backups of the data stored in the storage node and to determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point.

[0013] The building unit is used to construct a set of committable transactions and determine the pruning point of the storage node;

[0014] The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point.

[0015] The pruning point of the storage node includes the latest recorded preparation event end point in the operation log of the storage node between the two-phase commit transactions for that storage node in the set of committable transactions;

[0016] The backup unit is used to back up the set of committable transactions and the operation logs between the first point and the pruning point in the storage node.

[0017] The above technical solution can back up the data stored in the database through physical backup and backup operation logs, without the need for logical backup, which can reduce the impact of database data backup on the database's own business operations. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a flowchart illustrating a data backup method provided in an embodiment of this specification;

[0020] Figure 2 This is a schematic diagram of the structure of a database provided in the embodiments of this specification;

[0021] Figure 3 This is a schematic diagram illustrating the principle of operation log backup provided in the embodiments of this specification;

[0022] Figure 4 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0023] Figure 5 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0024] Figure 6 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0025] Figure 7 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0026] Figure 8 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0027] Figure 9 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0028] Figure 10 This is a schematic diagram illustrating the principle of another operation log backup provided in the embodiments of this specification;

[0029] Figure 11 This is a flowchart illustrating another data backup method provided in the embodiments of this specification;

[0030] Figure 12 This is a schematic flowchart of a data recovery method provided in the embodiments of this specification;

[0031] Figure 13 This is a flowchart illustrating another data backup method provided in the embodiments of this specification;

[0032] Figure 14 This is a schematic diagram of the structure of a data backup device provided in the embodiments of this specification;

[0033] Figure 15 This is a schematic diagram of another data backup device provided in the embodiments of this specification;

[0034] Figure 16 This is a schematic diagram of the architecture of a data backup system provided in the embodiments of this specification;

[0035] Figure 17 This is a schematic diagram of the structure of a device used to configure the methods of the embodiments of this specification. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solutions in the embodiments of this specification, the technical solutions in the embodiments of this specification will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art should fall within the scope of disclosure.

[0037] For databases where data is distributed across at least two storage nodes, backups need to be performed on multiple storage nodes.

[0038] In related technologies, logical backups of databases can be performed. Specifically, a logical backup can be achieved by retrieving data from all tables in the database using a `SELECT *` query. However, logical backups consume significant database resources, impacting the execution of the database's core business logic.

[0039] For example, logical backups require scanning large amounts of data, which can impact database performance and is typically scheduled during off-peak hours. Furthermore, because they are logical backups and can only be performed at the table level, the overall throughput is low, the time is long, and the backup efficiency is low for scenarios with extremely large database datasets.

[0040] To address the aforementioned problems, this specification provides a data backup method. This method can be used to back up stored data in a database.

[0041] This method allows for a physical backup of all stored data on each storage node in the database. For a detailed explanation of physical backup, please refer to relevant technical documentation. Specifically, the database can be a distributed database, where the stored data can be distributed across at least two storage nodes.

[0042] Compared to logical backups, physical backups are more efficient, faster, and have less impact on the database's business operations.

[0043] In the case of databases, data is distributed across different storage nodes, and there can be relationships between the data on different storage nodes.

[0044] For example, two storage nodes can each store a record for the same transaction, and these two records are related. If you need to modify the record for that transaction, you need to modify the corresponding records on both storage nodes.

[0045] Specifically, one of the two storage nodes can be used to store buyer data, and the other can be used to store seller data. For the same transaction, records from the seller's perspective and records from the buyer's perspective need to be stored on these two storage nodes respectively. These two records are therefore related.

[0046] When you need to query information about this transaction later, you can combine these two records to ensure the integrity of the transaction data.

[0047] Based on the data relationships in the database, the database needs to maintain transaction consistency, which means that transactions must adhere to data integrity constraints.

[0048] For example, when a transaction needs to modify related records on two storage nodes, the execution results of the transaction on these two storage nodes should be consistent, either both successfully rolled back or both successfully committed.

[0049] Specifically, regarding the seller's storage node and the buyer's storage node, if the buyer's storage node successfully commits the transaction and stores the buyer's perspective information, while the seller's data is successfully rolled back, the seller's perspective information for that transaction will no longer exist. This will result in incomplete transaction data.

[0050] To ensure that the stored data in the database can maintain transaction consistency after being restored from a backup, this method can further back up the necessary operation logs in each storage node during the backup process. This allows for maintaining transaction consistency based on the backed-up operation logs when restoring the stored data in the database.

[0051] For a detailed explanation, please refer to the explanation below.

[0052] Since the method provided in the embodiments of this specification can achieve the backup of data stored in the database through physical backup and backup operation log, without the need for logical backup, the impact of database backup on the database's own business execution can be reduced.

[0053] Furthermore, this method is more efficient than logical backup; and the backup of operation logs will not affect the database's business operations, further reducing the impact of database backup on database business operations.

[0054] like Figure 1 The diagram shown is a flowchart illustrating a data backup method provided in an embodiment of this specification.

[0055] This method is not limited to any particular entity; it can be applied to databases to back up stored data.

[0056] A database may include at least two storage nodes; the stored data in the database may be distributed across at least two storage nodes.

[0057] Optionally, the database may also include compute nodes. Compute nodes can be used to handle distributed data routing, computation, and dynamic scheduling, as well as distributed transaction 2PC coordination and global secondary index maintenance. Specifically, they can be used to initiate transactions against storage nodes, such as two-phase commit transactions.

[0058] For ease of understanding, such as Figure 2 The diagram shown is a schematic representation of a database structure provided in an embodiment of this specification. Figure 2 The database can include 2 compute nodes and 3 storage nodes.

[0059] Of course, this specification does not limit the specific number of compute nodes and storage nodes in the database. Figure 2 This is for illustrative purposes only.

[0060] Alternatively, the database may specifically be a distributed database, such as the PolarDB-X database.

[0061] Optionally, this method can be executed by a compute node in the database. The compute node executing this method can be called a control node.

[0062] Optionally, a new node can be added to the database to execute this method. This newly added node in the database can have control permissions over the stored nodes in the database.

[0063] Optionally, this method can also be executed by a device outside the database, which can be used to back up the stored data in the database. Specifically, this device can be a management device with control permissions over the database. Specifically, it can have control permissions over the storage nodes and compute nodes within the database.

[0064] Optionally, since each storage node can store a portion of the stored data in the database, different storage nodes can have different operation logs. Specifically, the operation logs can include data update logs, which are used to record the update status of the database data stored by the storage node.

[0065] Of course, operation logs can also include database logs, such as database logical logs or database transaction logs, which are used to record the update status of database data stored on the storage nodes.

[0066] This method may include the following steps.

[0067] S101: Perform a physical backup of all stored data in each storage node, and determine the operation log position corresponding to the start time of the physical backup in that storage node as the first point.

[0068] S102: If it is determined that all two-phase commit transactions in the incomplete transaction set have been completed, initiate a heartbeat transaction for all storage nodes.

[0069] Optionally, the set of incomplete transactions may include all two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0070] S103: If all storage nodes successfully commit heartbeat transactions, for each storage node, add all two-phase commit transactions that have commit events between the first moment and the heartbeat transaction preparation event to the current set of committable transactions; add the heartbeat transaction to the current set of committable transactions.

[0071] S104: Circularly update the current set of committable transactions and the current pruning point of each storage node until, for each storage node, all two-phase commit transactions that have a commit event with the current pruning point at the first moment are included in the current set of committable transactions.

[0072] The current pruning point of each storage node is the end point of the latest prepared event recorded in the node's operation log among all two-phase commit transactions for that node in the current set of committable transactions.

[0073] S105: After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in each storage node to the current pruning point.

[0074] The above method can back up the data stored in the database through physical backup and backup operation logs, without the need for logical backup, which can reduce the impact of database backup on the database's business operations.

[0075] Furthermore, this method is more efficient than logical backup; and the backup of operation logs will not affect the database's business operations, further reducing the impact of database backup on database business operations.

[0076] Furthermore, the full storage of data, operation logs, and the current set of committable transactions in the storage nodes backed up based on the above method and process can ensure transaction consistency when restoring the stored data in the database later.

[0077] The following explains how the data backed up in this process maintains transactional consistency after recovery.

[0078] In one alternative embodiment, transactions that need to be performed in the database can typically be divided into transactions for a single storage node and transactions for at least two storage nodes.

[0079] Transactions targeting a single storage node typically involve updating the database within that single storage node.

[0080] Transactions involving at least two storage nodes typically involve updating data that is related between at least two storage nodes.

[0081] Therefore, when considering transaction consistency in a database, transactions that only affect a single storage node are typically not considered. Regardless of whether such a transaction succeeds or fails, it only updates data in that single storage node and does not affect other storage nodes, thus not impacting transaction consistency. For example, a transaction that modifies data in a single storage node.

[0082] For transactions involving at least two storage nodes, it is necessary to ensure that the execution results of the transaction are consistent across all the targeted storage nodes. Either all nodes should be rolled back, or all nodes should be committed.

[0083] In an alternative embodiment, in the database, transactions for at least two storage nodes can be executed via a two-phase commit transaction.

[0084] Optionally, a two-phase commit transaction can specifically be a distributed transaction that follows a two-phase commit protocol. A two-phase commit transaction can specifically include: an XA transaction.

[0085] MySQL implements the open X / Open CAE specification, and XA transactions can specifically be a distributed transaction execution method based on the XA interface provided by MySQL.

[0086] For example, a transaction that modifies related data in two storage nodes can be executed following a two-phase commit protocol.

[0087] Since two-phase commit transactions involve at least two storage nodes, this method focuses on performing relevant steps for two-phase commit transactions to ensure transaction consistency between storage nodes.

[0088] I. The following is a detailed explanation of two-phase commit transactions.

[0089] For two-phase commit transactions, there are a prepare phase and a commit phase, also known as the prepare phase and the commit phase.

[0090] 1) During the prepare phase, the compute nodes in the database can send the prepare message, or prepare message, of the transaction to all storage nodes targeted by the two-phase commit transaction.

[0091] After receiving the transaction's commit message, the storage node can process the transaction locally and write the processing result to the log.

[0092] For example, a two-phase commit transaction specifically modifies a set of related data on two storage nodes. After receiving the transaction, the two storage nodes can retrieve the related data from their storage, modify it according to the transaction, and write the modification results to a log, specifically a redo log.

[0093] At this point, the two-phase commit transaction is in the prepare state. The storage node can then write the transaction's prepare event to the operation log, i.e., the prepare event. Specifically, this can be written to the database logical log.

[0094] The storage node can return a message indicating that preparation was successful to the compute node.

[0095] During the prepare phase, the storage node can continuously perform the following operations: after receiving the prepare to commit message for the transaction, it can process the transaction locally and write the processing result to the log; write the prepare event of the transaction to the operation log; and return a prepare success message to the compute node.

[0096] Therefore, the transaction preparation event in the operation log of the storage node can indicate that the transaction is in a state of being ready to commit, and the processing result has been written to the log, which makes it convenient to directly write the data into the database stored by the storage node based on the commit information.

[0097] In other words, a successful "prepare" of a transaction indicates that the transaction preparation event has been written to the operation log.

[0098] 2) During the commit phase, once the compute node has determined that all storage nodes targeted by the two-phase commit transaction have successfully prepared (i.e., all storage nodes targeted by the two-phase commit transaction are in the prepare state), it can send a commit message to all storage nodes targeted by the transaction.

[0099] In a two-phase commit transaction, all storage nodes can execute the commit operation after receiving the commit message for the transaction. This writes the transaction processing results from the log into the database stored on the storage nodes, completing the commit. From the perspective of the database as a whole, this can also be explained as the transaction processing results being written into the database.

[0100] At this point, the two-phase commit transaction is in the commit state. The storage node can then write the transaction's commit event to its operation log (i.e., a commit event) upon successful writing of the data to the database. Specifically, this can be written to the database logical log.

[0101] The storage node can return a successful commit message to the compute node.

[0102] During the commit phase, the storage node can perform the following operations continuously: after receiving the commit message of the transaction, it can execute the commit operation and write the transaction processing results in the log into the database data stored by the storage node; if the write is successful, it writes the commit event of the transaction into the operation log and returns the commit success message to the compute node.

[0103] Therefore, the transaction commit event in the operation log of the storage node can indicate that the transaction is in a committed state and the processing result has been successfully written to the database stored by the storage node, that is, the data has been successfully written to the database.

[0104] In other words, a successful transaction commit indicates that the transaction's commit event has been written to the operation log.

[0105] 3) In addition, if a compute node determines that either storage node's prepare operation for a two-phase commit transaction has failed or has timed out, it can initiate a rollback message for that transaction.

[0106] After receiving the rollback message for the two-phase commit transaction, all storage nodes can perform the rollback operation to complete the rollback.

[0107] At this point, the two-phase commit transaction is in a rollback state. The storage node can write the rollback event of this transaction to the operation log, i.e., the rollback event, to indicate that the transaction has been rolled back. Specifically, it can be written to the database logical log.

[0108] The storage node can send a successful rollback message back to the compute node.

[0109] The storage node can continuously perform the following operations: upon receiving a rollback message for the transaction, it can perform a rollback operation and delete the transaction's processing result from the log; if the deletion is successful, it writes the rollback event of the transaction to the operation log and returns a rollback success message to the compute node.

[0110] Therefore, the transaction rollback event in the operation log of the storage node can indicate that the transaction is in a rollback state and the processing result has been deleted.

[0111] In other words, a successful transaction rollback indicates that the transaction rollback event has been written to the operation log.

[0112] It should be noted that a two-phase commit transaction can be considered complete if all the target storage nodes have been successfully committed or rolled back.

[0113] If either of the storage nodes targeted by a two-phase commit transaction fails to commit or roll back successfully, the transaction is still in progress and remains incomplete.

[0114] For two-phase commit transactions, the operation log can correspond to prepare events, commit events, and rollback events.

[0115] After explaining the two-phase commit transaction, we will further explain the incomplete transaction set and heartbeat transaction in the above method flow.

[0116] II. Set of incomplete transactions.

[0117] It should be noted that after performing a physical backup of all stored data on each storage node, the backed-up database data may be inconsistent.

[0118] For example, a transaction may have been committed on one storage node but not on another. Alternatively, a transaction may have been rolled back on one storage node but not on another.

[0119] If subsequent log backups do not include transaction commit or rollback events, the outcome of the transaction cannot be determined. This can result in fragmented commit transactions appearing in full backups and backup logs on different nodes.

[0120] For example, if a transaction has been successfully committed on a storage node before a physical backup, the operation log after the first point of the subsequent backup of that storage node will not contain the commit event of that transaction.

[0121] The transaction has not yet been committed on another storage node, but the operation logs after the first backup point may not contain the transaction's commit event either.

[0122] During subsequent recovery, it becomes impossible to determine whether the transaction could be committed based on the operation logs after the first backup point.

[0123] Therefore, we can first address the two-phase commit transactions that are not yet completed in the database at a certain moment, and wait for all of these transactions to be completed.

[0124] Specifically, the set of incomplete transactions can be identified, including all two-phase commit transactions that were not completed in the database immediately after the physical backup.

[0125] Specifically, all incomplete two-phase commit transactions in the database can include: all two-phase commit transactions that have been initiated but not yet completed in the database.

[0126] After the physical backup, the set of incomplete transactions can be identified, and we can wait for all two-phase commit transactions in the set of incomplete transactions to complete.

[0127] Based on the above analysis, the completion of a two-phase commit transaction can include: each storage node targeted by the two-phase commit transaction successfully rolling back the two-phase commit transaction; or each storage node targeted by the two-phase commit transaction successfully committing the two-phase commit transaction.

[0128] It should be noted that since it has been determined that each two-phase commit transaction in the set of incomplete transactions has been completed, it can be determined that for each two-phase commit transaction in the set of incomplete transactions, all the storage nodes targeted have been successfully rolled back or successfully committed.

[0129] Based on the above explanation of the two-phase commit transaction, a successful transaction rollback indicates that the corresponding rollback event has been recorded in the storage node's operation log. A successful transaction commit indicates that the corresponding commit event has been recorded in the storage node's operation log.

[0130] Therefore, once it is determined that each two-phase commit transaction in the incomplete transaction set has been completed, the operation log of each storage node for that transaction has already recorded the processing result of the transaction completion. Specifically, this could be a rollback event or a commit event.

[0131] By physically backing up the full data stored in the storage nodes, as well as backing up the operation logs containing the completion of each two-phase commit transaction in the incomplete transaction set from the first point, the full data can be stored based on the physically backed-up storage nodes. In the subsequent recovery process, it is ensured that each storage node corresponding to each two-phase commit transaction in the incomplete transaction set can complete a consistent processing result based on the backed-up operation log content, thus guaranteeing the consistency of each two-phase commit transaction in the incomplete transaction set.

[0132] In this process, when backing up operation logs later, the operation logs from the first point (physical backup start time) to the current pruning point in each storage node can be backed up.

[0133] The current pruning point can be set on each storage node, either during or after the heartbeat transaction preparation event. See below for a detailed explanation.

[0134] Furthermore, since heartbeat transactions are initiated only after each two-phase commit transaction in the set of incomplete transactions has been completed, the preparation event for a heartbeat transaction on each storage node occurs after the completion of each two-phase commit transaction in the set of incomplete transactions.

[0135] In other words, the operation log of each backed-up storage node can contain the operation log content of each two-phase commit transaction from the first point to the completion of the incomplete transaction set.

[0136] Therefore, for all two-phase commit transactions in the incomplete transaction set, the processing result of each storage node can be determined by the backup operation log, which can be either rollback or commit. This ensures that when restoring using the backup operation log, the consistency of these transactions across different storage nodes can be guaranteed.

[0137] Therefore, for all two-phase commit transactions that are not completed in the database at the first moment, it is possible to ensure transaction consistency during subsequent recovery.

[0138] It should be noted that since physical backups are performed directly on all stored data in each storage node, transaction consistency between different storage nodes is not guaranteed. Subsequent log backups also start from the first point (the start time of the physical backup). Therefore, for transactions initiated before the physical backup, the logs of subsequent backups may lack relevant events for these transactions.

[0139] For example, the backup log may lack transaction preparation events initiated before the physical backup, or it may lack transaction commit events initiated before the physical backup.

[0140] By setting a set of incomplete transactions at the first moment, and by waiting for all two-phase commit transactions in the incomplete transaction set to complete, and by backing up the corresponding logs, the consistency of these transactions can be ensured, thereby avoiding the impact on transaction consistency due to missing transaction-related events in the backup logs.

[0141] Furthermore, since the incomplete transaction set includes all two-phase commit transactions that were not completed in the database at the first moment, the consistency of the processing results can be ensured based on the backup continuous logs for all two-phase commit transactions that were completed in the database at the first moment and before. This also ensures transaction consistency.

[0142] Then, for transactions initiated after the first moment, transaction consistency can be ensured.

[0143] Since transactions initiated after the initial transaction (e.g., prepare, commit, or rollback events) are recorded in the operation log after the corresponding point in the initial transaction, the relevant events can be found through the backup log to ensure transaction consistency. See the explanation below for details.

[0144] For ease of understanding, such as Figure 3 The diagram shown is a schematic representation of an operation log backup method provided in an embodiment of this specification.

[0145] It displays the operation logs of storage nodes 1-3, as well as the first point corresponding to the physical backup, the point corresponding to the first moment, and the point corresponding to the initiation of the heartbeat transaction. It also shows the positions of the preparation and commit events of transactions 1-3 in the operation logs.

[0146] For ease of description, the preparation event of transaction 1 is described as P1, and the commit event of transaction 1 is described as C1; the preparation event of transaction 2 is described as P2, and the commit event of transaction 2 is described as C2; the preparation event of transaction 3 is described as P3, and the commit event of transaction 3 is described as C3.

[0147] Among them, transaction 1 is a transaction for storage node 1 and storage node 2; transaction 2 is a transaction for storage node 1 and storage node 2; and transaction 3 is a transaction for storage node 2 and storage node 3.

[0148] When performing a physical backup of the full storage data on storage nodes 1-3 at the first location, it can be seen that transaction 1 is still in the "preparing to commit" state on storage node 1, while it is in the "committed" state on storage node 2. At this time, the data updated by transaction 1 is inconsistent between storage nodes 1 and 2.

[0149] Furthermore, at this point, transaction 2 is in the "ready to commit" state on its target storage node 1, but not yet successfully committed on its target storage node 2. Therefore, the data updated by transaction 2 is inconsistent between storage node 1 and storage node 2.

[0150] Immediately after the physical backup, Transaction 1 has been completed, and the operation logs of both storage node 1 and storage node 2 have recorded commit events, indicating that the transaction has been successfully committed.

[0151] When restoring using backup logs, the storage nodes store the full data based on the physical backup. The full data backed up by storage node 1 can successfully commit transaction 1 based on the C1 event, thus maintaining consistency with transaction 1 in the full data backed up by storage node 2, achieving consistency of transaction 1.

[0152] In other words, for two-phase commit transactions completed before the first moment, transaction consistency between different storage nodes can be ensured based on the backup logs.

[0153] At the first moment, the incomplete two-phase commit transactions in the database include transaction 2 and transaction 3. Transaction 2, which targets storage node 2, has completed its commit, but the storage node 1, which targets transaction 2, has not yet completed its commit. Therefore, transaction 2 is not yet complete.

[0154] Similarly, the storage node 2 targeted by transaction 3 has not yet completed the preparation for commit, and the storage node 3 targeted by transaction 3 has not yet completed the commit. Therefore, transaction 3 has not yet been completed.

[0155] Afterwards, we can wait for transactions 2 and 3 to complete before initiating heartbeat transactions for storage nodes 1-3.

[0156] In other words, once both storage nodes 1 and 2 targeted by transaction 2 have been committed, and both storage nodes 2 and 3 targeted by transaction 3 have been committed, then heartbeat transactions can be initiated for storage nodes 1-3.

[0157] When backing up logs later, the backed-up logs can include the logs from the first point in storage nodes 1-3 to the point where the heartbeat transaction was initiated.

[0158] When restoring using backup logs, the storage nodes store the full data based on the physical backup. The full data backed up by storage node 1 can be used to successfully commit transaction 2 based on the C2 event; the full data backed up by storage node 2 can be used to successfully commit transaction 2 based on the P2 and C2 events, thus ensuring that transaction 2 remains consistent between storage node 1 and storage node 2, achieving consistency of transaction 2.

[0159] Similarly, transaction 3 can also maintain consistency.

[0160] In other words, for two-phase commit transactions that are not completed in the first moment, transaction consistency between different storage nodes can be ensured based on the backup logs.

[0161] III. Heartbeat Matters.

[0162] For two-phase commit transactions initiated after the first moment, heartbeat transactions can be used to mark all storage nodes, thereby ensuring transaction consistency.

[0163] First, based on the above explanation of the two-phase commit transaction, and combined with the diagram, we can infer the relationship between the relevant events in the two-phase commit transaction.

[0164] 1) Corollary 1: For each two-phase commit transaction, in each storage node targeted by the transaction, the transaction preparation event is recorded before the time when the compute node issues the transaction commit information, and the transaction commit event is recorded after the time when the compute node issues the transaction commit information.

[0165] Based on the above explanation of two-phase commit transactions, a compute node will only issue a commit message for a transaction if it is certain that all storage nodes targeted by the two-phase commit transaction have successfully prepared.

[0166] Since a successful prepare operation for all target storage nodes may include recording the transaction preparation event in the operation log of all target storage nodes.

[0167] Therefore, the moment when the compute node issues the commit message can be regarded as the dividing line. In each storage node targeted by the transaction, the transaction preparation event is recorded before the moment when the compute node issues the transaction commit message.

[0168] Based on the above explanation of two-phase commit transactions, all storage nodes targeted by a two-phase commit transaction can execute a commit operation after receiving the commit message of the transaction, writing the transaction processing results in the log into the database data stored in the storage node, thus completing the commit.

[0169] The completion of the commit operation may include: receiving the commit message issued by the compute node, performing the subsequent commit, and recording the transaction commit event in the operation log if the data is successfully written to the database stored on the storage node.

[0170] Therefore, the moment when the compute node issues the commit message can be regarded as the dividing line. In the operation log of each storage node targeted by the transaction, the transaction commit event is recorded after the moment when the compute node issues the transaction commit message.

[0171] For ease of understanding, such as Figure 4 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0172] It displays the operation logs of storage nodes 1-3, and shows the positions of the prepare and commit events of transaction 1 in the operation logs.

[0173] For ease of description, the prepare event of transaction 1 is described as P1, and the commit event of transaction 1 is described as C1. Here, transaction 1 is a transaction targeting storage nodes 1-3.

[0174] As can be seen, after transaction 1 is ready to be committed successfully, storage nodes 1-3 record the preparation event in the log and send the message of successful preparation to the compute node.

[0175] Once the compute node has determined that transaction 1 in storage nodes 1-3 has been successfully prepared, it can issue a commit message for transaction 1.

[0176] Due to message sending delays, commit messages may arrive at different storage nodes at different times. Afterward, storage nodes 1-3 complete the commit and record the commit event C1 of transaction 1 in the log.

[0177] Therefore, taking the moment when the compute node issues the commit message of transaction 1 as the dividing line, it can be determined that in the operation log of each storage node targeted by transaction 1, P1 is before the dividing line and C1 is after the dividing line.

[0178] Based on Corollary 1, it can be determined that for the same two-phase commit transaction, the prepare event precedes the commit event in each storage node targeted by the transaction.

[0179] by Figure 3 For example, in storage nodes 1-3, P1 is always before the boundary line, which means it is always before C1.

[0180] Based on Corollary 1, it can also be determined that for each two-phase commit transaction, in each storage node targeted by the transaction, the moment when the compute node issues the transaction commit information is between the transaction preparation event and the transaction commit event.

[0181] by Figure 3 For example, in storage nodes 1-3, the commit message for transaction 1 is sent between P1 and C1.

[0182] 2) Corollary 2: If, in the operation log of a storage node, there is a preparation event for a two-phase commit transaction 2 after the commit event of two-phase commit transaction 1, then it can be determined that the commit message of transaction 1 was sent earlier than the commit message of transaction 2.

[0183] Based on Corollary 1, if there is a commit event for two-phase commit transaction 1 in the operation log of a storage node, then it can be determined that the commit message of transaction 1 was sent before the time corresponding to the commit event of transaction 1.

[0184] Based on inference 1, if there is a two-phase commit transaction 2 preparation event in the operation log of a storage node, then it can be determined that the commit message of transaction 2 was sent after the transaction 2 preparation event.

[0185] Furthermore, if in the operation log of a storage node, after the commit event of two-phase commit transaction 1, there is a preparation event of two-phase commit transaction 2, then it can be sorted in chronological order from front to back to obtain: the time when the commit message of transaction 1 was issued, the commit event of transaction 1, the preparation event of transaction 2, and the time when the message of transaction 2 was issued.

[0186] This also confirms that the commit message of transaction 1 was sent earlier than the commit message of transaction 2.

[0187] For ease of understanding, such as Figure 5 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0188] It displays the operation logs of the storage nodes, and shows the positions of the prepare and commit events of transaction 1-2 in the operation logs.

[0189] For ease of description, the prepare event of transaction 1 is described as P1, and the commit event of transaction 1 is described as C1. The prepare event of transaction 2 is described as P2, and the commit event of transaction 2 is described as C2.

[0190] In this operation log, C1 can precede P2. Furthermore, P1 must precede C1, and C2 must follow P2.

[0191] Therefore, the commit message for transaction 1 must be sent between P1 and C1, that is, before C1.

[0192] The commit message for transaction 2 must be sent between P2 and C2, that is, after P2.

[0193] Since C1 comes before P2, the sequence from beginning to end must be P1, send the commit message of transaction 1, C1, P2, send the commit message of transaction 2, C2.

[0194] Therefore, the commit message of transaction 1 is sent earlier than the message of transaction 2.

[0195] 3) Corollary 3: If, in the operation log of a storage node, there is a prepare event for a two-phase commit transaction 2 after the commit event of two-phase commit transaction 1, then it can be determined that, in the operation log of each storage node, there is no prepare event for transaction 1 after the commit event of transaction 2.

[0196] Based on Corollary 2, it can be determined that if, in the operation log of a storage node, after the commit event of two-phase commit transaction 1, there exists a preparation event for two-phase commit transaction 2, then it can be determined that the commit message of transaction 1 was sent earlier than the message of transaction 2.

[0197] Based on Corollary 2, it can also be determined that if there is a preparation event for transaction 1 after the commit event of transaction 2 in the operation log of a storage node, then it can be determined that the commit message of transaction 2 was sent earlier than the message sent by transaction 1.

[0198] It should be noted that in a database, there can be multiple compute nodes. A compute node is typically designed for a single transaction and can issue a commit message for that transaction only once.

[0199] Therefore, the order in which transaction commit messages are sent is fixed for different transactions. The scenario where the commit message of transaction 1 is sent earlier than the message of transaction 2, and the scenario where the commit message of transaction 2 is sent earlier than the message of transaction 1, will not exist simultaneously.

[0200] In other words, in the operation log of a storage node, the situation where the commit event of transaction 1 is followed by the prepare event of transaction 2, and the situation where the commit event of transaction 2 is followed by the prepare event of transaction 1, will not occur simultaneously.

[0201] Therefore, if in the operation log of a storage node, after the commit event of two-phase commit transaction 1, there exists a prepare event of two-phase commit transaction 2, then it can be determined that in the operation log of each storage node, after the commit event of transaction 2, there is no prepare event of transaction 1.

[0202] For ease of understanding, let P1 represent the prepare event of transaction 1, and C1 represent the commit event of transaction 1. Let P2 represent the prepare event of transaction 2, and C2 represent the commit event of transaction 2.

[0203] If the operation log of any storage node contains a sequential order of P1-C1-P2-C2, it indicates that the commit message of transaction 1 was sent before the commit message of transaction 2.

[0204] The sequential order of P2-C2-P1-C1 is not present in the operation logs of other storage nodes.

[0205] Furthermore, although the order in which transactions send commit messages is fixed, the order in which each storage node completes transaction commits cannot be controlled.

[0206] For example, if a compute node sends commit messages to transaction 1 and transaction 2 in succession, but transaction 1 needs to commit a larger amount of data while transaction 2 needs to commit a smaller amount of data, then the time when transaction 2 completes its commit may be earlier than the time when transaction 1 completes its commit.

[0207] This would result in the transaction log showing that the commit event of transaction 2 occurred before the commit event of transaction 1.

[0208] Therefore, if the operation log of any storage node contains the sequence P1-C1-P2-C2, then the operation logs of other storage nodes may contain the following possibilities: P1-P2-C1-C2, P1-P2-C2-C1, P2-P1-C1-C2, P2-P1-C2-C1.

[0209] For ease of understanding, such as Figure 6The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0210] It displays the operation logs of storage nodes 1-5, and shows the positions of the prepare and commit events of transaction 1-2 in the operation logs.

[0211] For ease of description, the prepare event of transaction 1 is described as P1, and the commit event of transaction 1 is described as C1. The prepare event of transaction 2 is described as P2, and the commit event of transaction 2 is described as C2.

[0212] 4) Corollary 4: If, in the operation log of a storage node, there is a prepare event for a two-phase commit transaction 2 after the commit event of two-phase commit transaction 1, then it can be determined that in the operation log of each storage node jointly targeted by transaction 1 and transaction 2, the prepare event of transaction 1 is before the commit event of transaction 2.

[0213] Based on Corollary 2, if in the operation log of a storage node, after the commit event of two-phase commit transaction 1, there exists a preparation event for two-phase commit transaction 2, then it can be determined that the commit message of transaction 1 was sent earlier than the commit message of transaction 2.

[0214] Because in the operation logs of each storage node targeted by transaction 1, the preparation events of transaction 1 are all before the time when the commit message of transaction 1 is sent, and the time when the commit message of transaction 1 is sent is earlier than the time when the commit message of transaction 2 is sent.

[0215] Therefore, the chronological order of the operation logs of each storage node targeted by transaction 1 can be obtained as follows: the preparation event of transaction 1, the time when the commit message of transaction 1 is sent, and the time when the commit message of transaction 2 is sent.

[0216] Furthermore, based on Inference 1, it can be determined that in the operation log of each storage node targeted by Transaction 2, the commit event of Transaction 2 is always after the time when the commit message of Transaction 2 is sent.

[0217] Therefore, the above sequence of events can be updated as follows: Transaction 1 preparation event, Transaction 1 commit message issuance time, Transaction 2 commit message issuance time, Transaction 2 commit event.

[0218] Therefore, it can be determined that in the operation logs of each storage node jointly targeted by Transaction 1 and Transaction 2, the preparation event of Transaction 1 occurs before the commit event of Transaction 2.

[0219] by Figure 6 For example, among all nodes 1-5 targeted by transaction 1, P1 is always before the time when transaction 1 sends its commit message, which is also before the time when transaction 2 sends its commit message, which is also before C2.

[0220] 5) Based on the four inferences derived above, the principle of using heartbeat transactions to ensure transaction consistency can be further explained.

[0221] Optionally, heartbeat transactions are initiated for all storage nodes; therefore, heartbeat transactions also belong to two-phase commit transactions.

[0222] Since heartbeat transactions are also two-phase commit transactions, with prepare and commit events, for any transaction in the operation log of any storage node that has a commit event before the heartbeat transaction's prepare event, we can use the above corollary four to determine that in the operation logs of all storage nodes targeted by the transaction, the prepare events of the transaction are all before the heartbeat transaction's commit events.

[0223] Optionally, if a transaction has a commit event in the operation log of any storage node before the heartbeat transaction preparation event, it can be determined that the compute node has sent the commit message for the transaction and carried out the subsequent commit operation.

[0224] Therefore, the outcome of this transaction can be determined as "commit" rather than "rollback".

[0225] For ease of description, this transaction, that is, a two-phase commit transaction in the operation log of any storage node that has a commit event before the heartbeat transaction preparation event, is called a specific transaction.

[0226] Using the above inference four, it can be determined that in the operation logs of all storage nodes targeted by a specific transaction, the preparation events of the specific transaction are all before the heartbeat transaction commit events.

[0227] To ensure the consistency of a specific transaction, it may be optional to limit the operation log of each storage node backup to include at least the preparation event of that specific transaction.

[0228] Based on the above explanation of two-phase commit transactions, the transaction preparation event in the operation log indicates that the transaction is in a state of being ready to commit, and the relevant processing results have been written to the log. It only needs to receive the commit message to write to the database and complete the commit.

[0229] Furthermore, for each specific transaction, the outcome has already been determined as "commit".

[0230] In the operation logs of each storage node backup for a specific transaction, since they all contain the preparation event of the specific transaction, regardless of whether the commit event of the specific transaction is included, when restoring the database stored data later, the processing result of the specific transaction can be determined to be "committed". Therefore, even if the commit event of the specific transaction is not included in the backup operation log, the "preparation to commit state" of the specific transaction can be restored based on the "preparation event" of the specific transaction. Then, the compute node can issue the commit message of the specific transaction to complete the subsequent commit steps, ensuring that the processing result of the specific transaction across different storage nodes is "committed successfully", thus ensuring transaction consistency.

[0231] If the backup operation log includes a specific transaction commit event, the specific transaction can be restored based on the operation log, or the commit step can be executed to obtain a "commit successful" result.

[0232] Based on the above explanation, the preparation event of a heartbeat transaction can be used to collect specific transactions into a transaction set for subsequent operations.

[0233] Optionally, all two-phase commit transactions that have commit events before the heartbeat transaction preparation event in the operation log of each storage node can be added to a transaction set.

[0234] In this context, the preparation event for each transaction in the transaction set occurs before the heartbeat transaction commit event in the operation log of each storage node targeted by that transaction.

[0235] Optionally, since the consistency of transactions that are not completed in the first moment can be ensured, the consistency of subsequent two-phase commit transactions initiated after the first moment needs to be ensured. However, the range of commit events for transactions initiated after the first moment is difficult to determine. Therefore, it can be limited to after the first moment.

[0236] Optionally, all two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event in the operation log of each storage node can be added to a transaction set.

[0237] Specifically, this can be achieved by adding all two-phase commit transactions from the operation log of each storage node that have a commit event between the corresponding position at the first moment and the heartbeat transaction preparation event to a transaction set.

[0238] Therefore, for each transaction in the transaction set, there exists a commit event in the operation log of at least one storage node between the first moment and the heartbeat transaction preparation event.

[0239] Since each transaction in the transaction set has recorded a commit event in the operation log of at least one storage node before the heartbeat transaction preparation event, the result of the transaction can be determined as "commit".

[0240] For ease of description, the above set of transactions will be referred to as the set of committable transactions.

[0241] Optionally, since the commit event of the heartbeat transaction is needed to determine the scope of the transaction preparation event, it is necessary to ensure that the heartbeat transaction of each storage node is successfully committed. This will determine that the processing result of the heartbeat transaction is "committed", and the heartbeat transaction can be added to the set of committable transactions.

[0242] After determining the set of committable transactions, the operation logs that need to be backed up can be determined for each storage node.

[0243] Specifically, this could involve determining the log pruning point for each storage node, so that the operation logs from the first point (the start time of the physical backup) to the pruning point can be backed up subsequently.

[0244] To ensure that each two-phase commit transaction in the set of committable transactions includes at least the preparation event of that transaction in the operation log of each storage node, optionally, the end point of the latest recorded preparation event in the operation log of that node among all two-phase commit transactions in the set of committable transactions for that node can be determined as the pruning point.

[0245] Specifically, the end point of the latest recorded preparation event can be the point where the preparation event ends, which is the point where the corresponding paragraph of the preparation event ends in the operation log.

[0246] This allows the preparation event to be included in the backup operation log when backing up the operation log prior to the pruning point.

[0247] Optionally, since the preparation event for each specific transaction in the set of committable transactions is before the heartbeat transaction commit event, the pruning point for each storage node will not be after the heartbeat transaction commit event, thus making it easy to determine the pruning point.

[0248] Furthermore, after all storage nodes have successfully committed the heartbeat transaction, it is necessary to ensure that the heartbeat transaction commit event has been recorded in the operation logs of all storage nodes, so as to easily determine the pruning point before the heartbeat transaction commit event.

[0249] When the pruning point for each storage node is determined, the operation log from the first backed-up point (the physical backup start time) to the pruning point can include preparation events for all two-phase commit transactions for that node in the set of committable transactions.

[0250] Correspondingly, when restoring each storage node using the backup operation logs, for transactions in the ready-to-commit state, if the transaction belongs to the set of committable transactions, the compute node can issue a commit message to complete the commit, thereby ensuring the consistency of transactions in the set of committable transactions.

[0251] Therefore, based on the above explanation, the consistency of each transaction in the set of committable transactions can be ensured by utilizing the set of committable transactions, pruning points, and backup operation logs.

[0252] For ease of understanding, such as Figure 7 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0253] It displays the operation logs of storage nodes 1-4, and shows the preparation and commit events of transactions 1-2, as well as the positions of the preparation and commit events of heartbeat transactions in the operation logs.

[0254] For ease of description, the prepare event of transaction 1 is described as P1, and the commit event of transaction 1 is described as C1. The prepare event of transaction 2 is described as P2, and the commit event of transaction 2 is described as C2. The prepare event of the heartbeat transaction is described as PH, and the commit event of the heartbeat transaction is described as CH.

[0255] Transaction 1 is for storage nodes 1-2. Transaction 2 is for storage nodes 3-4. The heartbeat transaction is for storage nodes 1-4.

[0256] exist Figure 7 In storage node 1, since C1 precedes PH, the set of committable transactions includes transaction 1. Similarly, in storage node 3, since C2 precedes PH, the set of committable transactions includes transaction 2.

[0257] The set of committable transactions also includes heartbeat transactions.

[0258] Based on the set of committable transactions, the pruning points for storage nodes 1-4 can be determined.

[0259] In storage nodes 1-2, among the transactions (transaction 1 and heartbeat transaction) for storage nodes 1-2 in the set of committable transactions (including transaction 1-2 and heartbeat transaction), the latest recorded preparation event is PH.

[0260] Therefore, the termination site corresponding to PH can be determined as the pruning site.

[0261] To facilitate the display of the pruning point, the paragraph occupied by PH in the operation log of storage node 1-2 is marked. The pruning point is the position where the paragraph occupied by PH ends.

[0262] This facilitates subsequent backups of operation logs prior to the pruning point, and also includes the PH in the backup operation logs.

[0263] In storage node 3, among the transactions (transaction 2 and heartbeat transaction) for storage node 3 in the set of committable transactions (including transactions 1-2 and heartbeat transactions), the latest recorded preparation event is PH.

[0264] Therefore, the termination site corresponding to PH can be determined as the pruning site.

[0265] In storage node 4, among the transactions (transaction 2 and heartbeat transaction) targeting storage node 4 in the set of committable transactions (including transactions 1-2 and heartbeat transactions), the latest recorded preparation event is P2.

[0266] Therefore, the end point corresponding to P2 can be determined as the pruning point.

[0267] When performing data recovery using the backup operation logs on storage nodes 1-4, the consistency of the committable transaction set (including transactions 1-2 and heartbeat transactions) can be ensured through operations.

[0268] For heartbeat transactions, after data recovery using the operation logs backed up to the pruning point, storage nodes 1-4 recover heartbeat transactions based on PH and are in a ready-to-commit state.

[0269] Since heartbeat transactions belong to the set of committable transactions, the processing result can be determined as "commit". Therefore, the compute node can send a commit message for the heartbeat transaction to storage nodes 1-4, so that the processing result of the heartbeat transaction between storage nodes 1-4 is "commit successful", thus ensuring the consistency of the heartbeat transaction.

[0270] For transaction 1, after storage nodes 1 and 2 recover using the operation logs backed up to the pruning point, transaction 1 in storage node 1 has been "committed successfully" based on P1 and C1, while transaction 1 in storage node 2 is based on P1 and is still in the state of preparing to commit.

[0271] Since transaction 1 belongs to the set of committable transactions, the processing result can be determined to be "commit". Therefore, the compute node can send a commit message for transaction 1 to the storage node 2, so that the processing result of transaction 1 is "committed successfully" between storage node 1 and storage node 2, thus ensuring the consistency of transaction 1.

[0272] Similarly, the consistency of transaction 2 can also be ensured. This will not be elaborated further here.

[0273] 6) The above operations ensure the consistency of transactions within the initial set of committable transactions. However, there are other two-phase commit transactions initiated after the initial commit, and consistency must be ensured for those as well.

[0274] The following sections will explain how to ensure consistency based on the classification of other two-phase commit transactions initiated after the first moment.

[0275] a) If the pruning point is after the PH (heartbeat transaction preparation event), then in the backup operation log, there may be a transaction commit event located between the PH and the pruning point, but the transaction is not in the set of committable transactions.

[0276] For ease of understanding, such as Figure 8 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0277] It displays the operation logs of storage nodes 1-4, and shows the preparation and commit events of transactions 2-3, as well as the positions of the preparation and commit events of heartbeat transactions in the operation logs.

[0278] For ease of description, the prepare event of transaction 2 is described as P2, and the commit event of transaction 2 is described as C2. The prepare event of transaction 3 is described as P3, and the commit event of transaction 3 is described as C3. The prepare event of the heartbeat transaction is described as PH, and the commit event of the heartbeat transaction is described as CH.

[0279] Transaction 2 is a transaction targeting storage nodes 3-4. Transaction 3 is a transaction targeting storage nodes 2-4. The heartbeat transaction is a transaction targeting storage nodes 1-4.

[0280] For determining the specific cutting points, please refer to... Figure 7 The explanation will not be repeated here.

[0281] After the pruning point is determined, the operation logs before the pruning point backed up in storage node 2 do not include P3 and C3; the operation logs before the pruning point backed up in storage node 3 only include P3 and do not include C3; the operation logs before the pruning point backed up in storage node 4 include P3 and C3.

[0282] Since C3 does not exist in the storage nodes 1-4 prior to PH, transaction 3 is not added to the set of committable transactions.

[0283] However, with the determination of the pruning point and the backup of the operation log, not all the preparation events of transaction 3 were retained between the backup operation logs of storage nodes 2 and 4. During subsequent recovery, storage node 2 did not have a backup of P3, and transaction 3 could not be restored to the "ready to commit" state based on P3. Therefore, it was impossible to use the issued commit message to restore the "commit successful" result of transaction 3 in storage node 4. As a result, the consistency of transaction 3 could not be guaranteed.

[0284] To ensure consistency of transactions outside the set of committable transactions, and between the PH and the pruning point in at least one storage node's operation log, the preparation events for each storage node targeted by such transactions can be included in the backup operation log.

[0285] For ease of description, a transaction that is outside the set of committable transactions and has a commit event between the PH and the pruning point in the operation log of at least one storage node is called a "supplementary transaction".

[0286] It should be noted that the initial pruning point is determined for each storage node by identifying the end point of the latest prepared event recorded in the node's operation log between the two-phase commit transactions for that node in the set of committable transactions.

[0287] In the initial set of committable transactions, each transaction other than the heartbeat transaction is in the operation log of at least one storage node, and there is a commit event before the heartbeat transaction preparation event.

[0288] Based on the above inference 2, if in the operation log of a storage node, after the commit event of two-phase commit transaction 1, there exists a preparation event for two-phase commit transaction 2, then it can be determined that the commit message of transaction 1 was sent earlier than the commit message of transaction 2.

[0289] Therefore, it can be determined that in the initial set of committable transactions, the commit message of each transaction other than the heartbeat transaction is sent earlier than the commit message of the heartbeat transaction.

[0290] Furthermore, for supplementary transactions, it can be determined that a commit event exists before the preparation event of the transaction corresponding to the pruning point in the initial set of committable transactions.

[0291] Furthermore, since the supplementary transaction has a commit event between the PH and the pruning point in at least one storage node's operation log, and the pruning point is not the end point of the PH, the transaction corresponding to the pruning point is not a heartbeat transaction.

[0292] Therefore, it can be determined that the commit message of the transaction corresponding to the pruning point is sent earlier than the commit message of the heartbeat transaction.

[0293] Based on the same inference 2 above, it can be determined that the commit message of the supplementary transaction is sent earlier than the commit message of the transaction corresponding to the pruning point in the initial set of committable transactions.

[0294] Therefore, the time sequence can be determined as follows: the time when the commit message of the supplementary transaction is sent, the time when the commit message of the transaction corresponding to the pruning point is sent, and the time when the commit message of the heartbeat transaction is sent.

[0295] In other words, the commit message of a supplementary transaction is sent earlier than the commit message of a heartbeat transaction.

[0296] Furthermore, based on the above inference 1: for each two-phase commit transaction, in each storage node targeted by the transaction, the transaction preparation event is recorded before the time when the compute node issues the transaction commit information, and the transaction commit event is recorded after the time when the compute node issues the transaction commit information.

[0297] It can be determined that in each storage node targeted by the supplementary transaction, the preparation event of the supplementary transaction occurs before the commit message of the supplementary transaction is sent.

[0298] It can also be determined that the commit event of the heartbeat transaction for each storage node (that is, all storage nodes in the database) is after the heartbeat transaction commit message is sent.

[0299] Therefore, it can be determined that in each storage node targeted by the supplementary transaction, the preparation event of the supplementary transaction occurs before the commit event of the heartbeat transaction.

[0300] For ease of understanding, such as Figure 9 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0301] Figure 9 For a detailed explanation, please refer to [link / reference]. Figure 8 Explanation.

[0302] Based on P3-C3-P2 in storage node 4, it can be determined that the commit message of transaction 3 was sent earlier than the commit message of transaction 2.

[0303] Furthermore, based on P2-C2-PH-CH in storage node 3, it can be determined that the commit message of transaction 2 was sent earlier than the commit message of the heartbeat transaction.

[0304] Therefore, it can be determined that in storage nodes 2-4, P3 will not exceed the "time when transaction 3 commit message is sent", and thus will not exceed the "time when transaction 2 commit message is sent", nor will it exceed the "time when heartbeat transaction commit message is sent", which is all before CH.

[0305] In other words, in storage nodes 2-4, P3 is always before CH.

[0306] If it is proven that the preparation events of the supplementary transaction are all before CH, in order to ensure the consistency of the supplementary transaction, the supplementary transaction can also be added to the set of committable transactions, using the same pruning point determination method, so that each storage node targeted by the supplementary transaction backs up the preparation events of the supplementary transaction.

[0307] Of course, as supplementary transactions are added to the set of committable transactions, the determined pruning point may be moved further back, resulting in new supplementary transactions.

[0308] Since the commit message of each supplementary transaction in the set of committable transactions is sent earlier than the commit message of the heartbeat transaction, even after adding supplementary transactions, the set of committable transactions still meets the requirement that "the commit message of each transaction other than the heartbeat transaction in the set of committable transactions is sent earlier than the commit message of the heartbeat transaction".

[0309] Furthermore, based on the same reasoning process, it can be proven that the preparation events for new supplementary transactions also occur before the heartbeat transaction commit events.

[0310] Therefore, the set of committable transactions and the pruning point of each storage node can be updated cyclically until no new supplementary transactions appear.

[0311] In this embodiment, by cyclically updating the set of committable transactions and the pruning point of each storage node, transactions with commit events between the PH of each storage node and the pruning point can be added to the set of committable transactions. This ensures that the operation log before the pruning point includes the preparation events of all transactions for that node in the current set of committable transactions.

[0312] This ensures that for transactions where a commit event exists between the PH of each storage node and the pruning point, the operation log of the transaction is included in the backup of all the storage nodes targeted, thus ensuring that the transaction can be restored to the "ready to commit" state during subsequent recovery.

[0313] Furthermore, since the transaction can be confirmed as "committed", the compute node can issue a commit message for the transaction, ensuring that the transaction result is "committed successfully" across all storage nodes to which the transaction is applied.

[0314] Therefore, it can be ensured that there is transaction consistency between the PH of each storage node and the pruning point, with commit events occurring between them.

[0315] Optionally, if all storage nodes successfully commit heartbeat transactions, for each storage node, all two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event can be added to the current set of committable transactions; and the heartbeat transaction can be added to the current set of committable transactions.

[0316] For each storage node, the end point corresponding to the latest prepared event recorded in the node's operation log among the two-phase commit transactions in the current set of committable transactions for that node can be determined as the current pruning point.

[0317] The current set of committable transactions and the current pruning point of each storage node are updated cyclically until all two-phase commit transactions that have a commit event with the current pruning point at the first moment for each storage node are included in the current set of committable transactions.

[0318] The current pruning point of each storage node is the end point of the latest prepared event recorded in the node's operation log among all two-phase commit transactions for that node in the current set of committable transactions.

[0319] This embodiment can include all two-phase commit transactions of each storage node that have commit events with the current pruning point at the first moment in the set of currently committable transactions after the cyclic update is completed.

[0320] Since the current pruning point is also determined based on the current set of committable transactions, it can be ensured that the operation logs before the current pruning point on each storage node include the preparation events of each transaction in the current set of committable transactions.

[0321] This ensures that each transaction in the current set of committable transactions after the cyclic update is completed can at least be restored to the "ready to commit" state during recovery.

[0322] Furthermore, it can be determined that the processing result of each transaction in the current set of committable transactions is "committed", and then the compute node can issue the corresponding commit message for transactions that have not yet been successfully committed.

[0323] Therefore, consistency of each transaction in the current set of committable transactions can be ensured after the cyclic update is completed.

[0324] For ease of understanding, such as Figure 10 The diagram shown is a schematic representation of another operation log backup method provided in the embodiments of this specification.

[0325] Figure 10 For a detailed explanation, please refer to [link / reference]. Figure 9 Explanation.

[0326] Through cyclical updates, it can be determined that the current set of committable transactions includes transaction 2, transaction 3, and the heartbeat transaction.

[0327] The current pruning site can be updated to the current site.

[0328] Specifically, the pruning point of storage node 2 can be updated to the end point of P3.

[0329] based on Figure 10 By backing up the operation log at the current pruning point, the preparation events of transactions 2, 3, and the heartbeat transaction can all be included in the backed-up log. This ensures the consistency of transactions 2, 3, and the heartbeat transaction.

[0330] b) Using the pruning point as a boundary, the above explains how transactions with commit events before the pruning point of at least one storage node can ensure consistency.

[0331] For two-phase commit transactions outside the current set of committable transactions that have completed a cyclic update, if they are in a completed state after being restored based on the backup operation logs—in other words, if these transactions were completed before the pruning point—then the processing results of these transactions are consistent, and no additional operations are needed to ensure transaction consistency.

[0332] For two-phase commit transactions outside the current set of committable transactions after the cyclic update has ended, if they are still in an incomplete state after being restored based on the backup operation log, the compute node can directly issue the corresponding rollback message to ensure the consistency of these transactions.

[0333] Specifically, unfinished tasks can be categorized into two types.

[0334] (1) There are no commit or rollback events in the operation logs of each storage node backed up by the firm (that is, before the pruning point).

[0335] In this scenario, there may or may not be a preparation event in the operation logs of each storage node backup performed by the firm (i.e., before the pruning point).

[0336] Since the outcome of the transaction cannot be determined, a rollback can be used to ensure that the transaction will be successfully rolled back during recovery, thus ensuring consistency.

[0337] (2) A rollback event exists in the operation log of at least one storage node backed up by the firm (i.e., before the pruning point).

[0338] In this case, the transaction has not been successfully rolled back to all the storage nodes it is targeting, so it is still in an incomplete state.

[0339] Since the outcome of the transaction has been determined to be "rollback", the rollback method can be used to ensure that the outcome of the transaction is "rollback successful" during recovery, thereby ensuring consistency.

[0340] In summary, by using the pruning point as the classification basis, consistency can be ensured for transactions initiated before the pruning point through the above embodiments.

[0341] Specifically, for transactions initiated before the pruning point, they may have been completed (all successfully committed or all successfully rolled back) before the pruning point, or they may not have been fully committed (incomplete), or they may not have been fully rolled back (incomplete), or they may not have had any processing results (incomplete).

[0342] Specifically, these can be summarized into the following four situations.

[0343] (1) Consistency can be ensured for transactions that have a commit event before the pruning point of at least one storage node.

[0344] Specifically, consistency can be ensured for transactions that have a commit event between the first moment of at least one storage node and the pruning point.

[0345] In other words, for transactions that are all successfully committed before the pruning point (which are already completed transactions), or transactions that are not all successfully committed at the time of commit, consistency can be ensured by issuing a commit message.

[0346] (2) For transactions that are completed before the pruning point (either all committed successfully or all rolled back successfully), consistency can be ensured through recovery based on backup operation logs.

[0347] (3) For transactions that were not fully rolled back before the pruning point, consistency can be ensured by sending a rollback message.

[0348] (4) For transactions that are not completed before the pruning point and whose processing result is uncertain (commit or rollback), consistency can be ensured by sending a rollback message.

[0349] Therefore, for two-phase commit transactions initiated between the first moment and the pruning point, the above embodiments can ensure transaction consistency.

[0350] IV. Additional Notes.

[0351] It should be noted that the pruning point was not directly determined by the commit event of the heartbeat transaction.

[0352] This is because if the commit event of the heartbeat transaction is used as the pruning point, then for supplementary transactions, it is impossible to determine whether the preparation event of the supplementary transaction is before the commit event of the heartbeat transaction.

[0353] In addition, when backing up the operation log, back up the continuous operation log directly, that is, the operation log from the first point to the current pruning point. This is because during subsequent recovery, the same data in the database may be continuously modified by different transactions. If only the log part required to maintain transaction consistency is backed up, it may cause the data in the database to be unable to be restored to the original modification result.

[0354] The following is a detailed explanation of each step.

[0355] 1. S101: Perform a physical backup of all stored data in each storage node, and determine the operation log point corresponding to the start time of the physical backup in that storage node as the first point.

[0356] This method does not limit the physical backup method. Optionally, the database storage data files of each storage node can be directly copied.

[0357] In this embodiment, since no logical backup is required and no data needs to be queried through SQL statements, the consumption of database resources can be reduced, thus improving backup efficiency.

[0358] In one alternative embodiment, physical backups of all stored data on different storage nodes can be performed simultaneously, making it easier to determine the timing of the physical backups.

[0359] Of course, different storage nodes can be physically backed up at different times.

[0360] Regarding the operation logs in the storage nodes, optionally, they may include database logical logs, i.e., binlogs. Optionally, the operation logs may also include transaction logs.

[0361] 2. S102: If it is determined that all two-phase commit transactions in the incomplete transaction set have been completed, initiate a heartbeat transaction for all storage nodes.

[0362] Optionally, the set of incomplete transactions may include all two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0363] In one alternative embodiment, a set of incomplete transactions may be obtained in advance.

[0364] Optionally, the database may include at least one compute node. The set of incomplete transactions can be obtained from the compute node. Different compute nodes can share the set of incomplete transactions.

[0365] This method does not limit the first moment, as long as the first moment is after the physical backup, that is, after the moment corresponding to the first point.

[0366] Optionally, the first moment can be determined after all storage nodes have completed physical backups. Specifically, the first moment can be the latest time when physical backups across all storage nodes are completed. Alternatively, the first moment can also be a preset time after the time when the last storage node's physical backup is completed.

[0367] Of course, the first moment can also be the moment when the physical backup is not yet fully completed.

[0368] In one optional embodiment, the completion of a two-phase commit transaction may include: each storage node to which the two-phase commit transaction is targeted successfully rolling back the two-phase commit transaction; or each storage node to which the two-phase commit transaction is targeted successfully committing the two-phase commit transaction.

[0369] Optionally, after waiting for all two-phase commit transactions in the incomplete transaction set to complete, a heartbeat transaction can be initiated for all storage nodes.

[0370] This method does not limit the timing of initiating a heartbeat transaction, as long as it is after all two-phase commit transactions in the incomplete transaction set have been completed.

[0371] Alternatively, a heartbeat transaction can be initiated for all storage nodes when it is determined that all two-phase commit transactions in the incomplete transaction set have been completed.

[0372] Optionally, a heartbeat transaction can be a two-phase commit transaction for each storage node in the database.

[0373] Since heartbeat transactions are primarily used to mark all storage nodes, the data modified by a heartbeat transaction does not necessarily have to be user data, thus reducing the impact on user data.

[0374] Alternatively, a heartbeat transaction can be a transaction that modifies the system tables in each storage node.

[0375] Optionally, heartbeat transactions can be initiated by compute nodes for all storage nodes.

[0376] Optionally, if this method is executed on a compute node in the database, the heartbeat transaction can be initiated directly by that compute node.

[0377] Optionally, when this method is executed on a device outside the database, that device can control the compute nodes to initiate heartbeat transactions. Specifically, the management device can control the compute nodes to initiate heartbeat transactions for all storage nodes in the database.

[0378] 3. S103: If all storage nodes successfully commit heartbeat transactions, for each storage node, add all two-phase commit transactions that have commit events between the first moment and the heartbeat transaction preparation event to the current set of committable transactions; add the heartbeat transaction to the current set of committable transactions.

[0379] S103 is to determine the initial set of currently committable transactions to facilitate subsequent cyclic updates.

[0380] Based on the above analysis of the principles of transaction consistency, if all storage nodes successfully commit the heartbeat transaction, it can be ensured that the heartbeat transaction commit event has been recorded in the operation log of each storage node.

[0381] This ensures that all transaction preparation events in the current set of committable transactions have been recorded in the operation log of each storage node, making it easier to determine the current pruning point of each storage node later.

[0382] It is important to emphasize that, because the commit event of the heartbeat transaction is needed to help determine the current pruning point of each storage node, the result of the heartbeat transaction is a successful commit, which requires all storage nodes to successfully commit the heartbeat transaction.

[0383] Furthermore, by adding heartbeat transactions to the initial set of currently committable transactions, the current pruning point of the storage node updated in each cycle is at or after the end point of the heartbeat transaction preparation event.

[0384] Even at the initial current pruning point of a storage node, since the initial set of currently committable transactions includes heartbeat transactions, the initial current pruning point is at or after the end point of the heartbeat transaction preparation event.

[0385] Since the heartbeat transaction is initiated on all storage nodes after confirming that all two-phase commit transactions in the incomplete transaction set have been completed, all two-phase commit transactions in the incomplete transaction set are completed before the heartbeat transaction preparation event in the operation log.

[0386] Since the pruning points are all at or after the end point of the heartbeat transaction preparation event, it can be ensured that the operation log before the current pruning point in the backup includes all relevant records (commit events or rollback events) of the completion of the two-phase commit transactions in the incomplete transaction set.

[0387] This ensures that during subsequent recovery, all two-phase commit transactions in the incomplete transaction set can be restored to a completed state based on the backed-up operation logs, thus ensuring the consistency of these transactions.

[0388] In an optional embodiment, since it has been determined through S102 that all two-phase commit transactions in the incomplete transaction set have been completed, the relevant operation logs can be backed up to ensure transaction consistency. Therefore, S103 ensures transaction consistency for two-phase commit transactions initiated after the first moment.

[0389] Therefore, for each storage node, all two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event need to be added to the current set of committable transactions.

[0390] Alternatively, all two-phase commit transactions that have a commit event between the first point and the heartbeat transaction preparation event can be added to the current set of committable transactions.

[0391] Alternatively, all two-phase commit transactions that have a commit event prior to the heartbeat transaction preparation event can be added to the current set of committable transactions.

[0392] In an alternative embodiment, a portion of the operation logs of each storage node can be copied out to determine the current set of committable transactions, and can also be used for subsequent cyclic updates.

[0393] Optionally, for each storage node, a second point that is not before the end point of the heartbeat transaction commit event can be determined.

[0394] The preset operation logs from all storage nodes can then be aggregated to determine the current set of committable transactions. Optionally, the preset operation logs may include at least the operation logs between the first and second points.

[0395] Of course, optionally, the preset operation log may also include: the operation log between the first point and the second point.

[0396] Since the second point is not before the end point of the heartbeat transaction commit event, that is, at or after the end point of the heartbeat transaction commit event, it is convenient to determine the current pruning point of each storage node subsequently.

[0397] Optionally, for each storage node, all two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event are added to the current set of committable transactions. This may include: a preset operation log for each storage node, which includes all two-phase commit transactions that have a commit event before the heartbeat transaction preparation event. The preset operation log includes: the operation log between the first moment and the second moment.

[0398] This method does not limit the way the second point is determined in the operation log of each storage node. As long as the determined second point is not before the end point of the heartbeat transaction commit event, it is sufficient to ensure that the preparation event of each transaction in the set of currently committable transactions determined later is before the second point.

[0399] Optionally, the second time point can be determined based on a specific moment. Specifically, this second moment can be determined after the heartbeat transaction in each storage node has been successfully committed. The second time point can be after the heartbeat transaction in each storage node has been successfully committed.

[0400] Then the operation log position corresponding to the second time point can be determined as the second position.

[0401] Determining the second site based on time can improve the efficiency of determining the second site, thereby improving backup efficiency.

[0402] Alternatively, the end point of the heartbeat transaction commit event can be directly determined as the second point.

[0403] In summary, optionally, for each storage node, determining the second position that is not before the end position of the heartbeat transaction commit event may include: for each storage node, determining the end position of the heartbeat transaction commit event as the second position; or, if all storage nodes have successfully committed heartbeat transactions, determining the position corresponding to the second moment in the operation log of each storage node as the second position; the second moment is not earlier than the latest recorded end position of the heartbeat transaction commit event among all storage nodes.

[0404] Optionally, the second moment can be the moment corresponding to the latest heartbeat transaction among all storage nodes, or it can be later than the moment corresponding to the latest heartbeat transaction among all storage nodes.

[0405] 4. S104: Circularly update the current set of committable transactions and the current pruning point of each storage node until all two-phase commit transactions that have a commit event with the current pruning point at the first moment for each storage node are included in the current set of committable transactions.

[0406] Optionally, the current pruning point of each storage node is the end point of the latest prepared event recorded in the node's operation log among all two-phase commit transactions for that node in the current set of committable transactions.

[0407] In one alternative embodiment, based on the above analysis of the principles of transaction consistency, it is necessary to perform cyclic updates on the current set of committable transactions and the current pruning point of each storage node.

[0408] During the cyclic update process, the current pruning point of each storage node needs to be determined using the above method based on the current set of committable transactions.

[0409] Furthermore, it is necessary to determine whether there are any new supplementary transactions based on the current pruning point of each storage node. These transactions are those outside the current set of committable transactions and have commit events between the heartbeat transaction preparation event and the current pruning point.

[0410] If a new supplementary transaction exists, it can be added to the current set of committable transactions for updating, thus executing the next round of cyclic updates.

[0411] If no new supplementary transactions exist, it can be said that for each storage node, all two-phase commit transactions that have commit events between the first moment and the current pruning point are included in the current set of committable transactions.

[0412] Based on the above analysis of the principles of transaction consistency, it can be determined that for transactions that have committed events before the current pruning point of at least one storage node, transaction consistency can be ensured through subsequent operations.

[0413] Of course, for other transactions, consistency can also be ensured through subsequent operations.

[0414] It should be noted that the current pruning point of each storage node will not exceed the heartbeat transaction commit event. For a detailed proof, please refer to the above analysis of the principle of transaction consistency.

[0415] The number of two-phase commit transactions between the first moment and the heartbeat commit event is finite. Therefore, the number of two-phase commit transactions between the first moment and the current pruning point is also finite.

[0416] In other words, the above-mentioned cyclic update can tend to converge, and the cyclic update can end after a finite number of iterations.

[0417] Optionally, cyclically updating the current set of committable transactions and the current pruning point of each storage node may include: cyclically executing the following steps until no new transactions are added to the current set of committable transactions: for each storage node, adding all two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point to the current set of committable transactions; for each storage node, determining the end point of the latest recorded preparation event in the node's operation log among all two-phase commit transactions for that node in the current set of committable transactions as the current pruning point.

[0418] If no new transactions are added to the current set of committable transactions, it means that for each storage node, all two-phase commit transactions that have a commit event between the first moment and the current pruning point are included in the current set of committable transactions.

[0419] This method does not limit the specific method for determining the current pruning point for each storage node.

[0420] Optionally, the process can traverse backwards from the heartbeat transaction preparation event, or it can traverse forwards from the heartbeat transaction commit event. Alternatively, it can determine all transaction preparation events between the heartbeat transaction preparation event and the heartbeat transaction commit event, as well as their corresponding order, to determine the current pruning point.

[0421] Optionally, the method for determining the current pruning point of each storage node may include: for each storage node, starting from the heartbeat transaction commit event, traversing backward through all preparation events in the operation log; if the preparation event currently being traversed corresponds to a two-phase commit transaction and belongs to the current set of committable transactions, determining the end point of the preparation event of that transaction as the current pruning point of that storage node.

[0422] 5. S105: After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in each storage node to the current pruning point.

[0423] Based on the above analysis of transaction consistency, in addition to physically backing up all the data in each storage node, it is also necessary to back up the current set of committable transactions and the operation log from the first point to the current pruning point in each storage node after the loop ends.

[0424] Optionally, the operation log from the first point to the clipping point can be cut and copied from the operation log.

[0425] Furthermore, in an optional embodiment, based on the above analysis regarding transaction consistency, for the set of committable transactions that includes heartbeat transactions in the above embodiments, apart from heartbeat transactions, all other transactions have commit events in the operation log of at least one storage node before the current pruning point.

[0426] For heartbeat transactions, the heartbeat transaction commit event on each storage node occurs after the current pruning point.

[0427] Therefore, heartbeat transactions may not be added to the set of committable transactions.

[0428] It should be noted that, on the one hand, because it is necessary to ensure that the operation logs of all two-phase commit transactions in the incomplete transaction set are backed up, the current pruning point must be at or after the end point of the heartbeat transaction preparation event. This is because the heartbeat transaction is initiated after all two-phase transactions in the incomplete transaction set have been completed. In other words, the heartbeat transaction will only be initiated and the heartbeat transaction preparation event will only be recorded after all two-phase transactions in the incomplete transaction set have been completed.

[0429] On the other hand, although the result of a heartbeat transaction in the database is a commit, the commit event of a heartbeat transaction cannot be backed up. Therefore, when restoring the database stored data, the heartbeat transaction can be rolled back to ensure its consistency.

[0430] Optionally, such as Figure 11 The diagram shown is a flowchart illustrating another data backup method provided in the embodiments of this specification.

[0431] This method is not limited to any particular entity; it can be applied to databases to back up stored data.

[0432] A database may include at least two storage nodes; the stored data in the database may be distributed across at least two storage nodes.

[0433] This method may include the following steps.

[0434] S201: Perform a physical backup of all stored data in each storage node, and determine the operation log position corresponding to the start time of the physical backup in that storage node as the first point.

[0435] S202: If it is determined that all two-phase commit transactions in the incomplete transaction set have been completed, initiate a heartbeat transaction for all storage nodes.

[0436] Optionally, the set of incomplete transactions may include all two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0437] S203: If all storage nodes successfully commit heartbeat transactions, for each storage node, add all two-phase commit transactions that have commit events between the first moment and the heartbeat transaction preparation event to the current set of committable transactions; determine the end point of the heartbeat transaction preparation event as the current pruning point for each storage node.

[0438] S204: Circularly update the current set of committable transactions and the current pruning point of each storage node until, for each storage node, all two-phase commit transactions that have a commit event with the current pruning point at the first moment are included in the current set of committable transactions.

[0439] Optionally, the current pruning point of each storage node is the latest recorded point between the candidate point and the end point of the heartbeat transaction preparation event; the candidate point is the end point of the latest recorded preparation event in the operation log of the node among all two-phase commit transactions for that node in the current set of committable transactions.

[0440] S205: After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in each storage node to the current pruning point.

[0441] It should be noted that the initial current pruning point can be determined as the end point of the heartbeat transaction preparation event.

[0442] When updating the current pruning point later, if the candidate point is before the end point of the heartbeat transaction preparation event, the operation log before the candidate point may not include the logs of all transactions in the incomplete transaction set being completed. Therefore, it is necessary to determine the latest recorded point between the candidate point and the end point of the heartbeat transaction preparation event as the current pruning point.

[0443] For an explanation of this method flow, please refer to the explanation of the method flow above.

[0444] Based on the above-described data backup method embodiments, this specification also provides a data recovery method embodiment based on the above-described backup method embodiments.

[0445] like Figure 12 The diagram shown is a flowchart illustrating a data recovery method provided in an embodiment of this specification.

[0446] The execution entity of this method is not limited; it can be a database, a computing node within the database, or a management device outside the database. See above for a detailed explanation.

[0447] This may include the following steps.

[0448] S301: For any storage node, obtain the full physical backup storage data and the operation log from the first backup point to the current pruning point, and restore the storage node based on the obtained full storage data and the obtained operation log.

[0449] Optionally, since the operation log includes the operations of the storage nodes, the storage nodes can execute the operations in the backup operation log one by one for the full physical backup of the stored data, thereby achieving data recovery.

[0450] S302: Get the current set of committable transactions in the backup.

[0451] S303: If the storage node has been recovered, for each two-phase commit transaction in the ready-to-commit state in the storage node, if the transaction belongs to the current set of committable transactions, commit the transaction; if the transaction does not belong to the current set of committable transactions, roll back the transaction.

[0452] Optionally, the execution order between the above steps S301 and S302 is not limited; they can be executed in parallel or sequentially.

[0453] By utilizing the current set of committable transactions, all transactions with commit events prior to the current pruning point in each storage node are committed, ensuring that the processing results of these transactions can be restored to a successful commit, thereby ensuring consistency.

[0454] In other words, consistency can be ensured by committing transactions that have a commit event before the current pruning point on at least one storage node.

[0455] For other transactions, that is, transactions for which there are no commit events before the current pruning point across all storage nodes, the backup operation logs for these transactions will not contain commit events for these transactions across all storage nodes targeted by these transactions.

[0456] Therefore, based on the backup operation logs, the processing result of such transactions is uncertain or may result in a rollback.

[0457] For this type of transaction, consistency can be ensured through rollback.

[0458] In summary, this method can ensure the transactional consistency of the restored database storage data based on the above-described data backup method embodiments.

[0459] It should be noted that for transactions that do not have commit events before the current pruning point in all storage nodes, even if there are preparation events for such transactions in the backup operation log, during recovery, only the data needs to be written to the log (e.g., the redo log), without affecting the recovery of the database stored data.

[0460] Even if such transactions are rolled back later, it will not affect the recovery of the database stored data.

[0461] In an alternative embodiment, if the storage node has completed recovery, for each incomplete two-phase commit transaction in the storage node, the transaction can be committed if it belongs to the current set of committable transactions, and rolled back if it does not belong to the current set of committable transactions.

[0462] Incomplete two-phase commit transactions can include: two-phase commit transactions with uncertain processing results, two-phase commit transactions that are not fully committed, and two-phase commit transactions that are not fully rolled back.

[0463] Since transactions in the current set of committable transactions have committed events before the current pruning point on at least one storage node, and the processing result can be determined as a commit, "two-phase commit transactions with uncertain processing results" are not in the current set of committable transactions, thus consistency can be ensured by rolling back transactions.

[0464] For a two-phase commit transaction that is not fully committed, there must be a commit event before the current pruning point of at least one storage node to determine that the processing result is a commit. Therefore, the "two-phase commit transaction that is not fully committed" is in the current set of committable transactions, and consistency can be ensured by committing the transaction.

[0465] For two-phase commit transactions that do not fully roll back, since the transaction's processing result has already been determined to be a rollback, consistency can be ensured by rolling back the transaction.

[0466] Of course, determining whether a transaction is incomplete usually requires considering multiple storage nodes. Therefore, in one optional embodiment, data recovery can be performed on multiple storage nodes in the database, specifically on all storage nodes in the database, so that incomplete transactions in the database can be identified after the recovery is completed.

[0467] Optionally, for each storage node, obtain the full physical backup storage data and the operation log from the first backup point to the current pruning point, and restore the storage node based on the obtained full storage data and the obtained operation log.

[0468] Get the backup of the current set of committable transactions.

[0469] Once each storage node has completed recovery, for each incomplete two-phase commit transaction in the database, if the transaction belongs to the current set of committable transactions, the transaction is committed; otherwise, the transaction is rolled back.

[0470] This method does not limit the specific implementation of transaction commit and rollback.

[0471] Optionally, a transaction can be committed by the compute node sending a transaction commit message, or a transaction can be rolled back by the compute node sending a transaction rollback message.

[0472] Alternatively, transaction commit and rollback can be implemented directly without going through the computing node. Specifically, for transactions that need to be committed, the data processing results in the log can be written to the database, specifically to the database data stored on the storage node; for transactions that need to be rolled back, rollback events can be recorded in the log.

[0473] In an alternative embodiment, when a two-phase commit transaction includes an XA transaction, XARECOVER can be used to obtain a list of dangling transactions, which is each two-phase commit transaction in the ready-to-commit state on the storage node.

[0474] You can also use XA COMMIT to commit a two-phase commit transaction, and use XA ROLLBACK to roll back the transaction.

[0475] The above method describes a method for restoring stored data for a single storage node. For multiple storage nodes, the database data stored on each storage node can be restored using the above method, thereby restoring the complete database.

[0476] Based on the above backup method implementation, database recovery can be achieved through simple operations, thereby improving recovery efficiency and ensuring transaction consistency.

[0477] Based on the above data backup method embodiments and corresponding recovery method embodiments, since physical backup is performed directly on all stored data in the storage node, and the backup is of operation logs, no query operations are performed on the data in the database, thus reducing the impact on the business running on the database.

[0478] Based on the above backup method embodiments, by determining a preset operation log for aggregation, instead of the full operation log, the amount of data that needs to be collaboratively determined to make up the set of committable transactions is effectively controlled, and the cost of collaboration is reduced.

[0479] Furthermore, each storage node can perform backup operations in parallel, effectively improving the concurrency of the backup process and increasing the overall efficiency of database backup.

[0480] Based on the above-described recovery method embodiments, each storage node can also perform recovery operations in parallel without interaction or coordination, effectively improving the concurrency of the recovery process and increasing the overall efficiency of database data recovery.

[0481] Therefore, the above method implementation provides a consistent, full, physical backup process and corresponding recovery process for the database, with minimal impact on the database's own business operations.

[0482] The backup and restore process can support most distributed transactions implemented based on the MySQL XA interface, namely XA transactions.

[0483] Furthermore, the backup and recovery process is efficient, low-cost, and requires minimal coordination. Specifically, this is reflected in the relatively low deterministic cost of the set of committable transactions.

[0484] In addition, this specification also provides an embodiment of another data backup method.

[0485] In this method embodiment, the consistency of some transactions after recovery can be ensured.

[0486] like Figure 13 The diagram shown is a flowchart illustrating another data backup method provided in the embodiments of this specification.

[0487] This method is not limited to any particular entity; it can be applied to databases to back up stored data. A database can include at least two storage nodes.

[0488] Optionally, different storage nodes in the database can store the stored data in the database in a distributed manner.

[0489] The method process may include the following steps.

[0490] S401: Perform physical backup of the data stored in the storage node, and determine the operation log position corresponding to the start time of the physical backup in the storage node as the first point.

[0491] S402: Construct a set of committable transactions and determine the pruning point for the storage nodes.

[0492] Optionally, the set of committable transactions may include two-phase commit transactions in which commit events exist between the first point and the pruning point.

[0493] Optionally, the pruning point of a storage node may include the latest recorded preparation event end point in the operation log of the storage node between two-phase commit transactions for that storage node in the set of committable transactions.

[0494] S403: Backup the set of committable transactions, and the operation log from the first point to the pruning point in the storage node.

[0495] Optionally, the storage node can specifically be each storage node in the database.

[0496] Optionally, the stored data in the storage node may specifically include all the stored data in the storage node.

[0497] Optionally, the set of committable transactions may include all or part of two-phase commit transactions where commit events exist between the first point and the pruning point.

[0498] Optionally, the set of committable transactions may include two-phase commit transactions in which commit events exist between the first point and the pruning point of all or some of the storage nodes.

[0499] In this method, a set of committable transactions can be constructed, and the latest prepared event end point recorded in the operation log of the storage node between the two-phase commit transactions in the set of committable transactions for that storage node can be determined as the pruning point. This ensures that the backup log includes at least the prepared events of the transactions in the set of committable transactions, so that the consistency of the transactions in the set of committable transactions can be ensured through the commit operation when restoring data later.

[0500] Specifically, this could be done by including, at least in the backup log, the preparation events for each transaction in the set of committable transactions.

[0501] The specific principles for ensuring transaction consistency within a set of committable transactions can be found in the explanation above.

[0502] In one optional embodiment, determining the pruning point of the storage node may include: determining the pruning point of the storage node after the operation log point corresponding to the preset transaction completion time.

[0503] Optionally, the preset transaction completion time may include the time when the two-phase commit transactions in the incomplete transaction set are completed.

[0504] Optionally, the set of incomplete transactions may include incomplete two-phase commit transactions in the database at the moment immediately following the physical backup.

[0505] In this embodiment, a pruning point can be set after the operation log point corresponding to the preset transaction completion time. This ensures that the backed-up log includes at least the events required for the completion of transactions in the incomplete transaction set. This facilitates the restoration of data based on the backed-up log and ensures the completion of transactions in the incomplete transaction set, thereby ensuring the consistency of transactions in the incomplete transaction set.

[0506] Specifically, this could be included in the backup logs, at least in the events required for the completion of each transaction in the set of incomplete transactions.

[0507] The specific principles for ensuring transaction consistency within an incomplete transaction set can be found in the explanation above.

[0508] Optionally, the set of incomplete transactions may include all or part of the two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0509] Optionally, the preset transaction completion time may include the time when all or part of the two-phase commit transactions in the incomplete transaction set are completed.

[0510] In one alternative embodiment, constructing a set of committable transactions and determining the pruning point of the storage node may include the following steps.

[0511] If the two-phase commit transactions in the incomplete transaction set are confirmed to be complete, a heartbeat transaction is initiated for the storage node.

[0512] The set of incomplete transactions may include two-phase commit transactions that were not completed in the database immediately after the physical backup.

[0513] If the heartbeat transaction is successfully committed on the storage node, for the storage node, the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event is added to the current set of committable transactions; the heartbeat transaction is added to the current set of committable transactions.

[0514] The current set of committable transactions and the current pruning point of the storage node are updated cyclically. A two-phase commit transaction is confirmed to exist between the storage node at the first moment and the current pruning point, and is included in the current set of committable transactions.

[0515] The current pruning point of a storage node may include the latest prepared event end point recorded in the operation log of the storage node between two-phase commit transactions for that storage node in the current set of committable transactions.

[0516] Optionally, a heartbeat transaction can be initiated for the storage nodes, specifically a heartbeat transaction can be initiated for all or some of the storage nodes.

[0517] Optionally, the successful submission of heartbeat transactions by storage nodes may include the successful submission of heartbeat transactions by all or some storage nodes.

[0518] Optionally, for all or some storage nodes, two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event can be added to the current set of committable transactions.

[0519] Optionally, for storage nodes, all or part of two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event can be added to the current set of committable transactions.

[0520] Optionally, the current pruning point of all or some of the storage nodes can be updated cyclically.

[0521] Optionally, it can be confirmed that there are two-phase commit transactions between all or some storage nodes and the current pruning point at the first moment, and these transactions are included in the current set of committable transactions.

[0522] Optionally, it can be confirmed that there are all or part of a two-phase commit transaction between the storage node at the first moment and the current pruning point, and that the commit event is included in the current set of committable transactions.

[0523] Optionally, the current pruning point of a storage node may include the latest recorded preparation event end point in the operation log of the storage node between all or part of the two-phase commit transactions for that storage node in the current set of committable transactions.

[0524] In this embodiment, by setting the heartbeat transaction, the pruning point of the storage node can be determined to be at or after the end point of the heartbeat transaction preparation event, thereby ensuring that the backup log includes at least the events required for the completion of transactions in the incomplete transaction set. Furthermore, the pruning point of the storage node can also be determined to be before the heartbeat transaction commit event.

[0525] The specific principles can be found in the explanation above.

[0526] Optionally, backing up the set of committable transactions and the operation log from the first point to the pruning point in the storage node may include: after the loop ends, backing up the current set of committable transactions and the operation log from the first point to the current pruning point in the storage node.

[0527] Optionally, cyclically updating the current pruning point of the current set of committable transactions and the storage node may include:

[0528] Repeat the following steps until no new transactions are added to the current set of committable transactions:

[0529] For storage nodes, two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point will be added to the set of currently committable transactions.

[0530] For a storage node, the latest recorded preparation event end point in the operation log of that storage node among the two-phase commit transactions in the current set of committable transactions is determined as the current pruning point.

[0531] Optionally, the above steps may be performed on one or more storage nodes.

[0532] Optionally, for all or some storage nodes, two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point can be added to the current set of committable transactions.

[0533] Optionally, for storage nodes, all or part of two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point can be added to the current set of committable transactions.

[0534] Optionally, for all or some storage nodes, the latest recorded preparation event end point in the operation log of the storage node between two-phase commit transactions in the current set of committable transactions for that storage node can be determined as the current pruning point.

[0535] Optionally, for a storage node, the latest recorded end point of the preparation event in the operation log of that storage node can be determined as the current pruning point among all or part of the two-phase commit transactions for that storage node in the current set of committable transactions.

[0536] Optionally, the method for determining the current pruning point of a storage node may include:

[0537] For storage nodes, starting from the heartbeat transaction commit event, traverse backwards through the preparation events in the operation log;

[0538] If the preparation event currently being traversed corresponds to a two-phase commit transaction and belongs to the current set of committable transactions, then the end point of the preparation event for that transaction is determined as the current pruning point of that storage node.

[0539] Optionally, for all or some storage nodes, the preparation events in the operation log can be traversed backward from the heartbeat transaction commit event.

[0540] Optionally, for a storage node, a second point that is not before the end point of the heartbeat transaction commit event can be determined;

[0541] Optionally, for storage nodes, two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event are added to the current set of committable transactions, which may include:

[0542] For the preset operation log of the storage node, two-phase commit transactions that have a commit event before the heartbeat transaction preparation event will be added to the current set of committable transactions; the preset operation log may include: the operation log between the first moment and the second point.

[0543] A two-phase commit transaction confirming the existence of a commit event between the storage node at the first moment and the current pruning point, included in the current set of committable transactions, may include:

[0544] Confirm that the two-phase commit transactions with commit events before the current pruning point in the preset operation log of the storage node are included in the current set of committable transactions.

[0545] Optionally, for a storage node, determining a second point that is not before the end point of the heartbeat transaction commit event may include:

[0546] For storage nodes, the end point of the heartbeat transaction commit event is determined as the second point; or

[0547] If the heartbeat transaction is successfully committed by the storage node, the position corresponding to the second moment in the operation log of the storage node is determined as the second position; the second moment is not earlier than the end position of the latest heartbeat transaction commit event recorded between the storage nodes.

[0548] Optionally, the completion of a two-phase commit transaction may include: the storage node to which the two-phase commit transaction is targeted successfully rolling back the two-phase commit transaction; or the storage node to which the two-phase commit transaction is targeted successfully committing the two-phase commit transaction.

[0549] Optionally, the operation log may include: database logical log; the two-phase commit transaction may include: XA transaction.

[0550] For a detailed explanation of this method flow, please refer to the explanation of the above method flow S101-S105.

[0551] This specification also provides an embodiment of another data backup method.

[0552] This method is not limited to any particular entity; it can be applied to databases to back up stored data. A database can include at least two storage nodes.

[0553] The method process may include the following steps.

[0554] S501: Perform physical backup of the data stored in the storage node, and determine the operation log position corresponding to the start time of the physical backup in the storage node as the first point.

[0555] S502: If it is determined that the two-phase commit transactions in the incomplete transaction set have been completed, initiate a heartbeat transaction for the storage node; the incomplete transaction set includes the two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0556] S503: If the storage node successfully commits a heartbeat transaction, for the storage node, add the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event to the current set of committable transactions; determine the end point of the heartbeat transaction preparation event as the current pruning point of the storage node.

[0557] S504: Circularly update the current set of committable transactions and the current pruning point of the storage node, and confirm that there is a two-phase commit transaction between the storage node and the current pruning point at the first moment, which is included in the current set of committable transactions.

[0558] The current pruning point of the storage node includes the latest recorded point between the candidate point and the end point of the heartbeat transaction preparation event; the candidate point is the latest recorded end point of the preparation event in the operation log of the storage node between the two-phase commit transactions for the storage node in the current set of committable transactions.

[0559] S505: After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in the storage node to the current pruning point.

[0560] The explanation of this method can be found above.

[0561] This specification also provides another data recovery method. Specifically, it can be a data recovery method based on the above-described method flow S401-S403.

[0562] This method may include the following steps.

[0563] S601: For a storage node, obtain the physical backup storage data and the operation log from the first backup point to the pruning point, and restore the storage node based on the storage data and the obtained operation log.

[0564] S602: Obtain the backup set of committable transactions.

[0565] S603: If the storage node has been recovered, for a two-phase commit transaction in the ready-to-commit state in the storage node, if the transaction belongs to the set of committable transactions, commit the transaction; if the transaction does not belong to the set of committable transactions, roll back the transaction.

[0566] This method can ensure the consistency of transactions in the set of committable transactions by performing commit operations on those transactions.

[0567] For a detailed explanation and explanation of the principles, please refer to the text above.

[0568] Optionally, for one or more storage nodes, the physically backed-up storage data and the operation logs from the first backup point to the pruning point can be obtained, and the storage node can be restored based on the storage data and the obtained operation logs. Specifically, this can be done for all or some of the storage nodes in the database.

[0569] Corresponding to the above method embodiments, this specification also provides corresponding device embodiments.

[0570] like Figure 14 The diagram shown is a structural schematic of a data backup device provided in an embodiment of this specification.

[0571] This device embodiment can be specifically applied to a database for backing up stored data in the database.

[0572] A database may include at least two storage nodes; the stored data in the database may be distributed across at least two storage nodes.

[0573] The device may include the following units.

[0574] The data backup unit 701 is used to perform physical backup of the data stored in the storage node and to determine the operation log position corresponding to the start time of the physical backup in the storage node as the first point.

[0575] Construction unit 702 is used to construct a set of committable transactions and determine the pruning point of the storage node.

[0576] Optionally, the set of committable transactions includes two-phase commit transactions in which commit events exist between the first point and the pruning point.

[0577] Optionally, the pruning point of a storage node includes the latest recorded preparation event end point in the operation log of the storage node between two-phase commit transactions for that storage node in the set of committable transactions.

[0578] The first backup unit 703 is used to back up the set of committable transactions and the operation log from the first point to the pruning point in the storage node.

[0579] Optionally, building unit 702 is used for:

[0580] After the operation log position corresponding to the preset transaction completion time, determine the pruning position of the storage node;

[0581] The preset transaction completion time includes: the time when two-phase commit transactions in the incomplete transaction set are completed; the incomplete transaction set includes the incomplete two-phase commit transactions in the database at the first moment after the physical backup.

[0582] Optionally, building unit 702 is used for:

[0583] If the two-phase commit transactions in the incomplete transaction set are confirmed to be completed, a heartbeat transaction is initiated for the storage node; the incomplete transaction set includes the two-phase commit transactions that were not completed in the database at the first moment after the physical backup.

[0584] If a storage node successfully commits a heartbeat transaction, for that storage node, the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event is added to the current set of committable transactions; the heartbeat transaction is also added to the current set of committable transactions.

[0585] Circularly update the current set of committable transactions and the current pruning point of the storage node, and confirm that there is a two-phase commit transaction between the storage node and the current pruning point at the first moment, which is included in the current set of committable transactions;

[0586] The current pruning point of a storage node includes the latest recorded preparation event end point in the operation log of the storage node between two-phase commit transactions for that storage node in the current set of committable transactions.

[0587] Optionally, the first backup unit 703 is used for:

[0588] After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in the storage node to the current pruning point.

[0589] Optionally, building unit 702 is used for:

[0590] Repeat the following steps until no new transactions are added to the current set of committable transactions:

[0591] For storage nodes, two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point will be added to the set of currently committable transactions.

[0592] For a storage node, the latest recorded preparation event end point in the operation log of that storage node among the two-phase commit transactions in the current set of committable transactions is determined as the current pruning point.

[0593] Optionally, the method for determining the current pruning point of the storage node includes:

[0594] For storage nodes, starting from the heartbeat transaction commit event, traverse backwards through the preparation events in the operation log;

[0595] If the preparation event currently being traversed corresponds to a two-phase commit transaction and belongs to the current set of committable transactions, then the end point of the preparation event for that transaction is determined as the current pruning point of that storage node.

[0596] Optionally, the building unit 702 is further configured to: for a storage node, determine a second point that is not before the end point of the heartbeat transaction commit event;

[0597] Optionally, building unit 702 is used for:

[0598] For the preset operation log of the storage node, two-phase commit transactions that have a commit event before the heartbeat transaction preparation event will be added to the current set of committable transactions; the preset operation log includes: the operation log between the first moment and the second moment;

[0599] Optionally, building unit 702 is used for:

[0600] Confirm that the two-phase commit transactions with commit events before the current pruning point in the preset operation log of the storage node are included in the current set of committable transactions.

[0601] Optionally, building unit 702 is used for:

[0602] For storage nodes, the end point of the heartbeat transaction commit event is determined as the second point; or

[0603] If the heartbeat transaction is successfully committed by the storage node, the position corresponding to the second moment in the operation log of the storage node is determined as the second position; the second moment is not earlier than the end position of the latest heartbeat transaction commit event recorded between the storage nodes.

[0604] Optionally, a two-phase commit transaction completion includes:

[0605] The two-phase commit transaction was successfully rolled back on the storage node it was targeting; or

[0606] The two-phase commit transaction is successfully committed to the storage node it is targeting.

[0607] Optionally, the operation log includes: database logical log; the two-phase commit transaction includes: XA transaction.

[0608] like Figure 15 The diagram shown is a structural schematic of another data backup device provided in the embodiments of this specification.

[0609] This device embodiment can be specifically applied to a database for backing up stored data in the database.

[0610] A database may include at least two storage nodes; the stored data in the database may be distributed across at least two storage nodes.

[0611] The device may include the following units.

[0612] The physical backup unit 801 is used to perform a physical backup of all stored data in each storage node and to determine the operation log position corresponding to the start time of the physical backup in that storage node as the first point.

[0613] Heartbeat transaction unit 802 is used to initiate a heartbeat transaction for all storage nodes when it is determined that all two-phase commit transactions in the incomplete transaction set have been completed; the incomplete transaction set includes all incomplete two-phase commit transactions in the database at the first moment after the physical backup.

[0614] The iterative update unit 803 is used to, when all storage nodes have successfully committed heartbeat transactions, add all two-phase commit transactions that have commit events between the first moment and the heartbeat transaction preparation event to the current set of committable transactions for each storage node; add heartbeat transactions to the current set of committable transactions; and iteratively update the current set of committable transactions and the current pruning point of each storage node until all two-phase commit transactions that have commit events between the first moment and the current pruning point for each storage node are included in the current set of committable transactions. The current pruning point of each storage node is the end point of the latest prepared event recorded in the operation log of the node among all two-phase commit transactions for that node in the current set of committable transactions.

[0615] Backup unit 804 is used to back up the current set of committable transactions and the operation log from the first point to the current pruning point in each storage node after the iteration is completed.

[0616] Optionally, the iterative update unit 803 is specifically used to: repeatedly execute the following steps until there are no new transactions in the current set of committable transactions: for each storage node, add all two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point to the current set of committable transactions; for each storage node, determine the end point of the latest recorded preparation event in the operation log of the node among all two-phase commit transactions for that node in the current set of committable transactions as the current pruning point.

[0617] Optionally, the method for determining the current pruning point of each storage node includes: for each storage node, starting from the heartbeat transaction commit event, traversing backwards through all preparation events in the operation log; if the preparation event currently being traversed corresponds to a two-phase commit transaction and belongs to the current set of committable transactions, determining the end point of the preparation event of that transaction as the current pruning point of that storage node.

[0618] Optionally, the iterative update unit 803 is also used to: for each storage node, determine a second point that is not before the end point of the heartbeat transaction commit event.

[0619] Optionally, the iterative update unit 803 is specifically used to: add all two-phase commit transactions that have commit events before the heartbeat transaction preparation event to the current set of committable transactions for each storage node's preset operation log; the preset operation log includes: the operation log between the first moment and the second point.

[0620] Optionally, the iterative update unit 803 is specifically used to iteratively update the current set of committable transactions and the current pruning point of each storage node until all two-phase commit transactions with commit events before the current pruning point are included in the current set of committable transactions in the preset operation log of each storage node.

[0621] Optionally, the iterative update unit 803 is specifically used to: for each storage node, determine the end point of the heartbeat transaction commit event as the second point; or, if all storage nodes successfully commit heartbeat transactions, determine the point corresponding to the second moment in the operation log of each storage node as the second point; the second moment is not earlier than the end point of the latest recorded heartbeat transaction commit event among all storage nodes.

[0622] Optionally, the completion of a two-phase commit transaction includes: each storage node targeted by the two-phase commit transaction successfully rolling back the two-phase commit transaction; or each storage node targeted by the two-phase commit transaction successfully committing the two-phase commit transaction.

[0623] Optionally, the operation log may include: database logical log; the two-phase commit transaction may include: XA transaction.

[0624] This specification also provides another embodiment of a data backup device.

[0625] This device embodiment can be specifically applied to a database for backing up stored data in the database.

[0626] A database may include at least two storage nodes; the stored data in the database may be distributed across at least two storage nodes.

[0627] The device may include the following units.

[0628] The database backup unit is used to perform a full physical backup of the stored data in each storage node, and to determine the operation log position corresponding to the start time of the physical backup in that storage node as the first point.

[0629] The heartbeat unit is used to initiate a heartbeat transaction for all storage nodes after determining that all two-phase commit transactions in the incomplete transaction set have been completed. The incomplete transaction set includes all incomplete two-phase commit transactions in the database at the first moment after the physical backup.

[0630] The update unit is used to, when all storage nodes have successfully committed heartbeat transactions, add all two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event to the current set of committable transactions for each storage node; determine the end point of the heartbeat transaction preparation event as the current pruning point for each storage node; and cyclically update the current set of committable transactions and the current pruning point for each storage node until all two-phase commit transactions that have a commit event between the first moment and the current pruning point for each storage node are included in the current set of committable transactions.

[0631] Optionally, the current pruning point of each storage node is the latest recorded point between the candidate point and the end point of the heartbeat transaction preparation event; the candidate point is the end point of the latest recorded preparation event in the operation log of the node among all two-phase commit transactions for that node in the current set of committable transactions.

[0632] The log backup unit is used to back up the current set of committable transactions and the operation log from the first point to the current pruning point in each storage node after the loop ends.

[0633] This specification also provides a data recovery device based on the above-described device embodiments, comprising:

[0634] The log recovery unit is used to obtain the full physical backup storage data and the operation log from the first backup point to the current pruning point for any storage node, and to restore the storage node based on the obtained full storage data and operation log.

[0635] The transaction recovery unit is used to obtain the set of currently committable transactions from the backup; when the storage node has been restored, for each two-phase commit transaction in the ready-to-commit state in the storage node, if the transaction belongs to the set of currently committable transactions, the transaction is committed; if the transaction does not belong to the set of currently committable transactions, the transaction is rolled back.

[0636] For a detailed explanation of the above device embodiments, please refer to the above method embodiments.

[0637] This specification also provides a system embodiment, which may specifically be a database system.

[0638] A database system may include a control node and at least two storage nodes.

[0639] The control node is used to: perform physical backups of the data stored in the storage node, and determine the operation log position corresponding to the start time of the physical backup in the storage node as the first point;

[0640] Construct a set of committable transactions and determine the pruning point for the storage nodes;

[0641] The set of committable transactions includes two-phase commit transactions in which commit events exist between the first point and the pruning point.

[0642] The pruning point of a storage node includes the latest prepared event end point recorded in the operation log of that storage node between two-phase commit transactions in the set of committable transactions for that storage node;

[0643] Backups can commit the set of transactions, as well as the operation logs from the first point to the pruning point in the storage node.

[0644] Optionally, the control node can be a computing node within the database system. The database system may include a computing node and at least two storage nodes.

[0645] For an explanation of the system embodiments, please refer to the above method embodiments.

[0646] The database system architecture diagram is provided as an example; for details, please refer to [link / reference]. Figure 2 .

[0647] This specification also provides a system embodiment, specifically a data backup system.

[0648] A data backup system may include a management device and a database system, the database system of which may include at least two storage nodes. The management device may have control over the database system.

[0649] The management device is used to: perform physical backups of the data stored in the storage node, and determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point;

[0650] Construct a set of committable transactions and determine the pruning point for the storage nodes;

[0651] The set of committable transactions includes two-phase commit transactions in which commit events exist between the first point and the pruning point.

[0652] The pruning point of a storage node includes the latest prepared event end point recorded in the operation log of that storage node between two-phase commit transactions in the set of committable transactions for that storage node;

[0653] Backups can commit the set of transactions, as well as the operation logs from the first point to the pruning point in the storage node.

[0654] For an explanation of the system embodiments, please refer to the above method embodiments.

[0655] For ease of understanding, such as Figure 16 The diagram shown is an architectural schematic of a data backup system provided in an embodiment of this specification.

[0656] This includes management equipment and a database system. The database system may include two compute nodes and three storage nodes.

[0657] Management devices can have control over database systems, thereby enabling the backup of data stored in the database system.

[0658] certainly, Figure 16 The number of computing nodes and storage nodes included in the database system are for illustrative purposes only and do not limit the scope of the embodiments described in this specification.

[0659] This specification also provides a computer device, which includes at least a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described method embodiments.

[0660] This specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement any of the above method embodiments.

[0661] Figure 17 This diagram illustrates a more specific hardware structure of a computer device provided in an embodiment of this specification. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0662] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0663] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0664] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0665] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0666] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0667] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0668] This specification also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements any of the above-described method embodiments.

[0669] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0670] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this specification can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this specification, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this specification.

[0671] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0672] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. When implementing the embodiments of this specification, the functions of each module can be implemented in one or more software and / or hardware. Alternatively, some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0673] The above description is merely a specific implementation of the embodiments of this specification. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles of the embodiments of this specification, and these improvements and modifications should also be considered as protection of the embodiments of this specification.

Claims

1. A data backup method applied to a database, the database including at least two storage nodes; the method includes: Perform physical backups on the data stored in the storage node, and determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point; A set of committable transactions is constructed, and the pruning point of the storage node is determined. Specifically, if a two-phase commit transaction in the incomplete transaction set is completed, a heartbeat transaction is initiated for the storage node. The incomplete transaction set includes the two-phase commit transactions in the database that were not completed at the first moment after the physical backup. If the storage node successfully commits the heartbeat transaction, for the storage node, two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event are added to the current set of committable transactions. The heartbeat transaction is added to the current set of committable transactions. The current set of committable transactions and the current pruning point of the storage node are updated cyclically to confirm that two-phase commit transactions that have a commit event between the first moment and the current pruning point of the storage node are included in the current set of committable transactions. In each loop, the current pruning point of the storage node is determined as: the latest recorded preparation event end point in the operation log of the storage node among the two-phase commit transactions for that storage node in the current set of committable transactions. The current pruning point after the loop ends is determined as the pruning point of the storage node. The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point. Backup the set of committable transactions, and the operation logs between the first point and the pruning point in the storage node.

2. The method according to claim 1, wherein determining the pruning point of the storage node includes: After the operation log position corresponding to the preset transaction completion time, determine the pruning position of the storage node; The preset transaction completion time includes: the time when the two-phase commit transactions in the incomplete transaction set are completed; the incomplete transaction set includes the incomplete two-phase commit transactions in the database at the first moment after the physical backup.

3. The method according to claim 1, wherein backing up the set of committable transactions and the operation log in the storage node from the first point to the pruning point includes: After the loop ends, back up the current set of committable transactions, as well as the operation logs from the first point to the current pruning point in the storage node.

4. The method according to claim 1, wherein the cyclical update of the current committable transaction set and the current pruning point of the storage node comprises: Repeat the following steps until no new transactions are added to the current set of committable transactions: For storage nodes, two-phase commit transactions that have a commit event between the heartbeat transaction preparation event and the current pruning point will be added to the set of currently committable transactions. For a storage node, the latest recorded preparation event end point in the operation log of that storage node among the two-phase commit transactions in the current set of committable transactions is determined as the current pruning point.

5. The method according to claim 1, the method for determining the current pruning point of a storage node, comprising: For storage nodes, starting from the heartbeat transaction commit event, traverse backwards through the preparation events in the operation log; If the preparation event currently being traversed corresponds to a two-phase commit transaction and belongs to the current set of committable transactions, then the end point of the preparation event for that transaction is determined as the current pruning point of that storage node.

6. The method according to claim 1, further comprising: For the storage node, determine the second point that is not before the end point of the heartbeat transaction commit event; For storage nodes, two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event are added to the current set of committable transactions, including: For the preset operation log of the storage node, two-phase commit transactions that have a commit event before the heartbeat transaction preparation event are added to the current set of committable transactions; the preset operation log includes: the operation log between the first moment and the second point; The two-phase commit transaction, in which the confirmation storage node has a commit event between the first moment and the current pruning point, is included in the current set of committable transactions, including: Confirm that the two-phase commit transactions with commit events before the current pruning point in the preset operation log of the storage node are included in the current set of committable transactions.

7. The method according to claim 6, wherein determining, for the storage node, a second point not preceding the end point of the heartbeat transaction commit event, comprises: For storage nodes, the end point of the heartbeat transaction commit event is determined as the second point; or If the heartbeat transaction is successfully committed by the storage node, the position corresponding to the second moment in the operation log of the storage node is determined as the second position; the second moment is not earlier than the end position of the latest heartbeat transaction commit event recorded between the storage nodes.

8. The method according to claim 2, wherein the completion of the two-phase commit transaction includes: The storage node targeted by the two-phase commit transaction successfully rolled back the two-phase commit transaction; or The two-phase commit transaction is successfully committed to the storage node targeted by the two-phase commit transaction.

9. The method according to claim 1, wherein the operation log includes: Database logical log; The two-phase commit transaction includes: XA transaction.

10. A data recovery method based on the data backup method according to any one of claims 1 to 9, comprising: For a storage node, obtain the physical backup storage data and the operation log from the first backup point to the pruning point, and restore the storage node based on the storage data and the obtained operation log; Obtain the backup set of committable transactions; If the storage node is recovered, for a two-phase commit transaction in the ready-to-commit state on that storage node, if the transaction belongs to the set of committable transactions, the transaction is committed; if the transaction does not belong to the set of committable transactions, the transaction is rolled back.

11. A data backup method applied to a database, the database including at least two storage nodes; the method includes: Perform physical backups on the data stored in the storage node, and determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point; If it is determined that a two-phase commit transaction in the incomplete transaction set has been completed, a heartbeat transaction is initiated for the storage node; the incomplete transaction set includes the incomplete two-phase commit transactions in the database at the first moment after the physical backup; If the heartbeat transaction is successfully committed on the storage node, for the storage node, the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event is added to the current set of committable transactions; The end point of the heartbeat transaction preparation event is determined as the current pruning point of the storage node; Circularly update the current set of committable transactions and the current pruning point of the storage node, and confirm that there is a two-phase commit transaction between the storage node and the current pruning point at the first moment, which is included in the current set of committable transactions; The current pruning point of the storage node includes the latest recorded point between the candidate point and the end point of the heartbeat transaction preparation event; the candidate point is the latest recorded end point of the preparation event in the operation log of the storage node between the two-phase commit transactions for the storage node in the current set of committable transactions. After the loop ends, back up the current set of committable transactions, as well as the operation log from the first point in the storage node to the current pruning point.

12. A data backup device applied to a database, the database including at least two storage nodes; the device comprising: The physical backup unit is used to perform physical backups of the data stored in the storage node and to determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point. A construction unit is used to construct a set of committable transactions and determine the pruning point of the storage node. Specifically, if a two-phase commit transaction in the incomplete transaction set is completed, a heartbeat transaction is initiated for the storage node. The incomplete transaction set includes the two-phase commit transactions in the database that were not completed at the first moment after the physical backup. If the storage node successfully commits the heartbeat transaction, for the storage node, two-phase commit transactions that have a commit event between the first moment and the heartbeat transaction preparation event are added to the current set of committable transactions. The heartbeat transaction is added to the current set of committable transactions. The current set of committable transactions and the current pruning point of the storage node are updated cyclically to confirm that two-phase commit transactions that have a commit event between the first moment and the current pruning point of the storage node are included in the current set of committable transactions. In each loop, the current pruning point of the storage node is determined as the latest recorded preparation event end point in the operation log of the storage node among the two-phase commit transactions for that storage node in the current set of committable transactions. The current pruning point after the loop ends is determined as the pruning point of the storage node. The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point. The backup unit is used to back up the set of committable transactions and the operation logs between the first point and the pruning point in the storage node.

13. A database system comprising a control node and at least two storage nodes; The control node is used to: perform physical backups of the data stored in the storage node, and determine the operation log position corresponding to the start time of the physical backup in the storage node as the first point; Construct a set of committable transactions and determine the pruning point for the storage nodes, where, If it is determined that a two-phase commit transaction in the incomplete transaction set has been completed, a heartbeat transaction is initiated for the storage node; the incomplete transaction set includes the incomplete two-phase commit transactions in the database at the first moment after the physical backup; If the heartbeat transaction is successfully committed on the storage node, for the storage node, the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event is added to the current set of committable transactions; Add the heartbeat transaction to the current set of committable transactions; Circularly update the current set of committable transactions and the current pruning point of the storage node, and confirm that there is a two-phase commit transaction between the storage node and the current pruning point at the first moment, which is included in the current set of committable transactions; In each loop, the current pruning point of the storage node is determined as: the latest recorded end point of the preparation event in the operation log of the storage node between the two-phase commit transactions for the storage node in the current set of committable transactions; The current pruning point after the loop ends is determined as the pruning point of the storage node; The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point. Backup the set of committable transactions, and the operation logs between the first point and the pruning point in the storage node.

14. A data backup system, comprising a management device and a database system; the database system comprising at least two storage nodes; the management device having control authority over the database system; The management device is used to: perform physical backups of the data stored in the storage node, and determine the operation log point corresponding to the start time of the physical backup in the storage node as the first point; Construct a set of committable transactions and determine the pruning point for the storage nodes, where, If it is determined that a two-phase commit transaction in the incomplete transaction set has been completed, a heartbeat transaction is initiated for the storage node; the incomplete transaction set includes the incomplete two-phase commit transactions in the database at the first moment after the physical backup; If the heartbeat transaction is successfully committed on the storage node, for the storage node, the two-phase commit transaction that has a commit event between the first moment and the heartbeat transaction preparation event is added to the current set of committable transactions; Add the heartbeat transaction to the current set of committable transactions; Circularly update the current set of committable transactions and the current pruning point of the storage node, and confirm that there is a two-phase commit transaction between the storage node and the current pruning point at the first moment, which is included in the current set of committable transactions; In each loop, the current pruning point of the storage node is determined as: the latest recorded end point of the preparation event in the operation log of the storage node between the two-phase commit transactions for the storage node in the current set of committable transactions; The current pruning point after the loop ends is determined as the pruning point of the storage node; The set of committable transactions includes two-phase commit transactions in which there is a commit event between the first point and the pruning point. Backup the set of committable transactions, and the operation logs between the first point and the pruning point in the storage node.

15. An electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 11.

16. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as claimed in any one of claims 1 to 11.

Images