Distributed database remote backup

Abstract

A method and system for data synchronization between a standby site and a primary site in a distributed database system is provided. A first primary site included in a primary site group of the distributed database system generates a commit action redo log message for a commit action performed by the first primary site on a first database transaction, the commit action redo log message including a transaction ID of the first database transaction and a transaction order indicator representing an order of database transactions in a group of database transactions executed on one or more primary sites in the primary site group. The first primary site sends the commit action redo log message to a corresponding first standby site backing up the first primary site.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 17 / 174,024, entitled “Distributed Database Remote Backup,” filed February 11, 2021, which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to data management, specifically to a method and system for synchronizing data between a backup site and a master site in a distributed computer system. Background Technology

[0004] In data management, a distributed database transaction can be a database transaction that is synchronized (or managed collaboratively) among multiple participating databases located in different physical locations. The multiple participating databases in a distributed system can include multiple primary databases, each corresponding to a primary site, and multiple standby or backup databases, each corresponding to a standby site. Each standby site corresponds to a primary site and synchronizes with the latest changes made in the primary site. Because the replicated database maintained by the standby site is close to the primary site database, but not a live copy of the primary site database, the standby site is a slightly outdated mirror of the primary site's data. Therefore, restoring data from the standby site (if necessary) is called asynchronous data recovery.

[0005] To support data recovery in the event of a failure, each primary site records all changes in its redo log. The primary site then sends a redo log update, recording the additional database changes, to its corresponding standby site. Once the standby site receives the redo log update, it applies it, thus synchronizing with the primary site.

[0006] In the event of a primary site failure, a backup site can be used to restore the primary site or upgraded to replace the primary site with minimal latency.

[0007] Therefore, it is desirable to provide a method and system that enables a standby site to accurately mirror changes made to its corresponding primary site, satisfying atomicity and dependency requirements with minimal divergence and maximum parallelism to support asynchronous data recovery scenarios. Summary of the Invention

[0008] According to a first example aspect, there is provided a method comprising: a first primary site included in a group of primary sites of a distributed database system generating a commit action redo log message for a commit action performed by the first primary site on a first database transaction, the commit action redo log message including a transaction ID of the first database transaction and a transaction order indicator indicating an order of database transactions in a group of database transactions executed on one or more primary sites in the group of primary sites; the first primary site sending the commit action redo log message to a corresponding first backup site backing up the first primary site.

[0009] In some examples of the first aspect, the transaction order indicator includes a vector including a respective log sequence number of each of the primary sites in the group of primary sites, the log sequence number of each of the primary sites corresponding to a commit action performed by the respective primary site.

[0010] In examples of one or more of the above aspects, the method includes: receiving, at the first primary site, log sequence numbers corresponding to commit actions performed by other primary sites in the group of primary sites, wherein the log sequence number of each primary site included in the transaction order indicator corresponds to a last commit action performed by the primary site known to the first primary site.

[0011] In examples of one or more of the above aspects, the method includes: the first primary site receiving a notification of the transaction ID of the first database transaction from a coordinator that allocates incremental transaction IDs for database transactions; the first primary site providing a notification to the coordinator that the first primary site is ready to commit the first database transaction, wherein the log sequence numbers corresponding to the commit actions performed by the other primary sites in the group of primary sites are received by the first primary site from the coordinator.

[0012] In examples of one or more of the above aspects, the method includes: receiving, at the first backup site, the commit action redo log message and determining when to commit the first database transaction based on the log sequence numbers included in the transaction order indicator.

[0013] In some examples of the first aspect, the transaction order indicator includes a commit sequence number assigned to the first database transaction by a coordinator, the commit sequence number indicating when the first database transaction is committed relative to other database transactions included in the group of database transactions on one or more of the primary sites.

[0014] In examples of the above aspect, the method includes receiving the commit action redo log message at the first standby site and determining when to commit the first database transaction based on a comparison of the commit sequence number assigned to the first database transaction to commit sequence numbers included in other commit action redo log messages received at other standby sites.

[0015] In some examples, the comparison includes comparing the commit sequence number assigned to the first database transaction to a consensus point value, where the consensus point value is a minimum commit sequence number of a group that includes a maximum commit sequence number received at each of the standby sites corresponding to the group of primary sites.

[0016] In some examples of the first aspect, when the first database transaction does not meet an importance criterion, the transaction order indicator includes a commit sequence number assigned to the first database transaction by a coordinator, the commit sequence number indicating when to commit the first database transaction relative to other database transactions included in the group of database transactions at one or more of the primary sites; when the first database transaction meets the importance criterion, the transaction order indicator includes (i) the commit sequence number assigned to the first database transaction by the coordinator and (ii) a vector including a respective log sequence number of each of the primary sites in the group of primary sites, the log sequence number of each of the primary sites corresponding to a commit action performed by the respective primary site.

[0017] According to a second example aspect, there is provided a first primary site included in a group of primary sites participating in database transactions. The first primary site includes a processing system including one or more processing units and one or more storage devices storing instructions that, when executed by the one or more processing units, are operable to cause the first primary site to perform operations including: generating a commit action redo log message for a commit action performed by the first primary site on a first database transaction, the commit action redo log message including a transaction ID of the first database transaction and a transaction order indicator representing an order of database transactions in a group of database transactions performed on one or more primary sites in the group of primary sites; sending the commit action redo log message to a corresponding first standby site backing up the first primary site.

[0018] According to a third example aspect, there is provided a method performed at a first standby site of a first primary site in a backup distributed database system, the distributed database system comprising a group of primary sites, each primary site having a respective standby site, the method comprising: receiving, at the first standby site, a redo log message regarding a first transaction performed at the first primary site, the redo log message comprising a transaction ID of the first transaction and a first transaction order indicator indicating an order of the first transaction in a group of transactions committed at the group of primary sites; receiving, at the first standby site, information regarding transaction order indicators received at other standby sites; determining, based on the redo log message and the information regarding transaction order indicators received at other standby sites, when to commit the first transaction at the standby site.

[0019] In some examples of the third aspect, each transaction order indicator comprises a vector, the vector comprising a respective log sequence number of each of the primary sites in the group of primary sites, the log sequence number of each of the primary sites corresponding to a commit action performed at the respective primary site.

[0020] In some examples of the third aspect, the first transaction order indicator comprises a commit sequence number of the first transaction, the commit sequence number indicating when the first database transaction is committed at one or more of the primary sites relative to other database transactions included in the group of database transactions, the transaction order indicators received at the other standby sites each indicating a commit sequence number of a transaction committed at the primary site corresponding to the other standby site.

[0021] In some examples of the third aspect, a consensus point value is determined, wherein the consensus point value is a minimum commit sequence number of a group, the group comprising a maximum commit sequence number received at each of the standby sites corresponding to the group of primary sites, wherein determining when to commit the first transaction at the standby site is based on a comparison of the commit sequence number of the first transaction and the consensus point value.

[0022] According to a fourth example aspect, there is provided a first backup site included in a backup site group backing up a primary site group participating in a database transaction. The first backup site comprises a processing system comprising one or more processing units and one or more storage devices storing instructions that, when executed by the one or more processing units, are operable to cause the first backup site to perform operations comprising: receiving, at the first backup site, a redo log message regarding a first transaction performed at the first primary site, the redo log message comprising a transaction ID of the first transaction and a first transaction order indicator indicating an order of the first transaction in a group of transactions committed at the primary site group; receiving, at the first backup site, information regarding transaction order indicators received at other backup sites; determining, based on the redo log message and the information regarding transaction order indicators received at other backup sites, when to commit the first transaction at the backup site.

[0023] According to a fifth example aspect, there is provided a computer readable medium storing instructions that, when executed by a processing unit of a distributed database site, can cause the site to perform one or more of the above described methods. BRIEF DESCRIPTION OF DRAWINGS

[0024] Reference will now be made to the drawings in which the exemplary embodiments of the present application will be illustrated, by way of example, and in which:

[0025] Figure 1 An exemplary schematic diagram of a distributed computer system is shown.

[0026] Figure 2A An example of redo logs provided by an exemplary embodiment in a scenario where a single transaction is committed at a primary site, each redo log comprising a log sequence number (LSN) within a succession dependency vector (SDV), is shown.

[0027] Figure 2B An example of SDVs provided by an exemplary embodiment in a scenario where multiple dependent transactions are committed at multiple primary sites is shown.

[0028] Figure 3 An example of SDVs provided by an exemplary embodiment in a scenario where transactions with a two-phase commit (2PC) protocol are committed at multiple primary sites is shown.

[0029] Figure 4AAn example redo log including a commit sequence number (CSN) in a scenario where a single transaction is committed at a primary site is shown, provided by an implementation of the present invention;

[0030] Figure 4B An example table and time graph of the relationship assigned between a transaction identifier (ID) and a commit sequence number (CSN), provided by an implementation of the present invention;

[0031] Figure 4C An example of redo log updates received at multiple secondary sites at different times is shown, provided by an implementation of the present invention;

[0032] Figure 4D Another example of redo log updates received at multiple secondary sites at different times is shown;

[0033] Figure 4E Another example of redo log updates received at multiple secondary sites at different times is shown;

[0034] Figure 5 is a block diagram of a processing system that can be used in Figures 1 to 4A one or more primary sites or Figure 1 , Figures 4C to 4E one or more secondary sites or one or more coordinators, provided by example embodiments.

[0035] The same reference numbers are used throughout the drawings to represent the same elements and features. While aspects of the present invention will be described in conjunction with the illustrated embodiments, it should be understood that the present invention is not limited to these embodiments. DETAILED DESCRIPTION

[0036] The present invention provides methods and systems for managing asynchronous data recovery in a distributed computer system in order to maintain database consistency and integrity between primary sites and corresponding secondary sites. In the present invention, a site can refer to a database instance, which is a collection of software-implemented memory structures for manipulating data in a database. A database can refer to a collection of files that store data and metadata. In some examples, database files can be written to persistent read / write storage, such as optical disk storage. A site (e.g., a database instance) can be implemented by a combination of a processing system and machine-readable instructions executable on the processing system. In some examples, each site can be hosted by a single processing system, such as a computer. In some examples, multiple sites can be hosted on a single processing system.

[0037] In the present invention, a database transaction refers to an independently executed logical atomic unit of work for data retrieval or update, and can include one or more actions (also referred to as operations) that implement one or more changes to a database. In this regard, a database transaction includes a series of indivisible and irreducible actions that must be completed. This means that, in order for a participating site to complete and save the transaction (also referred to as “commit”), all actions in the transaction must be completed. Otherwise, the transaction must be aborted and all actions rolled back. Examples of actions that a site can perform include actions that enable the site to store, modify, delete, and retrieve data.

[0038] Figure 1 An example of a distributed computer database system 100 provided by aspects of the present invention is shown. The distributed computer database system 100 includes a plurality of primary sites 102(1) through 102(n) (generally referred to collectively as primary sites 102 or individually as primary site 102(i)) and a plurality of backup sites 104(1) through 104(n) (generally referred to collectively as backup sites 104 and individually as primary site 104(i)). Each primary site 102(i) has a corresponding backup site 104(i). Each primary site 102(i) manages a database 105(i), and each corresponding backup site 104(i) manages a duplicate database 105'(i) that is a copy of the database 105(i) of its respective primary site 102(i). In some examples, when a primary site 102 becomes too large, it can be split into a plurality of primary shards. “When” as used in the present invention can refer to a time at which an environment causes something to happen, which time is not necessarily a particular clock time or a particular point or interval of time. Each primary shard will have a corresponding backup shard. For the present invention, a “shard” can also be considered a “site.” The primary sites 102 and backup sites 104 can be connected to a communication network system 150, which can include one or more networks, including, for example, an intranet, one or more intranets, a wired network, a wireless network, a virtual private network, and combinations thereof.

[0039] In example embodiments, transactions performed by the primary sites 102 are managed by a transaction coordinator 122. The transaction coordinator 122 can be a module implemented by a combination of machine-readable instructions executable on a processing system. In some examples, different coordinators 122 can be used to manage different transactions or groups of transactions. In some examples, the coordinator 122 can be co-hosted on a processing system with the primary sites 102.

[0040] Each primary site maintains a corresponding redo log file 110 in a defined format, which records the history of all changes made to the database 105(i) of the primary site 102(i). The primary site 102(i) generates (in any way) a new redo log 106(i) for each action performed by the primary site 102(i) to change its database 105(i). Whenever the primary site 102(i) generates a new redo log 106(i), it adds a copy of redo log 106(i) to its redo log file 110 and also provides a redo log message 112(i) that includes a copy of redo log 106(i) from its corresponding standby site 104(i). The standby site 104(i) performs the actions included in redo log 106(i) to manage the duplicate database 105'(i). Figure 1 An illustrative example of a redo log file 110 corresponding to the primary site 102(1) is shown. Redo log file 110 comprises multiple consecutively generated redo logs. Each transaction ends with a redo log entry recording a "commit" action. Figure 1 In the figure, the illustrative commit action redo log 106C(1) is enclosed in a dashed box. As shown, each redo log 106(1) identifies the corresponding action (in Figure 1 In the case of the commit action redo log 106C(1), the commit action is a transaction ID (e.g., Trx_1) that identifies the transaction whose action was performed by the master site 102(1), and the log sequence number (LSN) (e.g., LSN = 4). In an exemplary embodiment, the transaction ID is incrementally assigned by the coordinator 122 each time a new transaction is received. The LSN is incrementally assigned by the master site 102(i) to each redo log 106(i) it generates. Therefore, the LSN value of each master site 102(i) is a locally generated value, wherein a new LSN value is generated for each redo log 106(i) generated at the site (including the commit action redo log 106C(i)).

[0041] In an example, each primary site 102(i) maintains a local transaction order indicator, e.g., a site dependency vector (SDV) 108(i), stored with redo log files 110. The transaction order indicator represents (informally, represents or corresponds to) the order (sequence) of database transactions in a group of database transactions. A vector is an ordered set or array of numbers, often associated with the order in which the numerical values appear in the vector as well as the numerical values themselves. The SDV 108(i) is used to track the current (e.g., maximum) LSN values of commit actions performed at primary site 102(i) and other primary sites 102, thereby providing an indication of the location of each primary site 102 in the transaction log stream. The SDV 108(1) includes a slot or element (S1, S2,..., Sn) for each of the primary sites 102(1) through 102(n) that participated in at least some of the transactions included in the global transaction log stream in the distributed computer database system 100. The values included in the respective elements (S1, S2,..., Sn) of the SDV 108(1) identify the LSN values of the last commit action redo log 106C(1) through 106C(n) recorded by all of the primary sites 102(1) through 102(n) in the transaction log stream based on the current information available to the primary site 102(1). For example, vector element S1 can indicate the LSN of the commit action redo log 106C(1) of the last transaction committed by primary site 102(1), vector element S2 can indicate the LSN of the commit action redo log 106C(2) of the last transaction committed by primary site 102(2) as known by primary site 102(1), and vector element Sn can indicate the LSN of the commit action redo log 106C(n) of the last transaction committed by primary site 102(n) as known by primary site 102(1). In an example embodiment, the values of the vector elements (S1, S2,..., Sn) are each set to the log sequence number (LSN:Trx) of the last known commit action of a transaction indicated by the primary site that performed the commit action. As will be explained in greater detail below, the SDVs 108(1) through 108(n) are used to provide a vector of Lamport clocks that a site can use to determine its location in the transaction log stream. In an example embodiment, if the value of n exceeds a threshold, then the other primary sites 102 represented as commit action LSN slots in the SDV of a particular primary site 102(i) can be less than n. In this case, the represented primary sites 102 can be the sites most likely to be involved in a dependent transaction with the primary site 102(i). In some examples, the allocation of slots in the SDV for a particular primary site can be predetermined by a system administrator.

[0042] In example embodiments, when a primary site 102(i) performs an action with respect to a transaction, the action is recorded in a corresponding redo log 106(i) at the redo log file 110 of the primary site, and the redo log 106(i) is sent to its corresponding standby site 104(i) as part of a redo log message 112(i). In example embodiments, each time a primary site 102(i) becomes aware that a new commit action has been performed by itself or another primary site, the SDV 108(i) of the primary site is updated. In example embodiments, each time a primary site 102(i) reports a new commit action redo log 106C(i), the primary site 102(i) includes its current SDV 102(i) as part of the redo log message 112(i). The redo log message 112(i) reporting a new commit action redo log can be referred to as a “commit action redo log message”. In some alternatives, the current SDV 108(i) can be included in each redo log message 112(i) sent to a standby site 104(i).

[0043] Once a standby site 104(i) receives a redo log 106(i) from its corresponding primary site 102(i), the standby site 104(i) can perform the actions specified in the redo log 106(i) to update the replicated database 105'(i) and update its own copy of the redo log file to maintain synchronization with the primary site 102(i).

[0044] At some point, a standby site 104 participating in a transaction must make a decision to commit the transaction or to roll back the transaction. In this regard, a number of criteria are needed for each participating standby site 104 to determine whether to roll back or commit, including atomicity requirements and dependency requirements. Atomicity requirements mean that all actions in a transaction must be performed by the standby site 104 for the transaction to commit, otherwise the actions of the transaction must all be rolled back. Dependency requirements mean that unless a standby site 104(i) can confirm that a first transaction has committed, a second transaction that depends on the first transaction will not commit at the standby site 104(i).

[0045] Systems and methods for tracking transaction dependencies between standby sites 104 are described. Uncertainty can arise in a standby site 104(i) that does not know the progress of the actions of other standby sites 104 with respect to the execution of dependent transactions, which makes it difficult for the standby site 104(i) to determine when and whether to commit transactions described in redo log messages 112(i) that the standby site has received. Uncertainty can result in large recovery point objectives (RPOs) and long recovery time objectives (RTOs) for standby sites 104 in order to keep the data in the standby sites 104 consistent with changes that occur in the corresponding primary sites 106. Large RPOs can result in significant divergence between primary sites and corresponding standby sites, while long RTOs can result in backups of standby sites having low parallelism. Moreover, large RPOs and long RTOs can in turn result in inaccuracy and introduce errors in recovering data in a distributed computer database system when a primary site is shut down or otherwise fails. Accordingly, in at least some examples, the methods and systems disclosed herein can be used to improve the efficiency and accuracy of a distributed database system.

[0046] According to example embodiments, in order to reduce uncertainty that can arise due to unknown inter-transaction dependencies, the above-described site dependency vector (SDV) 108(i) is used as a mechanism to track transaction dependencies between backup sites 104.

[0047] Figure 2AA simplified example is shown of updating the site dependency vector (SDV) 108(1) of primary site 102(1) with respect to a single-site transaction Trx_1 that includes all one or more database change actions performed at primary site 102(1). Dotted box TO shows the SDV 108(1) stored as part of the log update file 110 at time TO prior to primary site 108(1) committing transaction Trx_1. As described above, the SDV 108(1) will include n elements (S1 :*, S2 :*,..., Sn :*), where, from the perspective of primary site 102(1), the elements S1 through Sn represent the last known transaction committed by itself and other primary sites 102(2) through 102(n), respectively. The value included in each of the elements S1 through Sn is the primary site 102(1)'s knowledge of the local LSN of the last transaction committed at each of the primary sites 102(i) through 102(n), respectively, performing a commit action. Thus, from the perspective of primary site 102(1), the element values in the SDV 108(1) can provide information about the offset of committed transactions in the overall log stream of the distributed computer database system 100. In this example, the "*" can represent a null value, indicating no prior transaction is known at time TO.

[0048] Dotted box T1 shows the SDV 108(1) at time T1 after primary site 102(1) has performed all database change actions that are part of transaction Trx_1 and commits transaction Trx_1. In particular, the SDV element S1 corresponding to primary site 102(i) has been set to S1 :LSN_j, where "LSN_j" is the log sequence value (LSV) corresponding to the commit action of transaction Trx_1. For example, if the SDV 108(1) vector element value S1 = "0" at the start of transaction Trx_1, and Trx_1 includes 2 database change actions at primary site 102(1), followed by a commit action at primary site 102(1), then the LSN value will have increased by 3 integer units, such that S1 :LSN_j = S1 :3 (assuming no intervening operations with respect to other transactions). In Figure 2A In the example of FIG. 1, primary site 102(1) is not aware of any other transactions committed by other primary sites, so all other SDV element values S2 through Sn retain their prior values (e.g., "*" or "null" in the illustrated example).

[0049] In the example real-time update embodiment, as the respective actions included in the transaction Trx_1 are executed, the primary site 102(1) will prepare and send respective redo logs 106(1) on an action-by-action basis to its standby site 104(1). When the primary site 102(1) executes a commit action (i.e., when it writes the transaction Trx_1 to disk or other non-transient database storage), it immediately provides the standby site 104(1) with a commit action redo log 106C(1) for the commit action. In the example embodiment, the currently updated SDV 108(1) will be included in a redo log message 112(1) that includes the commit action redo log 106C(1).

[0050] In some examples, the primary site 102(1) can send the redo log message 112(1) (including the commit action redo log 106C(1) and the SDV 108(1)) in a network communication addressed to the corresponding standby site 104(1). In some examples, the primary site 102(1) can send the redo log message 112(1) to an intermediate central storage site for retrieval by the standby site 104(1). At the corresponding standby site 104(1), once the redo log message 112(1) (including the commit action redo log 106C(1) and the updated SDV 108(1) (e.g., (S1 : LSN_i, S2 : *,..., Sn : *)) is received, the corresponding standby site 104(1) will determine whether the transaction Trx_1 can be committed at the standby site 104(1) based on the contents of the SDV 108(1) in the received SDV 108(1) (e.g., (S1 : LSN_i, S2 : *,..., Sn : *)).

[0051] In this regard, the standby site 104(1) analyzes all of the components included in the received SDV 108(1) (e.g., “S1 : LSN_i”, “S2 : *”) and finds that the value corresponding to the primary site S1 (e.g., S1 : LSN_i) is greater than 0 and the values corresponding to the other primary sites 102(2) through 102(n) (e.g., *) are empty (“*”). The standby site 104(1) will defer committing the transaction TRX_1 until it receives information indicating that the LSN values for the other sites are at least equal to LSN_i. The LSN information can come from one of two sources: in the SDV 108(1) received from the primary site 102(1), or by the standby site 104(1) directly polling another standby site 104(2). When the standby site 104(1) polls the other site standby site 104(2), it will update each value in the corresponding SDV 108(1) slot to the highest known LSN for each respective site.

[0052] In this regard, since each primary site 102(i) provides its corresponding standby site 104(i) with a corresponding commit action redo log 106(i) that includes a corresponding SDV 108(i), each standby site 104(i) can determine when and whether a transaction should be committed based on the value of the LSN included in the received SDV to support future possible asynchronous data recovery. This approach can help improve the accuracy of performing asynchronous data recovery at the standby site by using the vector as a Lamport clock in the received redo log, which in turn can result in a reduced RPO.

[0053] For the case of the number of primary sites n = 2, an example will now be described in the context of a multi-site transaction. In this regard, Figure 2B An example is shown of using the SDV vectors 108(1), 108(2) as Lamport clocks with respect to a first transaction Trxl and a second transaction Trx2. Figure 2B SDV vectors 108(1) and 180(2) are shown for the two primary sites 102(1) and 102(2), respectively, at different consecutive time periods (e.g., times Tl,..., T3), where Tl refers to the duration of time up to time T2, and so on. At time TO, both primary sites 102(1) and 102(2) are initialized, and their corresponding initialized SDVs 108(1), 108(2) are both filled with null values, denoted as (Sl:*, S2:*).

[0054] The instructions for the first transaction arrive at the coordinator 122, which assigns an incremental transaction ID, "Trx_l", to the first transaction. The first transaction, Trx_l, is a single-site transaction that includes an action performed at primary site 102(2) (e.g., add 10 books to site 2 inventory) and no action performed at primary site 102(1). By time Tl, all database change actions for the first transaction, Trx_l, are complete, and the corresponding redo logs have been sent to its standby site 104(2). Primary site 102(2) performs the commit action for the first transaction, Trx_l, and updates its SDV 108(2) to include the local LSN generated by primary site 102(2) for the commit action log 106C(2) of the first transaction, Trx_l (denoted as LSN_Trxl). Thus, at time Tl, the SDV 108(2) can be denoted as: (Sl:*, S2:LSN_Trxl). The component "S2:LSN_Trxl" of the vector (Sl:*, S2:Trx_l) identifies the LSN number of the last commit action performed by primary site 102(2). Primary site 102(2) sends a redo log message 112(2) to its corresponding standby site 104(2), which includes the commit action redo log 106C(2) and the current SDV 108(2) (Sl:*, S2:Trx_l).

[0055] The coordinator 122 receives the instructions for the second transaction and assigns an incremental transaction ID, "Trx_2", to the second transaction. In the example shown, the second transaction, Trx_2, includes a database change action that requires a change to the database of primary site 106(1) (e.g., add 5 books to site 1 inventory) and a retrieval action that requires information to be retrieved from the database of primary site 106(2) (e.g., does site 2 already have at least 5 books?). In this regard, the second transaction, Trx_2, includes a condition that the change action is only performed at primary site 102(1) if the response to the retrieval action from primary site 102(2) satisfies a defined criteria (e.g., only add 5 books to site 1 inventory if site 2 already has at least 5 books).

[0056] Before time T2, the coordinator 122 informs the primary site 102(1) of the incremental transaction ID for the second transaction (i.e., Trx_2). During time T2, the primary site 102(1) provides a request for information from the primary site 102(2) and receives a response from the primary site 102(2). In at least some examples, the request and response are facilitated by the coordinator 122. Further, as part of the response, the primary site 102(1) also receives a current copy of the SDV 108(2) of the primary site 102(2). The primary site 102(1) updates its own SDV 108(1) based on information included in the SDV 108(2) received from the primary site 102(2) by comparing each element individually and updating each element to the largest LSN value. In the illustrated example, the entry "S2:LSN_Trxl" in the SDV 108(2) will have a larger offset value in the transaction log stream value than the "null" value "S2:*" currently stored in the element position corresponding to the primary site 102(2) in the SDV 108(1). Thus, the primary site 102(1) updates its own SDV 108(1) to (Sl:*, S2:LSN_Trxl).

[0057] At time T3, all of the retrieval and change actions for the second transaction Trx_2 are complete, the corresponding redo log messages have been sent to the standby site 102(1), the primary site 102(1) performs the commit action for the second transaction Trx_2, and updates its SDV 108(1) to include the LSN for the commit action it performed for the second transaction Trx_2. Thus, at time T3, the SDV 108(1) can be represented as: (Sl:LSN_Trx2, S2:LSN_Trxl). The component "Sl:LSN_Trx2" of the vector (Sl:LSN_Trx2, S2:LSN_Trxl) identifies the local LSN of the last commit action for the primary site 102(1). The primary site 102(1) also provides a redo log message 112(1) for its corresponding standby site 104(1), which includes the commit action redo log 106C(1) for the transaction Trx_2, and the SDV 108(1) (Sl:LSN_Trx2, S2:LSN_Trxl).

[0058] Since the second primary site 102(2) does not perform any actions that require changes to its database or redo logs from time Tl to time T3, the SDV 108(2) for the second primary site 102(2) remains the same after time Tl (i.e., SDV 108(2) = (Sl:*, S2:LSN_Trxl) at times Tl, T2, and T3) in the example. Figure 2B

[0059] ​With respect to the first corresponding standby site 104(1) and the second corresponding standby site 104(2), the standby sites 104(1) and 104(2) receive the updated SDVs 108(1) and 108(2), respectively, provided at times T3 and Tl, respectively.

[0060] When receiving the commit action redo log 106C(2) provided by the corresponding primary site 102(2) at time Tl, the standby site 104(2) can compare the newly received SDV 108(2) (e.g., (S1 :*, S2:LSN_Trxl)) with its existing SDV (e.g., (S1 :*, S2:*)), and determine that the corresponding LSN value for its primary site 102(2) has changed from "*" to "LSN_Trxl," and that no other values in the SDV have changed. After time T3, if the standby site 102(2) polls the standby site 104(1), it will determine that the current SDV vector 108(1) is (S1 :LSN_Trx2, S2:LSN_Trxl), and update its own SDV vector accordingly. Assuming that the value of LSN_Trxl is less than or equal to LSN_Trx2, the standby site 102(2) will determine that it can commit transaction Trx 1.

[0061] Accordingly, the corresponding SDVs are in the form of vector clocks, enabling the standby sites 102 to determine whether transactions they are backing up are dependent on other transactions, and whether those other transactions have been successfully committed. This can support data recovery by keeping the backup sites 104 closely aligned with their corresponding primary sites. The SDVs 108 use a relatively small amount of memory and transmission resources (with being if only sent with the commit action redo logs) to track transaction dependencies between multiple sites, and thus have lower storage space requirements. In at least some applications, the use of SDVs enables computer resources used in the system 100 to ensure accurate recovery on the standby sites.

[0062] Reference is now made to Figure 3 , showing an example of a transaction Trx2 that requires a change in action on multiple primary sites 102(1), 102(2), provided by the example embodiments.

[0063] Transactions submitted on multiple primary sites (e.g., cross-storage transactions) typically rely on a two-phase commit (2PC) protocol, which requires a computer-implemented coordinator 122 to coordinate the actions of the sites participating in the transaction. A 2PC transaction includes a prepare phase and a commit phase. In the prepare phase, the participants (e.g., multiple primary sites) perform their respective actions without writing the results to permanent database storage (e.g., optical disk), including all necessary steps to prepare the resources for committing the transaction, and then notify the coordinator 122. In the commit phase, based on the received notifications of the participants’ preparations (e.g., votes), the coordinator decides whether to commit (if all participating sites voted “yes”) or abort the transaction, and notifies all participants of the decision. The participants then implement the decision (e.g., commit or abort the transaction) using the prepared resources. In some examples, in the distributed computer database system 100, a network node is designated as the coordinator 122 (which can also be a primary site or a different site), and multiple primary sites associated with the transaction are designated as participants.

[0064] Figure 3 An illustrative 2PC transaction Trx2 involving two primary sites 102(1) and 102(2) and a coordinator 122 is shown in FIG. 1C. Prior to time TO, both primary sites have committed a prior transaction Trxl, but are unaware that the other site committed transaction Trxl. Thus, the SDV 108(1) of primary site 102(1) is (S1:LSN_Trx1, S2:*) and the SDV 108(2) of primary site 102(2) is (S1:*, S2:LSN_Trx2). When the first primary site 102(1) and the second primary site 102(2) commit transaction Trxl, the SDVs 108(1) and 108(2) of these have previously been provided to the first standby site 104(1) and the second standby site 104(2), respectively. At time TO, the first primary site 102(1) has prepared the resources needed to perform transaction Trx2 and performed all actions included in transaction Trx2 in a buffer, except for the commit action. The redo logs 106(1) for each pre-commit action have been sent to the first standby site 104(1). The first primary site 102(1) also provides the coordinator 122 with a notification that it has prepared transaction Trx2 and provides the coordinator 122 with its current SDV vector 108(1) (S1:LSV_Trx1, S2: *).

[0065] Similarly, the second primary site 102(2): prepares transaction Trx2 and provides the coordinator 122 with a notification that it has prepared transaction Trx2 and provides the coordinator 122 with a copy of the SDV vector 108(2) (S1:*, S2:LSV_Trx1).

[0066] Once the coordinator 122 receives the notifications from all of the primary sites 102(1), 102(2) participating in the transaction Trx2, the coordinator 122 decides whether the primary sites collectively vote to commit the transaction Trx2 or abort the transaction 2. If the coordinator 122 determines that the transaction Trx2 is to be committed, the coordinator 122 extracts the respective maximum commit action LSN value for each primary site from the respective slot location in each SDV 108(1), 108(2) and merges all of the extracted maximum commit action LSN values into a merged SDV 124, Figure 3 denoted as (S1 :LSN_Trx1, S2 :LSN_Trx1). Note that S1 :LSN_Trx1 would be the local LSN generated by the primary site 102(1) for a commit action corresponding to the transaction Trx1 and S2 :LSN_Trx1 would be the local LSN generated by the primary site 102(2) for a commit action corresponding to the transaction Trx1, and thus, these two LSN values can not be equal, but would represent the position of each of the respective primary sites 102(1) and 102(2) in committing transactions included in a contiguous group of transactions (e.g., a transaction stream). The coordinator 122 provides a message to the first primary site 102(1) and the second primary site 102(2) (i) notifying the sites to proceed with committing the transaction Trx2 accordingly and (ii) including a copy of the merged SDV 124. After receiving the message (e.g., at time T2), each primary site 102(1) and 108(2) updates its respective SDV 108(1) and 108(2) based on the merged SDV 124. For example, after the update, both vectors 108(1), 108(2) are updated to be consistent with the merged SDV 124, (S1 :LSN_Trx1, S2 :LSN_Trx1).

[0067] Each primary site 102(1), 102(2) then enters a commit phase and commits the transaction Trx2 (e.g., writes the transaction to disk). After the commit phase is complete (e.g., at time T3), each primary site 102(1), 102(2) respectively: (i) generates a respective commit action redo log 106C(1), 106C(2) for the commit action of transaction Trx2; (ii) updates its respective SDV 108(1), 108(2) to include the LSN from the commit update log of transaction Trx2; and (iii) provides a respective redo log message 110(1), 110(2) (including the respective commit action redo log 106C(1), 106C(2) and updated SDV 108(1), 108(2), respectively) to its respective standby site 104(1), 104(2). In the case of primary site 102(1), at time T3, the updated SDV 108(1) will be: (S1:LSN_Trx2, S2:LSN_Trx1), indicating that primary site 102(1) is currently aware that transaction Trx2 has been committed at primary site 102(1). In the case of primary site 102(2), at time T3, the updated SDV 108(2) will be: (S1:LSN_Trx1, S2:LSN_Trx2), indicating that primary site 102(2) is currently aware that transaction Trx2 has been committed at primary site 102(2).

[0068] In the absence of any failures, standby site 104(1) will receive SDV 108(1) (S1:LSN_Trx2, S2:LSN_Trx1); standby site 104(2) will receive SDV 108(2) (S1:LSN_Trx1, S2:LSN_Trx2). If the redo logs and SDVs are received as expected, transaction Trx2 will execute at standby sites 104(1), 104(2), bringing the standby sites and databases into close alignment with the primary sites and databases. However, if the SDVs are not received or include lower than expected LSV values, standby sites 104(1), 104(2) can notify the coordinator site, which will then determine whether to abort, or take some other action (e.g., wait) with respect to transaction Trx2.

[0069] In the above embodiments, each of the group of primary sites 102 uses a respective SDV 108(i) as a transaction order indicator to track the respective position or offset of the commit action of the group of interacting primary sites 102 in the transaction stream. As described above, the SDV 108(i) of each site includes a respective value element for each primary site 102(i) in the group of interacting primary sites 102. Each primary site 102(i) tracks its own position in the transaction stream by setting its own corresponding commit action LSN in its SDV 108(i) to the LSV of the last commit action recorded in the commit action redo log 106C(i) of the primary site 102(i). Each primary site 102(i) tracks its own position in the transaction stream of the other primary sites 102 by setting the value elements in its SDV 108(i) for the other primary sites 102 based on received up-to-date site transaction stream position information for the other primary sites 102. This transaction stream position information can be obtained indirectly from the other primary sites 102 through the coordinator 122 (e.g., in the case of 2PC transactions of Figure 3 or, in some examples, directly from the other primary sites 102 (e.g., in the case of coordinator-less transactions of Figure 2B In examples, each time a primary site 102(i) sends a redo log 106(i) for a commit action to its backup site 104(i), it includes the latest version of its SDV 108(i).

[0070] At the corresponding backup sites 104, the received SDVs can be used to verify that the site backups between the multiple sites occurred in a manner that satisfies the transaction dependency requirements, and divergence between the primary sites and the backup sites is minimized. In at least some examples, the use of this Lamport clock-like synchronization for dependency relationships of the primary sites 102 can eliminate the need for checkpointing between the backup sites. The above-described Lamport clock-like synchronization method and system has some overhead, as it requires storage and updating of the SDVs at each site. However, in at least some examples, the above-described method and system enables redo logs to be applied on backup sites with minimal latency (i.e., small RPO) and minimal RTO, while satisfying atomicity requirements and dependency requirements.

[0071] A further example will now be described in which the SDV uses a commit sequence number (CSN) instead of an LSN as the transaction order indicator to track the position in the transaction stream. In the example of Figure 4A the primary sites 102 cooperate with the coordinator 122 to obtain the transaction ID and CSN of transactions committed at the primary sites 102. As described in Figure 4AAs shown, at time TO, the coordinator 122 receives instructions regarding a multi-site transaction involving primary sites 102(i) and 102(i+1). The coordinator 122 assigns an incremental transaction ID (e.g., Trx_j) to the transaction and notifies the primary sites 102(i) and 102(i+1) of the assigned transaction ID. The primary sites 102(i) and 102(i+1) each perform a prepare phase of the transaction Trx_j and then send respective prepare messages (e.g., at times T2 and T2', respectively) to the coordinator 122 indicating that the primary sites 102(i) and 102(i+1) are each ready to commit the transaction Trx_j. Upon receiving notification from all participating primary sites 102 that they are ready to commit the transaction Trx_j, the coordinator 122 increments a global transaction commit sequence number (CSN) to assign a CSN number to the commit transaction Trx_j (e.g., CSN of transaction Trx_j = CSNk).

[0072] The coordinator 122 then notifies each primary site 102(i), 102(i+1) that all sites are ready to commit the transaction Trx_j and the CSN assigned to the transaction (e.g., CSNk). Upon receiving the commit notification and the CSN of the transaction to commit the transaction Trx_j from the coordinator 122, each participating primary site 102(i), 102(i+1) (i) proceeds to commit the transaction Trx_j; (ii) generates a respective commit action redo log 106C(i), 106C(i+1) that is added to its local redo log file 110; and (iii) sends a respective redo log message 112(i), 112(i+1) (each including the respective commit action log record 106(i), 106(i+1) and the CSN value assigned to the transaction Trx_j) to its corresponding standby site 104(i), 104(i+1).

[0073] As will be explained in greater detail below, the redo log message 110(i) including the commit action redo log 106C(i) received at the standby site 104(i) includes the transaction ID and the CSN of the commit transaction (e.g., Trx_j, CSNk). The standby site 104(i) can compare the CSN information included in the commit action redo log 106C(i) of the transaction with CSN information received by the standby site 104(i) from other standby sites 104 to determine whether and when the transaction should be committed at the standby site 104(i). Including the CSN and transaction ID pair in the redo log message 110(i) of the commit transaction can enable atomicity requirements and dependency requirements to be accurately met. Moreover, RTO performance can be improved since the coordinator is responsible for coordinating actions of the participating sites.

[0074] Figure 4BTable 401 and sequence diagram 402 are illustrative examples showing different assigned transaction ID / CSN pairs. Each transaction is assigned a unique transaction ID and a unique CSN, with subsequent committed transactions in the transaction flow having a higher CSN value than previously committed transactions. For example, transactions 4 and 5 (transaction IDs Trx4 and Trx5) have a CSN value of 6, while transaction 5 has a CSN value of 5. Therefore, if the standby site receives a redo log update that includes a relationship identifying transaction ID Trx4 as corresponding to a CSN value of 6, and another redo log update that includes a Trx ID / CSN pair (Trx5:CSN 5), the standby site can determine that transaction 5 committed before transaction 4 by comparing the CSN values ​​from the received redo log updates (e.g., CSN 6 > CSN 5).

[0075] For two transactions with a dependency relationship (e.g., the second transaction depends on the first transaction), the CSN value of the second transaction will be greater than the CSN value of the first transaction. For example, if a standby site receives a redo log in which transaction Trx2 is assigned a CSN value of 3, the standby site can assume that transaction Trx2 will not depend on any other transactions with a CSN greater than 3.

[0076] The atomicity and dependency correctness theorems of the CSN-based transaction log stream tracing method can be explained as follows: For a transaction TrxA with a CSN value of X, if the maximum CSN of each standby site in the standby site group is at least as large as X, then transaction TrxA can be committed and the atomicity and dependency requirements are satisfied.

[0077] Now we will combine Figure 4C and Figure 4D An example is described in which a standby site uses a CSN value in the context of a 2PC transaction, wherein the distributed database system 100 includes a first standby site 104(1), a second standby site 104(2), and a third standby site 104(3).

[0078] exist Figure 4C In the middle, at time T0, the first backup station 104(1) and the second backup station 104(2) respectively start from the first main station 102(1) and the second main station 102(2). Figure 4CThe third standby site 104(3) receives a commit action redo log 106C(3) from the third primary site 102(3) that includes the CSN value of Trxl that has committed and assigned CSNl (e.g., indicating that the first transaction Trxl that assigned CSNl has committed at the third primary site 102(3)). Each of the first standby site 104(1), the second standby site 104(2), and the third standby site 104(3) track the transaction IDs for which they respectively received commit action redo logs and the CSN values assigned to those transactions. In addition, each of the first standby site 104(1), the second standby site 104(2), and the third standby site 104(3) can learn from the other standby sites 104(1), 104(2), and 104(3) what the maximum CSN value each of the other standby sites knows. In different examples, the transmission of this CSN information between the standby sites 104(1), 104(2), and 104(3) can occur through one or more of: (i) as part of standby site to standby site communications as part of transactions; (ii) a polling or reporting mechanism in which the standby sites poll each other or report to each other to determine the maximum CSN value each site sees; and / or (iii) through a standby coordinator 422, which can be present in some examples, to collect and disseminate information among the standby sites.

[0079] In the example embodiment, based on its own maximum CSN value and the maximum CSN value information it receives about the other standby sites, each standby site 104(i) can determine that all of the standby sites 104(1) through 104(3) have been informed by the respective primary sites 102(1) through 102(3) up to the point of the minimum of the maximum CSN values. The minimum of the maximum CSN values is referred to as the “consistent point (CP)” value. Thus, at time TO, the highest value CSN known by the standby sites 104(1) and 104(2) is the null value “*”. Based on the commit action redo log 106C(3) received from the primary site 102(3) for transaction Trxl, the standby site 104(3) knows the maximum CSN value of CSNl. Thus, at time TO, the highest value CSN known by the standby site 104(3) is CSNl. The consistent point (CP) value is the minimum of the maximum CSN values known by each of the standby sites 104(1), 104(2), and 104(3). Thus, the consistent point (CP) value is the minimum of CSNl and the null value “*”, which is the null value “*”. Thus, at time TO, the consistent point (CP) value is the null value “*”. Figure 4CIn the example, at time T0, the CP value (i.e., the minimum of all maximum CSN values ​​observed in the redo logs of the committed actions received by each of the standby sites 104(1), 104(2), and 104(3) so far) is a null value "*" because neither the primary site 104(1) nor 104(2) has received a CSN value. Each of the corresponding standby sites 104(1) through 104(3) can determine whether it can commit any transactions that have already received a CSN by comparing the received CSN number with the CP value. Figure 4C In the example shown, at time T0, backup sites 104(1) and 104(2) have no pending transactions with CSN numbers. Backup site 104(3) has received a notification of CSN1, but CSN1 is greater than the CP value of the group of backup sites 104(1) to 104(3) = "*", so backup site 104(3) chooses not to commit Trx1 at time T0. In the example where a backup coordinator 422 is present, the backup coordinator can collect the information needed to determine the CP value and then propagate that information to the corresponding backup sites 104(11) to 104(3). In the example without a backup coordinator, backup sites 104(1) to 104(3) can collect the information needed to determine the CP value directly or indirectly from each other.

[0080] exist Figure 4C At time T1, each of the first standby site 104(1) and the second standby site 104(2) receives from the first primary site 102(1) and the second primary site 102(2) a corresponding redo log message for transaction Trx2, including redo logs 106C(1) and 106C(3) for commit actions, and a CSN value CSN3. This indicates that the first primary site 102(1) and the second primary site 102(2) have each committed transaction Trx2, and that transaction Trx2 has been assigned CSN = 3 by the primary site coordinator 122. The maximum CSN values ​​observed on standby sites 104(1) to 104(3) are CSN3, CSN3, and CSN1, respectively.

[0081] Therefore, at time T1, the smallest maximum CSN is CSN=1, so the CP value is CP=1. Each of the standby stations 104(1) and 104(2) will determine that the CSN value of its corresponding transaction is greater than the current CP=1 value, and will choose not to commit any transaction. However, standby station 104(3) will determine that its transaction Trx1:CSN1, which has not yet been committed locally, has a CSN value less than or equal to the current CP=1 value, so standby station 102(3) will choose to commit transaction Trx1.

[0082] exist Figure 4CAt time T2, the third standby site 104(3) receives from the third primary site 102(3) another redo log message 112(3) including the commit action redo log 106C(3) indicating that transaction Trx4 can be committed at the third primary site 102(3) and assigned a CSN value CSN6. Thus, at time T2, the minimum of the maximum CSN with commit action redo logs received by each of the standby sites 104(1) to 104(3) is CSN=3, and thus the CP value is CP=3 at time T2. Standby sites 104(1) and 104(2) can decide to commit transaction Trx2:CSN3 because CSN=3 is equal to or less than CP=3. Standby site 104(3) will defer committing Trx4:CSN6 because CSN=6 is greater than CP=3.

[0083] Figure 4D An example is shown in which the redo log messages identifying committed transactions and their assigned CSNs are received at the standby sites 104 in an order different from the order in which the corresponding transactions are committed at the primary sites 102. Figure 4D Similarly to Figure 4C At time T1, instead of receiving a message identifying the commit action redo log 106C(2) for Trx2:CSN3 at standby site 104(2), the standby site 104(2) receives a commit action redo log 106C(2) for transaction Trx5 that has been assigned a CSN value CSN5. In this case, although transaction Trx2 with CSN3 is committed at primary site 102(2) before transaction Trx5 with CSN 5, the commit action redo log corresponding to Trx5:CSN5 is received at the second standby site 104(2) before the commit action redo log corresponding to Trx2:CSN3. Thus, at time T2, the CP value will be CP=3. Standby site 104(1) can commit transaction Trx2:CSN3 because CSN=3 is equal to or less than CP=3. Standby site 104(3) will defer committing Trx4:CSN6 because CSN=6 is greater than CP=3. Standby site 104(2) will defer committing Trx5:CSN5 because CSN=5 is greater than CP=3.

[0084] The processing of transaction Trx2 by standby site 104(2) will now be described. It is noted that, in Figure 4DIn the example, standby site 104(2) receives a redo log message for the action (but not the commit action) of transaction Trx2 from its primary site 102(2) at time T0. This indicates that primary site 102(2) is at least in the preparation phase of transaction Trx2 at time T0. However, by time T2, standby site 104(2) has not yet received the commit action redo log for transaction Trx2 with a CSN value from its primary site 104(2). As mentioned above, the first standby site 104(1), which also involves transaction Trx2, can commit Trx2:CSN3 because it has already received the commit action redo log for transaction Trx2, including Trx2:CSN3 (and CSN3 <= CP = 3). In some examples, the following combination can be interpreted as indicating that the second standby site 104(2) can also commit transaction Trx2 at time T2, even if the second standby site 104(2) has not yet received the commit action redo log update for transaction Trx2: (i) the second standby site 104(2) receives the preparation phase action redo log for Trx2 at time T0; (ii) the second standby site 104(2) receives the commit action redo log for transaction Trx5:CSN5 at time T1; and (iii) the first standby site 104(1) receives the commit action redo log for transaction Trx2:CSN3 at time T2.

[0085] Therefore, in at least some examples involving 2PC transactions, the above theorem can be extended to include transactions that have been prepared on multiple standby sites, but whose CSN numbers are received only on one or some standby sites, not all standby sites. In this case, the atomicity and dependency correctness theorems of the CSN-based transaction log stream tracing method can be stated as follows: For a 2PC transaction TrxA with a CSN value of X, if every standby site in the standby site group sees a maximum CSN at least as large as X, and at least one standby site in the group sees the CSN value of X, then the 2PC transaction TrxA can be committed on all standby sites that have been prepared for transaction TRxA.

[0086] Therefore, in the exemplary embodiment, standby stations 104(1) to 104(3) each track the current global CP, and these standby stations can each use the global CP to determine whether to commit a transaction that the standby station has received its redo log.

[0087] Figure 4E Another example of CP-based processing is shown in the context of a single-site transaction committed at the corresponding master site. In this example, the distributed database system 100 includes two master sites 102(1) and 102(2). Figure 4Egroup of primary site 102(1) and its corresponding first standby site 104(1) and second standby site (2) (not shown). In the example shown, transactions Trx2, Trx3, and Trx4 commit at the primary site. Transaction Trx2, Trx4 execute at primary site 102(1) and the primary coordinator 122 assigns CSN values 2 and 4, respectively. Transaction Trx3 executes at primary site 102(2) and assigns CSN value 3. At time TO, the first standby site 104(1) receives redo log message 112(1) that includes commit action redo log 106C(1) for transaction Trx2 with CSN value CSN2. At time Tl, the first standby site 104(1) receives a message that includes commit action redo log 106C(1) for transaction Trx4 with CSN value CSN4 and the second standby site 104(2) receives a message that includes commit action redo log 106C(2) for transaction Trx3 with CSN value CSN3. Thus, at time Tl, the current CP value is CP = 3. At time Tl, the first standby site 102(1) has two transactions waiting to commit, namely transaction Trx2 (with CSN2) and transaction Trx4 (with CSN4). Assuming that the CSN value CSN2 for transaction Trx2 is less than the current CP = 3, but the CSN value CSN4 for Trx4 is greater than the current CP = 3, at time Tl, the first standby site 102(1) will commit transaction Trx2, but will delay committing transaction Trx4. For the second standby site 104(2), the CSN value CSN3 for transaction 3 is equal to or less than the current CP = 3, so the standby site 104(2) will commit transaction Trx3.

[0088] Two schemes for tracking the position of transactions in a transaction stream are described above. In one scheme, the Lamport clock method based on SDVs is applied, where a vector on each master site is used to store information about the relative position of all sites in the transaction stream, and the local LSN of a commit action indicates the relative transaction offset in the transaction stream. In the second scheme, the transaction ID and CSN assigned by the coordinator are used to indicate the offset of a transaction in the transaction stream. In another example, a hybrid scheme that relies on both SDVs and transaction ID / CSN pairs can be employed. For example, the distributed computer system 100 can be used to provide levels of backup processing based on the importance of the transactions being backed up. In such an example, the master sites and / or the master coordinator 122 can classify transactions as "normal transactions" or "important transactions" based on predetermined importance criteria or thresholds. For example, the criteria can be defined based on one or more of the identity of the parties involved in the transaction, the size of the transaction, the nature of the items represented in the transaction, and / or other criteria and combinations thereof. In the case of normal transactions that do not meet the importance criteria or thresholds, transaction stream tracking can be performed based on the transaction ID / CSN values alone. In the case of important transactions that meet or satisfy the importance criteria, transaction stream tracking can be performed on the master site side and the backup site side of the database system 100 using the transaction ID / CSN values and SDVs. For example, when an important transaction is recognized as being prepared at one or more master sites, SDV tracking can be added for the important transaction and other transactions that occur within a defined time near the important transaction. The additional SDV information can be sent in the site-specific commit action redo log along with the CSN value assigned to the committed transaction. If the transaction CSN value (and thus the CP value derived therefrom) does not support backup updates for the important transaction to be processed, the backup site can refer to the SDV information, which can enable the transaction to be properly backed up.

[0089] Figure 5 An example processing system 500 is shown that can be used to implement the methods and systems described herein, such as instances of master sites 102, backup sites 104, coordinators 122, and coordinators 422 in a distributed computer system, such as database system 100. Other processing systems suitable for implementing the methods and systems described in this disclosure can be used, which can include different components than those discussed below. Although Figure 5 A single instance of each component is shown, but multiple instances of each component can be present in the processing system 500.

[0090] The processing system 500 can include one or more processing devices 502, such as a processor, microprocessor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), dedicated logic circuitry, or a combination thereof. The processing system 500 can also include one or more input / output (I / O) interfaces 514 to enable connection with one or more optional input devices and / or output devices (not shown). One or more of the input devices and / or output devices can be included as components of the processing system 500, or can be external to the processing system 500. The processing system 500 can include one or more network interfaces 508 for wired or wireless communication with a network. In an example embodiment, the network interfaces 508 include one or more wireless interfaces, such as a transmitter capable of communicating in a network. The one or more network interfaces 508 can include interfaces for wired links (such as Ethernet wires) and / or wireless links (such as one or more antennas) for intra-network and / or internetwork communication. The one or more network interfaces 508 can provide wireless communication through one or more transmitters or transmit antennas, one or more receivers or receive antennas, and various signal processing hardware and software. In this regard, some of the network interfaces 508 can include a respective processing system similar to the processing system 500. In the present example, a single antenna 516 is shown, which can be used as a transmit antenna and a receive antenna simultaneously. However, in other examples, multiple separate antennas can be present for transmission and reception.

[0091] The processing system 500 can also include one or more storage devices, such as a storage unit 513, which can include non-transitory storage elements, such as solid-state drives, hard disk drives, disk drives, and / or optical drives. The storage devices of the processing system 500 can include one or more memories 510, which can include volatile or non-volatile memory (e.g., flash memory, random access memory (RAM), and / or read-only memory (ROM)). The storage devices (e.g., the storage unit 513 and / or the one or more non-transitory memories 510) can store instructions for execution by the one or more processing devices 502, such as to perform the present application. The one or more memories 510 can include other software instructions, such as software instructions for implementing an operating system and other applications / functions. In some examples, one or more data sets and / or modules can be provided by external memory (e.g., an external drive in wired or wireless communication with the processing system 500) and / or by a transitory or non-transitory computer-readable medium. Examples of non-transitory computer-readable media include RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, CD-ROM, or other portable memory.

[0092] There can be a bus 514 providing communication among the components of the processing system 500, including the one or more processing devices 502, the one or more optional I / O interfaces 504, the one or more network interfaces 508, the one or more storage units 513, and / or the one or more memories 510. The bus 514 can be any suitable bus architecture, such as including a memory bus, a peripheral bus, or a video bus.

[0093] In some examples, the processing system 500 can be applied to Figures 1 to 4Eeach primary site 102 discussed in the examples. If changes are made to the primary site 102, the redo log 106 including the vector 108 of LSNs that make up the transaction offset for each associated primary site can be stored in the storage unit 513 or the memory 510. In cases where CSNs are used to record committed transactions at the primary site 102, the storage unit 513 or the memory 510 can store the redo log 106 including an assignment relationship that identifies each committed transaction ID corresponds to a CSN. The primary site 102 continuously sends updates regarding the stored redo log 106 to the corresponding standby site through the network interface 508 in order to support data backups that will support asynchronous data recovery.

[0094] In some examples, the processing system 500 can be applied in each standby site 104, as discussed in the examples of Figures 1 to 4E The standby site 104 receives the sent updates to the redo log 106 through the network interface 508 and determines whether to replay the received redo log 106 using the processing device 502 or roll back the received redo log 106.

[0095] In some applications, the processing system 500 can be used to implement the primary coordinator 122 and / or the standby coordinator 122' to coordinate actions between the primary sites and the standby sites, respectively. In at least one configuration, before changes are made (e.g., committed transactions) at the primary sites, the primary coordinator 122 determines which primary sites and how many primary sites are involved in or participate in the transactions using the processing device 502. In this way, the number of LSNs in the SDV 108 is determined based on the number of primary sites that participate in the transactions.

[0096] The present disclosure provides certain example algorithms and calculations for implementing examples of the disclosed methods and systems. However, the present disclosure is not bound by any particular algorithm or calculation. Although the present disclosure describes methods and processes by steps performed in a particular order, one or more steps of the methods and processes can be omitted or changed as appropriate. In appropriate circumstances, one or more steps can be performed in an order other than that described.

[0097] Through the description of the above embodiments, the present application can be implemented by hardware only, by software and necessary general hardware platform, or by a combination of hardware and software. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transitory storage medium, which can be a compact disk read-only memory (CD-ROM), a USB flash drive, or a hard disk. The software product includes many instructions, which enable a computer device (personal computer, server, or network device) to perform the method provided in the embodiments of the present application.

[0098] Although the present application and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the application as defined by the appended claims.

[0099] In addition, the scope of the present application is not limited to the specific embodiments of the processes, machines, manufactures, compositions of matter, means, methods and steps described in the specification. Those of ordinary skill in the art will readily understand that the present application can be practiced with processes, machines, products, compositions of matter, means, methods or steps that are different from those described in this document, or that are currently known or developed in the future, which have substantially the same function or achieve substantially the same results as the corresponding embodiments described herein. Therefore, the scope of the appended claims includes these processes, machines, products, compositions of matter, means, methods or steps.

Claims

1. A method, characterized in that, The method includes: The distributed database system's master site group includes a first master site, which generates a commit action redo log message for a commit action performed on a first database transaction. The commit action redo log message includes the transaction ID of the first database transaction and a transaction order indicator indicating the order of database transactions in the database transaction group executed on one or more master sites in the master site group. The transaction order indicator includes a commit sequence number assigned to the first database transaction by the coordinator, which indicates when the first database transaction is committed on one or more master sites relative to other database transactions included in the database transaction group. The first primary site sends the commit action redo log message to the corresponding first backup site that backs up the first primary site; The system receives the commit action redo log message at the first backup site and determines when to commit the first database transaction based on a comparison between the commit sequence number assigned to the first database transaction and the commit sequence numbers included in other commit action redo log messages received at other backup sites.

2. The method according to claim 1, characterized in that, The transaction sequence indicator includes a vector that includes the corresponding log sequence number of each master site in the master site group, the log sequence number of each master site corresponding to the commit action performed by the corresponding master site.

3. The method according to claim 2, characterized in that, include: The first master site receives the log sequence number corresponding to the commit action performed by other master sites in the master site group. Wherein, the log sequence number included in the transaction sequence indicator for each master site corresponds to the last commit action performed by the master site that is known to the first master site.

4. The method according to claim 3, characterized in that, include: The first master site receives a notification of the transaction ID of the first database transaction from the coordinator that assigns incremental transaction IDs to database transactions; The first master site provides the coordinator with a notification that the first master site is ready to commit the first database transaction. The log sequence number corresponding to the commit action performed by the other master sites in the master site group is received by the first master site from the coordinator.

5. The method according to claim 2, characterized in that, include: The first backup site receives the commit action redo log message and determines when to commit the first database transaction based on the log sequence number included in the transaction sequence indicator.

6. The method according to claim 1, characterized in that, The comparison includes comparing the commit sequence number assigned to the first database transaction with a consistency point value, wherein the consistency point value is the minimum commit sequence number of a group, the group comprising the maximum commit sequence number received by each standby site in the standby sites corresponding to the primary site group.

7. A first primary site included in a primary site group participating in database transactions, characterized in that, include: The processing system includes one or more processing units and one or more storage devices, wherein the one or more storage devices store instructions that, when executed by the one or more processing units, can be used to cause the first main site to perform operations including the following: A commit action redo log message is generated for the commit action performed by the first master site on the first database transaction. The commit action redo log message includes the transaction ID of the first database transaction and a transaction order indicator indicating the order of database transactions in the database transaction group executed on one or more master sites in the master site group. The transaction order indicator includes a commit sequence number assigned to the first database transaction by the coordinator, which indicates when the first database transaction is committed on one or more master sites in the master site group relative to other database transactions included in the database transaction group. Send the commit action redo log message to the corresponding first backup site that backs up the first primary site; The system receives the commit action redo log message at the first backup site and determines when to commit the first database transaction based on a comparison between the commit sequence number assigned to the first database transaction and the commit sequence numbers included in other commit action redo log messages received at other backup sites.

8. The first main site according to claim 7, characterized in that, The transaction sequence indicator includes a vector that includes the corresponding log sequence number of each master site in the master site group, the log sequence number of each master site corresponding to the commit action performed by the corresponding master site.

9. The first main site according to claim 8, characterized in that, When the instruction is executed by the one or more processing units, it causes the first main station to perform operations including the following: The first master site receives the log sequence number corresponding to the commit action performed by other master sites in the master site group. Wherein, the log sequence number included in the transaction sequence indicator for each master site corresponds to the last commit action performed by the master site that is known to the first master site.

10. The first main site according to claim 9, characterized in that, When the instruction is executed by the one or more processing units, it causes the first main station to perform operations including the following: The first master site receives a notification of the transaction ID of the first database transaction from the coordinator that assigns incremental transaction IDs to database transactions; The first master site provides the coordinator with a notification that the first master site is ready to commit the first database transaction. The log sequence number corresponding to the commit action performed by the other master sites in the master site group is received by the first master site from the coordinator.

11. The first main site according to claim 7, characterized in that, The comparison includes comparing the commit sequence number assigned to the first database transaction with a consistency point value, wherein the consistency point value is the minimum commit sequence number of a group, the group comprising the maximum commit sequence number received by each standby site in the standby sites corresponding to the primary site group.

12. A method for execution at a first standby site of a first primary site in a backup distributed database system, characterized in that, The distributed database system includes a group of master sites, each master site having a corresponding backup site, and the method includes: At the first backup site, a redo log message is received regarding a first transaction executed at the first primary site. The redo log message includes the transaction ID of the first transaction and a first transaction sequence indicator indicating the order of the first transaction in the transaction group committed by the primary site group. The first transaction sequence indicator includes a commit sequence number assigned to the first transaction by the coordinator, the commit sequence number indicating when the first transaction is committed at one or more master sites in the master site relative to other database transactions included in the transaction group, and the transaction sequence indicator received at other standby sites each indicates the commit sequence number of the transaction committed at the master site corresponding to the other standby site. Receive information at the first backup site regarding the transaction sequence indicator received at the other backup sites; Based on the redo log message and the information regarding the transaction sequence indicator received at the other standby site, it is determined when to commit the first transaction at the standby site.

13. The method according to claim 12, characterized in that, This includes determining a consensus point value, wherein the consensus point value is the minimum commit sequence number of a group, the group comprising the maximum commit sequence number received by each standby site in the standby sites corresponding to the primary site group, wherein determining when to commit the first transaction at the standby site is based on a comparison of the commit sequence number of the first transaction with the consensus point value.

Images