Using multiple blockchains to apply transactions to persistent data object sets in a persistent storage system.
By adopting an aggregated blockchain application approach in a persistent storage system, and utilizing consensus algorithms and a two-phase commit mechanism, the seriality problem of parallel transaction replication in blockchain applications is solved, achieving efficient parallel transaction processing and data consistency, and improving the throughput and security of the persistent storage system.
Patent Information
- Application Number
- CN202180062960.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-07-30
- Filing Date
- 2021-07-30
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-07-30
AI Technical Summary
The serial operation of blockchain application methods in traditional persistent storage systems hinders parallel transaction replication, resulting in low throughput and an inability to efficiently handle concurrent access to distributed ledger data.
By adopting an aggregated blockchain application approach, multiple blockchains are replicated in the network of each persistent data object. Through consensus algorithms and two-phase commit mechanisms, parallel application and efficient replication of transactions are achieved, thus solving the consistency problem of transaction dependencies.
It improves the parallelism and throughput of blockchain applications, enhances the replication efficiency of persistent data objects, and ensures data consistency and security.
Smart Images

Figure CN116134420B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to enterprise-level persistent storage systems, such as file systems, database management systems, key-value stores, and document storage systems, and particularly to natively implementing blockchain technology within an enterprise-level persistent storage system. Background Technology
[0002] Traditional persistent storage systems can handle large amounts of data read and / or modified by multiple concurrent users. Enterprise persistent storage systems (PSS) maintain the data integrity, consistency, and security of persistent data managed by the PSS.
[0003] PSS can be, for example, a database managed by a DBMS, a file stored in a file system, a key-value object managed by a key-value store, or a document (e.g., a JSON or XML document) managed by a document storage system (DOCS). PSS stores data in persistent data objects. For example, a set of persistent data objects can be a database table managed by a DBMS, key-value pairs managed by a key-value store, a collection of documents managed by a DOCS, or a file managed by a file system. A persistent data object can be a composite data object comprising multiple persistent data objects. For example, in a DBMS, a database table can include multiple rows as persistent data objects. In a file system, a file can include multiple blocks as persistent data objects, and a directory can be a persistent data object containing files and even other directories as persistent data objects. In a DOCS, a persistent data object can be a collection of JSON documents.
[0004] Various PSS (such as DBMS and key-value stores) offer superior performance for updating and querying PSS. PSS users can rely on the validity of persistent data objects stored in the PSS because changes to these objects are controlled by a centralized authority such as a company or organization. Typically, any changes made to data stored in the PSS are performed by users belonging to the same central authority and are therefore subject to its access policies.
[0005] For segregated entities (e.g., businesses, organizations, governments), it is becoming increasingly important to share read and write access to mission-critical shared data in a manner that allows entities to rely on the validity of the shared data. This type of data is referred to in this paper as distributed ledger data.
[0006] The approach used to implement distributed ledger data is a blockchain application approach. In this approach, distributed ledger data is replicated as persistent data objects at each PSS (Personal Subscriber Service) in a “ledger store of objects.” Changes to persistent data objects (“ledger objects”) are initiated but not committed by transactions at one of the PSSs (“initiator PSS”). The transaction and the corresponding changes are recorded in blocks added to the blockchain, which is replicated across each PSS. Other PSSs read information about the transaction from the blocks in the blockchain and replay the corresponding changes to the replica of the persistent data object at each of the other PSSs. The term replay means making changes without committing them. The changes are “approved” by the other PSSs. Once approved, the PSS commits the changes. The activity of approving the changes is recorded in blocks in the blockchain.
[0007] As shown above, blockchain application methods use the blockchain to propagate changes to a ledger store of objects. The problem with using the blockchain to propagate changes stems from the fact that adding a block to the blockchain is a serial operation, requiring multiple PSSs to participate in adding the block and approving the transaction. The serial nature of these operations hinders the throughput of replicating the ledger store of objects at the PSS, and in particular, impedes the ability to apply transactions in parallel to replicate transactions between PSSs. Attached Figure Description
[0008] In the attached diagram:
[0009] Figure 1A This is a diagram depicting a blockchain application network according to an embodiment of the present invention.
[0010] Figure 1B This is a diagram depicting the data structures used in embodiments of the present invention.
[0011] Figure 2 This is a diagram depicting a blockchain application process according to an embodiment of the present invention.
[0012] Figure 3 It is a diagram depicting a block in a blockchain according to an embodiment of the present invention.
[0013] Figure 4 This is a diagram depicting a blockchain application process using a dependency checkpointing method according to an embodiment of the present invention.
[0014] Figure 5 This is a diagram illustrating the process of avoiding replication timing inconsistencies of unique constraints by using tombstoning and induced transaction dependencies according to an embodiment.
[0015] Figure 6This is a diagram illustrating the process of avoiding replication timing inconsistencies of foreign key constraints by using transaction dependencies induced according to an embodiment.
[0016] Figure 7 This is a diagram depicting a computer system that can be used to implement embodiments of the present invention.
[0017] Figure 8 A software system that can be used to control the operation of a computer system according to an embodiment of the present invention is described. Detailed Implementation
[0018] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the invention. However, it will be apparent, however, that the invention may be practiced without these specific details.
[0019] General Overview
[0020] This paper describes a method for replicating a ledger store of persistent data objects on a PSS network using blockchain applications, achieving greater parallelism for replicating changes using blockchain applications. This method is referred to herein as an aggregated blockchain application because the method relies on an aggregated blockchain. The aggregated blockchain comprises multiple blockchains, each replicated on every PSS in the PSS network. Any PSS can be used to approve transactions and replicate them to the ledger store of persistent data objects. Each blockchain in the aggregated blockchain is referred to herein as a member blockchain or simply a blockchain.
[0021] Individual transactions for persistent data objects in the ledger repository can be processed through different member blockchains. In fact, as explained later, information regarding the approval of each PSS (Persistent Sequence Transaction) for a single replicated transaction can be recorded in a separate block, which can reside on a separate member blockchain. Therefore, information about a single replicated transaction can be recorded on different member blockchains.
[0022] In aggregated blockchain applications, the history of transactions in the ledger repository is recorded across member blockchains. Member blockchains can be merged in an appropriate order to provide historical ordering information about transactions performed on the ledger repository. In effect, the aggregated blockchain is a single logical blockchain that records historical ordering information about transactions. Therefore, member blockchains are collectively referred to as aggregated blockchains in this paper.
[0023] In this embodiment, adding a block to a single member blockchain is still a serial operation, which hinders the throughput of applying transactions in parallel across a single member blockchain. However, replicating transactions across multiple member blockchains enables the application of more transactions in parallel.
[0024] When applying transactions in parallel to replicate them to any PSS in the network, the application of transactions must be scheduled in a manner consistent with transaction dependencies or otherwise ordered. Consistency with transaction dependencies requires resolving the transaction dependencies of each transaction to be applied, a process that can be costly. This cost becomes more complex as more transactions are applied in parallel. Therefore, the more efficiently transaction dependencies are resolved, the greater the throughput and scalability of the applied transactions. This paper describes methods for resolving transaction dependencies more efficiently.
[0025] Transaction processing
[0026] In transaction processing, changes to persistent data objects are performed within a transaction. A transaction is a collection of operations that modify persistent data objects. All changes made within a transaction are performed atomically. Committing a transaction makes the changes permanent. If a transaction is aborted or otherwise not committed, it is rolled back.
[0027] In distributed transaction processing, multiple PSS participants commit a distributed transaction to modify corresponding data objects using a phased approach managed by a transaction manager. The phased approach includes at least a phase where participants reach a ready-to-complete state. When in the ready-to-complete state, a participant has performed the operations necessary to reach a state where the participant can be guaranteed to commit the transaction but has not yet committed it. A participant can be instructed by the transaction manager to reach the ready-to-complete state and communicate with the transaction manager whether the participant has reached the ready-to-complete state. Distributed transactions that modify persistent data objects in the ledger repository may be referred to as blockchain transactions in this paper.
[0028] As used in this article, transaction processing also requires journaling. In journaling, changes made by a transaction to persistent data objects are stored in a persistent journal. A persistent journal consists of journal entries that record the changes made by a transaction to persistent data objects. Each entry can specify which fields, columns, elements, or parts of the data object were changed and the new values created by those changes. It can also specify the old values that were replaced by the new values.
[0029] In a write-ahead journal, a record of the changes made by a transaction is persistently stored until the transaction is committed and / or reaches a ready-to-complete state. Furthermore, the journal can be read by other PSS instances to replicate persistent data objects.
[0030] Transaction dependency
[0031] According to an embodiment, parallel application of transactions is performed in a manner consistent with transaction dependencies at the persistent data object level. Based on the transaction's commit time, a particular transaction has a transaction dependency at the persistent data object level on that other transaction when another transaction adds or modifies a persistent data object before the particular transaction modifies it. To apply a particular transaction in accordance with transaction dependencies at the persistent data object level, the particular transaction is applied only after any other transaction to which it has a transaction dependency has been applied. Thereafter, when the term "transaction dependency" is used, unless otherwise stated, it refers to a transaction dependency at the persistent data object level.
[0032] For example, transaction TA, which modifies a row in a table in the source database, commits before transaction TB, which modifies the same row. Because transaction TA commits its changes to the same row before transaction TB, transaction TB has a transaction dependency on transaction TA. To apply these transactions in the target database in a manner consistent with the transaction dependency, transaction TA must be applied in the target database before transaction TB is applied.
[0033] Another example of different levels of transaction dependency is the database level. Based on database-level transaction dependency, transactions are applied in the order in which they are committed against any database object (e.g., a row) within the database.
[0034] For example, transaction TA, which modifies row A in a table in the source database, commits before transaction TB, which modifies a different row B in the same table. To replicate these transactions in the target database based on database-level transaction dependencies, transaction TA is applied to the target database before transaction TB, even if the transactions modify different rows.
[0035] When transactions from the source database are applied to the target database at the transaction-consistent database level, the source and target databases can be transformed through the same sequence of database states. However, when these transactions are applied at the consistent persistent data object level, the source and target databases cannot be transformed through the same sequence of database states.
[0036] For example, transaction TA, which modifies row A in a table in the source database, commits before transaction TB, which modifies a different row B in the same table. To replicate these transactions in the target database in a manner consistent with transaction dependencies at the persistent data object level, transaction TB can be applied in other databases before transaction TA is applied. This may place the target database in a state that does not exist in the source database.
[0037] Aggregated blockchain applications in a replicated DBMS
[0038] Aggregated blockchain applications are described within the context of DBMS. However, aggregated blockchain applications can be used in other types of PSSs that have the ability to make changes using distributed transaction processing.
[0039] In the context of a DBMS, the ledger store for persistent data objects comprises rows in a ledger set of replicated database tables. Each ledger table is referred to as a ledger database table or simply a ledger table. The ledger database table set is replicated at each DBMS. Transactions for any database table at a DBMS are recorded within blocks in the aggregate blockchain, specifically within blocks in the member blockchains replicated across each DBMS. The DBMS reads the changes from the member blockchains and applies the transactions to the corresponding copies of the ledger database table set.
[0040] The collection of multiple DBMSs that participate in the aggregation of blockchain applications to replicate a specific ledger table is collectively referred to in this document as the Blockchain Application Network (“BCA Network”). Each DBMS in the BCA Network may be simply referred to as a Participant DMBS (PDBMS) in this document. Changes that can be replicated using blockchain applications include DML (Database Manipulation Language) changes (i.e., changes to the data in the ledger table) or DDL (Database Definition Language) changes (which include changes to the database dictionary to achieve changes to the BCA Network, including the structure of the ledger table).
[0041] Each PDBMS stores and maintains a ledger set of database tables and a copy of each member blockchain. Therefore, if the aggregated blockchain comprises three member blockchains, each PDBMS includes a copy of each of the three member blockchains.
[0042] According to an embodiment, blockchain transactions for one or more ledgers are initiated by a single initiating PDBMS. The initiating PDBMS is the leader of the member blockchain, and uses the member blockchain to propagate changes made by the transaction to other PDBMSs. The blockchain leader is responsible for proposing blocks to be added to the member blockchain, as further described in U.S. Patent Application No. 16 / 932,633, "Native Persistent Store Support For Blockchains," filed July 17, 2020, by James Stamos et al. (Attorney General's File No. 50277-5374 / ORA190079-US-NP), the entire contents of which are incorporated herein by reference.
[0043] For illustrative purposes, the member blockchain in which PDBMS acts as the leader is referred to as the PDBMS member blockchain or PDBMS member blockchain. According to embodiments, regarding member blockchains, PDBMS frequently and / or periodically alternates leadership of the blockchain. At any given time, PDBMS can be the leader of zero or more member blockchains.
[0044] For blockchain transactions initiated by the initiating PDBMS, the initiating PDBMS coordinates the replication of the transaction with other PDBMSs through its member blockchain. Other PDBMSs are referred to herein as follower PDBMSs regarding their member blockchains and transactions initiated by the initiating PDBMS.
[0045] In this document, PDBMS is described as adding blocks to a blockchain. Adding blocks to a blockchain requires the participation of a PDBMS. Each PDBMS permanently appends blocks to the corresponding copy of its member blockchains according to the consensus protocol, which will be described later and in detail in Native Persistent Store Support For Blockchains.
[0046] Database application clients can request to execute transactions that modify ledger tables by simply issuing DML statements to the PDBMS via a database session established with it. The PDBMS then becomes the initiator of the transaction.
[0047] According to an embodiment, the aggregated blockchain application replicates transactions to a set of ledger tables created by the initiating PDBMS. Transactions are replicated at follower PDBMSs using a variant of two-phase commit, which begins when the initiating PDBMS receives a request to commit the transaction. In the first phase, the prepare-to-commit phase, the initiating PDBMS adds a block to the member blockchain for which it is the leader. This block specifies the changes made by the transaction. As a result, each PDBMS includes this block in its corresponding copy on the member blockchain.
[0048] Each follower PDBMS replays the specified changes in a block without committing them. Based on the results of the replay, the follower PDBMS votes on whether to commit the transaction or not. Each follower PDBMS then records its vote in a block, which is added to the corresponding member blockchain of the follower PDBMS. As a result, multiple blocks recording the votes on committing or not committing the transaction exist in multiple member blockchains.
[0049] The initiator PDBMS reads these blocks from local copies of multiple member blockchains to read the votes of each follower PDBMS. Based on the read votes, the initiator PDBMS determines the commit consensus result according to the consensus algorithm.
[0050] In the second phase, if the consensus submission result is "commit," the initiating PDBMS commits the transaction locally and adds the block recording the commit consensus result to the initiating PDBMS's member blockchain. Follower PDBMSs read the consensus submission result from the blocks recorded in their respective copies of the initiating PDBMS's member blockchain and commit their transactions locally. If the consensus submission result is "no commit," the initiating PDBMS aborts the blockchain transaction, and each follower PDBMS that executed the blockchain transaction aborts its transaction locally.
[0051] To explain in more detail, the blockchain is stored in a log that includes log records, each of which stores a block. Adding a block is proposed by suggesting each log record that contains a block.
[0052] Typically, BCA participants reach consensus by following a consensus algorithm. The algorithm is used to reach a consensus on whether to accept a proposal, and then an action is taken on the proposal if the consensus is to accept it. A proposal can be adding a log entry to the log (which includes a block), or taking another action.
[0053] In log-based consensus algorithms, participants reach a consensus on appending log entries, specifically on the order in which these entries are appended and the content of each entry. A copy of the log is maintained by each participant. In blockchain applications, each log entry stores a block in the blockchain, and the order of blocks in the blockchain mirrors the order of the log entries.
[0054] Aggregated blockchain applications are implemented as a native feature of PDBMS. Blockchain networks are created and configured by issuing DDL commands to each PDBMS. Changes to the ledger and associated blockchains are integrated into the PDBMS's native transaction and replication capabilities, including the ability to replicate changes in parallel. Performance can be monitored using available PDBMS tools. Database applications can be programmed to read and write to the ledger in the same way that applications are configured to read and write to traditional tables. Implementing the ledger and its associated blockchains does not require external (i.e., external to the PDBMS) data structures.
[0055] Explanatory BCA Network
[0056] Figure 1A A BCA network 101, which is a BCA network according to an embodiment of the present invention, is depicted. (Refer to...) Figure 1ABCA network 101 includes PDBMS 110, PDBMS 120 and PDBMS 130.
[0057] BCA Network 101 uses a clustered blockchain application to maintain the ledger set. Using a clustered blockchain application requires each of PDBMS 110, PDBMS 120, and PDBMS 130 to maintain replicated data structures, such as ledger tables and member blockchains. Figure 1B It is an organizational view that displays the replicated data structure and the logically contained replicas, but does not depict which participant hosts the replicas.
[0058] Reference Figure 1B , Figure 1B Aggregate BC 103 is described, which is an aggregate blockchain comprising three member blockchains: member BC103-B, member BC 104-B, and member BC 105-B. Each of member BC 103-B, member BC 104-B, and member BC 105-B includes replicas, namely replica member blockchains 103-B1, 103-B2 & 103-B3, 104-B1, 104-B2 & 104-B3, and 105-B1, 105-B2 & 105-B3.
[0059] The blockchain is used to replicate changes to ledger 102. Ledger 102 includes three copies: ledger copies 102-1, 102-2, and 102-3. No other tables are depicted.
[0060] The blockchain is stored in a blockchain log (BC log), which includes BC log records. Each log record stores a block in the blockchain. According to an embodiment, the BC log is a database table that includes at least one column for storing the blockchain, and each row of the BC log (BC log record) stores a block of the blockchain.
[0061] Figure 1BBC-Log 103-L, BC-Log 104-L, and BC-Log 105-L are depicted. Each of BC-Log 103-L, BC-Log 104-L, and BC-Log 105-L includes three copies. The copies of BC-Log 103-L, BC-Log 104-L, and BC-Log 105-L include BC-Log 103-L1, BC-Log 103-L2 & BC-Log 103-L3, BC-Log 104-L1, BC-Log 104-L2 & BC-Log 104-L3, and BC-Log 105-L1, BC-Log 105-L2 & BC-Log 105-L3, respectively; these BC log copies store member blockchain copies 103-B1, 103-B2 & 103-B3, 104-B1, 104-B2 & 104-B3, and 105-B1, 105-B2 & 105-B3, respectively.
[0062] Each log record includes a log identifier (log ID), which uniquely identifies the log record within the log. Each copy of a log record includes the same log ID. The log IDs are ordered such that the log ID of a log record is greater than the log ID of any other log record that maintains the record in the blockchain as an earlier block in the ledger.
[0063] In this embodiment, the log ID in the BC log is a monotonically increasing value, such as an integer. Regarding the blockchain stored in the BC log, the log ID of a log record in the BC log indicates the block position of the block contained in that log record, where the block position is the location of the block within the blockchain. For example, a log record with log ID 0 stores the genesis block in the corresponding blockchain, a log record with log ID 1 stores the second block in the corresponding blockchain, and so on.
[0064] Figure 1A This describes how the aforementioned copies are hosted across BCA network 101. PDBMS 110 hosts ledger copy 102-1 and BC logs 103-L1, 104-L1, and 105-L1. PDBMS 120 hosts ledger copy 102-2 and BC logs 103-L2, 104-L2, and 105-L2, and PDBMS 130 hosts ledger copy 102-3 and BC logs 103-L3, 104-L3, and 105-L3.
[0065] As previously stated, each PDBMS in the BCA network 101 is a leader of a member blockchain. They use this member blockchain to replicate transactions to other PDBMSs in the BCA network 101, while the other PDBMSs are followers of that member blockchain. The leader or follower of a member blockchain is also referred to as the leader or follower of the BC log containing that member blockchain. For example, PDBMS 110 is the leader of BC log 103-L and member blockchain 103-B, while PDBMSs 120 and 130 are followers.
[0066] Additional terminology for consensus-driven actions
[0067] Various operations of BCA network 101 or any of its participants are described as actions performed according to or by using a consensus algorithm. As previously stated, the consensus algorithm requires participants to follow a protocol used by the algorithm, which may require steps such as: the initiating PDBMS sends a proposal (e.g., a log record and log ID of the proposal); other PDBMSs receive the proposal and respond by voting to accept or reject it; the initiating PDBMS determines the consensus result based on the received votes according to a log-based consensus protocol; and then the initiating PDBMS sends the consensus result to the participants. When voting, accepting, or rejecting, or when an operation is described as being performed according to or by using a consensus algorithm, it can be assumed that participants are performing actions required by the protocol and according to the protocol.
[0068] As will be explained in more detail, BCA participants follow a consensus algorithm to determine whether to commit a blockchain transaction. As previously stated, the consensus result used to commit a blockchain transaction is referred to as the commit consensus result. The acceptance or rejection of the proposal to commit a distributed transaction by the follower PDBMS is referred to herein as the commit or non-commit vote, respectively.
[0069] Log-based consensus algorithms
[0070] According to an embodiment, BCA network 101 uses an initiator-based log-based consensus algorithm to control whether PDBMSs add log records to the BC log. Typically, an initiator-based log-based consensus algorithm requires the leader PDBMS to propose appending log records to the follower PDBMSs and manages the consensus among PDBMSs regarding whether to accept the proposal.
[0071] In response to the initiating PDBMS sending a message to propose a log entry, other PDBMSs determine whether to accept or reject the proposal and communicate this determination to the initiating PDBMS. The PDBMS determines acceptance or rejection based on one or more consensus criteria, which are at least partially based on the number of PDBMSs that accept or reject the proposal. This determination is referred to herein as the consensus result. An example of a consensus criterion is the quorum of PDBMSs that accept the log entry of the proposal. The initiating PDBMS then communicates the consensus result to the other PDBMSs.
[0072] PDBMS communicates acceptance of a proposal in several ways. One way is to send a digitally signed message to the initiating PDBMS, specifying whether the PDBMS accepts or rejects the proposal.
[0073] When the consensus result is to add a block to a member blockchain, the block is permanently appended to each copy of the member blockchain at each PDBMS. When the PDBMS or BCA network is described herein as adding a block to a member blockchain or adding a log record containing the block to the BC log, the log record containing the block is added to the corresponding local copy of the BC log of the PDBMS. Any PDBMS can subsequently read and access the log record and / or the block to perform processing based on the information in the block. Further details regarding adding blocks to the blockchain and adding log records to the BC log are described in Native Persistent Store Support For Blockchains.
[0074] Overview of Blockchain Application Replication
[0075] Figure 2 This is a flowchart depicting a process for aggregating blockchain applications according to an embodiment of the present invention. The process replicates DML changes to ledger tables across the BCA network.
[0076] This flowchart depicts the normal operation process, where an aggregated blockchain application is used to replicate DML changes made by a transaction without encountering error conditions or exceptions. Handling error conditions and exceptions is described in Native Persistent Store Support For Blockchains. The operation is illustrated in a scenario involving the replication of a blockchain transaction, where PDBMS 110 is the initiating PDBMS, and PDBMS 120 & 130 are the following PDBMSs. The blockchain transaction is initiated by a database client that has already established a database session with the initiating PDBMS 110.
[0077] Reference Figure 2The initiator PDBMS 110 performs database operations (205) within a blockchain transaction targeting ledger 102-1. Such database operations include executing one or more database statements (including DML statements and one or more blockchain procedures). Database operations may include savepoints, or even rollbacks. A blockchain procedure is a computer-executable process or function whose implementation is accepted and visible to members of BCA network 101, and is executed by the PDBMS of BCA network 101 to modify the ledger. Blockchain procedures are further described in Native Persistent Store Support For Blockchains.
[0078] The initiator PDBMS 110 receives a request to commit a database transaction from the client. In response, the initiator PDBMS 110 initiates a blockchain transaction and prepares to commit the blockchain transaction as part of a two-phase commit and is ready to commit (210).
[0079] A blockchain transaction modifies the database in each of the multiple participating PDBMSs. Changes at each PDBMS are made by local transactions initiated and / or executed by the PDBMS as part of the blockchain transaction, altering the data at that PDBMS. The initiating PDBMS assigns a distributed transaction ID (“distributed txid”). Local transactions are referred to as branch transactions of the blockchain transaction. Each PDBMS assigns a local transaction ID to the corresponding branch transaction and associates the local transaction ID with the distributed txid.
[0080] The execution, commit, and termination of a distributed transaction can be coordinated by the initiating PDBMS. In a preferred embodiment, the commit of the blockchain transaction is initiated by the initiating PDBMS, but the actual commit and termination of the blockchain transaction are carried out in a decentralized manner, as explained later. According to another embodiment, a distributed transaction protocol specified in the X / Open XA specification is used to execute the distributed transaction. Under this protocol, the initiating PDBMS is the transaction manager of the distributed transaction.
[0081] Initiator PDBMS 110 adds a “TC block” containing the transaction context to the member blockchain of Initiator PDBMS 110, which is a member of BC 103-B. The transaction context includes the specification of the database operations performed by Initiator PDBMS for the branch transaction and the “write set” of one or more rows that have been modified as a result of these database operations. The write set in the proposed TC block is referred to herein as the proposed write set. The TC block may include other information, such as the distributed txid of the blockchain transaction. The contents of the TC block will be described in more detail later.
[0082] Follower PDBMS 120 and 130 read the TC block from member BC 103-B and replay the transaction context (i.e., the sequence of database operations specified by the specification executed within the transaction context) within the branch transaction initiated by follower PDBMS 120 and 130 to generate a local write set (230). A comparison of the local write set with the proposed write set at least partially determines whether follower PDBMS 120 and 130 will vote to commit the distributed transaction. Follower PDBMS 120 and 130 initiate local transactions to replay the transaction context in the TC block to generate the local write set. The local write set is then compared with the proposed write set (235).
[0083] Follower PDBMS 120 and 130 determine which database transaction (240) to vote on and commit the proposed transaction based on a comparison between the local write set and the proposed write set. Typically, in determining whether to vote on and commit a distributed transaction, follower PDBMS 120 and 130 determine whether one or more DML commit criteria are met. DML commit criteria include whether the proposed write set matches the local write set, and if so, whether one or more other criteria (if any) are met. Rows in the write set should include a primary key or similar unique identifier from the ledger table to ensure that copies of identical rows can be matched for comparison. Set comparison is used when determining whether the proposed write set and the local write set match. The order in which rows are inserted in the local write set and the order in which rows are inserted in the proposed write set do not affect the comparison. For example, set {A, B, C} is equal to set {A, C, B} because both sets have the same three elements.
[0084] Next, followers PDBMS 120 and 130 prepare to submit the proposed DB transaction (245). After preparing to submit, followers PDBMS 120 and 130 vote to submit the blockchain transaction, recording their votes in separate blocks of their respective member blockchains BC 104-B and 105-B (250).
[0085] Initiator PDBMS 110 reads the blocks containing the votes of PDBMS120 and 130 from BC 104-B and 105-B, respectively. Based on the votes, Initiator PDBMS 110 determines that the consensus result for committing the blockchain transaction is to commit the blockchain transaction (260). In response to this determination, Initiator PDBMS 110 adds a block to member BC 103-B, which records the consensus result for committing the blockchain transaction as commit (265). Then, Initiator PDBMS 110 commits the blockchain transaction, which includes committing the corresponding local transaction of Initiator PDBMS 110 for the distributed transaction.
[0086] Follower PDBMS 120 and 130 read blocks from member BC 103-B, which record the commit consensus results of the distributed transaction. In response to determining the commit consensus results, follower PDBMS 120 and 130 commit their respective local transactions for the distributed transaction.
[0087] In embodiments of the present invention, the follower PDBMS may not have replayed or committed the blockchain transaction locally, but it can know that the consensus result of the blockchain transaction has been committed. The follower PDBMS (1) can replay the database operations in the blockchain transaction and verify whether the local write set and the proposed write set are the same, or (2) the follower PDBMS can ignore the database operations and replay the individual changes in the write set. The PDBMS that behaves as in (1) is called the transaction approver, while the PDBMS that behaves as in (2) is called the transaction replicator. The advantage of being a transaction replicator is that replication avoids any uncertainty that may exist in the original database operations in the blockchain transaction.
[0088] Figure 2 The aggregated blockchain application process described uses a centralized approach to determine the consensus result of blockchain transactions, where only the initiator PDBMS determines the consensus result and communicates it to the follower PDBMSs. This centralized approach may have a single point of failure and may not tolerate malicious behavior from the initiator PDBMS. Attaching digital signatures to the vote can mitigate, but not eliminate, the possibility of the initiator PDBMS engaging in malicious behavior using a centralized approach.
[0089] In this embodiment, decentralized determination of the consensus submission result can be used. In this decentralized determination, the follower PDBMS and the initiator PDBMS can independently determine the consensus submission result by examining the votes recorded for the blockchain transaction in their respective local copies across multiple member blockchains. Therefore, the follower PDBMS determines the consensus result without waiting for the block in which the initiator PDBMS determines the consensus submission result for the blockchain transaction. In response to the follower PDBMS independently determining the consensus submission result for the blockchain transaction, if the consensus submission result is "submit," the follower PDBMS submits the blockchain transaction locally; otherwise, if the consensus submission result is "not submit," the follower PDBMS aborts the blockchain transaction.
[0090] An alternative approach to tracking dependencies on individual transactions is to track dependencies on member blockchains in a conservative, coarse-grained manner and maintain a local watermark for each member blockchain. In this alternative approach, each PDBMS maintains a low watermark (LWM) for each member blockchain in the BCA network in persistent storage. Every transaction on a member blockchain (up to and including LWM transactions) has been applied by the PDBMS (i.e., executed and committed if quorum consensus is commit, or aborted or ignored if quorum consensus is abort). Before scheduling blockchain transaction T for local execution, the PDBMS (or the PDBMS that adds the transaction to the member blockchain) sorts the dependent transactions according to the position of the dependent transactions of T on each member blockchain. Once the low watermark of a member blockchain points to the last dependent transaction on that member blockchain or a subsequent transaction on that member blockchain, blockchain transaction T has no unresolved local dependencies on that member blockchain. Once blockchain transaction T has no unresolved local dependencies on any local member blockchain, blockchain transaction T can be scheduled for local execution.
[0091] Because blockchain transactions can be initiated and rows locked simultaneously at multiple PDBMSs, distributed deadlocks can occur. Existing techniques for detecting or avoiding distributed deadlocks can be employed. Where possible, transactions that have already been approved for submission through consensus should be given priority first, followed by transactions that are already in the member blockchain and are being replayed at another PDBMS, and then transactions that are not yet in the member blockchain.
[0092] Parallelism in Blockchain Applications
[0093] A PDBMS might be replaying multiple transactions within a single member blockchain and across records on multiple member blockchains to generate a set of writes. As mentioned earlier, transactions are applied in accordance with transaction dependencies, which requires resolving the transaction dependencies of individual transactions. At a general level, transaction dependency solutions can be solved using an exhaustive approach. For example, an exhaustive transaction dependency solution might require accessing and inspecting a map that is maintained to track persistent data objects changed by transactions, as well as the relative order in which transactions changed persistent data objects and the relative order in which transactions were committed. The map is inspected to determine if any transactions that previously modified persistent data objects have been replayed and committed.
[0094] This paper describes a novel approach to resolving transaction dependencies that resolves them significantly faster and with less overhead than exhaustive solutions. The new approach is referred to herein as the Dependency Checkpointing approach because it is based on efficiently maintained dependency checkpoints maintained at each BCA participant in the BCA network. At a high level, a dependency checkpoint is a global timestamp compared to a transaction promotion timestamp (“promotion timestamp”), which is associated with each persistent data object modified by a committed transaction. If a database object’s promotion timestamp is less than the dependency checkpoint, then any transaction that modified the persistent data object has been replayed and committed. This comparison can be performed to resolve transaction dependencies with significantly less overhead than exhaustive solutions.
[0095] The dependency checkpointing method is based on the following characteristics:
[0096] Timestamps: Timestamps represent logical or actual time and are associated with events such as changes to persistent data objects and the start and commit of transactions. A timestamp associated with an event represents the relative order in which that event occurred, based on the timestamps of other events. For example, a logical timestamp is a System Change Number (“SCN”).
[0097] Global timestamp: A global timestamp represents the logical or actual time across all participants in a BCA network and is used to indicate the relative order of events occurring across all participants. For example, the Lamporting scheme can be used to maintain global timestamps across BCA participants.
[0098] Promotion timestamp: In this embodiment, the promotion time is a global timestamp indicating when a transaction first changes any ledger persistent data object. A transaction in PSS can make a first change to a persistent data object that is not a ledger persistent data object, then a second change to a second persistent data object that is a ledger persistent data object, and then a third change to the ledger persistent data object. Each change occurs at a different timestamp. The promotion timestamp of the transaction is the timestamp when the second change occurs.
[0099] Object change metadata: Persistent data objects contain attributes describing the changes made to the last committed persistent data object. These attributes include the Distributed Transaction ID (LDTXID) of the last blockchain transaction that changed the persistent data object, and the Last Promotion Time (LPTIME), which is the promotion time of the last blockchain transaction that changed the persistent data object. In this embodiment, the object change metadata also includes a version number. When a transaction in a TC block changes a persistent data object, the object change metadata of the previous version of the persistent data object is included in the transaction context, while the new version of the persistent data object is included in the write set.
[0100] Earliest Locally Initiated Transaction (OLIT): In a PDBMS, there may be uncommitted pending transactions at some point in time. OLIT is the earliest promotion time associated with any uncommitted pending local transaction, or, if no uncommitted pending transactions exist, OLIT is the timestamp of the current time. Any transaction associated with a promotion time earlier than OLIT has been terminated, i.e., aborted or committed. For versions of persistent data objects with LPTIME less than OLIT, the transaction that created that version has already committed. OLIT depends only on blockchain transactions initiated at the local PDBMS. For a specific PDBMS, blockchain transactions initiated at other PDBMSs do not affect the OLIT of that PDBMS.
[0101] Block OLIT (BOLIT): TC blocks added to the chain include OLITs that existed when the leader generated the block to add it to the chain. This OLIT is called a BOLIT. Within a member blockchain, TC blocks are added by the leader of the member blockchain in commit time order. A blockchain transaction for a given TC block in a member blockchain is committed earlier than a blockchain transaction for any subsequent TC block in the member blockchain. For any blockchain transaction in a subsequent TC block in the blockchain, the blockchain transaction whose timestamp is earlier than the BOLIT of a particular TC block has been committed or aborted.
[0102] Explicit Dependency Checkpoints: Each BCA participant maintains a dependency checkpoint associated with each copy of the member blockchain for that participant. When a TC block in a member blockchain has been applied by a follower BCA participant, the follower BCA participant updates the dependency checkpoint to the value of BOLIT for the just applied TC block before applying subsequent TC blocks in the member blockchain. Applying a TC block means replaying the transaction context within the TC block and committing the transaction. Therefore, when applying subsequent TC blocks, follower participants can compare the LPTIME of the previous version of the persistent data object with the dependency checkpoint to determine whether the last transaction that modified the persistent data object has been committed.
[0103] Explanatory TC Block
[0104] A TC block is described within the context of a PDBMS. According to an embodiment, a TC block in a member blockchain comprises one or more block record sets. The block record set in a TC block includes a transaction context. As previously described, the transaction context describes the blockchain transaction in which the initiating PDBMS adds the TC block to the member blockchain of the initiating PDBMS. Figure 3 The TC block according to an embodiment of the present invention is depicted.
[0105] Reference Figure 3 , Figure 3 TC block 301 in BC log 103-L is depicted. TC block 301 was proposed and added by PDBMS 110. TC block 301 includes block record set 302 and... Figure 3 Other properties not described herein. For example, such properties include block hash and previous hash, as described in Native Persistent Store Support For Blockchains.
[0106] Block record set 302 includes transaction context 310, which includes fields database operation 312, write set 314, distributed TXID 316, BOLIT 320, and TxnPromTime 322. Database operation 312 specifies the database operations performed by the corresponding initiator PDBMS for the blockchain transaction. Write set 314 is the write set generated by performing database operations. Distributed TXID 316 contains the distributed txid of the blockchain transaction.
[0107] BOLIT 320 includes the BOLIT for TC block 301. TxnPromTime 322 includes the block promotion time for TC block 301.
[0108] Write set 314 is a view depicting write set 314, which contains some of the rows. Write set 314 includes rows 330 and 332. It depicts the previous and newer versions of these rows. Rows 330 and 332 include the following columns:
[0109] PK 342: The primary key of the row. The primary keys of rows 330 and 332 are pk1 and pk2, respectively.
[0110] Other column 350: These columns include changes made by blockchain transactions. Data columns include DML changes specified by clients of BCA network 101 via DML statements.
[0111] The following columns preserve the object's change metadata.
[0112] RowPromTime 344: Row's LPTIME. For previous versions of the row, LPTIME was the promotion time of the blockchain transaction that changed the row. For previous versions of rows 330 and 332, RowPromTime 344 was 10:40 and 11:05, respectively. For the new version of the row, the row's LPTIME is the block promotion time of TC block 301. Therefore, for the new version of rows 330 and 332, RowPromTime 344 is 11:10, which is the corresponding value of TxnPromTime 322. For illustrative purposes, timestamps are described in this document as standard times of day, not logical times.
[0113] RowLDTXID 346: The last LDTXID of the row's LDTX. For the new version of the row, RowLDTXID 346 is the value of 478810 as distributed TXID 316.
[0114] RowVersion 348: RowVersion 348 is the row version number. In this implementation, the row version number is not used for transaction dependency solutions. However, the version number is used to more conveniently sort the versions of row replicas on the PDBMS.
[0115] Replay using the dependency checkpointing method
[0116] Figure 4 This describes a process that can be merged to apply the transaction context using a dependency checkpointing approach by a follower PDBMS. In an embodiment, it can be used as... Figure 2 This process is performed as part of operation 230 described in the diagram. The process is described using follower PDBMS 120, which is replaying TC block 301 from replica member BC103-B2 in BC log 103-L2. The dependency checkpoint is 10:55.
[0117] Reference Figure 4 At 410, PDBMS 120 reads TC block 301. For each line in write set 314, operations 415-420 are performed.
[0118] At 415, PDBMS 120 determines a dependency checkpoint where the LPTIME (i.e., 10:40) of the previous version of line 330 is less than 10:55. Therefore, any previous changes to line 330 have been committed, and line 330 has no transactional dependencies. In response to this determination, another line in the write set is examined, and an exhaustive transactional solution is not performed on line 330 (420).
[0119] Next, PDBMS 120 determines a dependency checkpoint in line 332 of the previous version where the LPTIME (i.e., 11:05) is greater than 10:55. Therefore, the previous changes to line 330 may not have been committed. In response to this determination, an exhaustive transaction dependency solution is used to resolve the transaction dependency of line 332.
[0120] Once all rows have been examined at operation 415, at operation 430, PDBMS120 waits until any transaction dependencies in write set 314 have been resolved, i.e., any transaction that any row in write set 314 depends on has been committed. Such a wait may require further exhaustive transaction dependency solutions for any rows that were determined in operation 420 to have, or potentially have, transaction dependencies on uncommitted transactions. For example, for row 332 at operation 420, the exhaustive transaction dependency solution may have determined that row 332 depends on a transaction that modified row 332 and has not yet been committed. An exhaustive transaction dependency solution can then be performed again to determine if the transaction has been committed.
[0121] Next, transaction context 310 (435) is replayed and committed. Transaction context 310 is replayed by replaying the database operation specified in database operation 312.
[0122] Finally, PDBMS 120 waits until any transactions in replica member BC 103-B2 that have an earlier TC block for it have been committed or aborted (440). After waiting, if any, the dependency checkpoint on PDBMS 120 of replica member BC 103-B2 is updated to the value of BOLIT 320. As a result, the dependency checkpoint is updated from 10:55 to 11:00.
[0123] Replication timing inconsistency
[0124] Replication at the persistent object level (PSS) for transactional consistency may result in rows being applied at the target PSS before another row, previously committed at the source PSS, is applied. This inconsistency in the relative timing between the commit at the source and the application at the target is referred to herein as replication timing inconsistency. Replication timing inconsistency can create state at the target PSS that may not yet exist on the source PSS.
[0125] Under certain conditions, inconsistencies in replication timing can lead to inconsistencies and errors in the imposed relationships between values in persistent data objects. These relationships include uniqueness constraints and foreign key constraints.
[0126] Uniqueness constraint replication timing inconsistency
[0127] A uniqueness constraint requires that any value in a column of a table be unique relative to all other values in the column. When an attempt is made to add a duplicate value to a column subject to a uniqueness constraint, the PDBMS raises a uniqueness constraint violation error.
[0128] However, inconsistent replication timing can lead to erroneous uniqueness constraints being violated. Inconsistent replication timing can even result in inconsistent uniqueness constraints between PDBMSs.
[0129] For example, PDBMS 110 initiates and submits blockchain transaction BT1, which inserts row A into the ledger table 102-1. The value in the uniqueness constraint column of that row is 10.
[0130] PDBMS 110 initiates and commits blockchain transaction BT2, updating the uniqueness constraint column in row A from 10 to 11. After blockchain transaction BT2 is committed, PDBMS 120 initiates and commits blockchain transaction BT3, inserting a different row B with a uniqueness constraint column value of 10.
[0131] PDBMS 130 was offline during the execution of these three blockchain transactions. The blockchain transactions were accepted according to the commit consensus algorithm, as the quorum members of PDBMS, PDBMS 110 and PDBMS 120, voted to commit these transactions. Once PDBMS 130 came online, the TC blocks for blockchain transactions BT1 and BT2 were added to replica member blockchain 103-B3, and the TC block for blockchain transaction BT3 was added to replica member blockchain 104-B3.
[0132] Between three transactions, PDBMS 130 applies blockchain transaction BT3, recorded in the TC block of BT3 in the replica member blockchain 104-B3, and commits row B with a uniqueness constraint column value of 10. Next, PDBMS 130 begins applying blockchain BT1 and encounters an incorrect uniqueness constraint violation when attempting to insert row A with a uniqueness constraint column value of 10.
[0133] Avoiding replication timing inconsistencies due to uniqueness constraints
[0134] One measure to avoid timing inconsistencies in the replication of unique constraints is the use of "tombstones". A tombstone is applied when a row in a table is deleted or a value in a unique constraint column is replaced. According to an embodiment, tombstones preserve the deleted or replaced column values ("tombstone values") by inserting or updating rows ("tombstone rows") in another table ("tombstone table").
[0135] Transactions that delete or replace tombstone values also insert or update tombstone rows. Subsequent transactions reuse tombstone values either by inserting another row where the unique constraint column is set to the tombstone value or by updating the unique constraint column in another row to the tombstone value. The tombstone table is checked in conjunction with inserts or updates targeting tombstone values. In response to finding a tombstone value, the tombstone row is updated to indicate either the reuse of the tombstone value or the deletion of the tombstone row.
[0136] As a result, earlier transactions that delete or replace tombstone values and subsequent transactions that reuse tombstone values have "induced" transaction dependencies because they are performed to modify the same tombstone rows. Under replication based on applying transactions in accordance with persistent data object-level transaction dependencies, subsequent transactions will be applied later than earlier transactions.
[0137] Figure 5 This document describes the use of gravestones in the context of a PDBMS to avoid timing inconsistencies in the replication of uniqueness constraints in a PSS. The description is based in part on a gravestone table with a primary key column (PK) for storing gravestone values. The gravestone table also includes columns for recording distributed txids of blockchain transactions that delete or replace values in the uniqueness constraint column of the ledger table. The gravestone table consists of gravestone rows, each storing a gravestone value as the primary key in the PK, and a distributed txid of the blockchain transaction that deletes, replaces, or reuses the gravestone value in the uniqueness constraint column. The distributed txid is used for exhaustive transaction dependency solutions if needed. Another column, referred to as the reuse column, indicates whether a gravestone value has been reused.
[0138] The description pertains to BCA network 101. PDBMS 110 and PDBMS 120 are online, while PDBMS 130 is offline. PDBMS 130 will eventually come online.
[0139] Reference Figure 5 PDBMS 110 commits blockchain transaction BT1, inserting row A with a unique constraint column value of 10 into ledger table (510). PDBMS 110 also adds the TC block TB1 of blockchain transaction BT1 to member blockchain 103-B, which results in TC block TB1 being added to replica member blockchain 103-B3 when PDBMS 130 goes live.
[0140] PDBMS 110 commits blockchain transaction BT2, which updates the unique constraint column in row A from 10 to 11. PDBMS 110 also adds the TC block TB2 of blockchain transaction BT2 to member blockchain 103-B, which results in TC block TB2 being added to replica member blockchain 103-B3 when PDBMS 130 comes online. Blockchain transaction BT2 also inserts a gravestone row with a PK value of 10 into the gravestone table. This gravestone row also includes the distributed txid BT2(515).
[0141] PDBMS 120 commits blockchain transaction BT3, which inserts row B with a unique constraint column value of 10 into the ledger. PDBMS 120 also adds the TC block TB3 of blockchain transaction BT3 to member blockchain 104-B, which results in TC block TB3 being added to replica member blockchain 104-B3 when PDBMS 130 goes live. Blockchain transaction BT3 also updates the reuse column in the gravestone row with a PK value of 10 to mark that the value 10 has been reused as a unique value for the uniqueness constraint column. The gravestone row also includes distributed txidBT3(520). As a result of the gravestone row updates, an induced transaction dependency has been created between blockchain transactions BT2 and BT3.
[0142] Next, PDBMS 130 was brought back online. PDBMS 130 began using the aggregated blockchain application to apply TC block TB3 (540) before applying TC block TB1.
[0143] PDBMS detects an induced transaction dependency between blockchain transactions BT3 and BT2 indirectly through a dependency checkpointing method or by using an exhaustive solution of transaction dependencies using the distributed txid BT3 recorded in the tombstone row (545). In response to the detection of the induced transaction dependency, PDBMS 130 applies TC block TB3 until TC block TB2 is applied. Because of the transaction dependency between blockchain transactions BT2 and BT1 (since both modified row A), PDBMS 130 applies BT1 before applying BT2.
[0144] Various embodiments of tombstone inscriptions
[0145] In this embodiment, a tombstone table exists for each unique constraint column. Instead of having a tombstone table for each unique constraint column, an alternative is to have a combined tombstone table for multiple unique constraint columns, where the unique constraint columns have the same column(s) type(s). The combined tombstone table used for deleting or replacing unique values has a CONSTRAINT_NAME column to separate values from different unique constraint columns. For multi-column unique constraints, multiple columns can be combined into a single binary value, rather than having multiple columns in the tombstone table for each of the multiple unique constraint columns. This approach allows a single tombstone table to be used for multiple unique constraint columns in a ledger tableset, regardless of the number of unique constraint columns or their data types.
[0146] Gravestone rows can also be garbage collected. A gravestone row can be removed from the gravestone table or otherwise cleared when the last modification to it satisfies the age criterion and is an UPDATE that changes a non-NULL unique value or a DELETE that deletes a row with a non-NULL value in a uniqueness constraint column. An example of an age criterion could be based on the timestamp of the last modification stored in a column of the gravestone row. The age criterion is satisfied when an administrator-defined retention threshold (e.g., 1 week) for a replay of a blockchain transaction after point-in-time recovery is reached, or when the timestamp on a particular PDBMS is earlier than the minimum LWM maintained by the PDBMS.
[0147] Inconsistent replication timing of foreign key constraints
[0148] A foreign key constraint on a column ("foreign key") in a child table ("child table") requires that the value in that column exists as a primary key value in another table ("parent table"). Attempting to modify a row may violate this foreign key constraint, potentially leading to an error. For example, attempting to insert a "child row" in the child table whose foreign key column value does not exist as a primary key value in the parent table violates the foreign key constraint and raises an error. Similarly, attempting to update a foreign key column with a value that does not exist as a primary key in the parent table violates the foreign key constraint and raises an error. Inconsistent replication timing can also cause foreign key violation errors.
[0149] In the description of replication inconsistency in foreign key constraints, the ledger table named CHILD is a child table, while the ledger table named PARENT is the parent table relative to CHILD. PARENT and CHILD are ledger tables of ledger table 102.
[0150] PDBMS 110 initiates blockchain transaction BT4. Blockchain transaction BT4 commits the insertion of rows into PARENT (hereinafter referred to as the parent row). Next, PDBMS 120 initiates and commits blockchain transaction BT5, which inserts the child rows of the parent row into CHILD. Next, PDBMS 120 initiates and commits blockchain transaction BT6, which deletes the child rows in CHILD. Finally, PDBMS 110 initiates and commits blockchain transaction BT7, which deletes the parent row.
[0151] Since PDBMS 130 was offline during the execution of these four blockchain transactions, the blockchain transactions were accepted according to the commit consensus algorithm, as the quorum of PDBMS, PDBMS 110 and PDBMS 120, voted to commit these transactions. Once online, PDBMS 130 added the TC blocks for blockchain transactions BT4 and BT7 to replica member blockchain 103-B3 and the TC blocks for blockchain transactions BT5 and BT6 to replica member blockchain 104-B3.
[0152] PDBMS 130 first applies blockchain transactions BT4 and BT7, which insert and then delete the parent row. Afterward, PDBMS 130 attempts to apply blockchain transaction BT5 to insert the child row. However, because the parent row has already been deleted, a foreign key constraint violation error occurs.
[0153] Replication timing inconsistencies can be avoided by creating induced transaction dependencies involving the parent row when inserting and / or modifying child rows. This measure will cause any blockchain transactions that create child rows (via inserting or updating foreign key columns) to be applied before any blockchain transaction that deletes the parent row is deleted. Similarly, blockchain transactions that delete or modify the parent row's primary key will only be applied after the child row has been deleted or modified to remove the foreign key relationship with the parent row. "Parent update" conditions that trigger the creation of induced transaction dependencies by modifying the parent row include:
[0154] 1. Insert a subrow with a non-NULL foreign key;
[0155] 2. Delete sub-rows with non-NULL foreign keys;
[0156] 3. Change the foreign key of the sub-row from NULL to a non-NULL value;
[0157] 4. Change the foreign key in the sub-row from a non-NULL value to NULL; or
[0158] 5. The foreign key of the sub-row changes from a non-NULL value to a different non-NULL value.
[0159] In cases 1-4 above, one parent row is updated, while in case 5, two parent rows are updated. Figure 6 A diagram illustrates how updating the parent row uses induced transaction dependencies to avoid replication timing inconsistencies. In this diagram, PDBMS 130 is offline, while PDBMS 110 and PDBMS 120 execute blockchain transactions BT4, BT5, BT6, and BT7.
[0160] Reference Figure 6 PDBMS 110 initiates blockchain transaction BT4. Blockchain transaction BT4 commits the insertion of the parent row into PARENT and also adds the TC block TB4 of blockchain transaction BT4 to member blockchain 103-B, which results in TC block TB4 being added to replica member blockchain 103-B3 (610) when PDBMS 130 comes online.
[0161] Next, PDBMS 120 initiates and commits blockchain transaction BT5, which inserts the child row into the parent row's CHILD. PDBMS 120 also adds the TC block TB5 of blockchain transaction BT5 to member blockchain 104-B, which results in TC block TB5 being added to replica member blockchain 104-B3(615) when PDBMS 130 comes online.
[0162] In addition, blockchain transaction BT5 updates the parent row. This creates a transaction dependency induced between blockchain transactions BT4 and BT5 due to the parent row update, thus ensuring that BT4 is applied before BT5 is applied, so that the parent row is inserted before the child row is inserted.
[0163] Next, PDBMS 120 initiates and commits blockchain transaction BT6, which deletes the child row in CHILD. PDBMS 120 also adds the TC block TB6 of blockchain transaction BT6 to member blockchain 104-B, which results in TC block TB6 being added to replica member blockchain 104-B3(620) when PDBMS 130 comes online.
[0164] In addition, blockchain transaction BT6 updates the parent row. This update will ensure that induced transaction dependencies are created in subsequent events that update or delete the parent row, as will occur in this diagram.
[0165] Finally, PDBMS 110 initiates and commits blockchain transaction BT7, which deletes its parent line. PDBMS 110 also adds the TC block TB7 of blockchain transaction BT7 to member blockchain 103-B, which results in TC block TB7 being added to replica member blockchain 103-B3 when PDBMS 130 goes live. Furthermore, as a result of blockchain transaction BT6 previously updating its parent line, an induced transaction dependency now exists between blockchain transactions BT7 and BT6.
[0166] Once online, in response to the detection of an induced transaction dependency between blockchain transactions BT4 and BT5, the PDBMS 130 may be forced to apply block TC TB4 before applying block TC TB5. This avoids inserting a child row before inserting the parent row, thus preventing a foreign key violation. Similarly, in response to the detection of an induced transaction dependency between blockchain transactions BT6 and BT7, the PDBMS 130 may be forced to apply block TC TB6 before applying block TC TB7. This avoids deleting the parent row before deleting the child row, thus preventing a foreign key violation.
[0167] Applied to DOCS
[0168] In DOCS, the attributes of documents in a document collection correspond to the columns of rows in a database table. In DOCS, a document collection can be a collection of XML documents, and attributes can be elements from those XML documents; a document collection can also be a collection of JSON documents, and attributes can be fields from those JSON documents.
[0169] Attributes in a DOCS collection may be subject to uniqueness constraints, which require that an attribute value be unique among other attribute values in the collection. Similarly, a foreign key constraint on a child attribute of a child collection in a DOCS can require that the child attribute value exist as a value in the parent attribute of the document's parent collection.
[0170] The procedures described above for avoiding replication timing inconsistencies can be used in DOCS. For example, a set of tombstones for documents can be used similarly to a tombstone table. When a uniqueness constraint attribute in a particular document is updated to a new value, an induced transactional dependency can be created between documents in the tombstone set and that particular document. Similarly, when a child document has a condition similar to the parent update condition described earlier, the parent document is updated.
[0171] Hardware Overview
[0172] According to one embodiment, the techniques described herein are implemented by one or more dedicated computing devices. The dedicated computing device may be hardwired to execute the techniques, or may include digital electronic devices permanently programmed to execute the techniques, such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), or may include one or more general-purpose hardware processors programmed to execute the techniques according to program instructions in firmware, memory, other storage devices, or combinations thereof. Such a dedicated computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to implement the techniques. The dedicated computing device may be a desktop computer system, a portable computer system, a handheld device, a networking device, or any other device that combines hardwired and / or program logic to implement the techniques.
[0173] For example, Figure 7 This is a block diagram illustrating a computer system 700 on which embodiments of the present invention may be implemented. The computer system 700 includes a bus 702 or other communication mechanism for conveying information and a hardware processor 704 coupled to the bus 702 for processing information. The hardware processor 704 may be, for example, a general-purpose microprocessor.
[0174] Computer system 700 also includes main memory 706, such as random access memory (RAM) or other dynamic storage devices, coupled to bus 702 for storing instructions and information to be executed by processor 704. Main memory 706 can also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 704. When such instructions are stored in a non-transitory storage medium accessible to processor 704, computer system 700 becomes a dedicated machine customized for performing the operations specified in the instructions.
[0175] The computer system 700 also includes a read-only memory (ROM) 708 or other static storage device coupled to the bus 702 for storing static information and instructions of the processor 704. A storage device 710, such as a disk, optical disk, or solid-state drive, is provided and coupled to the bus 702 for storing information and instructions.
[0176] Computer system 700 can be coupled via bus 702 to a display 712, such as a cathode ray tube (CRT), for displaying information to the computer user. Input device 714, including alphanumeric keys and other keys, is coupled to bus 702 to transmit information and command selections to processor 704. Another type of user input device is a cursor control 716, such as a mouse, trackball, or arrow keys, used to transmit directional information and command selections to processor 704 and to control cursor movement on display 712. Such input devices typically have two degrees of freedom on two axes (a first axis (e.g., x) and a second axis (e.g., y)), allowing the device to specify a position in a plane.
[0177] Computer system 700 may implement the techniques described herein using custom hard-wired logic, one or more ASICs or FPGAs, firmware, and / or program logic. This custom hard-wired logic, one or more ASICs or FPGAs, firmware, and / or program logic, combined with the computer system, enables computer system 700 to become a special-purpose machine or to program computer system 700 as a special-purpose machine. According to one embodiment, the techniques herein are executed by computer system 700 in response to processor 704 executing one or more sequences of one or more instructions contained in main memory 706. Such instructions may be read into main memory 706 from another storage medium, such as storage device 710. Execution of the instruction sequence contained in main memory 706 causes processor 704 to perform the processing steps described herein. In alternative embodiments, hard-wired circuitry may be used instead of software instructions or in combination with software instructions.
[0178] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, or solid-state drives, such as storage device 710. Volatile media include dynamic memory, such as main memory 706. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips, or magnetic tape cassettes.
[0179] Storage media differ from transmission media but can be used in conjunction with them. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including wires containing bus 702. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.
[0180] Carrying one or more sequences of instructions to processor 704 for execution can involve various forms of media. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit them over a telephone line using a modem. A modem local to computer system 700 may receive data over the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on bus 702. Bus 702 carries the data to main memory 706, from which processor 704 retrieves and executes the instructions. The instructions received by main memory 706 may optionally be stored on storage device 710 before or after execution by processor 704.
[0181] Computer system 700 also includes a communication interface 718 coupled to bus 702. Communication interface 718 provides bidirectional data communication coupled to network link 720, which connects to local network 722. For example, communication interface 718 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem providing data communication connectivity to a corresponding type of telephone line. As another example, communication interface 718 may be a LAN card providing data communication connectivity to a compatible local area network (LAN). A wireless link may also be implemented. In any such implementation, communication interface 718 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.
[0182] Network link 720 typically provides data communication to other data devices via one or more networks. For example, network link 720 may provide a connection to host computer 724 or to data devices operated by Internet Service Provider (ISP) 726 via local network 722. ISP 726, in turn, provides data communication services via a global packet data communication network now commonly referred to as the “Internet” 728. Both local network 722 and Internet 728 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks, as well as signals on network link 720 and through communication interface 718, are example forms of transmission media that carry digital data to or from computer system 700.
[0183] Computer system 700 can send messages and receive data, including program code, through one or more networks, network links 720, and communication interfaces 718. In the Internet example, server 730 can transmit code requesting applications via the Internet 728, ISP 726, local network 722, and communication interface 718.
[0184] The received code can be executed by processor 704 when it is received, and / or stored in storage device 710 or other non-volatile storage device for later execution.
[0185] In the foregoing description, embodiments of the invention have been described with reference to numerous specific details that may vary depending on the implementation. Therefore, the description and drawings are to be viewed in an illustrative rather than restrictive sense. The sole and exclusive indication of the scope of the invention, and what the applicant intends to define as the scope of the invention, is the literal and equivalent scope of the set of claims published in the specific form of the set of claims issued from this application, including any subsequent amendments.
[0186] Software Overview
[0187] Figure 8 This is a block diagram of a basic software system 800 that can be used to control the operation of computer system 700. The software system 800 and its components, including their connections, relationships, and functions, are merely exemplary and do not imply limitation on the implementation of one or more example embodiments. Other software systems suitable for implementing one or more example embodiments may have different components, including components with different connections, relationships, and functions.
[0188] A software system 800 is provided to guide the operation of the computer system 700. The software system 800, which may be stored on system memory (RAM) 706 and fixed storage devices (e.g., hard disk or flash memory) 710, includes a kernel or operating system (OS) 810.
[0189] OS 810 manages the low-level aspects of computer operations, including managing process execution, memory allocation, file input and output (I / O), and device I / O. One or more applications, designated 802A, 802B, 802C…802N, can be “loaded” by system 800 (e.g., transferred from fixed storage device 710 to memory 706) for execution. Applications or other software intended for use on computer system 700 can also be stored as a set of downloadable computer-executable instructions, for example, for downloading and installing from Internet locations (e.g., web servers, app stores, or other online services).
[0190] Software system 800 includes a graphical user interface (GUI) 815 for receiving user commands and data graphically (e.g., "click" or "touch gestures"). System 800 can then act on these inputs according to instructions from operating system 810 and / or (one or more) applications 802. GUI 815 is also used to display the results of the actions from OS 810 and (one or more) applications 802, to which the user can provide additional input or terminate the session (e.g., log off).
[0191] OS 810 can execute directly on the bare hardware 820 of computer system 700 (e.g., one or more processors 704). Alternatively, a hypervisor or virtual machine monitor (VMM) 830 can be inserted between the bare hardware 820 and OS 810. In this configuration, VMM 830 acts as a software “buffer” or virtualization layer between the bare hardware 820 and OS 810 of computer system 700.
[0192] VMM 830 instantiates and runs one or more virtual machine instances (“guest machines”). Each guest machine includes a “guest” operating system, such as OS 810, and one or more applications designed to run on the guest operating system, such as application(s)802. VMM 830 presents a virtual operating platform to the guest operating system and manages the execution of the guest operating system.
[0193] In some cases, VMM 830 can allow a guest operating system to run as if it were running directly on the bare hardware 820 of the computer system 700. In these cases, the same version of the guest operating system configured to run directly on the bare hardware 820 can also run on VMM 830 without modification or reconfiguration. In other words, in some situations, VMM 830 can provide complete hardware and CPU virtualization to the guest operating system.
[0194] In other cases, the guest operating system can be specifically designed or configured to run on the VMM830 for improved efficiency. In these cases, the guest operating system "knows" that it is running on the virtual machine monitor. In other words, in some situations, the VMM830 can provide paravirtualization to the guest operating system.
[0195] A computer system process includes the allocation of hardware processor time and memory allocation (physical and / or virtual). Memory allocation is used to store instructions executed by the hardware processor, to store data generated by the hardware processor executing instructions, and / or to store the hardware processor state (e.g., register contents) between the allocation of hardware processor time and when the computer system process is not running. Computer system processes run under the control of the operating system and can also run under the control of other programs executing on the computer system.
[0196] cloud computing
[0197] This article generally uses the term "cloud computing" to describe a computing model that enables on-demand access to a shared pool of computing resources, such as computer networks, servers, software applications, and services, and allows for the rapid provisioning and release of resources with minimal management effort or service provider interaction.
[0198] Cloud computing environments (sometimes called cloud environments or the cloud itself) can be implemented in various ways to best suit different requirements. For example, in a public cloud environment, the underlying computing infrastructure is owned by an organization that makes its cloud services available to other organizations or the public. In contrast, private cloud environments are generally designed for use by a single organization or within a single organization. Community clouds are designed to be shared by multiple organizations within a community; while hybrid clouds include two or more types of clouds (e.g., private, community, or public) bound together by data and application portability.
[0199] Typically, cloud computing models deliver some of the responsibilities that might have previously been provided by an organization's own IT department as service layers within the cloud environment for use by consumers (inside or outside the organization, depending on the public / private nature of the cloud). Depending on the specific implementation, the precise definition of the components or features provided by or within each cloud service layer can vary, but common examples include: Software as a Service (SaaS), where consumers use software applications running on cloud infrastructure while the SaaS provider manages or controls the underlying cloud infrastructure and applications; Platform as a Service (PaaS), where consumers can use software programming languages and development tools supported by the PaaS provider to develop, deploy, and otherwise control their own applications while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything in the runtime execution environment); and Infrastructure as a Service (IaaS), where consumers can deploy and run arbitrary software applications and / or provide processing, storage, networking, and other basic computing resources while the IaaS provider manages or controls the underlying physical cloud infrastructure (i.e., everything below the operating system layer). Database as a Service (DBaaS) is a service in which consumers use database servers or database management systems running on cloud infrastructure, while the DbaaS provider manages or controls the underlying cloud infrastructure, applications, and servers, including one or more database servers.
Claims
1. A method, the method comprising: In a Blockchain Application (BCA) network, a specific BCA participant of the BCA network reads a first block from a specific blockchain copy of a specific blockchain, wherein the first block includes a specific transaction context of a specific transaction executed by the initiator BCA participant of the BCA network; The multiple transactions executed by the BCA network include the specific transaction, each of the multiple transactions is recorded by the blockchain of the BCA network, and each of the multiple transactions is associated with a corresponding transaction promotion time, the corresponding transaction promotion time specifying when each transaction modified a persistent object; The specific transaction context includes a first set of one or more records, each of which corresponds to a corresponding persistent object modified by the specific transaction. For each record in the first set of the one or more records, each record specifies a corresponding last promotion time, and for the corresponding persistent object of each record, the corresponding last promotion time specifies the corresponding transaction promotion time of the transaction in the plurality of transactions that previously modified the corresponding persistent object; The specific BCA participant maintains a checkpoint value, which specifies the previous transaction promotion time of a transaction recorded by a previously applied block of the specific blockchain; To determine if any transactions that a particular transaction depends on have been committed; A specific record in the first set of one or more records specifies the corresponding last promotion time for a particular persistent object. The determination includes determining that the last promotion time of the specific persistent object is less than the checkpoint value.
2. The method according to claim 1, The second record in the first set of one or more records specifies the corresponding last promotion time of the second persistent object and identifies the transaction identifier of the previous transaction of the second persistent object; The determination includes determining that the last promotion time of the second persistent object is not less than the checkpoint value; In response to determining that the last promotion time of the second persistent object is not less than the checkpoint value, it is determined whether the previous transaction has been committed based on the transaction identifier.
3. The method according to claim 2, further comprising: After determining that the previous transaction has been committed, the specific transaction is applied.
4. The method according to claim 2, The first set of one or more records specifies the previous and new values of the attribute that was changed by the specific transaction at the initiator BCA participant; The method includes: The specific BCA participant replays the specific transaction context to generate a second set of one or more records, the second set of one or more records specifying the previous and new values of attributes changed by the specific transaction at the specific BCA participant; The specific BCA participant votes to submit the specific transaction based on a comparison of a first set of one or more records with a second set of one or more records.
5. The method according to any one of claims 1-4, comprising: The first block includes the transaction promotion time in the earliest processing of one or more transactions in process when a copy of the first block is added to the copy of the blockchain at the initiator BCA participant; The method further includes changing the checkpoint value to the earliest processed transaction promotion time.
6. The method according to any one of claims 1-4, wherein the BCA network is a network of a database management system, and the particular persistent object is a row in a database table.
7. The method according to any one of claims 1-4, wherein the first set of said one or more records comprises: A first record in a first set of one or more records, the first record specifying a change to an attribute of a specific persistent object from a first set of persistent objects, the attribute being subject to uniqueness constraints, the change to the attribute of the specific persistent object changing the attribute from a first value to a second value. A second record in a first set of one or more records, the second record specifying the insertion of a persistent object having an attribute set to the first value.
8. The method according to any one of claims 1-4, wherein the first set of said one or more records comprises: A first record in a first set of one or more records, the first record specifying a change to a sub-property of a sub-persistent object from a first set of sub-persistent objects, the sub-property being subject to a foreign key constraint, the sub-property having a value equal to the primary key value of the parent persistent object; The second record in the first set of one or more records, the second record specifying the modification to the parent persistent object.
9. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the steps of the method according to any one of claims 1-8 to be performed.
10. A computing device, comprising: One or more processors; as well as One or more non-transitory computer-readable storage media storing instructions that, when executed by the one or more processors, cause the steps of the method according to any one of claims 1-8 to be performed.
11. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1-8.
Citation Information
Patent Citations
Native persistent store support for blockchains
US20210034606A1
Transaction processing for database in persistent system
CN105210060A
Multi-database log with multi-item transaction support
CN106462449A
