Methods, media, and systems for providing persistent storage support for a blockchain

Abstract

Distributed ledger data is stored in a distributed persistent storage system that includes multiple persistent storage systems as distributed ledger participants. The distributed ledger data is maintained using native capabilities of the persistent storage systems. The distributed ledger data is replicated as persistent data objects in an "object ledger repository" that are replicated at each persistent storage system. Changes at one persistent storage system are recorded in a block in a distributed blockchain that is distributed across each of the other distributed ledger participants. The other distributed ledger participants read the changes from the blockchain and apply the changes to the respective copies at each of the other distributed ledger participants.

Description

Technical Field

[0001] This invention relates to enterprise-level persistent storage systems, such as file systems, database management systems, key-value stores, and document storage systems, and particularly to the native implementation of blockchain technology within enterprise-level persistent storage systems. Background Technology

[0002] Traditional persistent storage systems can handle large amounts of data read and / or modified by multiple concurrent users. Enterprise persistent storage systems (PSS) maintain the data integrity, consistency, and security of persistent data managed by the PSS.

[0003] PSS can be, for example, a database managed by a DBMS, a file stored in a file system, a key-value object managed by a key-value store, or a document managed by a Document Storage System (DOCS) (e.g., a JSON document). PSS stores data in persistent data objects. For example, persistent data objects can be database tables managed by a DBMS, key-value pairs managed by a key-value store, documents managed by a DOCS, or files managed by a file system. Persistent data objects can be composite data objects comprising multiple persistent data objects. For example, in a DBMS, a database table can include multiple rows as persistent data objects. In a file system, a file can include multiple blocks as data objects, and a directory can be a persistent object containing files and can even contain other directories as persistent data objects. In a DOCS, a persistent data object can be a collection of JSON documents.

[0004] Various PSSs (such as DBMSs and key-value stores) offer superior performance for updating and querying persistent repositories. PSS users can rely on the validity of persistent data objects stored in the PSS because changes to these objects are controlled by a centralized authority such as a company or organization. Generally, any changes made to data stored in the PSS are performed by users belonging to the same central authority and are therefore subject to its access policies.

[0005] For segregated entities (e.g., businesses, organizations, governments), it is becoming increasingly important to share read and write access to mission-critical shared data in a manner that allows entities to rely on the validity of the shared data. This type of data is referred to in this paper as distributed ledger data.

[0006] For traditional PSSs, systems used to manage distributed ledger data can be implemented using a federated approach. This approach involves multiple traditional PSSs, each controlled by an independent storage system participant trusted by the other participants in the federation. Each participant maintains its own replica of the distributed ledger data in its own private PSS, trusted by the other participants in the federation to make only valid changes to the replica and to transmit such changes to the other participants in a trusted manner.

[0007] Unfortunately, synchronization issues can lead to out-of-sync copies, requiring dispute resolution among participants in the consortium, or even transaction reversal, which can be inefficient. Important decisions and operations relying on distributed ledger data may be delayed until participants can reconcile their copies. Furthermore, consortium participation is limited to participants who can truly be trusted and can be reasonably protected from malicious attacks on the distributed ledger data.

[0008] On the other hand, blockchain technology provides a means of maintaining distributed ledger data without requiring every participant to be trusted, at least at the same level as in a consortium approach. Each participant maintains a copy of the distributed ledger data. Changes to the distributed ledger data are recorded in the blockchain, which includes linked blocks with cryptographic hashes. Participants use a consensus mechanism to coordinate the addition of blocks, ensuring that participants record the same changes to the distributed ledger data. Even if some participants attempt to corrupt their copies of the distributed ledger data, the validity of the distributed ledger data is ensured.

[0009] Unfortunately, using blockchain technology to store distributed ledger data requires sacrificing many of the performance benefits offered by numerous PSSs (Programmable Securing Servers). To provide the benefits of both blockchain technology and such PSSs, an "off-chain storage" approach can be used. Under the off-chain storage approach, a version of the distributed ledger data resides in the PSS, while the blockchain itself is stored in a blockchain repository managed by the blockchain application.

[0010] Off-chain storage methods make application development extremely difficult because applications must manage data in both the PSS and blockchain storage simultaneously. Even more challenging are handling rollback and recovery operations, both of which must be performed by both the PSS and blockchain repositories.

[0011] Based on the foregoing, it is hoped that methods for managing distributed ledger data that can be stored and accessed within the PSS will be facilitated. Attached Figure Description

[0012] In the attached diagram:

[0013] Figure 1AThis is a diagram depicting a blockchain application network according to an embodiment of the present invention.

[0014] Figure 1B This is a diagram depicting the data structures used in embodiments of the present invention.

[0015] Figure 2 This is a diagram depicting a blockchain application process according to an embodiment of the present invention.

[0016] Figure 3 This is a diagram describing the data structures used in embodiments of the present invention.

[0017] Figure 4 It is a diagram depicting blocks in a blockchain according to an embodiment of the present invention.

[0018] Figure 5 This is a diagram depicting the recovery process of a blockchain application according to an embodiment of the present invention.

[0019] Figure 6 This is a diagram depicting the recovery process of a blockchain application according to an embodiment of the present invention.

[0020] Figure 7 This is a diagram depicting a blockchain process for applying DDL changes according to an embodiment of the present invention.

[0021] Figure 8 This is a diagram depicting the process of bootstrapping a blockchain application according to an embodiment of the present invention.

[0022] Figure 9 This is a diagram depicting a blockchain application network for a file system according to an embodiment of the present invention.

[0023] Figure 10 This is a diagram depicting a blockchain application network for a document storage system according to an embodiment of the present invention.

[0024] Figure 11 This is a diagram depicting a computer system that can be used to implement embodiments of the present invention.

[0025] Figure 12 A software system that can be used to control the operation of a computer system according to an embodiment of the present invention is described. Detailed Implementation

[0026] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the invention. However, it will be apparent that the invention may be practiced without these specific details.

[0027] General Overview

[0028] This document describes a method for maintaining distributed ledger data within a replicated PSS, which comprises multiple PSSs as participants in the distributed ledger. In various embodiments, the distributed ledger data is maintained using the native capabilities of the PSS. The distributed ledger data is replicated as a persistent data object in a "ledger of objects," which is copied at each PSS. Changes at one PSS are recorded within a block in a distributed blockchain distributed across each PSS. Other PSSs read the changes from the blockchain and apply them to the corresponding copies at each of the other PSSs. Therefore, this method is referred to as blockchain application.

[0029] Blockchain applications can be used to replicate various forms of PSS (Physical Storage Service) repository objects. In a DBMS (DBMS), the repository of objects is a table, where each record or row is an object in the repository. In a file system, the repository of objects is a directory, where each directory and its files are objects in the repository. In a Document Storage System (DOCS), the repository of objects is a collection of documents, where each document is an object in the repository.

[0030] Transaction processing

[0031] In transaction processing, changes to persistent data objects are performed by transactions. A transaction is a collection of operations that modify persistent data objects. All changes made within a transaction are atomic. Changes are made permanent by committing the transaction. If a transaction cannot be committed, aborts, or otherwise fails to commit, it is rolled back.

[0032] In distributed transaction processing, multiple PSS participants commit a distributed transaction to modify corresponding data objects using a phased approach managed by a transaction manager. The phased approach includes at least a phase where a participant reaches a ready state. When in the ready state, a participant has performed the operations necessary to reach a state where the participant can guarantee that it can commit the transaction but has not yet committed it. A participant can be instructed by the transaction manager to reach the ready state and communicate to the transaction manager whether it has reached the ready state.

[0033] As used in this article, transaction processing also requires journaling. In journaling, changes made by a transaction to persistent data objects are stored in a persistent journal. A persistent journal consists of journal entries that record the changes made to data objects by a transaction. Each entry can specify which fields, columns, elements, or parts of the data object were changed, and the new values ​​created by those changes. It can also specify the old values ​​that were replaced by the new values.

[0034] In a write-ahead journal, a record of the changes made by a transaction is persistently stored until the transaction is committed and / or reaches a ready state. Furthermore, the journal can be read by other PSS instances to replicate persistent data objects.

[0035] Blockchain applications in replicated DBMS

[0036] Blockchain applications were initially described in the context of DBMS. However, blockchain applications can be used in other types of PSSs that have the ability to make changes using distributed transaction processing.

[0037] In the context of a DBMS, distributed ledger data is maintained as distributed database tables using the DBMS's native capabilities. The distributed database is replicated at each DBMS. Changes at one DBMS are recorded within blocks in a distributed blockchain, which is distributed across each DBMS. The DBMS reads the changes from the blockchain and applies them to the corresponding copies of the distributed database tables.

[0038] The distributed database tables replicated using this method are referred to as ledger tables in this document. The term ledger table does not imply that a ledger is merely an append-only data structure. A ledger table can be any database table. The collection of multiple DBMSs participating in a blockchain application to replicate a specific ledger table is collectively referred to in this document as the Blockchain Application Network (“BCA Network”). Each DBMS in the BCA Network may be simply referred to as a DBMS participant in this document. Changes that can be replicated using a blockchain application include DML (Database Manipulation Language) changes (i.e., changes to the data in the ledger table) or DDL (Database Definition Language) changes (which include changes to the database dictionary (including the structure of the ledger table) to achieve the desired change to the BCA Network).

[0039] In blockchain applications, a key characteristic of blockchain is chain-state consensus, meaning that participants have agreed (through a consensus protocol) on the state of the blockchain and what changes to that state are. Chain-state consensus is fully utilized in blockchain applications. In blockchain applications, changes are recorded on the ledger. In effect, the blockchain represents the state of the ledger and its corresponding copies. Because participants have agreed on the state of the blockchain, they have also agreed on the state of the ledger and what changes to that state are possible.

[0040] In this embodiment, any changes to the ledger must be initiated by the leader of the BCA network (referred to as the initiator DBMS). The ledger and blockchain are distributed. Therefore, each DBMS participant maintains a copy of the ledger and blockchain. The initiator DBMS coordinates updates to the copies of the ledger and blockchain with other DBMS participants (referred to herein as follower DBMSs). DBMS participants frequently and / or periodically alternate in the role of initiator DBMS, switching from the initiator DBMS role to the follower DBMS role.

[0041] DBMS participants have been configured by, for example, the database administrator of each of the respective DBMS participants to participate in the BCA network to update distributed tables. Changes to the ledger tables can be initiated by the database application client by simply issuing a DML statement to the initiating DBMS through a database session established with the initiating DBMS.

[0042] According to an embodiment, a blockchain application applies changes to a ledger and the corresponding blockchain by executing a variant of a two-phase commit. In the first phase, the prepare-to-commit phase, blocks specifying the changes are added to the blockchain at each DBMS participant, who then enters a prepare-to-commit state for the data changes to their copy of the ledger. In the second phase, DBMS participants add blocks recording the results of their commit decisions and commit or abort the changes accordingly. DBMS participants use a log-based consensus algorithm to control whether blocks are added to the blockchain.

[0043] Generally, in a consensus algorithm, the participants reach a consensus on whether to accept a proposal, and then take an action on the proposal when the consensus is to accept it. A proposal can be adding the proposed block to the blockchain, adding the proposed log record (including the block) to the log, or taking another action.

[0044] In log-based consensus algorithms, participants reach a consensus on how to append log entries to the log (specifically, the order in which these entries are appended and what the content of each entry should be). A copy of the log is maintained by each participant. In blockchain applications, each log entry is stored as a block in the blockchain, and the order of the blocks mirrors the order of the log entries. Log-based consensus algorithms will be explained in more detail later.

[0045] Blockchain applications are implemented as a native feature of the DBMS. The blockchain network is created and configured by issuing DDL commands to each DBMS participant. Changes to the ledger and associated blockchain are integrated into the DBMS's native transaction and replication capabilities, including the ability to replicate changes in parallel. Performance can be monitored using available DBMS tools. Database applications can be programmed to read and write to the ledger in the same way that applications are configured to read and write to traditional tables. Implementing the ledger and its associated blockchain does not require external data structures (i.e., outside the DBMS).

[0046] Explanatory BCA Network

[0047] Figure 1A A BCA network 101 according to an embodiment of the present invention is depicted. (Reference) Figure 1A The BCA network 101 includes initiator DBMS 110 and follower DBMS 120 and follower DBMS 130. Each of these DBMSs participates in maintaining a distributed ledger, which is securely updated via protocols used to append blocks to the distributed blockchain. Changes to the distributed ledger occur at and are initiated by initiator DBMS 110.

[0048] As mentioned earlier, BCA participants rotate roles with the initiator DBMS. Therefore, Figure 1A This indicates the role of a BCA participant at a specific point in time. At another point in time, follower DBMS120 or follower DBMS130 could be the initiator DBMS of BCA network 101. Switching the role of the initiator DBMS among BCA participants is referred to as a change of leadership in this paper, where the initiator DBMS switching to the role of a follower DBMS is said to have lost leadership, and the new initiator DBMS is said to have gained leadership. The initiator DBMS retains leadership during its leadership term.

[0049] BCA Data Structure

[0050] Figure 1A The diagram illustrates the various data structures distributed as replicas across the BCA network 101. Figure 1B The constituent copies of these distributed data structures are depicted, and these will be described further below.

[0051] refer to Figure 1A and 1B BCA network 101 includes distributed ledger table 102 (see...) Figure 1B Distributed ledger table 102 includes copies stored as ledger table 102-1, ledger table 102-2, and ledger table 102-3 on the initiator DBMS 110, follower DBMS 120, and follower DBMS 130, respectively. (See also...) Figure 1A )

[0052] Distributed blockchain 104 includes copies of the distributed blockchain, namely blockchain 104-1, blockchain 104-2, and blockchain 104-3. These blockchain copies are stored on the initiator DBMS110, follower DBMS120, and follower DBMS130, respectively.

[0053] Blockchains 104-1, 104-2, and 104-3 are stored in a distributed blockchain log (“BC log”) BC log 103, which includes BC logs 103-1, 103-2, and 103-3, and are stored on the initiator DBMS 110, follower DBMS 120, and follower DBMS 130, respectively. According to an embodiment, the BC log is a database table including at least one column for storing the blockchain, and each row of the BC log (BC log record) stores a block of the blockchain.

[0054] BC Log 103-1, BC Log 103-2, and BC Log 103-3 comprise a distributed BC Log 103 and each contains a copy of the log records in BC Log 103. Each log record includes a log identifier (log ID), which uniquely identifies the log record in BC Log 103. Each copy of the log record includes the same log ID. The log IDs are ordered such that the log ID of a log record is greater than any other log record that maintains an earlier block in the blockchain stored in the ledger.

[0055] In this embodiment, the log ID in the BC log is a monotonically increasing value, such as an integer. Regarding the blockchain stored in the BC log, the log ID of a log record in the BC log indicates the block position of the block contained in that log record, where the block position is the location of the block within the blockchain. For example, a log record with log ID 0 stores the originating block in the corresponding blockchain, a log record with log ID 1 stores the second block in the corresponding blockchain, and so on.

[0056] BCA network 101 also includes a distributed collection of blockchain processes 105. A blockchain process is a computer-executable process or function whose implementation is accepted and visible to members of BCA network 101, and executed by members of BCA network 101 to modify the ledger. According to embodiments, blockchain processes can be implemented using database computer languages ​​such as PL / SQL or other languages ​​such as Java, JavaScript, Python, or Solidity. Blockchain process 105 includes replicas of blockchain processes 105-1, 105-2, and 105-3, which are stored on initiator DBMS 110, follower DBMS 120, and follower DBMS 130, respectively.

[0057] A BCA network client can request changes to the ledger table by initiating a database transaction 115 with the initiator DBMS 110. Through this operation, the client can issue database statements to modify ledger table 102-1. Issuing database statements initiates database transaction 115.

[0058] Database statements can invoke blockchain procedures, passing the values ​​of the arguments used in the blockchain procedure. Alternatively, database statements can directly specify DML operations on the distributed ledger.

[0059] To explain in more detail, the initiator DBMS110 initiates a distributed transaction to replicate DML operations across the BCA network 101. Changes are replicated by replicating log records at follower DBMS120 and 130. The initiator DBMS110 coordinates with follower DBMS120 and 130 to add log records to the distributed BC log 103. Adding log records effectively adds blocks to the distributed blockchain 104. Using the distributed transaction processing coordinated by the initiator DBMS110, local transactions 125 executed within follower DBMS120 and 135 executed within follower DBMS130 apply the changes specified in the log records to ledger table copies 102-2 and 102-3, respectively.

[0060] A DBMS can be a DBMS participant in multiple different BCA networks. As will be explained in more detail, the database dictionary of each DBMS participant in a BCA network defines the BCA network, its constituent components, and its configuration.

[0061] Log-based consensus algorithms

[0062] According to an embodiment, BCA network 101 uses a proponent-based log-based consensus algorithm to control whether DBMS participants add log records to the distributed BC log or whether DBMS participants should take other types of actions. Generally, the proponent-based log-based consensus algorithm requires the proponent DBMS to propose to the follower DBMSs to append log records to the log and manages the consensus among the participating DBMSs regarding whether to accept the proposal.

[0063] In a proponent-based log-based consensus algorithm, participants in the BCA network periodically follow an election protocol to elect a proponent DBMS for each leadership term. Leadership terms are not necessarily of equal duration, and elections are not necessarily held at fixed intervals. For the BCA network, at any given time, there is only one active leadership term (“current leadership term”) and one proponent DBMS. Each leadership term is identified by a “term ID,” a value from an ordered value field (e.g., an integer) that uniquely identifies the leadership term and its order of appearance relative to other leadership terms. For example, term IDs 1, 2, and 3 uniquely identify three consecutive leadership terms.

[0064] Initiator-based consensus protocols require the initiator DBMS to frequently send different types of messages to each of the other participant DBMSs. These messages include proposal log messages and heartbeat messages. These messages can include common "piggybacked payloads," which contain fields that transmit information about the BCA network state when the message is sent. The piggybacked payload includes the term ID of the leader's term of office when the initiator DBMS sends the message. As will be explained later, the piggybacked payload can include other types of information.

[0065] In response to the initiating DBMS sending a message to propose a log entry, other DBMS participants determine whether to accept or reject the proposal and transmit this determination to the initiator. One or more criteria governing DBMS participants' acceptance of a proposal are referred to herein as proposal acceptance criteria. Proposal acceptance criteria may include, for example, that the proposed log entry includes a valid block to be added to the blockchain, or that the term ID in the payload of the proposed log entry message is the ID of the current leader's term.

[0066] DBMS participants determine whether a proposal is accepted or rejected based on one or more consensus criteria, which are at least in part based on how many DBMS participants accept or reject the proposal. This determination is referred to herein as the consensus outcome. An example of a consensus criterion is the quorum of DBMS participants who log in accepting the proposal. The initiating DBMS transmits the consensus outcome to the other DBMS participants, which will be described in further detail. Alternatively, a decentralized communication model can be used to determine the consensus outcome.

[0067] DBMS participants transmit their acceptance of the proposal. In one embodiment, a DBMS participant sends a digitally signed message to the initiating DBMS participant, specifying whether the DBMS participant accepts or rejects the proposal.

[0068] When acceptance is a consensus outcome related to adding a log entry to the BC log, adding a block to the blockchain, or taking any other kind of action, the log entry, blockchain, or other action is said to be accepted. When rejection is a consensus outcome related to adding a log entry to the BC log, adding a block to the blockchain, or taking any other kind of action, the log entry, blockchain, or other kind of action is said to be rejected.

[0069] As a clarification, in an initiator-based log-based consensus algorithm, follower DBMSs do not accept or reject log entries; instead, they accept or reject proposed log entries. The initiator DBMS accepts or rejects log entries on behalf of the BCA network by determining the consensus result. When log entries are accepted in this way, the log entries and the blocks contained therein are referred to in this document as being added to or appended to the BC log or blockchain, respectively.

[0070] Log-based example algorithms include Raft and Paxos. An example of Raft is described in June 2014 in *In Search of an Underable Consensus Algorithm* by Diego Ongaro and John Ousterhout of Stanford University, the entire contents of which are incorporated herein by reference. Embodiments of this invention are not limited to any particular consensus algorithm. The consensus algorithm used to determine whether to accept or reject a log entry or to add a block to the blockchain is referred to herein as a blockchain consensus algorithm.

[0071] Log ID

[0072] A log ID is an identifier derived from an ordered field of a value (e.g., an integer) that uniquely identifies an accepted log record in the BC log. This order is referred to herein as the log ID order. Within the BC log, a log record's log ID is unique relative to the log IDs of other accepted log records.

[0073] The log IDs of the log records reflect the order in which they were accepted. The log ID of the first accepted log record is followed by the log ID of the second accepted log record, and so on.

[0074] A proposal to add a log record includes the proposed log ID. The log ID of an accepted log record is referred to as the accepted log ID. Follower DBMSs do not accept proposals to add log records unless the corresponding proposed log ID is the next log ID in the log ID order after the last accepted log record.

[0075] The initiating DBMS can publish multiple proposals before any proposal is accepted by the BCA network. Follower DBMSs only accept proposals in the order of their log IDs. Therefore, follower DBMSs can postpone accepting a proposal's log entry until they are informed that the consensus result for the log entry with the most recent log ID is acceptance.

[0076] The acceptance / rejection of blocks by the transmission participants

[0077] According to an embodiment, the consensus result of one or more blocks is signaled via a global log ID watermark (“watermark”) transmitted by the initiating DBMS in the payload. Any log ID less than or equal to the watermark is considered by the participating DBMS to have an accepted consensus result. When the initiating DBMS determines that the consensus result of a proposed log record is accepted, the initiating DBMS adjusts the watermark to the proposed log ID, provided that any proposal with the previous log ID has an accepted consensus result. When a participating DBMS determines that the log ID of a proposed log record is covered by the watermark (i.e., less than or equal to the watermark), the participating DBMS assumes that the consensus result of the log record is accepted.

[0078] A participating DBMS may receive one or more other proposed log records with the same log ID as a previously proposed log record. In this case, when a participant learns that the consensus result for one of the proposed log records is acceptance, that participant can discard any other proposed log records. It should be noted that the local watermark maintained by the participant should not overwrite the log ID at this time, because the previously proposed log record should not be accepted.

[0079] Additional terminology for consensus-driven actions

[0080] Various operations of BCA Network 101 or any of its components are described as performing actions according to or by using a consensus algorithm. The consensus algorithm requires participants to follow a protocol used by the algorithm, which may require steps such as: the initiating DBMS transmits a proposal (e.g., a log record and log ID of the proposal); other DBMS participants receive the proposal and respond by voting to accept or reject it; the initiating DBMS determines the consensus result based on the received votes according to a log-based consensus protocol; and then the initiating DBMS transmits the consensus result to the participants. When voting, accepting, or rejecting, or when an operation is described as being performed according to or by using a consensus algorithm, it can be assumed that participants are performing actions required by the protocol and according to the protocol.

[0081] A proposal has a proposal state, which indicates the progress towards determining a consensus outcome for the proposal. Before a consensus outcome is determined for a proposal, the proposal state is pending, and the proposal is referred to as pending in this document. When a consensus outcome has been determined for a proposal, the proposal state is resolved, and the proposal is referred to as resolved in this document.

[0082] As will be explained in more detail, BCA participants follow a consensus algorithm to determine whether to commit the distributed transaction. The consensus result used to commit the distributed transaction is called the commit consensus result. The follower DBMS's acceptance or rejection of the proposal to commit the distributed transaction is referred to in this document as the commit or non-commit vote, respectively.

[0083] Overview of Blockchain Application Replication

[0084] Figure 2 This is a flowchart depicting a blockchain application according to an embodiment of the present invention. This process is referred to as the DML blockchain application process because it replicates DML changes to ledgers across the BCA network.

[0085] This flowchart depicts the normal process of the operation, where a blockchain application is used to replicate DML changes made by transactions without encountering error conditions or exceptions. Handling error conditions and exceptions will be explained later. The operation is executed by the initiator DBMS110 and follower DBMSs 120 & 130 in response to transactions initiated by database clients that have already established database sessions with the initiator DBMS110.

[0086] refer to Figure 2 The initiating DBMS 110 performs database operations (205) on ledger table 102-1 within the initiated database transaction. Such database operations include executing one or more database statements (including DML statements and one or more blockchain procedures of blockchain procedure 105). Database operations may include savepoints or even rollbacks.

[0087] The initiator DBMS110 receives a request from the client to commit the database transaction. In response, the initiator DBMS110 initiates a distributed transaction and prepares to commit the distributed transaction as part of a two-phase commit and is ready to commit (210).

[0088] A distributed transaction modifies the databases in each of the multiple participating DBMSs. The changes at each participating DBMS are made by local transactions initiated and / or executed by the participating DBMS to modify the data there as part of the distributed transaction. The initiator (“coordinator DBMS”) assigns a distributed transaction ID (“distributed txid”). Local transactions are referred to as branch transactions of the distributed transaction. Each DBMS assigns a local transaction ID to the corresponding branch transaction and associates the local transaction ID with the distributed txid.

[0089] The execution, commit, and termination of distributed transactions are coordinated by the initiating DBMS. According to an embodiment, distributed transactions are executed using a distributed transaction protocol specified in the X / OpenXA specification. Under this protocol, the initiating DBMS acts as the transaction manager for the distributed transaction.

[0090] The initiator DBMS110 prepares and submits a proposed log record with a log ID for BC log 103-1. The proposed log record is committed in a separate local transaction, although it has not yet been accepted by the follower DBMSs, for recovery purposes, which will be explained in more detail. According to the blockchain consensus algorithm, the initiator DBMS110 proposes to add the log record for BC log 103 to followers DBMS120 and 130. The proposed log record includes the proposed TC block (transaction context block) to be added to the distributed blockchain 104, along with the proposed log ID.

[0091] A TC block includes a transaction context. The transaction context includes the specification of the database operations performed by the initiating DBMS for the branch transaction, and the write set of one or more rows produced by performing these database operations. The write set in the proposed TC block is referred to herein as the proposed write set. TC blocks may include other information, such as the distributed txid of the distributed transaction.

[0092] The specifications for database operations may include the input argument values ​​of one or more blockchain procedures executed by the database operations. The execution of one or more blockchain procedures may result in the execution of one or more database statements that are not explicitly listed in the specifications.

[0093] As described later, follower DBMS120 and 130 replay the transaction context within the branch transaction initiated by follower DBMS120 and 130 (i.e., execute the sequence of database operations specified by the specification within the transaction context without committing the database operations) to generate a local write set. The comparison of the local write set with the proposed write set determines at least in part whether follower DBMS120 and 130 will vote to commit the distributed transaction.

[0094] Follower DBMS120 and 130 receive and accept proposed log entries. Follower DBMS120 and 130 insert and submit proposed log entries for BC logs 103-2 and 103-3, respectively (220).

[0095] The initiator DBMS110 determines that the consensus result of the proposed log record is acceptance based on the blockchain consensus algorithm. In response, the initiator DBMS110 updates the log record to indicate its acceptance and sends messages to the followers DBMS120 and 130 to notify them of the consensus result (230). As explained earlier, sending the message may include updating the global watermark by carrying a message transmitted by the initiator DBMS. After learning that the consensus result of the proposed log record is acceptance, the followers DBMS120 and 130 then determine that the block consensus result is acceptance and, in response, update the proposed log record in BC logs 103-2 and 103-3 respectively to indicate acceptance. The updating of the log record by the initiator DBMS110 and the followers DBMS120 and 130 in this way effectively adds the TC block to blockchains 104-1, 104-2, and 104-3 respectively.

[0096] Next, follower DBMSs 120 and 130 initiate local transactions (separate from the transaction used to commit log records) to replay the transaction context in the TC block to generate a local write set (235). The local write set is then compared with the proposed write set (240).

[0097] Follower DBMSs 120 and 130 determine which database transaction to vote on and commit based on a comparison of the local write set with the proposed write set (245). Generally, when determining whether to vote on and commit a distributed transaction, follower DBMSs 120 and 130 determine whether one or more DML commit criteria are met. DML commit criteria include whether the proposed write set matches the local write set, and if so, whether one or more other criteria (if any) are met. Rows in the write set should include the primary key of the ledger table or a similar unique identifier to ensure that copies of the same row can be matched for comparison.

[0098] Next, follower DBMS120 and 130 prepare to commit the proposed DB transaction (250). After preparing to commit, follower DBMS120 and 130 vote to commit the distributed transaction (255).

[0099] The initiator DBMS 110 determines the commit consensus result for submitting the distributed transaction based on a consensus algorithm that may differ from the blockchain consensus algorithm (260). The initiator DBMS 110 then generates a log record including a TA block (transaction management block) and commits this log record to BC log 103-1 (265) in a separate local transaction. The TA block includes information about the result of the distributed transaction, including whether the distributed database transaction has been committed, and the votes of the follower DBMSs.

[0100] The log record for the TA block is then proposed to follower DBMS120 and 130, who then accept the proposed log record and submit it to BC logs 103-2 and 103-3, respectively. In response to the consensus result for the log record indicating acceptance, initiator DBMS110 updates the log record to BC log 103-1 to indicate acceptance and commits the distributed transaction and the corresponding local transaction (270). In response to initiator DBMS110 transmitting the message indicating acceptance to follower DBMS120 and 130, follower DBMS120 and 130 update the log records in BC logs 103-2 and 103-3, respectively, to indicate acceptance of the log record (275).

[0101] In response to the update log record, follower DBMS120 and 130 read the TA block to determine from the TA block that the consensus result of the distributed transaction commit is acceptance (280). In response, follower DBMS120 and 130 then commit their respective local transactions (285).

[0102] Add log entries or blocks - Clarify

[0103] As shown above, adding a log entry to the BC log or adding a block to the blockchain involves various operations. These operations may include the initiating DBMS inserting the log entry into the BC log, proposing the log entry to the follower DBMSs, the follower DBMSs voting to accept the proposal, the initiating DBMS determining the block consensus result and transmitting the block consensus result to the follower DBMSs, and modifying the BC log to affect the block consensus result. For illustrative purposes, adding a log entry or block is described without specifying any or all of these different operations. However, it should be understood that such operations are performed as part of adding a log entry or block, even if such operations are not specifically mentioned.

[0104] Secure communication between DBMS participants and clients

[0105] The initiator DBMS 110 and follower DBMSs 120 and 130 can be connected via a wide area network (WAN), which can be public or private. Generally, secure network communication protocols are used to transmit messages between these DBMSs. Furthermore, the message may contain content digitally signed by the initiator or sender to authenticate that the content originates from the initiator or sender.

[0106] Blockchain applications can be used between DBMS participants and database clients in restricted or untrusted environments. To allow DBMS participants to securely participate in the BCA network, each DBMS participant runs a proxy server. Messages transmitted as part of the block application are sent to the proxy server using protocols such as HTTPS. Using a proxy server avoids the need for one database to directly connect to another, which might be unacceptable in deployments with the lowest trust levels.

[0107] Parallelization in Blockchain Applications

[0108] In practice, the re-implementation of a transaction context by a follower DBMS involves copying the DML changes specified within the transaction context. Parallel replication techniques can be used to apply DML changes. Generally, parallel processing of DML changes within a transaction involves each of one or more slave processes executing the corresponding portion of the DML changes.

[0109] In fact, follower DBMSs in the BCA network can each execute local transactions to generate a set of writes for multiple uncommitted distributed transactions that have been added to the TC block. In this case, among these local transactions, "dependent" local transactions can include DML changes that are transactionally dependent on the commit of an earlier local transaction. Enforcing transaction dependencies can require that the application and commit of DML changes be postponed until the commit of the earlier transaction. Blockchain applications should apply and commit DML changes in this manner.

[0110] The use of parallel replication in blockchain applications leads to a delay in applying DML changes in subordinate local transactions. BCA participants cannot unconditionally complete the local write set that generates subordinate local transactions, let alone unconditionally vote to commit the corresponding distributed transactions until an earlier local transaction is committed or guaranteed to be committed.

[0111] Supplementary data structures for blockchain applications

[0112] Figure 3 A more detailed description is provided of data objects for supporting blockchain applications targeting ledger tables, according to embodiments of the present invention. (Reference) Figure 3 It describes BC log 103-1, as well as table blockchain transactions 310 and blockchain voting 320, which are data objects used to support blockchain applications on DBMS 110 for ledger table 102-1.

[0113] BC Log 103-1 contains columns and multiple rows, namely, log records. Each log record contains the block from Blockchain 104-1 in the BLOCK column. Additionally, each log record includes its log ID in the LOGID column. Each log record in BC Log 103-1 specifies the block consensus result. Information about the block consensus result is stored in the OUTCOME column.

[0114] Information about DBMS participants and their roles in the block consensus results is stored in the CONSENSUSACTIVITY column. For each row in BC log 103-1, CONSENSUSACTIVITY identifies the initiating DBMS, follower DBMS, and their respective acceptance or rejection of the block.

[0115] Blockchain transaction 310 and blockchain vote 320 may contain information redundant with the information stored in BC log 103-1, but they can store such information in a more easily accessible manner. Blockchain transaction 310 and blockchain vote 320 are used for information and auditing purposes.

[0116] Blockchain transaction 310 stores information about transactions executed against ledger table 102-1. Each row in blockchain transaction 310 includes a distributed txid identifying the distributed transaction executed against ledger table 102-1 in the DISTR TXID (Distributed TXID) column, a log ID of the corresponding TC block of the distributed transaction in the LOG ID column, a user identifier of the user who published the distributed transaction in the USER (User) column, and a transaction status of the distributed transaction in the STATUS (Status) column. The status of the distributed transaction can include when or whether the transaction has been committed or aborted, or is in progress.

[0117] For example, line 311 records information about the distributed transaction. DISTR TXID contains 4001 as the distributed txid of the distributed transaction, LOG ID stores log record 313 as log ID 1 storing the TC block used for the distributed transaction, STATUS stores the transaction status of the specified distributed transaction that has been committed, and USER stores the user identifier ENTA that identifies the user who initiated the distributed transaction.

[0118] Blockchain vote 320 includes information about the commit consensus result for the log record of the TA block in BC log 103-1. Each row in Blockchain vote 320 includes voting information for the corresponding log record in BC log 103-1. The LOG ID column includes the log ID of the corresponding log record. The DISTR TXID includes the distributed txid of the distributed transaction that added the block. The COMMIT VOTE column describes the vote of each DBMS participant on the commit consensus result. Specific DBMS participants can be stored in Blockchain vote 320, or they can be determined by checking the corresponding log record in BC log 103-1.

[0119] For example, line 321 records the commit consensus result of the TA block stored in log record 312. In blockchain vote 320, LOG ID stores the log ID 2 that identifies log record 312, DISTR TXID includes the distributed txid 4001 of the corresponding distributed transaction of log record 312, and COMMIT VOTE specifies the voting result for distributed transaction 4001.

[0120] Blockchain transactions 310 and blockchain votes 320 can be used to retrieve the current state of operations affecting the ledger and the corresponding blockchain, as well as information about the activities of DBMS participants in the BCA network 101. While such information can be derived from the BC log 103-1, storing it in blockchain transactions 310 and blockchain votes 320 facilitates more efficient information retrieval. For example, to determine the state of a distributed transaction, the distributed txid can be used to retrieve the transaction's state from the STATUS column.

[0121] Database dictionary for blockchain application networks

[0122] The database dictionary includes metadata that defines the database objects and other configuration aspects of the DBMS. The database dictionary may include multiple data structures that store database metadata. For example, the database dictionary may include multiple files and tables. Parts of the data structures can be cached in the database server's main memory.

[0123] A subset of metadata that defines a specific database object or a specific aspect of the DBMS configuration is referred to herein as a metadata definition or simply a definition. A definition may also include multiple data structures and tables. The definition of a specific database object may include the definitions that make up the data object. For example, the definition of a table may include multiple definitions of the columns of that table.

[0124] The database dictionary used by DBMS participants includes a blockchain schema (“BC schema”). A BC schema is a named set of definitions for database objects and one or more configurations of the BCA network. The BCA defines the ledger, the blockchain process for the ledger (if any), supplementary database objects, and DBMS participants in the BCA network. A BC schema can be created by the DBMS in response to DDL database statements that specify the definitions for various aspects of the BC schema.

[0125] refer to Figure 3 It describes DB dictionary 330 (the database dictionary of the initiator DBMS 110). DB dictionary 330 includes BC schema 332, which includes ledger table definition 333, which defines ledger table 102-1 (and its columns) and database objects specifically for supporting blockchain applications of ledger table 102-1 within BCA network 101, as well as various aspects of the configuration of BCA network 101 specific to ledger table 102-1.

[0126] Blockchain process 105-1 defines the blockchain process for ledger table 102-1. The definition may include the source code of the process and / or its compiled version. According to an embodiment, the blockchain process is associated with only one BCA network and can therefore be used to update one or more ledger tables of the BCA network. Participant definition 334 defines the DBMS participants of BCA network 101. This information may include the network address and public encryption key of the DBMS participant.

[0127] Blockchain configuration 335 includes configuration details for the distributed blockchain maintained for ledger table 102-1. Such information may include, for example, a “block hash algorithm” (as described later) used to generate block hash values ​​for blocks in the blockchain, and a cryptographic algorithm used to digitally sign the content included in the blocks.

[0128] at last, Figure 3 This represents the database objects and data structures on follower DBMSs 120 and 130 used to support blockchain applications. Therefore, follower DBMSs 120 and 130 each have peer-specific database objects—a blockchain voting table, a blockchain transaction table, a DB dictionary, and a BC schema—defined structurally similar or identical to those of blockchain voting 320, blockchain transactions 310, DB dictionary 330, and BC schema 332, respectively. Furthermore, the structures of BC logs 103-2 and 103-3 are identical or similar to those of BC log 103-1.

[0129] Private subpattern and callback process

[0130] The BC schema can include a private BCA subschema that is not replicated and distributed across the BCA network. A private BCA subschema includes database objects that the administrators of the DBMS participants may not expect or need to be shared across the BCA network, and which the administrators can use to manage, monitor, or supplement the security of the BCA network. For example, database objects such as views or supplementary tables, which are not intended to be shared with other BCA network participants, can be created. (See reference) Figure 3 BC mode 332 includes a private sub-mode 340. Private sub-mode 340 includes a callback procedure 342 and a view 344.

[0131] Callback procedure 342 may include a BCA DML callback procedure and / or a BCA DDL callback procedure. These are procedures that can be privately implemented by the administrator of the DBMS participant to implement additional logic to determine whether to vote to commit the DML or DDL changes specified in the block. The BCA DML callback procedure is used to determine whether to vote to commit the DML changes specified in the TC block. The BCA DDL callback procedure is used to determine whether to vote to commit the DDL changes in the TD block. The TD block specifies the DDL changes and will be described in more detail later.

[0132] The blockchain application invokes the BCA DML callback procedure to determine whether the specific "DML commit criteria" for the DML change are met. The returned result indicates whether the specific DML commit criteria are met. The vote to accept or reject the DML change specified in the TC block is based on the result.

[0133] For example, a ledger in a BCA network stores bank account transfers between banks. Each bank operates a DBMS participant within the BCA network. A particular bank of a DBMS participant may require approval from a bank official for any bank transaction exceeding $1 million. In this case, the DML submission guidelines include bank official approval for bank transactions exceeding $1 million. The bank implements a BCA DML callback procedure, which processes the bank official's approval and returns a result indicating whether the transfer was approved. For example, the BCA DML callback procedure is invoked at operation 245.

[0134] Multi-version database dictionary

[0135] According to the embodiment, DB dictionary 330 is a multi-version dictionary. The multi-version control capabilities of the multi-version dictionary are fully utilized to influence DDL changes in the distributed BC mode within the BCA network.

[0136] In a multi-version dictionary, one or more sets of defined versions are tracked. Each set is called an epoch group, and each version within that epoch group is called an epoch. Each epoch is identified by an epoch identifier (“epoch id”). The epoch ids of an epoch group are ordered such that an epoch id is greater than any epoch id of any earlier epoch in the epoch group.

[0137] In this embodiment, the BC pattern is treated as a period group. Therefore, multiple periods of the BC pattern are tracked and associated with a period ID.

[0138] Periods can be generated in response to the execution of DDL statements issued to the initiator DBMS110. For example, in response to a DDL command received by the initiator DBMS110 to add a column to ledger table 102-1, a new period and period ID for BC schema 332 are created. This period includes the definition for the new column within ledger table definition 333. As another example, in response to a DDL statement received by the initiator DBMS110 to add a new DBMS participant to BCA network 101, a new period and period ID for BC schema 332 are created. This period includes a new version of participant definition 334, which includes metadata about the new DBMS participant.

[0139] According to the implementation, only one period in a period group is active. DBMS participants can only compile database statements using the active BC schema period. For example, when compiling a database statement that references a column name in a ledger table, and the column name is not defined for the active period of the ledger table but for another period of the table, the column name is not resolved, and a compiler error is generated.

[0140] When a period is created within a period group, it is not immediately active; it can be activated later. A period can also expire later. A period can be pending, active, or invalid. A period cannot be active when it is invalid or pending. When a period is created, it can initially be pending.

[0141] Activity period 336 maintains the activity period used for BC mode. Period 337 is a list of periods generated for BC mode 332. This list includes EPOCH ID and EPOCH STATUS columns. Each entry corresponds to a period and includes the period ID and period status in the EPOCHID and EPOCH STATUS columns, respectively.

[0142] Explanatory blocks in a blockchain

[0143] Figure 4 An illustrative subchain 401 is depicted according to an embodiment of the present invention. As used herein, the term blockchain may also be used to refer to a subchain of a “complete” blockchain. A complete blockchain begins with the origin block.

[0144] Subchain 401 includes four block types. These include TC blocks and TA blocks. In addition to these block types, subchain 401 also includes DDL blocks and DDL management blocks. DDL blocks specify DDL changes to BC mode for replication across the BCA network. DDL management blocks describe whether DDL changes are actually replicated and / or accepted.

[0145] refer to Figure 4 It depicts blocks 410, 420, 430, and 440, which belong to the undepicted complete blockchain. Each block has a shared set of attributes. Furthermore, each block has a record set. However, different record set types exist for each block type.

[0146] Each block in subchain 401 is included in the BC log record and has a "blockchain position" in the blockchain's "blockchain order". The log ID of the log record reflects the block's blockchain position.

[0147] A block may include digitally signed block content (“signed content”) digitally signed by one or more DBMS participants. As described later, the majority of the block content is “initiator-signed content” digitally signed by the initiating DBMS. Some of the block content may be provided by follower DBMSs, such as votes from follower DBMSs. Generally, specific content in a block provided by a particular follower DBMS should be “follower-signed content” digitally signed by the follower DBMS. The block also includes one or more digital signatures of the initiator-signed content and the follower-signed content.

[0148] A block contains a block hash and, optionally, previous block hashes, which are particularly relevant to the blockchain order and the validity of the block. The block hash is generated by applying the blockchain's hash algorithm to the block's "hash content." Previous hashes in a block are the block hashes of previous blocks in the blockchain order. The hash content of a block hash includes the digitally signed content, previous block hashes, and other content from that block.

[0149] Refer again Figure 4 Each of blocks 410, 420, 430, and 440 contains the attributes of a block hash value and a previous hash value. Specifically, blocks 410, 420, 430, and 440 include previous hash 417 and block hash 419, previous hash 427 and block hash 429, previous hash 437 and block hash 439, and previous hash 447 and block hash 449, respectively.

[0150] In block 420, the hash value of the previous hash 427 is the hash value of block hash 419. The hash value of the previous hash 437 is the hash value of block hash 429, and so on.

[0151] Each block in subchain 401 includes an initiator DBMS attribute, which comprises at least two fields. One field identifies the initiator DBMS that created and proposed the block, and the other is the term ID of the leader's term during the proposed block. Blocks 410, 420, 430, and 440 include initiator DBMS attributes for initiator DBMSs 415, 425, 435, and 445, respectively.

[0152] Each block in subchain 401 includes a log ID attribute, which stores the log ID of the block's log records. Blocks 410, 420, 430, and 440 include attribute log IDs 411, 421, 431, and 441, respectively.

[0153] Each block in subchain 401 includes an initiator digital signature attribute, which holds a digital signature of the content signed by the block's initiator. For example, in block 410, the initiator's signed content includes log ID 411, block recordset 412, initiator DBMS 415, and previous hash 417. Initiator digital signature 418 holds a digital signature of the content signed by the initiator of block 410. For initiator-signed content similar to that of block 410, blocks 420, 430, and 440 include initiator digital signatures 428, 438, and 448, respectively. Figure 4 As shown in the image.

[0154] In addition to the originating block in the blockchain, a valid block requires that its hash content includes a previous hash value and that the block's block hash value is equal to the hash value generated by the block hashing algorithm based on the block's hash content. When a block is proposed to be added to the blockchain, the proposal acceptance criteria used by BCA participants to determine whether to accept or reject the proposed block include that the proposed block's hash value is equal to the hash value generated by applying the block hashing algorithm to the proposed block's hash content, where the hash content includes a previous hash value.

[0155] Record set

[0156] The structure and content of a block's record set depend on the type of block. Block record set 412 in block 410 represents the block record set of the TC block.

[0157] Block record set 412 includes a block type attribute, block type 412-1, which specifies the block type of the block containing the record set. Block record set 412 also includes a transaction context 412-2, which includes database operations 412-21 and a distributed TXID 412-23. Database operations 412-21 specify the database operations performed by the appropriate initiating DBMS for the distributed transaction. Write set 412-22 is the write set generated by performing database operations. Distributed TXID 412-23 contains the distributed txid of the distributed transaction that proposed block 410 to the BCA network.

[0158] Block record set 422 in block 420 represents the block record set of the TA block. Block record set 422 includes the block type attribute (block type 422-1), the distributed TXID (422-2), the transaction result (422-3), and the vote (422-4). The distributed TXID (422-2) is the distributed txid of the distributed transaction, and block 420 represents its result. The transaction result (422-3) specifies the outcome, i.e., whether the distributed transaction was committed or aborted.

[0159] Voting 422-4 lists the votes cast by follower DBMSs on committing or aborting transactions. Each vote of a follower DBMS is a digitally signed copy of the vote. Voting 422-4 includes the corresponding digital signature for each vote.

[0160] Block record set 432 in block 430 represents the block record set for the DDL block. Block record set 432 includes the block type attribute 432-1 and the DDL operation 432-2, which specifies one or more DDL changes to be performed on the corresponding BC mode. DDL changes can be implemented through one or more DDL statements.

[0161] Block record set 442 in block 440 represents the block record set used for DDL management blocks. Block record set 442 includes the block type attribute block type 442-1 and DDL operation 442-2, which represents the proposed DDL change, and block 440 represents its result. DDL transaction result 442-3 specifies this result. Votes 442-4 lists the votes of DBMS participants that led to the result.

[0162] Voting 442-4 lists the votes cast by follower DBMSs to commit or abort transactions. Each vote by a follower DBMS is a digitally signed copy of the vote. Voting 442-4 includes the corresponding digital signature of the vote.

[0163] In this embodiment, some of the information described in a TA block or TDA block can be carried over to other types of blocks. For example, after proposing a TC block for a specific distributed transaction, the initiating DBMS receives some votes for the distributed transaction from the follower DBMSs. Before receiving all the votes for the specific distributed transaction, the initiating DBMS proposes a subsequent TC block for another distributed transaction. To record the votes received for a specific distributed transaction, the initiating DBMS may include the votes received in the subsequent TC block.

[0164] In an embodiment, a block may include multiple record sets. For example, a TC block may include two record sets for two distributed transactions.

[0165] In an embodiment, the record set can be of different record set types, thus making the block a hybrid block. For example, a block can include a record set for proposing distributed transactions, such as record set 412 of TC block 410, and can include a record set for recording the voting results of another distributed transaction, such as record set 422 of TA block 420.

[0166] Handling consensus issues related to adding blocks

[0167] In this embodiment, DBMS participants insert and commit log records while they are in a pending state. Committing log records is important even before a block consensus result is reached, as it allows for handling various recovery scenarios, as explained in more detail later. However, proposed log records may be rejected at different stages of the blockchain application. Handling the log records of proposals that have been submitted and then rejected introduces some complexity.

[0168] First, log-based consensus algorithms may require that only a single log ID be assigned to an accepted proposal, and that log IDs be assigned in the order in which the corresponding log records were accepted. Therefore, measures are needed to ensure that a log ID is not assigned to a committed log record that was ultimately rejected. Second, since the initiation and / or completion of local transactions that modify the ledger are associated with adding a block to the blockchain when a log record is rejected, any incomplete local transactions need to be addressed.

[0169] Figure 5 The diagram illustrates the process of resolving the denial of logging for TC blocks in various scenarios. (Reference) Figure 5 Operations 505-520 are executed by the initiator DBMS 110 after proposing logging for the TC block for the distributed transaction. The logging has been committed by the initiator DBMS 110 to BC log 103-1.

[0170] The initiator DBMS110 determines that the block consensus result for the log record is rejected (505). The initiator DBMS110 marks BC log 103-1 as rejected by updating the OUTCOME column of BC log 103-1 to indicate that the log record has been rejected (510).

[0171] Log IDs marked as rejected in this way are not considered to be assigned to valid log records and blocks, and are effectively ignored. The log ID is used for another accepted log record.

[0172] Initiator DBMS110 rolls back the local transaction (515) for the distributed transaction of the proposed TC block. Initiator DBMS110 updates supplementary tables. These updates may include updating rows for the distributed transaction in blockchain transaction 310 (i.e., rows including the distributed txid) to indicate that the transaction has been aborted.

[0173] Operations 530-535 are performed by follower DBMS120 and 130 after they have submitted log entries for which their block consensus result was rejected. The operations are described in relation to follower DBMS120. Follower DBMS120 determines that the proposed log entry has been rejected (530).

[0174] The follower DBMS120 marks its corresponding BC log 103-2 as rejected by updating the OUTCOME column, indicating that the log record has been rejected. The row in the blockchain transaction table on the follower DBMS120 for the distributed transaction is updated to indicate that the distributed transaction has been aborted. Because the proposed log record was rejected, the follower DBMS120 never initiated a local transaction for the distributed transaction. Therefore, no branch transaction was rolled back.

[0175] The operation performed when logging is refused for TA blocks is similar to the operation performed for TC blocks.

[0176] Proposal to restore

[0177] The initiating DBMS can lose leadership before coordinating the completion of distributed transactions for the ledger. Leadership can be lost at various points in the blockchain application: for example, before reaching consensus on the TC block, before reaching consensus on the commit of the distributed transaction, or before reaching consensus on the TA block. Leadership may be lost because the leadership term has expired or because the initiating DBMS fails.

[0178] When a leader is lost, pending proposals initiated by the initiating DBMS can be considered as having a rejected consensus outcome and handled accordingly by other DBMS participants. This handling may include marking the committed log entries for the proposal as rejected. The initiating DBMS can simply abort the distributed transaction and notify the requesting client. The client can then initiate a transaction with the new leader.

[0179] However, in some cases, it may be desirable to complete a distributed transaction. For example, a distributed transaction can be a long-running transaction, and aborting it may be relatively costly for the requesting client. According to an embodiment, DBMS participants can resubmit the proposal to the current initiating DBMS to complete the processing of the distributed transaction through a process referred to herein as proposal recovery.

[0180] Figure 6 The process for proposing a restoration is described. Figure 6 Some operations proposed for recovery are not illustrated. Many operations proposed for recovery are related to... Figure 2 The operations described in the previous section are similar, but may not be described in as much detail as before, or may not be described at all.

[0181] This process can be executed in response to various events that lead to the loss of leadership. However, for illustrative purposes, the proposal to revert is initiated by the former initiating DBMS (“Recommit DBMS”) after losing leadership due to the election of a new leader and after processing the log entries for TC blocks following the commit of the log entries.

[0182] refer to Figure 6 The recommitting DBMS submits a "recommit request" (610) to the current initiating DBMS. The recommit request includes the "recommitted transaction context," which is the transaction context previously generated by the recommitting DBMS for the distributed transaction. The recommitting DBMS can retrieve the transaction context from its BC log.

[0183] The initiating DBMS initiates a local transaction and replays the transaction context to generate a local write set (615) and compares the local write set with the proposed write set in the recommitted transaction context and determines that the write set matches (620).

[0184] In response to the determination of a write set match, the initiating DBMS determines the voting proposal log record (625) and prepares the proposed log record, including the TC block (630) for the proposed log record. The log record is committed and proposed to the DBMS participants, including resubmitting to the DBMS.

[0185] and Figure 2 Similarly, as described in the text, DBMS participants execute a set of operations, in which DBMS participants vote to accept log entries, prepare to commit transactions, vote to commit transactions, accept TA blocks, and commit distributed transactions (including corresponding local branch transactions).

[0186] However, the resubmitted DBMS performs a different set of operations. Similar to other DBMS participants, the resubmitted DBMS receives the proposed log record from the initiating DBMS and votes to add the log record to the BC log (650). The resubmitted DBMS then determines that the proposed log record is accepted (655).

[0187] However, a recommitted DBMS does not need to replay the transaction context of the TC block in the log records because the recommitted DBMS has already generated a local write set for the distributed transaction. Furthermore, the recommitted DBMS is already prepared to commit the local transaction for the distributed transaction and does not need to prepare to commit again.

[0188] Based on a comparison of the proposed write sets in the TC blocks of local writes and log records (660), the resubmitted DBMS determines to vote to commit the distributed transaction (665). Finally, the distributed transaction is committed, and the initiating DBMS commits its local branch transaction (640), the resubmitted DBMS commits its local branch transaction, and the other DBMS participants commit theirs (670).

[0189] Change BC mode

[0190] Similar to distributed ledgers, the BC schema is distributed among DBMS participants in the BCA network. Changes to the distributed BC schema are propagated throughout the BCA network using blocks applied to the blockchain.

[0191] If the DBMS participants support distributed transaction processing for DDL statements, then it is possible to use... Figure 2 The described DML blockchain application process is similar to the blockchain application process that replicates DDL changes to the BC model.

[0192] However, some DBMSs do not support distributed transaction processing for DDL changes, or may only support statement-level local transactions. For such DBMSs, there is no protocol (such as two-phase commit) that allows the DBMS to arbitrate whether DDL changes should be made permanent. In these cases, a process such as “DDL blockchain application” as described below can be used.

[0193] Figure 7 The process of "DDL blockchain application" is described, which uses a multi-version data dictionary supporting the periods described above. According to an embodiment, the process is initiated in response to receiving a DDL statement from the initiating DBMS specifying a DDL change to BC schema 332. For example, the DDL change could be a DDL change to ledger table definition 333 to add column restrictions or a change to participant definition 334 to modify participant connection information.

[0194] refer to Figure 7 In response to receiving a DDL statement, according to the log-based consensus algorithm, the initiator DBMS 110 adds a log record for the DDL block to BC log 103-1 (705), and the follower DBMSs 120 and 130 add log records for the DDL block to BC logs 103-2 and 103-3 (750), respectively. The DDL block specifies the DDL statement. After adding the log record, the initiator DBMS 110 makes a DDL change to ledger table definition 333, thereby creating a new pending period (710).

[0195] Follower DBMS120 and 130 read the DDL statements from their respective DDL blocks from the newly added log record and execute the DDL statements to create a new pending period (755). According to the log-based consensus protocol, follower DBMS120 and 130 vote to accept or reject the DDL change (760).

[0196] The initiator DBMS110 determines the "DDL consensus result" (715) for the DDL change. If the "DDL consensus result" is acceptance, then log entries including the TDA blocks specifying that the DDL change is accepted are added to BC log 103-1 (720). DB dictionary 330 is updated to mark the new era as valid (725). If the DDL consensus result is rejection, then log entries including the TDA blocks specifying that the DDL change is rejected are added to BC log 103-1 (730). BC mode 332 is updated to mark the new era as invalid (735).

[0197] Follower DBMSs 120 and 130 add TDA log records for the DDL blocks to BC logs 103-2 and 103-3, respectively (765). Follower DBMSs 120 and 130 read the TDA blocks to determine whether to accept or reject the DDL changes (770). If accepted, the new period is marked as valid (775). If rejected, the new period is marked as invalid.

[0198] Staged data dictionary

[0199] In the staging data dictionary, defined versions are not tracked individually. However, DDL changes to the data dictionary can be staging. When DDL changes to a definition are made via a DDL transaction, these changes are marked as pending in the data dictionary. The changes can later be made effective by marking the DDL changes as valid or otherwise removing them from the pending list. Alternatively, changes can be marked as valid or removed by compensating for the DDL transaction.

[0200] Copying DDL changes to DBMS participants that use a temporary dictionary allows them to interact with... Figure 7 This is a similar DDL blockchain application described in [the text]. However, DDL changes are initially marked as pending, instead of creating a new pending period (which is later verified or invalidated). If accepted, the DDL change can be marked as valid. If not accepted, the DDL change will be marked as invalid or removed by compensating the DDL transaction.

[0201] Bootstrapping BCA network

[0202] Bootstrapping the BC network (“bootstrapping”) refers to configuring BCA participants to initiate blockchain applications for a specific BC schema initially created at one of the BCA participants. Bootstrapping requires proposing a BC schema to BCA participants for acceptance, and if accepted, then creating the BC schema at each BCA participant.

[0203] The bootstrapping process begins with the initiating DBMS, where the BC schema is created by, for example, an administrator of the initiating DBMS. Once created, DDL statements are issued to the initiator to bootstrap the BCA network.

[0204] Figure 8 This is a flowchart illustrating the process of using a guide according to an embodiment of the present invention. (Reference) Figure 8 The BC schema is created by the initiating DBMS in response to DDL statements published to the initiating DBMS (805). The BC schema specifies the BCA participant.

[0205] Next, the initiating DBMS receives the bootstrap command to bootstrap the BC schema and create a pending period for the BC schema (810). The initiating DBMS generates log records, one for the origin block and one for the TD block. The TD block specifies the DDL changes to create the BC schema. Then, the initiating DBMS proposes log records for the origin block and TD block to the other DBMS participants defined by the BC schema (815).

[0206] Other DBMS participants receive the proposed log entries for the origin block and TD block. In response, other DBMS participants create a BCA schema and create a pending period (845). Other DBMS participants vote to accept or reject the BC schema (850).

[0207] Based on the votes of the DBMS participants, the initiating DBMS determines whether the proposed origin block and TD block are accepted or rejected (820). If rejected, the pending period of the BC mode is marked as invalid (825).

[0208] If accepted, the initiating DBMS will mark the ledger records for the origin block and TD block as accepted (830). The initiating DBMS will then mark the pending period as valid.

[0209] Regarding other DBMS participants, they determine whether the proposed log record is accepted (855). If accepted, the other BCA participants mark the ledger records used for the origin block and TD block as accepted (860), and then mark the period as valid (865). Otherwise, the other BCA participants mark the pending period as invalid (870).

[0210] Blockchain applications in file systems

[0211] As mentioned earlier, as long as PSS supports distributed transaction processing (especially the ability to make transactions ready), other forms of PSS (instead of DBMS) can be combined with blockchain applications to manage the ledger store of objects, to confirm to the transaction coordinator that the transaction has reached the ready state, and to commit or abort the ready transaction when requested by the coordinator.

[0212] Figure 9 The diagram illustrates a replicated file system network implementing blockchain applications within a BCA network, where BCA participants are file systems (“FS”). This replicated file system network is referred to herein as an FS-based BCA network. An FS may include one or more storage servers with one or more processors, memory, and persistent storage devices on which files are stored. The FS provides access to files via a network or through an interface using a file system protocol.

[0213] The FS-based BCA network supports distributed transaction processing and journaling to replicate files across each FS-based BCA participant. Each FS-based BCA participant is an independent file system, for example, connected via a wide area network. The distributed ledger data is a directory (“ledger directory”), which is replicated as a directory copy on each FS-based BCA participant.

[0214] refer to Figure 9 It describes a BCA network 901, which is a FS-based BCA network according to an embodiment of the present invention. (Reference) Figure 9 The FS-based BCA network 901 includes initiator FS 910 and followers FS 920 and FS 930. Each of these file systems participates in maintaining a copy of the ledger directory using blockchain applications. Changes include adding, deleting, and moving files and subdirectories, modifying data within files, and altering the properties of files and subdirectories. Changes to the ledger directory are initiated by initiator FS 910.

[0215] Similar to the example mentioned earlier for BCA network 101, in FS-based BCA network 901, BCA participants rotate the role of the initiator FS. Therefore, Figure 9 This indicates the role of a BCA participant at a specific point in time. At another point in time, follower FS920 or follower FS930 could be the initiator FS of BCA network 901. The initiator FS retains leadership during their leadership term.

[0216] refer to Figure 9The FS-based BCA network 901 includes a ledger directory, which includes copies of ledger directory 902-1, ledger directory 902-2, and ledger directory 902-3 stored on the initiator FS 910, follower FS 920, and follower FS 930, respectively.

[0217] Distributed Blockchain 904 includes copies of the distributed blockchain, namely Blockchain 904-1, Blockchain 904-2, and Blockchain 904-3. These blockchain copies are stored on the initiator FS 910, the follower FS 920, and the follower FS 930, respectively.

[0218] Blockchains 904-1, 904-2, and 904-3 are stored in a distributed blockchain log, which includes BC logs 903-1, 903-2, and 903-3, and are stored on the initiator FS 910, the follower FS 920, and the follower FS930, respectively.

[0219] Since BC logs 903-1, BC logs 903-2, and BC logs 903-3 comprise distributed BC logs, each contains a copy of the log records in the distributed BC logs. Each log record includes a log identifier (log ID), which uniquely identifies the log record in BC log 903. Each copy of the log record includes the same log ID. The log IDs are generated in a similar manner to those described for BCA network 101.

[0220] To explain in more detail, initiator FS 910 initiates a distributed transaction to replicate changes to a directory across the BCA network 901. Changes are replicated by copying log records at follower FS 920 and 930. Initiator FS 910 coordinates with follower FS 920 and 930 to add log records to the distributed BC log 903. Adding log records effectively adds blocks to the distributed blockchain. Using the distributed transaction processing coordinated by initiator FS 910, local transactions 925 and 935, executed in follower FS 920 and 930 respectively, apply the changes specified in the log records to the replicated directory replica ledger directories 902-2 and 902-3.

[0221] BCA network 901 also includes a distributed collection of blockchain processes 905. A blockchain process is a computer-executable process or function whose implementation is accepted and visible to members of the FS-based BCA network 901, and executed by members of the FS-based BCA network 901 to modify a replicated directory. According to embodiments, blockchain processes can be implemented using computer languages ​​such as Java, JavaScript, Python, or Solidity. Blockchain process 905 includes replicating blockchain processes 905-1, 905-2, and 905-3, which are stored on initiator FS 910, follower FS 920, and follower FS 930, respectively.

[0222] The FS-based BCA network 901 client can request changes to the replicated directory by initiating a transaction 915 with the initiator FS 910, where the client can issue file system commands to change the ledger directory 902-1. File commands can invoke stored blockchain processes.

[0223] Transaction and file commands

[0224] BCA Network 901 clients can request changes to the replicated directory by initiating transaction 915 with initiator FS 910. The transaction includes file system commands issued by the client to change the ledger directory 902-1.

[0225] File system commands include commands to open files, close files, rename, move, or delete files, write or delete blocks within files, and write or modify lines within files. File commands can also specify the creation, moving, renaming, or deletion of directories, and can invoke stored blockchain processes.

[0226] Logs, block structure, record sets, and supplementary structures

[0227] Log records (including blockchain blocks) are stored in one or more data blocks within a file. These data blocks are referred to below as "file data units" to distinguish them from blocks in the blockchain. Furthermore, the memory address boundaries of blocks within a log record are not necessarily aligned with file data units. For example, a log record can be stored in a single file data unit. A single file data unit stores the block along with other data from the log record.

[0228] According to an embodiment, in a FS-based BCA network, blocks include TC blocks and TA blocks, similar to TC block 410 and TA block 420. The TC block or TA block does not identify the initiator DBMS and its term of office, but rather identifies the initiator FS and its term of office.

[0229] The TC block of the FS-based BCA network also includes a transaction context. The transaction context comprises specifications that identify file system commands and / or blockchain procedures and corresponding independent variable values, rather than specifications that identify database operations. The proposed write set of the transaction context includes the results of executing file system commands and / or blockchain procedures. Such results include, for example, (1) overwriting one or more file data units, a new version of the overwritten one or more file data units, and (optionally) a previous version of one or more file data units; (2) renaming a file, a new name for the file, and (optionally) the previous name for the file; and (3) moving a file, a new directory for the file, and (optionally) the old directory for the file.

[0230] Similar to what has been described previously, followers FS120 and FS130 reload within the transaction context of the branch transaction initiated by followers FS120 and 130 for the distributed transaction (i.e., executing the commands and / or procedures specified in the specification within the transaction context) to generate a local write set for comparison with the proposed write set. This comparison at least partially determines whether followers FS120 and 130 will vote to commit the distributed transaction.

[0231] Using the TC and TA blocks described above for FS-based BCA networks, and... Figure 2 The similar processing described in [the text] is for blockchain applications on the FS-based BCA network 901. It handles non-consensus mechanisms related to adding blocks. Figure 5 The same execution as described in [the text] is proposed, and restoration is suggested. Figure 6 It is executed similarly to what is described in the text.

[0232] The FS-based BCA network 901 can store information in a similar structure to the information stored in table blockchain transactions 310 and blockchain votes 320 in an "embedded database" of FS-based BCA participants. Any embedded database can be a relational database or a NoSQL database, such as a key-value store or document store.

[0233] FS-based BCA pattern

[0234] Similar to BC pattern 332 of BCA network 101, FS-based BCA network 901 also includes a pattern that defines the blockchain. This pattern is referred to in this paper as the FS-based BCA pattern and is distributed among FS-based BCA participants.

[0235] In the FS-based BCA network 901, each FS-based BCA participant supports a command set similar to the DDL commands described earlier for creating and / or modifying FS-based BCA schemas. FS-based BCA schemas can be stored in one or more files or in an embedded database. Figure 7 Similar to those described above, using blocks similar to the DDL blocks and DDL management blocks described earlier, FS-based BCA schema changes are propagated and coordinated among FS-based BCA participants, the difference being that these blocks identify the initiator FS rather than the initiator DBMS.

[0236] Blockchain applications in document storage systems

[0237] Figure 10 The illustration depicts a document storage system network that enables the replication of blockchain applications within the BCA network, where BCA participants are document storage systems (“DOCS”) that store copies of a collection of documents (“Collection”). This replicated document storage system is referred to herein as a DOCS-based BCA network. Furthermore, in this embodiment, DOCS is an extended version of a key-value store system. In a key-value store system, the stored values ​​do not need to conform to a markup language.

[0238] A Document Storage Service (DOCS) may include one or more document storage servers, having one or more processors, memory, and persistent storage devices for storing documents. DOCS provides access to documents through a document storage protocol, which in particular supports CRUD ("Create, Read, Update, Delete") document modification features. Documents are objects stored according to markup languages ​​such as XML or JSON.

[0239] In DOCS, documents are contained within document collections, which can be named in a manner similar to table names. Collections are analogous to tables in a relational database. In a table, each record is a row; in a collection, each record can be an XML or JSON document, or an XML element or JSON field within a document.

[0240] The DOCS-based BCA network supports distributed transaction processing and journaling for modifying copies of documents within a collection. The distributed ledger data in the DOCS-based BCA network is a collection replicated on each DOCS-based BCA participant using blockchain applications.

[0241] refer to Figure 10 It depicts a BCA network 1001, which is a DOCS-based BCA network according to an embodiment of the present invention. (Reference) Figure 10The BCA network 1001, based on DOCS, includes initiator DOCS1010 and followers DOCS1020 and DOCS1030. Each of these DOCSs participates in securely maintaining CRUD changes to the ledger set using blockchain applications. Changes to the ledger set are initiated by initiator DOCS1010.

[0242] Similar to the example mentioned earlier for BCA network 101, in the DOCS-based BCA network 1001, the BCA participants rotate the role of the initiator, DOCS. Therefore, Figure 10 This indicates the role of a BCA participant based on DOCS at a specific point in time. At another point in time, follower DOCS1020 or follower DOCS1030 could be the initiator DOCS of BCA network 1001. The initiator DOCS retains leadership during its leadership term.

[0243] refer to Figure 10 The DOCS-based BCA network 1001 includes a ledger set, which includes copies of ledger set 1002-1, ledger set 1002-2, and ledger set 1002-3 stored on initiator DOCS1010, follower DOCS1020, and follower DOCS1030, respectively.

[0244] Distributed Blockchain 1004 includes copies of the distributed blockchain, namely Blockchain 1004-1, Blockchain 1004-2, and Blockchain 1004-3. These blockchain copies are stored on the initiator DOCS1010, the follower DOCS1020, and the follower DOCS1030, respectively.

[0245] Blockchains 1004-1, 1004-2, and 1004-3 are stored in a distributed blockchain log, which includes BC log 1003-1, BC log 1003-2, and BC log 1003-3, and are stored on the initiator DOCS1010, follower DOCS1020, and follower DOCS 1030, respectively.

[0246] Since BC logs 1003-1, BC logs 1003-2, and BC logs 1003-3 comprise distributed BC logs, each BC log contains a copy of the log records in the distributed BC logs. Each log record includes a log identifier (log ID), which uniquely identifies the log record in BC log 1003. Each copy of the log record includes the same log ID. The log IDs are generated in a manner similar to that described for BCA network 101.

[0247] To explain in more detail, the initiator DOCS1010 initiates a distributed transaction to replicate changes across the ledger sets of the BCA network 1001. Changes are replicated by copying log records at follower DOCs 1020 and 1030. The initiator DOCS1010 coordinates with followers DOCS1020 and 1030 to add log records to the distributed BC log 1003. Adding log records effectively adds blocks to the distributed blockchain. Using the distributed transaction processing coordinated by the initiator DOCS1010, local transactions 1025 and 1035, executed within followers DOCS1020 and DOCS1030, respectively apply the changes specified in the log records to ledger set replicas 1002-2 and 1002-3.

[0248] BCA network 1001 also includes a distributed collection of blockchain processes 1005. A blockchain process is a computer-executable process or function whose implementation is accepted and visible to members of the DOCS-based BCA network 1001, and executed by members of the DOCS-based BCA network 1001 to modify a replicated directory. According to embodiments, blockchain processes can be implemented using computer languages ​​such as Java, JavaScript, Python, or Solidity. Blockchain process 1005 includes replicas DOCS 1005-1, DOCS 1005-2, and DOCS 1005-3, which are stored on initiator DOCS 1010, follower DOCS 1020, and follower DOCS 1030, respectively.

[0249] Transactions and CRUD commands

[0250] A BCA network 1001 client can request changes to a ledger collection by initiating a transaction 1015 with initiator DOCS 1010 (where the client can issue DOCS commands to modify ledger catalog 1002-1). DOCS commands can include CRUD commands and can invoke stored blockchain procedures. DOCS commands also include commands to change the name or key of a document and to open or close a collection or document.

[0251] Logs, block structure, record sets, and supplementary structures

[0252] Each log record is stored as a document in a collection. This collection serves the BC log, and each log record in the BC log is a document. Blocks include TC blocks and TA blocks, similar to TC block 410 and TA block 420. TC blocks or TA blocks do not identify the initiating DBMS and its tenure; instead, they identify the initiating DOCS and its tenure.

[0253] The TC block for the DOCS-based BCA network also includes a transaction context. The transaction context comprises specifications that identify DOCS commands and / or blockchain processes and their corresponding independent variable values, rather than specifications that identify database operations. The proposed write set of the transaction context includes the results of executing DOCS commands and / or blockchain processes. Such results include documents in the case of document creation, as well as new values ​​generated by document updates.

[0254] Similar to what was described previously, followers DOCS1020 and DOCS1030 reload into the transaction context within the branch transaction initiated by followers DOCS1020 and 1030 for the distributed transaction to generate a local write set for comparison with the proposed write set. This comparison at least partially determines whether followers DOCS1020 and 1030 will vote to commit the distributed transaction.

[0255] Using the TC and TA blocks described above for DOCS-based BCA networks, and... Figure 2 The similar processing described in [the original text] applies to the DOCS-based BCA network 1001 blockchain. This includes handling non-consensus mechanisms related to adding blocks. Figure 5 The same execution as described in [the text] is proposed, and restoration is suggested. Figure 6 The same applies as described in [the document]. The DOCS-based BCA network 1001 can store information similar to that stored in table blockchain transactions 310 and blockchain votes 320 in one or more sets.

[0256] BCA model based on DOCS

[0257] Similar to BC Mode 332 of BCA Network 101, DOCS-based BCA Network 1001 also includes a mode that defines the blockchain. This mode is referred to herein as the DOCS-based BCA Mode and is distributed among DOCS-based BCA participants.

[0258] In the DOCS-based BCA network 1001, each DOCS-based BCA participant supports a set of DDL commands for creating and / or modifying DOCS-based BCA schemas, similar to the DDL commands described earlier for creating BC schemas. DOCS-based BCA schemas can be stored in one or more sets. Figure 7 Similar to those described above, using blocks similar to the DDL blocks and DDL management blocks described earlier, DOCS-based BCA schema changes are propagated and coordinated among DOCS-based BCA participants, the difference being that these blocks identify the initiator DOCS rather than the initiator DBMS.

[0259] Finally, DOCS is an extended version of key-value stores. However, in key-value stores, the stored values ​​do not need to conform to a markup language. Nevertheless, DOCS-based BCA networks can still include key-value stores as BCA participants.

[0260] Applications of non-transaction processing systems

[0261] If a PSS does not support transactions about data, there are many ways to implement the all-or-nothing nature of transaction processing. One approach uses a list of intents with idempotent operations to mimic a prepared transaction. Copies of the relevant data are operated on before deciding how to vote on a distributed transaction. For example, creating a new file in the file system is done in a system-managed directory, and hidden files are appropriately renamed when the distributed transaction commits. In Unix / Linux systems, renaming can be done using the "mv" command. Existing files in the file system are updated with copies of the files in the system-managed directory, and temporary files are used to overwrite user files when the distributed transaction commits. In Unix / Linux systems, overwriting can be done using the "mv" command. Deleting a file is simply an entry on the intent list if and when the transaction commits, the user file is actually deleted. If the distributed transaction aborts, each PSS discards the local intent list associated with the distributed transaction, as well as any temporary objects in the system-managed object set. If the distributed transaction commits, each PSS applies the local intent list associated with the distributed transaction and then discards that list. Crashes during application can be handled by reapplying the local intent list from the beginning. For large or expensive distributed transactions, local checkpoints can be used to record progress and minimize operations that are attempted multiple times in the intent list.

[0262] Database system

[0263] A Database Management System (DBMS) manages a database. A DBMS may include one or more database servers. A database consists of database data and a database dictionary stored on a persistent storage mechanism (such as a collection of disks). Database data can be stored in one or more data containers. Each container contains records. The data in each record is organized into one or more fields. In a relational DBMS, the data container is called a table, the record is called a row, and the field is called a column. In an object-oriented database, the data container is called an object class, the record is called an object, and the field is called an attribute. Other database architectures may use different terminology.

[0264] Users interact with the DBMS database server by submitting commands to the database server, causing the database server to perform operations on the data stored in the database. A user can be one or more applications running on the client computer interacting with the database server. Multiple users may also be collectively referred to as users in this document.

[0265] Database commands can be in the form of database statements conforming to a database language. The database language used to express database commands is Structured Query Language (SQL). Many different versions of SQL exist; some are standard, some are proprietary, and there are various extensions. Data Definition Language (“DDL”) commands are issued to the database server to create or configure database objects, such as tables, views, or complex data types. SQL / XML is a common extension of SQL used when manipulating XML data in object-relational databases.

[0266] Transaction processing is used to modify the database in a DBMS. A database transaction is a set of operations that change database data. In a DBMS, a database transaction is initiated in response to a database statement that requests changes (such as a DML statement requesting an update, insert, or delete row). Committing a transaction makes the changes made by that transaction permanent.

[0267] In transaction processing, all changes made to a transaction are performed atomically. When a transaction is committed, either all changes are committed, or the transaction is rolled back. DML statements or commands are statements that specify changes to data, such as INSERT and UPDATE statements. DML statements or commands are not statements that only query database data.

[0268] In distributed transactions, multiple DBMSs use a two-phase commit approach to commit the distributed transaction. Each DBMS executes its local transaction within a branch of the distributed transaction. One DBMS (the coordinating DBMS) is responsible for coordinating the commit of the transaction on one or more other database systems. These other DBMSs are referred to as participating DBMSs in this document.

[0269] Two-phase commit consists of two phases: the prepare-to-commit phase and the commit phase. In the prepare-to-commit phase, branch transactions are prepared in each participating database system. When a branch transaction is prepared on a DBMS, the database is in a "ready state," ensuring that modifications to database data performed as part of the branch transaction can be committed. This guarantee may require persistently storing the change records of the branch transaction. Participating DBMSs confirm the changes upon completing the prepare-to-commit phase and entering the ready state for the corresponding branch transaction in their respective participating DBMSs.

[0270] According to an embodiment, the preparation for submission phase includes adding a TC block to the blockchain or a log record containing the blockchain. Therefore, the participating DBMS cannot enter the preparation state or confirm this state until a TC block is added.

[0271] During the commit phase, the coordinating database system commits transactions on both the coordinating and participating database systems. Specifically, the coordinating database system sends a message to the participants, requesting them to commit the modifications specified by the transaction to the participating database systems. The participating and coordinating database systems then commit the transactions.

[0272] On the other hand, if the participating database system is unable to prepare, or the coordinating database system is unable to commit, then at least one database system cannot make the changes specified by the transaction. In this case, all modifications at every point in both the participating and coordinating database systems are rolled back, restoring each database system to its state before the changes.

[0273] A client can send a series of requests to a DBMS, such as requests to execute queries, by establishing a database session. A database session involves establishing a specific connection between the client and the database server, through which the client can send a series of requests. Database session processes execute and process the requests sent by the client through the database session. The database session can generate an execution plan for the queries sent by the database session client and marshal slave processes to execute that execution plan.

[0274] Database servers can maintain session state data about database sessions. Session state data reflects the current state of the session and may include the identity of the user who created the session, the services used by the user, instances of object types, language and character set data, statistics on the session's resource usage, temporary variable values ​​generated by processes executing software in the session, and storage for cursors, variables, and other information.

[0275] A database server comprises multiple database processes. These processes run under the control of the database server (i.e., they can be created or terminated by the database server) and perform various database server functions. Database processes include those running within the database sessions established for clients.

[0276] A database process is a unit of execution. A database process can be a computer system process, a thread, or a user-defined execution context, such as a user thread or a fiber optic thread. Database processes can also include "database server system" processes that provide services and / or perform functions on behalf of the entire database server. Such database server system processes include listeners, garbage collectors, log writers, and recovery processes.

[0277] A multi-node database management system consists of interconnected nodes, each running a database server that shares access to the same database. Typically, nodes are interconnected via a network and share access to shared storage devices to varying degrees, such as a collection of disk drives and shared access to the blocks of data stored thereon. Nodes in a multi-node database system can be in the form of a group of computers (e.g., workstations, personal computers) interconnected via a network. Alternatively, nodes can be in the form of a grid of server blades interconnected with other server blades on a rack.

[0278] In a multi-node database system, each node hosts a database server. A server (such as a database server) is a combination of integrated software components and computing resources (such as memory, nodes, and processes on the nodes for executing the integrated software components on the processors), a combination of software and computing resources dedicated to performing a specific function on behalf of one or more clients.

[0279] Resources from multiple nodes in a multi-node database system can be allocated to software running a specific database server. Each combination of software and resource allocation from the nodes is a server, referred to herein as a "server instance" or "instance". A database server may include multiple database instances, some or all of which may run on separate computers (including separate server blades).

[0280] A database dictionary can include multiple data structures that store database metadata. For example, a database dictionary can include multiple files and tables. Parts of the data structures can be cached in the database server's main memory.

[0281] When we say that a database object is defined by a database dictionary, the database dictionary contains metadata that defines the characteristics of the database object. For example, the metadata in the database dictionary defining a database table may specify column names and data types of the columns, as well as one or more files or portions thereof for storing the data in the table. The metadata in the database dictionary defining a procedure may specify the procedure's name, the procedure's arguments, and the return data type and the data types of the arguments, and may include the source code and its compiled version.

[0282] Database objects can be defined by a database dictionary, but the metadata in the database dictionary itself can only partially specify the characteristics of the database object. Other characteristics can be defined by data structures that cannot be considered part of the database dictionary. For example, a user-defined function implemented in a Java class can be partially defined by the database dictionary by specifying the name of the user-defined function and specifying a reference to the file containing the source code of the Java class (i.e., the .java file) and the compiled version of the class (i.e., the .class file).

[0283] Hardware Overview

[0284] According to one embodiment, the techniques described herein are implemented by one or more dedicated computing devices. The dedicated computing device may be hardwired to execute the techniques, or may include digital electronic devices permanently programmed to execute the techniques, such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), or may include one or more general-purpose hardware processors programmed to execute the techniques according to program instructions in firmware, memory, other storage devices, or combinations thereof. Such a dedicated computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to implement the techniques. The dedicated computing device may be a desktop computer system, a portable computer system, a handheld device, a networking device, or any other device that combines hardwired and / or program logic to implement the techniques.

[0285] For example, Figure 11 This is a block diagram illustrating a computer system 1100 on which embodiments of the present invention may be implemented. The computer system 1100 includes a bus 1102 or other communication mechanism for transmitting information and a hardware processor 1104 coupled to the bus 1102 for processing information. The hardware processor 1104 may be, for example, a general-purpose microprocessor.

[0286] Computer system 1100 also includes main memory 1106, such as random access memory (RAM) or other dynamic storage devices, coupled to bus 1102 for storing information and instructions to be executed by processor 1104. Main memory 1106 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by processor 1104. When such instructions are stored in non-transitory storage media accessible to processor 1104, computer system 1100 becomes a dedicated machine customized for performing the operations specified in the instructions.

[0287] Computer system 1100 also includes a read-only memory (ROM) 1108 or other static storage device coupled to bus 1102 for storing static information and instructions of processor 1104. Storage device 1110, such as a disk, optical disk, or solid-state drive, is provided and coupled to bus 1102 for storing information and instructions.

[0288] Computer system 1100 may be coupled via bus 1102 to a display 1112, such as a cathode ray tube (CRT), for displaying information to a computer user. Input device 1114, including alphanumeric keys and other keys, is coupled to bus 1102 for transmitting information and command selections to processor 1104. Another type of user input device is a cursor control 1116, such as a mouse, trackball, or arrow keys, for transmitting directional information and command selections to processor 1104 and for controlling cursor movement on display 1112. Such input devices typically have two degrees of freedom on two axes (a first axis (e.g., x) and a second axis (e.g., y)) to allow the device to specify a position in a plane.

[0289] Computer system 1100 may implement the techniques described herein using custom hard-wired logic, one or more ASICs or FPGAs, firmware, and / or program logic. This custom hard-wired logic, one or more ASICs or FPGAs, firmware, and / or program logic, combined with the computer system, enables computer system 1100 to become a special-purpose machine or to program computer system 1100 as a special-purpose machine. According to one embodiment, the techniques herein are executed by computer system 1100 in response to processor 1104 executing one or more sequences of one or more instructions contained in main memory 1106. These instructions may be read into main memory 1106 from another storage medium, such as storage device 1110. Execution of the instruction sequence contained in main memory 1106 causes processor 1104 to perform the processing steps described herein. In alternative embodiments, hard-wired circuitry may be used instead of software instructions or in combination with software instructions.

[0290] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media can include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, or solid-state drives, such as storage device 1110. Volatile media include dynamic memory, such as main memory 1106. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, NVRAMs, any other memory chips, or magnetic tape cassettes.

[0291] Storage media differ from transmission media but can be used in conjunction with them. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including wires containing bus 1102. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0292] Various forms of media can involve carrying one or more sequences of one or more instructions to processor 1104 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit them over a telephone line using a modem. A modem local to computer system 1100 may receive data over the telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector may receive the data carried in the infrared signal, and appropriate circuitry may place the data on bus 1102. Bus 1102 carries the data to main memory 1106, from which processor 1104 retrieves and executes the instructions. The instructions received by main memory 1106 may optionally be stored on storage device 1110 before or after execution by processor 1104.

[0293] Computer system 1100 also includes a communication interface 1118 coupled to bus 1102. Communication interface 1118 provides bidirectional data communication coupled to network link 1120, which is connected to local network 1122. For example, communication interface 1118 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem providing data communication connectivity to a corresponding type of telephone line. As another example, communication interface 1118 may be a LAN card providing data communication connectivity to a compatible local area network (LAN). A wireless link may also be implemented. In any such implementation, communication interface 1118 transmits and receives electrical, electromagnetic, or optical signals carrying streams of digital data representing various types of information.

[0294] Network link 1120 typically provides data communication to other data devices via one or more networks. For example, network link 1120 may provide a connection to host computer 1124 or to data devices operated by Internet Service Provider (ISP) 1126 via local network 1122. ISP 1126, in turn, provides data communication services via a global packet data communication network now commonly referred to as the “Internet” 1128. Both local network 1122 and Internet 1128 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks, as well as signals on network link 1120 and through communication interface 1118, are example forms of transmission media that carry digital data to or from computer system 1100.

[0295] Computer system 1100 can send messages and receive data, including program code, through one or more networks, network links 1120, and communication interfaces 1118. In the Internet example, server 1130 can transmit requested code for the application through the Internet 1128, ISP 1126, local network 1122, and communication interface 1118.

[0296] The received code can be executed by processor 1104 when it is received, and / or stored in storage device 1110 or other non-volatile storage device for later execution.

[0297] In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details, which may vary depending on the implementation. Therefore, the specification and drawings should be viewed in an illustrative rather than restrictive sense. The sole and exclusive indication of the scope of the invention, and what the applicant intends to define as the scope of the invention, is the literal and equivalent scope of the claims generated in the specific form of the set of claims arising from this application, including any subsequent corrections.

[0298] Software Overview

[0299] Figure 12 This is a block diagram of a basic software system 1200 that can be used to control the operation of computer system 1100. Software system 1200 and its components, including their connections, relationships, and functions, are merely exemplary and are not intended to limit the implementation of one or more example embodiments. Other software systems suitable for implementing one or more example embodiments may have different components, including components with different connections, relationships, and functions.

[0300] Software system 1200 is provided to guide the operation of computer system 1100. Software system 1200, which may be stored on system memory (RAM) 1106 and fixed storage device (e.g., hard disk or flash memory) 1110, includes a kernel or operating system (OS) 1210.

[0301] OS1210 manages the low-level aspects of computer operations, including managing process execution, memory allocation, file input and output (I / O), and device I / O. One or more applications, designated 1202A, 1202B, 1202C…1202N, can be “loaded” by system 1200 (e.g., transferred from fixed storage device 1110 to memory 1106) for execution. Applications or other software intended for use on computer system 1100 can also be stored as a set of downloadable computer-executable instructions for downloading and installation, for example, from an internet location (e.g., a web server, app store, or other online service).

[0302] Software system 1200 includes a graphical user interface (GUI) 1215 for receiving user commands and data graphically (e.g., "click" or "touch gestures"). These inputs can then be manipulated by system 1200 according to instructions from operating system 1210 and / or (one or more) applications 1202. GUI 1215 is also used to display the results of operations from OS 1210 and (one or more) applications 1202, whereby the user can provide additional input or terminate the session (e.g., log off).

[0303] OS 1210 can execute directly on the bare hardware 1220 of computer system 1100 (e.g., one or more processors 1104). Alternatively, a hypervisor or virtual machine monitor (VMM) 1230 can be inserted between the bare hardware 1220 and OS 1210. In this configuration, VMM 1230 acts as a software “buffer” or virtualization layer between the bare hardware 1220 and OS 1210 of computer system 1100.

[0304] VMM 1230 instantiates and runs one or more virtual machine instances (“guest machines”). Each guest machine includes a “guest” operating system, such as OS1210, and one or more applications designed to run on the guest operating system, such as application(s)1202. VMM 1230 presents a virtual operating platform for the guest operating system and manages the execution of the guest operating system.

[0305] In some cases, VMM 1230 can allow a guest operating system to run as if it were running directly on the bare hardware 1220 of computer system 1100. In these cases, the same version of the guest operating system configured to run directly on the bare hardware 1220 can also run on VMM 1230 without modification or reconfiguration. In other words, in some situations, VMM 1230 can provide complete hardware and CPU virtualization for a guest operating system.

[0306] In other cases, the guest operating system can be specifically designed or configured to run on the VMM 1230 for improved efficiency. In these cases, the guest operating system "knows" that it is running on the virtual machine monitor. In other words, in some situations, the VMM 1230 can provide paravirtualization to the guest operating system.

[0307] Computer system processes include the allocation of hardware processor time, as well as the allocation of memory (physical and / or virtual), the allocation of memory for storing instructions executed by the hardware processor, the allocation of memory for storing data generated by the execution of instructions by the hardware processor, and / or the storage of hardware processor state (e.g., register contents) between hardware processor time allocations when the computer system process is not running. Computer system processes run under the control of the operating system and can also run under the control of other programs executing on the computer system.

[0308] cloud computing

[0309] This article generally uses the term "cloud computing" to describe a computing model that enables on-demand access to a shared pool of computing resources, such as computer networks, servers, software applications, and services, and allows for the rapid provisioning and release of resources with minimal management effort or service provider interaction.

[0310] Cloud computing environments (sometimes called cloud environments or the cloud itself) can be implemented in various ways to best suit different requirements. For example, in a public cloud environment, the underlying computing infrastructure is owned by an organization that makes its cloud services available to other organizations or the public. In contrast, private cloud environments are generally used only by a single organization or within a single organization. Community clouds are designed to be shared by several organizations within a community; while hybrid clouds include two or more types of clouds (e.g., private, community, or public) bound together by data and application portability.

[0311] Generally, cloud computing models enable some of the responsibilities that might have previously been provided by an organization's own IT department to be delivered as service layers within the cloud environment for consumer use (depending on the public / private nature of the cloud, whether within or outside the organization). Depending on the specific implementation, the precise definition of the components or features provided by or within each cloud service layer can vary, but common examples include: Software as a Service (SaaS), where consumers use software applications running on cloud infrastructure while the SaaS provider manages or controls the underlying cloud infrastructure and applications; Platform as a Service (PaaS), where consumers can use software programming languages ​​and development tools supported by the PaaS provider to develop, deploy, and otherwise control their own applications while the PaaS provider manages or controls other aspects of the cloud environment (i.e., everything in the runtime execution environment); and Infrastructure as a Service (IaaS), where consumers can deploy and run arbitrary software applications and / or provide processes, storage devices, networks, and other basic computing resources while the IaaS provider manages or controls the underlying physical cloud infrastructure (i.e., everything below the operating system layer). Database as a Service (DBaaS) is a service in which consumers use database servers or database management systems running on cloud infrastructure, while the DbaaS provider manages or controls the underlying cloud infrastructure, applications, and servers, including one or more database servers.

Claims

1. A method for providing persistent storage support for a blockchain, the method comprising: A specific BCA participant in the blockchain application BCA network receives a request to submit a specific transaction to a ledger store object stored in a persistent storage system, the specific transaction including one or more operations performed on the ledger store object; The BCA network comprises multiple BCA participants, including the specific BCA participant and multiple other BCA participants, wherein each of the multiple BCA participants includes: The corresponding ledger repository object copy of the ledger repository object; A corresponding BC log copy of the blockchain BC log is stored, wherein the corresponding BC log copy stores a corresponding blockchain copy of the blockchain; In response to receiving the request to submit the specific transaction: Initiate a first distributed transaction for the aforementioned ledger repository object; The specific BCA participant sends a first proposal to the plurality of other BCA participants to add a first log record to the BC log, wherein the first log record includes a first block to be added to the blockchain, and the first block includes a first transaction context specifying the one or more operations and a first set of writes generated by the specific transaction; The consensus outcome of the BCA network in response to the first proposal is determined to be acceptance; In response to determining that the consensus result for the first proposal is acceptance, the corresponding BC log copy of the specific BCA participant is modified to add a first log copy of the first log record to the corresponding BC log. The consensus result of the BCA network submitting the first distributed transaction is determined to be acceptance; the specific BCA participant sends a second proposal to add a second log record to the BC log, the second log record including a second block to be added to the blockchain, the second block including content indicating that the consensus result of the BCA network submitting the first distributed transaction is acceptance; The specific BCA participant determines that the second consensus outcome of the BCA network for the second proposal is acceptance; and In response to the consensus outcome of accepting the second proposal, the specific BCA participant: Modify the corresponding BC log copy of the specific BCA participant to add the second log record copy of the second log record to the corresponding BC log; Submit the first distributed transaction.

2. The method of claim 1, wherein each of the plurality of BCA participants comprises: Define the corresponding BC schema for each BCA participant's corresponding ledger repository object copy and corresponding BC log copy; as well as The corresponding blockchain copy of each BCA participant includes one or more blocks specifying the DDL operations that define the corresponding BC pattern.

3. The method of claim 2, wherein the corresponding BC pattern of each of the plurality of BCA participants defines a blockchain process, wherein the specific transaction includes the execution of the blockchain process.

4. The method of claim 2, further comprising: The specific BCA participant makes pending DDL changes to the corresponding BC pattern of the specific BCA participant; The specific BCA participant sends a third proposal to the plurality of other BCA participants to add a third log entry to the BC log, wherein the third log entry includes a third block to be added to the blockchain, the third block specifying the DDL change; The specific BCA participant determines that the consensus outcome of the BCA network for the third proposal is acceptance; In response to determining that the consensus result for the third proposal is acceptance, the corresponding BC log copy of the particular BCA participant is modified to add the third log copy of the third log record to the corresponding BC log of the particular BCA participant. The consensus outcome of the BCA network in response to the DDL change is determined to be acceptance; Based on the consensus result of the BCA network regarding the DDL change, the DDL change is activated to the corresponding BC mode of the specific BCA participant.

5. The method of claim 4, wherein making pending DDL changes to the corresponding BC pattern of the particular BCA participant includes the period of creating the corresponding BC pattern.

6. The method of claim 1, further comprising: The specific BCA participant adds a third log record to the corresponding BC log of the specific BCA participant, wherein the third log record includes a third block to be added to the blockchain, the third block includes a third transaction context, which specifies the operations performed by a third transaction executed by another BCA participant among the plurality of other BCA participants and a third write set generated by the third transaction, the third transaction being a branch transaction of a third distributed transaction initiated by the other BCA participant; The specific BCA participant executes a local transaction as a branch transaction of the third distributed transaction, thereby generating a local write set; The specific BCA participant compares the third write set and the local write set; Based on the comparison, a vote is taken to submit the third distributed transaction.

7. The method of claim 6, wherein executing the local transaction includes multiple slave processes executing the branch transaction.

8. The method of claim 2, further comprising: The specific BCA participant adds a third log record to the corresponding BC log of the specific BCA participant, wherein the third log record includes a third block to be added to the blockchain, the third block including a second transaction context, which specifies the operations performed by a second transaction executed by another BCA participant among the plurality of other BCA participants and a second set of writes generated by the second transaction, the second transaction being a branch transaction of a second distributed transaction initiated by the other BCA participant; The specific BCA participant executes the process defined by the private schema within the corresponding BC schema of the specific BCA participant; Based on the result returned by the execution of the process, a vote is made to commit the second distributed transaction.

9. The method of claim 8, further comprising: The specific BCA participant adds a third log record to the corresponding BC log copy of the specific BCA participant, wherein the third log record includes a third block to be added to the blockchain, the third block specifying that the consensus result of the second distributed transaction submitted by the BCA network is acceptance; as well as Based on the consensus result of the second distributed transaction specified by the third block, the specific BCA participant submits a branch transaction of the second distributed transaction.

10. The method of claim 1, wherein: The specific BCA participant is assigned the initiator role and each of the plurality of other BCA participants is assigned the follower role; For a given time period among multiple time periods, only one specific BCA participant among the multiple BCA participants acts as the initiator; During the aforementioned multiple time periods, only BCA participants who are assigned the role of initiator are able to propose adding blocks to the BC log chain.

11. The method as described in claim 10, During the multiple time periods, and when another BCA participant among the multiple other BCA participants is assigned the role of the initiator DBMS: The specific BCA participant determines that a specific block proposed by the specific BCA participant is pending; and In response to the specific BCA participant determining that a block proposed by the specific BCA participant is in a pending state, a request to propose a block based on that specific block is sent to the other BCA participant.

12. The method of claim 1, wherein a particular BCA participant among the plurality of BCA participants is a database management system, a file system, a document storage system, or a key-value pair storage system.

13. A non-transitory computer-readable medium having executable instructions stored thereon, the executable instructions, when executed by a processor of a computer, controlling the computer to perform the method as described in any one of claims 1-12.

14. A system for providing persistent storage support for a blockchain, the system comprising: processor; as well as A memory coupled to the processor and having executable instructions stored thereon, which, when executed by the processor, cause the processor to perform the method as described in any one of claims 1-12.

15. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-12.

Images