Method for searching basic page and / or incremental page and computer equipment

Abstract

The invention discloses a method for searching a base page and / or an increment page and computer equipment. The embodiment of the method comprises the following steps: receiving a read state key and a version; according to the state key and the version, matching the version and the page ID in the page identifier of the logic page according to the lexicographical order; searching in an index file of a basic page / incremental page according to the version and the page ID; and after the data file is found, reading the corresponding data file from the persistent medium according to the persistent medium position in the entry in the index.

Description

Technical Field

[0001] The embodiments of this specification belong to the field of blockchain technology, and in particular, relate to a method and computer device for searching a base page and / or an incremental page. Background Art

[0002] Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. In the blockchain system, data blocks are combined into a chain data structure in a sequential manner according to time sequence, and a distributed ledger that cannot be tampered with or forged is guaranteed by cryptography. Due to the characteristics of blockchain such as decentralization, information cannot be tampered with, and autonomy, blockchain has also received more and more attention and application. Summary of the invention

[0003] The object of the present invention is to provide a method and computer device for searching a base page and / or a delta page, comprising:

[0004] A method for finding a base page and / or a delta page, comprising:

[0005] Receive the read status key and version;

[0006] According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order;

[0007] Search in the index file of the base page / delta page according to the version and page ID;

[0008] After finding it, the corresponding data file is read from the persistent medium according to the persistent medium position in the entry in the index.

[0009] A method for searching a base page and / or an incremental page, wherein a file storage location of a previous predecessor version is set in a file of the incremental page, the method comprising:

[0010] Receive the read status key and version;

[0011] According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order;

[0012] Search in the index file according to the version and page ID;

[0013] After finding it, recursively read the file position of the base page / incremental page of its predecessor stored in the found incremental page until the file position of the base page is found;

[0014] According to the persistent media location of the found base page / incremental page, the corresponding data file is read from the persistent media.

[0015] A method for searching a base page and / or a delta page, wherein a version of a previous predecessor base page / delta page is set in a delta page entry of an index, the method comprising:

[0016] Receive the read status key and version;

[0017] According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order;

[0018] Find the incremental page entry in the index file according to the version and page ID;

[0019] After finding the incremental page entry, recursively read the version of the predecessor base page / incremental page stored in the found incremental page entry and search in the index file according to the found version and page ID until the base page is found;

[0020] The location of the corresponding data file in the persistent medium is determined in the index based on the found version of the base page / incremental page and the page ID, and the corresponding data file is read from the persistent medium.

[0021] A computer device comprising:

[0022] processor;

[0023] and a memory storing a program, wherein when the processor executes the program, any of the operations above is performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 is a schematic diagram of deploying a smart contract in one embodiment;

[0026] Figure 2 is a schematic diagram of calling a smart contract in one embodiment;

[0027] Figure 3 is a schematic diagram of a block storage structure in one embodiment;

[0028] Figure 4 is a schematic diagram of a block storage structure in one embodiment;

[0029] Figure 5 is a schematic diagram of an MPT tree in one embodiment;

[0030] Figure 6 It is a schematic diagram of the modules involved in the transaction processing process and the relationship between the CPU, memory and disk in one embodiment;

[0031] Figure 7 It is a schematic diagram of the modules involved in the transaction processing process and the relationship between the CPU, memory and disk in one embodiment;

[0032] Figure 8 is a schematic diagram of the logical relationship of the state database in one embodiment;

[0033] Fig. 9 is a flow chart of reading status data in one embodiment;

[0034] Fig.10 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;

[0035] Fig.11 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;

[0036] Fig.12 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;

[0037] Fig.13 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;

[0038] Fig.14 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;

[0039] Fig.15 is a schematic diagram of a block storage structure in one embodiment;

[0040] Fig.16 is a schematic diagram of a block storage structure in one embodiment;

[0041] Fig.17 is a schematic diagram of a block storage structure in one embodiment;

[0042] Fig.18 is a schematic diagram of a block storage structure in one embodiment;

[0043] Fig.19 is a schematic diagram of a block storage structure in one embodiment;

[0044] Fig. 20 is a schematic diagram of a block storage structure in one embodiment;

[0045] Fig.21 is a schematic diagram of a storage structure in one embodiment;

[0046] Fig. 22 is a schematic diagram of a storage structure in one embodiment;

[0047] Fig.23 is a schematic diagram of a storage structure in one embodiment. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0049] Blockchain is generally divided into three types: public blockchain, private blockchain and consortium blockchain. In addition, there are many types of combinations, such as private blockchain + consortium blockchain, consortium blockchain + public blockchain and other different combinations. Among them, the public chain has the highest degree of decentralization. Public chains are represented by Bitcoin and Ethereum. Participants who join the public chain can read data records on the chain, participate in transactions, and compete for the right to record new blocks. Moreover, each participant (reflected as the participant's node on the blockchain) can freely join and exit the network and perform related operations. On the contrary, the private chain has the write permission of the network controlled by a certain organization or institution, and the data reading permission is stipulated by the organization. In short, the private chain can be a weakly centralized system with strictly limited and few participating nodes. This type of blockchain is more suitable for use within a specific organization. The consortium chain is a blockchain between the public chain and the private chain, which can achieve "partial decentralization". Each node in the alliance chain usually has a corresponding entity or organization; participants join the network through authorization and form a stakeholder alliance to jointly maintain the operation of the blockchain.

[0050] Whether it is a public chain, a private chain or a consortium chain, in addition to supporting the transfer of native assets on the blockchain between accounts, such as the transfer of Bitcoin and Ethereum, it can also provide smart contract functions. Smart contracts on the blockchain are contracts that can be triggered and executed by transactions on the blockchain system. Smart contracts can be defined in the form of code.

[0051] Taking Ethereum as an example, supporting users to create and call some complex logic in the Ethereum network is the biggest challenge that distinguishes Ethereum from Bitcoin blockchain technology. The core of Ethereum as a programmable blockchain is the Ethereum Virtual Machine (EVM), and each Ethereum node can run EVM. EVM is a Turing-complete virtual machine, which means that various complex logics can be implemented through it. Users publish and call smart contracts in Ethereum on EVM. In fact, the virtual machine directly runs the virtual machine code (virtual machine bytecode, hereinafter referred to as "bytecode"). Smart contracts deployed on the blockchain can be in the form of bytecode.

[0052] For example Figure 1 As shown in the figure, after Bob sends a transaction containing the information of creating a smart contract to the Ethereum network, the EVM of node 1 can execute this transaction and generate the corresponding contract instance. "0x6f8ae93..." in Figure 1 represents the address of this contract. The data field of the transaction can store bytecode, and the to field of the transaction is an empty account. After the nodes reach an agreement through the consensus mechanism, the contract is successfully created and subsequent users can call this contract.

[0053] After the contract is created, a contract account corresponding to the smart contract is generated on the blockchain, and has a specific address. The contract code and account storage will be saved in the contract account. The behavior of the smart contract is controlled by the contract code, and the account storage of the smart contract saves the state of the contract. In other words, the smart contract generates a virtual account on the blockchain that contains the contract code and account storage.

[0054] As mentioned above, the data field of the transaction that creates a smart contract can store the bytecode of the smart contract. The bytecode consists of a series of bytes, each of which can identify an operation. Based on development efficiency, readability and other considerations, developers can choose a high-level language to write smart contract code instead of writing bytecode directly. The smart contract code written in a high-level language is compiled by a compiler to generate bytecode, which can then be deployed on the blockchain. Ethereum supports many high-level languages, such as Solidity, Serpent, LLL, etc.

[0055] Taking Solidity as an example, contracts written in it are very similar to classes in object-oriented programming languages. Multiple members can be declared in a contract, including state variables, functions, function modifiers, events, etc. State variables are values ​​stored in the account storage of smart contracts and are used to save the state of the contract.

[0056] The following is a code example of a simple smart contract written in Solidity language 1:

[0057] Contract SimpleStorage {

[0058] string storedData;

[0059] function set(string s){

[0060] storedData = s;

[0061] }

[0062] function get()constant returns(string){

[0063] return storedData;

[0064] }

[0065] }

[0066] In addition, if Figure 2 As shown, still taking Ethereum as an example, after Bob sends a transaction containing information about calling a smart contract to the Ethereum network, the EVM of node 1 can execute this transaction and generate a corresponding contract instance. The from field of the transaction in Figure 2 is the address of the account that initiates the call to the smart contract, the "0x6f8ae93..." in the to field represents the address of the smart contract being called, and the data field of the transaction stores the method and parameters for calling the smart contract. In addition, the value field can also be included to represent the value of the Ether in the transaction. After calling the smart contract, the value of storedData may change. Later, a client can call a blockchain node (for example Figure 2 Node 6) in the view the current value of storedData.

[0067] Smart contracts can be executed independently on each node in the blockchain network in a prescribed manner, and all execution records and data are stored on the blockchain. Therefore, when such a transaction is completed, the blockchain stores transaction credentials that cannot be tampered with or lost.

[0068] As mentioned above, the storedData in the above example is a state variable, which is stored in the account storage of the smart contract. In various blockchain networks that introduce smart contracts, such as Ethereum, accounts can usually include two types:

[0069] Contract account: stores the executed smart contract code and the value of the state in the smart contract code, and can usually only be activated by calling an external account;

[0070] Externally owned account: A user's account, such as an Ethereum owner account.

[0071] The design of external accounts and contract accounts is actually a mapping from account addresses to account status. Account status usually includes fields such as Nonce, Balance, Storage root, CodeHash, etc. Nonce and Balance exist in both external accounts and contract accounts. CodeHash and Storage root properties are generally only valid on contract accounts.

[0072] Nonce: Counter. For external accounts, this number can represent the number of transactions sent from the account address; for contract accounts, it can be the number of contracts created by the account.

[0073] Balance: The amount of ether owned by this address.

[0074] Storage root: The hash of the root node of an MPT tree, which organizes the storage of state variables of the contract account.

[0075] CodeHash: The hash value of the smart contract code. For contract accounts, this is the hash value of the smart contract; for external accounts, since they do not include smart contracts, the CodeHash field can generally be an empty string / all-0 string.

[0076] MPT stands for Merkle Patricia Tree, which is a tree structure that combines Merkle Tree and PatriciaTree (compressed prefix tree, a more space-saving Trie tree, dictionary tree). Merkle Tree, the Merkle tree algorithm calculates a hash value for each transaction, and then connects two by two to calculate the hash again until the top Merkle root. Ethereum uses an improved MPT tree, such as a 16-way tree structure, which is usually referred to as MPT tree.

[0077] The data structure of the Ethereum MPT tree includes a state trie. The state trie contains key-value pairs (also written as key-value, abbreviated as kv or kv) of the storage content corresponding to each account in the Ethereum network. The "key" in the state trie can be a 160-bit identifier (such as the address of an Ethereum account or part of the hash value of an address, collectively referred to as an account address below), which is distributed in the storage from the root node of the state trie to the leaf node. The "value" in the state trie is generated by encoding the information of the Ethereum account (using the Recursive-Length Prefix encoding (RLP) method). As mentioned above, for external accounts, the values ​​include nonce and balance; for contract accounts, the values ​​include nonce, balance, codehash, and storageroot.

[0078] The contract account is used to store the states related to the smart contract. After the smart contract is deployed on the blockchain, a corresponding contract account will be generated. This contract account generally has some states, which are defined by the state variables in the smart contract and generate new values ​​when the smart contract is created and executed. The smart contract generally refers to a contract that can automatically execute clauses defined in the form of code in the blockchain environment. Once an event triggers the clauses in the contract (meets the execution conditions), the code can be automatically executed. In the blockchain, the relevant states of the contract are saved in the storage trie, and the hash value of the storage trie root node is stored in the above storageroot, so that all states of the contract are locked to the contract account through hash. The storage trie is also an MPT tree structure that stores the key-value mapping from the state address to the state value. Part of the information from the root node to the leaf node of the storage trie tree is arranged in sequence to store the address of a state, and the leaf node stores the value of the state.

[0079] like Figure 3In some blockchain data storages shown, the block header of each block includes several fields, such as the previous block hash previous_Hash (Prev Hash in the figure), the random number Nonce (in some blockchain systems, this Nonce is not a random number, or in some blockchain systems, the Nonce in the block header is not enabled), the timestamp Timestamp, the block number Block Num, the state root hash State_Root, the transaction root hash Transaction_Root, the receipt root hash Receipt_Root, etc. Among them, the Prev Hash in the block header of the next block (such as block N+1) points to the previous block (such as block N), which is the hash value of the previous block. In this way, the next block on the blockchain is locked to the previous block through the block header. Among them, State_Root, Transaction_Root and Receipt_Root lock the state set, transaction set and receipt set respectively. The state set, transaction set and receipt set organize the state, transaction and receipt in the form of a tree respectively. Generally, it can be the same tree structure or different tree structures. For example, in Ethereum, the same MPT structure is used. In some tree structures including the state set of smart contracts such as Ethereum, there are two levels of MPT structures: the leaf nodes of the upper-level MPT structure include two types: external accounts and contract accounts; each contract account includes the MPT structure of the lower level, and the leaf nodes of the lower level include the value of the state in the contract account.

[0080] Figure 4 This is a schematic diagram of the structure of blockchain data storage. Taking Ethereum as an example, we can combine Figure 3As shown in the figure, state_root is the hash value of the root of the MPT tree composed of the status of all accounts in the current block, that is, the state trie pointing to state_root is a state tree in the form of MPT. The root node of this MPT tree is generally an extension node (Extension Node) or a branch node (Branch Node), and the hash value of this root node is generally stored in state_root. The root node can be connected to one or more layers of Extension Node / Branch Node below, and these multi-layer tree nodes can be collectively referred to as intermediate nodes (Internal Node). From the root node of this MPT to the leaf node, a part of the value in each node is connected in sequence to form an account address and serve as a key. The account information stored in the leaf node is the value corresponding to this account address, thus forming a key-value pair. This key can also be a part of sha3(Address), that is, a part of the hash value of the account address (the hash algorithm uses the sha3 algorithm, for example), and the value stored can be rlp(Account), that is, the rlp encoding of the account information. Among them, the account information is

[0081] A tuple consisting of [nonce, balance, storageRoot, codeHash]. As mentioned above, for external accounts, there are generally only two items, nonce and balance, and the storageRoot and codeHash fields store empty strings / all zero strings by default. In other words, external accounts do not store contracts, nor do they store state variables generated after contract execution. Contract accounts generally include Nonce, Balance, Storage root, and CodeHash. Among them, Nonce is the transaction counter of the contract account; Balance is the account balance; Storage root corresponds to another MPT, through which the storage root can be linked to the information of the contract-related status; CodeHash is the hash value of the contract code. Whether it is an external account or a contract account, its account information is generally located in a separate leaf node (Leaf Node). From the Extension Node / Branch Node of the root node to the Leaf Node of each account, there may be several branch nodes and extension nodes in the middle.

[0082] The state trie can be a tree in the form of MPT, generally a 16-way tree, that is, each layer can have up to 16 child nodes. For Extension Node, it is used to store common prefixes, and it generally has 1 child node, which can be Branch Node. For Branch Node, it can have up to 16 child nodes, which may include ExtensionNode and / or Leaf Node.

[0083] Among them, for a contract account in the state trie, its storage_Root points to another tree in the form of MPT, which stores the data of the state variables involved in the contract execution. The MPT-style tree pointed to by this storage_Root is Storage Trie, that is, the hash value of the root node of Storage Trie. Generally, this Storage Trie tree also stores key-value pairs. The key indicates the address of the state variable, and its value can be the result obtained after the position of the state variable declaration in the contract (the value counting from 0) is processed by certain rules, such as sha3 (the position of the state variable declaration) or sha3 (the contract name + the position of the state variable declaration). The value is used to store the value of the state variable (for example, the value encoded by RLP). A part of the data stored on the path from the root node to the intermediate node to the leaf node is connected to form the key, and the value is stored in the leaf node. As mentioned above, the Storage trie can also be a tree in the form of MPT, which is generally a 16-way tree, that is, for a Branch Node, it can have up to 16 child nodes, which may include Extension Nodes and / or Leaf Nodes. For an ExtensionNode, it can generally have 1 child node, which can be a Branch Node or a Leaf Node.

[0084] For example Figure 4The Leaf Node Account P of the state trie in the contract is a contract account, and its Storage Root locks all states in the contract storage. These states are organized into an MPT tree, and the tree structure is like the Storage trie linked to the Storage Root. In this linked Storage trie, taking Leaf Node StateVariable N as an example, for example, the value of storedData in the aforementioned contract code example, its key is sha3 (the declaration position of storedData, that is, the second line of the code), and its value is s (for simplicity, the encoding format of value is omitted here, for example, RLP, which will be similar in the future and will not be repeated). Among them, the key values ​​are distributed sequentially from the root node to the leaf node (that is, Leaf Node Variable N) of the storage trie.

[0085] For another example, Figure 4 Leaf Node Account C in the state Trie is an external account, and its key is sha3(Address C), which is the hash value of the address of account C (the hash algorithm uses the sha3 algorithm, for example). The stored value value can be (Account), where the account information Account is a tuple consisting of [nonce, balance]. As mentioned above, since Account C is an external account, its account information is two items, nonce and balance (codehash and storage root are omitted here, and the following are similar). For example, an external account, whose nonce is 20 and Balance is 4550, then the Leaf Node State Variable C stores nonce=20 and balance=4550. The address of Account C is the key, and its values ​​are distributed sequentially from the root node to the leaf node (i.e., Leaf NodeVariable C) of the state Trie.

[0086] These states, including the kv of external accounts and the kv of contract accounts, are ultimately stored in the database. The storage in the database does not directly store the states of these accounts, that is, it does not directly store the kv of these accounts, but stores the kv value of each tree node itself.

[0087] like Figure 5As shown in the example, in the upper-level MPT structure, for leaf node A1, the key of the leaf node is composed of a7 of the shared nibble in the root node A8 (Extension Node)—slot 1 of the intermediate node A7 (Branch Node)—1335 of the key-end in the leaf node A1, which is a711335, and Balance=45.0ETH and Nonce=n1 are stored in the leaf node. For leaf node A2, the key of the leaf node is composed of a7 of the shared nibble in the root node A8 (Extension Node)—slot 7 of the intermediate node A7 (Branch Node)—d3 of the shared nibbles in the node A6 (Extension Node)—slot 3 of the intermediate node A5 (Branch Node)—7 of the key-end in the leaf node A2, which is a77d337, and Balance=1.00WEI and Nonce=n2 are stored in the leaf node. For leaf node A3, the key of the leaf node is composed of a7 of the shared nibble in root node A8 (Extension Node), slot f of intermediate node A7 (Branch Node), and 9365 of key-end in leaf node A3, which is a7f9365. Balance = 1.1ETH and Nonce = n3 are stored in the leaf node. For leaf node A4, the key of the leaf node is composed of a7 of the shared nibble in root node A8 (Extension Node), slot 7 of intermediate node A7 (Branch Node), d3 of the shared nibbles in node A6 (Extension Node), slot 9 of intermediate node A5 (Branch Node), and 7 of key-end in leaf node A4, which is a77d397. Balance = 0.12ETH, Nonce = n4, CodeHash = c1, and Storage root = s1 are stored in the leaf node. s1 can be H(A10), which is the hash of the root node A10 of the next tree layer. The leaf nodes of A1, A2 and A3 store the information of external accounts, and the leaf node of A4 stores the information of contract accounts. For contract accounts, it contains the next level MPT, forming a Storage Trie, which is used to store the state variables in the contract account.

[0088] like Figure 5As shown in the example, in the next-level MPT structure, for leaf node A11, slot 3 in root node A10 (Branch Node) - 35b2e4 of key-end in leaf node A11 are sequentially combined to form the key of the leaf node, that is, 335b2e4, and "Zhang San_A = 20" is stored in the leaf node, for example, indicating that the share of type A digital assets defined in the contract belonging to Zhang San is 20, that is, the balance of Zhang San's type A assets is 20. For leaf node A12, slot 7 in root node A10 (Branch Node) - c25988 of key-end in leaf node A12 are sequentially combined to form the key of the leaf node, that is, 7c25988, and "Li Si_B = 20" is stored in the leaf node, for example, indicating that the share of type B digital assets defined in the contract belonging to Li Si is 50, that is, the balance of Li Si's type B assets is 50. For leaf node A15, the key of the leaf node is composed of slot f in root node A10 (Branch Node) - a in shared nibble in intermediate node A13 (Extension Node) - slot 6 in intermediate node A14 (Branch Node) - be33 of key-end in leaf node A15, which is fa6be33, and "storedData=s" is stored in the leaf node. For leaf node A16, the key of the leaf node is composed of slot f in root node A10 (Branch Node), a in shared nibble in intermediate node A13 (Extension Node), slot 9 in intermediate node A14 (Branch Node), and 9365 in key-end in leaf node A16, which is fa99365. "Wang Wu_A=35" is stored in the leaf node, which means that the share of type A digital assets defined in the contract belonging to Wang Wu is 35, that is, the balance of Wang Wu's type A assets is 35.

[0089] In the node structure of the above MPT tree, the prefix prefix is ​​used to indicate the tree node type, for example, 0 indicates an Extension Node containing an even number of shared nibbles (shared half bytes), 1 indicates an Extension Node containing an odd number of shared nibble(s), 2 indicates a Leaf Node containing an even number of nibbles, and 3 indicates a Leaf Node containing an odd number of nibble(s).

[0090] In the above node structure, the hash value of the entire content of the next tree node is filled in the corresponding position of the previous tree node. In the database, the key-value mapping of each tree node is actually stored, where the value includes the content stored in this tree node, and the corresponding key is the hash value of the entire content of this tree node. In this way, the tree node kv actually stored in the database is as follows:

[0091]

[0092]

[0093] Table 1. Tree node kv actually stored in the database

[0094] In Table 1 above, H() is used to represent hash calculation. In this way, the hash value of the next tree node is anchored in the previous tree node. Through such layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and the root hash is locked in the state root field of the block header.

[0095] In some blockchain systems, the code of the blockchain platform may include a P2P (Peer to Peer) module, a consensus module, an execution module, and a storage module. P2P is a way of forming a computer network. Unlike common web networks, P2P is decentralized. The P2P module can complete the distributed dissemination of data. For blockchain nodes, the P2P module can be used to disseminate and receive receipts in a peer-to-peer manner. Different participants can establish a distributed blockchain network through deployed nodes. The ledger constructed using a chain block structure is stored on each node (or most nodes, such as consensus nodes) in the distributed blockchain network. This is also called a decentralized (or multi-centralized) distributed ledger. Such a blockchain system needs to solve the problem of consistency and correctness of the ledger data on multiple decentralized (or multi-centralized) nodes. The same blockchain platform program is running on each node. Under certain fault tolerance requirements, the consensus module can ensure that all loyal nodes have the same transactions, thereby ensuring that all loyal nodes have consistent execution results for the same transaction, and packaging the transactions and execution results to generate blocks. The current mainstream consensus mechanisms include: Proof of Work (POW), Proof of Stake (POS), Delegated Proof of Stake (DPOS), Practical Byzantine Fault Tolerance (PBFT) algorithm, Honey Badger BFT algorithm, etc. During the consensus process, the consensus module can generally generate the timestamp of the block corresponding to the current transaction set. The execution module can execute transactions, including ordinary transfer transactions and transactions involving contracts, before or after the consensus module completes the consensus. For transactions involving contracts, the execution module can introduce a virtual machine to execute the code of the smart contract, such as the Ethereum Virtual Machine (EVM), so that the differences in the hardware configuration and software environment of each node are shielded by the EVM to ensure that the process and results of executing the smart contract on each node are the same, and the sandbox environment is used to prevent the execution of the smart contract from affecting the blockchain platform code, other programs or operating systems on the host. For a situation of a consortium chain, the nodes can determine the transaction content and transaction order in a transaction set through the consensus module, and then output a deterministic transaction set of the consensus result to the execution module. The execution module generates the execution result by executing ordinary transfer transactions / transactions involving contracts, and sends it to the storage module. The storage module can be responsible for storing the execution result in the local persistent storage medium of the node.

[0096] like Figure 6 In the blockchain node shown, the physical components include CPU, memory, disk, etc. The blockchain platform code executed by this blockchain node may include P2P module, consensus module, execution module and storage module. The functional implementation of P2P module, consensus module and execution module generally requires the participation of CPU and memory. The storage module may include a tree building module, a block header generation module, a WAL (Write Ahead Log) module and a state database module. Among them, the tree building module is used to build a tree (for example, an MPT tree) based on the state kv passed in by the execution module, such as the aforementioned state trie and storage trie, so as to obtain the kv of the tree node, which generally requires the participation of CPU and memory. The block header generation module is used to generate a block header based on the root node of the tree built by the tree building module and some other data (such as the hash of the previous block, timestamp, block number, etc.), which generally requires the participation of CPU and memory. The WAL module is used to persistently store the leaf node kv of the tree generated by the tree building module before writing the leaf node kv of the tree generated by the tree building module to the state database module, so as to prevent data loss caused by power failure and other situations during the process of writing the leaf node kv of the tree generated by the tree building module to the state database module, and to restore the data when this happens, which generally requires the participation of the CPU, memory and disk. The state database module is used to store the tree node kv in Table 1 constructed by the tree building module on a persistent storage device; since the tree node data will eventually be written to a persistent storage medium (such as the disk in the figure), the state database module generally requires the participation of the disk in addition to the CPU and memory.

[0097] From the perspective of storage structure, the above-mentioned Merkle tree structure, such as Ethereum's MPT and Libra's SMT (SparseMerkle Tree, similar to MPT), can be located in the tree construction module in the form of the corresponding relationship in Table 1 above and stored in the memory. Among them, the upper Merkle tree is a prefix tree (dictionary tree), which can realize the organization of data and obtain a unique Merkle root for the organized data. The leaf node can save the state Value, and the root node to the intermediate node to the leaf node implements the lexicographic index of the state key. These tree nodes are encoded as Key according to certain rules, and their contents are encoded as Value, and finally stored in the lower database. Most databases use NoSQL Key-Value DB (DataBase, database; Key-Value DB is also referred to as KVDB) with LSM (Log-Structured Merge-Tree, log structure merge tree) class structure, which is located in the state database module and finally saved on disk. Specifically, the database is, for example, Ethereum's levelDB and Libra's RocksDB. Both KVDBs are based on the LSM storage engine.

[0098] The LSM storage engine is a hierarchical, ordered, disk-oriented storage engine. It draws on the characteristics of continuous appending (rather than modification) of Logs. The core idea is to make full use of the fact that sequential batch writes of disks are much more efficient than random writes, and give up some read efficiency in exchange for maximum write efficiency. Generally speaking, the way to maximize the use of disk characteristics is to read or write a fixed-size block of data at one time and reduce random addressing operations as much as possible. The design idea of ​​LSM is based on this characteristic of disks and assumes that there is enough memory. It does not need to write data to disk every time it is updated. Instead, the latest data resides in memory first. After the amount of data accumulates enough, the data in memory is merged with the data on disk using merge sort and appended to disk in batches.

[0099] The following Figure 7As an example, the storage process of the state data module is further explained. After the tree building module generates the tree node kv data and sends it to the storage module, these kv data are first written to WAL in an Append-Only manner and stored on the disk. Then, the tree node kv data is passed to the state database module. As mentioned above, writing to WAL first can prevent data loss caused by power outages and other situations during the process of writing the tree node kv generated by the tree building module to the state database module, and restore the data when this happens. After the state database module receives the tree node kv data, it is first stored in the memory, specifically in the MemTable in the memory. MemTable can provide concurrent read and write operations. There can be multiple MemTables in the memory. When the amount of data in the MemTable reaches a certain threshold, for example, when it reaches 256MB, the data in the MemTable can be written (flush) to the disk. In order to avoid the blockage of flush caused by the operation of writing MemTable, this MemTable is converted into an immutable Immutable Memtable, that is, Immutable Memtable is set to read-only, and a new MemTable is generated to receive the newly passed in tree node kv. This new MemTable can provide concurrent read and write operations (such as New MemTable in the figure). The storage engine writes the data in Immutable MemTable to disk.

[0100] In the disk, KVDB is stored in multiple levels of SST (Sorted String Table, SSTable) files. SST comes from the paper of Google Bigtable. BigTable is a distributed data storage system designed by Google, a non-relational database used to process massive amounts of data. SST can be used to store data files inside Bigtable, and is an efficient key-value file storage format. Content stored in persistent storage media can also be called files.

[0101] The SST may include multiple layers, for example, 3 layers, 4 layers, 5 layers, 6 layers, 7 layers or more. Figure 7The figure shows the situation of three layers, namely level 0, level 1, and level 2. Generally, the total capacity of the SST of the previous layer is significantly smaller than that of the next layer. As an example, the total capacity of the SST of level 0 is 1GB, the total capacity of the SST of level 1 is 10GB, and the total capacity of the SST of level 2 is 100GB. Assume that the capacity of MemTable is 256MB. The tree module can be written with the tree node kv corresponding to the block, and then written to MemTable. After the tree kv corresponding to one or more blocks is written to MemTable, the space occupied by MemTable may reach 256MB. Then, this MemTable is converted into Immutable MemTable, and the data in ImmutableMemTable can be written (flush, here refers to the operation of writing data in memory to disk) to disk. Specifically, the data in MemTable can be flushed to the SST of Level 0 on the disk. On the other hand, as mentioned above, a new MemTable is generated to receive the newly incoming tree node kv and provide concurrent read and write operations.

[0102] As mentioned above, each layer of disk storage, that is, each level of disk storage, can include multiple SST files. The kv stored in the SST of Level 1 and lower levels is generally stored in the order of the key, and the key values ​​in different SSTs of the same layer are also related in size and are not repeated. For example, Level 1 includes 3 SSTs, the SST on the left 11 It includes 1000 kv, for example <k 1 -v 1 > <k 2 -v 2 > <k 3 -v 3 >... <k 1000 -v 1000 >, and arrange them in order from small to large k, for example, k 1 =10b43c032a46,k 2 =1118d46f41e1,k 3 =1b44942a3d6e,...,k 1000 =2018df9326a7. It can be seen that the SST on the left 11 The k contained in 1 <k 2 <k 3 <...<k 1000 , but k 1 , k 2 , k 3 ,...k1000 are not necessarily continuous. Similarly, the SST in the middle 12 For example, it includes 800 kv, <k 1 -v 1 > <k 2 -v 2 > <k 3 -v 3 >... <k 800 -v 800 >, and arrange them in order from small to large k, for example, k 1 =21a514c0a7e1,k 2 =21b46b994214,k 3 =21b489571b6d,...,k 800 =284a5df3fca7. It can be seen that the SST in the middle 12 The k contained in 1 <k 2 <k 3 <...<k 800 , but similarly, k 1 , k 2 , k 3 ,...k 800 are not necessarily continuous. Similarly, the SST on the right 13 For example, it includes 1100 kv, <k 1 -v 1 > <k 2 -v 2 > <k 3 -v 3 >... <k 1100 -v 1100 >, where k is arranged in ascending order, for example, k 1 =3a5e55ac3fe1,k 2 =3a5e55ac3fe2,k 3 =3afb8ed31526,...,k 1100 =400a51ef1405. It can be seen that the SST on the right 13 The k contained in 1 <k 2 <k 3 <...<k 1100 , but similarly, k 1 , k 2 , k 3 ,...k 1100 Moreover, overall, the left SST 11 The minimum value of k is less than the left SST 11The maximum value of k is less than the intermediate SST 12 The minimum value of k < the intermediate SST 12 The maximum value of k is less than the right SST 13 The minimum value of k is less than the right SST 13 The same is true for each layer of Level 2, Level 3, etc., that is, within each SST in each layer, there is a size order relationship in the value of k, and between multiple SSTs in each layer, there is a size relationship in the range of k.

[0103] The kv in an SST in Level 0 can be sorted according to the size of k, and there may be no size relationship between k in multiple SSTs. This is because MemTable generally stores multiple kvs corresponding to each block in the order of blocks from old to new (or block numbers from small to large, because the block numbers of new blocks are larger and the block numbers of old blocks are smaller), and these k are the hash values ​​of tree nodes. The changes in the content of tree nodes are likely to cause the hash values ​​of tree nodes (i.e., k values) to be very different. The kv in an SST in Level 0 can be sorted according to the size of k. Such sorting is efficient and feasible in memory; of course, it is also possible not to sort according to the size of k values, which is not excluded here. Taking sorting as an example, when the capacity of MemTable reaches 256MB, it may store multiple kvs corresponding to multiple blocks, and this MemTable is converted to Immutable MemTable. These kvs in Immutable MemTable can be sorted in the order of k and written into the SST on the left of Level 0. There may be multiple different Immutable MemTables written to the same SST before the SST on the left side of Level 0 is written. 01 Once it is full, the SST on the right side of Level 0 can be written in a similar way. 02. Although the kv written by a single Immutable MemTable to a certain SST in Level 0 is sorted, since the kv sets written by different Immutable MemTables in the same SST in Level 0 are generally not sorted by size, and the kvs between different SSTs (such as the left and right SSTs) are also not sorted by size, that is, the ranges of k in the left and right SSTs in Level 0 may overlap. Similarly, since the kv written to an SST in Level 0 may be written after multiple MemTables are full, and the kv ranges in the writing process after multiple MemTables are full generally overlap, there is generally no specific order between the multiple kvs stored in each SST in Level 0, and there is generally no specific range order between the SSTs in Level 0. Of course, if an SST is written by an Immutable MemTable in Level 0, since the Immutable MemTable sorts the kvs when writing to the SST, the kvs stored in the SST are also sorted by k.

[0104] Furthermore, when the storage capacity of Level 0 reaches or approaches the upper limit, the data of Level 0 is written to Level 1 through a process called "compaction". During this compaction process, the kv in each SST in Level 0 and the kv of some or all SSTs in Level 1 can be transferred to the memory first, sorted in the memory, and then written to the SST of Level 1. Since sorting is performed during the compaction process, there is an order relationship in the value of k in the SST in Level 1, and there is a size relationship in the range of k between multiple SSTs in Level 1. In other words, in each SST of Level 1, kv is sorted according to <k 1 -v 1 > <k 2 -v 2 > <k 3 -v 3 >... <k n -v n >, and k 1 <k 2 <k 3 <...<k n , and k 1 , k 2 , k 3 , ...k nThey are not necessarily continuous. Moreover, overall, for two adjacent SSTs, the minimum value of k of the left SST < the maximum value of k of the left SST < the minimum value of k of the right SST < the maximum value of k of the right SST. Figure 8 as shown in .

[0105] During the compaction process, the kvs in each SST in Level 0 and the kvs in some SSTs in Level 1 can be transferred to the memory for sorting. The said some SSTs in Level 1 can be the SSTs in Level 1 that have overlapping ranges with the kvs in Level 0, that is, the SSTs in Level 1 that have overlapping ranges with the kvs in each SST in Level 0. Specifically, for example, in the above example, Level 1 includes 3 SSTs, SST 11 The key range is 10b43c032a46~2018df9326a7, SST 12 The key range is 21a514c0a7e1~284a5df3fca7, SST 13 The key range is 3a5e55ac3fe1~400a51ef1405. For example, the key range of each SST in Level 0 is 18c49a405e32~24d35f39ab56. It can be seen that there are SSTs in Level 1 that overlap with the key range in Level 0. 11 and SST 12 In this way, during the compaction process, the kv in each SST in Level 0 and the SST in Level 1 11 、SST 12 The kv can be transferred to the memory and sorted, and then the sorted results are written into the SST of Level 1.

[0106] Similarly, when the storage capacity of Level 1 reaches the upper limit, the data of Level 1 is written to Level 2 through the compaction process. Similarly, during this compaction process, the kvs in each SST in Level 1 and the kvs of some or all SSTs in Level 2 can be first transferred to the memory, sorted in the memory, and then written to the SST of Level 2. Similarly, if there is Level 3 under Level 2, when the storage capacity of Level 2 reaches the upper limit, the data of Level 2 is written to Level 3 through the compaction process. Similarly, during this compaction process, the kvs in each SST in Level 2 and some or all SSTs in Level 3 can be first transferred to the memory, sorted in the memory, and then written to the SST of Level 3. And so on.

[0107] In this way, overall, the data stored in the upper SST is newer than that in the lower SST. The latest data is stored in memory, the next latest data is stored in Level 0, and the oldest data is stored in the SST in the lowest level. In addition, compaction can also be used to perform garbage collection, such as deleting data marked for deletion.

[0108] The sorted kv can be retrieved efficiently, for example, by binary search. Binary search is also called binary search, and the algorithm is as follows: define left as the leftmost element position of the sequence list, right as the rightmost element position of the sequence list, define mid = (left + right) / 2, that is, the middle position of the sequence list; compare the value being searched with the value at the mid position; since the list is ordered, if the value being searched is smaller than mid, then only the first half of the table needs to be searched, otherwise only the second half of the table needs to be searched (if the two values ​​are found to be equal in the first comparison, the position of the current value is directly returned). And so on, until the value being searched is found or it is determined that the value being searched is not in the list (that is, the search fails).

[0109] Such a storage engine has read amplification and write amplification problems when reading and writing data, which are described in detail below.

[0110] In the blockchain system using the above storage engine, when the execution module executes a transaction, it may be necessary to read an existing state. For example, for a general transfer transaction, when external account A transfers a certain amount of assets (for example, f) to external account B, it is necessary to increase or decrease the account balance. Assuming that before the execution of the transaction, the balance of account A is a and the balance of account B is b, then the execution of the transaction is to set the balance of account A to the value of (af) and the balance of account B to the value of (b+f). Then this transaction needs to read the values ​​of the two state variables a and b when it is executed. For another example, in a transaction that calls a contract, external account C initiates a call to contract D, then C initiates a transaction pointing to the address of contract account D, and specifies the called function and input parameters in the data field of the transaction. Assuming that the logic of the function called in contract D includes judging the value of a global state d in the contract, and assuming that the value of this d is a Boolean type, the judgment logic includes, if it is true, execute branch 1, and if it is false, execute branch 2. Then this transaction that calls the contract needs to read the value of the state d when it is executed.

[0111] During the transaction execution process, whether it is reading the values ​​of the state variables a and b as mentioned above, or reading the value of the state variable d, the execution module can first search whether the value of the state variable exists in the memory of the execution module. If not, a read instruction is sent to the storage module. The read instruction may include the address of the state to be read, that is, the key value of the state. After receiving the read instruction, the storage module finds the root node of the state tree node in the construction tree module, and matches the state key value to be searched with the shared nibble(s) field (for Extension Node) or slot (for Branch Node) of the root node from the beginning, and then reads the hash of the next layer of tree nodes (which can be Internal Node or Leaf Node, taking Internal Node as an example) from the matching position, and then first searches for the hash in the memory of the construction tree module. If it is not found, it searches in the memory of the state database module. If it is still not found, it needs to be searched in the file of the state database module. Specifically, you can search in the SST in Level 0 first. If you still can't find it, continue to search in a certain SST in Level 1. Continue to search in a certain SST in the lower level until you find the next Internal Node pointed to by the hash value. After unlocking this Internal Node, continue to match the remaining fields of the state address key value to be read from the front to the back. If it matches, read the hash value from the matching point and jump to the next level tree node pointed to by the hash value. Repeat this process, unlock the Internal Node level by level and match the remaining fields of the state address key value to be read from the front to the back. The hash value at the matching point is used as the basis for the next search for the intermediate node or leaf node until the Leaf node is matched, so as to read the value of the state from the Leaf node. In this process, some of the kvs of the tree nodes at each level may not exist in the memory of the tree construction module, but in the state database module. Therefore, it may be necessary to read from the state database module to the memory for operation. You can first search for this key in the MemTable or Immutable MemTable of the state database module. If it does not exist, you need to further search the SST files at all levels on the disk. For the Internal Node and Leaf node that exist on the disk, you need to load them into the memory and decode (deserialize) them to get the content. Specifically, for example, first search the sst files of level 0 one by one. If each SST of Level 0 is sorted, you can do a binary search in each SST based on the hash value of the intermediate node / leaf node. After finding it, load v into the memory and decode and match it.If the key is not found in the SST of Level 0, a binary search is performed in Level 1 based on the hash value of the intermediate node / leaf node. If the key is not found, a binary search is performed in Level 2. If the key is found, the v corresponding to the k is loaded into memory and decoded. This process is repeated until the leaf node of the tree structure is matched and the value is read from it. When searching in Level 1, Level 2, and lower levels, since the overall SST in these levels is sorted, a binary search can generally be performed in this level.

[0112] From the above process of reading a state kv, it can be seen that this reading process needs to perform binary search in SSTs of multiple different levels according to the hash value of the tree node. This may require reading a large number of files in SSTs of multiple levels and loading them into memory, which brings about the problem of read amplification. Assuming that the database includes Level 0, Level 1, Level 2, ..., Level M-1, a total of M layers, then in the worst case, reading the data of a state requires at least M accesses to the disk (greater than M because in the worst case, the N SSTs of Level 0 may require more than 1 and less than or equal to N searches).

[0113] In addition, if the tree building module lacks the kv of a certain tree node in the process of building the kv of all tree nodes on the path to the root node according to the state kv, it may also need to do a binary search from Level 0, Level 1, Level 2, ..., etc., and the principle is similar to the above process. In the above transaction example, for example, after the transaction is executed, the value of the execution result (af) / the value of (b+f) / the value of the contract state d needs to be written into the state database. For the above values ​​to be written to the database, the execution module calls the storage interface of the storage module and outputs the key-value value of the state to the storage module. The tree building module in the storage module generates the tree node kv from the root of the tree structure to the intermediate node and the leaf node according to the state kv, and also writes these tree node kvs into the WAL module, and then writes the kv of this series of tree nodes into the MemTable of the state database module and marks it as dirty data. Dirty data usually refers to data that has been written to the MemTable but has not yet been written to the persistent storage (such as SSTable). Dirty data can be written to the disk in the aforementioned flush process. The state database module then writes some kvs in MemTable, including the tree node kv, into the SST file on disk. Once MemTable is persisted to SSTable, the data in it is no longer dirty data, that is, it becomes "clean" data. It can be seen that the above process of persistently storing a state kv is relatively complicated, consumes more computing and memory resources, and brings about write amplification problems.

[0114] The complexity of the above-mentioned reading and writing processes and the problems of read amplification and write amplification pose challenges to the high performance requirements and large-scale applications of blockchain.

[0115] The present application provides a method embodiment for reading blockchain state data. The method embodiment can be applied to a storage module of a blockchain node. The storage module constructs a world state with a tree structure in a memory. The leaf nodes of the tree structure may include the content of the state data or the position index of the state data in a data file in a persistent storage medium.

[0116] In an example, the tree structure as a whole can be Fig.10 As shown, this is also consistent with Figure 5 In particular, if the leaf node in memory contains state data, then Figure 5 As shown, for example Figure 5 The leaf node A11 in the memory has state data, that is, Zhang San_A=20. If there is no state data in the leaf node in the memory, the position index of the state data in the data file can be used.

[0117] In this application, the actual data and index are separated, which helps to improve the flexibility and performance of the system. The basic concepts involved here include data files and index files. Among them, the data file is a file that stores the actual content; the index file is an index that stores the location of the data file. Its working principle is: write the data content to the data file, record the location of the data in the file (such as file name, offset and length); then, create an entry in the index file, including: a key (a unique identifier for search) pointing to a reference to the data file (file name, offset, etc.). In this way, when retrieving data, the data location can be quickly located through the index file, and then the actual content can be read from the data file according to the location information. The advantages of this method are: ① flexibility, data and indexes can be managed and optimized independently; ② performance, indexes are usually small and can be fully loaded into memory to speed up queries; ③ space efficiency: avoid storing a large amount of duplicate data in the index; ④ update efficiency: when modifying data, only the location information in the index needs to be updated.

[0118] For example Fig.10 In the example, location is used to represent the location index. Still taking the A11 node as an example, if there is no state data in the memory, the location field included in it can be used to store the location index of the data file where the state data is located. The location index is, for example, <2, 750, 210>. This <2, 750, 210> can indicate that the state data is located in "File 2" in the data file, the starting position is the 750KB of "File 2", and the length is 210KB. Fig.10 As shown in the data file in the lower left corner, assuming that the size of each file is 1024KB, the logical address range of the index is 0KB to 1023KB. In this way, the status data of A11 actually occupies the range from 750KB to 959KB in file 2.

[0119] like Fig. 9 As shown, this embodiment includes:

[0120] S910: The storage module of the blockchain node receives an instruction for the state key to be searched, and searches for a leaf node corresponding to the state key on the tree structure in the memory.

[0121] As mentioned above Figure 6 As shown in , the execution module can generate an execution result by executing a normal transfer transaction / a transaction involving a contract, and send it to the storage module. Specifically, for example, the value of a state variable needs to be read during the execution of a contract. The execution module can first search the memory of the execution module to see whether the value of the state variable exists. If not, a read instruction is sent to the storage module. The read instruction may include the address of the state to be read, that is, the key value of the state.

[0122] The storage module includes a tree construction module. The tree construction module constructs a tree structure with the world state in the memory. If the blockchain node is cold started, that is, there is no tree structure of the world state in the memory, then the tree nodes of the tree structure can be loaded from the disk and organized into a tree in the memory. This process also involves loading the leaf nodes in the tree structure. Here, an example is taken to illustrate the case where there is an incomplete tree structure in the tree construction module. The case of cold start loading is similar. Since the capacity of the memory is generally limited, the LRU (Least Recently Used, a commonly used page replacement algorithm, selects the page that has not been used for the longest time to be eliminated) algorithm is generally used to eliminate tree nodes that have not been accessed for a long time from the memory. In this way, each leaf node on the tree structure constructed in the memory by the tree construction module has a certain possibility of being eliminated. For the leaf nodes of the world state tree, some may be in the memory and some may not be in the memory. In addition, for the content in the tree node, such as the state data in the leaf node, a similar elimination mechanism can also be used, or the state data may not be kept in the memory.

[0123] For the state key to be found, if the memory of the tree building module has the leaf node corresponding to the state key, it can be found directly in the memory. In this case, you can refer to Figure 5 , directly find the key of the corresponding leaf node in the memory, and then read the value from it. Then, the state data can be read from the value. If the memory of the tree building module does not have the leaf node corresponding to the state key, or even if there is the leaf node, the state data to be found does not exist in it, then it is necessary to further search from the state database module. For the case where the state database uses an LSM tree, as mentioned above, you can first search in the MemTable, and if it is not found, search in the file on the disk.

[0124] S920: Obtain a position index contained in a leaf node corresponding to the tree node key, where the position index points to a position in a data file in the persistent storage medium.

[0125] If the leaf node to be searched is present in the memory of the tree building module, but the state data does not exist in it, it is necessary to further search from the state database module. As mentioned above, if there is no state data in the memory, the location field included in it can be used, and its content is, for example, Fig.10 The location of the leaf node A11 is <2, 750, 210>. This <2, 750, 210> is the location index, which indicates that the status data is located in file 2 of the data file, starting at the 750KB and with a length of 210KB. Fig.10 As shown in the data file in the lower left corner, assuming that the size of each file is 1024MB, the logical address range of the index is 0KB to 1023KB. In this way, the status data of A11 actually occupies the range from 750KB to 959KB in file 2.

[0126] If the leaf node to be found is not in the memory of the tree building module, it is necessary to find and load the leaf node from the state database module. Finding and loading the leaf node from the state database module specifically includes loading the index file and data file of the leaf node. Assume Fig.10 The corresponding underlying database stores the kv of all intermediate nodes in the tree structure, as shown in the following table:

[0127]

[0128]

[0129] Table 2. Tree node kv actually stored in the database

[0130] In Table 2 above, similar to Table 1, H() is used to represent hash calculation. In this way, the hash value of the next tree node is anchored in the previous tree node. Through such layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and the root hash is locked in the state root field of the block header. Assume that the kv pairs in this table are stored in the underlying state database module, saved on disk, and use a similar LSM structure. In this way, similar to the above, after the storage module receives the read instruction, it can find the root node of the state tree node in the construction tree module, and match the state key value to be searched with the shared nibble(s) field (for Extension Node) or slot (for Branch Node) of the root node from the beginning, read the hash of the next layer of tree nodes from the matching position, and then, preferentially search for the hash in the memory of the construction tree module. If it has not been found, search in the memory of the state database module. If it still has not been found, it needs to be searched in the file of the state database module. Specifically, you can search in the SST in Level 0 first. If you still can't find it, continue to search in a certain SST in Level 1. Continue to search in a certain SST in the lower level until you find the next Internal Node pointed to by the hash value. After unlocking this Internal Node, continue to match the remaining fields of the state address key value to be read from the front to the back. If it matches, read the hash value from the matching point and jump to the next level tree node pointed to by the hash value. Repeat this process, unlock the Internal Node level by level and match the remaining fields of the state address key value to be read from the front to the back. The hash value at the matching point is used as the basis for the next search for the intermediate node or leaf node until the Leaf node is matched, so as to read the value of the state from the Leaf node. In this process, some of the kvs of the tree nodes at each level may not exist in the memory of the tree construction module, but in the state database module. Therefore, it may be necessary to read from the state database module. You can first search for this key in the MemTable or Immutable MemTable of the state database module. If it does not exist, you need to further search the SST files at all levels on the disk. For the Internal Node and Leaf node that exist on the disk, you need to load them into the memory and decode (deserialize) them to get the content. Specifically, for example, first search the sst files of level 0 one by one. If each SST of Level 0 is sorted, you can do a binary search in each SST based on the hash value of the intermediate node / leaf node. After finding it, load v into the memory and decode and match it.If the key is not found in the SST of Level 0, a binary search is performed in Level 1 according to the hash value of the intermediate node / leaf node. If the key is not found, a binary search is performed in Level 2. If the key is found, the v corresponding to the key is loaded into the memory and decoded. This process is repeated until the key is matched to the leaf node of the tree structure and the value is read from it. When searching in Level 1, Level 2, and lower levels, since the overall SST in these levels is sorted, a binary search can generally be performed in this level.

[0131] In this way, the tree node kv of the found leaf node can be loaded into the memory. For example, A11 loads the content prefix:2,Key-end:35b2e4,location:(2,750,210) in the leaf node value into the memory. The <2,750,210> is the position index of the value content in the file.

[0132] S930: Load the value of the data file location in the persistent storage medium pointed to by the location index into the memory and read the status data therein.

[0133] Still taking A11 as an example, the position index <2,750,210> points to the position in the data file in the persistent storage medium. In this way, the value of the position of the data file in the persistent storage medium pointed to by the position index <2,750,210> can be loaded into the memory and the status data therein can be read, that is, the data of file 2 in the data file starting from the starting position 750KB and the length of 210KB is loaded into the memory, thereby reading the status data therein, that is, "Zhang San_A=20".

[0134] Through the position index, you can directly locate the position in the data file, so that you can directly load the data file and read the status data from it.

[0135] In the case where the leaf node to be searched is present in the memory of the tree building module but the state data does not exist, it can be directly read from the corresponding data file according to the location of the leaf node in the memory. In this way, the kv of the leaf node is not stored in the persistent storage SST file, and the binary search process in the SST is required multiple times.

[0136] In the case where the leaf node to be searched does not exist in the memory of the tree building module, it is necessary to search and load the leaf node from the state database module. In fact, it is also necessary to read a certain amount of content from the disk into the memory before searching. In the implementation method corresponding to Table 1, the kv of the tree node is stored in the disk, and the value of the leaf node includes the state data, for example, H(11) in Table 1 corresponds to Zhang San_A=20 in the content, H(12) corresponds to Li Si_B=20 in the content, H(15) corresponds to storedData=s in the content, and H(16) corresponds to Wang Wu_A=20 in the content. Generally speaking, the space occupied by state data is relatively large, accounting for more than 90% of the total space of a kv. In the implementation method corresponding to Table 2, the index and data are separated. The index stores the location of the data file where the status data is located, but not the status data itself. For example, in Table 2, H(11) corresponds to location (2,750,210) in the content, H(12) corresponds to location (3,350,210) in the content, H(15) corresponds to location (5,760,140) in the content, and H(16) corresponds to location (5,170,210) in the content. The actual status is stored in the corresponding position in the data file, such as Fig.10 As shown in the data files and status data in . In the process of searching for leaf nodes, only index files can be searched. In this way, more index files can be loaded into the memory within the unit space for searching. The search efficiency of memory is significantly higher than that of disk. Generally, the content to be searched will be loaded into the memory before searching. The implementation method of Table 2 can find leaf nodes faster than the implementation method of Table 1 because more indexes can be loaded into the memory space of unit capacity. However, in the implementation method of Table 1, for example, after reading a batch of kv into the memory, if the leaf node is not found, another batch of kv needs to be re-read into the memory for searching again. Moreover, due to the state content that occupies a large space, the number of kv in the memory space of unit capacity is relatively small, so the number of times to read from the disk will generally be more frequent, and the reading speed of the disk is much slower than that of the memory. Therefore, the search efficiency of the implementation method of Table 1 is low.

[0137] The above combination Fig.10 Explained Fig. 9 The specific process of the method embodiment. Fig.10 The structure of the MPT tree is still two levels, and the structure of the intermediate nodes of each level of the tree includes two types of tree nodes: Extension Node and Branch Node. In a variation, a tree node such as Branch Node can be used as an intermediate node and a root node, for example Fig.11As shown in . In this way, from the root node through the intermediate node to the leaf node, the sequential combination of the slots in the branch nodes passed through can constitute part of the content in the state key. The hash value of the next level node can be stored in the slot of the previous level node. Fig.11 In the example, a two-level tree structure can still be used. The leaf nodes in the upper tree can store account status (including the status of external accounts and contract accounts), and the leaf nodes in the lower tree can store the status of contracts. In the leaf nodes that store external account status, contract account status, or contract status, position indexes can be used. For example, the leaf nodes of the upper tree include account status in the following four rows: H(M1), H(M2), H(M3), and H(M4):

[0138]

[0139]

[0140] Table 3. Tree node kv actually stored in the database

[0141] Through the position index, you can directly locate the position in the data file, so that you can directly load the data file and read the status data from it. Furthermore, the contract status included in the leaf nodes of the next level tree is as shown in the four rows of H(Z1), H(Z2), H(Z3), and H(Z4) in the table above.

[0142] Data files such as Fig.11 As shown in , the content in the data file is as follows:

[0143] M1: Nonce: n1, balance: 45.0 ETH

[0144] M2: Nonce: n2, balance: 1.00WEI

[0145] M3: Nonce: n3, balance: 1.1ETH

[0146] M4:Nonce:n1,balance:0.12ETH,codehash:c1,storage:s1

[0147] Z1: Zhang San_A=20

[0148] Z2: Li Si_B = 50

[0149] Z3: Wang Wu_A=35

[0150] Z4: storedData = s

[0151] Whether it is the previous level tree or the next level tree, the location in the data file can be quickly located through the location in the leaf node. In this way, the kv of the leaf node is not stored in the persistent storage SST file, and the binary search process in the SST is required multiple times according to the key (hash value of the content) of the leaf node. Fig.11 In the example, according to Fig. 9 The blockchain state data can be read in the same way as in the previous step. According to the position index, the position in the data file can be directly located. Therefore, the content at the corresponding position in the data file can be directly loaded into the memory. This also avoids the read amplification problem caused by reading a large number of files in multiple levels of SST and loading them into the memory. Fig.10 The storage method shown in Fig.11 Stored in the same way.

[0152] For example, external accounts and contract accounts use fixed-length strings as keys, and the key of the contract status can be a string of the same length. Specifically, for example, the key of an external account or contract account can be an account address, while the key of the contract status can be a string of the same length. For another example, the key of an external account or contract account can be a hash value of an account address according to a certain hash algorithm, such as the sha3 algorithm; and the key of the contract status can be sha3 (contract name + location of the state variable declaration). The above-mentioned external accounts, contract accounts, and contract status are strings of the same length and format, collectively referred to as state keys. For example, the length of the state key is 4 bytes, or 32 bits. In this way, assuming that the root node (B0), intermediate nodes (..., C5, ..., C13, ..., C5D1, ..., C5D9, ..., C5D14, ..., C13D2, ..., C13D10, ...) are each a hexadecimal tree, then on the one hand, the slot of each intermediate node (such as 0, 1, 2, ..., d, e, f in B0, a total of 16 slots) can be filled with the hash value of the next tree node, and on the other hand, the position of the slot of this root node / intermediate node can represent a 4-bit key. 4 bits constitute a half-byte, so a total of 8 layers of root nodes and intermediate nodes can be loaded with 32 bits, a total of 8 bytes, that is, all state keys. The 32 bits of the state key can be stored in one half-byte of the key from the intermediate node at the top level to the intermediate node at the bottom level to the leaf node. In this way, for the case where the intermediate nodes are fixedly expanded, the last 4 bits of the key can be stored in the leaf node. In the case where the intermediate nodes are fixedly expanded, Fig.11In the example, there are 8 layers from C* to C13D*, all the way to the bottom layer M*. In another case, the prefix and / or infix and / or suffix compression method can also be adopted, so that one or some leaf nodes plus the intermediate nodes above can be less than 8 layers.

[0153] In addition, a tree structure can also be used to store account status and contract status, for example Fig.12 As shown in . Since the key of the contract status is a hash value, based on the discrete characteristics of the hash algorithm, the status key of different contract status may be the path from the root node to different leaf nodes in this layer of tree structure. In this way, the value of different states of the same contract may exist in different leaf nodes at the bottom of the tree structure, and will no longer be gathered under the contract account to which it belongs. In fact, with a one-layer tree structure, the account status and contract status are scattered in the leaf nodes at the bottom.

[0154] In addition, Fig.12 On the basis of Fig.13 In the case of infix compression, for example, the tree node key of M1 is 0x52d1479e (0x indicates that each bit is represented by a hexadecimal number), and the tree node key of Z1 is 0x52d9ab36, then M1 and Z1 have a common infix 2d. In addition to the 1 and 9 separated by the hexadecimal tree, the tree node C5 can also have a compressed infix 2d, such as Fig.13 In the case of suffix compression, for example, key-end is used to represent the key-end in the tree node M1, and key-end in the tree node Z1 is 36. By using prefix and / or infix compression, the number of tree layers can be reduced.

[0155] Regardless of whether it is compressed or not, for the state key being searched, you can start from the root node of the tree structure and match the path of the state key segment by segment until you find the leaf node of the tree structure corresponding to the state key. In one way, the root node and the intermediate node are stored in a key-value format, where the key is the hash value of the value, similar to the structure in Table 3. In this way, you can match the hash of the path of the state key in order starting from the previous tree node, and search for the next tree node based on the hash until you find the leaf node of the tree structure corresponding to the state key.

[0156] In addition, the key of the tree node kv actually stored in the database may not be the hash value of the value, but a partial value on the state key path, such as the value of a partial key from the root node to the current tree node (combined with Fig.12, without compressed prefixes / infixes, as shown in Table 4), or the value of a partial key after the previous tree node (combined with Fig.13 , with compressed prefixes / infixes, as shown in Table 5).

[0157]

[0158]

[0159] Table 4. Tree node kv actually stored in the database

[0160] According to Table 4, combined Fig.12 For example, if the state key to be searched is 0x59...36 (leaf node Z1), starting from the root node of the tree structure, the tree node with key 0x5 in the database can be matched according to the first digit 5 ​​from the left of the state key (h(C5) in the value of the tree node with key 0x5 can be used to verify with the hash value on slot 5 in the root node. If they are the same, it means that the tree node 0x5 is the child node in slot 5 of the 0x tree node); further, the tree node with key 0x9 in the database can be matched according to the second digit 9 from the left of the state key (h(C5D9) in the value of the tree node with key 0x9 can be used to verify with the hash value on slot 9 in the parent node. If they are the same, it means that the tree node 0x9 is the child node of the 0x5 tree node); and so on. Finally, the last digit 6 of the state key can be matched to the tree node with key 0x6 in the database (h(Z1) in the value of the tree node with key 0x6 can be used to verify with the hash value on slot 6 in the parent node. If they are the same, it means that the tree node 0x6 is the child node of the parent node).

[0161]

[0162]

[0163] Table 5. Tree node kv actually stored in the database

[0164] According to Table 5, for example, if the state key to be searched is 0x52d9...36 (leaf node Z1), starting from the root node of the tree structure, the first bit 5 from the left of the state key can be matched in the database. If the same key is not matched, or the hash value in the value of the tree node that matches the key is inconsistent with h(C5) in the root node, then the state key is matched again in the database according to the first-second bits 52 from the left, and the cycle continues until a match is found. When the tree node with the key 0x52d is matched in the database according to the first-third bits 52d from the left of the state key, and the H(C5) in its value is the same as the hash value on slot 5 in the root node, it means that a match is found. Then, we can match the database according to the 1st to 4th bits 52d9 from the left of the state key. If the same key is not matched, or the hash value in the value of the tree node of the matched key is inconsistent with h(C5) in the root node, we can match the database according to the 1st to 2nd bits 52 from the left of the state key again. If a match is found, and H(C5D9) in the matched tree node value is the same as the hash value on slot 9 in the parent node, it means that the match is found. Furthermore, we match the database according to the 1st to 5th bits from the left of the state key, match the database according to the 1st to 6th bits from the left of the state key, and so on, until a leaf node is matched.

[0165] In another implementation, the complete state key or the corresponding complete state key can be stored in the leaf node. For example, the complete state key is stored in the leaf node. Fig.14 As shown in , the key-full field of the leaf node can store the complete key of the corresponding state. The key-value of the leaf node stored in the database is the complete key of the corresponding state of the leaf node. In this way, when searching for a state key, you can directly match the key of the leaf node in the database according to the key without matching the key of the intermediate node. Obviously, this search method is extremely efficient for query operations. In fact, the above Figure 10-13 In the example above, the complete state key can also be used as the key of the leaf node.

[0166] Above Figure 9-14In the embodiment, the location index location is stored in the leaf node, and the position of the value of the leaf node in the data file can be quickly located through the location index after the leaf node is found. In this way, the content of the corresponding position in the located data file can be directly loaded into the memory, which also avoids the read amplification problem caused by reading a large number of files in multiple Levels of SST and loading them into the memory. This also avoids the kv of the leaf node being stored in the persistently stored SST file, and the process of performing binary searches in the SST multiple times according to the key (hash value of the value content) of the leaf node. Moreover, similarly, for the leaf node to be found that is not in the memory of the tree construction module, the above embodiment can only search the index file, so that more index files can be loaded into the memory within the unit space for search, and the leaf node can be found faster.

[0167] As mentioned above, the state root in the block header stores the hash value of the root of the underlying MPT tree to which it is anchored. This hash value is actually obtained by calculating the hash from the leaf node layer by layer, as described above. Generally, the storage module of the blockchain node will build the world state of the tree structure in memory through the tree construction module. Specifically, in addition to the above Figures 12-14 In addition to the leaf nodes, it also includes intermediate nodes and root nodes, as shown in the above Tables 2 to 5.

[0168] Moreover, the chain data structure of the blockchain is realized by anchoring the hash of the next block to the PrevHash of the previous block, such as Figure 3 , 4 As shown in , there are blocks identified by different block numbers on the blockchain. These block numbers are also called versions. Under different versions of blocks, the corresponding tree structure may change.

[0169] People in this field know that the essence of blockchain is a chain structure composed of a series of data blocks (blocks) linked by hashes. Each block has a world state corresponding to the block, but each block does not necessarily contain a complete world state, but in most cases contains a reference to the previous world state. The following explains this:

[0170] In the block structure of Ethereum, when the value in a key-value changes, the content of the leaf node of the corresponding MPT tree changes. Accordingly, the relevant hash values ​​stored in a series of intermediate nodes and the root node above the leaf node will also change. In the structure shown in Table 1, the changed leaf nodes, intermediate nodes, and root nodes will also be added to the end of Table 1 in an appended manner. At the same time, the values ​​of more state variables in a contract storage may not change.

[0171] like Fig.15 The Ethereum block structure diagram shown in Fig.15 and Figure 3 The difference is that the references of the root nodes and intermediate nodes of the two-level Merkle tree under the State Root in block N+1 to the state variables that have not changed in block N are added. For example, in the contract storage of a contract, the value of the same state variable is changed from 29 in block N to 45 in block N+1. Assuming that other state variables have not changed, in block N+1, except for the leaf node that stores the value changed to 45 and the intermediate nodes and intermediate nodes related to it, the other leaf nodes and intermediate nodes that have not changed in block N+1 directly reference the same leaf nodes and intermediate nodes in block N. Two adjacent blocks N and N+1 are shown here. It can be understood that for the block structure of multiple consecutive blocks on the blockchain, the leaf nodes and intermediate nodes in block N+1 that have not changed relative to the previous blocks all reference the same leaf nodes and intermediate nodes that were last updated to the value in block N+1. The same leaf node and intermediate node that were last updated to the value, for example, the same leaf node and related intermediate node that were updated to the value in block N-1, the same leaf node and related intermediate node that were updated to the value in block 5, and so on. The different blocks here can also be called different versions, and the block number can be the version number. The reference here is achieved by recording the hash value of the lower node in the upper node. The benefit of this is obvious, that is, the leaf nodes and intermediate nodes that have not changed will not be repeatedly stored in the underlying database.

[0172] Here, we still use the change of the state in the same contract storage in version N+1 relative to version N as a simple explanation. For example, if the content of a tree node changes, its hash will change accordingly. MPT uses the hash as the index in the database, which means that for each value, there is a certain record in the database. MPT associates parent and child nodes based on the node hash. Therefore, whenever the content of a node changes, ultimately for the parent node, only a hash index value changes. The content of the parent node also changes, creating a new higher-level parent node, and recursively passing this impact to the root node. Ultimately, a change creates a new path from the modified node to the root node, and the old node can still be accessed through the old path based on the old root node.

[0173] like Fig.15 As shown in the figure, when the content of a node in MPT changes from 29 in version N to 45 in version N+1, assuming that the other states in the contract state have not changed in the previous and next two versions, the changed 45 in the world state of version N+1 will create a new path to the root node (Storage root) of this contract. In addition, the other leaf nodes and intermediate nodes that have not changed reuse the leaf nodes and / or intermediate nodes in the previous version (i.e. the version with block number N) through hash pointers, thereby constructing a new MPT tree for the storage of the contract in version N+1, while still retaining its old path. Fig.18 In the same contract storage, the leaf nodes / intermediate nodes whose values ​​in version N+1 have not changed relative to version N include V, U, T, S, etc. Therefore, in the storage of the contract in version N+1, in addition to creating a new path from 45 to the Storage root, the Storage root of version N+1 is indexed to V, U, T, S in version N through hash, but does not include the leaf node with a value of 29 and related intermediate nodes. Assuming that the contract is deployed in version N, CodeHash is similar. The contract code corresponding to CodeHash exists in the state database (StateDB) corresponding to version N, and CodeHash of version N+1 also points to the contract code in version N.

[0174] In addition to contract storage, other account storage (including storage of external accounts and contract accounts) can also adopt similar methods, such as Fig.15For the account storage that has not changed in version N, the State Root in the block header of N+1 points to the storage of external accounts or contract accounts such as L, M, N, P, Q, and R in version N. Only the leaf nodes and related intermediate nodes that have changed are stored in the account storage corresponding to version N+1, which will not be described one by one.

[0175] All states under the current block, in fact, many of them are likely to have been updated in a previous block and have not changed until the current block. This is similar to Ethereum, which is also called current state and is stored in currentDB. CurrentDB is used to store the latest "current state" data, such as the latest account status, contract code, and contract storage. Moreover, as the block grows, the data in currentDB can be continuously modified.

[0176] As mentioned above Figure 3 and above Fig.15 The content of the blockchain is very redundant. It is an efficient way to store data by referencing the same leaf node and intermediate node that were last updated to the same value in the previous block. Moreover, the state changed in each block is likely to be part of the global state. This is also similar to Ethereum, also known as history state, which is stored in historyDB. historyDB is used to store historical state data, thereby providing a complete historical view of the entire chain.

[0177] The present application provides a method for constructing a world state based on a tree structure, comprising:

[0178] S170: Receive status key-value.

[0179] S172: Generate leaf nodes of the tree structure corresponding to the state key-value according to the construction rules of the tree structure; calculate the hash value of the leaf node according to the content in the leaf node, and generate or update the upper tree node according to the hash value of the leaf node and the state key, until the root node of the tree.

[0180] The leaf node may be in the form of the aforementioned data file or the aforementioned KVDB. For the aforementioned data file, the leaf node may generate a hash value including the position index of the state value in the data file in the persistent storage medium and the state value content.

[0181] S174: Generate corresponding memory pages for the generated / updated tree nodes according to the divided logical page structure, and the memory pages are used to represent the contents of all corresponding tree nodes in the logical page.

[0182] In the present application, the entire Merkle tree structure can be divided into multiple LogicalPages (logical pages). According to the node association relationship of the tree structure, several upper and lower adjacent tree nodes can be aggregated into one LogicalPage, such as a 2-level 16-branch dictionary tree with a total of 256 child nodes aggregated into one LogicalPage. Among them, there are multiple layers of LogicalPages from the root node to the leaf node of the Merkle tree, and there are parent-child and brother relationships between LogicalPages, thus forming a complete Merkle tree. In this way, the tree structure can be divided into multiple logical pages, and a logical page contains at least one tree node. Different logical pages contain different tree nodes, and the logical page maintains the content of the corresponding tree node of the current latest version.

[0183] Inside each LogicalPage, a memory page (MemoryPage) can be maintained to represent the contents of all tree nodes corresponding to the logical page of the latest version of the logical page. Furthermore, the contents of the tree nodes maintained by the logical page can be generated into base pages and delta pages according to the generation / modification behavior based on the continuous version update. Specifically, BasePage (base page) and DeltaPage (delta page) are generated according to the generation / modification behavior. For example, for continuous state changes, such as the state variables a=5, b=8, c=3 included in version N, a=5, b=8, c=3 can be used as the contents of the base page BasePage. For a=6 in version N+1 and a=8 in version N+2, assuming that b and c have not changed, a=6 in version N+1 and a=8 in version N+2 can be used as DeltaPages based on this BasePage, such as DeltaPage1, and the case where b and c have not changed is not included in DeltaPage1. For a=11 in version N+3 and a=15 in version N+4, a=11 in version N+3 and a=15 in version N+4 can be used as DeltaPages based on this BasePage, such as DeltaPage2. Similarly, assuming that b and c have not changed, b and c are not included in DeltaPage2, and so on. Among them, BasePage and DeltaPage can be used as the smallest unit of the tree's memory structure. In addition, BasePage can be used as the smallest unit of the tree's disk persistence. On this basis, DeltaPage can also be used as the smallest unit of the tree's disk persistence.

[0184] It should be noted that the BasePage here is a checkpoint of the entire LogicalPage content, which can contain the full content of the LogicalPage at that time, such as all the tree node information contained. Checkpoint is generally a snapshot of a specific state saved in a computer system, so that it can be restored from this saved state and continue execution when the system fails or fails. In many cases, checkpoints are used for long-running computing tasks, safety-critical systems, distributed systems, and database systems.

[0185] For BasePage, a BasePage can be generated every predetermined number of state modifications (such as m times), such as Fig.16 As shown in the basepage2 on the right side of the lower right red box. As in the above example, in version N, the state variable a=5 can be used as the content of the base page BasePage1. For version N+1, a=6, and version N+2, a=8, a=6 in version N+1 and a=8 in version N+2 are used as DeltaPage1 based on this BasePage. For version N+3, a=11, and version N+4, a=15, a=11 in version N+3 and a=15 in version N+4 are used as DeltaPage2 based on this BasePage. For version N+5, a=20, assuming that the modification reaches the predetermined number of times 5, the value a=20 in version N+5, b=8, and c=3 can be used as the content of this BasePage2.

[0186] Similarly, DeltaPage describes several version modifications of LogicalPage, and aggregates multiple modifications into a set, such as generating a DeltaPage every M modifications, and creating a new DeltaPage to collect the subsequent version modification operations. As in the above example, for a=6 in version N+1 and a=8 in version N+2, a=6 in version N+1 and a=8 in version N+2 are used as DeltaPage1 based on this BasePage. For a=11 in version N+3 and a=15 in version N+4, a=11 in version N+3 and a=15 in version N+4 can be used as DeltaPage2 based on this BasePage. It can be seen that each DeltaPage is set to create an incremental page for 2 version modification operations.

[0187] For each modification of BasePage and DeltaPage, corresponding dirty data is generated in the memory. When the memory occupied by BasePage and DeltaPage reaches a predetermined amount, BasePage and DeltaPage can be batch-persisted to disk, thereby avoiding persisting all dirty data for each modification and continuously occupying memory and CPU resources. The memory generation and disk persistence of BasePage and DeltaPage can be performed in the background.

[0188] When the storage module persists the world state of the tree structure built in the memory through the structure of the logical page, it can cache the state data according to the predetermined strategy based on the hot and cold characteristics of the state data, such as the (Least Recently Used, LRU) strategy, with the logical page as the granularity. In this way, if the MemoryPage is eliminated, the corresponding BasePage and DeltaPage are read to replay the MemoryPage.

[0189] The above scheme provided by this application describes a scheme for dividing the world state of a tree structure according to Page segmentation, dividing the tree structure into LogicalPage, and managing MemoryPage / BasePage / DeltaPage internally in LogicalPage, optimizing each state tree modification into multiple version modifications at the granularity of LogicalPage, and reducing the overall write amplification. For the current version, it can be directly obtained through the memory page, and for the historical version, the BasePage and several associated DeltaPages can be read to replay. The entire MemoryPage is cached in memory without affecting the contract execution performance.

[0190] In the above scheme, a LogicalPage internally manages MemoryPage / BasePage / DeltaPage, and as mentioned above, several upper and lower adjacent tree nodes can be aggregated into a LogicalPage according to the node association relationship of the tree structure, such as aggregating 256 child nodes in a 2-level 16-branch dictionary tree into a LogicalPage. Among them, there are multiple layers of LogicalPages from the root node to the leaf node of the Merkle tree, and there are parent-child and brother relationships between LogicalPages, thus forming a complete Merkle tree. In this way, the tree structure can be divided into multiple logical pages, and a logical page contains at least one tree node. Different logical pages contain different tree nodes, and the logical page maintains the content of the corresponding tree node of the current latest version.

[0191] Then, when constructing a world state based on a tree structure, in addition to generating a leaf node of the tree according to the state key-value, changes will also occur from the generated leaf node to the intermediate node to the root node, such as the hash value described in S172 above. Moreover, the key of the state also needs to be designed, especially for the LogicalPage and the corresponding MemoryPage / BasePage / DeltaPage.

[0192] Therefore, this application also proposes the following method for constructing a world state based on a tree structure, including:

[0193] S180: Receive status key-value.

[0194] S182: Generate leaf nodes of the tree structure corresponding to the state key-value according to the construction rules of the tree structure; calculate the hash value of the leaf node according to the content in the leaf node, and generate or update the upper tree node according to the hash value of the leaf node and the state key, until the root node of the tree.

[0195] The leaf node may be in the form of the aforementioned data file or the aforementioned KVDB. For the aforementioned data file, the leaf node may generate a hash value including the position index of the state value in the data file in the persistent storage medium and the state value content.

[0196] The process of S182 can be specifically referred to in the above content, which is briefly described again as follows:

[0197] Like Ethereum, when the storage module receives a key-value pair (kv) request to write the state, the key (k) will first be hashed (such as Keccak-256). In addition, the state key in S182 here can also be the state key received in S180, that is, it can also be not hashed. Next, use this key to search in the MPT. This search process starts from the root node of the MPT and traverses and matches down along the path of the tree in the lexicographic order of the key. In this process, the key is generally compared bit by bit (or half a byte by half a byte, also known as a nibble, generally a hexadecimal number of one bit) with the path stored in the node. During the traversal and matching process, three types of nodes may be encountered: branch nodes (if the path forks at a certain point), extension nodes (if there is a shared prefix), and leaf nodes (storing the final value). Branch nodes are used to fork in the tree, extension nodes are used to optimize storage (through path compression), and leaf nodes store actual values. If a complete lexicographic path for a matching key is found to exist during the search process, a leaf node will be reached. At this point, the value (v) stored in this leaf node can be updated. If the search and matching process finds that only part of the path is matched, it may be necessary to create a new node (which may be a branch node, an extension node, or a leaf node) to complete the remaining path and eventually reach the leaf node, and store or update the value in the leaf node.

[0198] Once a leaf node is updated or created, a backtracking process begins. This process starts with the modified leaf node and recalculates the hash value of the node. Then, it moves up to the parent node, updates the relevant reference in the parent node (this reference may be the direct content of the child node, or the hash value of the child node content, depending on the size of the child node), and recalculates the hash value of the parent node. This process continues until the root node of the MPT is reached.

[0199] Eventually, when the backtracking process reaches the root node, the root hash value of the entire MPT will be updated. This new root hash value represents the new state of the entire state tree, which will be included in the block header of the next block as the new state root. In this way, each block not only contains transaction information, but also saves the hash value of the global state after executing these transactions through the state root.

[0200] S184: Update the version of the generated / updated tree node in the corresponding memory page according to the divided logical page structure, the memory page is used to represent the content of all corresponding tree nodes in the logical page, the logical page contains a page identifier, and the page identifier includes the lexicographic order content and version of the upper tree node in the logical page.

[0201] It should be noted that the process of S184 can occur after S182 or during S182.

[0202] Fig.18 It is a complete diagram of another embodiment of the present application. Fig.18 and Fig.13 The overall structure is similar, with the main difference being that, for example, the intermediate node labeled C5 and the intermediate nodes labeled C5D1, C5D9, and C5D14 below it together form a LogicalPage, and in the leaf nodes M1, M2, M3, and M4, key-end is stored instead of key-full, and in addition to the position index contained in the leaf node, it also contains the hash value of the corresponding state value content. Of course, in this embodiment, as mentioned above, the leaf nodes M1, M2, M3, and M4 can also be key-full.

[0203] The leaf node may also include a hash value (valuehash) of the state value content and a key-end field, wherein the hash value of the leaf node is calculated based on the content in the leaf node, including the calculation based on the hash value of the state value content in the leaf node and the key-end field, but does not include the location index of the state value in the data file in the persistent storage medium.

[0204] The leaf node may also include a hash value and a key-full field of the state value content, wherein the hash value of the leaf node is calculated based on the content in the leaf node, including the calculation based on the hash value of the state value content in the leaf node and the key-full field, but does not include the position index of the state value in the data file in the persistent storage medium. Fig.18 The valuehash in the leaf node is omitted. And, Fig.18 The data structure of the tree node is used instead of Fig.14 A graphical description of .

[0205] In the aforementioned S182, a tree node is generated / updated. The updated tree node is integrated into the corresponding memory page according to the aforementioned logical page structure, indicating the world state tree structure under the current block number. In this application, a version number can be set for the logical page. The version number can typically be a block number. In addition to the block number, the version number can also be data such as a block hash, timestamp, state root, transaction root, receipt root, consensus round number, etc. that can uniquely identify a block. For simplicity, the following is explained in terms of block number.

[0206] The version number of the logical page can be used as the content in the page identification of the logical page. In addition to the version number, the page identification of the logical page can also include the lexicographical content of the tree nodes included. Specifically, the page identification can include the lexicographical content of the upper tree nodes in the logical page. Fig.18 As shown in , in a LogicalPage composed of the intermediate node labeled C5 and the intermediate nodes labeled C5D1, C5D9, and C5D14 below it, the upper tree node has a node ID (NodeID), for example, 0xd2345 (0x represents hexadecimal), and the intermediate nodes C5D1, C5D9, and C5D14 below it also have a NodeID. The PageID of the LogicalPage can be set to the NodeID of the upper tree node. These page IDs are the lexicographic content from the root node to the upper tree node in the LogicalPage (that is, the memory page), that is, the key of the lexicographic content distribution from the root node through the intermediate node to the upper tree node in the LogicalPage. Similarly, NodeID is the lexicographic content from the root node to the tree node, that is, the key of the lexicographic content distribution from the root node through the intermediate node to the tree node.

[0207] As described above, in the process of generating / updating the current version of the tree nodes, the changed tree nodes constitute dirty data. And as mentioned above, the base page and incremental page can be generated based on the changes in the logical pages of the continuous version. On this basis, the base page can be used as the minimum unit of disk persistence of the tree, and the incremental page can also be used as the minimum unit of disk persistence of the tree. A base page can be generated once for every predetermined number of state modifications, and an incremental page can be generated once for every predetermined number of state modifications. When the base page and incremental page occupy a certain amount of memory, the base page and incremental page are batched for disk persistence.

[0208] The base page and incremental page are persistent objects of the memory page corresponding to the logical page, and the memory page is the memorized object of the logical page. The base page generally corresponds to the memory page, contains global state variables, and can have the same page ID as the corresponding memory page, and the memory page has the same page ID as the corresponding logical page. The incremental page can also correspond to the memory page, containing tree nodes in one or more consecutive versions of the memory page that have changed relative to the previous adjacent version of the base page / incremental page, which also contains the changed state variables, but generally not the global state variables, and the structure of the tree node in the incremental page itself generally does not need to change, so the page ID of the corresponding memory page can be the same as the NodeID of the upper tree node.

[0209] In addition, the base page and the incremental page may have versions. After the above-mentioned S184 operation, the base page and the incremental page are generated based on the changes of the continuous version of the memory page, and the corresponding version of the base page and / or the incremental page may be generated based on the changes of the continuous version of the memory page. Specifically, the base page generally corresponds to a memory page, including all tree nodes in the corresponding memory page, and also including global state variables, and therefore may have the same version as the corresponding memory page. The incremental page may correspond to one or more memory pages, including tree nodes in one or more continuous versions of memory pages that have changed relative to the previous adjacent version of the base page / incremental page, and also including changed state variables, which are generally not global state variables, so the version of an incremental page may be the lowest or highest version in the one or more corresponding memory pages.

[0210] For example, a base page with version 4 contains state variables a=1, b=2, c=3 and corresponding intermediate nodes and root nodes, and the subsequent incremental page may include a=2 with version 5 and corresponding intermediate nodes and root nodes, a=1 with version 6 and corresponding intermediate nodes and root nodes, and b=4 with version 7 and corresponding intermediate nodes and root nodes. Among them, although a=1 with version 6 is the same as the state variable a=1 contained in the base page, it is different from the previous adjacent a=2 with version 5, so a=1 with version 6 and corresponding intermediate nodes and root nodes are also included in the incremental page. In addition, for the subsequent incremental page, such as the incremental page containing tree nodes with versions 7, 8 and 9, it is actually in order relative to the previous adjacent incremental page and version of the tree node. The tree nodes with versions 5, 6, and 7 correspond to the memory pages with versions 5, 6, and 7 respectively. The incremental page containing tree nodes with versions 5, 6, and 7 can have a version of 5 or 7, that is, it can be the lowest or highest version among the corresponding multiple memory pages.

[0211] Furthermore, a page type may be set in the base page and the incremental page to distinguish between the base page and the incremental page.

[0212] The above scheme is a multi-version Page segmentation scheme for the Merkle state data dictionary tree. The tree is switched to LogicalPage. Both the tree and LogicalPage maintain the version (such as the block number). MemoryPage / BasePage / DeltaPage are managed internally. BasePage / DeltaPage maintains the memory structure and file persistence by version. By reducing the overall write amplification through page aggregation, the version ordering feature is utilized to avoid compaction after the page is dropped to the disk, which can reduce the write amplification of the page being dropped to the disk. At the same time, through version-based replay, it supports the reading of state data of any historical version and SPV functions. In addition, combined with the aforementioned Figures 9 to 14 The state value can be stored in a data file of a persistent storage medium, and the leaf node includes a position index of the state value in the data file of the persistent storage medium, thereby further reducing the problem of write amplification.

[0213] Corresponding to the world state constructed in the above manner, the present application also provides a reading method, including:

[0214] S190: Receive the read status key and version.

[0215] S192: According to the state key and version, match the version and page ID in the page identifier of the memory page in lexicographic order. After matching, if the memory page includes at least two layers of tree nodes, continue to match the version and page ID of the tree nodes below the top tree node in lexicographic order until a leaf node is found.

[0216] For the received state key and version, the version and page ID in the page identifier of the memory page can be matched in lexicographic order. For example, for Fig.18 The intermediate node labeled C5 and the intermediate nodes labeled C5D1, C5D9, and C5D14 below it together constitute a LogicalPage. The corresponding memory page can be found first according to the version and page ID, and then the received state key can be used to further match the lower tree nodes in the memory page.

[0217] For example, for the received status key 0xd2345167890ab and version 20, the version + page ID of a memory page is 20 + 0xd2345, then Fig.18 The middle node labeled C5 and the middle nodes labeled C5D1, C5D9, and C5D14 below it together form a memory page composed of a LogicalPage, which can be matched to (the first 5 bits of 0xd2345167890ab, i.e., 0xd2345). Furthermore, in the lower tree nodes in the memory page, assuming that the tree node C5D1 is matched, that is, the NodeID in the tree node C5D1 is matched to 0xd23451; further, the memory page below the memory page can be matched. Fig.19As shown, the lower memory page is matched. According to 0xd2345167890ab, the memory page with PageID 0xd234516 and the same version is matched. The memory page includes upper and lower tree nodes E5 and E5F2, E5F7, and E5F8. Furthermore, in the lower tree node of the memory page, it is assumed that the tree node E5F7 is matched, that is, the NodeID in the tree node E5F7 is matched to 0xd2345167. This goes back and forth, matching the version and page ID in the page identifier of the memory page in lexicographic order until the last memory page and the leaf node therein are found, and the search is completed.

[0218] As mentioned above, a memory page is a memorized object of a logical page. However, based on the cache elimination strategy in the memory, there may be no currently matching memory page. At the same time, as mentioned above, base pages and incremental pages can be generated based on the changes in the continuous versions of the logical page, and the base pages and incremental pages are persistent objects of the memory pages corresponding to the logical pages. Therefore, in the case where the memory page does not exist in the memory, the corresponding base page and incremental page can also be loaded from the persistent storage medium to the memory according to the page identifier, and the loaded base page and incremental page can be played back to obtain the memory page.

[0219] Specifically, for example, if no memory page with the same version and matching the remaining lexicographic content is found in the memory, the corresponding base page and incremental page matching the remaining lexicographic content can be found in the persistent storage medium according to the page identifier. As mentioned above, since base pages and incremental pages also have versions, and base pages generally have the same version as the corresponding memory pages, and the version of an incremental page can be the lowest version among one or more corresponding memory pages, there are the following two situations:

[0220] 1. In the case where a base page with the same version is found in the persistent storage medium, the corresponding base page can be loaded from the persistent storage medium to the memory according to the page identifier, and then the loaded base page can be restored to the memory page.

[0221] 2. In the case where the base page with the same version is not found in the persistent storage medium, the corresponding base page and incremental page are loaded from the persistent storage medium to the memory according to the page identifier; the loaded base page and incremental page are replayed to obtain the memory page.

[0222] The page identifiers of the base page and the incremental page may also include page types.

[0223] In this way, the loading of the corresponding base page from the persistent storage medium to the memory according to the page identifier can be loading a base page of the maximum version not greater than the read version, with the same page ID and the page type of the base page from the persistent storage medium to the memory. For the above situation 1, that is, the situation where the base page with the same version is found in the persistent storage medium, the base page with the same version, the same page ID and the page type of the base page can be loaded into the memory. For example, the version to be searched is 20, and there is no memory page with version 20 in the memory, and the base page with the same version 20 is found in the persistent storage medium, and the base page with the same version 20, the same page ID and the page type of the base page can be loaded into the memory. For the above situation 2, that is, the situation where the base page with the same version is not found in the persistent storage medium, the base page of the maximum version not greater than the read version, the same page ID and the page type of the base page can be loaded from the persistent storage medium to the memory. For example, the version to be found is 20, but there is no memory page with version 20 in the memory. Of course, the base page with version 20 is not found in the persistent storage medium. In fact, there are BasePage1 with version 17, DeltaPage1 with version 18, DeltaPage2 with version 20, and BasePage2 with version 22 in the persistent storage medium. DeltaPage1 with version 18 can include tree nodes with version 18 and tree nodes with version 19, and DeltaPage2 with version 20 can include tree nodes with version 20 and tree nodes with version 21. Then, BasePage1 with a version not greater than 20, a consistent page ID, and a page type of the base page can be loaded into the memory. That is, the version of the base page that meets the conditions here is 17.

[0224] For the above situation 1, that is, the situation where the base page with the same version is found in the persistent storage medium, the base page with the same version, the same page ID and the page type of the base page can be loaded into the memory, and then the memory page with the same version can be restored. For example, if the version to be found is 20, and there is no memory page with version 20 in the memory, and the base page with the same version 20 is found in the persistent storage medium, the base page with the same version 20, the same page ID and the page type of the base page can be loaded into the memory, and then the loaded base page with version 20 can be restored to the memory page with version 20.

[0225] For the above situation 2, that is, the situation where the base page with the same version as the version is not found in the persistent storage medium, a base page with a maximum version not greater than the read version, a consistent page ID and a page type of the base page can be loaded from the persistent storage medium to the memory, and then the loaded base page and incremental page can be replayed to obtain the memory page. The loaded base page and incremental page can be replayed to obtain the memory page, specifically, a base page with a maximum version not greater than the read version, a consistent page ID and a page type of the base page can be loaded from the persistent storage medium to the memory, and then an incremental page with a minimum version not less than the read version, a consistent page ID and a page type of the incremental page can be loaded from the persistent storage medium to the memory. For example, if the version to be searched is 20, but there is no memory page with version 20 in the memory, then of course there is no base page with version 20 in the persistent storage medium. In fact, there are BasePage1 with version 17, DeltaPage1 with version 18, DeltaPage2 with version 20, and BasePage2 with version 22 in the persistent storage medium. DeltaPage1 with version 18 can include tree nodes with version 18 and tree nodes with version 19, and DeltaPage2 with version 20 can include For tree nodes with version 20 and tree nodes with version 21, BasePage1 with a version not greater than 20, consistent page ID and page type of the base page with a maximum version of 17 can be loaded into memory, that is, the version of the eligible base page here is 17, and then DeltaPage1 with version 17 and later and a minimum version 20 not less than the read version 20, consistent page ID and page type of the incremental page with version 18 and DeltaPage2 with version 20 are loaded into memory, and then the loaded base page and incremental page are played back to obtain the memory page.

[0226] For the above situation 2, specifically, the loaded base page and incremental page can be replayed in the order of page type and version to obtain the memory page. Still according to the above example, the loaded base page is BasePage1 with version 17, and the loaded incremental pages include DeltaPage1 with version 18 and DeltaPage2 with version 20. Then, the incremental pages can be replayed in the order of versions, that is, based on the BasePage1 with version 17, in the order of DeltaPage1 with version 18 → DeltaPage2 with version 20 to obtain the memory page. Since DeltaPage2 with version 20 includes tree nodes with version 20 and tree nodes with version 21, after playing back the page in sequence to the incremental page with the maximum version not greater than the read version, that is, playing back the loaded DeltaPage1 with version 18, when playing back in sequence to DeltaPage2 with the maximum version 20 loaded, not all tree nodes in DeltaPage2 are played back, but the tree nodes in the incremental page with the maximum version loaded are continued to be played back in sequence until they are the same as the read version, that is, the tree node with version 20 in DeltaPage2 is played back, thereby obtaining the memory page.

[0227] In another example, assuming that an incremental page contains a version of a tree node, that is, the version of the incremental page is the same as the version of the tree node contained therein, then based on the base page, the incremental page can be replayed in version order to an incremental page equal to the read version to obtain a memory page.

[0228] The reason why it is necessary to replay from the base page to the incremental page to obtain the memory page is due to the trustless characteristics of the blockchain, which requires not only providing data but also verifiable data. Specifically, based on the leaf nodes and all related intermediate nodes of the global state in the base page, the incremental page is replayed in sequence, that is, the leaf nodes, intermediate nodes and root nodes in the base page are updated according to the leaf nodes, intermediate nodes and root nodes contained in the incremental page in version order. Only then can all tree nodes in the restored memory page maintain the hash anchor relationship from the leaf node to the intermediate node to the root node, such as the hash value of the correct lower-level tree node stored in the branch node slot, so that data can be verified.

[0229] As mentioned above, the version of a delta page can be the lowest or highest version among the one or more memory pages corresponding to it. In the above example, the case where the version of a delta page is the lowest version among the multiple memory pages corresponding to it is given. The following describes the case where the version of a delta page is the highest version among the multiple memory pages corresponding to it.

[0230] For the aforementioned situation 2, i.e., the situation where the base page with the same version as the version is not found in the persistent storage medium, a base page with a maximum version not greater than the version read, a consistent page ID and a page type of a base page can be loaded from the persistent storage medium to the memory. On this basis, an incremental page with a version after the base page and a minimum version not less than the version read, a consistent page ID and a page type of an incremental page can also be loaded from the persistent storage medium to the memory and loaded. For example, the version to be searched is 20, but there is no memory page with version 20 in the memory. Of course, the base page with the same version as 20 is not found in the persistent storage medium. In fact, there are, for example, BasePage1 with version 17, DeltaPage1 with version 19, DeltaPage2 with version 21 and BasePage2 with version 22 in the persistent storage medium. DeltaPage1 with version 19 can include tree nodes with version 18 and tree nodes with version 19, and DeltaPage2 with version 21 can include For tree nodes with version 20 and tree nodes with version 21, BasePage1 with a version not greater than 20, consistent page ID and page type of the base page with a maximum version of 17 can be loaded into memory, that is, the version of the eligible base page here is 17, and then version 17 of BasePage1 is loaded to the minimum version 21 not less than the read version 20, the page ID is consistent and the page type is the incremental page with version 19 and DeltaPage2 with version 21 into memory, and then the loaded base page and incremental page are played back to obtain the memory page.

[0231] For the above situation 2, specifically, the loaded base page and incremental page can be replayed in the order of page type and version to obtain the memory page. Still according to the above example, the loaded base page is BasePage1 with version 17, and the loaded incremental pages include DeltaPage1 with version 19 and DeltaPage2 with version 21. Then, the incremental pages can be replayed in the order of versions, that is, based on the BasePage1 with version 17, in the order of DeltaPage1 with version 19 → DeltaPage2 with version 21 to obtain the memory page. Since DeltaPage2 with version 21 includes tree nodes with version 20 and tree nodes with version 21, after playing back the page in sequence to the incremental page with the maximum version smaller than the read version, that is, playing back DeltaPage1 with the loaded version 19, when playing back in sequence to DeltaPage2 with the loaded maximum version 21, not all tree nodes in DeltaPage2 are played back, but the tree nodes in the incremental page with the loaded maximum version are continued to be played back in sequence until they are the same as the read version, that is, the tree node with version 20 in DeltaPage2 is played back, thereby obtaining the memory page.

[0232] S194: Read the status in the leaf node.

[0233] As mentioned above, the leaf node may be in the form of the aforementioned data file or the aforementioned KVDB. In the form of the aforementioned data file, the leaf node may include the location index of the state value in the data file in the persistent storage medium and the hash value of the state value content. Thus, the location index contained in the leaf node can be obtained, and the location index points to the location in the data file in the persistent storage medium. Then, the value of the location of the data file in the persistent storage medium pointed to by the location index can be loaded into the memory and the state data therein can be read. In the form of KVDB, the cache can be searched according to the hash value of the leaf node. If the cache is not found, the aforementioned Figure 7 The search in the state database module shown may involve a binary search in the sst of each Level, as described above and will not be repeated here.

[0234] As mentioned above, the essence of blockchain is a chain structure composed of a series of data blocks (blocks) linked by hashes. A memory page can include all states in the current version. In fact, many of these states are likely to have been updated in a previous block and have not changed until the current block. This is similar to Ethereum, which is also called current state. CurrentDB is used to store the latest "current state" data, such as the latest account status, contract code, and contract storage. Moreover, as the block grows, the data in currentDB can be continuously modified. There is a large redundancy in the chain structure of multiple blocks in the blockchain. It is an efficient way to store data by referencing the same leaf node and intermediate node that was last updated to the value in the previous block. Moreover, the changed state in each block is likely to be part of the global state. This is also similar to Ethereum, also called history state.

[0235] In some current implementations of blockchain technology, there is a practice of storing the current state and history state in currentDB and historyDB respectively. These two databases are usually independent entities. In some implementations, both databases are persisted independently. However, this dual database architecture has obvious defects: First, it leads to redundant storage of state data, which increases storage overhead. For example, if currentDB occupies 10GB of disk space and historyDB occupies 100GB, the total storage requirement reaches 110GB. Second, this architecture increases the complexity of the system. The usual operation process is to update currentDB first, then generate a binary log (Binlog), then synchronize Binlog to historyDB, and finally execute Binlog by historyDB to generate the corresponding version of the state record. This mechanism of keeping the dual databases synchronized not only increases the complexity of the system, but also may introduce potential risks of data consistency.

[0236] On the other hand, some blockchain implementations adopt a solution that only retains currentDB without maintaining historyDB. Although this method greatly reduces storage requirements (for example, it only takes up 10GB of disk space), it also brings new problems. When it is necessary to restore to the state of a historical version, if the read-write set is saved, the system needs to start from the genesis block and replay the read-write set in version order until the required historical version state is reached; if the read-write set is not saved, it is necessary to replay all transactions starting from the genesis block until the target state is reached. This method is extremely inefficient, time-consuming and computationally intensive when performing historical state recovery, especially when dealing with long-running blockchain systems, this problem becomes more prominent.

[0237] This application is in the above Figures 9 to 19 Based on the scheme, a method for storing the world state in a tree structure is provided, including:

[0238] S200: Generate corresponding memory pages for the generated / updated tree nodes according to the divided logical page structure, and the memory pages are used to represent the contents of all the corresponding tree nodes in the logical page; the logical page is the result of aggregating several upper and lower adjacent tree nodes according to the node association relationship of the tree structure, and a logical page contains at least one tree node, and different logical pages contain different tree nodes.

[0239] S202: Generate a base page and an incremental page based on changes in the contents of all tree nodes in the logical pages of the successive versions;

[0240] S200 and S202 are similar to the above and will not be described in detail.

[0241] S204: persisting the basic page and the incremental page in one database in a manner that does not distinguish between currentDB and historyDB.

[0242] In this application, a database is set. When data is written to disk, historyDB and currentDB are no longer distinguished, but the base page and incremental page are written into this database. In this way, unlike the prior art, two databases, currentDB and historyDB, are used, so data redundancy and storage overhead can be reduced. Moreover, unlike the prior art, only currentDB is retained without maintaining historyDB, so restoring any version status becomes faster and more efficient.

[0243] Furthermore, as described above, a base page may be generated once for each predetermined number of state modifications, and a delta page may be generated once for each predetermined number of state modifications.

[0244] The base page and the incremental page may contain a page identifier, which includes a version, a page ID, and a page type; the logical page and the memory page contain a page identifier, which includes a version and a page ID.

[0245] Moreover, the state value may be stored in a data file of a persistent storage medium, and the leaf node includes a position index of the state value in the data file of the persistent storage medium.

[0246] Based on the solutions in Figures S200 to S202 above, this application also provides a method for restoring the state of any version of the world, including:

[0247] S210: Read the version to be restored;

[0248] S212: Reading a basic page of the same version as the restored version from the database;

[0249] S214: If a base page of the same version as the restored version is read from the database, the base page of the restored version is loaded from the database into the memory to obtain a memory page;

[0250] S216: If the base page of the same version as the restored version is not read in the database, a base page of a maximum version not greater than the restored version and a page type of a base page is loaded from the database to the memory, and an incremental page of a version after the base page to a minimum version not less than the restored version and a page type of an incremental page is loaded from the database to the memory; the loaded base page and incremental page are replayed in the order of page type and version to obtain the memory page of the restored version;

[0251] S218: For a memory page including a leaf node, read the status of the leaf node into the memory.

[0252] For restoring the world state of any version, such as restoring the current version or any historical version, the above recovery process can be performed according to the version that needs to be restored. Of course, for the memory pages existing in the memory, as mentioned above, based on the elimination mechanism in the cache, it is also possible that some MemoryPages are eliminated. If a MemoryPage that needs to be restored is eliminated, the corresponding BasePage and DeltaPage can be read to replay the MemoryPage. As mentioned above, the base page and incremental page can be generated based on the changes in the logical page of the continuous version, and the base page and incremental page are persistent objects of the memory page corresponding to the logical page. Therefore, in the case where the memory page does not exist in the memory, the corresponding base page and incremental page can also be loaded from the persistent storage medium to the memory, and the loaded base page and incremental page can be replayed to obtain the memory page.

[0253] In addition, when the blockchain node is cold started, there are generally no memory pages in the memory. At this time, the solution of the present application can be used to load the base page and incremental page corresponding to the current latest version of the block from the persistent storage medium to the memory, and the loaded base page and incremental page can be replayed to obtain the memory page, thereby obtaining the current state.

[0254] In the prior art, for restoring any historical version, in addition to the above-mentioned solution of replaying all transactions from the genesis block until the target state is reached, it is also possible to start from the root node of the MPT tree locked by the state root of the specified historical version and search for tree nodes in the database. This process involves decoding and then continuing to search for the next level of tree nodes according to the slot relationship. It also involves searching layer by layer in LevelDB, including traversal and a large number of binary search operations. As mentioned above, it is also computationally intensive and has the problem of read amplification. In fact, the same problem exists for restoring the current version.

[0255] The reading solution provided in the present application adopts the method of marking the base page and incremental page of the version. When searching for the base page and incremental page in the database, the base page and incremental page are generally distributed in the order of the version, so the search can be done quickly, thereby improving the recovery efficiency.

[0256] The method may also include: verifying the hash locking relationship between the upper and lower adjacent tree nodes from the leaf node to the root node, and verifying the consistency of the root node hash value with the state root in the latest version of the block. This is because, in the world state of the tree structure in the blockchain, the upper layer of tree nodes anchors the hash value of the lower layer of tree nodes, thereby achieving verifiable storage as a whole. Therefore, after restoring a version of the world state in the above manner, further verification of the anchored hash can ensure the correctness of the world state.

[0257] As mentioned above, the state value can be stored in a data file of a persistent storage medium, and the leaf node includes the position index of the state value in the data file of the persistent storage medium. Correspondingly, reading the state in the leaf node here can be to obtain the position index contained in the leaf node, and the position index points to the position in the data file of the persistent storage medium, and then load the value of the data file position in the persistent storage medium pointed to by the position index into the memory and read the state data therein.

[0258] As in the example mentioned above, the leaf node may include a hash value (valuehash) of the state value content, wherein the hash value of the leaf node is calculated based on the content in the leaf node, including the calculation based on the hash value of the state value content in the leaf node, but excluding the position index of the state value in the data file in the persistent storage medium. Here, based on the valuehash field included in the leaf node, this solution can also verify that the hash value of the data file in the persistent storage medium pointed to by the position index is consistent with the valuehash in the leaf node.

[0259] The loaded base page and incremental page are replayed in the order of page type and version to obtain the memory page of the restored version. Similar to the above, it can be based on the base page, and the incremental page is replayed in the order of version to the incremental page of the maximum version smaller than the restored version, and the tree nodes in the loaded incremental page of the maximum version are continued to be replayed in sequence until they are the same as the restored version to obtain the memory page.

[0260] For example, if the version to be restored is 20, and the base page with the same version as 20 is not found in the database, as in the above example, BasePage1 with a version not greater than 20 and a page type of the base page with a maximum version of 17 can be loaded into the memory, that is, the version of the eligible base page here is 17, and then the version 17 of the BasePage1 to the minimum version 21 not less than the read version 20, the page ID is consistent and the page type is the incremental page with a version of 19 DeltaPage1 and a version of 21 DeltaPage2 are loaded into the memory, and then the loaded base page and incremental page are replayed to obtain the memory page. Specifically, the loaded base page and incremental page can be replayed in the order of page type and version to obtain the memory page. Still according to the above example, the loaded base page is BasePage1 with version 17, and the loaded incremental pages include DeltaPage1 with version 19 and DeltaPage2 with version 21. Then, the incremental pages can be replayed in the order of versions, that is, based on the BasePage1 with version 17, in the order of DeltaPage1 with version 19 → DeltaPage2 with version 21, to obtain the memory page. Since DeltaPage2 with version 21 includes tree nodes with version 20 and tree nodes with version 21, after replaying the page in sequence to the incremental page with the maximum version smaller than the read version, that is, replaying the loaded DeltaPage1 with version 19, when replaying in sequence to DeltaPage2 with the maximum version 21 loaded, all tree nodes in DeltaPage2 are not replayed, but the tree nodes in the incremental page with the maximum version loaded are continuously replayed in sequence until the version is the same as the read version, that is, the tree nodes with version 20 in DeltaPage2 are replayed, thereby obtaining the memory page.

[0261] It should be noted that for multiple logical pages, although a base page is generated for each predetermined number of state modifications, and an incremental page is generated for each predetermined number of state modifications, and this predetermined number is generally set to be the same, different logical pages contain different tree nodes. Therefore, for a tree structure containing multiple logical pages, the number of tree nodes in different logical pages may not be synchronized. In this way, there may be Fig. 20 In the case of , the tree node containing version 17 is located in the DeltaPage with version number 17 in the left LogicalPage, and in the BasePage with version number 17 in the right LogicalPage, that is, the version numbers of the BasePage and DeltaPage in different logical pages are not synchronized. When restoring the world state of a specified version, the aforementioned S214 and S216 may be parallel operations, and there is no order of precedence.

[0262] The aforementioned application also proposes a method for constructing a world state based on a tree structure, such as S180 to S184 and S200 to S204. In S184, the generated / updated tree node is updated according to the divided logical page structure, and the memory page is used to represent the content of all corresponding tree nodes in the logical page. The logical page contains a page identifier, and the page identifier includes the lexicographic order content and version of the upper tree node in the logical page.

[0263] Based on this, the aforementioned present application also provides a reading method. The reading method involves searching for corresponding base pages and incremental pages from disk. The base page and incremental page may include a page identifier, and the page identifier includes a version, a page type, and a page ID. In the database, the scheme can match the base page and incremental page according to the page identifier on the disk according to the state key and version to be found, and after matching, the base page and incremental page are loaded into memory and played back to obtain MemoryPage.

[0264] First, if Fig.21 As shown, a State Trie locked to the State Root in the block header of block number N is shown. This State Trie can be the aforementioned Figures 11 to 14Any of the above. The State Trie tree structure adopts the aforementioned LogicalPage structure. Among them, MemoryPage is used to represent the contents of all tree nodes corresponding to the logical page of the current latest version, and is a memorized object of the logical page. The contents of the tree nodes maintained by the logical page are updated according to the contents of the tree nodes maintained by the logical page in the continuous version, and BasePage and DeltaPage can be generated. If MemoryPage is eliminated, the corresponding BasePage and DeltaPage can be read to replay the MemoryPage. To read the corresponding BasePage and DeltaPage, the corresponding BasePage and DeltaPage need to be found on the disk. BasePage and DeltaPage can be used as the minimum unit of disk persistence of the tree. Each BasePage and DeltaPage on the disk has a specific position on the disk, such as the disk block number, the starting position and the continuous length, so that the BasePage / DeltaPage can be located on the disk. As mentioned above, the page identifier can typically include the version, page type and page ID. Then, an index file can be set to store the corresponding relationship between the page identifier and the position of the page file (i.e., the data file of the page) on the disk. When searching for any BasePage and DeltaPage on disk, the index file can be loaded first, that is, the index file is loaded into memory, and matching is performed based on the loaded index file. After matching the corresponding page identifier, the corresponding disk location can be found, and the BasePage and DeltaPage are loaded from the corresponding disk location. It should be noted that, generally speaking, the entries in the index can be sorted according to the page identifier, so that binary search can be implemented.

[0265] The index files stored on disk are similar to BasePage and DeltaPage. Fig.21As shown in . In particular, a BasePage generally contains the contents of the full amount of tree nodes in a memory page, so the version of the top-level tree node in the BasePage is the version of the BasePage, and the lexicographic order of the contents from the root node to the top-level tree node in the BasePage can be used as the PageID of the BasePage. A DeltaPage can be a collection of multiple consecutive modifications to the MemoryPage, so that a DeltaPage can correspond to multiple consecutive versions, and for any version, it is generally not the full amount of tree node content, and generally speaking, a DeltaPage does not contain the full amount of tree node content. The version of an incremental page can be the lowest version in the corresponding one or more memory pages, or it can be the highest version in the corresponding one or more memory pages. The following is explained using the lowest version as an example.

[0266] After receiving the read state key and version, for example, after the storage module receives the read state key and version, a series of page IDs corresponding to LogicalPage can be constructed according to the state key, and the memory page can be matched according to the version and page ID. If the memory page does not exist in the memory, the corresponding base page and incremental page can be further searched and loaded from the persistent storage medium according to the page identifier.

[0267] Specifically, for the page ID of the version build, if there is a BasePage corresponding to the version and page ID on the disk, it can be found in the index. For example, to find the version 24 and the same page ID as MemoryPage B5 in the figure, you can traverse or perform a binary search in the index according to the version number, find several entries matching 24, and search for entries matching PageID in these matching entries. For PageID, you can match the suffix. Assuming that PageID also matches, that is, the PageType in the matching entry is BasePage, it means that the full tree node content for restoring the MemoryPage is already available in the BasePage, so the Page file on the disk can be read and loaded according to the disk location corresponding to the version + page type + ID, thereby restoring the MemoryPage.

[0268] If the search version is 25 and the same page ID as MemoryPage B5 in the figure, you can traverse or perform a binary search in the index according to version + page type + PageID. If you find several adjacent entries that match version 25, and further search for entries with matching PageID in these adjacent entries, as mentioned above, perform a suffix match with PageID in the adjacent entries of version 25 + page type + PageID in the index. Fig.21 As shown in , since the versions of the DeltaPage corresponding to B5 are 25 and 27 respectively, that is, 25 exists, assuming that PageID also matches, this search is a hit, and the hit PageType is DeltaType. Then, further search for the previous DeltaPage or BaseType. The version 25 can be -1 operated to get 24, and then search in the index with version 24 and PageID. The result is as described above, and the file with PageType of BasePage in the matching entry is found. After finding the file of the BaseType type, since the BasePage file already has the full tree node content of a MemoryPage, the BaseType with version 24 and PageID matching can be combined with the DeltaType with version 25 and PageID matching to restore the target MemoryPage with version 25, thereby ending the search. Then, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and PageID matching can be loaded, and the BaseType with version 24 and PageID matching and the DeltaType with version 25 and PageID matching can be replayed in the order of versions, thereby restoring the MemoryPage with version 25. The specific recovery method is as described in S192 or S216 above, and will not be repeated here.

[0269] Similarly, if the search version is 26 and the same page ID as MemoryPage B5 in the figure, you can traverse or perform a binary search in the index according to the version number. If you find several entries matching 26, you can further search for entries with the same PageID among these matching entries. Fig.21As shown in , since the IDs of the DeltaPage corresponding to B5 are 25 and 27 respectively, and 26 does not exist, this search did not hit. Since the example here sets the version of the incremental page to the lowest version of the corresponding one or more memory pages, if there is no entry matching the ID of version 26, the version is subtracted by 1 and searched again. As shown in the previous paragraph, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and PageID matching are finally found. Then, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and PageID matching can be loaded, and the BaseType with version 24 and PageID matching and the DeltaType with version 25 and PageID matching can be replayed in version order, thereby restoring the MemoryPage with version 26. The specific recovery process is as described above and will not be repeated here.

[0270] Similarly, if the search version is 27 and the same page ID as MemoryPage B5 in the figure, the index can be traversed or binary searched according to version + page type + PageID. If several adjacent entries matching version 27 are found, and further search for entries with matching PageID in these adjacent entries, as mentioned above, suffix matching is performed with PageID in the adjacent entries of version 27 + page type + PageID in the index. Fig.21As shown in , since the versions of the DeltaPage corresponding to B5 are 25 and 27 respectively, that is, 27 exists, assuming that PageID also matches, this search is a hit, and the hit PageType is DeltaType. Then, further search for the previous DeltaPage or BaseType. The version 27 can be -1 operated to get 26, and then search in the index with version 26 and PageID. After recursive search, as described in the above paragraphs, the file with PageType as BasePage in the matching entry is finally found. After finding the file of the BaseType type, since the BasePage file already has the full tree node content of a MemoryPage, the BaseType with version 24 and PageID matching can be combined with the DeltaType with versions 25 and 27 and PageID matching to restore the target version 27 of MemoryPage, thereby ending the search. Furthermore, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and 27 PageID matching can be loaded, and the BaseType with version 24 and PageID matching and the DeltaType with version 25 and 27 PageID matching can be replayed in version order, thereby restoring the MemoryPage with version 27. The specific recovery process is as described above and will not be repeated here. The above search process shows that if a matching incremental page is found in the index file of the base page / incremental page according to the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding base page / incremental page is searched in the index file according to the decremented version and page ID until the base page is found, and the search process ends.

[0271] Similarly, if the search version is 28 and the same page ID as MemoryPage B5 in the figure, you can traverse or perform a binary search in the index according to the version number. If you find several entries matching 28, you can further search for entries with the same PageID among these matching entries. Fig.21As shown in , since the IDs of the DeltaPage corresponding to B5 are 25 and 27 respectively, and 28 does not exist, this search did not hit. Since the example here sets the version of the incremental page to the lowest version of the one or more memory pages corresponding to it, if there is no entry matching the ID of version 28, the version is minus 1 and searched again. After recursive search, as shown in the above paragraphs, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and 27 PageID matching are finally found. Then, the BaseType with version 24 and PageID matching and the DeltaType with version 25 and 27 PageID matching can be loaded, and the BaseType with version 24 and PageID matching and the DeltaType with version 25 and 27 PageID matching can be replayed in version order, thereby restoring the MemoryPage with version 28. The specific recovery process is as described above and will not be repeated here. The above search process shows that if no matching incremental page is found in the index file of the base page / incremental page based on the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding base page / incremental page is searched in the index file based on the decremented version and page ID until the base page is found, and the search process ends.

[0272] In the above search process, if the PageType in the matching entry is DeltaPage, it means that the tree node content for recovering part of the increment of the MemoryPage is already available in the DeltaPage, but the possible previous version of DeltaPage and BasePage are also needed. Therefore, it is necessary to decrement the version again and continue to traverse in the index or perform a binary search with the suffix ID until a BasePage with the same page ID and lower than the largest version in the version is found. After finding all the BasePages and DeltaPages, they are loaded into memory, and can be played back in order according to these BasePages and DeltaPages, and finally the specified version of MemoryPage is recovered.

[0273] The above solution is a general feasible solution. For each version and PageID to be found, it is necessary to match them in the index according to the version and PageID, which is a relatively complex task involving a large amount of searching and comparison.

[0274] Furthermore, the present application proposes a method for matching base pages and delta pages to speed up the search process in the disk. In terms of chronological order, BasePage / DeltaPage is generated by time, or in other words, by version order. In this way, the generation time of the subsequent version is later, and the generation time of the predecessor version is earlier. In this scheme, the data file storage location of the previous predecessor version can be set in the Page file of the disk, so that as the subsequent version is formed, a series of forward linked lists pointing to the previous predecessor file are formed in each version, such as Fig. 22 shown.

[0275] The method for finding a base page and / or an incremental page proposed in the present application includes:

[0276] S220: receiving the read status key and version;

[0277] S222: Match the version and page ID in the page identifier of the logical page in lexicographic order according to the state key and version;

[0278] S224: searching for an incremental page in the index file according to the version and the page ID;

[0279] S226: After the incremental page is found, the data file position of the base page / incremental page of the predecessor stored in the found incremental page is recursively read until the data file position of the base page is found;

[0280] S228: Read the corresponding data file from the persistent medium according to the file location in the found base page / incremental page.

[0281] The data files here are the base pages / incremental pages linked by the forward linked list.

[0282] S220 and S222 are similar to the above and will not be described in detail.

[0283] The above process mentioned:

[0284] If a matching incremental page is found in the index file of the base page / incremental page according to the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding base page / incremental page is searched in the index file according to the decremented version and page ID until the base page is found, and the search process ends; and,

[0285] If no matching incremental page is found in the index file of the base page / incremental page based on the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding base page / incremental page is searched in the index file based on the decremented version and page ID until the base page is found, and the search process ends.

[0286] From the above content, we can know that for the version and page ID that are being searched, there is a corresponding incremental page on disk. According to the above method, the corresponding incremental page entry can always be found first in the index. Specifically, it includes:

[0287] If a matching delta page entry is found in the index file based on the version and page identifier, or,

[0288] If no matching incremental page entry is found in the index file according to the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding incremental page entry is searched in the index file according to the decremented version and page ID, until an incremental page entry matching the version is found.

[0289] Next, the recursive search for the corresponding predecessor base page / incremental page entry in the index file according to the decremented version and page ID can be avoided, and the file location of the predecessor base page / incremental page stored in the first found incremental page entry can be read.

[0290] like Fig. 22As shown in , for example, if the search is for Page 28 pointing to B5 (the details of PageID are ignored here), the incremental page DeltaPage27 pointing to Page B5 can be first found through S220, S222 and S224. Since DeltaPage27 stores the file location of its predecessor DeltaPage, that is, DeltaPage25, it is possible to avoid searching in the index again, and instead find DeltaPage27 directly on the disk based on the file location of its predecessor DeltaPage25 stored in DeltaPage27. Similarly, since DeltaPage25 stores the file location of its predecessor DeltaPage, that is, DeltaPage24, it is possible to avoid searching in the index again, and instead find BasePage24 directly on the disk based on the file location of its predecessor BasePage24 stored in DeltaPage25. At this point, the base page is found, and as mentioned above, after finding the file of the BaseType type, since the BasePage file already has the full tree node content of a MemoryPage, the BaseType with version 24 and PageID matching can be combined with the DeltaType with versions 25 and 27 of PageID matching to restore the target MemoryPage with version 28, thus ending the search. Furthermore, the BaseType with version 24 PageID matching and the DeltaType with versions 25 and 27 of PageID matching can be loaded, and the BaseType with version 24 and PageID matching and the DeltaType with versions 25 and 27 of PageID matching can be replayed in version order, thereby restoring the MemoryPage with version 28.

[0291] It should be noted that, in the embodiment of the present method, since the BasePage file already has the full tree node content of a MemoryPage, the data file storage location of the previous predecessor version can be set in the data file of the incremental page instead of setting the data file storage location of the previous predecessor version in the data file of the base page.

[0292] In order to avoid matching each version and PageID to be searched in the index according to the version and PageID, that is, to reduce the work of searching and comparing, the present application also proposes a method embodiment for matching the base page and the incremental page. In this method embodiment, the previous predecessor version can be recorded in the incremental page entry of the index, including the version of the previous predecessor base page / incremental page. As the subsequent version is formed, its predecessor version is recorded in each incremental page entry in the index, such as Fig.23 shown.

[0293] The method for finding a base page and / or an incremental page proposed in the present application includes:

[0294] S230: receiving the read status key and version;

[0295] S232: Match the version and page ID in the page identifier of the logical page in lexicographic order according to the state key and version;

[0296] S234: searching for an incremental page entry in the index file according to the version and the page ID;

[0297] S236: After finding the incremental page entry, recursively read the version of the predecessor base page / incremental page stored in the found incremental page entry and search in the index file according to the found version and page ID until the base page is found;

[0298] S238: Determine the location of the corresponding data file in the persistent medium according to the found version of the base page / incremental page and the page ID, and read the corresponding data file from the persistent medium.

[0299] S230 and S232 are similar to the above and will not be described in detail.

[0300] In S234, the incremental page entry may be first searched in the index file according to the version and the page ID. If a matching incremental page entry is found in the index file according to the version and the page ID, the version of the predecessor base page / incremental page may be read from the found incremental page entry.

[0301] In this way, after finding the incremental page entry in S236, the version of the predecessor base page / incremental page stored in the found incremental page entry can be recursively read and searched in the index file according to the found version and page ID until the base page entry is found.

[0302] like Fig.23 As shown in, for example, if the page with version 27 pointing to B5 (the details of PageID are ignored here), the delta page entry DeltaPage27 pointing to Page B5 can be first found through S230, S232 and S234. Since the version of the previous predecessor base page / delta page is set in the delta page entry of the index, that is, the entry DeltaPage27 stores the version 25 of its predecessor DeltaPage, DeltaPage25, so the predecessor version 25 in the index can be located. In the next search, only the entry with version 25 needs to be searched, thereby avoiding the search in all index files.

[0303] In addition, in S234, the incremental page entry is searched in the index file according to the version and the page ID. If no matching incremental page entry is found in the index file according to the version and the page ID, the following operations are recursively performed: the version is decremented, and the corresponding incremental page entry is searched in the index file according to the decremented version and the page ID until an incremental page entry with a matching version is found. Thus, the version of the predecessor base page / incremental page stored in the found incremental page entry can be read.

[0304] In this way, after finding the incremental page entry in S236, the version of the predecessor base page / incremental page stored in the found incremental page entry can be recursively read and searched in the index file according to the found version and page ID until the base page entry is found.

[0305] like Fig.23 As shown in , for example, if the page with version 28 pointing to B5 (the details of PageID are ignored here) is searched, then the incremental page entry DeltaPage28 pointing to Page B5 can be first found through S230, S232 and S234. Since there is no DeltaPage with version 28 and matching PageID in the index, the version is decremented, and here the version is 28 after decrement. The corresponding incremental page entry is searched in the index file according to the decremented version 28 and the page ID, and the incremental page entry with matching version can be found (PageID also matches). Furthermore, since the version of the previous predecessor base page / incremental page is set in the incremental page entry of the index, that is, the entry DeltaPage27 stores its predecessor DeltaPage, that is, version 25 of DeltaPage25, so the predecessor version 25 in the index can be located. In the next search, it is only necessary to search in the entry with version 25, thereby avoiding searching in all index files.

[0306] Furthermore, as described in S238, the position of the corresponding data file in the persistent medium can be determined based on the found base page / incremental page version and the page ID in the index, and the corresponding data file can be read from the persistent medium. That is, the position of the corresponding data file in the persistent medium can be determined based on the found base page version 24 and the entry of the page ID in the index, and the position of the corresponding data file in the persistent medium can be determined based on the entry of the incremental page 25 and the page ID in the index, and the position of the corresponding data file in the persistent medium can be determined based on the entry of the incremental page 27 and the page ID in the index, and the corresponding data file can be read from the persistent medium.

[0307] The above search in the index file according to the version and the page ID may be a search in the index file according to the version, and further a search according to the page ID in the index file entry matching the found version. The search may be a traversal search or a binary search.

[0308] The present application also provides a computer device, comprising:

[0309] processor;

[0310] and a memory storing a program, wherein when the processor executes the program, the operation of any one of the above methods is performed.

[0311] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0312] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0313] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0314] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0315] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0316] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0317] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0318] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0319] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0320] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0321] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0322] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0323] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0324] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0325] The above description is only an example of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims.

Claims

1. A method for finding a base page and / or a delta page, comprising: Receive the read status key and version; According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order; Search in the index file of the base page / delta page according to the version and page ID; After finding the file, the corresponding data file is read from the persistent medium according to the file position in the persistent medium in the entry in the index.

2. The method of claim 1, wherein searching in the index file of the base page / delta page according to the version and the page ID comprises: If a matching base page is found in the index file of the base page / delta page according to the version and page identifier, the search process ends.

3. The method of claim 1, wherein searching in the index file of the base page / delta page according to the version and the page ID comprises: If a matching delta page is found in the index file of the base page / delta page according to the version and page identifier, the following operations are recursively performed: The version is decremented, and the corresponding base page / incremental page is searched in the index file according to the decremented version and the page ID, until the base page is found, and the search process ends.

4. The method according to claim 2 or 3, wherein searching in the index file of the base page / delta page according to the version and the page ID comprises: If no matching incremental page is found in the index file of the base page / incremental page according to the version and page identifier, the following operations are recursively performed: The version is decremented, and the corresponding base page / incremental page entry is searched in the index file according to the decremented version and the page ID, until the base page is found, and the search process ends.

5. The method of claim 1, searching in the index file of the base page / delta page according to the version and the page ID, comprises: The version is searched in the index file of the base page / delta page, and the page ID is further searched in the index file entry that matches the found version.

6. The method according to claims 1 to 5, wherein the search comprises a traversal search or a binary search.

7. A method for searching a base page and / or an incremental page, wherein a data file of the incremental page is provided with a data file storage location of a previous predecessor version, the method comprising: Receive the read status key and version; According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order; Find the incremental page entry in the index file according to the version and page ID; After finding the incremental page, recursively read the data file location of the base page / incremental page of the predecessor stored in the found incremental page until the data file location of the base page is found; According to the persistent media location of the found base page / incremental page, the corresponding data file is read from the persistent media.

8. The method of claim 7, wherein searching in the index file of the delta page according to the version and the page ID comprises: If a matching delta page entry is found in the index file based on the version and page identifier, or, If no matching incremental page entry is found in the index file according to the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding incremental page entry is searched in the index file according to the decremented version and page ID, until an incremental page entry matching the version is found; The file position of the predecessor base page / delta page stored in the delta page entry found is read.

9. The method of claim 7, searching in an index file according to the version and page ID, comprising: The version is searched in the index file, and the page ID is further searched in the index file entry matching the found version; the search includes a traversal search or a binary search.

10. A method for searching a base page and / or a delta page, wherein a version of a previous predecessor base page / delta page is set in a delta page entry of an index, the method comprising: Receive the read status key and version; According to the state key and version, match the version and page ID in the page identifier of the logical page in lexicographic order; Find the incremental page entry in the index file according to the version and page ID; After finding the incremental page entry, recursively read the version of the predecessor base page / incremental page stored in the found incremental page entry and search in the index file according to the found version and page ID until the base page is found; The location of the corresponding data file in the persistent medium is determined in the index based on the found version of the base page / incremental page and the page ID, and the corresponding data file is read from the persistent medium.

11. The method of claim 10, wherein searching in the index file of the delta page according to the version and the page ID comprises: If a matching delta page entry is found in the index file based on the version and page identifier, or, If no matching incremental page entry is found in the index file according to the version and page ID, the following operations are recursively performed: the version is decremented, and the corresponding incremental page entry is searched in the index file according to the decremented version and page ID, until an incremental page entry matching the version is found; The data file location of the predecessor base page / delta page stored in the found delta page is read.

12. The method of claim 10, searching in an index file according to the version and the page ID, comprising: The version is searched in the index file, and the page ID is further searched in the index file entry matching the found version; the search includes a traversal search or a binary search.

13. A computer device comprising: processor; and a memory in which a program is stored, wherein when the processor executes the program, the operation of any one of claims 1 to 12 is performed.

Images