Method for reading block chain state data and block chain node
By building a tree structure in the memory of blockchain nodes and using position indexes, the read amplification problem of blockchain system when reading state data is solved, and the efficiency and performance of data reading are improved.
Patent Information
- Application Number
- CN202410933441.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-26
AI Technical Summary
Blockchain systems have read amplification problems when reading state data, resulting in low performance and high load, especially in the process of binary search in multi-layer SST files, which is less efficient.
The tree structure is used to build the world state in memory and store the location index in the leaf nodes. The location in the data file is quickly positioned through the index, avoiding binary searches in SST multiple times, and improving search efficiency.
Directly positioning the location in the data file through position index reduces the need to read a large number of files and load it into memory, improves the speed and efficiency of reading state data, and reduces the problem of read amplification.
Smart Images

Figure CN119025519A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of this specification belong to the field of blockchain technology, and in particular, relate to a method for reading blockchain status data and a blockchain node. Background Art
[0002] Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. In the blockchain system, data blocks are combined into a chain data structure in a sequential manner according to time sequence, and a distributed ledger that cannot be tampered with or forged is guaranteed by cryptography. Due to the characteristics of blockchain such as decentralization, information cannot be tampered with, and autonomy, blockchain has also received more and more attention and application. Summary of the invention
[0003] The object of the present invention is to provide a method for reading blockchain status data and a blockchain node, comprising:
[0004] A method for reading blockchain state data is applied to a storage module of a blockchain node, wherein the storage module constructs a world state with a tree structure in a memory, and the leaf nodes of the tree structure include a position index of the state data in a data file in a persistent storage medium; the method comprises:
[0005] The storage module of the blockchain node receives an instruction for the state key to be searched, and searches for a leaf node corresponding to the state key on the tree structure in the memory;
[0006] Obtaining a location index contained in the leaf node, wherein the location index points to a location in a data file in the persistent storage medium;
[0007] The value of the data file location in the persistent storage medium pointed to by the location index is loaded into the memory and the status data therein is read.
[0008] A blockchain node, the blockchain node includes a storage module, the storage module includes a tree building module and a persistent storage, the persistent storage medium includes an index file and a data file; the storage module also includes a search unit and a reading unit; the leaf node of the tree structure includes a position index of state data in the data file in the persistent storage medium;
[0009] The tree building module is used to build a world state of a tree structure in memory;
[0010] The search unit receives an instruction for a state key to be searched, searches for a leaf node corresponding to the state key on the tree structure in the memory; obtains a position index contained in the leaf node, the position index pointing to a position in a data file in the persistent storage medium;
[0011] The reading unit loads the value of the data file position in the persistent storage medium pointed to by the position index into the memory and reads the status data therein.
[0012] In the above embodiment, the location index location is stored in the leaf node, which can be used to quickly locate the position of the value of the leaf node in the data file through the location index after the leaf node is found. In this way, the content of the corresponding position in the located data file can be directly loaded into the memory, which also avoids the read amplification problem caused by reading a large number of files in multiple Level SSTs and loading them into the memory. This also avoids the kv of the leaf node being stored in the persistently stored SST file, and the process of performing binary searches in the SST multiple times according to the key (hash value of the value content) of the leaf node. Moreover, similarly, for the leaf node to be found that is not in the memory of the tree construction module, the above embodiment can only search the index file, so that more index files can be loaded into the memory within the unit space for search, and the leaf node can be found faster. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0014] Figure 1 is a schematic diagram of deploying a smart contract in one embodiment;
[0015] Figure 2 is a schematic diagram of calling a smart contract in one embodiment;
[0016] Figure 3 is a schematic diagram of a block storage structure in one embodiment;
[0017] Figure 4 is a schematic diagram of a block storage structure in one embodiment;
[0018] Figure 5 is a schematic diagram of an MPT tree in one embodiment;
[0019] Figure 6It is a schematic diagram of the modules involved in the transaction processing process and the relationship between the CPU, memory and disk in one embodiment;
[0020] Figure 7 It is a schematic diagram of the modules involved in the transaction processing process and the relationship between the CPU, memory and disk in one embodiment;
[0021] Figure 8 is a schematic diagram of the logical relationship of the state database in one embodiment;
[0022] Fig. 9 is a flow chart of reading status data in one embodiment;
[0023] Fig.10 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;
[0024] Fig.11 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;
[0025] Fig.12 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;
[0026] Fig.13 is a schematic diagram illustrating reading status data in combination with a tree structure in an embodiment;
[0027] Fig.14 It is a schematic diagram of an embodiment of the present invention in combination with a tree structure to illustrate reading status data. DETAILED DESCRIPTION
[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.
[0029] Blockchain is generally divided into three types: public blockchain, private blockchain and consortium blockchain. In addition, there are many types of combinations, such as private blockchain + consortium blockchain, consortium blockchain + public blockchain and other different combinations. Among them, the public chain has the highest degree of decentralization. Public chains are represented by Bitcoin and Ethereum. Participants who join the public chain can read data records on the chain, participate in transactions, and compete for the right to record new blocks. Moreover, each participant (reflected as the participant's node on the blockchain) can freely join and exit the network and perform related operations. On the contrary, the private chain has the write permission of the network controlled by a certain organization or institution, and the data reading permission is stipulated by the organization. In short, the private chain can be a weakly centralized system with strictly limited and few participating nodes. This type of blockchain is more suitable for use within a specific organization. The consortium chain is a blockchain between the public chain and the private chain, which can achieve "partial decentralization". Each node in the alliance chain usually has a corresponding entity or organization; participants join the network through authorization and form a stakeholder alliance to jointly maintain the operation of the blockchain.
[0030] Whether it is a public chain, a private chain or a consortium chain, in addition to supporting the transfer of native assets on the blockchain between accounts, such as the transfer of Bitcoin and Ethereum, it can also provide smart contract functions. Smart contracts on the blockchain are contracts that can be triggered and executed by transactions on the blockchain system. Smart contracts can be defined in the form of code.
[0031] Taking Ethereum as an example, supporting users to create and call some complex logic in the Ethereum network is the biggest challenge that distinguishes Ethereum from Bitcoin blockchain technology. The core of Ethereum as a programmable blockchain is the Ethereum Virtual Machine (EVM), and each Ethereum node can run EVM. EVM is a Turing-complete virtual machine, which means that various complex logics can be implemented through it. Users publish and call smart contracts in Ethereum on EVM. In fact, the virtual machine directly runs the virtual machine code (virtual machine bytecode, hereinafter referred to as "bytecode"). Smart contracts deployed on the blockchain can be in the form of bytecode.
[0032] For example Figure 1 As shown in the figure, after Bob sends a transaction containing the information of creating a smart contract to the Ethereum network, the EVM of node 1 can execute this transaction and generate the corresponding contract instance. "0x6f8ae93..." in Figure 1 represents the address of this contract. The data field of the transaction can store bytecode, and the to field of the transaction is an empty account. After the nodes reach an agreement through the consensus mechanism, the contract is successfully created and subsequent users can call this contract.
[0033] After the contract is created, a contract account corresponding to the smart contract is generated on the blockchain, and has a specific address. The contract code and account storage will be saved in the contract account. The behavior of the smart contract is controlled by the contract code, and the account storage of the smart contract saves the state of the contract. In other words, the smart contract generates a virtual account on the blockchain that contains the contract code and account storage.
[0034] As mentioned above, the data field of the transaction that creates a smart contract can store the bytecode of the smart contract. The bytecode consists of a series of bytes, each of which can identify an operation. Based on development efficiency, readability and other considerations, developers can choose a high-level language to write smart contract code instead of writing bytecode directly. The smart contract code written in a high-level language is compiled by a compiler to generate bytecode, which can then be deployed on the blockchain. Ethereum supports many high-level languages, such as Solidity, Serpent, LLL, etc.
[0035] Taking Solidity as an example, contracts written in it are very similar to classes in object-oriented programming languages. Multiple members can be declared in a contract, including state variables, functions, function modifiers, events, etc. State variables are values stored in the account storage of smart contracts and are used to save the state of the contract.
[0036] The following is a code example of a simple smart contract written in Solidity language 1:
[0037] Contract SimpleStorage {
[0038] string storedData;
[0039] function set(string s){
[0040] storedData = s;
[0041] }
[0042] function get()constant returns(string){
[0043] return storedData;
[0044] }
[0045] }
[0046] In addition, if Figure 2As shown, still taking Ethereum as an example, after Bob sends a transaction containing information about calling a smart contract to the Ethereum network, the EVM of node 1 can execute this transaction and generate a corresponding contract instance. The from field of the transaction in Figure 2 is the address of the account that initiates the call to the smart contract, the "0x6f8ae93..." in the to field represents the address of the smart contract being called, and the data field of the transaction stores the method and parameters for calling the smart contract. In addition, the value field can also be included to represent the value of the Ether in the transaction. After calling the smart contract, the value of storedData may change. Later, a client can call a blockchain node (for example Figure 2 Node 6) in the view the current value of storedData.
[0047] Smart contracts can be executed independently on each node in the blockchain network in a prescribed manner, and all execution records and data are stored on the blockchain. Therefore, when such a transaction is completed, the blockchain stores transaction credentials that cannot be tampered with or lost.
[0048] As mentioned above, the storedData in the above example is a state variable, which is stored in the account storage of the smart contract. In various blockchain networks that introduce smart contracts, such as Ethereum, accounts can usually include two types:
[0049] Contract account: stores the executed smart contract code and the value of the state in the smart contract code, and can usually only be activated by calling an external account;
[0050] Externally owned account: A user's account, such as an Ethereum owner account.
[0051] The design of external accounts and contract accounts is actually a mapping from account addresses to account status. Account status usually includes fields such as Nonce, Balance, Storage root, CodeHash, etc. Nonce and Balance exist in both external accounts and contract accounts. CodeHash and Storage root properties are generally only valid on contract accounts.
[0052] Nonce: Counter. For external accounts, this number can represent the number of transactions sent from the account address; for contract accounts, it can be the number of contracts created by the account.
[0053] Balance: The amount of ether owned by this address.
[0054] Storage root: The hash of the root node of an MPT tree, which organizes the storage of state variables of the contract account.
[0055] CodeHash: The hash value of the smart contract code. For contract accounts, this is the hash value of the smart contract; for external accounts, since they do not include smart contracts, the CodeHash field can generally be an empty string / all-0 string.
[0056] MPT stands for Merkle Patricia Tree, which is a tree structure that combines Merkle Tree and PatriciaTree (compressed prefix tree, a more space-saving Trie tree, dictionary tree). Merkle Tree, the Merkle tree algorithm calculates a hash value for each transaction, and then connects two by two to calculate the hash again until the top Merkle root. Ethereum uses an improved MPT tree, such as a 16-way tree structure, which is usually referred to as MPT tree.
[0057] The data structure of the Ethereum MPT tree includes a state trie. The state trie contains key-value pairs (also written as key-value, abbreviated as kv or kv) of the storage content corresponding to each account in the Ethereum network. The "key" in the state trie can be a 160-bit identifier (such as the address of an Ethereum account or part of the hash value of an address, collectively referred to as an account address below), which is distributed in the storage from the root node of the state trie to the leaf node. The "value" in the state trie is generated by encoding the information of the Ethereum account (using the Recursive-Length Prefix encoding (RLP) method). As mentioned above, for external accounts, the values include nonce and balance; for contract accounts, the values include nonce, balance, codehash, and storageroot.
[0058] The contract account is used to store the states related to the smart contract. After the smart contract is deployed on the blockchain, a corresponding contract account will be generated. This contract account generally has some states, which are defined by the state variables in the smart contract and generate new values when the smart contract is created and executed. The smart contract generally refers to a contract that can automatically execute clauses defined in the form of code in the blockchain environment. Once an event triggers the clauses in the contract (meets the execution conditions), the code can be automatically executed. In the blockchain, the relevant states of the contract are saved in the storage trie, and the hash value of the storage trie root node is stored in the above storageroot, so that all states of the contract are locked to the contract account through hash. The storage trie is also an MPT tree structure that stores the key-value mapping from the state address to the state value. Part of the information from the root node to the leaf node of the storage trie tree is arranged in sequence to store the address of a state, and the leaf node stores the value of the state.
[0059] like Figure 3 In some blockchain data storages shown, the block header of each block includes several fields, such as the previous block hash previous_Hash (Prev Hash in the figure), the random number Nonce (in some blockchain systems, this Nonce is not a random number, or in some blockchain systems, the Nonce in the block header is not enabled), the timestamp Timestamp, the block number Block Num, the state root hash State_Root, the transaction root hash Transaction_Root, the receipt root hash Receipt_Root, etc. Among them, the Prev Hash in the block header of the next block (such as block N+1) points to the previous block (such as block N), which is the hash value of the previous block. In this way, the next block on the blockchain is locked to the previous block through the block header. Among them, State_Root, Transaction_Root and Receipt_Root lock the state set, transaction set and receipt set respectively. The state set, transaction set and receipt set organize the state, transaction and receipt in the form of a tree respectively. Generally, it can be the same tree structure or different tree structures. For example, in Ethereum, the same MPT structure is used. In some tree structures including the state set of smart contracts such as Ethereum, there are two levels of MPT structures: the leaf nodes of the upper-level MPT structure include two types: external accounts and contract accounts; each contract account includes the MPT structure of the lower level, and the leaf nodes of the lower level include the value of the state in the contract account.
[0060] Figure 4This is a schematic diagram of the structure of blockchain data storage. Taking Ethereum as an example, we can combine Figure 3 As shown in the figure, state_root is the hash value of the root of the MPT tree composed of the status of all accounts in the current block, that is, the state trie pointing to state_root is a state tree in the form of MPT. The root node of this MPT tree is generally an extension node (Extension Node) or a branch node (Branch Node), and the hash value of this root node is generally stored in state_root. The root node can be connected to one or more layers of Extension Node / Branch Node below, and these multi-layer tree nodes can be collectively referred to as intermediate nodes (Internal Node). From the root node of this MPT to the leaf node, a part of the value in each node is connected in sequence to form an account address and serve as a key. The account information stored in the leaf node is the value corresponding to this account address, thus forming a key-value pair. This key can also be a part of sha3(Address), that is, a part of the hash value of the account address (the hash algorithm uses the sha3 algorithm, for example), and the value stored can be rlp(Account), that is, the rlp encoding of the account information. The account information is
[0061] A tuple consisting of [nonce, balance, storageRoot, codeHash]. As mentioned above, for external accounts, there are generally only two items, nonce and balance, and the storageRoot and codeHash fields store empty strings / all zero strings by default. In other words, external accounts do not store contracts, nor do they store state variables generated after contract execution. Contract accounts generally include Nonce, Balance, Storage root, and CodeHash. Among them, Nonce is the transaction counter of the contract account; Balance is the account balance; Storage root corresponds to another MPT, through which the storage root can be linked to the information of the contract-related status; CodeHash is the hash value of the contract code. Whether it is an external account or a contract account, its account information is generally located in a separate leaf node (Leaf Node). From the Extension Node / Branch Node of the root node to the Leaf Node of each account, there may be several branch nodes and extension nodes in the middle.
[0062] The state trie can be a tree in the form of MPT, generally a 16-way tree, that is, each layer can have up to 16 child nodes. For Extension Node, it is used to store common prefixes, and it generally has 1 child node, which can be Branch Node. For Branch Node, it can have up to 16 child nodes, which may include ExtensionNode and / or Leaf Node.
[0063] Among them, for a contract account in the state trie, its storage_Root points to another tree in the form of MPT, which stores the data of the state variables involved in the contract execution. The MPT-style tree pointed to by this storage_Root is Storage Trie, that is, the hash value of the root node of Storage Trie. Generally, this Storage Trie tree also stores key-value pairs. The key indicates the address of the state variable, and its value can be the result obtained after the position of the state variable declaration in the contract (the value counting from 0) is processed by certain rules, such as sha3 (the position of the state variable declaration) or sha3 (the contract name + the position of the state variable declaration). The value is used to store the value of the state variable (for example, the value encoded by RLP). A part of the data stored on the path from the root node to the intermediate node to the leaf node is connected to form the key, and the value is stored in the leaf node. As mentioned above, the Storage trie can also be a tree in the form of MPT, which is generally a 16-way tree, that is, for a Branch Node, it can have up to 16 child nodes, which may include Extension Nodes and / or Leaf Nodes. For an ExtensionNode, it can generally have 1 child node, which can be a Branch Node or a Leaf Node.
[0064] For example Figure 4The Leaf Node Account P of the state trie in the contract is a contract account, and its Storage Root locks all states in the contract storage. These states are organized into an MPT tree, and the tree structure is like the Storage trie linked to the Storage Root. In this linked Storage trie, taking Leaf Node StateVariable N as an example, for example, the value of storedData in the aforementioned contract code example, its key is sha3 (the declaration position of storedData, that is, the second line of the code), and its value is s (for simplicity, the encoding format of value is omitted here, for example, RLP, which will be similar in the future and will not be repeated). Among them, the key values are distributed sequentially from the root node to the leaf node (that is, Leaf Node Variable N) of the storage trie.
[0065] For another example, Figure 4 Leaf Node Account C in the state Trie is an external account, and its key is sha3(Address C), which is the hash value of the address of account C (the hash algorithm uses the sha3 algorithm, for example). The stored value value can be (Account), where the account information Account is a tuple consisting of [nonce, balance]. As mentioned above, since Account C is an external account, its account information is two items, nonce and balance (codehash and storage root are omitted here, and the following are similar). For example, an external account, whose nonce is 20 and Balance is 4550, then the Leaf Node State Variable C stores nonce=20 and balance=4550. The address of Account C is the key, and its values are distributed sequentially from the root node to the leaf node (i.e., Leaf NodeVariable C) of the state Trie.
[0066] These states, including the kv of external accounts and the kv of contract accounts, are ultimately stored in the database. The storage in the database does not directly store the states of these accounts, that is, it does not directly store the kv of these accounts, but stores the kv value of each tree node itself.
[0067] like Figure 5As shown in the example, in the upper-level MPT structure, for leaf node A1, the key of the leaf node is composed of a7 of the shared nibble in the root node A8 (Extension Node)—slot 1 of the intermediate node A7 (Branch Node)—1335 of the key-end in the leaf node A1, which is a711335, and Balance=45.0ETH and Nonce=n1 are stored in the leaf node. For leaf node A2, the key of the leaf node is composed of a7 of the shared nibble in the root node A8 (Extension Node)—slot 7 of the intermediate node A7 (Branch Node)—d3 of the shared nibbles in the node A6 (Extension Node)—slot 3 of the intermediate node A5 (Branch Node)—7 of the key-end in the leaf node A2, which is a77d337, and Balance=1.00WEI and Nonce=n2 are stored in the leaf node. For leaf node A3, the key of the leaf node is composed of a7 of the shared nibble in root node A8 (Extension Node), slot f of intermediate node A7 (Branch Node), and 9365 of key-end in leaf node A3, which is a7f9365. Balance = 1.1ETH and Nonce = n3 are stored in the leaf node. For leaf node A4, the key of the leaf node is composed of a7 of the shared nibble in root node A8 (Extension Node), slot 7 of intermediate node A7 (Branch Node), d3 of the shared nibbles in node A6 (Extension Node), slot 9 of intermediate node A5 (Branch Node), and 7 of key-end in leaf node A4, which is a77d397. Balance = 0.12ETH, Nonce = n4, CodeHash = c1, and Storage root = s1 are stored in the leaf node. s1 can be H(A10), which is the hash of the root node A10 of the next tree layer. The leaf nodes of A1, A2 and A3 store the information of external accounts, and the leaf node of A4 stores the information of contract accounts. For contract accounts, it contains the next level MPT, forming a Storage Trie, which is used to store the state variables in the contract account.
[0068] like Figure 5As shown in the example, in the next-level MPT structure, for leaf node A11, slot 3 in root node A10 (Branch Node) - 35b2e4 of key-end in leaf node A11 are sequentially combined to form the key of the leaf node, that is, 335b2e4, and "Zhang San_A = 20" is stored in the leaf node, for example, indicating that the share of type A digital assets defined in the contract belonging to Zhang San is 20, that is, the balance of Zhang San's type A assets is 20. For leaf node A12, slot 7 in root node A10 (Branch Node) - c25988 of key-end in leaf node A12 are sequentially combined to form the key of the leaf node, that is, 7c25988, and "Li Si_B = 20" is stored in the leaf node, for example, indicating that the share of type B digital assets defined in the contract belonging to Li Si is 50, that is, the balance of Li Si's type B assets is 50. For leaf node A15, the key of the leaf node is composed of slot f in root node A10 (Branch Node) - a in shared nibble in intermediate node A13 (Extension Node) - slot 6 in intermediate node A14 (Branch Node) - be33 of key-end in leaf node A15, which is fa6be33, and "storedData=s" is stored in the leaf node. For leaf node A16, the key of the leaf node is composed of slot f in root node A10 (Branch Node), a in shared nibble in intermediate node A13 (Extension Node), slot 9 in intermediate node A14 (Branch Node), and 9365 in key-end in leaf node A16, which is fa99365. "Wang Wu_A=35" is stored in the leaf node, which means that the share of type A digital assets defined in the contract belonging to Wang Wu is 35, that is, the balance of Wang Wu's type A assets is 35.
[0069] In the node structure of the above MPT tree, the prefix prefix is used to indicate the tree node type, for example, 0 indicates an Extension Node containing an even number of shared nibbles (shared half bytes), 1 indicates an Extension Node containing an odd number of shared nibble(s), 2 indicates a Leaf Node containing an even number of nibbles, and 3 indicates a Leaf Node containing an odd number of nibble(s).
[0070] In the above node structure, the hash value of the entire content of the next tree node is filled in the corresponding position of the previous tree node. In the database, the key-value mapping of each tree node is actually stored, where the value includes the content stored in this tree node, and the corresponding key is the hash value of the entire content of this tree node. In this way, the tree node kv actually stored in the database is as follows:
[0071]
[0072]
[0073] Table 1. Tree node kv actually stored in the database
[0074] In Table 1 above, H() is used to represent hash calculation. In this way, the hash value of the next tree node is anchored in the previous tree node. Through such layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and the root hash is locked in the state root field of the block header.
[0075] In some blockchain systems, the code of the blockchain platform may include a P2P (Peer to Peer) module, a consensus module, an execution module, and a storage module. P2P is a way of forming a computer network. Unlike common web networks, P2P is decentralized. The P2P module can complete the distributed dissemination of data. For blockchain nodes, the P2P module can be used to disseminate and receive receipts in a peer-to-peer manner. Different participants can establish a distributed blockchain network through deployed nodes. The ledger constructed using a chain block structure is stored on each node (or most nodes, such as consensus nodes) in the distributed blockchain network. This is also called a decentralized (or multi-centralized) distributed ledger. Such a blockchain system needs to solve the problem of consistency and correctness of the ledger data on multiple decentralized (or multi-centralized) nodes. The same blockchain platform program is running on each node. Under certain fault tolerance requirements, the consensus module can ensure that all loyal nodes have the same transactions, thereby ensuring that all loyal nodes have consistent execution results for the same transaction, and packaging the transactions and execution results to generate blocks. The current mainstream consensus mechanisms include: Proof of Work (POW), Proof of Stake (POS), Delegated Proof of Stake (DPOS), Practical Byzantine Fault Tolerance (PBFT) algorithm, Honey Badger BFT algorithm, etc. During the consensus process, the consensus module can generally generate the timestamp of the block corresponding to the current transaction set. The execution module can execute transactions, including ordinary transfer transactions and transactions involving contracts, before or after the consensus module completes the consensus. For transactions involving contracts, the execution module can introduce a virtual machine to execute the code of the smart contract, such as the Ethereum Virtual Machine (EVM), so that the differences in the hardware configuration and software environment of each node are shielded by the EVM to ensure that the process and results of executing the smart contract on each node are the same, and the sandbox environment is used to prevent the execution of the smart contract from affecting the blockchain platform code, other programs or operating systems on the host. For a situation of a consortium chain, the nodes can determine the transaction content and transaction order in a transaction set through the consensus module, and then output a deterministic transaction set of the consensus result to the execution module. The execution module generates the execution result by executing ordinary transfer transactions / transactions involving contracts, and sends it to the storage module. The storage module can be responsible for storing the execution result in the local persistent storage medium of the node.
[0076] like Figure 6 In the blockchain node shown, the physical components include CPU, memory, disk, etc. The blockchain platform code executed by this blockchain node may include P2P module, consensus module, execution module and storage module. The functional implementation of P2P module, consensus module and execution module generally requires the participation of CPU and memory. The storage module may include a tree building module, a block header generation module, a WAL (Write Ahead Log) module and a state database module. Among them, the tree building module is used to build a tree (for example, an MPT tree) based on the state kv passed in by the execution module, such as the aforementioned state trie and storage trie, so as to obtain the kv of the tree node, which generally requires the participation of CPU and memory. The block header generation module is used to generate a block header based on the root node of the tree built by the tree building module and some other data (such as the hash of the previous block, timestamp, block number, etc.), which generally requires the participation of CPU and memory. The WAL module is used to persistently store the leaf node kv of the tree generated by the tree building module before writing the leaf node kv of the tree generated by the tree building module to the state database module, so as to prevent data loss caused by power failure and other situations during the process of writing the leaf node kv of the tree generated by the tree building module to the state database module, and to restore the data when this happens, which generally requires the participation of the CPU, memory and disk. The state database module is used to store the tree node kv in Table 1 constructed by the tree building module on a persistent storage device; since the tree node data will eventually be written to a persistent storage medium (such as the disk in the figure), the state database module generally requires the participation of the disk in addition to the CPU and memory.
[0077] From the perspective of storage structure, the above-mentioned Merkle tree structure, such as Ethereum's MPT and Libra's SMT (SparseMerkle Tree, similar to MPT), can be located in the tree construction module in the form of the corresponding relationship in Table 1 above and stored in the memory. Among them, the upper Merkle tree is a prefix tree (dictionary tree), which can realize the organization of data and obtain a unique Merkle root for the organized data. The leaf node can save the state Value, and the root node to the intermediate node to the leaf node implements the lexicographic index of the state key. These tree nodes are encoded as Key according to certain rules, and their contents are encoded as Value, and finally stored in the lower database. Most databases use NoSQL Key-Value DB (DataBase, database; Key-Value DB is also referred to as KVDB) with LSM (Log-Structured Merge-Tree, log structure merge tree) class structure, which is located in the state database module and finally saved on disk. Specifically, the database is, for example, Ethereum's levelDB and Libra's RocksDB. Both KVDBs are based on the LSM storage engine.
[0078] The LSM storage engine is a hierarchical, ordered, disk-oriented storage engine. It draws on the characteristics of continuous appending (rather than modification) of Logs. The core idea is to make full use of the fact that sequential batch writes of disks are much more efficient than random writes, and give up some read efficiency in exchange for maximum write efficiency. Generally speaking, the way to maximize the use of disk characteristics is to read or write a fixed-size block of data at one time and reduce random addressing operations as much as possible. The design idea of LSM is based on this characteristic of disks and assumes that there is enough memory. It does not need to write data to disk every time it is updated. Instead, the latest data resides in memory first. After the amount of data accumulates enough, the data in memory is merged with the data on disk using merge sort and appended to disk in batches.
[0079] The following Figure 7As an example, the storage process of the state data module is further explained. After the tree building module generates the tree node kv data and sends it to the storage module, these kv data are first written to WAL in an Append-Only manner and stored on the disk. Then, the tree node kv data is passed to the state database module. As mentioned above, writing to WAL first can prevent data loss caused by power outages and other situations during the process of writing the tree node kv generated by the tree building module to the state database module, and restore the data when this happens. After the state database module receives the tree node kv data, it is first stored in the memory, specifically in the MemTable in the memory. MemTable can provide concurrent read and write operations. There can be multiple MemTables in the memory. When the amount of data in the MemTable reaches a certain threshold, for example, when it reaches 256MB, the data in the MemTable can be written (flush) to the disk. In order to avoid the blockage of flush caused by the operation of writing MemTable, this MemTable is converted into an immutable Immutable Memtable, that is, Immutable Memtable is set to read-only, and a new MemTable is generated to receive the newly passed in tree node kv. This new MemTable can provide concurrent read and write operations (such as New MemTable in the figure). The storage engine writes the data in Immutable MemTable to disk.
[0080] In the disk, KVDB is stored in multiple levels of SST (Sorted String Table, SSTable) files. SST comes from the paper of Google Bigtable. BigTable is a distributed data storage system designed by Google, a non-relational database used to process massive amounts of data. SST can be used to store data files inside Bigtable, and is an efficient key-value file storage format. Content stored in persistent storage media can also be called files.
[0081] The SST may include multiple layers, for example, 3 layers, 4 layers, 5 layers, 6 layers, 7 layers or more. Figure 7The figure shows the situation of three layers, namely level 0, level 1, and level 2. Generally, the total capacity of the SST of the previous layer is significantly smaller than that of the next layer. As an example, the total capacity of the SST of level 0 is 1GB, the total capacity of the SST of level 1 is 10GB, and the total capacity of the SST of level 2 is 100GB. Assume that the capacity of MemTable is 256MB. The tree module can be written with the tree node kv corresponding to the block, and then written to MemTable. After the tree kv corresponding to one or more blocks is written to MemTable, the space occupied by MemTable may reach 256MB. Then, this MemTable is converted into Immutable MemTable, and the data in ImmutableMemTable can be written (flush, here refers to the operation of writing data in memory to disk) to disk. Specifically, the data in MemTable can be flushed to the SST of Level 0 on the disk. On the other hand, as mentioned above, a new MemTable is generated to receive the newly incoming tree node kv and provide concurrent read and write operations.
[0082] As mentioned above, each layer of disk storage, that is, each level of disk storage, can include multiple SST files. The kv stored in the SST of Level 1 and lower levels is generally stored in the order of the key, and the key values in different SSTs of the same layer are also related in size and are not repeated. For example, Level 1 includes 3 SSTs, the SST on the left 11 It includes 1000 kv, for example <k1-v1> <k2-v2> <k3-v3>... <k 1000 -v 1000 >, and arranged in order from small to large k, for example, k1 = 10b43c032a46, k2 = 1118d46f41e1, k3 = 1b44942a3d6e, ..., k 1000 =2018df9326a7. It can be seen that the SST on the left 11 k1<k2<k3<...<k 1000 , but k1, k2, k3, ... k 1000 are not necessarily continuous. Similarly, the SST in the middle 12 For example, it includes 800 kv, <k1-v1> <k2-v2> <k3-v3>... <k 800 -v 800 >, and arranged in order from small to large k, for example, k1 = 21a514c0a7e1, k2 = 21b46b994214, k3 = 21b489571b6d, ..., k 800 =284a5df3fca7. It can be seen that the SST in the middle 12 k1<k2<k3<...<k 800 , but similarly, k1, k2, k3, ... k 800 are not necessarily continuous. Similarly, the SST on the right 13 For example, it includes 1100 kv, <k1-v1> <k2-v2> <k3-v3>... <k 1100 -v 1100 >, where k is arranged in ascending order, for example, k1 = 3a5e55ac3fe1, k2 = 3a5e55ac3fe2, k3 = 3afb8ed31526, ..., k 1100 =400a51ef1405. It can be seen that the SST on the right 13 k1<k2<k3<...<k 1100 , but similarly, k1, k2, k3, ... k 1100 Moreover, overall, the left SST 11 The minimum value of k is less than the left SST 11 The maximum value of k is less than the intermediate SST 12 The minimum value of k < the intermediate SST 12 The maximum value of k is less than the right SST 13 The minimum value of k is less than the right SST 13 The same is true for each layer of Level 2, Level 3, etc., that is, within each SST in each layer, there is a size order relationship in the value of k, and between multiple SSTs in each layer, there is a size relationship in the range of k.
[0083] The kv in an SST in Level 0 can be sorted according to the size of k, and there may be no size relationship between k in multiple SSTs. This is because MemTable generally stores multiple kvs corresponding to each block in the order of blocks from old to new (or block numbers from small to large, because the block numbers of new blocks are larger and the block numbers of old blocks are smaller), and these k are the hash values of tree nodes. The changes in the content of tree nodes are likely to cause the hash values of tree nodes (i.e., k values) to be very different. The kv in an SST in Level 0 can be sorted according to the size of k. Such sorting is efficient and feasible in memory; of course, it is also possible not to sort according to the size of k values, which is not excluded here. Taking sorting as an example, when the capacity of MemTable reaches 256MB, it may store multiple kvs corresponding to multiple blocks, and this MemTable is converted to Immutable MemTable. These kvs in Immutable MemTable can be sorted in the order of k and written into the SST on the left of Level 0. There may be multiple different Immutable MemTables written to the same SST before the SST on the left side of Level 0 is written. 01 Once it is full, the SST on the right side of Level 0 can be written in a similar way. 02 . Although the kv written by a single Immutable MemTable to a certain SST in Level 0 is sorted, since the kv sets written by different Immutable MemTables in the same SST in Level 0 are generally not sorted by size, and the kvs between different SSTs (such as the left and right SSTs) are also not sorted by size, that is, the ranges of k in the left and right SSTs in Level 0 may overlap. Similarly, since the kv written to an SST in Level 0 may be written after multiple MemTables are full, and the kv ranges in the writing process after multiple MemTables are full generally overlap, there is generally no specific order between the multiple kvs stored in each SST in Level 0, and there is generally no specific range order between the SSTs in Level 0. Of course, if an SST is written by an Immutable MemTable in Level 0, since the Immutable MemTable sorts the kvs when writing to the SST, the kvs stored in the SST are also sorted by k.
[0084] Furthermore, when the storage capacity of Level 0 reaches or approaches the upper limit, the data of Level 0 is written to Level 1 through a process called "compaction". During this compaction process, the kv in each SST in Level 0 and the kv of some or all SSTs in Level 1 can be transferred to the memory first, sorted in the memory, and then written to the SST of Level 1. Since sorting is performed during the compaction process, there is an order relationship in the value of k in the SST in Level 1, and there is a size relationship in the range of k between multiple SSTs in Level 1. In other words, in each SST in Level 1, kv is sorted according to <k1-v1> <k2-v2> <k3-v3>... <k n -v n >, and k1<k2<k3<...<k n , and k1, k2, k3, ... k n They are not necessarily continuous. Moreover, overall, for two adjacent SSTs, the minimum value of k of the left SST < the maximum value of k of the left SST < the minimum value of k of the right SST < the maximum value of k of the right SST. Figure 8 as shown in .
[0085] During the compaction process, the kvs in each SST in Level 0 and the kvs in some SSTs in Level 1 can be transferred to the memory for sorting. The said some SSTs in Level 1 can be the SSTs in Level 1 that have overlapping ranges with the kvs in Level 0, that is, the SSTs in Level 1 that have overlapping ranges with the kvs in each SST in Level 0. Specifically, for example, in the above example, Level 1 includes 3 SSTs, SST 11 The key range is 10b43c032a46~2018df9326a7, SST 12 The key range is 21a514c0a7e1~284a5df3fca7, SST 13 The key range is 3a5e55ac3fe1~400a51ef1405. For example, the key range of each SST in Level 0 is 18c49a405e32~24d35f39ab56. It can be seen that there are SSTs in Level 1 that overlap with the key range in Level 0. 11 and SST 12 In this way, during the compaction process, the kv in each SST in Level 0 and the SST in Level 1 11 , SST 12 The kv can be transferred to the memory and sorted, and then the sorted results are written into the SST of Level 1.
[0086] Similarly, when the storage capacity of Level 1 reaches the upper limit, the data of Level 1 is written to Level 2 through the compaction process. Similarly, during this compaction process, the kvs in each SST in Level 1 and the kvs of some or all SSTs in Level 2 can be first transferred to the memory, sorted in the memory, and then written to the SST of Level 2. Similarly, if there is Level 3 under Level 2, when the storage capacity of Level 2 reaches the upper limit, the data of Level 2 is written to Level 3 through the compaction process. Similarly, during this compaction process, the kvs in each SST in Level 2 and some or all SSTs in Level 3 can be first transferred to the memory, sorted in the memory, and then written to the SST of Level 3. And so on.
[0087] In this way, overall, the data stored in the upper SST is newer than that in the lower SST. The latest data is stored in memory, the next latest data is stored in Level 0, and the oldest data is stored in the SST in the lowest level. In addition, compaction can also be used to perform garbage collection, such as deleting data marked for deletion.
[0088] The sorted kv can be retrieved efficiently, for example, by binary search. Binary search is also called binary search, and the algorithm is as follows: define left as the leftmost element position of the sequence list, right as the rightmost element position of the sequence list, define mid = (left + right) / 2, that is, the middle position of the sequence list; compare the value being searched with the value at the mid position; since the list is ordered, if the value being searched is smaller than mid, then only the first half of the table needs to be searched, otherwise only the second half of the table needs to be searched (if the two values are found to be equal in the first comparison, the position of the current value is directly returned). And so on, until the value being searched is found or it is determined that the value being searched is not in the list (that is, the search fails).
[0089] Such a storage engine has read amplification and write amplification problems when reading and writing data, which are described in detail below.
[0090] In the blockchain system using the above storage engine, when the execution module executes a transaction, it may be necessary to read an existing state. For example, for a general transfer transaction, when external account A transfers a certain amount of assets (for example, f) to external account B, it is necessary to increase or decrease the account balance. Assuming that before the execution of the transaction, the balance of account A is a and the balance of account B is b, then the execution of the transaction is to set the balance of account A to the value of (af) and the balance of account B to the value of (b+f). Then this transaction needs to read the values of the two state variables a and b when it is executed. For another example, in a transaction that calls a contract, external account C initiates a call to contract D, then C initiates a transaction pointing to the address of contract account D, and specifies the called function and input parameters in the data field of the transaction. Assuming that the logic of the function called in contract D includes judging the value of a global state d in the contract, and assuming that the value of this d is a Boolean type, the judgment logic includes, if it is true, execute branch 1, and if it is false, execute branch 2. Then this transaction that calls the contract needs to read the value of the state d when it is executed.
[0091] During the transaction execution process, whether it is reading the values of the state variables a and b as mentioned above, or reading the value of the state variable d, the execution module can first search whether the value of the state variable exists in the memory of the execution module. If not, a read instruction is sent to the storage module. The read instruction may include the address of the state to be read, that is, the key value of the state. After receiving the read instruction, the storage module finds the root node of the state tree node in the construction tree module, and matches the state key value to be searched with the shared nibble(s) field (for Extension Node) or slot (for Branch Node) of the root node from the beginning, and then reads the hash of the next layer of tree nodes (which can be Internal Node or Leaf Node, taking Internal Node as an example) from the matching position, and then first searches for the hash in the memory of the construction tree module. If it is not found, it searches in the memory of the state database module. If it is still not found, it needs to be searched in the file of the state database module. Specifically, you can search in the SST in Level 0 first. If you still can't find it, continue to search in a certain SST in Level 1. Continue to search in a certain SST in the lower level until you find the next Internal Node pointed to by the hash value. After unlocking this Internal Node, continue to match the remaining fields of the state address key value to be read from the front to the back. If it matches, read the hash value from the matching point and jump to the next level tree node pointed to by the hash value. Repeat this process, unlock the Internal Node level by level and match the remaining fields of the state address key value to be read from the front to the back. The hash value at the matching point is used as the basis for the next search for the intermediate node or leaf node until the Leaf node is matched, so as to read the value of the state from the Leaf node. In this process, some of the kvs of the tree nodes at each level may not exist in the memory of the tree construction module, but in the state database module. Therefore, it may be necessary to read from the state database module to the memory for operation. You can first search for this key in the MemTable or Immutable MemTable of the state database module. If it does not exist, you need to further search the SST files at all levels on the disk. For the Internal Node and Leaf node that exist on the disk, you need to load them into the memory and decode (deserialize) them to get the content. Specifically, for example, first search the sst files of level 0 one by one. If each SST of Level 0 is sorted, you can do a binary search in each SST based on the hash value of the intermediate node / leaf node. After finding it, load v into the memory and decode and match it.If the key is not found in the SST of Level 0, a binary search is performed in Level 1 based on the hash value of the intermediate node / leaf node. If the key is not found, a binary search is performed in Level 2. If the key is found, the v corresponding to the k is loaded into memory and decoded. This process is repeated until the leaf node of the tree structure is matched and the value is read from it. When searching in Level 1, Level 2, and lower levels, since the overall SST in these levels is sorted, a binary search can generally be performed in this level.
[0092] From the above process of reading a state kv, it can be seen that this reading process needs to perform binary search in SSTs of multiple different levels according to the hash value of the tree node. This may require reading a large number of files in SSTs of multiple levels and loading them into memory, which brings about the problem of read amplification. Assuming that the database includes Level 0, Level 1, Level 2, ..., Level M-1, a total of M layers, then in the worst case, reading the data of a state requires at least M accesses to the disk (greater than M because in the worst case, the N SSTs of Level 0 may require more than 1 and less than or equal to N searches).
[0093] In addition, if the tree building module lacks the kv of a certain tree node in the process of building the kv of all tree nodes on the path to the root node according to the state kv, it may also need to do a binary search from Level 0, Level 1, Level 2, ..., etc., and the principle is similar to the above process. In the above transaction example, for example, after the transaction is executed, the value of the execution result (af) / the value of (b+f) / the value of the contract state d needs to be written into the state database. For the above values to be written to the database, the execution module calls the storage interface of the storage module and outputs the key-value value of the state to the storage module. The tree building module in the storage module generates the tree node kv from the root of the tree structure to the intermediate node and the leaf node according to the state kv, and also writes these tree node kvs into the WAL module, and then writes this series of tree node kvs into the MemTable of the state database module and marks them as dirty data. Dirty data usually refers to data that has been written to the MemTable but has not yet been written to the persistent storage (such as SSTable). Dirty data can be written to disk in the aforementioned flush process. The state database module then writes some kvs in MemTable, including the tree node kv, into the SST file on disk. Once MemTable is persisted to SSTable, the data in it is no longer dirty data, that is, it becomes "clean" data. It can be seen that the above process of persistently storing a state kv is relatively complicated, consumes more computing and memory resources, and brings about the problem of write amplification.
[0094] The complexity of the above-mentioned reading and writing processes and the problems of read amplification and write amplification pose challenges to the high performance requirements and large-scale applications of blockchain.
[0095] The present application provides a method embodiment for reading blockchain state data. The method embodiment can be applied to a storage module of a blockchain node. The storage module constructs a world state with a tree structure in a memory. The leaf nodes of the tree structure may include the content of the state data or the position index of the state data in a data file in a persistent storage medium.
[0096] In an example, the tree structure as a whole can be Fig.10 As shown, this is also consistent with Figure 5 In particular, if the leaf node in memory contains state data, then Figure 5 As shown, for example Figure 5 The leaf node A11 in the memory has state data, that is, Zhang San_A=20. If there is no state data in the leaf node in the memory, the position index of the state data in the data file can be used.
[0097] In this application, the actual data and index are separated, which helps to improve the flexibility and performance of the system. The basic concepts involved here include data files and index files. Among them, the data file is a file that stores the actual content; the index file is an index that stores the location of the data file. Its working principle is: write the data content to the data file, record the location of the data in the file (such as file name, offset and length); then, create an entry in the index file, including: a key (a unique identifier for search) pointing to a reference to the data file (file name, offset, etc.). In this way, when retrieving data, the data location can be quickly located through the index file, and then the actual content can be read from the data file according to the location information. The advantages of this method are: ① flexibility, data and indexes can be managed and optimized independently; ② performance, indexes are usually small and can be fully loaded into memory to speed up queries; ③ space efficiency: avoid storing a large amount of duplicate data in the index; ④ update efficiency: when modifying data, only the location information in the index needs to be updated.
[0098] For example Fig.10 In the example, location is used to represent the location index. Still taking the A11 node as an example, if there is no state data in the memory, the location field included in it can be used to store the location index of the data file where the state data is located. The location index is, for example, <2,750,210>. This <2,750,210> can indicate that the state data is located in "file 2" in the data file, the starting position is the 750KB of "file 2", and the length is 210KB. Fig.10 As shown in the data file in the lower left corner, assuming that the size of each file is 1024KB, the logical address range of the index is 0KB to 1023KB. In this way, the status data of A11 actually occupies the range from 750KB to 959KB in file 2.
[0099] like Fig. 9 As shown, this embodiment includes:
[0100] S910: The storage module of the blockchain node receives an instruction for the state key to be searched, and searches for a leaf node corresponding to the state key on the tree structure in the memory.
[0101] As mentioned above Figure 6 As shown in , the execution module can generate an execution result by executing a normal transfer transaction / a transaction involving a contract, and send it to the storage module. Specifically, for example, the value of a state variable needs to be read during the execution of a contract. The execution module can first search the memory of the execution module to see whether the value of the state variable exists. If not, a read instruction is sent to the storage module. The read instruction may include the address of the state to be read, that is, the key value of the state.
[0102] The storage module includes a tree construction module. The tree construction module constructs a tree structure with the world state in the memory. If the blockchain node is cold started, that is, there is no tree structure of the world state in the memory, then the tree nodes of the tree structure can be loaded from the disk and organized into a tree in the memory. This process also involves loading the leaf nodes in the tree structure. Here, an example is taken to illustrate the case where there is an incomplete tree structure in the tree construction module. The case of cold start loading is similar. Since the capacity of the memory is generally limited, the LRU (Least Recently Used, a commonly used page replacement algorithm, selects the page that has not been used for the longest time to be eliminated) algorithm is generally used to eliminate tree nodes that have not been accessed for a long time from the memory. In this way, each leaf node on the tree structure constructed in the memory by the tree construction module has a certain possibility of being eliminated. For the leaf nodes of the world state tree, some may be in the memory and some may not be in the memory. In addition, for the content in the tree node, such as the state data in the leaf node, a similar elimination mechanism can also be used, or the state data may not be kept in the memory.
[0103] For the state key to be found, if the memory of the tree building module has the leaf node corresponding to the state key, it can be found directly in the memory. In this case, you can refer to Figure 5 , directly find the key of the corresponding leaf node in the memory, and then read the value from it. Then, the state data can be read from the value. If the memory of the tree building module does not have the leaf node corresponding to the state key, or even if there is the leaf node, the state data to be found does not exist in it, then it is necessary to further search from the state database module. For the case where the state database uses an LSM tree, as mentioned above, you can first search in the MemTable, and if it is not found, search in the file on the disk.
[0104] S920: Obtain a position index contained in a leaf node corresponding to the tree node key, where the position index points to a position in a data file in the persistent storage medium.
[0105] If the leaf node to be searched is present in the memory of the tree building module, but the state data does not exist in it, it is necessary to further search from the state database module. As mentioned above, if there is no state data in the memory, the location field included in it can be used, and its content is, for example, Fig.10 The location of the leaf node A11 is <2, 750, 210>. This <2, 750, 210> is the location index, which indicates that the status data is located in file 2 of the data file, starting at the 750KB and with a length of 210KB. Fig.10 As shown in the data file in the lower left corner, assuming that the size of each file is 1024MB, the logical address range of the index is 0KB to 1023KB. In this way, the status data of A11 actually occupies the range from 750KB to 959KB in file 2.
[0106] If the leaf node to be found is not in the memory of the tree building module, it is necessary to find and load the leaf node from the state database module. Finding and loading the leaf node from the state database module specifically includes loading the index file and data file of the leaf node. Assume Fig.10 The corresponding underlying database stores the kv of all intermediate nodes in the tree structure, as shown in the following table:
[0107]
[0108]
[0109] Table 2. Tree node kv actually stored in the database
[0110] In Table 2 above, similar to Table 1, H() is used to represent hash calculation. In this way, the hash value of the next tree node is anchored in the previous tree node. Through such layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and the root hash is locked in the state root field of the block header. Assume that the kv pairs in this table are stored in the underlying state database module, saved on disk, and use a similar LSM structure. In this way, similar to the above, after the storage module receives the read instruction, it can find the root node of the state tree node in the construction tree module, and match the state key value to be searched with the shared nibble(s) field (for Extension Node) or slot (for Branch Node) of the root node from the beginning, read the hash of the next layer of tree nodes from the matching position, and then, preferentially search for the hash in the memory of the construction tree module. If it has not been found, search in the memory of the state database module. If it still has not been found, it needs to be searched in the file of the state database module. Specifically, you can search in the SST in Level 0 first. If you still can't find it, continue to search in a certain SST in Level 1. Continue to search in a certain SST in the lower level until you find the next Internal Node pointed to by the hash value. After unlocking this Internal Node, continue to match the remaining fields of the key value of the state address to be read from the front to the back. If it matches, read the hash value from the matching point and jump to the next level tree node pointed to by the hash value. Repeat this process, unlock the Internal Node level by level and match the remaining fields of the key value of the state address to be read from the front to the back. The hash value at the matching point is used as the basis for the next search for the intermediate node or leaf node until the Leaf node is matched, so as to read the value of the state from the Leaf node. In this process, some of the kvs of the tree nodes at each level may not exist in the memory of the tree construction module, but in the state database module. Therefore, it may be necessary to read from the state database module. You can first search for this key in the MemTable or Immutable MemTable of the state database module. If it does not exist, you need to further search the SST files at all levels on the disk. For the Internal Node and Leaf node that exist on the disk, you need to load them into the memory and decode (deserialize) them to get the content. Specifically, for example, first search the sst files of level 0 one by one. If each SST of Level 0 is sorted, you can do a binary search in each SST based on the hash value of the intermediate node / leaf node. After finding it, load v into the memory and decode and match it.If the key is not found in the SST of Level 0, a binary search is performed in Level 1 according to the hash value of the intermediate node / leaf node. If the key is not found, a binary search is performed in Level 2. If the key is found, the v corresponding to the key is loaded into the memory and decoded. This process is repeated until the key is matched to the leaf node of the tree structure and the value is read from it. When searching in Level 1, Level 2, and lower levels, since the overall SST in these levels is sorted, a binary search can generally be performed in this level.
[0111] In this way, the tree node kv of the found leaf node can be loaded into the memory. For example, A11 loads the content prefix:2,Key-end:35b2e4,location:(2,750,210) in the leaf node value into the memory. The <2,750,210> is the position index of the value content in the file.
[0112] S930: Load the value of the data file location in the persistent storage medium pointed to by the location index into the memory and read the status data therein.
[0113] Still taking A11 as an example, the position index <2,750,210> points to the position in the data file in the persistent storage medium. In this way, the value of the position of the data file in the persistent storage medium pointed to by the position index <2,750,210> can be loaded into the memory and the status data therein can be read, that is, the data of file 2 in the data file starting from the starting position 750KB and the length of 210KB is loaded into the memory, thereby reading the status data therein, that is, "Zhang San_A=20".
[0114] Through the position index, you can directly locate the position in the data file, so that you can directly load the data file and read the status data from it.
[0115] In the case where the leaf node to be searched is present in the memory of the tree building module but the state data does not exist, it can be directly read from the corresponding data file according to the location of the leaf node in the memory. In this way, the kv of the leaf node is not stored in the persistent storage SST file, and the binary search process in the SST is required multiple times.
[0116] In the case where the leaf node to be searched does not exist in the memory of the tree building module, it is necessary to search and load the leaf node from the state database module. In fact, it is also necessary to read a certain amount of content from the disk into the memory before searching. In the implementation method corresponding to Table 1, the kv of the tree node is stored in the disk, and the value of the leaf node includes the state data, for example, H(11) in Table 1 corresponds to Zhang San_A=20 in the content, H(12) corresponds to Li Si_B=20 in the content, H(15) corresponds to storedData=s in the content, and H(16) corresponds to Wang Wu_A=20 in the content. Generally speaking, the space occupied by state data is relatively large, accounting for more than 90% of the total space of a kv. In the implementation method corresponding to Table 2, the index and data are separated. The index stores the location of the data file where the status data is located, but not the status data itself. For example, in Table 2, H(11) corresponds to location (2,750,210) in the content, H(12) corresponds to location (3,350,210) in the content, H(15) corresponds to location (5,760,140) in the content, and H(16) corresponds to location (5,170,210) in the content. The actual status is stored in the corresponding position in the data file, such as Fig.10 As shown in the data files and status data in . In the process of searching for leaf nodes, only index files can be searched. In this way, more index files can be loaded into the memory within the unit space for searching. The search efficiency of memory is significantly higher than that of disk. Generally, the content to be searched will be loaded into the memory before searching. The implementation method of Table 2 can find leaf nodes faster than the implementation method of Table 1 because more indexes can be loaded into the memory space of unit capacity. However, in the implementation method of Table 1, for example, after reading a batch of kv into the memory, if the leaf node is not found, another batch of kv needs to be re-read into the memory for searching again. Moreover, due to the state content that occupies a large space, the number of kv in the memory space of unit capacity is relatively small, so the number of times to read from the disk will generally be more frequent, and the reading speed of the disk is much slower than that of the memory. Therefore, the search efficiency of the implementation method of Table 1 is low.
[0117] The above combination Fig.10 Explained Fig. 9 The specific process of the method embodiment. Fig.10 The structure of the MPT tree is still two levels, and the structure of the intermediate nodes of each level of the tree includes two types of tree nodes: Extension Node and Branch Node. In a variation, a tree node such as Branch Node can be used as an intermediate node and a root node, for example Fig.11 As shown in . In this way, from the root node through the intermediate node to the leaf node, the sequential combination of the slots in the branch nodes passed through can constitute part of the content in the state key. The hash value of the next level node can be stored in the slot of the previous level node. Fig.11 In the example, a two-level tree structure can still be used. The leaf nodes in the upper tree can store account status (including the status of external accounts and contract accounts), and the leaf nodes in the lower tree can store the status of contracts. In the leaf nodes that store external account status, contract account status, or contract status, position indexes can be used. For example, the leaf nodes of the upper tree include account status in the following four rows: H(M1), H(M2), H(M3), and H(M4):
[0118]
[0119]
[0120] Table 3. Tree node kv actually stored in the database
[0121] Through the position index, you can directly locate the position in the data file, so that you can directly load the data file and read the status data from it. Furthermore, the contract status included in the leaf nodes of the next level tree is as shown in the four rows of H(Z1), H(Z2), H(Z3), and H(Z4) in the table above.
[0122] Data files such as Fig.11 As shown in , the content in the data file is as follows:
[0123] M1: Nonce: n1, balance: 45.0 ETH
[0124] M2: Nonce: n2, balance: 1.00WEI
[0125] M3: Nonce: n3, balance: 1.1ETH
[0126] M4:Nonce:n1,balance:0.12ETH,codehash:c1,storage:s1
[0127] Z1: Zhang San_A=20
[0128] Z2: Li Si_B = 50
[0129] Z3: Wang Wu_A=35
[0130] Z4: storedData = s
[0131] Whether it is the previous level tree or the next level tree, the location in the data file can be quickly located through the location in the leaf node. In this way, the kv of the leaf node is not stored in the persistent storage SST file, and the binary search process in the SST is required multiple times according to the key (hash value of the content) of the leaf node. Fig.11 In the example, according to Fig. 9 The blockchain state data can be read in the same way as in the previous step. According to the position index, the position in the data file can be directly located. Therefore, the content at the corresponding position in the data file can be directly loaded into the memory. This also avoids the read amplification problem caused by reading a large number of files in multiple levels of SST and loading them into the memory. Fig.10 The storage method shown in Fig.11 Stored in the same way.
[0132] For example, external accounts and contract accounts use fixed-length strings as keys, and the key of the contract status can be a string of the same length. Specifically, for example, the key of an external account or contract account can be an account address, while the key of the contract status can be a string of the same length. For another example, the key of an external account or contract account can be a hash value of an account address according to a certain hash algorithm, such as the sha3 algorithm; and the key of the contract status can be sha3 (contract name + location of the state variable declaration). The above-mentioned external accounts, contract accounts, and contract status are strings of the same length and format, collectively referred to as state keys. For example, the length of the state key is 4 bytes, or 32 bits. In this way, assuming that the root node (B0), intermediate nodes (..., C5, ..., C13, ..., C5D1, ..., C5D9, ..., C5D14, ..., C13D2, ..., C13D10, ...) are each a hexadecimal tree, then on the one hand, the slot of each intermediate node (such as 0, 1, 2, ..., d, e, f in B0, a total of 16 slots) can be filled with the hash value of the next tree node, and on the other hand, the position of the slot of this root node / intermediate node can represent a 4-bit key. 4 bits constitute a half-byte, so a total of 8 layers of root nodes and intermediate nodes can be loaded with 32 bits, a total of 8 bytes, that is, all state keys. The 32 bits of the state key can be stored in one half-byte of the key from the intermediate node at the top level to the intermediate node at the bottom level to the leaf node. In this way, for the case where the intermediate nodes are fixedly expanded, the last 4 bits of the key can be stored in the leaf node. In the case where the intermediate nodes are fixedly expanded, Fig.11 In the example, there are 8 layers from C* to C13D*, all the way to the bottom layer M*. In another case, the prefix and / or infix and / or suffix compression method can also be adopted, so that one or some leaf nodes plus the intermediate nodes above can be less than 8 layers.
[0133] In addition, a tree structure can also be used to store account status and contract status, for example Fig.12 As shown in . Since the key of the contract status is a hash value, based on the discrete characteristics of the hash algorithm, the status key of different contract status may be the path from the root node to different leaf nodes in this layer of tree structure. In this way, the value of different states of the same contract may exist in different leaf nodes at the bottom of the tree structure, and will no longer be gathered under the contract account to which it belongs. In fact, with a one-layer tree structure, the account status and contract status are scattered in the leaf nodes at the bottom.
[0134] In addition, Fig.12 On the basis of Fig.13 In the case of not many leaf nodes, prefix, infix and / or affix compression can be performed. In the case of infix compression, for example, the tree node key of M1 is 0x52d1479e (0x indicates that each bit is represented by a hexadecimal number), and the tree node key of Z1 is 0x52d9ab36, then M1 and Z1 have a common infix 2d. In the tree node C5, in addition to the 1 and 9 separated by the hexadecimal tree, C5 can also have a compressed infix 2d, such as Fig.13 In the case of suffix compression, for example, key-end is used to represent the key-end in the tree node M1, and key-end in the tree node Z1 is 36. By using prefix, infix and / or affix compression, the number of tree layers can be reduced.
[0135] Regardless of whether it is compressed or not, for the state key being searched, you can start from the root node of the tree structure and match the path of the state key segment by segment until you find the leaf node of the tree structure corresponding to the state key. In one way, the root node and the intermediate node are stored in a key-value format, where the key is the hash value of the value, similar to the structure in Table 3. In this way, you can match the hash of the path of the state key in order starting from the previous tree node, and search for the next tree node based on the hash until you find the leaf node of the tree structure corresponding to the state key.
[0136] In addition, the key of the tree node kv actually stored in the database may not be the hash value of the value, but a partial value on the state key path, such as the value of a partial key from the root node to the current tree node (combined with Fig.12 , without compression prefix / infix / suffix, as shown in Table 4), or the value of the partial key after the previous tree node (combined with Fig.13 , with compressed prefixes / infixes / suffixes, as shown in Table 5).
[0137]
[0138]
[0139] Table 4. Tree node kv actually stored in the database
[0140] According to Table 4, combined Fig.12 For example, if the state key to be searched is 0x59...36 (leaf node Z1), starting from the root node of the tree structure, the tree node with key 0x5 in the database can be matched according to the first digit 5 from the left of the state key (h(C5) in the value of the tree node with key 0x5 can be used to verify with the hash value on slot 5 in the root node. If they are the same, it means that the tree node 0x5 is the child node in slot 5 of the 0x tree node); further, the tree node with key 0x9 in the database can be matched according to the second digit 9 from the left of the state key (h(C5D9) in the value of the tree node with key 0x9 can be used to verify with the hash value on slot 9 in the parent node. If they are the same, it means that the tree node 0x9 is the child node of the 0x5 tree node); and so on. Finally, the last digit 6 of the state key can be matched to the tree node with key 0x6 in the database (h(Z1) in the value of the tree node with key 0x6 can be used to verify with the hash value on slot 6 in the parent node. If they are the same, it means that the tree node 0x6 is the child node of the parent node).
[0141]
[0142]
[0143] Table 5. Tree node kv actually stored in the database
[0144] According to Table 5, for example, if the state key to be searched is 0x52d9...36 (leaf node Z1), starting from the root node of the tree structure, the first bit 5 from the left of the state key can be matched in the database. If the same key is not matched, or the hash value in the value of the tree node that matches the key is inconsistent with h(C5) in the root node, then the state key is matched again in the database according to the first-second bits 52 from the left, and the cycle continues until a match is found. When the tree node with the key 0x52d is matched in the database according to the first-third bits 52d from the left of the state key, and the H(C5) in its value is the same as the hash value on slot 5 in the root node, it means that a match is found. Then, we can match the database according to the 1st to 4th bits 52d9 from the left of the state key. If the same key is not matched, or the hash value in the value of the tree node of the matched key is inconsistent with h(C5) in the root node, we can match the database according to the 1st to 2nd bits 52 from the left of the state key again. If a match is found, and H(C5D9) in the matched tree node value is the same as the hash value on slot 9 in the parent node, it means that the match is found. Furthermore, we match the database according to the 1st to 5th bits from the left of the state key, match the database according to the 1st to 6th bits from the left of the state key, and so on, until a leaf node is matched.
[0145] In another implementation, the complete state key or the corresponding complete state key can be stored in the leaf node. For example, the complete state key is stored in the leaf node. Fig.14 As shown in , the key-full field of the leaf node can store the complete key of the corresponding state. The key-value of the leaf node stored in the database is the complete key of the corresponding state of the leaf node. In this way, when searching for a state key, you can directly match the key of the leaf node in the database according to the key without matching the key of the intermediate node. Obviously, this search method is extremely efficient for query operations. In fact, the above Figure 10-13 In the example above, the complete state key can be used as the key of the leaf node.
[0146] Above Figure 9-14 In the embodiment, the location index location is stored in the leaf node, and the position of the value of the leaf node in the data file can be quickly located through the location index after the leaf node is found. In this way, the content of the corresponding position in the located data file can be directly loaded into the memory, which also avoids the read amplification problem caused by reading a large number of files in multiple Levels of SST and loading them into the memory. This also avoids the kv of the leaf node being stored in the persistently stored SST file, and the process of performing binary searches in the SST multiple times according to the key (hash value of the value content) of the leaf node. Moreover, similarly, for the leaf node to be found that is not in the memory of the tree construction module, the above embodiment can only search the index file, so that more index files can be loaded into the memory within the unit space for search, and the leaf node can be found faster.
[0147] The present application also provides a blockchain node, the blockchain node includes a storage module, the storage module includes a tree building module and a persistent storage, the persistent storage medium includes an index file and a data file; the storage module also includes a search unit and a reading unit; the leaf node of the tree structure includes a position index of the state data in the data file in the persistent storage medium;
[0148] The tree building module is used to build a world state of a tree structure in memory;
[0149] The search unit receives an instruction for a state key to be searched, searches for a leaf node corresponding to the state key on the tree structure in the memory; obtains a position index contained in the leaf node, the position index pointing to a position in a data file in the persistent storage medium;
[0150] The reading unit loads the value of the data file position in the persistent storage medium pointed to by the position index into the memory and reads the status data therein.
[0151] The search unit matches the path of the state key segment by segment starting from the root node of the tree structure until a leaf node of the tree structure corresponding to the state key is found.
[0152] The search unit matches the hash of the path of the state key in order starting from the tree node of the previous layer, and searches for the tree node of the next layer according to the hash, until a leaf node of the tree structure corresponding to the state key is found.
[0153] The search unit matches the keys of the path of the state key in order starting from the tree node of the previous layer, and searches for the tree node of the next layer according to the matched key until a leaf node of the tree structure corresponding to the state key is found.
[0154] When there is no status data in the leaf point in the memory, the position index contained in the leaf node is obtained.
[0155] When the leaf node does not exist in the memory, the tree node key of the leaf node is searched from the index file in the persistent storage, the value in the data file corresponding to the key is loaded into the memory, and the position index is obtained from the value.
[0156] The tree node key of the leaf node of the tree structure corresponds to the state key, then when the leaf node does not exist in the memory, the tree node key of the leaf node is searched from the persistent storage,
[0157] When the leaf node does not exist in the memory, the search unit searches for the tree node key of the leaf node in the persistently stored index file according to the state key.
[0158] The tree structure includes a one-layer tree structure or a two-layer tree structure.
[0159] In the 1990s, it was very clear whether the improvement of a technology was a hardware improvement (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements in the method flow today can be regarded as direct improvements in the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs. The original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0160] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0161] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0162] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.
[0163] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0164] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0167] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0168] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0169] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0170] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0171] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0172] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference term "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.
[0173] The above description is only an example of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims. < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1>
Claims
1. A method for reading blockchain state data, applied to a storage module of a blockchain node, wherein the storage module constructs a world state with a tree structure in a memory, and the leaf nodes of the tree structure include a position index of the state data in a data file in a persistent storage medium; the method comprises: The storage module of the blockchain node receives an instruction for the state key to be searched, and searches for a leaf node corresponding to the state key on the tree structure in the memory; Obtaining a location index contained in the leaf node, wherein the location index points to a location in a data file in the persistent storage medium; The value of the data file location in the persistent storage medium pointed to by the location index is loaded into the memory and the status data therein is read.
2. The method according to claim 1, wherein searching for a leaf node corresponding to a state key in the tree structure in the memory comprises: The path of the state key is matched piece by piece starting from the root node of the tree structure until a leaf node of the tree structure corresponding to the state key is found.
3. The method according to claim 2, wherein the path of matching the state key segment by segment starting from the root node of the tree structure comprises: The hash of the path of the state key is matched in order starting from the tree node of the previous layer, and the tree node of the next layer is searched according to the hash until the leaf node of the tree structure corresponding to the state key is found.
4. The method according to claim 2, wherein the path of matching the state key segment by segment starting from the root node of the tree structure comprises: Starting from the tree node of the previous layer, the keys of the path of the state key are matched in order, and the tree node of the next layer is searched according to the matched key until the leaf node of the tree structure corresponding to the state key is found.
5. The method according to any one of claims 1 to 4, wherein when there is no status data in the leaf point in the memory, the position index contained in the leaf node is obtained.
6. According to the method described in any one of claims 1-4, when the leaf node does not exist in the memory, the tree node key of the leaf node is searched from the persistent storage, the value corresponding to the key is loaded into the memory, and the position index is obtained from the value.
7. The method of claim 6, wherein the tree node key of the leaf node of the tree structure corresponds to the state key, and searching the tree node key of the leaf node from the persistent storage when the leaf node does not exist in the memory comprises: When the leaf node does not exist in the memory, the tree node key of the leaf node is searched in the persistent storage according to the state key.
8. The method according to any one of claims 1 to 7, wherein the tree structure comprises a one-layer tree structure or a two-layer tree structure.
9. A blockchain node, the blockchain node comprising a storage module, the storage module comprising a tree building module and a persistent storage, the persistent storage medium comprising an index file and a data file; the storage module further comprising a search unit and a reading unit; the leaf node of the tree structure comprises a position index of the state data in the data file in the persistent storage medium; The tree building module is used to build a world state of a tree structure in memory; The search unit receives an instruction for a state key to be searched, searches for a leaf node corresponding to the state key on the tree structure in the memory; obtains a position index contained in the leaf node, the position index pointing to a position in a data file in the persistent storage medium; The reading unit loads the value of the data file position in the persistent storage medium pointed to by the position index into the memory and reads the status data therein.
10. In the blockchain node as described in claim 9, the search unit matches the path of the state key segment by segment starting from the root node of the tree structure until a leaf node of the tree structure corresponding to the state key is found.
11. In the blockchain node as described in claim 10, the search unit matches the hash of the path of the state key in sequence starting from the tree node of the previous layer, and searches for the tree node of the next layer according to the hash until the leaf node of the tree structure corresponding to the state key is found.
12. In the blockchain node as described in claim 10, the search unit matches the key of the path of the state key in sequence starting from the tree node of the previous layer, and searches for the tree node of the next layer according to the matched key until the leaf node of the tree structure corresponding to the state key is found.
13. A blockchain node as described in any one of claims 9 to 12, when there is no state data in the leaf point in the memory, obtaining the location index contained in the leaf node.
14. A blockchain node as described in any one of claims 9 to 12, when the leaf node does not exist in the memory, searches for the tree node key of the leaf node from the index file in the persistent storage, loads the value in the data file corresponding to the key into the memory, and obtains the position index from the value.
15. The blockchain node as claimed in claim 14, wherein the tree node key of the leaf node of the tree structure corresponds to the state key, and when the leaf node does not exist in the memory, the tree node key of the leaf node is searched from the persistent storage, When the leaf node does not exist in the memory, the search unit searches for the tree node key of the leaf node in the persistently stored index file according to the state key.
16. The blockchain node according to any one of claims 9 to 15, wherein the tree structure comprises a one-layer tree structure or a two-layer tree structure.
Citation Information
Patent Citations
Data storage method and device
CN115374127A
Block chain-based state data storage method and device
CN116610670A
Database index processing method, device and equipment and readable storage medium
CN118051502A
Systems, methods, and apparatuses for implementing efficient storage and validation of data and metadata within a blockchain using distributed ledger technology (DLT)
US20200250174A1
