Method for constructing tree structure-based world state, and computer device

Abstract

A method for constructing a tree structure-based world state, and a computer device. The method comprises: receiving a state key-value; on the basis of a construction rule of a tree structure, generating / updating leaf nodes of the tree structure corresponding to the state key-value; on the basis of content in each leaf node, calculating a hash value of the leaf node, and on the basis of the hash value of the leaf node and a state key, generating or updating an upper-level tree node until a root node of a tree; and updating the version of the generated / updated tree node in a corresponding memory page on the basis of a divided logical page structure, wherein the memory page is used for representing the content of all the corresponding tree nodes in a logical page, the logical page comprises a page identifier, and the page identifier comprises the lexicographical order content and version of the upper-level tree node in the logical page.

Description

A method and computer device for constructing a tree-based world state

[0001] This application claims priority to Chinese Patent Application No. 2024109809138, filed on July 19, 2024, entitled "A Method and Computer Device for Constructing a World State Based on a Tree Structure", the entire contents of which are incorporated herein by reference. Technical Field

[0002] The embodiments in this specification belong to the field of blockchain technology, and in particular relate to a method and computer device for constructing a world state based on a tree structure. Background Technology

[0003] Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. In a blockchain system, data blocks are sequentially linked together to form a chain-like data structure, and a distributed ledger is cryptographically guaranteed to be immutable and unforgeable. Due to its decentralized, immutable, and autonomous characteristics, blockchain is receiving increasing attention and application.

[0004] Summary of the Invention

[0005] The purpose of this invention is to provide a method and computer device for constructing a tree-based world state, comprising:

[0006] A method for constructing a tree-structured world state, applied to the storage module of a blockchain node, wherein the storage module constructs a tree-structured world state in memory, the method comprising:

[0007] Receive status key-value;

[0008] According to the tree structure construction rules, generate / update the leaf nodes of the tree structure corresponding to the state key-value; calculate the hash value of the leaf node based on the content of the leaf node, and generate or update the upper-level tree nodes based on the hash value of the leaf node and the state key, until the root node of the tree;

[0009] The generated / updated tree nodes are updated with versions in the corresponding memory pages according to the divided logical page structure. The memory pages represent the contents of all tree nodes in the logical pages. The logical pages contain page identifiers, which include the lexicographical order and version of the upper-level tree nodes in the logical pages.

[0010] A computer device includes: a processor; and a memory storing a program, wherein when the processor executes the program, it performs any of the operations described above.

[0011] The above embodiments describe a scheme for partitioning the world state of a tree structure according to Pages. The tree structure is divided into LogicalPages, each of which manages MemoryPage, BasePage, and DeltaPage. Each state tree modification is optimized into multiple version modifications at the LogicalPage granularity, reducing overall write amplification. The current version can be directly obtained through memory pages, while historical versions can be obtained by replaying the BasePage and several associated DeltaPages. The entire MemoryPage is cached in memory, without affecting contract execution performance. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 is a schematic diagram of deploying a smart contract in one embodiment;

[0014] Figure 2 is a schematic diagram of calling a smart contract in one embodiment;

[0015] Figure 3 is a schematic diagram of a block storage structure in one embodiment;

[0016] Figure 4 is a schematic diagram of a block storage structure in one embodiment;

[0017] Figure 5 is a schematic diagram of an MPT tree in one embodiment;

[0018] Figure 6 is a schematic diagram of the modules involved in the transaction processing process and the relationship between CPU, memory and disk in one embodiment;

[0019] Figure 7 is a schematic diagram of the modules involved in the transaction processing process and the relationship between CPU, memory and disk in one embodiment;

[0020] Figure 8 is a schematic diagram of the logical relationship of the state database in one embodiment;

[0021] Figure 9 is a flowchart of reading status data in one embodiment;

[0022] Figure 10 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure;

[0023] Figure 11 is a schematic diagram illustrating the reading of status data in conjunction with a tree structure in one embodiment;

[0024] Figure 12 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure;

[0025] Figure 13 is a schematic diagram illustrating the reading of status data in one embodiment using a tree structure;

[0026] Figure 14 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure;

[0027] Figure 15 is a schematic diagram of a block storage structure in one embodiment;

[0028] Figure 16 is a schematic diagram of a block storage structure in one embodiment;

[0029] Figure 17 is a schematic diagram of a block storage structure in one embodiment;

[0030] Figure 18 is a schematic diagram of a block storage structure in one embodiment. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0032] Blockchains are generally classified into three types: public blockchains, private blockchains, and consortium blockchains. In addition, there are various combinations of these types, such as private blockchains + consortium blockchains, consortium blockchains + public blockchains, etc. Public blockchains offer the highest degree of decentralization. Represented by Bitcoin and Ethereum, public blockchains allow participants to read data records on the chain, participate in transactions, and compete for the right to record new blocks. Furthermore, participants (represented by nodes on the blockchain) can freely join and leave the network and perform related operations. Private blockchains, on the other hand, have write permissions controlled by a specific organization or institution, and data read permissions are regulated by that organization. Simply put, a private blockchain can be a weakly centralized system with strictly limited and few participating nodes. This type of blockchain is more suitable for internal use within specific institutions. Consortium blockchains fall between public and private blockchains, achieving "partial decentralization." Each node in a consortium blockchain typically has a corresponding entity or organization; participants join the network through authorization and form a consortium of stakeholders to jointly maintain the operation of the blockchain.

[0033] Whether it's a public, private, or consortium blockchain, besides supporting the transfer of native assets on the blockchain between accounts, such as transferring Bitcoin and Ethereum, it can also provide smart contract functionality. A smart contract on the blockchain is a contract that can be triggered and executed by transactions on the blockchain system. Smart contracts can be defined in the form of code.

[0034] Taking Ethereum as an example, enabling users to create and invoke complex logic within the Ethereum network is the biggest challenge that distinguishes Ethereum from Bitcoin's blockchain technology. At the core of Ethereum as a programmable blockchain is the Ethereum Virtual Machine (EVM), which can be run by every Ethereum node. The EVM is a Turing-complete virtual machine, meaning that various complex logics can be implemented through it. Users publishing and invoking smart contracts in Ethereum run on the EVM. In fact, the virtual machine directly runs virtual machine code (virtual machine bytecode, hereinafter referred to as "bytecode"). Smart contracts deployed on the blockchain can be in the form of bytecode.

[0035] For example, as shown in Figure 1, after Bob sends a transaction containing information about creating a smart contract to the Ethereum network, Node 1's EVM can execute this transaction and generate the corresponding contract instance. In Figure 1, "0x6f8ae93…" represents the address of this contract. The transaction's data field can store bytecode, and the transaction's to field is an empty account. After the nodes reach a consensus through the consensus mechanism, the contract is successfully created, and users can subsequently call this contract.

[0036] After a contract is created, a contract account corresponding to that smart contract is generated on the blockchain, with a specific address. The contract code and account storage are stored in this contract account. The behavior of the smart contract is controlled by the contract code, while the smart contract's account storage preserves the contract's state. In other words, a smart contract enables the creation of a virtual account on the blockchain that contains both contract code and account storage.

[0037] As mentioned earlier, the `data` field of a transaction that creates a smart contract can store the bytecode of that smart contract. Bytecode consists of a series of bytes, with each byte representing an operation. For reasons of development efficiency and readability, developers can choose to write smart contract code in a high-level language instead of directly writing bytecode. The smart contract code written in a high-level language is compiled by a compiler to generate bytecode, which can then be deployed to the blockchain. Ethereum supports many high-level languages, such as Solidity, Serpent, and LLL.

[0038] Taking Solidity as an example, contracts written in it are very similar to classes in object-oriented programming languages. A contract can declare various members, including state variables, functions, function modifiers, events, etc. State variables are values ​​stored in the smart contract's account store and are used to save the contract's state.

[0039] The following is a simple code example 1 of a smart contract written in the Solidity language:

[0040] Furthermore, as shown in Figure 2, still using Ethereum as an example, after Bob sends a transaction containing information about calling a smart contract to the Ethereum network, the EVM of node 1 can execute this transaction and generate the corresponding contract instance. In Figure 2, the `from` field of the transaction is the address of the account initiating the smart contract call, and "0x6f8ae93…" in the `to` field represents the address of the called smart contract. The `data` field of the transaction stores the method and parameters for calling the smart contract. Additionally, it may include a `value` field to represent the value of Ether in the transaction. After calling the smart contract, the value of `storedData` may change. Subsequently, a client can view the current value of `storedData` through a specific blockchain node (e.g., node 6 in Figure 2).

[0041] Smart contracts can be executed independently on each node of the blockchain network in a prescribed manner. All execution records and data are stored on the blockchain. Therefore, when such a transaction is completed, the blockchain stores an immutable and unlost transaction certificate.

[0042] As mentioned earlier, `storedData` in the example above is the state variable, which is stored in the smart contract's account storage. In various blockchain networks that introduce smart contracts, such as Ethereum, accounts typically include two types:

[0043] Contract account: Stores the executed smart contract code and the values ​​of the state within the smart contract code; it can typically only be activated by an external account.

[0044] Externally owned account: A user's account, such as an Ethereum owner's account.

[0045] The design of external and contract accounts is essentially a mapping from account addresses to account states. Account states typically include fields such as Nonce, Balance, Storage root, and CodeHash. Nonce and Balance exist in both external and contract accounts. CodeHash and Storage root attributes are generally only valid for contract accounts.

[0046] Nonce: A counter. For external accounts, this number can represent the number of transactions sent from the account address; for contract accounts, it can be the number of contracts created by the account.

[0047] Balance: The amount of Ether held by this address.

[0048] Storage root: The hash of the root node of an MPT tree, which organizes the storage of state variables for contract accounts.

[0049] CodeHash: The hash value of the smart contract code. For contract accounts, this is the hash value of the smart contract; for external accounts, since the smart contract is not included, the CodeHash field can generally be an empty string or a string of all zeros.

[0050] MPT stands for Merkle Patricia Tree, a tree structure that combines the features of Merkle Tree and Patricia Tree (a compressed prefix tree, a more space-efficient Trie tree, also known as a dictionary tree). The Merkle Tree algorithm calculates a hash value for each transaction, then joins each pair of transactions and calculates the hash again, continuing until the top-level Merkle root. Ethereum uses an improved MPT tree, such as a 16-ary tree structure, which is often simply referred to as an MPT tree.

[0051] The Ethereum MPT tree data structure includes a state trie. The state trie contains key-value pairs (kv) representing the storage content for each account in the Ethereum network. The "key" in the state trie can be a 160-bit identifier (such as the address of an Ethereum account or part of its hash value, hereinafter referred to as the account address), distributed across storage from the root node to the leaf nodes of the state trie. The "value" in the state trie is generated by encoding the information of the Ethereum account using recursive-length prefix encoding (RLP). As mentioned earlier, for external accounts, the value includes nonce and balance; for contract accounts, the value includes nonce, balance, codehash, and storageroot.

[0052] A contract account is used to store the state of a smart contract. Once a smart contract is deployed on the blockchain, a corresponding contract account is created. This contract account typically has several states defined by state variables within the smart contract, which generate new values ​​during the smart contract's creation and execution. A smart contract generally refers to a contract defined in code within a blockchain environment that can automatically execute its terms. Once an event triggers a term in the contract (meeting the execution conditions), the code can execute automatically. In the blockchain, the contract's state is stored in a storage trie. The hash value of the storage trie's root node is stored in the aforementioned storage root, thus locking all the contract's states to this contract account through hashing. The storage trie is also an MPT tree structure, storing a key-value mapping from state addresses to state values. Information from the root node to the leaf node of the storage trie tree is arranged sequentially to store the address of a state, and the leaf node stores the state's value.

[0053] As shown in Figure 3, in some blockchain data storage, each block header includes several fields, such as the previous block hash (prev_Hash in the figure), a nonce (in some blockchain systems, this nonce is not random, or the nonce in the block header is not enabled), a timestamp, a block number, a state root hash (State_Root), a transaction root hash (Transaction_Root), and a receipt root hash (Receipt_Root). The Prev Hash in the header of the next block (e.g., block N+1) points to the previous block (e.g., block N), which is the hash value of the previous block. In this way, the blockchain achieves the locking of the previous block by the next block through the block header. Specifically, State_Root, Transaction_Root, and Receipt_Root lock the state set, transaction set, and receipt set, respectively. The state set, transaction set, and receipt set are organized in a tree structure. Generally, they can be the same tree structure or different tree structures. For example, Ethereum uses the same MPT structure. In some tree structures like Ethereum that include a set of smart contract states, there is a two-level MPT structure: the leaf nodes of the upper-level MPT structure include two types: external accounts and contract accounts; each contract account includes the lower-level MPT structure, and the leaf nodes of the lower-level structure include the values ​​of the state in the contract account.

[0054] Figure 4 is a schematic diagram of a blockchain data storage structure. Taking Ethereum as an example, and referring to Figure 3, `state_root` is the hash value of the root of the MPT tree composed of the states of all accounts in the current block. In other words, pointing to `state_root` is an MPT-form state trie. The root node of this MPT tree is generally an extension node or a branch node, and `state_root` typically stores the hash value of this root node. The root node can be connected to one or more layers of extension nodes / branch nodes below it; these multi-layered tree nodes can be collectively referred to as internal nodes. A portion of the values ​​from each node from the root node to the leaf node in this MPT can be concatenated sequentially to form an account address as the key. The account information stored in the leaf node is the value corresponding to this account address, thus forming a key-value pair. This key can also be a partial SHA3 hash of the address, i.e., a portion of the hash value of the account address (using a hash algorithm such as SHA3). The stored value can be an RLP (Account), i.e., the RLP encoding of the account information. The account information is a four-tuple consisting of [nonce, balance, storageRoot, codeHash]. As mentioned earlier, for external accounts, generally only the nonce and balance fields are present, while the storageRoot and codeHash fields default to storing empty strings / strings of all zeros. That is, external accounts do not store contracts or the state variables generated after contract execution. Contract accounts generally include Nonce, Balance, Storage root, and CodeHash. Nonce is the transaction counter for the contract account; Balance is the account balance; Storage root corresponds to another MPT, which links to contract-related state information; CodeHash is the hash value of the contract code. Whether external or contract accounts, their account information is generally located in a single leaf node. From the root node's Extension Node / Branch Node to each account's Leaf Node, there may be several branch nodes and extension nodes in between.

[0055] The state trie can be an MPT-style tree, typically a 16-ary tree, meaning each level can have a maximum of 16 child nodes. An Extension Node, used to store a common prefix, typically has one child node, which can be a Branch Node. A Branch Node can have a maximum of 16 child nodes, which may include Extension Nodes and / or Leaf Nodes.

[0056] For a contract account within the state trie, its `storage_root` points to another tree, also in MPT form, which stores data related to the contract's execution state variables. This `storage_root` points to an MPT-form tree called the Storage Trie, specifically the hash value of the root node. Generally, this Storage Trie also stores key-value pairs. The key represents the address of the state variable, and its value can be the result of processing the state variable's declaration position (counting from 0) according to certain rules, such as SHA3(state variable declaration position) or SHA3(contract name + state variable declaration position). The value stores the state variable's value (e.g., an RLP-encoded value). A portion of the data stored along the path from the root node through intermediate nodes to the leaf node forms the key, and the leaf node stores the value. As mentioned earlier, this Storage trie can also be an MPT-type tree, typically a 16-ary tree. This means that a Branch Node can have a maximum of 16 child nodes, which may include Extension Nodes and / or Leaf Nodes. An Extension Node typically has one child node, which can be either a Branch Node or a Leaf Node.

[0057] For example, Leaf Node Account P in the state trie in Figure 4 is a contract account whose Storage Root locks all the states stored in the contract. These states are organized as an MPT tree, with a tree structure like the Storage trie linked by the Storage Root. In this linked Storage trie, taking Leaf Node State Variable N as an example, if it is the value of storedData in the aforementioned contract code example, then its key is sha3 (the declaration location of storedData, i.e., line 2 of the code), and its value is s (for simplicity, the encoding format of the value is omitted here, such as RLP, and similarly, it will not be elaborated further). The key values ​​are distributed sequentially from the root node to the leaf node (i.e., Leaf Node Variable N) of the storage trie.

[0058] For example, in the state Trie in Figure 4, Leaf Node Account C is an external account. Its key is sha3(Address C), which is the hash value of the address of account C (using a hash algorithm such as sha3). Its stored value can be (Account), where the account information Account is a tuple of [nonce, balance]. As mentioned earlier, since Account C is an external account, its account information consists of nonce and balance (codehash and storage root are omitted here, but similarly below). For example, if an external account has a nonce of 20 and a balance of 4550, then the leaf node State Variable C will store nonce = 20 and balance = 4550. The address of Account C is the key, and its values ​​are distributed sequentially from the root node to the leaf node (i.e., Leaf Node Variable C) of the state Trie.

[0059] These states, including the key-value pairs of external accounts and contract accounts, are ultimately stored in the database. However, the database does not directly store the states of these accounts (i.e., it does not directly store their key-value pairs), but rather stores the key-value values ​​of each tree node itself.

[0060] As shown in the example in Figure 5, in the upper-level MPT structure, for leaf node A1, the key of the leaf node is formed by sequentially combining a7 of the shared nibble in the root node A8 (Extension Node), slot 1 of the intermediate node A7 (Branch Node), and key-end 1335 in leaf node A1, which is a711335. Balance = 45.0ETH and Nonce = n1 are stored in this leaf node. For leaf node A2, the key of the leaf node is formed by sequentially combining a7 of the shared nibble in the root node A8 (Extension Node), slot 7 of the intermediate node A7 (Branch Node), d3 of the shared nibbles in node A6 (Extension Node), slot 3 of the intermediate node A5 (Branch Node), and key-end 7 in leaf node A2, which is a77d337. Balance = 1.00ETH and Nonce = n2 are stored in this leaf node. For leaf node A3, the key of this leaf node is formed by sequentially combining a7 from the shared nibble in root node A8 (Extension Node), slot f from intermediate node A7 (Branch Node), and key-end 9365 in leaf node A3, resulting in a7f9365. This leaf node stores Balance = 1.1 ETH and Nonce = n3. For leaf node A4, the key of this leaf node is formed by sequentially combining a7 from the shared nibble in root node A8 (Extension Node), slot 7 from intermediate node A7 (Branch Node), slot d3 from the shared nibbles in node A6 (Extension Node), slot 9 from intermediate node A5 (Branch Node), and key-end 7 in leaf node A4, resulting in a77d397. This leaf node stores Balance = 0.12 ETH, Nonce = n4, CodeHash = c1, and Storage root = s1. s1 can be H(A10), which is the hash of the root node A10 of the next level tree. The leaf nodes of A1, A2, and A3 store information about external accounts, while the leaf node of A4 stores information about contract accounts. For each contract account, there is a next-level MPT, forming a Storage Trie used to store the state variables of that contract account.

[0061] As shown in the example of Figure 5, in the next-level MPT structure, for leaf node A11, through slot 3 in root node A10 (Branch Node) - key-end 35b2e4 in leaf node A11, they are sequentially combined to form the key of this leaf node, which is 335b2e4. In this leaf node, "Zhang San_A = 20" is stored. For example, it means that the share of type A digital assets defined in the contract belonging to Zhang San is 20, that is, the balance of Zhang San's type A assets is 20. For leaf node A12, through slot 7 in root node A10 (Branch Node) - key-end c25988 in leaf node A12, they are sequentially combined to form the key of this leaf node, which is 7c25988. In this leaf node, "Li Si_B = 20" is stored. For example, it means that the share of type B digital assets defined in the contract belonging to Li Si is 50, that is, the balance of Li Si's type B assets is 50. For leaf node A15, through slot f in root node A10 (Branch Node) - shared nibble a in intermediate node A13 (Extension Node) - slot 6 in intermediate node A14 (Branch Node) - key-end be33 in leaf node A15, they are sequentially combined to form the key of this leaf node, which is fa6be33. In this leaf node, "storedData = s" is stored. For leaf node A16, through slot f in root node A10 (Branch Node) - shared nibble a in intermediate node A13 (Extension Node) - slot 9 in intermediate node A14 (Branch Node) - key-end 9365 in leaf node A16, they are sequentially combined to form the key of this leaf node, which is fa99365. In this leaf node, "Wang Wu_A = 35" is stored. For example, it means that the share of type A digital assets defined in the contract belonging to Wang Wu is 35, that is, the balance of Wang Wu's type A assets is 35.

[0062] In the node composition of the above MPT tree, the tree node type is represented by prefix. For example, 0 represents an Extension Node containing an even number of shared nibbles, 1 represents an Extension Node containing an odd number of shared nibbles, 2 represents a Leaf Node containing an even number of nibbles, and 3 represents a Leaf Node containing an odd number of nibbles.

[0063] In the above node structure, the hash value of the entire content of the next tree node is filled into the corresponding position of the previous tree node. The database actually stores the key-value mapping for each tree node, where the value includes the content stored in that tree node, and the corresponding key is the hash value of the entire content of that tree node. Thus, the actual key-value pairs of the tree nodes stored in the database are shown in the following table:

[0064] Table 1. Key-value pairs of tree nodes actually stored in the database.

[0065] In Table 1 above, H() represents hash calculation. This anchors the hash value of the next tree node to the previous tree node. Through this layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and this root hash is locked into the state root field of the block header.

[0066] In some blockchain systems, the blockchain platform's code may include P2P (Peer-to-Peer) modules, consensus modules, execution modules, and storage modules. P2P is a way of organizing computer networks; unlike common web networks, P2P is decentralized and distributed. P2P modules enable the distributed propagation of data. For blockchain nodes, P2P modules allow for the peer-to-peer transmission and reception of receipts. Different participants can establish a distributed blockchain network through deployed nodes. The ledger constructed using a chain-like block structure is stored on each node (or most nodes, such as consensus nodes) in the distributed blockchain network; this is also known as a decentralized (or multi-centralized) distributed ledger. Such blockchain systems need to address the issue of consistency and correctness of the ledger data across multiple decentralized (or multi-centralized) nodes. Each node runs the same blockchain platform program. With a certain level of fault tolerance, the consensus module ensures that all loyal nodes have the same transactions, guaranteeing consistent execution results for the same transactions. The transactions and execution results are then packaged into blocks. Current mainstream consensus mechanisms include Proof of Work (POW), Proof of Stake (POS), Delegated Proof of Stake (DPoS), Practical Byzantine Fault Tolerance (PBFT), and Honey Badger Byzantine Fault Tolerance (HoneyBadgerBFT). During the consensus process, the consensus module can also generate timestamps for the blocks corresponding to the current transaction set. The execution module can execute transactions, including ordinary transfers and transactions involving contracts, either before or after the consensus module completes consensus. For transactions involving contracts, the execution module can introduce a virtual machine, such as the Ethereum Virtual Machine (EVM), to execute the smart contract code. This EVM shields the differences in hardware configuration and software environment across nodes, ensuring that the process and results of smart contract execution are identical on all nodes. Furthermore, the sandbox environment prevents smart contract execution from impacting the blockchain platform code, other programs, or the operating system on the host machine. In one scenario for a consortium blockchain, nodes can determine the content and order of transactions in a transaction set through a consensus module, and then output a deterministic transaction set as the consensus result to the execution module. The execution module generates execution results by executing ordinary transfer transactions or transactions involving contracts, and sends them to the storage module. The storage module is responsible for storing the execution results in persistent storage media on the node's local machine.

[0067] As shown in Figure 6, a blockchain node physically includes a CPU, memory, and disk. The blockchain platform code executed by this node can include a P2P module, a consensus module, an execution module, and a storage module. The P2P, consensus, and execution modules generally require CPU and memory to function. The storage module can include a tree construction module, a block header generation module, a WAL (Write Ahead Log) module, and a state database module. The tree construction module is used to build a tree (e.g., an MPT tree) based on the state key-value pairs passed from the execution module, such as the aforementioned state trie and storage trie, thus obtaining the key-value pairs of the tree nodes; this generally requires CPU and memory. The block header generation module is used to generate block headers based on the root node of the tree constructed by the tree construction module and other data (such as the previous block hash, timestamp, block number, etc.); this generally requires CPU and memory. The WAL module persistently stores the key-value pairs (kv) of the leaf nodes of the tree generated by the tree-building module before writing them to the state database module. This prevents data loss due to power outages or other unforeseen circumstances during the writing process and allows for data recovery in such cases. It typically requires the participation of CPU, memory, and disk. The state database module stores the key-value pairs of the tree nodes (kv) constructed by the tree-building module, as shown in Table 1, on a persistent storage device. Since the tree node data will ultimately be written to persistent storage media (such as the disk in the diagram), the state database module generally requires the participation of disk in addition to CPU and memory.

[0068] From a storage structure perspective, the aforementioned Merkle tree structures, such as Ethereum's MPT and Libra's SMT (Sparse Merkle Tree, similar to MPT), can reside in the tree construction module and be stored in memory, according to the correspondence shown in Table 1. The upper-level Merkle tree is a prefix tree (prefix tree), capable of organizing data and providing a unique Merkle root for the organized data. Leaf nodes store state values, and the root node, intermediate nodes, and leaf nodes provide a lexicographical index of the state keys. These tree nodes are encoded as keys according to certain rules, and their contents are encoded as values, ultimately stored in the lower-level database. The database mostly adopts an LSM (Log-Structured Merge-Tree) type NoSQL Key-Value DB (Database; Key-Value DB is also simply KVDB), located in the state database module and ultimately stored on disk. Specific examples include Ethereum's LevelDB and Libra's RocksDB. Both of these KVDBs are based on the LSM storage engine.

[0069] The LSM storage engine is a tiered, ordered, disk-oriented storage engine that borrows from the append-only (rather than modify) nature of the Log. Its core idea is to fully leverage the fact that sequential, batch writes to disk are far more efficient than random writes, sacrificing some read efficiency for maximum write efficiency. Generally, maximizing disk performance involves reading or writing a fixed-size block of data at a time, minimizing random access operations. LSM's design is based on this disk characteristic and assumes sufficient memory. Instead of writing data to disk every time it's updated, it first keeps the latest data in memory. Once a sufficient amount of data has accumulated, it uses merge sort to combine the data in memory with the data on disk and appends it to disk in batches.

[0070] The following uses Figure 7 as an example to further illustrate the storage process of the state data module. After the tree building module generates tree node key-value data and sends it to the storage module, this key-value data is first written to the Write-Ahead Log (WAL) in an append-only manner and stored on the disk. Then, the tree node key-value data is passed to the state database module. As mentioned earlier, writing to the WAL first prevents data loss due to power outages or other events during the writing of the tree node key-value data generated by the tree building module to the state database module, and allows for data recovery in such cases. After receiving the tree node key-value data, the state database module first stores it in memory, specifically in a MemTable within memory. The MemTable can provide concurrent read and write operations. Multiple MemTables can exist in memory. When the amount of data in a MemTable reaches a certain threshold, such as 256MB, the data in the MemTable can be flushed to the disk. To avoid write operations to the MemTable blocking the flush process, this MemTable is converted into an immutable Memtable, meaning the Immutable Memtable is set to read-only, and a new MemTable is created to receive the newly passed tree node key-value pairs. This new MemTable can provide concurrent read and write operations (as shown in the New MemTable diagram). The storage engine writes the data in the Immutable MemTable to disk.

[0071] On disk, KVDB stores data in multiple levels of SST (Sorted String Table) files. SST originates from the Google Bigtable paper. Bigtable is a distributed data storage system designed by Google to handle massive amounts of data; it's a non-relational database. SST can be used to store data files within Bigtable and is an efficient key-value file storage format. Content stored on persistent storage media can also be called a file.

[0072] SSTs can include multiple layers, such as 3, 4, 5, 6, 7, or more. Figure 7 shows a 3-layer case, namely level 0, level 1, and level 2. Generally, the total capacity of the SST in the upper layer is significantly smaller than that in the lower layer. For example, the total capacity of the SST in level 0 is 1GB, the total capacity of the SST in level 1 is 10GB, and the total capacity of the SST in level 2 is 100GB. Assume the capacity of the MemTable is 256MB. Tree construction modules can be written using the key-value pairs (kv) of the tree nodes corresponding to blocks, and then written to the MemTable. After writing the tree kv corresponding to one or more blocks to the MemTable, the space occupied by the MemTable may reach 256MB. Then, this MemTable is converted into an Immutable MemTable, and the data in the Immutable MemTable can be written (flush, which refers to the operation of writing data in memory to disk) to disk. Specifically, the data in the MemTable can be flushed to the Level 0 SST on disk. On the other hand, as mentioned earlier, a new MemTable is generated to receive the newly passed tree node kv and to provide concurrent read and write operations.

[0073] As mentioned earlier, each level of disk storage can include multiple SST files. The key-value pairs stored in the SSTs of Level 1 and lower levels are generally stored in key order, and the key values ​​in different SSTs within the same level are also ordered and unique. For example, Level 1 includes three SSTs, the leftmost one... 11 It includes 1000 kV, for example, <k1-v1> <k2-v2> <k3-v3>... <k 1000 -v 1000 > and arrange them in ascending order of k, for example, k1=10b43c032a46, k2=1118d46f41e1, k3=1b44942a3d6e, ..., k 1000 =2018df9326a7. It can be seen that the left-hand SST... 11 It contains k1 < k2 < k3 < ... < k 1000 But k1, k2, k3, ... k 1000 They are not necessarily consecutive. Similarly, the intermediate SSTs... 12 For example, it includes 800 kV. <k1-v1> <k2-v2> <k3-v3>... <k 800 -v 800 > and arrange them in ascending order of k, for example, k1=21a514c0a7e1, k2=21b46b994214, k3=21b489571b6d, ..., k 800 =284a5df3fca7. It can be seen that the middle SST... 12 It contains k1 < k2 < k3 < ... < k 800 However, similarly, k1, k2, k3, ... k 800 They are not necessarily consecutive. Similarly, the SST on the right... 13 For example, it includes 1100 kV. <k1-v1> <k2-v2> <k3-v3>... <k 1100 -v 1100 > where k is arranged in ascending order, for example k1=3a5e55ac3fe1, k2=3a5e55ac3fe2, k3=3afb8ed31526, ..., k 1100 =400a51ef1405. It can be seen that the SST on the right... 13 It contains k1 < k2 < k3 < ... < k 1100 However, similarly, k1, k2, k3, ... k 1100 They are not necessarily continuous. Furthermore, overall, the left SST... 11 The minimum value of k is less than the left-hand side SST. 11 The maximum value of k is less than the middle SST. 12 The minimum value of k < the middle SST 12 The maximum value of k is less than the right-hand side SST. 13 The minimum value of k is less than the SST on the right. 13 The maximum value of k. Level 2, Level 3... are similar, that is, within each SST in each level, there is a size order relationship in terms of the value of k, and among multiple SSTs in each level, there is a size relationship in terms of the range of k.

[0074] In Level 0, key-value pairs (kv) within a single SST can be sorted according to the size of their k values. However, the k values ​​across multiple SSTs do not necessarily have a range-based relationship. This is because the MemTable typically stores multiple kv values ​​corresponding to each block in ascending order (either from oldest to newest block number or from smallest to largest block number, since newer blocks have larger block numbers and older blocks have smaller block numbers). These k values ​​are the hash values ​​of tree nodes, and changes in the content of tree nodes are highly likely to result in significantly different hash values ​​(i.e., k values). Sorting the kv values ​​within an SST in Level 0 according to their k values ​​is efficient and feasible in memory; however, sorting by k values ​​is not strictly prohibited. For example, when the MemTable's capacity reaches 256MB, it may store multiple kv values ​​corresponding to multiple blocks. This MemTable is then converted into an Immutable MemTable. These kv values ​​in the Immutable MemTable can be sorted by their k values ​​and written to the left-hand SSTs in Level 0. There may be multiple processes of writing different Immutable MemTables to the same SST before the SST on the left side of Level 0 is updated. 01 Once the space is full, it can then be written to the SST on the right side of Level 0 in a similar manner. 02 Although the key-value pairs (kv) written to a Level 0 SST by an Immutable MemTable in a single instance are sorted, the sets of kv written to the same SST by different Immutable MemTables in Level 0 are generally not sorted by size. Similarly, the kv sets between different SSTs (e.g., left and right SSTs) are also not sorted by size. This means that the ranges of k in the left and right SSTs in Level 0 may overlap. Likewise, since the kv written to an SST in Level 0 can be written multiple times after a MemTable has been full, and the ranges of kv written during these multiple MemTable writes generally overlap, the multiple kv sets stored in each SST in Level 0 generally do not have a specific order, nor do the ranges between different SSTs in Level 0 generally have a specific order. Of course, if an Immutable MemTable writes to an SST in Level 0 in a single instance, then because the Immutable MemTable sorts the kv sets when writing to that SST, the kv sets stored in that SST are also sorted by k.

[0075] Furthermore, when the storage capacity of Level 0 reaches or approaches its limit, a process called "compaction" is used to write the data from Level 0 into Level 1. During this compaction process, the key-value pairs (kv) in each SST within Level 0 and some or all of the kv in Level 1 can be transferred to memory, sorted in memory, and then written to the SSTs of Level 1. Because sorting occurs during compaction, within each SST of Level 1, there is a size order relationship in the value of k, and across multiple SSTs in Level 1, there is a size order relationship in the range of k. In other words, within each SST of Level 1, the kv values ​​are ordered according to... <k1-v1> <k2-v2> <k3-v3>... <k n -v n Arranged in the order of >, and k1 < k2 < k3 < ... < k n And k1, k2, k3, ... k n The values ​​are not necessarily consecutive. Moreover, overall, for two adjacent SSTs, the minimum value of k in the left SST < the maximum value of k in the left SST < the minimum value of k in the right SST < the maximum value of k in the right SST, as shown in Figure 8.

[0076] During compaction, the key-value pairs (kv) in each SST within Level 0 and the key-value pairs in some SSTs within Level 1 can be loaded into memory for sorting. The SSTs within Level 1 can be SSTs whose key-value pairs overlap with those in Level 0; that is, SSTs whose key-value pairs overlap with those in each SST of Level 0. Specifically, for example, in the above example, Level 1 includes 3 SSTs... 11 The key range is 10b43c032a46~2018df9326a7, SST 12 The key range is 21a514c0a7e1~284a5df3fca7, SST 13 The key range is 3a5e55ac3fe1~400a51ef1405. For example, the key range in each SST within Level 0 is 18c49a405e32~24d35f39ab56. It can be seen that the key ranges in Level 1 overlap with those in Level 0 for some SSTs. 11 and SST 12 Thus, during compaction, the key-value pairs (kv) in each SST within Level 0 and the key-value pairs (kv) in each SST within Level 1 are... 11 SST 12 The key-value pairs can be loaded into memory and sorted, and then the sorted results can be written back to the Level 1 SST.

[0077] Similarly, when the storage capacity of Level 1 reaches its limit, a compaction process is used to write the data from Level 1 to Level 2. During this compaction process, the key-value pairs (kv) of each SST in Level 1 and some or all of the kv of some SSTs in Level 2 can be first loaded into memory, sorted in memory, and then written to the SSTs of Level 2. Likewise, if there is a Level 3 below Level 2, when the storage capacity of Level 2 reaches its limit, a compaction process is used to write the data from Level 2 to Level 3. Again, during this compaction process, the key-value pairs of each SST in Level 2 and some or all of the SSTs in Level 3 can be first loaded into memory, sorted in memory, and then written to the SSTs of Level 3. And so on.

[0078] In this way, overall, the data stored in the upper-level SST is newer than that in the lower-level SST. The newest data is stored in memory, the second newest data is stored in Level 0, and so on, with the oldest data stored in the SST within the lowest level. Furthermore, compaction can be used for garbage collection, such as deleting data marked for deletion.

[0079] Sorted key-value pairs can be efficiently retrieved, for example, using binary search. Binary search, also known as half-interval search, has the following algorithm: Define `left` as the leftmost element position of the sequential list, `right` as the rightmost element position, and `mid` as (left + right) / 2, which is the middle position of the sequential list; compare the value to be searched with the value at position `mid`; since the list is ordered, if the value to be searched is smaller than `mid`, only the first half of the list needs to be searched; otherwise, only the second half needs to be searched (if the two values ​​are equal on the first comparison, the current value's position is returned directly). This process continues until the value is found or it is determined that the value to be searched is not in the list (i.e., the search fails).

[0080] Such storage engines suffer from read amplification and write amplification issues when reading and writing data, which will be explained in detail below.

[0081] In a blockchain system employing the aforementioned storage engine, the execution module may need to read an existing state during transaction execution. For example, in a typical transfer transaction, external account A transfers a certain amount (e.g., f) of assets to external account B, requiring adjustments to the account balances. Assuming that before the transaction, account A's balance is a and account B's balance is b, the transaction execution sets account A's balance to (af) and account B's balance to (b+f). Therefore, this transaction needs to read the values ​​of the state variables a and b during execution. Another example is a contract call transaction. External account C initiates a call to contract D. C initiates a transaction pointing to the address of contract account D, specifying the called function and input parameters in the transaction's data field. Assuming the called function in contract D includes judging the value of a global state d within the contract (assuming d is a boolean), the judgment logic includes executing branch 1 if true and branch 2 if false. Therefore, this contract call transaction needs to read the value of state d during execution.

[0082] During transaction execution, whether reading the values ​​of state variables a and b, or reading the value of state variable d, the execution module can first check if the value of that state variable exists in its own memory. If it does not exist, a read command is sent to the storage module. This read command can include the address of the state to be read, i.e., the state's key value. Upon receiving the read command, the storage module locates the root node of the state tree in the tree construction module. It then matches the state key value to be searched against the root node's shared nibble(s) field (for Extension Nodes) or slot (for Branch Nodes) from the beginning. Next, it reads the hash of the next-level tree node (which can be an Internal Node or a Leaf Node; we will use an Internal Node as an example) from the matched position. It then first searches for the hash in the tree construction module's memory. If it is not found there, it searches in the state database module's memory. If it is still not found there, it searches in the state database module's file. Specifically, the search can begin with a lookup within the SST at Level 0. If the hash value is not found there, the search continues within an SST at Level 1, and so on, down to the next SST at each lower level, until the next Internal Node pointed to by the hash value is found. After unpacking this Internal Node, the remaining fields of the key value of the state address to be read are matched from front to back. If a match is found, the hash value is read from the match point, and the search jumps to the next level tree node pointed to by that hash value. This process is repeated, unpacking Internal Nodes level by level and matching the remaining fields of the key value of the state address to be read from front to back. The hash value at the match point serves as the basis for the next lookup of intermediate or leaf nodes, until a Leaf node is matched, from which the state value is read. During this process, some key-value pairs of tree nodes at each level may not exist in the memory of the tree building module but are located in the state database module. Therefore, it may be necessary to read them from the state database module into memory for further operations. The key can be searched first in the MemTable or Immutable MemTable of the state database module. If it does not exist, further searching of the various levels of SST files on disk is required. For Internal Nodes and Leaf Nodes existing on disk, they need to be loaded into memory and decoded (deserialized) to obtain their contents. Specifically, for example, the search can begin by searching each SST file in level 0. If each SST in level 0 is sorted, a binary search can be performed within each SST based on the hash value of the intermediate / leaf node. Once found, the key (v) is loaded into memory and decoded and matched.If the value is not found in the SST at Level 0, a binary search is performed in Level 1 based on the hash value of the intermediate / leaf node. If the value is still not found, a binary search is performed in Level 2. If the value is found, the value corresponding to the k is loaded into memory and decoded. This process continues until a leaf node of the tree structure is matched and its value is read. Since the overall SST at Levels 1, 2, and lower levels is sorted, a binary search can generally be performed at each level.

[0083] From the above process of reading a state key-value pair, it can be seen that this reading process requires binary search in multiple SSTs at different levels based on the hash value of the tree node. This may require reading a large number of files in multiple SSTs and loading them into memory, which leads to read amplification. Assuming the database includes Level 0, Level 1, Level 2, ..., Level M-1, for a total of M levels, then in the worst case, reading the data of a state requires at least M disk accesses (greater than M because in the worst case, N SSTs at Level 0 may require more than 1 but less than or equal to N search operations).

[0084] Furthermore, if the tree-building module is missing a key-value pair (kv) for a particular tree node while constructing the path from the state key-value pair to the root node, it may need to perform a binary search from Level 0, Level 1, Level 2, etc., with a similar principle to the process described above. In the transaction example above, for instance, after the transaction is executed, the value of the execution result (af), the value of (b+f), and the value of the contract state d need to be written to the state database. For the values ​​to be written to the database, the execution module calls the storage interface of the storage module, outputting the key-value pairs of the state to the storage module. The tree-building module in the storage module generates the key-value pairs of the tree nodes from the root to the intermediate and leaf nodes based on the state key-value pairs, writes these key-value pairs to the WAL module, and then writes this series of key-value pairs to the MemTable of the state database module, marking them as dirty data. Dirty data typically refers to data that has been written to the MemTable but has not yet been written to persistent storage (such as SSTable). Dirty data can be written to disk during the aforementioned flush process. The state database module then writes some key-value pairs, including the tree node key-value pairs, from the MemTable to the SST file on disk. Once the MemTable is persisted to the SSTable, the data within it is no longer dirty data, i.e., it becomes "clean" data. It is evident that the above process of persistently storing a state key-value pair is quite complex, consumes significant computational and memory resources, and introduces write amplification issues.

[0085] The complexity of the read and write processes, as well as the read amplification and write amplification issues, pose challenges to the high-performance requirements and large-scale applications of blockchain.

[0086] This application provides an embodiment of a method for reading blockchain state data. This method embodiment can be applied to the storage module of a blockchain node. The storage module constructs a tree-structured world state in memory. The leaf nodes of the tree structure may include the content of the state data or the position index of the state data within a data file in a persistent storage medium.

[0087] In one example, the overall tree structure can be shown in Figure 10, which is somewhat similar to Figure 5. Specifically, if the leaf nodes in memory contain state data, it can be as shown in Figure 5. For example, leaf node A11 in Figure 5 contains state data in memory, i.e., Zhang San_A = 20. If the leaf nodes in memory do not contain state data, the index storing the state data in the data file can be used.

[0088] In this application, separating the actual data and the index helps improve the system's flexibility and performance. The basic concepts involved include data files and index files. The data file stores the actual content; the index file stores an index pointing to the location of the data file. Its working principle is as follows: data content is written to the data file, recording the data's position within the file (such as filename, offset, and length); then, an entry is created in the index file, containing a key (a unique identifier used for lookup) pointing to a reference in the data file (filename, offset, etc.). Thus, during data retrieval, the data location can be quickly located through the index file, and the actual content can be read from the data file based on the location information. The advantages of this approach are: ① Flexibility: data and indexes can be managed and optimized independently; ② Performance: indexes are typically small and can be fully loaded into memory, speeding up queries; ③ Space efficiency: avoiding the storage of large amounts of duplicate data in the index; ④ Update efficiency: when modifying data, only the location information in the index needs to be updated.

[0089] For example, in Figure 10, the location index is used to represent this position. Taking node A11 as an example again, if there is no state data in memory, the location field can be used to store the position index of the data file containing the state data. This position index is, for example, <2,750,210>. This <2,750,210> indicates that the state data is located in "File 2" within the data file, starting at the 750th KB position of "File 2" and having a length of 210KB. As shown in the lower left corner of Figure 10, assuming each file is 1024KB in size, the logical address range of the index is 0KB to 1023KB. Thus, the state data of A11 actually occupies the range from the 750th KB to the 959th KB position in File 2.

[0090] As shown in Figure 9, this embodiment includes:

[0091] S910: The storage module of the blockchain node receives the instruction to look up the state key and searches for the leaf node corresponding to the state key in the tree structure in memory.

[0092] As shown in Figure 6 above, the execution module can generate execution results by executing ordinary transfer transactions or transactions involving contracts, and then send them to the storage module. Specifically, for example, during contract execution, it may be necessary to read the value of a state variable. The execution module can first check if the value of the state variable exists in its own memory. If it does not exist, it sends a read instruction to the storage module. This read instruction can include the address of the state to be read, i.e., the state's key value.

[0093] The storage module includes a tree-building module. This module constructs a tree structure representing the world state in memory. In the case of a cold start of a blockchain node (i.e., before the world state tree structure exists in memory), the individual tree nodes can be loaded from disk and organized into a tree in memory. This process also involves loading leaf nodes. An example is given where the tree-building module contains an incomplete tree structure; the cold start loading process is similar. Since memory capacity is generally limited, the LRU (Least Recently Used) algorithm is typically used to evict tree nodes that haven't been accessed for a long time. Therefore, leaf nodes in the tree structure constructed in memory by the tree-building module may be subject to eviction. For the leaf nodes of the world state tree, some may be in memory while others may not. Furthermore, a similar eviction mechanism can be used for the content within tree nodes, such as the state data in leaf nodes; alternatively, state data may not be maintained in memory.

[0094] For the state key to be searched, if the tree construction module has a leaf node corresponding to the state key in memory, it can be found directly in memory. Referring to Figure 5, the key of the corresponding leaf node can be found directly in memory, and the value can be read from it. Then, the state data can be read from the value. If the tree construction module does not have a leaf node corresponding to the state key in memory, or if the leaf node exists but does not contain the state data to be searched, further searching is required in the state database module. For cases where the state database uses an LSM tree, as mentioned earlier, a search can be performed first in the MemTable; if not found, a search can be performed in the files on disk.

[0095] S920: Obtain the position index contained in the leaf node corresponding to the tree node key, where the position index points to the position within the data file in the persistent storage medium.

[0096] Even if the tree construction module has the leaf node to be searched in memory, but it does not contain state data, a further search is needed from the state database module. As mentioned earlier, if there is no state data in memory, the location field can be used. For example, in Figure 10, the location of leaf node A11 is <2, 750, 210>. This <2, 750, 210> is the location index, which indicates that the state data is located in file 2 of the data file, starting at position 750KB and with a length of 210KB. As shown in the data file in the lower left corner of Figure 10, assuming each file is 1024MB in size, the logical address range of the index is 0KB to 1023KB. Thus, the state data of A11 actually occupies the range from 750KB to 959KB in file 2.

[0097] If the leaf node to be searched is not in the memory of the tree construction module, it needs to be searched for and loaded from the state database module. Searching for and loading the leaf node from the state database module specifically includes loading the index file and data file of the leaf node. Assume that the underlying database corresponding to Figure 10 stores the key-value pairs (kv) of all intermediate nodes in the tree structure, as shown in the table below:

[0098] Table 2. Key-value pairs of tree nodes actually stored in the database.

[0099] In Table 2 above, similar to Table 1, H() represents hash calculation. This anchors the hash value of the next tree node to the previous tree node. Through this layer-by-layer hashing, the root hash of the entire state trie tree is obtained and locked to the state root field in the block header. Assume the key-value pairs in this table are stored in the underlying state database module, saved on disk, and using a similar LSM structure. Thus, similar to the above, after receiving a read command, the storage module can find the root node of the state tree in the tree construction module. It then matches the state key value to be searched against the root node's shared nibble(s) field (for Extension Nodes) or slot (for Branch Nodes) from the beginning. It reads the hash of the next level tree node from the matching position, and then first searches for the hash in the memory of the tree construction module. If it is not found there, it searches in the memory of the state database module. If it is still not found there, it searches in the state database module's file. Specifically, the search can begin within the SST at Level 0. If the hash value is not found there, the search continues within an SST at Level 1, and so on, down to the next SST at each lower level, until the next Internal Node pointed to by the hash value is found. After unpacking this Internal Node, the remaining fields of the key value of the state address to be read are matched from front to back. If a match is found, the hash value is read from the match point, and the search jumps to the next level tree node pointed to by that hash value. This process is repeated, unpacking Internal Nodes level by level and matching the remaining fields of the key value of the state address to be read from front to back. The hash value at the match point serves as the basis for the next search of intermediate or leaf nodes, until a Leaf node is matched, from which the state value is read. During this process, some key-value pairs of tree nodes at each level may not exist in the memory of the tree building module but are located in the state database module. Therefore, it may be necessary to read them from the state database module. The key can be searched first in the MemTable or Immutable MemTable of the state database module. If it does not exist, further searching of the various levels of SST files on disk is required. For Internal Nodes and Leaf Nodes existing on disk, they need to be loaded into memory and decoded (deserialized) to obtain their contents. Specifically, for example, the search can begin by sequentially searching the SST files of level 0. If each SST in level 0 is sorted, a binary search can be performed within each SST based on the hash value of the intermediate / leaf node. Once found, the key (v) is loaded into memory and decoded and matched.If the value is not found in the SST at Level 0, a binary search is performed in Level 1 based on the hash value of the intermediate / leaf node. If the value is still not found, a binary search is performed in Level 2. If the value is found, the value corresponding to the k is loaded into memory and decoded. This process continues until a leaf node of the tree structure is matched and its value is read. Since the overall SST at Levels 1, 2, and lower levels is sorted, a binary search can generally be performed at each level.

[0100] In this way, the key-value pairs of the found leaf nodes can be loaded into memory. For example, in A11, the content of the leaf node value, prefix:2, key-end:35b2e4, location: (2,750,210), is loaded into memory. Here, <2,750,210> is the position index of the value content in the file.

[0101] S930: Load the value of the data file location in the persistent storage medium pointed to by the location index into memory and read the status data therein.

[0102] Taking A11 as an example, the position index <2,750,210> points to the location within the data file in the persistent storage medium. Thus, the value at the location in the data file pointed to by the position index <2,750,210> can be loaded into memory and its status data read. Specifically, the 210KB data starting from the 750KB mark in file 2 of the data file can be loaded into memory, and its status data, "Zhang San_A = 20", can be read.

[0103] The location index allows direct location within the data file, enabling the loading of the data file and the reading of status data from it.

[0104] If the tree-building module has the leaf node to be searched in memory, but it does not contain state data, it can directly read the data from the corresponding data file based on the location within the leaf node in memory. This avoids storing the key-value pairs of the leaf node together in the persistent storage SST file, which would require multiple binary search operations within the SST.

[0105] If the leaf node to be searched is not in the memory of the tree construction module, and it is necessary to search for and load the leaf node from the state database module, it is actually necessary to read a certain amount of content from the disk into memory before searching. In the implementation corresponding to Table 1, the key-value pairs of the tree node are stored together on disk, and the value of the leaf node includes state data. For example, in Table 1, H(11) corresponds to Zhang San_A=20, H(12) corresponds to Li Si_B=20, H(15) corresponds to storedData=s, and H(16) corresponds to Wang Wu_A=20. Generally speaking, the state data occupies a large space, about 90% or more of the total space of a key-value pair. In the implementation corresponding to Table 2, the index and data are separated. The index stores the location of the data file containing the state data, but not the state data itself. For example, in Table 2, H(11) corresponds to location (2,750,210), H(12) corresponds to (3,350,210), H(15) corresponds to (5,760,140), and H(16) corresponds to location (5,170,210). The actual state is stored in the corresponding location in the data file, as shown in Figure 10. Therefore, in the process of searching for leaf nodes, only the index file can be searched. In this way, more index files can be loaded into memory within a unit space for searching. The search efficiency in memory is significantly higher than that on disk. Generally, the content to be searched is loaded into memory before searching. The implementation in Table 2 can find leaf nodes faster than the implementation in Table 1 because more indexes can be loaded within a unit capacity of memory space. The implementation of Table 1, for example, requires reading a batch of key-value pairs into memory. If a leaf node is not found, another batch of key-value pairs needs to be read into memory again for another search. Since the state content occupies a large amount of space, the number of key-value pairs in a unit of memory space is relatively small. Therefore, the number of reads from the disk will generally be more frequent. Since the disk read speed is much slower than memory, the search efficiency of the implementation of Table 1 is low.

[0106] The specific process of the method embodiment in Figure 9 is illustrated above with reference to Figure 10. Figure 10 still shows a two-level MPT tree structure, and the structure of the intermediate nodes in each level includes two types of tree nodes: Extension Node and Branch Node. In a variation, a Branch Node can be used as both the intermediate node and the root node, as shown in Figure 11. Thus, the sequential combination of slots in the branch nodes traversed from the root node through the intermediate nodes to the leaf nodes can constitute part of the content in the state key. The hash value of the next-level node can be stored in the slot of the previous-level node. In Figure 11, a two-level tree structure can still be used. The leaf nodes in the previous-level tree can store account states (including the states of external accounts and contract accounts), and the leaf nodes in the next-level tree store the contract states. Position indexes can be used in the leaf nodes storing external account states, contract account states, or contract states. For example, the leaf nodes of the previous-level tree include the account states in the following four rows: H(M1), H(M2), H(M3), and H(M4) in the table below:

[0107] Table 3. Key-value pairs of tree nodes actually stored in the database.

[0108] The location index allows direct access to the data file, enabling the loading of the data file and the reading of state data. Furthermore, the leaf nodes of the next-level tree contain the contract states as shown in rows H(Z1), H(Z2), H(Z3), and H(Z4) in the table above.

[0109] The data file is shown in Figure 11, and its contents are as follows:

[0110] M1: Nonce: n1, balance: 45.0 ETH

[0111] M2: Nonce: n2, balance: 1.00WEI

[0112] M3: Nonce: n3, balance: 1.1 ETH

[0113] M4:Nonce:n1,balance:0.12ETH,codehash:c1,storage:s1

[0114] Z1: Zhang San_A=20

[0115] Z2: Li Si_B=50

[0116] Z3: Wang Wu_A=35

[0117] Z4:storedData=s

[0118] Regardless of whether it's the parent or child level of the tree, the location in the leaf node can be quickly located to the position in the data file. This avoids the need for multiple binary search operations in the SST based on the leaf node's key (hash value) when storing the key-value pairs of the leaf nodes together in the persistent storage SST file. In the example of Figure 11, when reading blockchain state data as shown in Figure 9, the location in the data file can be directly located based on the location index. Therefore, the content located at the corresponding position in the data file can be directly loaded into memory, thus avoiding the read amplification problem caused by reading a large number of files from multiple levels of SST and loading them into memory. Similarly, the leaf node A4, which is also the parent level of the tree, can be stored as shown in Figure 10 or as shown in Figure 11.

[0119] External accounts and contract accounts typically use fixed-length strings as keys, while contract status keys can also be strings of the same length. Specifically, for example, the key for an external account or contract account could be an account address, while the key for the contract status could be a string of the same length. Another example is that the key for an external account or contract account could be the account address hashed using a specific algorithm, such as SHA3; while the key for the contract status could be SHA3(contract name + position of status variable declaration). These external accounts, contract accounts, and contract statuses are all strings of the same length and format, collectively referred to as the status key. For example, the length of the status key might be 4 bytes, or 32 bits. Thus, assuming the root node (B0) and intermediate nodes (..., C5, ..., C13, ..., C5D1, ..., C5D9, ..., C5D14, ..., C13D2, ..., C13D10, ...) are each 16-ary trees, then on the one hand, each slot in the intermediate node (such as slots 0, 1, 2, ..., d, e, f in B0, a total of 16 slots) can be filled with the hash value of the next tree node; on the other hand, the position of this slot in the root / intermediate node can represent a 4-bit key. 4 bits constitute a half-byte, so the root node and intermediate nodes, a total of 8 levels, can store 32 bits, totaling 8 bytes, which is the entire state key. The 32 bits of the state key can be stored from the top-level intermediate node to the bottom-level intermediate node and down to the leaf node, with each level storing one half-byte of the key. Thus, in the case where the intermediate nodes are fixed and expanded, the leaf nodes can store the last 4 bits of the key. With the intermediate nodes fixed and expanded, Figure 11 shows a total of 8 layers from C* to C13D*, all the way to the bottommost M*. Alternatively, a compressed prefix and / or infix and / or suffix approach can be used, in which case one or more leaf nodes plus the intermediate nodes above them can be fewer than 8 layers.

[0120] Alternatively, a single-layer tree structure can be used to store account and contract states, as shown in Figure 12. Since the key for a contract state is a hash value, due to the discrete nature of hash algorithms, the state keys for different contract states may follow paths from the root node to different leaf nodes within this single-layer tree structure. Thus, different states of the same contract may have values ​​residing in different leaf nodes at the bottom layer of the tree structure, rather than clustered under their respective contract accounts. In effect, using a single-layer tree structure, both account and contract states are distributed across the bottom-layer leaf nodes.

[0121] Furthermore, based on Figure 12, as shown in Figure 13, prefix and / or infix compression can be performed. In the case of infix compression, for example, if the key of tree node M1 is 0x52d1479e (0x represents each digit in hexadecimal), and the key of tree node Z1 is 0x52d9ab36, then M1 and Z1 share a common infix 2d. Therefore, in tree node C5, besides the 1 and 9 separated by the hexadecimal tree, C5 can also have compressed infix 2d, as shown in the shared nibbles field in Figure 13. In the case of postfix compression, for example, using key-end representation, the key-end in tree node M1 is, for example, 479e, and the key-end in tree node Z1 is, for example, 36. Using prefix and / or infix compression, the number of tree levels can be reduced.

[0122] Regardless of compression, to find a state key, one can start from the root node of the tree structure and match the path of the state key segment by segment until the leaf node of the tree structure corresponding to the state key is found. In one approach, the root node and intermediate nodes are stored in key-value format, where the key is the hash value of the value, similar to the structure in Table 3. In this way, one can start from the upper-level tree node, sequentially match the hash of the path of the state key, and use the hash to find the lower-level tree node until the leaf node of the tree structure corresponding to the state key is found.

[0123] In addition, the key of the tree node kv actually stored in the database may not be the hash value of the value, but a part of the value on the state key path. For example, it may be a part of the key value from the root node to the current tree node (refer to Figure 12, without compressed prefix / infix, as shown in Table 4), or a part of the key value from the previous tree node (refer to Figure 13, with compressed prefix / infix, as shown in Table 5).

[0124] Table 4. Tree node key-value pairs actually stored in the database

[0125] According to Table 4 and Figure 12, for example, if the search state key is 0x59...36 (leaf node Z1), starting from the root node of the tree structure, we can match the first digit (5) of the state key to the tree node in the database with the key 0x5 (the h (C5) in the value of the tree node with the key 0x5 can be used to verify with the hash value in slot 5 of the root node; if they are the same, it means that the tree node with the key 0x5 is a child node in slot 5 of the 0x tree node); then, we can match the second digit (9) of the state key to the tree node in the database with the key 0x9 (the h (C5D9) in the value of the tree node with the key 0x9 can be used to verify with the hash value in slot 9 of the parent node; if they are the same, it means that the tree node with the key 0x9 is a child node of the 0x5 tree node); and so on. Finally, the last digit 6 of the status key can be used to match the tree node in the database with the key 0x6 (the h(Z1) in the value of the tree node with key 0x6 can be used to verify with the hash value on slot 6 in the parent node; if they are the same, it means that the tree node with key 0x6 is a child node of the parent node).

[0126] Table 5. Tree node key-value pairs actually stored in the database

[0127] According to Table 5, for example, if the search query for the state key is 0x52d9...36 (leaf node Z1), starting from the root node of the tree structure, the system can match the first digit (5) of the state key in the database. If no matching key is found, or if the hash value in the value of the tree node matching the key is inconsistent with h(C5) in the root node, the system will again match the first two digits (52) of the state key in the database. This process is repeated until a match is found. When the system matches the first three digits (52d) of the state key in the database and finds a tree node with the key 0x52d, and the hash value H(C5) in its value is the same as the hash value in slot 5 of the root node, then a match has been found. Then, the system can match the first four characters (52d9) of the status key in the database. If no matching key is found, or if the hash value of the value in the tree node containing the matched key is inconsistent with h(C5) in the root node, the system can re-match the first two characters (52) of the status key in the database. If a match is found, and the H(C5D9) in the value of the matched tree node is the same as the hash value in slot 9 of the parent node, then a match has been found. Further, the system matches the first five characters (1-5) of the status key in the database, the first six characters (1-6) of the status key in the database, and so on, until a leaf node is matched.

[0128] In another implementation, the complete state key or the corresponding complete state key can be stored in the leaf nodes. Taking storing the complete state key in the leaf nodes as an example, as shown in Figure 14, the `key-full` field of the leaf node can store the complete key of the corresponding state. The key-value pair of the leaf node stored in the database has the key that is the complete key of the corresponding state for that leaf node. In this way, when searching for a state key, the key can be directly matched against the key of the leaf node in the database, without needing to match the keys of intermediate nodes. Obviously, this search method is extremely efficient for query operations. In fact, in Figures 10-13 above, the complete state key can also be used as the key of the leaf node.

[0129] In the embodiments shown in Figures 9-14 above, the location index is stored in the leaf node. This allows for quick location of the leaf node's value in the data file after the leaf node is found. This enables direct loading of the content at the corresponding location in the data file into memory, avoiding the read amplification problem caused by reading and loading large amounts of files from multiple levels of the SST. It also avoids the need for multiple binary search operations within the SST based on the leaf node's key (the hash value of the value) when storing the leaf node's key and value together in the persistent SST file. Furthermore, similarly, if the leaf node to be searched is not present in the memory of the tree-building module, the above embodiments can search only the index file. This allows more index files to be loaded into memory within a unit of space for searching, thus finding the leaf node faster.

[0130] As mentioned earlier, the state root in the block header stores the hash value of the root of the lower-level MPT tree to which it is anchored. This hash value is actually obtained by calculating the hash value layer by layer upwards from the leaf nodes, as described above. Generally, the storage module of a blockchain node constructs the world state of the tree structure in memory through the tree construction module. Specifically, in addition to the leaf nodes shown in Figures 12-14, it also includes intermediate nodes and the root node, as shown in Tables 2-5.

[0131] Furthermore, the chain-like data structure of the blockchain is implemented by anchoring the hash of the subsequent block to the Prev Hash of the previous block, as shown in Figures 3 and 4. The blockchain as a whole contains blocks identified by different block numbers. These block numbers are also called versions. Under different versions of blocks, the corresponding tree structure may change.

[0132] Those skilled in the art know that the essence of a blockchain is a chain-like structure composed of a series of data blocks (blocks) linked by hashes. Each block has a corresponding world state, but each block does not necessarily contain the complete world state; rather, it usually contains a reference to a previous world state. This will be explained below:

[0133] In Ethereum's block structure, when the value in a key-value pair changes, the content of the leaf node in its corresponding MPT tree changes. Correspondingly, the hash values ​​stored in a series of intermediate nodes above that leaf node, as well as the root node, will also change. In the structure shown in Table 1, the changed leaf nodes, intermediate nodes, and root node will be appended to the end of Table 1. Meanwhile, it's possible that many state variables in a contract's storage may not have changed their values.

[0134] Figure 15 shows the Ethereum block structure. The difference between Figure 15 and Figure 3 is that Figure 15 adds references to the root and intermediate nodes of the two-level Merkle tree under the State Root in block N+1, showing state variables that haven't changed in block N. For example, in a contract's contract storage, if the value of the same state variable changes from 29 in block N to 45 in block N+1, assuming other state variables haven't changed, then in block N+1, besides the leaf node storing the changed value of 45 and its associated intermediate nodes, all other unchanged leaf and intermediate nodes directly reference the same leaf and intermediate nodes from block N. This shows two adjacent blocks, N and N+1. It can be understood that for the block structure of multiple consecutive blocks on the blockchain, leaf and intermediate nodes in block N+1 that haven't changed compared to previous blocks all reference the same leaf and intermediate nodes that were last updated to that value. The last leaf node and intermediate node updated to this value, for example, the same leaf node and related intermediate node updated to this value in block N-1, the same leaf node and related intermediate node updated to this value in block 5, and so on. These different blocks can also be called different versions, and the block number can be the version number. This referencing is implemented by the upper-level node recording the hash value of the lower-level node. The advantage of this is obvious: it avoids repeatedly storing unchanged leaf nodes and intermediate nodes in the underlying database.

[0135] To illustrate this simply, let's consider a change in the state of the same contract in version N+1 compared to version N. For example, if the content of a tree node changes, its hash will change accordingly. MPT uses the hash as an index in the database, ensuring that each value has a unique record in the database. MPT associates parent and child nodes based on their hashes. Therefore, whenever the content of a node changes, only the hash index value changes for the parent node; the parent node's content also changes, creating a new, higher-level parent node, and this effect is recursively propagated to the root node. Ultimately, each change creates a new path from the modified node to the root node, while the old node can still be accessed via the old path from the old root node.

[0136] As shown in Figure 15, when the content of a node in the MPT changes from 29 in version N to 45 in version N+1, assuming that the other states in the contract state have not changed between the two versions, the changed 45 corresponds to the creation of a new path to the contract's storage root in the world state of version N+1. Furthermore, the other unchanged leaf nodes and intermediate nodes reuse the leaf nodes and / or intermediate nodes from the previous version (i.e., the version with block number N) using hash pointers, thereby constructing a new MPT tree for the contract's storage in version N+1 while still retaining its old paths. As shown in Figure 18, within the same contract storage, leaf nodes / intermediate nodes whose values ​​in version N+1 have not changed compared to version N include V, U, T, and S. Therefore, in the storage of this contract in version N+1, besides creating a new path from 45 to the Storage root, the Storage root in version N+1 is indexed by hash to V, U, T, and S in version N, but does not include the leaf node with a value of 29 and its related intermediate nodes. Assuming the contract is deployed in version N, the CodeHash works similarly. The contract code corresponding to the CodeHash exists in the StateDB corresponding to version N, and the CodeHash in version N+1 also points to the contract code in version N.

[0137] In addition to contract storage, other account storage (including the storage of external accounts and contract accounts) can also adopt a similar approach. As shown in Figure 15, for the account storage that has not changed in version N+1, the State Root in the N+1 block header points to the storage of external accounts or contract accounts such as L, M, N, P, Q, and R in version N. Only the leaf nodes and related intermediate nodes that have changed are stored in the account storage corresponding to version N+1. These will not be elaborated on in detail.

[0138] The current state contains many states that were likely updated in a previous block and have remained unchanged until now. This is similar to Ethereum, and is known as the current state, stored in the currentDB. The currentDB stores the latest "current state" data, such as the latest account state, contract code, and contract storage. Moreover, the data in the currentDB can be continuously modified as the number of blocks grows.

[0139] As shown in Figure 3 and Figure 15 above, building a global MPT tree for all states in each block is highly redundant. Using the same leaf and intermediate nodes from previous blocks that were last updated to that value is a more efficient data storage method. Furthermore, the changed state in each block is likely a part of the global state. This is similar to Ethereum, also known as history state, stored in historyDB. historyDB stores historical state data, providing a complete historical view of the entire chain.

[0140] This application provides a method for constructing a tree-based world state, including:

[0141] S170: Receive status key-value.

[0142] S172: According to the tree structure construction rules, generate leaf nodes of the tree structure corresponding to the state key-value; calculate the hash value of the leaf node according to the content in the leaf node, and generate or update the upper-level tree nodes according to the hash value of the leaf node and the state key, until the root node of the tree.

[0143] The leaf nodes can be in the form of the aforementioned data file or the aforementioned KVDB. If the aforementioned data file format is used, the leaf node can generate a value including the position index of the state value within the data file in the persistent storage medium and the hash value of the state value content.

[0144] S174: Generate corresponding memory pages for the generated / updated tree nodes according to the divided logical page structure. The memory pages are used to represent the contents of all tree nodes in the logical pages.

[0145] In this application, the entire Merkle tree structure can be divided into multiple LogicalPages. This can be achieved by aggregating several adjacent tree nodes into a single LogicalPage based on the node relationships within the tree structure, such as a 2-level, 16-branch trie with 256 child nodes. There are multiple levels of LogicalPages from the root node to the leaf node of the Merkle tree, and these LogicalPages have parent-child and sibling relationships, thus forming a complete Merkle tree. In this way, the tree structure can be divided into multiple LogicalPages, each containing at least one tree node. Different LogicalPages contain different tree nodes, and each LogicalPage maintains the latest version of the corresponding tree node's content.

[0146] As shown in Figure 16, each LogicalPage can maintain a MemoryPage, which represents the content of all tree nodes corresponding to the current latest version of the LogicalPage. Furthermore, the content of the tree nodes maintained by the LogicalPage can be used to generate base pages and incremental pages based on continuous version updates and generation / modification behavior. Specifically, BasePage and DeltaPage are generated according to generation / modification behavior. For example, for continuous state changes, such as version N including state variables a=5, b=8, c=3, a=5, b=8, c=3 can be used as the content of the base page BasePage. For a=6 in version N+1 and a=8 in version N+2, assuming b and c remain unchanged, a=6 in version N+1 and a=8 in version N+2 can be used as DeltaPage based on this BasePage, such as DeltaPage1. Cases where b and c remain unchanged are not included in DeltaPage1. If a = 11 in version N+3 and a = 15 in version N+4, then a = 11 in version N+3 and a = 15 in version N+4 can be used as DeltaPages based on this BasePage, such as DeltaPage2. Similarly, assuming b and c remain unchanged, b and c are not included in DeltaPage2, and so on. BasePage and DeltaPage can serve as the smallest unit of the tree's memory structure. Furthermore, BasePage can serve as the smallest unit of the tree's disk persistence. Based on this, DeltaPage can also serve as the smallest unit of the tree's disk persistence.

[0147] It's important to note that the BasePage here refers to a checkpoint of the entire LogicalPage content. It can contain the full content of the LogicalPage at that particular moment, such as all included tree node information. A checkpoint is generally a snapshot of a specific state saved in a computer system, allowing recovery and continuation of execution in the event of a system failure. In many cases, checkpoints are used for long-running computational tasks, safety-critical systems, distributed systems, and database systems.

[0148] [Corrected according to Rule 91, December 18, 2024] For BasePage, the state can be modified and a BasePage can be generated once every predetermined number of times (e.g., m times), as shown in basepage2 on the right side of the gray box in the lower right corner of Figure 17. In the example above, the state variable a = 5 in version N, and this a = 5 can be used as the content of the base page BasePage1. In version N+1, a = 6, and in version N+2, a = 8. The a = 6 in version N+1 and the a = 8 in version N+2 are used as DeltaPage1 based on this BasePage. If a=11 in version N+3 and a=15 in version N+4, then a=11 in version N+3 and a=15 in version N+4 can be used as DeltaPage2 based on this BasePage. If a=20 in version N+5, assuming the modification reaches the predetermined number of 5, then the values ​​a=20, b=8, and c=3 in version N+5 can be used as the content of this BasePage2.

[0149] Similarly, a DeltaPage describes several version modifications of a LogicalPage, aggregating multiple modifications into a set. For example, a DeltaPage is generated every M modifications, and a new DeltaPage is created to collect subsequent version modification operations. In the previous example, if a=6 in version N+1 and a=8 in version N+2, then a=6 in version N+1 and a=8 in version N+2 are used as DeltaPage1 based on this BasePage. If a=11 in version N+3 and a=15 in version N+4, then a=11 in version N+3 and a=15 in version N+4 are used as DeltaPage2 based on this BasePage. It can be seen that each DeltaPage is configured to create an incremental page for two version modification operations.

[0150] For each modification to BasePage and DeltaPage, corresponding dirty data is generated in memory. When the memory usage of BasePage and DeltaPage reaches a predetermined level, they can be batch-persisted to disk, thus avoiding the need to persist all dirty data with every modification, which continuously consumes memory and CPU resources. The memory generation and disk persistence of BasePage and DeltaPage can be performed in the background.

[0151] When the storage module persists the world state of the tree structure built in memory through logical pages, it can cache the state data at the logical page level according to a predetermined strategy, such as the Least Recently Used (LRU) strategy, based on the hot / cold characteristics of the state data. In this way, if a MemoryPage is evicted, the corresponding BasePage and DeltaPage are read, and the MemoryPage is replayed.

[0152] The scheme described in this application proposes a method for partitioning the world state of a tree structure according to Pages. The tree structure is divided into LogicalPages, each of which manages MemoryPage, BasePage, and DeltaPage. Each state tree modification is optimized into multiple version modifications at the LogicalPage level, reducing overall write amplification. The current version can be directly obtained from the memory page, while historical versions can be obtained by replaying the BasePage and several associated DeltaPages. The entire MemoryPage is cached in memory, without affecting contract execution performance.

[0153] In the above scheme, a LogicalPage internally manages MemoryPage / BasePage / DeltaPage. As mentioned earlier, it can be aggregated into a LogicalPage by following the node association relationships in a tree structure, such as aggregating 256 child nodes (2 levels, 16 branches) of a trie into a LogicalPage. There are multiple levels of LogicalPages from the root node to the leaf node of the Merkle tree, with parent-child and sibling relationships between them, thus forming a complete Merkle tree. In this way, the tree structure can be divided into multiple logical pages, each containing at least one tree node. Different logical pages contain different tree nodes, and each logical page maintains the latest version of the corresponding tree node's content.

[0154] Therefore, when constructing a tree-based world state, in addition to generating leaf nodes of the tree based on the state key-value pair, changes also occur from the generated leaf nodes to the intermediate nodes to the root node, as described in S172 above regarding the hash value. Furthermore, the state key needs to be designed, especially for the LogicalPage and its corresponding MemoryPage / BasePage / DeltaPage.

[0155] Therefore, this application also proposes the following methods for constructing world states based on tree structures, including:

[0156] S180: Receive status key-value.

[0157] S182: According to the tree structure construction rules, generate leaf nodes of the tree structure corresponding to the state key-value; calculate the hash value of the leaf node according to the content in the leaf node, and generate or update the upper-level tree nodes according to the hash value of the leaf node and the state key, until the root node of the tree.

[0158] The leaf nodes can be in the form of the aforementioned data file or the aforementioned KVDB. If the aforementioned data file format is used, the leaf node can generate a value including the position index of the state value within the data file in the persistent storage medium and the hash value of the state value content.

[0159] For details regarding the process of S182, please refer to the aforementioned content. Here it is briefly summarized again as follows:

[0160] Similar to Ethereum, when the storage module receives a key-value pair (kv) request to write state, it first hashes the key (k) (e.g., using Keccak-256). Alternatively, the state key in S172 could be the same as the state key received in S170, meaning it could be used without hashing. Next, this key is used to perform a lookup in the MPT. This lookup process starts from the root node of the MPT and traverses and matches downwards along the tree path in lexicographical order. During this process, the key is typically compared bit-by-bit (or nibble, typically a single hexadecimal digit) with the path stored in the node. During traversal and matching, three types of nodes may be encountered: branch nodes (if the path forks at some point), extension nodes (if there is a shared prefix), and leaf nodes (which store the final value). Branch nodes are used to branch in the tree, extension nodes are used to optimize storage (through path compression), and leaf nodes store the actual value. If a complete lexicographical path to the matching key already exists during the lookup, a leaf node is reached. At this point, the value (v) stored in this leaf node can be updated. If the search and matching process finds that only part of the path is matched, it may be necessary to create a new node (which may be a branch node, an extension node, or a leaf node) to complete the remaining path and eventually reach the leaf node, where the value is stored or updated.

[0161] Once a leaf node is updated or created, an upward backtracking process begins. This process starts by recalculating the hash value of the modified leaf node. Then, it moves upwards to the parent node, updating the relevant references in the parent node (this reference could be the direct content of the child node, or the hash value of the child node's content, depending on the size of the child node), and recalculating the hash value of the parent node. This process continues until the root node of the MPT is reached.

[0162] Finally, when the backtracking process reaches the root node, the root hash of the entire MPT is updated. This new root hash represents the new state of the entire state tree and will be included in the block header of the next block as the new state root. In this way, each block not only contains transaction information but also stores the hash value of the global state after these transactions are executed through the state root.

[0163] S184: Update the version of the generated / updated tree node in the corresponding memory page according to the divided logical page structure. The memory page is used to represent the content of all tree nodes in the logical page. The logical page contains a page identifier, which includes the lexicographical content and version of the upper-level tree node in the logical page.

[0164] It should be noted that the process of S184 can occur after S182 or during S182.

[0165] Figure 18 is a complete illustration of another embodiment of this application. Figure 18 is similar to Figure 13 in general, with the main difference being that, for example, the intermediate node labeled C5 and its subordinate intermediate nodes labeled C5D1, C5D9, and C5D14 together constitute a LogicalPage. Furthermore, the leaf nodes M1, M2, M3, and M4 store key-end data instead of key-full data, and in addition to the position index, the leaf nodes also contain the hash value of the corresponding state value. Of course, in this embodiment, as mentioned above, the leaf nodes M1, M2, M3, and M4 can also be key-full data.

[0166] The leaf node may also include the hash value of the state value content and a key-end field. The calculation of the hash value of the leaf node based on the content of the leaf node includes the calculation based on the hash value of the state value content of the leaf node and the key-end field, but does not include the position index of the state value in the data file in the persistent storage medium.

[0167] The leaf node may also include the hash value of the state value content and a key-full field. Calculating the hash value of the leaf node based on its content includes calculating it based on the hash value of the state value content and the key-full field, but does not include the location index of the state value within the data file in the persistent storage medium. The value hash in the leaf node is omitted in Figure 18. Furthermore, Figure 18 uses a tree node data structure instead of the graphical description in Figure 14.

[0168] In step S182 above, tree nodes were generated / updated. These updated tree nodes are integrated into the corresponding memory pages according to the aforementioned logical page structure, representing the world state tree structure under the current block number. In this application, a version number can be set for the memory page corresponding to the logical page. This version number is typically the block number. Besides the block number, the version number can also be data that uniquely identifies a block, such as block hash, timestamp, state root, transaction root, receipt root, consensus round number, etc. For simplicity, the following explanation uses the block number.

[0169] The version number of a logical page can be used as content in the page identifier of the logical page. Besides the version number, the page identifier of a logical page can also include the lexicographical order of the contained tree nodes. Specifically, the page identifier can include the lexicographical order of the upper-level tree nodes in the logical page. As shown in Figure 18, in a LogicalPage composed of the intermediate node labeled C5 and its subordinate intermediate nodes labeled C5D1, C5D9, and C5D14, the upper-level tree nodes have a NodeID, for example, 0xd2345 (0x represents hexadecimal), and the subordinate intermediate nodes C5D1, C5D9, and C5D14 also each have a NodeID. The PageID of this LogicalPage can be set to the NodeID of the upper-level tree nodes. These page IDs represent the lexicographical order of the upper-level tree nodes from the root node to the upper-level tree nodes in this LogicalPage (i.e., the memory page), that is, the key of the lexicographical order distribution from the root node through the intermediate node to the upper-level tree nodes in the LogicalPage. Similarly, NodeID is the lexicographical content from the root node to the node in the tree, that is, the key of the lexicographical content distribution from the root node through intermediate nodes to the node in the tree.

[0170] As mentioned above, during the generation / updating of the current version of the tree nodes, the changed tree nodes constitute dirty data. Furthermore, as previously stated, base pages and incremental pages can be generated based on changes in memory pages across consecutive versions. Based on this, either base pages or incremental pages can be used as the smallest unit for disk persistence of the tree. A base page can be generated after a predetermined number of state modifications, or an incremental page can be generated after a predetermined number of state modifications. When the memory occupied by base pages and incremental pages reaches a certain amount, the base pages and incremental pages are batch-persisted to disk.

[0171] The base page and incremental page are persistent objects of the memory pages corresponding to logical pages, while memory pages are in-memory objects of logical pages. A base page generally corresponds to a memory page, contains global state variables, and can have the same page ID as the corresponding memory page, while the memory page has the same page ID as the corresponding logical page. An incremental page can also correspond to a memory page, containing tree nodes from one or more consecutive versions of memory pages that have changed relative to the previous adjacent base page / incremental page / version. This includes changed state variables, which are generally not global state variables. The structure of the tree nodes in an incremental page generally does not change, therefore the page ID of the corresponding memory page can be the same as the NodeID of its parent tree node.

[0172] Furthermore, base pages and incremental pages can have versions. After the operation in S184 above, the generation of base pages and incremental pages based on changes in consecutive versions of memory pages can be based on generating corresponding versions of base pages and / or incremental pages based on changes in consecutive versions of memory pages. Specifically, a base page generally corresponds to one memory page and contains all tree nodes in that memory page, which also includes global state variables, and therefore can have the same version as the corresponding memory page. An incremental page can correspond to one or more memory pages and contains tree nodes in one or more consecutive versions of memory pages that have changed relative to the previous adjacent base page / incremental page, which also includes changed state variables, but generally not global state variables. Therefore, the version of an incremental page can be the lowest or highest version among the one or more memory pages it corresponds to.

[0173] For example, a base page of version 4 contains state variables a=1, b=2, c=3, and the corresponding intermediate and root nodes. Subsequent incremental pages may include version 5's a=2, its corresponding intermediate and root nodes; version 6's a=1, its corresponding intermediate and root nodes; and version 7's b=4, its corresponding intermediate and root nodes. While version 6's a=1 shares the same state variable as the base page, it differs from the preceding adjacent version 5's a=2. Therefore, version 6's a=1 and its corresponding intermediate and root nodes are also included in the incremental page. Furthermore, for subsequent incremental pages, such as those containing tree nodes of versions 7, 8, and 9, the version is actually relative to the immediately preceding incremental page and the version of the tree nodes. The tree nodes of versions 5, 6, and 7 correspond to memory pages of versions 5, 6, and 7, respectively. This incremental page containing tree nodes of versions 5, 6, and 7 can itself be version 5 or 7, meaning it can be the lowest or highest version among the corresponding memory pages.

[0174] Furthermore, the page identifiers of the base page and the incremental page may also include the page type to distinguish between the base page and the incremental page.

[0175] The above scheme, a multi-version page partitioning scheme for Merkle state data dictionary tree, switches the tree to LogicalPage. Both the tree and LogicalPage maintain versions (such as block numbers). Internally, it manages MemoryPage / BasePage / DeltaPage. BasePage / DeltaPage maintains the memory structure and file persistence according to the version. While reducing overall write amplification through page aggregation, it utilizes the ordered nature of versions to avoid compaction after page write to disk, thus reducing write amplification. Furthermore, through version-based replay, it supports reading state data from any historical version and SPV functionality. In addition, combined with the content of Figures 9-14 above, the state value can be stored in the data file of the persistent storage medium. The leaf nodes include the position index of the state value within the data file of the persistent storage medium, thereby further reducing the write amplification problem.

[0176] This application also provides a computer device, including:

[0177] processor;

[0178] And a memory containing a program, wherein, when the processor executes the program, any of the above methods are performed.

[0179] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0180] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0181] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0182] While one or more embodiments of this specification provide the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0183] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0184] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0185] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0186] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0187] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0188] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0189] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0190] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0191] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0192] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0193] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims. < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1> < / k2-v2> < / k1-v1>

Claims

1. A method for constructing a tree-structure-based world state, applied to a storage module of a blockchain node, the storage module constructing a tree-structure-based world state in memory, the method comprising: receiving a state key-value; generating / updating a leaf node of a tree structure corresponding to the state key-value according to a construction rule of the tree structure; calculating a hash value of the leaf node according to content in the leaf node, and generating or updating an upper-layer tree node according to the hash value of the leaf node and the state key, until a root node of the tree; updating a version in a corresponding memory page according to a divided logical page structure, the memory page being used to represent content of all tree nodes in the logical page, the logical page containing a page identifier, the page identifier including a lexicographic content and a version of an upper-layer tree node in the logical page. 2.The method of claim 1, wherein the logical page is a result of aggregating a plurality of adjacent tree nodes according to a node association relationship of the tree structure, at least one tree node is contained in one logical page, and different tree nodes are contained in different logical pages. 3.The method of claim 2, wherein the base page contains a page identifier, the page identifier including a lexicographic content and a version of an upper-layer tree node in the logical page.

4. The method of claim 1, further comprising: generating a base page and a delta page based on a change of a logical page with consecutive versions. 5.The method of claim 4, wherein the base page corresponds to one memory page and contains all tree nodes in the corresponding memory page. 6.The method of claim 4, wherein the delta page corresponds to one or more memory pages and contains tree nodes in one or more memory pages with consecutive versions that have changed relative to a previous adjacent base page / delta page / version.

7. The method of claim 1, further comprising: storing the state value in a data file of a persistent storage medium, the leaf node including a location index of the state value in the data file of the persistent storage medium. 8.The method of claim 4, wherein the base page and the delta page have the same page ID as the corresponding memory page. 9.The method of claim 4, wherein the base page has the same version as the corresponding memory page, and the version of the delta page is the lowest or highest version in one or more memory pages corresponding to the delta page. 10.The method of claim 9, wherein the generating a base page and a delta page based on a change of a logical page with consecutive versions comprises: generating a base page and / or a delta page corresponding to a version based on a change of a memory page with consecutive versions. 11.The method of claim 4, wherein the page identifier of the base page and the delta page further includes a page type to distinguish the base page and the delta page. 12.A computer device, comprising: a processor; and a memory having a program stored therein, wherein the processor executes the program to perform operations of any one of claims 1-11.

Images