Sparse Merkel tree storage and verification method and device, medium and equipment

Abstract

The invention relates to the technical field of data storage, and particularly provides a sparse Merkel tree storage and verification method and device, a medium and equipment, and the method can comprise the steps: constructing a multi-way tree structure for storing original data based on a key value corresponding to the original data; determining tree path coordinates of the original data in the multi-way tree structure; the tree path coordinates comprise all nodes involved in storing the original data; and storing the original data into a compressed target leaf node corresponding to the tree path coordinate, and locally updating a related node hash value from bottom to top along a path corresponding to the tree path coordinate to generate a newest Merkel root hash value. Through the hierarchical compression structure and dynamic node management, the sparse Merkel tree calculation complexity can be remarkably reduced, the storage overhead is reduced, the certificate data volume is reduced, and the method is suitable for storing massive sparse data scenes.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and more specifically, to a method, apparatus, medium, and device for storing and verifying sparse Merkle trees. Background Technology

[0002] A Sparse Merkle Tree (SMR) is a tree structure optimized for sparse datasets. It efficiently stores and verifies data through hash calculations and path compression. Unlike a regular Merkle tree, the leaf nodes of a SMR are not completely filled; they are filled only as needed. Currently, while SMRs theoretically only store non-empty leaf nodes, intermediate nodes still require storing a large number of pointers or hash values ​​to their child nodes. When the data key-value pairs are extremely long, resulting in a very deep tree, the storage overhead of intermediate nodes is very high, even if the data is sparse.

[0003] Therefore, how to provide a technical solution for sparse Merkle tree storage with low storage overhead has become an urgent technical problem to be solved. Summary of the Invention

[0004] The purpose of some embodiments of this application is to provide a method, apparatus, medium and device for storing and verifying sparse Merkle trees. The technical solutions of the embodiments of this application can reduce the storage overhead of sparse Merkle trees, reduce the complexity of sparse Merkle tree update calculations, and improve update and verification efficiency.

[0005] In a first aspect, some embodiments of this application provide a method for storing sparse Merkle trees, comprising: constructing a multi-branch tree structure for storing the original data based on the key values ​​corresponding to the original data; the tree depth of the multi-branch tree structure being related to the number of key values; determining the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates including all nodes involved in storing the original data; storing the original data in the compressed target leaf node corresponding to the tree path coordinates, and locally updating the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value.

[0006] Some embodiments of this application construct a multi-branch tree structure according to the number of key-value groups. After determining the tree path coordinates of the original data, the response data is stored in the compressed target leaf node pointed to by the tree path coordinates, and the latest Merkle root hash value is obtained by updating. Embodiments of this application can reduce the storage overhead of the sparse Merkle tree, while also reducing the computational complexity of updating the sparse Merkle tree and improving update and verification efficiency.

[0007] In some embodiments, constructing a multi-branch tree structure for storing the original data based on the key value corresponding to the original data includes: grouping the key value according to a fixed number of bits to obtain multiple grouping sequences; constructing the multi-branch tree structure based on the number of groups in the grouping sequence; wherein each node in the multi-branch tree structure includes a node hash value; each intermediate node also includes a left branch node and a right branch node.

[0008] Some embodiments of this application can reconstruct multi-branch trees by grouping key values ​​according to a fixed number of bits and constructing a multi-branch tree structure with the number of groups as the tree depth. This achieves compression of the original extremely deep binary tree and reduces storage overhead.

[0009] In some embodiments, determining the tree path coordinates of the original data in the multi-branch tree structure includes: mapping the multiple grouping sequences layer by layer to the multi-branch tree structure to generate the tree path coordinates; wherein the tree path coordinates are represented in hexadecimal format.

[0010] Some embodiments of this application generate tree path coordinates by mapping multiple grouping sequences to a multi-branch tree structure, providing a basis for subsequent data storage.

[0011] In some embodiments, storing the original data into the compressed target leaf node corresponding to the tree path coordinates includes: obtaining the compressed target leaf node pointed to by the tree path coordinates of any group sequence in the original data; and storing the data corresponding to the any group sequence and the tree path coordinates into the compressed target leaf node.

[0012] Some embodiments of this application achieve effective data storage by determining the compressed target leaf node pointed to by the tree path coordinates and then storing the corresponding content therein.

[0013] In some embodiments, storing the original data into the compressed target leaf node corresponding to the tree path coordinates includes: determining that there exists target node data in the multi-branch tree structure that has the same prefix path as any tree path coordinate of any grouping sequence of the original data; adding a branch node under the same prefix path, and storing the data corresponding to any grouping sequence and the any tree path coordinate into the branch node; wherein the branch node serves as the compressed target leaf node.

[0014] Some embodiments of this application can achieve node reuse by determining the same prefix path between nodes, eliminating the need to create redundant nodes and reducing storage overhead.

[0015] In some embodiments, storing the original data in the compressed target leaf node corresponding to the tree path coordinates, and locally updating the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value includes: starting from the compressed target leaf node, updating the hash values ​​of all related intermediate nodes layer by layer along the tree path coordinates until the latest Merkle root hash value is obtained.

[0016] Some embodiments of this application update the tree hash value by updating the hash value of all intermediate nodes involved in the tree path coordinates until the Merkle root node is updated, thus reducing computational overhead.

[0017] In some embodiments, the method further includes setting the hash value of child nodes in the multi-way tree structure that do not store any data to a default value.

[0018] Secondly, some embodiments of this application provide a method for verifying a sparse Merkle tree, including: obtaining the node hash value of the leaf node corresponding to the data to be verified; starting from the leaf node, calculating the hash value of the associated node layer by layer based on the node hash value until obtaining the root hash value to be verified; determining the verification result by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein, the verification result indicates whether the data to be verified has been tampered with.

[0019] Thirdly, some embodiments of this application provide an apparatus for sparse Merkle tree storage, comprising: a construction module for constructing a multi-branch tree structure for storing the original data based on the key values ​​corresponding to the original data; the tree depth of the multi-branch tree structure is related to the number of key values; a determination module for determining the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates include all nodes involved in storing the original data; and a storage module for storing the original data into the compressed target leaf node corresponding to the tree path coordinates, and locally updating the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value.

[0020] Fourthly, some embodiments of this application provide an apparatus for verifying a sparse Merkle tree, comprising: an acquisition module for acquiring the node hash value of the leaf node corresponding to the data to be verified; a calculation module for calculating the hash value of associated nodes layer by layer based on the node hash value, starting from the leaf node, until the root hash value to be verified is acquired; and a verification module for determining the verification result by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein the verification result indicates whether the data to be verified has been tampered with.

[0021] Fifthly, some embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0022] In a sixth aspect, some embodiments of this application provide an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the method as described in any embodiment of the first aspect.

[0023] In a seventh aspect, some embodiments of this application provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of some embodiments of this application, the accompanying drawings used in some embodiments of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart of a method for sparse Merkle tree storage provided for some embodiments of this application; Figure 2 A schematic diagram of a multi-branch tree structure is provided for some embodiments of this application; Figure 3 One of the schematic diagrams of leaf node insertion provided for some embodiments of this application; Figure 4 One or two schematic diagrams of leaf node insertion provided for some embodiments of this application; Figure 5 A flowchart of a method for verifying sparse Merkle trees provided for some embodiments of this application; Figure 6 A block diagram of an apparatus for sparse Merkle tree storage provided for some embodiments of this application; Figure 7 A block diagram of the apparatus for verifying sparse Merkle trees provided for some embodiments of this application; Figure 8 A schematic diagram of an electronic device provided for some embodiments of this application. Detailed Implementation

[0026] The technical solutions of some embodiments of this application will now be described with reference to the accompanying drawings.

[0027] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] In related technologies, sparse Merkle trees are well-suited for storing sparse datasets. The design of sparse Merkle trees makes verifying the existence of a data item in the tree highly efficient. Even with a very sparse dataset, only the path (i.e., hash chain) related to the target data item needs to be verified. This saves storage space and improves verification speed. However, while traditional sparse Merkle trees theoretically only store non-empty leaf nodes, intermediate nodes still need to store a large number of pointers or hash values ​​pointing to child nodes (mostly H0). When the key length is extremely large (e.g., 160 bits), resulting in a very deep tree (160 levels), the storage overhead of intermediate nodes is significant even with sparse data. Inserting or updating a leaf node requires calculating and updating the hash values ​​of all nodes along the entire path from the leaf to the root (160 levels), resulting in a time complexity of O(log N), where the cardinality of log N is too large (160). Consequently, generating and verifying existence proofs requires passing and processing paths with up to 160 hash values. This efficient handling of a large number of empty nodes (H0) and path processing logic increases the implementation complexity.

[0029] In view of this, some embodiments of this application provide a method for sparse Merkle tree storage. This method groups the key-value pairs corresponding to the original data, constructs a multi-branch tree structure according to the number of groups, stores the corresponding data according to the nodes pointed to by the tree path coordinates, and updates the hash values ​​of the relevant nodes until the Merkle root hash value of the root node of the tree is updated. Since the depth of the multi-branch tree structure is set according to the number of scores, compared to directly constructing a binary tree using the key-value length, the tree depth can be compressed, thereby reducing storage overhead and computational complexity. Simultaneously, it can also reduce data computational complexity during verification, improving verification efficiency.

[0030] The following is in conjunction with the appendix Figure 1 The implementation process of sparse Merkle tree storage provided in some embodiments of this application is illustrated by way of example. The implementation process of sparse Merkle tree storage can be executed by a terminal, which can be a computer terminal, a processor terminal, or a server terminal. The embodiments of this application are not specifically limited to this.

[0031] Please see the appendix Figure 1 , Figure 1A flowchart of a method for sparse Merkle tree storage is provided for some embodiments of this application. The method for sparse Merkle tree storage may include: S110, Based on the key values ​​corresponding to the original data, construct a multi-branch tree structure to store the original data; the tree depth of the multi-branch tree structure is related to the number of key values.

[0032] For example, in a specific embodiment of this application, the long key value K (i.e., the key value) corresponding to the original data (such as an account address) is subjected to hierarchical compression to construct a multi-way tree structure (referred to as a multi-way tree) that can store the original data. The key value K adopts a binary representation. The tree depth is related to the number of layers (i.e., the number or the number of groups) of K in the hierarchical compression.

[0033] In some embodiments of this application, S110 may include: grouping the key value according to a fixed number of bits to obtain multiple grouping sequences; constructing the multi-branch tree structure based on the number of groups in the grouping sequence; wherein each node in the multi-branch tree structure includes a node hash value; each intermediate node also includes a left branch node and a right branch node.

[0034] For example, in a specific embodiment of this application, key values ​​of length K are grouped into fixed number of bits B, and a Y-ary tree (Y=2^B) with a tree depth of M = L / B is constructed to compress the original binary tree based on the key value from the K-level to the M-level.

[0035] For example, a long key value K (e.g., a 160-bit account address) is grouped into fixed-length segments (e.g., B=4, with each group of 4 bits corresponding to a hexadecimal character). The 160-bit key value is divided into 40 4-bit sequence segments (as a concrete example of multiple grouping sequences); that is: K = Frag1 || Frag2 || ... || Frag40. Each sequence segment represents a portion of the original data. A multi-way tree is constructed with a depth equal to the number of groups, i.e., M=40 levels; for example, this multi-way tree is a 16-way tree (i.e., Y=16) because the 4-bit groups have 16 possible values. Each level of the multi-way tree corresponds to a sequence segment Frag_i (where i takes values ​​from 1 to 40) of the key. Figure 2A structural diagram of a 16-ary tree is shown. The bottom layer of this 16-ary tree is the leaf node children, which contains 16 nodes Node

[16] ; it can be called a child node mapping table, using Index to represent fragment values ​​(0-15), and Value to represent the reference of the child node or its hash value. There are multiple intermediate nodes between the leaf nodes and the top-level root node, and each intermediate node Node contains the node hash value, left node, and right node. The root node represents the current state of the entire multi-ary tree, and its hash value RootHash is the state digest (i.e., the root hash value). By compressing the tree height (i.e., tree depth) from an exponential depth determined by the key length (e.g., 160 layers) to a linearly manageable number of layers (e.g., 40 layers), the path length for traversal and operation is greatly reduced.

[0036] S120, determine the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates include all nodes involved in storing the original data.

[0037] For example, in a specific embodiment of this application, tree path coordinates corresponding to each group sequence are generated by mapping multiple grouping sequences corresponding to the original data into a 16-ary tree. These tree path coordinates represent the storage path of the data corresponding to each group sequence, and this storage path contains multiple nodes in the tree.

[0038] In some embodiments of this application, S120 may include: mapping the plurality of grouping sequences layer by layer to the multi-branch tree structure to generate the tree path coordinates; wherein the tree path coordinates are represented in hexadecimal.

[0039] For example, in a specific embodiment of this application, by mapping each grouping sequence layer by layer into a 16-ary tree, tree path coordinates represented in hexadecimal can be obtained. For example, a tree path coordinate is 0x326A1C45 or 0x326AB162. Taking 0x326A1C45 as an example, 0x represents hexadecimal, 3 represents Index=3, and Value is the node pointing to the remaining path 26A1C45.

[0040] S130, the original data is stored in the compressed target leaf node corresponding to the tree path coordinates, and the relevant node hash values ​​are locally updated from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value.

[0041] For example, in a specific embodiment of this application, the target child node (i.e., the compressed target leaf node) under which the data Data corresponding to each group sequence should be stored can be read from the tree path coordinates. When data storage is completed, the state storage of the sparse Merkle tree is completed by updating the hash value of the corresponding target child node and other hash values ​​on its storage path until the root hash value (i.e., the latest Merkle root hash value) is updated.

[0042] In the Y-ary tree, each intermediate node stores only the mapping of the actual non-empty child nodes (the key is the B-bit grouping value, and the value is the child node information); for unstored child node positions, a predefined default zero hash value H0 is used to replace the parent node hash when calculating the parent node hash; when inserting a new leaf node, the node is dynamically created along its path, and existing nodes with shared path prefixes are reused.

[0043] The above process is illustrated below by example.

[0044] In some embodiments of this application, S130 may include: obtaining the compressed target leaf node pointed to by the tree path coordinates of any group sequence in the original data; and storing the data corresponding to the any group sequence and the tree path coordinates into the compressed target leaf node.

[0045] For example, in a specific embodiment of this application, Figure 3 This demonstrates the initial state of inserting the first target leaf node 0x326A... From the tree path coordinates 0x326A1C45, we know the target leaf node has Index=3, which is the 4th position in the table. 26A1C45 is the subsequent storage path for Data starting from the target leaf node; at this point, 0x326A1C45 and Data are stored under this target leaf node. The root node, children Node, only stores the entry with Index=3, whose value points to a node representing the remaining path 26A... (37 levels), or to the root hash of the subtree corresponding to that path.

[0046] For example, a certain child node 3 (i.e. Figure 3 The third cell in the children node (in the example) stores only the entry with index=2. The location and path of data storage are determined in the same way as above. Unused branches (such as the root node with key=0,1,2,4..15) do not exist in the children node, do not physically store any data, and are implicitly represented as H0 (as a concrete example of the default value).

[0047] In some embodiments of this application, S130 may include: determining that there is target node data in the multi-branch tree structure that has the same prefix path as any tree path coordinate of any grouping sequence of the original data; adding a branch node under the same prefix path, and storing the data corresponding to any grouping sequence and the any tree path coordinate to the branch node; wherein, the branch node serves as the compressed target leaf node.

[0048] For example, in a specific embodiment of this application, nodes with the same prefix can be merged to avoid creating redundant nodes. For instance, a new node needs to be inserted, with a tree path coordinate of 0x326AB162 (as a specific example of any tree path coordinate). First, searching along the path corresponding to 0x326AB162 reveals that the path of a previously inserted node was 3->2->6->A->..., and the path of the new node is 3->2->6->A->B->... They share a prefix at Frag1=3, Frag2=2, Frag3=6, and Frag4=A. The current storage location of the previous node is found (e.g., under the node representing 326A). When inserting the new node, it is discovered that a branch B needs to be added under the 326A node (as a specific example of a branch node). The system simply adds a new child node with Index=B (pointing to the 162... path) to the child nodes of the 326A node. Figure 4 As shown, a node is created under 0x326A as the target leaf node, storing 0x326AB162 and Data, i.e., 654. The shared prefix path 326A is reused, and no redundant nodes are created.

[0049] In some embodiments of this application, S130 may include: starting from the compressed target leaf node, updating the hash values ​​of all related intermediate nodes layer by layer along the tree path coordinates until the latest Merkle root hash value is obtained.

[0050] For example, in a specific embodiment of this application, after storing data in the corresponding target leaf node, the state of the entire tree will change. Therefore, it is necessary to use it as the starting point to update the hash values ​​of all intermediate nodes on the tree path coordinates until the latest root hash value (i.e., the latest Merkle root hash value) is obtained, thus completing the state storage of the sparse Merkle tree. For example, after adding a branch node to 326A, the hash value of node 326A itself is updated (because its child node set has changed), and then the hash values ​​of the parent nodes are updated upwards along the path until the root hash value is reached. In some embodiments of this application, the method for storing sparse Merkle trees further includes setting the hash value of child nodes in the multi-branch tree structure that do not store any data to a default value.

[0051] For example, in a specific embodiment of this application, any child node location that is not physically stored is considered as a default zero hash value H0 when a hash calculation or proof verification is required (as a specific example of a default value). H0 is a pre-calculated constant, such as H(0).

[0052] In addition, a leaf node can be inserted, updated, or deleted in a multi-way tree.

[0053] When inserting a node, following the method described above, calculate the path segments Frag1 to Frag40 based on the key K of the leaf node. Update from bottom to top, starting from the bottom-level leaf node, and find or create the next-level node based on Frag1. Recursively go down until the location of the target leaf node is found or created.

[0054] When updating leaf node data, its new hash value H_leaf_new is calculated. Starting from the current leaf node and tracing back to the root node, for each current node (leaf or intermediate node), if its content or its child node mapping changes, its hash value H_node_new is recalculated. The calculation rule is: H_node_new = H( H_child0 || H_child1 || ... || H_child15 ). The value of H_childX is determined as follows: If an Index=X exists in the children node, then take the hash value of that child node.

[0055] If Index=X does not exist in the children Node, then the default zero hash H0 is used.

[0056] The new hash value H_node_new of the current node is informed to its parent node. Upon receiving the hash change from the current node, the parent node updates the corresponding entry in its child nodes (if the value is a reference, the child node's internal state has changed; if it's a stored hash, the stored hash value needs to be updated), and then triggers a recalculation of its own hash value. Simultaneously, it aggregates the hash values ​​of all 16 underlying logical child nodes (if existing, the stored / reference value is used; otherwise, H0 is used). This process is repeated layer by layer upwards until the root node calculates the new RootHash (i.e., the Merkle root hash value).

[0057] It is understandable that when updating the leaf node data, the hash values ​​of each intermediate node on the path of the leaf node in the Y-ary tree are recalculated and updated layer by layer from bottom to top; the hash value of each intermediate node is calculated based on all its Y logical child nodes (the stored hash of the existing child node is taken, and the H0 of the non-existent child node is taken).

[0058] When deleting a child node, the hashes of all nodes related to that child node are also updated, and the updates are performed layer by layer upwards until the root node calculates a new RootHash.

[0059] The computational cost and number of nodes involved in each update operation are only related to the tree depth (40 levels), which is constant time O(1). Compared to the total data size N or O(log N), the cardinality decreases from 2^160 to 16^40, and the actual average path length is much less than 40, avoiding the need to rebuild the entire tree. That is, the storage space is only related to the number of non-empty leaf nodes and the uniqueness of their paths, with a complexity of approximately O(kN), where k is the average path length (much less than 40) and N is the number of leaf nodes, completely avoiding the need to store 2^160 level nodes.

[0060] The following is in conjunction with the appendix Figure 5 The implementation process of sparse Merkle tree verification provided by some embodiments of this application is illustrated by way of example.

[0061] Please see the appendix Figure 5 , Figure 5 A flowchart of a sparse Merkle tree verification method is provided for some embodiments of this application. The sparse Merkle tree verification method may include: S510, obtain the node hash value of the leaf node corresponding to the data to be verified.

[0062] For example, in a specific embodiment of this application, to prove the existence of data D (as a specific example of data to be verified) corresponding to key value K, the leaf node corresponding to K is located. The hash value H_leaf of the leaf node is calculated using data D (as a specific example of a node hash value).

[0063] Additionally, Merkel paths need to be collected, tracing back from the leaf nodes to the root node. For each node Node_i on the path, e.g., at level L; level L is a dynamic value, where L=1 when the root node is at the highest level, representing the current node's vertical position in the tree; the level number increases with each level deeper towards the leaf node (i.e., L gradually increases from root to intermediate level to leaf). At this point, all sibling nodes of Node_i need to be provided, i.e., the hash values ​​of the 15 child nodes under the parent node of Node_i, excluding the child node Frag_L pointing to Node_i. For sibling nodes that do not exist in the parent node's children Node, their hash values ​​are directly provided as the default zero hash H0. There is no need to store or transmit complete information about these non-existent nodes. Provide the leaf node data D or its hash value H(D), and finally provide the current root hash value RootHash.

[0064] S520, starting from the leaf node, calculate the hash value of the associated node layer by layer based on the node hash value until the root hash value to be verified is obtained.

[0065] For example, in a specific embodiment of this application, the hash value H_current (initially H_leaf) of the current leaf node and the hash values ​​H_sib1...H_sib15 of all its 15 sibling nodes are known. Based on the current leaf node's position index (i.e., Frag_L) within its parent node, these 16 hash values ​​(H_current and 15 H_sib15) are... Arrange them in order; the order must be consistent with the tree definition, such as indices 0 to 15. Calculate the hash value of the parent node of the current leaf node: H_parent_calc = H( H0 || H1 || ... || H15 ). Since the root node is compressed into the root node and its sixteen child nodes in this embodiment, the hash value of the root node will change when any node in the tree changes. Here, H_index is H_current, and the other positions are the corresponding sibling hash values, i.e., H0~H15. Use the calculated H_parent_calc as the target node hash value for the next layer calculation; calculate layer by layer in this order until the root hash value to be verified, RootHash_calc, is calculated.

[0066] S530, the verification result is determined by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein, the verification result indicates whether the data to be verified has been tampered with.

[0067] For example, in a specific embodiment of this application, RootHash_calc and RootHash are compared. If they are the same, it indicates that D has not been tampered with; otherwise, it has been tampered with.

[0068] As can be seen from the above, the amount of proof data (the list of sibling hashes on the path) is significantly reduced because the tree depth has been compressed during the multi-way tree construction process; this is because a large number of non-existent sibling nodes only need to be represented by a known constant H0. The computational cost of verification is also only related to the tree depth (40 levels).

[0069] As can be seen from the above embodiments of this application, this application achieves the following technical effects: 1) Significantly improved storage efficiency: Sparse Merkle trees only store non-empty nodes (nodes that actually contain data), eliminating the need to store all empty nodes. By using hash calculations to replace empty nodes with default values ​​(such as H0), physical storage requirements are drastically reduced. Through a hierarchical compression structure (e.g., 160-bit -> 40-level 16-ary tree) and a dynamic sparse node management mechanism (storing only non-empty nodes and merging nodes with the same prefix), physical storage requirements are reduced from a theoretical O(2^160) to an actual O(kN) (k << 40, N is the number of leaves). For example, storage requirements for accounts with tens of millions of records are reduced by more than 98%.

[0070] 2) Dynamic Update Performance Optimization: When performing local updates such as inserting, modifying, or deleting data, only the hash values ​​of nodes on the affected path need to be updated, without reconstructing the entire tree. The compressed storage structure further reduces the computational and storage overhead during updates. The incremental hash update algorithm ensures that insertion, modification, and deletion operations only need to update the hashes of up to 40 nodes on the affected path, optimizing the computational complexity from O(N) or O(160) to approximately O(1) (constant time, only related to a fixed number of levels), making it very suitable for high-frequency write scenarios.

[0071] 3) Verification efficiency and bandwidth saving: The compressed existence proof protocol uses the default zero hash H0 to represent unstored sibling nodes, which reduces the amount of data in the proof path by more than 90% (a large number of H0s do not need to be transmitted). The number of hash calculations required for verification is fixed to the tree height (40 times), which greatly improves the verification speed of light nodes and reduces network bandwidth consumption.

[0072] 4) Security Preservation: The cryptographic properties of Merkle trees are retained. The root hash fully represents the state of the dataset, and any data tampering will cause the root hash to change. The uniform use of the default zero hash H0 avoids the leakage of data distribution information.

[0073] 5) Scalability and compatibility: This method effectively overcomes the depth problem caused by ultra-long key values ​​(such as 160-bit addresses), making sparse Merkle trees truly feasible and efficient for use in storing massive sparse data (such as the global state of the blockchain).

[0074] Please refer to Figure 6 , Figure 6 The diagram illustrates a block diagram of a sparse Merkle tree storage apparatus provided in some embodiments of this application. It should be understood that this sparse Merkle tree storage apparatus corresponds to the method embodiments described above and is capable of performing the various steps involved in the method embodiments. The specific functions of this sparse Merkle tree storage apparatus can be found in the description above; detailed descriptions are omitted here to avoid repetition.

[0075] Figure 6The sparse Merkle tree storage device includes at least one software functional module that can be stored in memory or embedded in the sparse Merkle tree storage device in the form of software or firmware. The sparse Merkle tree storage device includes: a construction module 610, used to construct a multi-branch tree structure for storing the original data based on the key values ​​corresponding to the original data; the tree depth of the multi-branch tree structure is related to the number of key values; a determination module 620, used to determine the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates include all nodes involved in storing the original data; and a storage module 630, used to store the original data in the compressed target leaf node corresponding to the tree path coordinates, and locally update the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest updated Merkle root hash value for storing the multi-branch tree structure.

[0076] Please refer to Figure 7 , Figure 7 The diagram illustrates the composition of an apparatus for sparse Merkle tree verification provided in some embodiments of this application. It should be understood that this sparse Merkle tree verification apparatus corresponds to the method embodiments described above and is capable of performing the various steps involved in the method embodiments. The specific functions of this sparse Merkle tree verification apparatus can be found in the description above; detailed descriptions are omitted here to avoid repetition.

[0077] Figure 7 The sparse Merkle tree verification apparatus includes at least one software functional module that can be stored in a memory or embedded in the sparse Merkle tree verification apparatus in the form of software or firmware. The apparatus includes: an acquisition module 710, used to acquire the node hash value of the leaf node corresponding to the data to be verified; a calculation module 720, used to calculate the hash values ​​of associated nodes layer by layer based on the node hash value, starting from the leaf node, until the root hash value to be verified is obtained; and a verification module 730, used to determine the verification result by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein the verification result indicates whether the data to be verified has been tampered with.

[0078] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0079] Some embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can perform the operation of any of the methods corresponding to the methods provided in the above embodiments.

[0080] Some embodiments of this application also provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operation of any of the methods corresponding to the above embodiments provided in the above embodiments.

[0081] like Figure 8 As shown, some embodiments of this application provide an electronic device 800, which includes a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820. When the processor 820 reads the program from the memory 810 via a bus 830 and executes the program, it can implement the methods of any of the above embodiments.

[0082] Processor 820 can process digital signals and can include various computing architectures. Examples include complex instruction set computer architectures, reduced instruction set computer architectures, or architectures that implement multiple instruction set combinations. In some examples, processor 820 can be a microprocessor.

[0083] The memory 810 can be used to store instructions executed by the processor 820 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 820 of this disclosure embodiment can be used to execute the instructions in the memory 810 to implement the methods shown above. The memory 810 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0084] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0085] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0086] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for sparse Merkle tree storage, characterized in that, include: Based on the key values ​​corresponding to the original data, a multi-branch tree structure is constructed to store the original data; the depth of the multi-branch tree structure is related to the number of key values. Determine the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates include all nodes involved in storing the original data; The original data is stored in the compressed target leaf node corresponding to the tree path coordinates, and the relevant node hash values ​​are locally updated from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value.

2. The method as described in claim 1, characterized in that, The step of constructing a multi-way tree structure to store the original data based on the key-value pairs corresponding to the original data includes: The key values ​​are grouped according to a fixed number of bits to obtain multiple grouping sequences; The multi-branch tree structure is constructed based on the number of groups in the grouping sequence; wherein each node in the multi-branch tree structure includes a node hash value; each intermediate node also includes a left branch node and a right branch node.

3. The method as described in claim 2, characterized in that, Determining the tree path coordinates of the original data in the multi-branch tree structure includes: The multiple grouping sequences are mapped layer by layer into the multi-branch tree structure to generate the tree path coordinates; wherein the tree path coordinates are represented in hexadecimal.

4. The method according to any one of claims 2-3, characterized in that, The step of storing the original data into the compressed target leaf node corresponding to the tree path coordinates includes: Obtain the compressed target leaf node pointed to by the tree path coordinates of any group sequence in the original data; Store the data corresponding to any group sequence and the tree path coordinates into the compressed target leaf node.

5. The method according to any one of claims 2-3, characterized in that, The step of storing the original data into the compressed target leaf node corresponding to the tree path coordinates includes: Determine if there exists a target node in the multi-branch tree structure that has the same prefix path as any tree path coordinate of any grouping sequence of the original data; A branch node is added under the same prefix path, and the data corresponding to any group sequence and the coordinates of any tree path are stored in the branch node; wherein, the branch node serves as the leaf node of the compression target.

6. The method according to any one of claims 1-3, characterized in that, The step of storing the original data into the compressed target leaf node corresponding to the tree path coordinates, and locally updating the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value includes: Starting from the compressed target leaf node, update the hash values ​​of all related intermediate nodes layer by layer along the tree path coordinates until the latest Merkle root hash value is obtained.

7. The method according to any one of claims 1-3, characterized in that, The method further includes setting the hash value of child nodes in the multi-way tree structure that do not store any data to a default value.

8. A method for sparse Merkle tree verification, characterized in that, The sparse Merkle tree is stored using the method described in any one of claims 1-7; the method includes: Obtain the node hash value of the leaf node corresponding to the data to be verified; Starting from the leaf node, calculate the hash value of the associated node layer by layer based on the node hash value until the root hash value to be verified is obtained; The verification result is determined by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein, the verification result indicates whether the data to be verified has been tampered with.

9. A device for sparse Merkle tree storage, characterized in that, include: The construction module is used to construct a multi-branch tree structure for storing the original data based on the key values ​​corresponding to the original data. The tree depth of the multi-branch tree structure is related to the number of key values; A determining module is used to determine the tree path coordinates of the original data in the multi-branch tree structure; the tree path coordinates include all nodes involved in storing the original data; The storage module is used to store the original data into the compressed target leaf node corresponding to the tree path coordinates, and locally update the hash values ​​of related nodes from bottom to top along the path corresponding to the tree path coordinates to generate the latest Merkle root hash value.

10. An apparatus for sparse Merkle tree verification, characterized in that, The sparse Merkle tree is stored using the method described in any one of claims 1-7; the method includes: The acquisition module is used to obtain the node hash value of the leaf node corresponding to the data to be verified; The calculation module is used to calculate the hash value of the associated nodes layer by layer, starting from the leaf node and based on the node hash value, until the root hash value to be verified is obtained; The verification module is used to determine the verification result by judging whether the root hash value to be verified is the same as the Merkle root hash value; wherein the verification result indicates whether the data to be verified has been tampered with.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the method as described in any one of claims 1-8.

12. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the computer program is executed by the processor to perform the method as claimed in any one of claims 1-8.

Images