A method for improving verifiable storage efficiency through multi-tier AMTs

Abstract

A method for improving the efficiency of verifiable storage by multi-level AMTs, the multi-level AMTs tree includes N-level AMTs, the first level AMT is the root AMT, the second level to the Nth level are all child AMTs of the previous level, each level AMT includes a verifiable storage commitment, M1 storage vector spaces, and each storage vector space includes M2 vector space elements. The method includes a construction step S1 and a verification step S2, and the multi-level AMTs share the same set of pre-computed and cached elliptic curve points, so that the application can solve the limited key-value pair storage problem in the current verifiable storage scheme by AMTs, the application can expand the limited key-value pair storage to support unlimited key-value pair storage and effectively avoid expensive elliptic curve multiplication operations, which can greatly reduce I / O read and write and improve the execution throughput of the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of big data processing and information technology, and more particularly to a method for improving verifiable storage efficiency through multi-level Authenticated Multipoint Evaluation Trees (AMTs). Background Art

[0002] As blockchain technology develops and its application scenarios gradually expand, a complete blockchain system will inevitably include a storage module. Overall, blockchain systems can indeed play a role in data persistence. Currently, typical blockchain systems use a key-value format to store blockchain ledger state, with the primary key being the keyword and the value being the value.

[0003] Key-value distributed storage systems offer fast query speeds, large data volumes, and high concurrency, making them ideal for queries based on primary keys. A key-value database stores data as key-value pairs, similar to a Java map. The entire database can be thought of as a large map, where each key corresponds to a unique value.

[0004] However, key-value distributed storage systems cannot perform complex conditional queries. When modifying a key-value pair, the hashes of all nodes on the path from the leaf node to the root node need to be calculated and updated.

[0005] See also Figure 1 , Figure 1 The following is a schematic diagram of the structure of a key-value distributed storage system. Figure 1 As shown, each update along the path amplifies the I / O read and write operations of the underlying storage. For example, a simple payment transaction involves at least two state updates to the ledger, namely, deducting and increasing the balances of two accounts. As the throughput of new blockchains approaches thousands of transactions per second, storage has become a new bottleneck in blockchain development.

[0006] Currently, there are some solutions in the industry to optimize the above problems, such as the AMTs (authenticated multipoint evaluation tree)-based method for scheduled updates, which can reduce I / O reading and writing.

[0007] AMTs calculates its Lagrangian interpolation, that is, satisfies f(x)=a i (where 1≤i≤8), the principle is as follows:

[0008] First, split f(x) into two functions: f0(x) and f1(x), that is

[0009] f(x)=f0(x)+f1(x)

[0010] in,

[0011]

[0012] Similarly, by splitting down in sequence, f0(x) can be split into f 0,0 (x) and f 0,1 (x), that is, f0(x)=f 0,0 (x)+f 0,1 (x), repeat the above steps to obtain the composition of each node tree of AMTs.

[0013] See also Figure 2 , Figure 2 The figure shows the structure diagram of AMTs that can verify the composition of each node tree. Figure 2 As shown, each node in the AMTs tree consists of two parts:

[0014] (1) The commitment form of the function;

[0015] (2) Batch proof of the index for which this function is always 0.

[0016] Please refer to Figure 3 , Figure 3 The diagram shows the change path involved when the value at x=7 is updated in AMTs verifiable storage and the verifiable proof points. Figure 3 As shown in the figure, when a storage entry needs to be updated, AMTs will go from the root to the leaf node corresponding to the entry and update the associated elements on the path. The other nodes remain unchanged, and when proving, it is only necessary to prove that the value of its neighbor node is 0; therefore, the AMTs scheme can spend less time maintaining the proof than the MPT scheme.

[0017] However, the AMT method also has its shortcomings. A k-bit key space parameter requires 2 k In order to update the elliptic curve parameter points more efficiently, AMTs improves the update efficiency by pre-calculating and caching these parameters. However, its pre-calculation and caching have certain requirements on the cache space. For example, for a 32-bit tree, the pre-calculation and cache space exceeds 256G, and most ledgers use 256 bits. It is conceivable how much space is needed for this pre-calculation and cache. In other words, the current AMTs solution is not enough to support unlimited bits to support its storage of key values, that is, its storage of key-value pairs is limited. Summary of the Invention

[0018] The purpose of the present invention is to provide a method for improving the efficiency of verifiable storage through multi-level AMTs, which can effectively utilize the elliptic curve parameters pre-calculated and stored in the AMTs tree, and expand the limited storage of key-value pairs to unlimited storage by using multiple levels and each level sharing the elliptic curve parameters of AMTs.

[0019] To achieve the above object, the technical solution of the present invention is as follows:

[0020] A method for improving verifiable storage efficiency using multi-level AMTs, characterized in that the multi-level AMT tree includes N levels of AMTs, with the first level AMT serving as the root AMT and the second to Nth levels being sub-AMTs of the previous level. Each level AMT includes the following information:

[0021] i. Verifiable storage commitment, i.e., the verifiable storage commitment of the layer for the entire AMTs tree;

[0022] ii.M1 storage vector space;

[0023] iii. Each storage vector space includes M2 vector space elements; wherein the first M2-1 storage slots of the storage vector space elements are used to store key and version information (key, version) that can verify the stored data, and the Nth slot of each storage vector space element is used to mark the identifier or index of the next level AMT tree; the storage vector space elements are stored in the leaf nodes of the AMTs tree at that level;

[0024] The method includes a blockchain storage step S1 and a verification step S2:

[0025] The step S1 specifically includes:

[0026] Step S11: constructing the multi-level AMTs, wherein the AMTs tree is used to store key and version information of verifiable stored data; Step S11 includes:

[0027] Step S111: When assigning verifiable storage to a new storage entry, the node's key is first calculated based on the storage entry, and the version is set to an initial value. A search is then performed on the root AMT. If the first-level sub-AMT has no available slots, the sub-AMTs of the first-level sub-AMT are searched, and so on, until a sub-AMT at a certain level is found with an available slot for storing the key and version information (key, version) of the verifiable storage data.

[0028] Step S112: When the current node finds a blank slot to update a storage entry, the version number of the verifiable storage commitment of the level is incremented by 1. Correspondingly, the version numbers of the verifiable storage commitments of all levels above it are also incremented by 1. From the root of the AMTs tree to the node corresponding to the storage entry, the version information in the key and version (key, version) of the verifiable storage data on the node is updated. That is, when the data of the leaf node changes, only the version number of the associated node on the path from the node to the root node needs to be incremented by 1.

[0029] Step S12: Expand the data of the storage entries in each block to construct a Merkle tree to store the keyword, version and data value information (key, version, value) of each transaction block;

[0030] When the AMTs tree needs to add a storage entry, the Merkle tree adds a new node and stores the key, version number information and value in one of the nodes of the Merkle tree to form the key, version and data value information (key, version, value) and generate a verifiable storage proof;

[0031] When the AMTs tree needs to modify a storage entry, the Merkle tree adds a new node and stores the keyword Key, the version number information increased by 1, and the value of Value on the node of the Merkle tree to form the keyword, version, and data value information (key, version, value) and generate a verifiable storage proof;

[0032] The verifiable step S2 includes:

[0033] Using the node added to the Merkle tree, the verifiable storage proof finds the value of the Value of the node on the AMTs tree according to the keyword and version number information (key, version) of the node on the AMTs tree.

[0034] Furthermore, the multi-level AMTs all share the same set of pre-calculated and cached elliptic curve parameters.

[0035] The present invention also provides a readable storage medium storing executable instructions for executing the method for improving the efficiency of verifiable storage through multi-level AMTs.

[0036] The application further provides a data processing terminal used as a consensus node of a P2P network, receiving, consensus verifying and storing blocks in the P2P network, comprising:

[0037] The readable storage medium described above;

[0038] A processor, which invokes and executes executable instructions in the readable storage medium to perform the method for improving the verifiable storage efficiency by multi-level AMTs.

[0039] The application further provides a P2P network, wherein the consensus node of the P2P network is the data processing terminal described above.

[0040] From the technical solution described above, it can be seen that the application provides a new type of blockchain verifiable storage method based on AMTs tree and updating the proof by versioning, which can solve the limited key-value pair storage problem in the current verifiable storage scheme by AMTs, and can expand the limited key-value pair storage to support unlimited key-value pair storage and effectively avoid the expensive elliptic curve multiplication operation, thereby greatly reducing the I / O read and write and improving the blockchain execution throughput. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 Fig. 1 shows a structure schematic diagram of a distributed storage system in the form of key-value;

[0042] Figure 2 Fig. 5 shows a structure schematic diagram of the composition of each node tree of AMTs verifiable storage;

[0043] Figure 3 Fig. 8 shows a schematic diagram of the change path and verifiable proof point involved in the value update at x=7 of AMTs verifiable storage;

[0044] Figure 4 Fig. 11 shows a structure schematic diagram of multi-level and versioning based on AMTs in the embodiment of the application;

[0045] Figure 5 Fig. 14 shows a schematic diagram of storing version number information by AMTs tree in the embodiment of the application; DETAILED DESCRIPTION

[0046] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings.

[0047] It should be noted that in the following specific embodiments, in order to clearly show the structure of the application for the purpose of illustration, the structure in the drawings is not drawn according to the general proportion, and is partially enlarged, deformed and simplified, therefore, it should be avoided to be understood as the limitation of the application.

[0048] In the specific embodiments of the present application, the main idea is to propose a new type of blockchain storage method based on multi-level and versioning, in order to make AMTs scalable and store unlimited keys, it introduces the concept of multi-level AMTs, that is, the multi-level AMTs tree includes N-level AMTs, the first level is the root AMT as the root AMT, the second level to the Nth level are the child AMTs of the last level, and each level AMT includes the following information:

[0049] i. Verifiable storage commitment, that is, the level of verifiable storage commitment for the whole AMTs tree;

[0050] ii. M1 storage vector spaces;

[0051] iii. Each storage vector space includes M2 vector space elements; wherein the first M2-1 storage slots of the storage vector space element are used to store the key and version information (key, version) of the verifiable storage data, and the Nth slot of each storage vector space element is used to mark the identification or index of the next level AMT tree; the storage vector space element is stored in the leaf node of the AMTs tree of the level.

[0052] Wherein, in the verifiable storage commitment, the elliptic curve parameters of each level are the same, that is, the same set of elliptic curve parameters is shared by the multi-level AMTs, and the way of indexing each level AMT to the next level AMT is designed to expand the limited key-value pair storage to unlimited.

[0053] Please refer to Figure 4 , Figure 4 The figure shows the multi-level and versioning structure based on AMTs in the embodiments of the present application. The present application proposes a new type of versioned blockchain verifiable storage method, which stores the key and version number information of the verifiable storage data (key, version) through the AMTs tree, and stores the information of the key, version and data value (key, version, value) by using the merkle tree.

[0054] The method can include an AMTs-based blockchain storage step S1 and a verifiable step S2.

[0055] The step S1 specifically includes:

[0056] Step S11: constructing the multi-level AMTs, the AMTs tree is used to store the information of key and version (key, version) of verifiable storage data; the step S11 can include:

[0057] Step S111: when a new storage entry data is assigned verifiable storage, first calculate the key of the node according to the storage entry, and set the version to the initial value; then, search from the root AMT, if the first level child AMT has no free Slot, search the child AMT of the first level child AMT, and so on, until a certain level child AMT has a free Slot to store the key and version information (key, version) of verifiable storage data.

[0058] As shown in Figure 4 , the figure is exemplarily illustrated with N equal to 3, M1 being 4, and M2 being 5.

[0059] Specifically, when a new data needs to be assigned verifiable storage, first search from the root AMT, if the first level has no free Slot, search the child AMT thereof, as shown in Figure 4 , find the AMT of the third level by this method, find a free Slot to store the key, that is, the Slot of key 100111 in the figure (the slot with a pentagram in the box in the figure).

[0060] Step S112: when the current node finds a blank Slot to update the storage entry, add 1 to the version number of the verifiable storage commitment of the level, and correspondingly, add 1 to the version number of the verifiable storage commitment of all levels above it, from the root of the AMTs tree to the node corresponding to the storage entry, update the version information of the key and version (key, version) of the verifiable storage data on the node, that is, when the data of the leaf node changes, only need to add 1 to the version number of the associated node on the path from the node to the root node.

[0061] That is, the AMTs tree is used to update the change of version, that is, the change of key and version (key, version) information data of the node, when the node information data changes, only need to add 1 to the version number on the path from the node to the root node, and the operation performed is addition operation, C' = C + G, wherein C' represents the new state to be calculated, C represents the previous state, and G represents the starting parameter of the elliptic curve, which is a constant under the condition of the elliptic curve.

[0062] Specifically, as shown in Figure 5As shown on the left, when the current node finds a blank slot to update the key value, Sub-AMT-(2, 1) in the figure stores the new key, and the version number of the commitment at that layer (i.e., commitment B) is incremented by 1. Correspondingly, the version number of Sub-AMT-2 (i.e., commitment A) above it is also incremented by 1, and Sub-AMT-2j is incremented by +1. Recursively, the version number of the Root AMT (i.e., commitment R) above it also needs to be incremented by 1.

[0063] Please refer to Figure 5 As shown on the right:

[0064] Step S12: Expand the data of the storage entries in each block to construct a Merkle tree for storing the keyword, version and data value information (key, version, value) of each transaction block.

[0065] When the AMTs tree needs to add a storage entry, the Merkle tree adds a new node and stores the keyword Key, version number information and Value value on a node of the Merkle tree to form the keyword, version and data value information (key, version, value) and generate a verifiable storage certificate (such as Figure 5 shown on the right).

[0066] When the AMTs tree needs to modify a storage entry, the Merkle tree adds a new node and stores the keyword Key, the version number information increased by 1, and the value of Value on one of the nodes of the Merkle tree, forming the keyword, version and data value information (key, version, value) and generating a verifiable storage proof.

[0067] In other words, the extended Merkle tree can be used to store keywords, versions, and data value information (key, version, value). It takes advantage of the Merkle tree's fast node addition and proof generation (since data modification and updating are slow, the Merkle tree is not updated). It only adds data, so that each block generates a corresponding Merkle tree, and each version change must be added as a node in the Merkle tree.

[0068] Therefore, it can be seen from the above technical solution of the present invention that it is unnecessary to perform the multiplication calculation for proof on the AMTs tree, but instead turns it into an addition calculation (the multiplication calculation requires 160us, while the addition calculation only requires 1us), which saves a lot of time in the generation and maintenance of verifiable storage.

[0069] Furthermore, by generating keyword, version, and data value information (key, version, value) nodes on the extended merkle tree, changes in value will not be updated on the merkle tree. Instead, new nodes will be added to the merkle tree through version changes and proofs will be generated, saving a lot of time.

[0070] The verifiable step S2 includes:

[0071] Using the node added to the Merkle tree, the verifiable storage proof finds the value of the Value of the node on the AMT tree according to the keyword and version number information (key, version) of the node on the AMTs tree.

[0072] To summarize, the core idea of ​​the technical solution of the present invention is that it uses multi-level AMTs to store the keyword and version (key, version) information of verifiable storage data and generate verifiable proof, and uses an extended Merkle tree to store (key, version, value) and generate verifiable proof, thereby achieving verifiable storage. At the same time, since multi-level AMTs share the same set of elliptic curve points, it greatly saves AMTs pre-calculation and caching space.

[0073] Furthermore, the present invention does not need to perform real-time updates on the AMT and pre-calculate and cache a large number of proof parameters, but can simply update (key, ver, value) and provide proof on the extended Merkle tree.

[0074] In an embodiment of the present invention, if the stored value Value is to be modified, it is necessary to add 1 to the version information on the AMTs and generate a certificate, store the (key, version, value) information on the extended merkle tree and generate a certificate.

[0075] Specifically, to modify the value of a storage node, first find the AMT point corresponding to the key value and increment the version number by 1. At the same time, the node versions along the line are also incremented by 1. In other words, the AMTs tree is used to store version number information, and the change in the node value is replaced by the change in the version number. The polynomial multiplication operation of AMTs is converted into an addition operation, saving computing time and reducing the number of read and write operations of the underlying I / O.

[0076] And, on the extended merkle tree, a node is added to store the key, the version number information after adding 1, and the value of the value in a new node of the merkle tree, and a verifiable storage proof is generated to prove that the value of the value can be found according to (key, version). That is, (key, version, value) is stored by the extended merkle tree and a proof is generated, and the tree is only used to store values by adding nodes without modifying values, thereby improving the efficiency of verifiable storage.

[0077] The application further provides a readable storage medium storing executable instructions for executing the method for improving the efficiency of verifiable storage by using multi-level AMTs.

[0078] The application further provides a data processing terminal used as a consensus node of a P2P network, receiving, consensus verifying and storing blocks in the P2P network, comprising a readable storage medium and a processor used for calling and executing executable instructions in the readable storage medium to execute the method for improving the efficiency of verifiable storage by using multi-level AMTs.

[0079] Those skilled in the art can understand that all or part of the steps of the above method can be instructed by a program to related hardware (for example, a processor), and the program can be stored in a readable storage medium, such as a read-only memory, a magnetic disk or an optical disk. All or part of the steps of the above embodiment can also be implemented by using one or more integrated circuits. Accordingly, each module in the above embodiment can be implemented in the form of hardware, for example, by using an integrated circuit to implement its corresponding function, or in the form of a software function module, for example, by using a processor to execute a program / instruction stored in a memory to implement its corresponding function. The embodiment of the application is not limited to any specific form of combination of hardware and software.

[0080] The application further provides a P2P network, wherein the consensus node of the P2P network is the data processing terminal.

[0081] The above description is only the preferred embodiment of the application, and the embodiment is not used to limit the patent protection scope of the application, so that any equivalent structural changes made according to the content of the specification and drawings of the application should also be included in the protection scope of the application.

Claims

1. A method for improving verifiable storage efficiency through multi-level AMTs, characterized in that: The multi-level AMTs tree includes N levels of AMTs. The first level AMT is the root AMT, and the second to Nth levels are all sub-AMTs of the previous level. Each level AMT includes the following information: i. Verifiable storage commitment, i.e., the verifiable storage commitment of the layer for the entire AMTs tree; ii.M1 storage vector space; iii. Each storage vector space includes M2 vector space elements; wherein the first M2-1 storage slots of the storage vector space elements are used to store keywords and version information that can verify the stored data, and the Nth slot of each storage vector space element is used to mark the identifier or index of the next level AMT tree; the storage vector space elements are stored in the leaf nodes of the AMTs tree at the level; The method includes a blockchain storage step S1 and a verification step S2: The step S1 specifically includes: Step S11: constructing the multi-level AMTs, wherein the AMTs tree is used to store information about keywords and versions of verifiable stored data; Step S11 includes: Step S111: When assigning verifiable storage to a new storage entry, the node's key is first calculated based on the storage entry, and the version is set to an initial value. A search is then performed on the root AMT. If the first-level sub-AMT has no available slots, the sub-AMTs of the first-level sub-AMT are searched, and so on, until a sub-AMT at a certain level is found with an available slot for storing the key and version information of the verifiable storage data. Step S112: When the current node finds a blank slot to update a storage entry, the version number of the verifiable storage commitment at that level is incremented by 1. Correspondingly, the version numbers of the verifiable storage commitments at all levels above it are also incremented by 1. From the root of the AMTs tree to the node corresponding to the storage entry, the version information in the keywords and versions of the verifiable storage data on the node is updated. That is, when the data of the leaf node changes, only the version number of the associated node on the path from the node to the root node needs to be incremented by 1. Step S12: Expand the data of the storage entries in each block to construct a Merkle tree for storing the keyword, version, and data value information of each transaction block; When the AMTs tree needs to add a storage entry, the Merkle tree adds a new node and stores the keyword Key, version number information and Value value on one of the nodes in the Merkle tree to form the data of the keyword, version and data value information, and generate a verifiable storage proof; When the AMTs tree needs to modify a storage entry, the Merkle tree adds a new node and stores the keyword Key, the version number information increased by 1, and the value of Value in one of the nodes of the Merkle tree to form the data of the keyword, version and data value information, and generate a verifiable storage proof; The verifiable step S2 includes: Using the node added to the Merkle tree, the verifiable storage proof finds the value of the Value of the node on the AMTs tree according to the keyword and version number information of the node on the AMTs tree.

2. The method for improving verifiable storage efficiency through multi-level AMTs according to claim 1, characterized in that: Multi-level AMTs all share the same set of pre-computed and cached elliptic curve points.

3. A readable storage medium, characterized in that: Executable instructions are stored, and the executable instructions are used to execute the method for improving verifiable storage efficiency through multi-level AMTs as described in claim 1 or 2.

4. A data processing terminal, characterized in that: The data processing terminal is used as a consensus node in a P2P network to receive, verify consensus, and store blocks in the P2P network. The data processing terminal includes: The readable storage medium according to claim 3; A processor is configured to retrieve and execute executable instructions in the readable storage medium to implement a method for improving verifiable storage efficiency through multi-level AMTs.

5. A P2P network system, characterized in that: The consensus node of the P2P network system is the data processing terminal described in claim 4.

Images