A blockchain-oriented method and system supporting efficient verifiable query

Abstract

The application discloses a kind of methods for supporting efficient verifiable query oriented to blockchain, using the disk friendly verifiable structure based on vector commitment, for the characteristics of only append write of state log, with disk block as unit organization and construction verifiable structure, and introduce vector commitment in cryptography to traditional Merkle hash tree, realize the constant level of proof size, to solve the problem that the proof overhead increases with the data to be proved when traditional Merkle hash tree is used to prove.At the same time, for the above-mentioned verifiable structure, the corresponding persistent storage scheme is proposed, combining Merkle hash tree and the characteristics of disk, solving the problem that the original Merkle hash tree is difficult to realize efficient persistence.Under as little additional storage as possible, strong reading and proof performance is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of blockchains, and relates to a method for supporting efficient and verifiable queries for blockchains, which uses a disk-friendly verifiable structure based on vector commitments. BACKGROUND

[0002] Now, the blockchain technology is also often used to solve the synchronization and verifiable storage of data of nodes in the case where there is no unified trusted central node. Unlike traditional storage systems, because the blockchain is based on the Byzantine assumption, in order to prevent the blockchain nodes from returning maliciously tampered data, the blockchain system often needs to provide verifiable queries to users. That is, the blockchain system not only needs to return the query result to the user, but also needs to return the corresponding proof so that the user can verify the correctness of the query result.

[0003] Common verifiable data structures mainly include Merkle hash tree and its variants such as Merkle Patricia Tree (MPT) and the like. MPT is the most commonly used data structure, which has the functions of indexing, multi-version storage and verifiable query. MPT stores one or several key characters at each layer, and each node can have multiple child nodes. Keys with common prefixes in the tree share paths. Each non-leaf node in MPT stores the hash of its child nodes, so that any modification of a child node will affect the hash of the parent node, and finally reflect on the root hash. For query, MPT can prove the leaf node by providing a proof. For a certain leaf node, the proof contains the hash of each node corresponding to the brother node on the root path, and the final root hash. For verification, if the hash result from the leaf to the root is recalculated according to the proof, it is the given root hash, then the value belongs to the state tree corresponding to the root. For updating, MPT adopts an incremental updating method similar to version control tree to provide version proof, that is, after updating the corresponding node in the original MPT, a new root node is generated by recalculating the hash, and the new root node represents the new version of MPT, and the unchanged node is shared by the new and old MPT. Although MPT better solves the problems of integrity proof and version proof, the maintenance of MPT often becomes a bottleneck of the blockchain storage system, and the main problem is that MPT is a memory index structure, and the existing scheme using a log-structured merge tree (LSM-Tree) for persistence has a large performance overhead; at the same time, in order to realize the version proof, MPT needs to adopt the update method of copy-on-write, and with the increase of the depth of the updated leaf node, the update overhead is large, including a large number of new node writing and data copying. The existing researches mainly start from the optimization of verifiable structure, or separately optimize the write amplification of LSM-Tree. For the problem of write amplification caused by the combination of MPT and LSM-Tree in the tamper-proof storage system, little optimization and improvement is made.

[0004] Therefore, in order to solve the read-write amplification problem caused by updating and reading in the original MPT scheme as much as possible, it is necessary to propose a method supporting efficient verifiable query. SUMMARY

[0005] In order to solve the problems in the prior art, the purpose of the present application is to provide a method for supporting efficient verifiable query for blockchains, which uses a disk-friendly verifiable structure based on vector commitment. This method introduces the cryptographic primitive vector commitment to compress the traditional Merkle tree verification structure. According to the characteristics of the state log that only appends writing, the verifiable structure is organized and constructed in units of disk blocks, so as to speed up the reading and proof of the verifiable structure.

[0006] The application provides a blockchain-oriented method for supporting efficient verifiable query, which comprises the following steps:

[0007] Step one, constructing a disk-friendly verifiable structure based on vector commitment;

[0008] Step two, proving and verifying based on the verifiable structure;

[0009] Step three, persistently storing the verifiable structure.

[0010] The application provides a blockchain-oriented method for supporting efficient verifiable query, which uses a disk-friendly verifiable structure based on vector commitment, and mainly comprises the construction, proving, verifying and persisting processes. The construction process of the verifiable structure comprises the following steps:

[0011] Step 1: After sorting the input state data according to the account address, the data is split into m state groups according to the maximum number of leaf nodes of the set sub-Merkle hash tree, and each state group corresponds to a sub-Merkle hash tree;

[0012] Step 2: The root of the Merkle hash tree corresponding to each state group is calculated as the data of the vector commitment, and the Merkle hash tree nodes are saved in the cache during the calculation process to facilitate the storage of the verification structure on the disk;

[0013] Step 3: The roots of the m sub-Merkle hash trees are constructed into a vector commitment according to the input vector commitment public parameters, and the corresponding commitment representing all states and the vector commitment proof of each position corresponding to all Merkle hash trees are calculated to accelerate the subsequent proving operation.

[0014] The core content of the verifiable structure obtained by construction is as follows Figure 1As shown, from top to bottom are the vector commitment, the Merkle hash tree and the actual state data, and the general idea of constructing it is to build a Merkle hash tree by grouping the state data and use vector commitment connection. The overhead brought by this construction process is mainly: the calculation of m sub-Merkle hash trees and the pre-computation of the vector commitment proof of all Merkle hash trees. Since these two parts are divided into m tasks due to the vector commitment, in the actual implementation process, multi-threading can be used to speed up. At the same time, for the pre-computation of the vector commitment proof of the second part, the main purpose of this part is to make full use of the advantage of fixed size of vector commitment proof, to optimize the efficiency of proof data generation and persistence, and to improve the proof performance of the user for the state data. Therefore, this step can be updated in an asynchronous or delayed manner, which will not affect the submission of the actual block state. And the setting of the vector commitment public parameter in the construction method can be set and saved by the trusted node responsible for controlling the access in the consortium chain, and the corresponding public parameters are generated according to the security parameters and broadcast to all nodes. And for the maximum number of leaf nodes of each small Merkle hash tree and the number of Merkle hash trees that need to be persisted, it will affect the proof size and the persistence way of the whole verification structure.

[0015] Corresponding to the construction process, the proof and verification of the verification structure can also be divided into two parts, corresponding to the Merkle hash tree and the vector commitment, wherein the proof process specifically includes:

[0016] Step 1: Find the account address to be proved on the state list, if the address is found, process the existence proof, directly prove the existence of the account address; otherwise, process the non-existence proof, which needs to provide the range proof of the left and right boundaries;

[0017] Step 2: For both existence proof and non-existence proof, the position information of the account to be proved needs to be generated, including the Merkle hash tree serial number and the position in the Merkle hash tree, since the state list has been sorted by account address, so it can be quickly queried. For the existence proof, the account to be proved is the account address to be proved, and for the non-existence proof, the range proof of the left and right boundaries of the account address to be proved needs to be returned;

[0018] Step 3: Give existence proof to all accounts in the list to be proved, each proof includes Merkle hash tree proof and vector commitment proof, wherein the Merkle hash tree proof needs to reconstruct the part of the Merkle hash tree required for proof according to the state list and the Merkle hash tree cache part generated during construction. For the vector commitment proof, the pre-computed vector commitment proof representing all Merkle hash trees can be directly read.

[0019] The overhead of the proof part is mainly concentrated in the proof construction part of the Merkle hash tree. In order to save the verification data storage overhead and reading performance as much as possible, this part takes the time-space strategy, and does not store the whole Merkle hash tree, but reconstructs the Merkle hash tree according to the state list in the proof process. It can be further found that in the Merkle hash tree, the storage size required to store any node is consistent, that is, 32B (SHA256 hash algorithm is adopted), however, the calculation overhead required for each node recalculation is different. The calculation overhead in the present application refers to the total number of hash calculations performed when each node is recalculated. Take Figure 2 for example, which shows the structure of a Merkle hash tree, where each node is marked with the number of hash calculations required to calculate the node. Take the proof of the leaf node d0 as an example, the proof data to be returned is Figure 2 Node 1 and Node 3 in the above formula, it can be found that the closer to the leaf node, the less the hash number required for recalculation. For example, node 3 needs to read the state of d2, d3 and perform 3 hash calculations to obtain. And node 1 only needs to read d1 and perform 1 hash calculation to obtain again. This calculation overhead is related to the number of leaf nodes n of the binary tree with the node as the root, which is (2*n-1) times. Through the above conclusion, it can be known that under the same storage overhead, storing the hash value closer to the root node can save more hash calculation overhead. Take Figure 2 for example, if node 3 is cached, for the verification of d0, only d1 needs to be read and the leaf node 1 needs to be calculated, only one hash calculation is required. This saves a large part of the hash recalculation overhead compared to the total of 4 hash calculations of the proof operation without caching any data.

[0020] The verification process of the verifiable structure can be divided into existence verification and non-existence verification. First, according to the query result, it is judged whether the returned proof is an existence proof or a non-existence proof. If the result is an existence proof, it is relatively simple. According to the Merkle proof in the proof data and the query result, the corresponding Merkle tree root hash is reconstructed, and then the vector commitment is used to verify whether the Merkle root hash is correct. If the result is a non-existence proof, it is first necessary to check whether the non-existence proof meets the conditions. First, whether the left and right boundaries of the range are legal, that is, whether the left and right boundaries contain the queried account address. If the left and right boundaries contain the queried account address, it means that the left and right boundaries are legal, and the subsequent judgment logic can be entered. If the left and right boundaries do not contain the queried account address, it means that the left and right boundaries are not legal, and the verification process is terminated directly. Second, it is necessary to judge whether the Merkle hash tree id where the left and right boundaries are located is one of equal or adjacent. If the Merkle hash tree id where the left and right boundaries are located is not one of equal or adjacent, the verification process is terminated. Otherwise, the Merkle hash tree root hash needs to be reconstructed according to the range query proof. Since there are two Merkle hash trees where the left and right boundaries are located, the list of root hashes is returned here. Finally, it is necessary to verify whether the root hash is correct through the vector commitment.

[0021] The overall idea of the persistent storage process of the verifiable structure is: in the storage process, the storage overhead and the number of I / O times during proof are mainly considered, and the verification data and the state data are stored in the same disk block as much as possible, so that only one disk I / O is consumed during proof. First, analyze the data required for state proof. The data required for verification structure proof mainly includes corresponding state hash, cached nodes of Merkle hash tree and corresponding vector commitment proof. For existence proof, since it can be indexed by multi-version query structure, only the leaf node hash of the sub-Merkle hash tree where it is located and the corresponding vector commitment proof need to be stored. After reading, the Merkle proof is reconstructed. Therefore, by adjusting the size and the number of cache layers of each sub-Merkle hash tree during the construction of the verification structure, the data required for proof and the state hash are stored in the same disk block.

[0022] For the non-existence proof, since there is no corresponding account address in the multi-version query structure, and in order to provide the non-existence proof, it is necessary to locate the left and right boundaries on the Merkle hash tree. Therefore, first, the maximum and minimum values of the account address in each sub-Merkle hash tree need to be stored to locate the corresponding account in the Merkle hash tree. Secondly, for the located sub-Merkle hash tree, the account data corresponding to the sub-Merkle hash tree needs to be read in the state log in the multi-version verifiable LSM, and the account data in the same Merkle hash tree must be continuous in the state log, so this part only needs the start address and end address in the state log for each sub-Merkle hash tree. Finally, on the read state data, the left and right boundaries are located by binary search, and the non-existence proof is provided on the sub-Merkle hash tree. This part needs to read the state data stored in the state log again, so the overhead will be large, but since the Bloom filter can achieve low false positives, the non-existence proof is rarely needed in actual process.

[0023] According to the above idea, the following persistence scheme can be taken Figure 3 The persistence scheme shown in the figure can be roughly divided into the following two parts:

[0024] The first part, for each sub-Merkle hash tree, takes a disk block to save the related data, including the cached Merkle hash tree node to speed up the proof, the Merkle hash tree node includes the corresponding state hash and vector commitment proof (VC proof) and some related metadata. Different sub-Merkle hash trees are written to the disk in sequence.

[0025] The second part, for the entire verifiable structure, also uses a disk block to save the related index data, Bloom filter, and vector commitment related data. The index includes the maximum and minimum values of the account address in each subsequent Merkle hash tree and the corresponding address on the state log, the Bloom filter saves all account addresses in the entire verifiable structure, and the vector commitment related data saves the root hash of each hash tree and the corresponding commitment.

[0026] For the existence proof of the verification structure, that is, the corresponding index data can be queried in the multi-version index, only the disk block where the state is located needs to be read. By using the characteristic that the minimum read-write unit of the operating system is the disk block, the number of I / O of the disk is reduced as much as possible. In the read disk block, since the vector commitment proof, that is, the VC proof in the figure, and the Merkle cache are fixedly stored at the head and tail addresses of the disk block, and the system can calculate the size of the Merkle cache area according to the number of layers of the Merkle hash tree that needs to be persisted. Therefore, the data in the area can be decoded again, so that the proof and reading of the state are realized. Therefore, the existence proof only needs to perform an additional disk I / O on the verifiable storage part after the system reads the corresponding account state value. For the non-existence proof of the verification structure, since there is no related data account address in the multi-version index, the verification structure needs to be queried. At this time, the first disk block of the verification structure needs to be read, and the maximum and minimum value ranges of the Merkle hash tree on each disk block in the index data are used to determine the disk blocks and state data that need to be read.

[0027] Based on the above method, the application further provides a blockchain-oriented system supporting efficient verifiable query, comprising a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the above method is realized.

[0028] Compared with the prior art, the beneficial effects of the application are that while effectively providing integrity and version verification, the performance and overhead are also optimized, the overhead of the verifiable structure proof is reduced from O(log(n)) of the conventional Merkle hash tree to O(1), so that the efficient operation of the blockchain system can be effectively supported. The original structure based on the Merkle hash tree is difficult to perform efficient persistence and indexing on the disk, so that the read amplification in the proof process is large, and the overhead of the proof based on the Merkle hash tree also increases with the increase of the size of the set to be proved. Therefore, the application introduces the vector commitment cryptographic primitive, combines the Merkle hash tree and the disk characteristics, and designs a disk-friendly verifiable structure. The structure can provide higher reading and proof performance with as little additional storage overhead as possible. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0030] Figure 1A verifiable structure based on vector commitment for the present application.

[0031] Figure 2 A Merkle cache strategy diagram for the present application.

[0032] Figure 3 A verifiable structure storage diagram for the present application. DETAILED DESCRIPTION

[0033] The application will be further described in conjunction with the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present application are the general knowledge and common sense in the art, and the present application does not have special limitations.

[0034] The present application discloses a blockchain-oriented method for supporting efficient verifiable query, which uses a disk-friendly verifiable structure based on vector commitment. In view of the characteristics of state log only appending write, the verifiable structure is organized and constructed in units of disk blocks, and a vector commitment in cryptography is introduced to the traditional Merkle hash tree, realizing constant level of proof size, thereby solving the problem that the proof overhead increases with the increase of the data to be proved when using the traditional Merkle hash tree to prove. Meanwhile, in view of the above-mentioned verifiable structure, a corresponding persistent storage scheme is proposed, which combines the characteristics of Merkle hash tree and disk, and solves the problem that the original Merkle hash tree is difficult to realize efficient persistence. With as little additional storage as possible, strong read and proof performance is realized.

[0035] The present application proposes a blockchain-oriented method for supporting efficient verifiable query, which involves the process mainly including construction, proof, verification and persistence of the disk-friendly verifiable structure based on vector commitment. The method includes the following steps:

[0036] Step one, constructing a disk-friendly verifiable structure based on vector commitment;

[0037] Step two, proving and verifying based on the verifiable structure;

[0038] Step three, persistently storing the verifiable structure.

[0039] The verifiable structure construction process includes the following steps:

[0040] Step 1, after sorting the input state list according to the account address, the maximum number of leaf nodes of the set Merkle hash tree is split into m state groups, and each state group corresponds to a set Merkle hash tree;

[0041] Step 2, the algorithm calculates the corresponding Merkle hash tree root of each state group as the data of the vector commitment, and saves the Merkle hash tree nodes to the cache during the calculation process to facilitate subsequent storage of the verification structure on the disk;

[0042] Step 3, the algorithm constructs the vector commitment according to the input vector commitment public parameters, and calculates the corresponding commitment representing all states and the vector commitment proof of all Merkle hash trees corresponding to each position, in order to speed up the subsequent proof operation.

[0043] Figure 1 is the core structure diagram of the verifiable structure, from top to bottom, respectively, the vector commitment, the Merkle hash tree and the actual state data, the construction idea is to construct the Merkle hash tree by grouping the state data using the hash function, and to connect using the vector commitment.

[0044] Figure 2 It shows how to give a non-existence proof through range proof, and the figure is a Merkle hash tree established on the state data sorted by account. If the user queries d2 data, the verification structure will give a non-existence proof through range proof. At this time, the left and right boundaries of d2, i.e. d1 and d3, will be found, and the relevant intermediate nodes, i.e. node 1 and node 2, will be given. The user receives the proof containing d1, d3, node 1 and node 2, and according to these data, the root hash can be recalculated. Thus, it is proved that d1 and d3 are adjacent in the Merkle hash tree. Since the state data has been sorted, it can be concluded that there is no d2. It is worth noting that in the verification structure, there may be a case where the left and right boundaries cross the Merkle hash tree. At this time, although two Merkle hash trees are involved, similar range proof can still be used, i.e. it is necessary to prove that the left boundary is the rightmost value of the left Merkle hash tree, and the right boundary is the leftmost value of the right Merkle hash tree, and it is proved that the two trees are adjacent through the vector commitment.

[0045] Figure 3 It shows the non-existence proof process of the verifiable structure combined with persistence. When the left and right boundaries fall on the same disk block, three disk I / Os are needed for proof, one for disk block 0 to read index data, the second for the disk block where the corresponding state address is located, and the third for the disk block where the corresponding Merkle hash tree is located. When the left and right boundaries fall on two disk blocks, four disk I / Os are needed. It is worth noting that the index data is usually small, and in the specific implementation, the cache can be used to save this part of disk I / O.

[0046] The main application scenario of the application is to optimize the verifiable query in the blockchain. The blockchain is a data management system with the characteristics of decentralization, tamper resistance and traceability, and can act as an intermediary for different organizations. The existing blockchain system uses MPT as a verifiable structure and uses LSM tree for persistence, and this scheme has a large performance overhead. The method supporting efficient verifiable query proposed by the application can be used to optimize the above scheme.

[0047] The protection scope of the application is not limited to the above embodiments. Changes and advantages that can be thought of by those skilled in the art without departing from the spirit and scope of the application are included in the application, and are protected by the appended claims.

Claims

1. A blockchain-oriented method of supporting efficient verifiable queries, characterized in that, Comprising the following steps: Step one, constructing a disk-friendly verifiable structure based on vector commitment; the verifiable structure construction process comprises the following steps: Step 1, after sorting the input state list according to the account address, split it into m state groups according to the maximum leaf node number of the set sub-Merkle hash tree, and each state group corresponds to a sub-Merkle hash tree; Step 2, calculate the Merkle hash tree root of each state group as the data of the vector commitment, and save the Merkle hash tree nodes to the cache during the calculation process to facilitate subsequent storage of the verification structure on the disk; Step 3, construct the vector commitment according to the input vector commitment public parameter, and calculate the corresponding commitment representing all states and the vector commitment proof of each position, which is used to speed up the subsequent proof operation; Step two, proof and verification based on the verifiable structure; The verification process of the verifiable structure is as follows: the verifiable query is divided into existence verification and non-existence verification; first, determine whether the returned proof is an existence proof or a non-existence proof according to the query result; if the result is an existence proof, reconstruct the corresponding Merkle root hash according to the Merkle proof in the proof data and the query result, and then verify whether the Merkle root hash is correct using the vector commitment; if the result is a non-existence proof, first check whether the non-existence proof meets the conditions, one is whether the left and right boundaries are legal, that is, whether the left and right boundaries contain the queried account address, if the left and right boundaries contain the queried account address, the left and right boundaries are legal, and the subsequent judgment logic can be entered; if the left and right boundaries do not contain the queried account address, the left and right boundaries are illegal, and the verification process is terminated; two is to judge whether the Merkle hash tree id of the left and right boundaries is one of equal or adjacent, if the Merkle hash tree id of the left and right boundaries is not one of equal or adjacent, the verification process is terminated; otherwise, reconstruct the Merkle hash tree root hash according to the range query proof, since there are two Merkle hash trees with left and right boundaries, return the list of root hashes here; finally, verify whether the root hash is correct through the vector commitment; Step three, persistently store the verifiable structure.

2. The blockchain-oriented method of supporting efficient verifiable queries of claim 1, wherein, In the construction process, multi-threading is used to accelerate the calculation of m sub-Merkle hash trees and the pre-computation of the vector commitment proof of all Merkle hash trees; at the same time, the pre-computation of the vector commitment proof is updated using an asynchronous or delayed manner. 3.The blockchain-oriented method of supporting efficient verifiable query according to claim 1, wherein, The security parameters of the vector commitment public parameter are set and saved by the trusted node controlling the access, and the corresponding public parameters are generated according to the security parameters and broadcast to all nodes.

4. The blockchain-oriented method of supporting efficient verifiable queries of claim 1, wherein, The proof of the verifiable structure comprises the following steps: Step 1, find the account address to be proved on the state list, if the address is found, process the existence proof, and directly prove that the account address exists; otherwise, process the non-existence proof, and provide the range proof of the left and right boundaries; Step 2, for both existence proof and non-existence proof, the location information of the account to be proved needs to be generated, including the sequence number of the Merkle hash tree where it is located and the location in the Merkle hash tree, wherein the state list is sorted by account address; for existence proof, the account to be proved is the account address to be proved; for non-existence proof, the range proof of the left and right boundaries of the account to be proved is returned; Step 3, give existence proof to all accounts in the list to be proved, each proof includes two parts of Merkle hash tree proof and vector commitment proof, wherein the Merkle hash tree proof needs to reconstruct the part of the Merkle hash tree required for proof according to the state list and the part of the Merkle hash tree cache generated when constructing; for the vector commitment proof, directly read the precomputed vector commitment proof representing all Merkle hash trees when constructing.

5. The blockchain-oriented method of supporting efficient verifiable queries of claim 4, wherein, The strategy of time for space is adopted, that is, the whole Merkle hash tree is not stored, but the Merkle hash tree is reconstructed according to the state list during the proof process; In the Merkle hash tree, the storage size required to store any node is consistent.

6. The blockchain-oriented method of supporting efficient verifiable queries of claim 1, wherein, The persistent storage of the verifiable structure considers the storage overhead and the number of I / O during proof, and tries to store the verification data and the state data in the same disk block, so that only one disk I / O is consumed during proof; The storage structure adopted includes the following two parts: The first part, for each sub-Merkle hash tree, a disk block is used to save the related data, including the cached Merkle hash tree node to speed up the proof, the Merkle hash tree node includes the corresponding state hash and the vector commitment proof and some related metadata; and different sub-Merkle hash trees are written to the disk in sequence; The second part, for the whole verifiable structure, a disk block is also used to save the related index data, Bloom filter, and vector commitment related data; wherein the index includes the maximum and minimum values of the account address in each subsequent Merkle hash tree and the corresponding address on the state log, the Bloom filter saves all account addresses in the whole verifiable structure, and the vector commitment related data saves the root hash of each hash tree and the corresponding commitment. 7.A blockchain-oriented system supporting efficient verifiable queries, characterized in that, Comprise: a memory and a processor; The memory has stored thereon a computer program which, when executed by the processor, implements the method of any one of claims 1-6.

Images