Multi-version verifiable query method oriented to block chain data

Abstract

The invention discloses a multi-version verifiable query method oriented to block chain data. The method comprises an index structure supporting efficient multi-version query and a verifiable structure supporting quick generation and verification of multi-version proof. Firstly, an index structure supporting efficient multi-version query can efficiently locate all historical versions in multi-version traceability query crossing multiple blocks at the same time. The index structure supporting efficient multi-version query comprises an index structure based on the LSM-Tree and a new filter based on Huffman coding, the ordered character string table where different versions of data are located is queried at a time through the filter, and the efficiency of multi-version query is remarkably improved. Secondly, a verifiable structure supporting rapid generation and verification of multi-version proof can generate proof of multi-version data in parallel and efficiently aggregate the proof, and the size of the proof is reduced so as to facilitate batch verification; by means of the scheme, the query and verification process of the multi-version data can be effectively optimized, and rapid positioning of the data and efficient generation and verification of the certification are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of blockchain, and in particular relates to a multi-version verifiable query method for blockchain data. Background Art

[0002] As a distributed ledger technology, blockchain has the important characteristics of data immutability and traceability. With the widespread application of blockchain technology in various fields, the demand for data integrity, traceability and security is growing. However, with the continuous expansion of the scale of blockchain systems and the increase in application scenarios, traditional data storage, query and verification methods may face challenges. Especially when it comes to multi-version data, how to efficiently store, query and verify has become an urgent problem to be solved. Existing blockchain data storage is mainly divided into two types: one is to separate the underlying persistent storage from the verifiable index structure, and the other is to directly build a verifiable structure on the underlying persistent storage.

[0003] The way to separate the underlying persistent storage from the verifiable index structure is that the key-value database (level db) is responsible for the persistent storage of the underlying data, while the verifiable index structure is used for the upper-level index and provides verifiable queries. This separated architecture causes the underlying database to store a large amount of data to provide indexes for the verifiable structure, and the actual part used for data storage is relatively small. The redundant storage of the intermediate index nodes results in a small part of the LSM-Tree actually used to store account data. The index structure built in blocks needs to be traversed to process multi-version data queries, which reduces the efficiency of multi-version queries. Because it is impossible to determine the starting and final positions of the state change, when processing multi-version data queries, traversal is often used. Traverse from the latest block to the oldest block to determine the block where the state change occurs. The verifiable structure built in blocks often provides multi-version proofs by merging multiple single-version proofs, resulting in the size of the multi-version proof increasing linearly with the number of versions.

[0004] Although building a verifiable structure directly on the underlying persistent storage reduces the redundant storage of index nodes, the performance of multi-version queries has not been improved. Existing work directly combines the verifiable structure with LSM-Tree to reduce the redundant storage of index nodes by removing the upper index. However, since the update of the verification structure often incurs a large overhead, this approach will result in excessive overhead in rebuilding the verification structure during compression. Multi-version queries still need to be traversed, which reduces the efficiency of multi-version queries. With the introduction of some more efficient consensus algorithms and some transaction concurrency control algorithms, the above-mentioned problem system in blockchain storage has become the main bottleneck restricting the performance of blockchain systems. Summary of the invention

[0005] The purpose of the present invention is to propose a multi-version verifiable query method for blockchain data, aiming to solve the problems of low efficiency of multi-version data query, large size of generated proofs and high verification cost in existing blockchain systems. In order to quickly locate multi-version data, the present invention designs an efficient structure that supports multi-version indexing. This structure can realize rapid retrieval of all versions of data by effectively organizing and managing different versions of data in the blockchain. At the same time, in order to improve the efficiency of multi-version queries, the present invention designs a new filter based on Huffman coding to accelerate the data location process. The present invention also designs a new verifiable structure specifically for the generation and verification of multi-version proofs. By introducing vector commitments, the proof size is reduced, and parallel generation of proofs and aggregated proofs are supported. At the same time, a batch verification mechanism is introduced so that multiple versions of proofs can be efficiently verified in a single verification process, greatly reducing the verification overhead.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A multi-version efficient and verifiable query method for blockchain data, comprising: an index structure supporting efficient multi-version query and a verifiable structure supporting fast application and verification of multi-version proofs; wherein:

[0008] The index structure supporting efficient multi-version query is implemented based on LSM-Tree. The multi-version index relies on various LSM-Tree components, avoiding additional maintenance of the index structure and reducing duplicate hash storage overhead by merging. By designing new filters, the access overhead of the original multi-layer filters is reduced, accelerating multi-version query;

[0009] The verifiable structure that supports the rapid generation and verification of multiple versions of proofs adopts a scheme that combines vector commitments with Merkle hash trees, which not only retains the advantage of vector commitments in terms of constant proof size, but also minimizes the use of vector commitments. Proofs are generated for multiple versions of data in parallel, and are efficiently aggregated to reduce the size of proofs and facilitate batch verification.

[0010] The index structure supporting efficient multi-version query specifically includes:

[0011] A1: In the write phase, the index data is written to the components of the LSM-Tree, including the in-memory MemTable, the immutable ImmTable, and the SSTable on the disk.

[0012] A2: In the merging phase, different versions of the same key are merged to reduce storage overhead. In the compaction process of LSM-Tree, this method can increase the size of key-value pairs while reducing the storage of duplicate keys.

[0013] A3: In the query phase, all versions of a given key are queried at once. The in-memory table MemTable and the immutable memory table ImmTable are searched, and the SSTable on the disk is retrieved to ensure that all historical versions of the key are obtained.

[0014] The verifiable structure that supports rapid generation and verification of multi-version proofs specifically includes:

[0015] B1: Construct Merkle hash tree and vector commitment VC;

[0016] B2: Proof and verification of Merkle hash tree and vector commitment VC;

[0017] B3: Only the upper vector commitment VC is persisted.

[0018] Furthermore, the A1 specifically includes:

[0019] A11: Write to memory, write the index data to the MemTable in memory;

[0020] A12: When the size of the memory table MemTable reaches a predetermined threshold, it is converted to an immutable memory table ImmTable, which is then persisted as an SSTable and written to disk.

[0021] A13: When the data in the memory table is refreshed to the disk, the fingerprint of the corresponding key and the SSTable number need to be inserted into the cuckoo filter. It should be noted that the number is not inserted directly, but the number after Huffman encoding is inserted into the cuckoo filter to reduce the storage overhead of the SSTable number.

[0022] Furthermore, the A3 specifically includes:

[0023] A31: Memory query; first, use the skip table to query MemTable and return whether there is any relevant key change information in MemTable; then query ImmTable to check whether there is any corresponding key change information;

[0024] A32: Based on the designed filter, determine all SSTables where the key is located; in the SSTables that meet the conditions, use binary search to obtain the corresponding value and return it.

[0025] Furthermore, the B1 specifically includes:

[0026] B11: First, sort the input status list by account address in blocks, build a corresponding Merkle hash tree for each block, and calculate its corresponding Merkle hash tree root;

[0027] B12: Use the Merkle hash tree root of each block as the data of the vector commitment, and save some Merkle hash tree nodes to the cache during the calculation process to facilitate the subsequent storage of the verification structure on disk;

[0028] B13: Construct the vector commitment VC based on the root hash of the Merkle hash tree of all blocks according to the input public parameter pp, and calculate the corresponding commitment and the corresponding proof for each position.

[0029] Furthermore, the B2 specifically includes:

[0030] B21: Search the account address to be proved in the status list, and generate the location information of the account to be proved, including its Merkle hash tree number m_id and its position pos in the Merkle hash tree. Since the status list is sorted by account address, it can be queried quickly.

[0031] B22: Proof of existence is given to all accounts in the proof list. Each proof includes a Merkle hash tree proof and a vector commitment proof. The Merkle hash tree proof is to reconstruct the part of the Merkle hash tree required for the proof based on the state list and the Merkle hash tree generated during construction. For the vector commitment proof, the proofs pre-calculated during construction are directly read.

[0032] The beneficial effects of the present invention include:

[0033] The present invention aims to solve the problems of low efficiency of multi-version data query, large size of generated proofs and high verification cost in the existing blockchain system, and focuses on developing an efficient and verifiable multi-version query method for blockchain data. Its purpose is to quickly locate multi-version data, generate multi-version proofs and effectively verify these proofs. Specifically, the present invention designs an efficient indexing mechanism that can efficiently locate all historical versions simultaneously in multi-version traceability queries across multiple blocks. The storage overhead of duplicate hash values ​​is reduced by merging based on LSM multi-version indexes, and all sstables where key-value pairs are located are queried at one time by designing new filters to accelerate multi-version queries. In addition, an efficient and verifiable structure is designed that can generate proofs for multiple versions of data in parallel at the same time, and perform efficient aggregation to reduce the size of proofs and facilitate batch verification. Combined with Merkle trees and vector commitments, the size of multi-version proofs can be reduced while improving verification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of the present invention;

[0035] Figure 2 is an example diagram of the encoding of the filter in the multi-version index proposed by the present invention;

[0036] Figure 3 It is a schematic diagram of the Level ID encoding algorithm;

[0037] Figure 4 is an example diagram of the multi-version query proposed by the present invention;

[0038] Figure 5 It is a schematic diagram of the multi-version verifiable structure of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0040] The present invention discloses an efficient and verifiable multi-version query method for blockchain data, which supports rapid positioning of multi-version data, generates multi-version proofs, and effectively verifies these proofs. The method includes two core contents: first, an index structure that supports efficient multi-version queries, which can efficiently locate all historical versions simultaneously in multi-version traceability queries across multiple blocks; second, a verifiable structure that supports rapid generation and verification of multi-version proofs, generates proofs for multiple versions of data in parallel, and performs efficient aggregation to reduce the size of proofs, facilitating batch verification. Through a series of optimization schemes, the present invention can efficiently optimize the verifiable query process of multi-version data, quickly locate multi-version data, generate multi-version proofs, and verify proofs.

[0041] In the efficient and verifiable multi-version query method for blockchain data, the transaction life cycle will go through the following three stages: (1) execution stage; (2) writing, merging and querying stage; (3) building a verifiable structure.

[0042] Execution phase: In the transaction execution phase of the blockchain, first, the verified transactions are packaged into a new block. This block contains multiple transaction records, the hash value of the previous block, a timestamp, and other relevant metadata. Nodes verify the new block through a consensus mechanism (such as proof of work or proof of stake) to ensure the validity of the block. Each transaction triggers a status update for the corresponding account, and these status changes are reflected in the blockchain's state database. When all transactions are executed, the node updates its local state database to reflect the latest account balances and contract status.

[0043] Write, merge and query phase: Once the transaction execution is completed, it enters the write subphase. In this phase, an index structure that supports efficient multi-version queries is required. First, the index data is written to the memory table (MemTable) in the component related to LSM-Tree. When the size of the memory table reaches a predetermined threshold, it will be converted to an immutable memory table (ImmTable) and persisted as an SSTable stored on disk. Next, the system enters the merge subphase, where the merge is performed on different versions of the same key to ensure the integrity of the blockchain data. This process increases the size of the key-value pair while reducing storage redundancy. While the data is refreshed to disk, the filter must be updated to include the latest information of the key-value pair in the SSTable. Finally, the system enters the query subphase, which aims to retrieve all version information of a given key at one time, and collect and output all relevant version data to the log, in preparation for the next step of building a verifiable structure.

[0044] Phase of building a verifiable structure: A scheme combining vector commitment and Merkle hash tree was adopted, which not only retained the advantage of constant-level proof size of vector commitment, but also reduced the use of vector commitment as much as possible, thereby alleviating performance problems caused by excessive computational overhead of vector commitment. According to the log information obtained in the previous phase, Merkle tree is built in blocks, and blocks are connected by vector commitment, which reduces the query overhead of single-version proof by reducing the tree height. Through maintainable and aggregatable vector commitment scheme, proof generation and verification are aggregated across blocks to reduce proof size and speed up verification.

[0045] Figure 1The overall process of the multi-version verifiable query method proposed in the present invention is shown. The block contains account transaction data, which can trigger the modification of account status. Account information is stored in LSM-Tree for indexing. LSM-Tree manages write and query operations in an efficient way, supports fast indexing and retrieval, and the corresponding status data is attached to the status log file, recording the status and change history of the account after each transaction for subsequent auditing and querying. In order to cope with the problem of redundant data storage, the storage occupancy is reduced and the access speed of the index is improved by deleting the intermediate index nodes and merging the indexes in LSM-Tree. A new filter is designed to support one-time query of all storage tables (SSTables), which can effectively locate relevant data and solve the problem of inefficient multi-version query. The verifiable structure is built based on the state log, combining Merkle tree and vector commitment to speed up the proof calculation as much as possible while reducing the size of the proof.

[0046] Figure 2 The encoding process of filters in multi-version indexes is illustrated. LSM-Tree has a three-layer structure. The first and second layers contain 4 SSTables, and the last layer contains 1 SSTable. The proportion of each SSTable in each layer in the entire LSM-Tree structure can be calculated. For example, SSTable 1 The ratio is 1 / 125, SSTable 5 The ratio is 5 / 125, SSTable 9 The ratio is 100 / 125. Use Huffman coding to encode the Level ID of all SSTables, and get the following Figure 2 The Huffman tree shown in the upper right corner. Among them, SSTable 1 Contains k 1 , SSTable 5 Contains k 2 and k 3 , SSTable 9 Contains k 4 , k 5 , k 6 and k 7 Assuming that the original filter does not use encoding and directly uses int32 to store the corresponding Level ID, then 3 int32 are needed to store k 1 , k 2 and k 4 The Level ID requires a total of 12 bytes. When using Huffman coding, k 1 Requires 6 bits, k 2 Requires 3 bits, k4 Only 1 bit is needed. A total of 1.25 bytes are needed, which greatly reduces the storage overhead of the Level ID.

[0047] like Figure 3 As shown, Algorithm 1 shows the encoding process of Level ID. Encode the Level ID based on L (the number of layers of the LSM-Tree), K (the number of SSTables in each layer except the last layer), Z (the number of SSTables in the last layer), and T (the size ratio of the next layer to the previous layer). Calculate the proportion of each SSTable in each layer in the entire LSM-Tree in turn (CalculateSSTableRatio). According to the proportion of each SSTable in the LSM-Tree, perform Huffman encoding on the SSTable (EncodeLevelID). The table with more key values ​​at the bottom corresponds to a shorter Level ID encoding length, thereby reducing the storage overhead of the Level ID, and then reducing the storage overhead of the cuckoo filter.

[0048] like Figure 4 As shown in the figure, for index query, when the user gives an account Acc, the system needs to return all version information of the given account. This process is different from the LSM-Tree query process. The LSM-Tree query process only needs to find the latest data that meets the conditions, while the multi-version query needs to query all version information of the given account at one time. Figure 3 The actual storage structure is shown in the article, including MemTable, ImmTable, and the SSTables in the first two layers, Level 0 and Level 1. The specific query process will be introduced. For example, querying all state values ​​of Acc6 is divided into two stages. The first stage queries the data in the memory, and the second stage queries the data on the disk. When querying the Memtable using the skip table, it will return that there is no change information related to Acc6 in the MemTable. When querying the ImmTable, there is no change information corresponding to Acc6. First, based on the FHE filter, query the SSTables corresponding to Acc6. The corresponding SSTables are {1, 3}. Perform a binary search in the SSTables to obtain the corresponding value and return it.

[0049] Figure 5In this paper, a new verifiable structure BVM (Block Vector Merkle) is designed, which combines vector commitments and Merkle trees to reduce the size of multi-version proofs while maximizing the generation and verification performance of multi-version proofs. Verifiable structures are crucial in tamper-proof storage because they support verifiable queries, ensuring the correctness of query results based on the provided proofs. Existing blockchain storage usually designs verifiable structures based on Merkle hash trees, verifying the integrity and immutability of data through the difficulty of hash inversion operations. Another scheme is based on vector commitments, which improves verification efficiency by generating small and fixed-size proofs. However, there are limitations to using only one of the schemes. Merkle trees have large proof sizes, while vector commitments have high initialization and verification overheads. Therefore, BVM combines the advantages of these two schemes to provide a more efficient method for generating and verifying multi-version data proofs.

[0050] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A multi-version verifiable query method for blockchain data, characterized in that: include: An index structure that supports efficient multi-version queries and a verifiable structure that supports fast generation and verification of multi-version proofs; The index structure supporting efficient multi-version query specifically includes: A1: In the write phase, the index data is written to the components of the LSM-Tree, including the in-memory MemTable, the immutable ImmTable, and the SSTable on the disk. A2: In the merging phase, different versions of the same key are merged to reduce storage overhead. In the compaction process of LSM-Tree, this method can increase the size of key-value pairs while reducing the storage of duplicate keys. A3: In the query phase, all versions of a given key are queried at once. The in-memory table MemTable and the immutable memory table ImmTable are searched, and the SSTable on the disk is retrieved to ensure that all historical versions of the key are obtained. The verifiable structure that supports rapid generation and verification of multi-version proofs specifically includes: B1: Construct Merkle hash tree and vector commitment VC; B2: Proof and verification of Merkle hash tree and vector commitment VC; B3: Only the upper vector commitment VC is persisted.

2. The multi-version verifiable query method according to claim 1, characterized in that: The A1 specifically includes: A11: Write to memory, write the index data to the MemTable in memory; A12: When the size of the memory table MemTable reaches a predetermined threshold, it is converted to an immutable memory table ImmTable, which is then persisted as an SSTable and written to disk. A13: When the data in the memory table is refreshed to the disk, the fingerprint of the corresponding key and the SSTable number need to be inserted into the cuckoo filter. Instead of inserting the number directly, the Huffman-encoded number is inserted into the cuckoo filter to reduce the storage overhead of the SSTable number.

3. The multi-version verifiable query method according to claim 1, characterized in that: The A3 specifically includes: A31: Memory query; first, use the skip table to query MemTable and return whether there is any relevant key change information in MemTable; then query ImmTable to check whether there is any corresponding key change information; A32: Based on the designed filter, determine all SSTables where the key is located; in the SSTables that meet the conditions, use binary search to obtain the corresponding value and return it.

4. The multi-version verifiable query method according to claim 1, characterized in that: The B1 specifically includes: B11: First, sort the input status list by account address in blocks, build a corresponding Merkle hash tree for each block, and calculate its corresponding Merkle hash tree root; B12: Use the Merkle hash tree root of each block as the data of the vector commitment, and save some Merkle hash tree nodes to the cache during the calculation process to facilitate the subsequent storage of the verification structure on disk; B13: Construct the vector commitment VC based on the root hash of the Merkle hash tree of all blocks according to the input public parameter pp, and calculate the corresponding commitment and the corresponding proof for each position.

5. The multi-version verifiable query method according to claim 1, characterized in that: The B2 specifically includes: B21: Search the account address to be proved in the status list, and generate the location information of the account to be proved, including its Merkle hash tree number m_id and its position pos in the Merkle hash tree; B22: Provide proof of existence for all accounts in the proof list. Each proof includes a Merkle hash tree proof and a vector commitment proof. The Merkle hash tree proof needs to reconstruct the part of the Merkle hash tree required for the proof based on the state list and the Merkle hash tree generated during construction. For the vector commitment proof, directly read the proofs pre-calculated during construction.

Images