Tamper-proof index structure and its construction, storage and query methods

Abstract

The present invention proposes a tamper-proof index structure and its construction, storage and query methods, which can support efficient storage and efficient query, and meet the actual industrial usage scenarios. Compared with the traditional B+ tree, the index structure of the present invention adds an associated hash field for each keyword to provide tamper-proof function; adds left and right sibling pointers between each layer of nodes to support bidirectional query, and adds a High Key field to each layer of non-rightmost nodes to identify the keyword range to accelerate the query; in addition, abandons the B+ tree's assumption of fixed keyword length, and instead uses the remaining available storage space of the node as the judgment basis, constructing an efficient tamper-proof index structure that supports variable-length key storage and bidirectional query. The present invention describes the index structure and its construction method in detail, providing a new solution for the safe use and efficient query of data.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a tamper-proof index structure and a construction, storage and query method thereof. Background Art

[0002] Data management technology, which involves classifying, encoding, storing, retrieving, and maintaining data, is a central aspect of data processing. With the continuous development of computer technology and driven by application needs, data management technology has evolved through three stages: manual management, file systems, and database systems, building on advances in computer hardware and software.

[0003] A database is a warehouse that organizes, stores, and manages data according to its structure. It can provide efficient data storage and management services for massive amounts of data, primarily utilizing B-tree or B+ tree indexing structures. It is currently the most widely used data management technology and plays a significant role in industry. However, while users enjoy the convenience provided by databases, they also face a series of security risks, such as data leakage and tampering, that may arise from the transfer of direct control over data. Although some current database products have implemented computer security technologies such as public and private keys and password authentication to some extent to protect against unauthorized users, this security mechanism, which relies on role-based permission management, still cannot detect malicious behavior by system administrators with special permissions.

[0004] Blockchain, through its chain-like data structure and distributed consensus mechanism, strictly ensures that data, once written, cannot be tampered with in a decentralized, trustless environment. It is considered a key approach to addressing data security. However, the index structure designed for file systems and KV key-value storage systems only supports basic query operations and results in relatively low query efficiency.

[0005] An Authenticated Data Structure (ADS) is a data structure that not only provides data query functionality but also provides verification objects for a given query, thereby verifying the correctness and integrity of the query results. It is an organic combination of an Authenticated Data Structure and an Index Structure. Merkle trees and B-trees are currently the most widely used Authenticated Data Structures and Index Structures. However, some of their designs are overly idealistic and not suitable for certain real-world scenarios. Summary of the Invention

[0006] In view of this, the present invention proposes a tamper-proof index structure and its construction, storage and query methods, which can support efficient storage and efficient query and meet the actual industrial usage scenarios.

[0007] To achieve the above object, the technical solution of the present invention is:

[0008] The present invention provides a tamper-proof index structure, the basic structure of which is a B+ index tree, with the bottom layer being the leaf node layer and the remaining layers being non-leaf node layers; each node can store multiple index tuples, and does not strictly stipulate a unified node fan-out number. The remaining available storage space of the node is used as the basis for judgment, and the case of variable key length is supported; each index tuple includes a keyword key, a pointer, and a hash field;

[0009] Among them, the pointer of the index tuple of the non-leaf node layer points to the index node of the next layer; the pointer of the index tuple of the leaf node layer points to the data block; the hash field of the index tuple of the leaf node layer is obtained by the hash algorithm for the data block it points to; the calculation method of the hash field of the index tuple of the non-leaf node layer is: cascade the hash fields of all index tuples of the lower-level index node it points to and perform the hash operation again.

[0010] Among them, the High Key field is set for each non-rightmost node in each layer to identify the maximum key value of the node.

[0011] Among them, bidirectional left and right sibling pointers are set between nodes in each layer.

[0012] The present invention also provides a method for constructing a tamper-proof index structure, comprising the following steps:

[0013] Construct the first node and allocate the set storage space for the node;

[0014] Insert the index tuples corresponding to the data blocks into the first node one by one: the search key, the pointer to the data block, and the hash value of the data block. Before inserting each index tuple, determine whether the remaining storage space of the current node is sufficient to store the index tuple. If sufficient, continue inserting; otherwise, proceed to the next step.

[0015] Construct a new node as the right sibling node of the first node, create a parent node for the first node and write the HK of the first node into the parent node, let the parent node point to the first node, the right sibling pointer of the first node point to the second node, and the left sibling pointer of the second node point to the first node;

[0016] When the second node is full, apply for a third new node and create a pointer from the first parent node of the first layer to the second node; create another parent node for the parent node of the first layer, and set the right sibling pointer of the second node and the left sibling pointer of the third node;

[0017] When the third node is also filled, create a parent node pointing to this node as in the beginning, set the left and right sibling pointers between the two first-level parent nodes, and apply for a right sibling node again, and set the corresponding left and right sibling pointers;

[0018] This recursive process is repeated until a complete EMBT index tree is built.

[0019] Among them, each node is pre-assigned with a field identifying HK. After each layer of the tree is filled, if it is found to be the rightmost node, its HK field will be deleted.

[0020] The present invention also provides a method for storing a tamper-proof index structure, which is characterized in that the method uses the tamper-proof index structure of the present invention to store data records, including the following steps:

[0021] Step 21: Encapsulate the data records to be inserted into index tuples;

[0022] Step 22: Determine the inserted leaf node through a single value query based on the key value of the index tuple;

[0023] Step 23: Further determine the insertion position in the leaf node; if the current leaf node has sufficient space, execute step 24; if the current leaf node has insufficient space, execute step 25;

[0024] Step 24: Insert the index tuple and update the value of the affected hash field in each node on the path to the root node;

[0025] Step 25: Perform split operation on the current leaf node;

[0026] The split operation first creates a new right sibling node of the current node, then selects the specific split location according to the set copy algorithm, and copies part of the index tuple of the original child node to the newly created node; then updates the parent node related information and the left and right sibling pointers of the leaf node and the values ​​of all affected hash fields on the path from the leaf node to the root node. If the parent node has enough space, the insertion is completed; otherwise, this step is repeated in the parent node until the insertion is completed.

[0027] The present invention also provides a query method of the tamper-proof index structure, which uses the tamper-proof index structure of the present invention to query data records, including single-value query and range query;

[0028] The single value query includes the following steps:

[0029] Step 31: Start searching from the root node, compare the size of each keyword of the root node with the keyword searchkey to be queried, and determine the descending position;

[0030] Step 32: Compare the size of the child node HK with the search key. If the search key is smaller than HK, continue to compare the size of each keyword of the child node with the search key to determine the position to descend from the current layer. Otherwise, find the right sibling node through the right sibling pointer right link. Repeat this step until you descend to the leaf node layer.

[0031] Step 33: Find the index tuple corresponding to the search key in the leaf node, and query the data that meets the conditions; if there is no index tuple corresponding to the search key at the leaf node level, the query ends;

[0032] Range queries include determining the query range through one keyword and determining the search range through two keywords.

[0033] Beneficial effects:

[0034] 1. The present invention's index structure, EMBT (Efficient Merkle Blink Tree), supports efficient storage and querying. EMBT is a variant of Merkle trees and B+ trees, with a basic structure similar to a regular B+ tree. The linked list structure formed by the ordered arrangement of keywords in leaf nodes ensures support for range queries. EMBT uses the available storage space of a node as its basis for decision making, without strict restrictions on node fan-out. To support data tamper-proofing, EMBT adds a hash field to each keyword, ensuring that any tampering with a record will result in a mismatch in the hash field corresponding to the keyword pointing to the node. The hash field value corresponding to a keyword in a leaf node is calculated using a specific hash algorithm for the data record it points to. The hash field value corresponding to a keyword in a non-leaf node is the concatenation of all hash fields in the index node it points to, followed by a hash calculation. Because each hash field occupies only a small amount of storage space, EMBT can still store massive amounts of data while maintaining a low tree height, thereby ensuring low I / O disk consumption during queries.

[0035] 2. In the present invention, each non-rightmost node in each layer is added with a HK (High Key) field to identify the maximum key value of the current node's child nodes to accelerate query positioning and thereby determine whether the child node is split. HK is not a tuple in the index structure, but only marks a maximum range.

[0036] 3. In the present invention, in order to further speed up the query speed and support bidirectional query, EMBT adds bidirectional left and right sibling pointers to each layer of nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a non-leaf node structure diagram of the EMBT of the present invention. If the node shown in the diagram is not the rightmost node, the first key k0 is the HK of the node.

[0038] Figure 2 This is the leaf node structure diagram of the EMBT of the present invention. The only difference from the non-leaf node is that the index tuple of the leaf node points to the corresponding data record, while the non-leaf node points to the index node of the lower layer.

[0039] Figure 3 A schematic diagram of the tree structure of the EMBT filling the first leaf node in the EMBT construction process of the present invention.

[0040] Figure 4 A schematic diagram of the tree structure of the EMBT filling the second leaf node in the EMBT construction process of the present invention.

[0041] Figure 5 A schematic diagram of the tree structure of the EMBT filling the third leaf node in the EMBT construction process of the present invention.

[0042] Figure 6 A schematic diagram of the tree structure of the EMBT with the fourth leaf node filled in during the EMBT construction process of the present invention.

[0043] Figure 7 1 is a schematic diagram of an EMBT tree structure with 7 leaf nodes, showing a query path for searching for a customer with ID number 16 in an embodiment of the present invention.

[0044] Figure 8 FIG. 1 is a schematic diagram of an EMBT tree structure with 7 leaf nodes, showing a query path for searching for customers with ID numbers between 49 and 81 in an embodiment of the present invention.

[0045] Figure 9 This is a schematic diagram of the EMBT tree structure of the present invention. DETAILED DESCRIPTION

[0046] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0047] The present invention proposes a tamper-proof index structure and its construction, storage and query methods, which can support efficient storage and efficient query, and meet the actual industrial usage scenarios. Compared with the traditional B+ tree, the index structure of the present invention adds an associated hash field for each keyword to provide tamper-proof function; adds left and right sibling pointers between each layer of nodes to support bidirectional query, and adds a High Key field to each layer of non-rightmost nodes to identify the keyword range to accelerate the query; in addition, abandons the B+ tree's assumption of fixed keyword length, and instead uses the remaining available storage space of the node as the judgment basis, constructing an efficient tamper-proof index structure that supports variable-length key storage and bidirectional query. The present invention describes the index structure and its construction method in detail, providing a new solution for the safe use and efficient query of data.

[0048] The present invention provides a tamper-proof index structure that is efficient in storage and query, such as Figure 9 As shown in the figure, the basic structure is a B+ index tree, with the bottom layer being the leaf node layer and the remaining layers being the non-leaf node layers. Each node can store multiple index tuples, and there is no strict requirement for a uniform fan-out number of nodes. The remaining available storage space of the node is used as the basis for judgment, and the case of variable key length is supported. Each index tuple includes the keyword key, pointer, and hash field.

[0049] Among them, the pointer of the index tuple of the non-leaf node layer points to the index node of the next layer; the pointer of the index tuple of the leaf node layer points to the data block; the hash field of the index tuple of the leaf node layer is obtained by the hash algorithm for the data block it points to; the calculation method of the hash field of the index tuple of the non-leaf node layer is: cascade the hash fields of all index tuples of the lower-level index node it points to and perform the hash operation again.

[0050] The tamper-proof index structure of the present invention is a variant of the Merkle tree and the B+ tree, specifically an EMBT (Efficient Merkle Blink Tree) index structure that supports efficient storage and querying. The basic structure of an EMBT is similar to that of a regular B+ tree. The linked list structure formed by the ordered arrangement of keywords in leaf nodes ensures support for range queries. The B+ tree assumes that each keyword is of fixed length and therefore sets a strict fan-out number for each node. However, in real-world scenarios, key values ​​are often of variable length. Therefore, EMBT uses the available storage space of the node as the basis for judgment, thus eliminating strict node fan-out limits. To support data tamper-proofing, EMBT adds a hash field to each keyword. This ensures that no matter which record is tampered with, the hash field corresponding to the keyword pointing to the node will not match. The value of the hash field corresponding to the keyword in a leaf node is the value calculated by a specific hash algorithm for the data record it points to. The value of the hash field corresponding to the keyword in a non-leaf node is the value calculated by hashing the concatenated hash fields of all the hash fields in the index node it points to. Since a hash field only takes up a small amount of storage space, EMBT can still meet the storage needs of massive data while maintaining a low tree height, thereby ensuring low I / O disk consumption during queries.

[0051] In addition, in the present invention, each non-rightmost node adds a new HK (High Key) field to identify the maximum key value of the current node's child nodes to accelerate query positioning and determine whether the child node is split. HK is not a tuple in the index structure, but only marks a maximum range. The non-leaf node structure diagram of the EMBT of the present invention is shown in the figure. Figure 1 As shown, if Figure 1 If the node shown in is not the rightmost node, the first key k0 is the HK of the node.

[0052] To further speed up the query and support bidirectional queries, EMBT adds bidirectional left and right sibling pointers to each layer of nodes to support bidirectional queries and range queries.

[0053] The leaf node structure diagram of EMBT of the present invention is as follows Figure 2 As shown, the only difference from non-leaf nodes is that the index tuple of a leaf node points to the corresponding data record, while the non-leaf node points to the index node of the lower layer.

[0054] The present invention also provides a method for constructing a tamper-proof index structure for storage and querying. Constructing the EMBT of the present invention comprises the following steps:

[0055] Step 1: Construct the first node and allocate the set storage space for the node;

[0056] Step 2: Insert the index tuples corresponding to the data blocks into the first node one by one: the search key, the pointer to the data block, and the hash value of the data block. Before inserting each index tuple, determine whether the remaining storage space of the current node is sufficient to store the index tuple. If sufficient, continue inserting. If not, proceed to step 3.

[0057] Step 3: Construct a new node as the right sibling of the first node, create a parent node for the first node and write the HK of the first node to the parent node. Let the parent node point to the first node, the right sibling pointer of the first node point to the second node, and the left sibling pointer of the second node point to the first node;

[0058] Step 4: When the second node is full, apply for a third new node and create a pointer from the first parent node of the first layer to the second node; create another parent node for the parent node of the first layer, and set the right sibling pointer of the second node and the left sibling pointer of the third node;

[0059] Step 5: When the third node is also filled, create a parent node pointing to this node as in the beginning, set the left and right sibling pointers between the two first-level parent nodes, and apply for a right sibling node again, and set the corresponding left and right sibling pointers;

[0060] Step 6: Repeat this recursion repeatedly until a complete EMBT index tree is established.

[0061] It is worth noting that each node is pre-assigned a field identifying HK. After each layer of the tree is filled, if it is found to be the rightmost node, the node no longer needs to maintain HK, and its HK field needs to be deleted.

[0062] The tree structure diagram of the EMBT construction process of the present invention from filling the first leaf node to the fourth leaf node is as follows Figure 3-Figure 6 shown.

[0063] This embodiment establishes an EMBT with 7 leaf nodes based on the customer table, using the ID attribute in the customer table as the key. The query path for finding the customer with ID number 16 is shown in the following diagram: Figure 7As shown in the figure, to find the customer with ID number 16, we will start from the root node and go down layer by layer until we find a leaf node. We will continue to locate in the leaf node until we find the index tuple with key = 16, and then find the customer information it points to. The query path for finding customers with ID numbers between 49 and 81 is shown in the figure below. The EMBT tree structure with 7 leaf nodes is shown in the figure below. Figure 8 As shown, in order to find customers with ID numbers between 49 and 81, we will start searching from the root node and go down layer by layer until we find the index tuple with ID = 49 at the leaf node layer. Then, we will traverse to the right through the right sibling pointer to obtain all index tuples that satisfy ID <= 81, thereby querying all customers who meet the conditions. Figure 7 The hash value of each node index tuple is not identified in the tree. In order to support variable key value storage, the tree structure does not strictly require the fan-out number of each node to be consistent. The tree is built on the customer table. Figure 7 The query path for finding the customer with ID number 16 is shown in FIG. Figure 8 The hash value of each node index tuple is not identified in the tree. To support mutable key-value storage, the tree structure does not strictly require the fanout number of each node to be consistent. The tree is built on the customer table and finds the query path for customers with ID numbers between 49 and 81.

[0064] The present invention also provides a method for storing a tamper-proof index structure, which uses the tamper-proof index structure of the present invention to store data records, including the following steps:

[0065] Step 21: Encapsulate the data records to be inserted into index tuples;

[0066] Step 22: Determine the inserted leaf node through a single value query based on the key value of the index tuple;

[0067] Step 23: Further determine the insertion position in the leaf node; if the current leaf node has sufficient space, execute step 24; if the current leaf node has insufficient space, execute step 25;

[0068] Step 24: Insert the index tuple and update the value of the affected hash field in each node on the path to the root node;

[0069] Step 25: Perform a split operation on the current leaf node. The split operation first creates a new right sibling node for the current node. Then, based on the configured replication algorithm, the split location is selected and the partial index tuple of the original child node is copied to the newly created node. Next, the parent node information, the left and right sibling pointers of the leaf node, and the values ​​of all affected hash fields on the path from the leaf node to the root node are updated. If the parent node has sufficient space, the insertion is complete; otherwise, this step is repeated in the parent node until the insertion is complete.

[0070] The present invention also provides a query method for a tamper-proof index structure, which uses the tamper-proof index structure of the present invention to query data records, wherein the single-value query of EMBT includes the following steps:

[0071] Step 31: Start searching from the root node, compare the size of each keyword of the root node with the keyword searchkey to be queried, and determine the descending position;

[0072] Step 32: Compare the size of the child node HK with the search key. If the search key is smaller than HK, continue comparing the size of each keyword of the child node with the search key to determine the position to descend from the current layer. Otherwise, it means that this child node must have split during the downward search and has not been updated in the parent node. At this time, it is necessary to find the right sibling node through the right sibling pointer. The child node may have split multiple times, so it is necessary to keep searching to the right. Repeat this step until you descend to the leaf node layer.

[0073] Step 33: Find the index tuple corresponding to the search key in the leaf node to retrieve data that meets the conditions. If there is no index tuple corresponding to the search key at the leaf node level, it means that there is no data that meets the conditions. The query ends.

[0074] EMBT range queries include two types: the first type determines the query range through a keyword, for example, searchkey<=key1; the second type determines the search range through two keywords, for example, key2<=searchkey<=key3.

[0075] The first type of range query with search key <= key1 includes the following steps: searching for the index tuple with search key = key1 through a single-value query; if the query fails, the query ends; if an index tuple that meets the conditions is found, the query traverses the leaf node layer in the appropriate direction to retrieve the data that meets the conditions until the query ends.

[0076] The second type of range query with key2<=search key<=key3 includes the following steps:

[0077] Through single-value query, find the location of the leaf node index tuple where search key = key2; if the query fails, end; if the index tuple that meets the conditions is found, traverse the leaf node layer in the appropriate direction until searchkey <= key3 is no longer satisfied, and end the query.

[0078] Vice versa, start the query from key3 and traverse in reverse until the search key>=key2 is no longer satisfied and end the query.

[0079] Due to the setting of left and right sibling pointers, efficient bidirectional queries are supported, and both types of range queries can benefit from this.

[0080] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a tamper-proof index structure, characterized in that: The basic structure is a B+ index tree, with the bottom layer being the leaf node layer and the remaining layers being the non-leaf node layers. Each node can store multiple index tuples, with no strict requirement for a uniform fan-out number for each node. The number of fan-outs is determined based on the available storage space on the node, and variable key lengths are supported. Each index tuple consists of a key, a pointer, and a hash field. The pointer of the index tuple at the non-leaf node level points to the index node at the next level; the pointer of the index tuple at the leaf node level points to the data block; the hash field of the index tuple at the leaf node level is obtained by using the hash algorithm to calculate the data block it points to; the hash field of the index tuple at the non-leaf node level is calculated by concatenating the hash fields of all the index tuples of the lower-level index nodes it points to and performing the hash operation again; Construct the first node and allocate the set storage space for the node; Insert the index tuples corresponding to the data blocks into the first node one by one: the search key, the pointer to the data block, and the hash value of the data block. Before inserting each index tuple, determine whether the remaining storage space of the current node is sufficient to store the index tuple. If it is sufficient, continue inserting; otherwise, proceed to the next step. Construct a new node as the right sibling node of the first node, create a parent node for the first node and write the HK of the first node to the parent node, let the parent node point to the first node, the right sibling pointer of the first node points to the second node, and the left sibling pointer of the second node points to the first node; When the second node is full, a third new node is requested and a pointer from the first parent node of the first layer to the second node is created. Another parent node is created for the parent node of the first layer, and the right sibling pointer of the second node and the left sibling pointer of the third node are set. When the third node is also filled, create a parent node pointing to this node as in the beginning, set the left and right sibling pointers between the two first-level parent nodes, and apply for a right sibling node again, and set the corresponding left and right sibling pointers; This process is repeated recursively until a complete EMBT index tree is built.

2. The method for constructing a tamper-proof index structure according to claim 1, characterized in that: The High Key field is set for each non-rightmost node in each layer to identify the maximum key value of the node.

3. A method for constructing a tamper-proof index structure according to claim 1 or 2, characterized in that: Bidirectional left and right sibling pointers are set between nodes in each layer.

4. A method for constructing a tamper-proof index structure according to claim 1, characterized in that: Each node is pre-assigned a field identifying HK. After each level of the tree is filled, if it is the rightmost node, its HK field will be deleted.

5. A method for storing a tamper-proof index structure, characterized in that: Storing data records using the tamper-proof index structure according to claim 3 comprises the following steps: Step 21: Encapsulate the data records to be inserted into index tuples; Step 22: Determine the inserted leaf node through a single-value query based on the key value of the index tuple; Step 23: Further determine the insertion position in the leaf node; if the current leaf node has sufficient space, execute step 24; if the current leaf node has insufficient space, execute step 25; Step 24: Insert the index tuple and update the value of the affected hash field in each node on the path to the root node; Step 25: Perform split operation on the current leaf node; The split operation first creates a new right sibling node of the current node, then selects the specific split location according to the set copy algorithm, and copies part of the index tuple of the original child node to the newly created node; then updates the parent node related information and the left and right sibling pointers of the leaf node and the values ​​of all affected hash fields on the path from the leaf node to the root node. If the parent node has enough space, the insertion is completed; otherwise, this step is repeated in the parent node until the insertion is completed.

6. A query method for an anti-tampering index structure, characterized in that: Utilizing the tamper-proof index structure as claimed in claim 3 to query data records, including single-value query and range query; The single value query includes the following steps: Step 31: Start searching from the root node, compare the size of each keyword of the root node with the search key of the keyword to be queried, and determine the descending position; Step 32: Compare the size of the child node HK with the search key. If the search key is smaller than HK, continue to compare the size of each keyword of the child node with the search key to determine the position to descend from the current layer. Otherwise, find the right sibling node through the right sibling pointer right link. Repeat this step until you descend to the leaf node layer. Step 33: Find the index tuple corresponding to the search key in the leaf node, and query the data that meets the conditions; if there is no index tuple corresponding to the search key at the leaf node level, the query ends; Range queries include determining the query range through one keyword and determining the search range through two keywords.

Images