Verification structure updating method, data verification method, device, medium and product

Abstract

Provided in the present description are a verification structure updating method, a data verification method, an electronic device, a storage medium and a product. The method comprises: in response to an updating instruction from a first client for a tuple in a data table, acquiring a hash value of the updated tuple, recording the hash value of the updated tuple in a log, and synchronizing the log to a second client having a query requirement for the data table; and, in response to the log meeting an updating condition, using the hash value of each tuple recorded in the log to update a leaf node corresponding to the tuple in a tree, updating non-leaf nodes in the tree on the basis of hash values recorded by the updated leaf nodes, and synchronizing root hash values recorded by root nodes of the updated tree and an updated log to the second client having the query requirement on the data table.

Description

Verification Structure Update Method, Data Verification Method, Device, Medium and Product Technical Field

[0001] One or more embodiments of this specification relate to the field of database technology, and in particular to a verification structure update method, a data verification method based on a verification structure, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the related art, database service providers can provide various database services for different users based on the databases they hold, such as data query services, data update services, etc. In order to enable the client to effectively verify the correctness of the data returned by the server of the database service provider, when the server responds to the data query request of the client, it will return the query result and the query result proof to the client. The client can verify the correctness of the query result based on the query result proof and the verification certificate obtained in advance.

[0003] Among them, the verification certificate can be obtained based on the ADS (authenticated data structure) structure maintained by the server. When the server updates the data in the database it manages based on the data update request sent by the client, it also needs to update the ADS structure accordingly; the server synchronizes the updated verification certificate to all clients based on the updated ADS structure, and the client can verify the latest updated data retrieved based on the updated verification certificate.

[0004] However, the resources consumed for updating the ADS structure in the related art are relatively large. In order to ensure that the data update performance and data query performance in the database are not affected, a method of updating the ADS structure by bypass is usually adopted. However, this method will result in the inability to ensure the real-time nature of the verification certificate, and thus it is impossible to perform reliable verification on the latest updated data retrieved. Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide a verification structure update method, a data verification method based on a verification structure, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] To achieve the above object, one or more embodiments of this specification provide the following technical solutions.

[0007] According to the first aspect of one or more embodiments of this specification, a method for verifying structure update is proposed. The verification structure is used to perform data verification on data tables in a database. The verification structure includes a tree and a log. Each leaf node in the tree is used to record the hash values of each tuple in the data table, and the hash values recorded by non-leaf nodes in the tree are aggregated from the hash values recorded by their child nodes. Among them, the root hash value recorded by the root node of the tree and the log are synchronized to a second client that has a query requirement for the data table as verification vouchers. The method includes: in response to an update instruction from a first client for a tuple in the data table, obtaining the hash value of the updated tuple, recording the hash value of the updated tuple in the log, and synchronizing the log to a second client that has a query requirement for the data table; in response to the log satisfying the update condition, updating the leaf nodes in the tree corresponding to each tuple with the hash values of each tuple recorded in the log, and updating the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes, to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronizing the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table. In response to an update instruction from a first client for a tuple in the data table, obtain the hash value of the updated tuple, record the hash value of the updated tuple in the log, and synchronize the log to a second client that has a query requirement for the data table; in response to the log satisfying the update condition, update the leaf nodes in the tree corresponding to each tuple with the hash values of each tuple recorded in the log, and update the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes, to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronize the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table.

[0008] According to the second aspect of one or more embodiments of this specification, a data verification method based on a verification structure is proposed. The verification structure is obtained based on the verification structure update method described in the first aspect. The method includes: in response to a query instruction from a second client for a data table in the database, querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table; if so, returning the hash value of the target tuple recorded in the log to the second client, so that the second client performs data verification on the target tuple based on the hash value of the target tuple and the log stored locally; if not, querying the tree corresponding to the data table, determining the proof path of the leaf node corresponding to the target tuple in the tree and returning it to the second client, so that the second client performs data verification based on the proof path and the root hash value stored locally.

[0009] According to a third aspect of the embodiments of the present specification, a data verification method based on a verification structure is provided, which is applied to a second client. The verification certificate held by the second client is obtained based on the verification structure update method described in the first aspect. The verification certificate includes a root hash value corresponding to a data table in the database and a log, and the log records the hash values of at least one tuple in the data table. The method includes: sending a query instruction for the data table to the database; if the database returns the hash value of the target tuple indicated by the query instruction, performing data verification on the target tuple based on the hash value of the target tuple and the log stored locally; if the database returns a proof path of the target tuple indicated by the query instruction, performing data verification based on the proof path and the root hash value stored locally. Verify the data of the target tuple based on the hash value of the target tuple and the log stored locally; if the database returns a proof path of the target tuple indicated by the query instruction, perform data verification based on the proof path and the root hash value stored locally.

[0010] According to a fourth aspect of the embodiments of the present specification, an electronic device is provided, including: a processor; a memory for storing executable instructions of the processor; wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0011] According to a fifth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the method described in any one of the above.

[0012] According to a sixth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in any one of the above.

[0013] The technical solutions provided by the embodiments of the present specification may include the following beneficial effects.

[0014] In the embodiments of the present specification, first, a verification structure is provided, which is used to perform data verification on a data table in a database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to the second client that has a query requirement for the data table as verification certificates.

[0015] The verification structure update method provided in this embodiment uses the root hash value and the log as verification vouchers. During the verification structure update process, after receiving an update instruction, the log in the verification structure can be updated immediately and synchronized to the client that has a query requirement for the data table, thereby ensuring the real-time nature of data verification. And only when the update condition is met, all the hash values recorded in the log will be updated to the tree at once, avoiding the resource consumption problem caused by real-time tree updates, providing better performance and response speed in the data query scenario, and maintaining the high efficiency of data updates.

[0016] The data verification method provided in this embodiment can first access the log and then access the tree when performing data queries. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency. For the target tuple not recorded in the log, a proof path can be obtained using the tree in the verification structure and returned to the second client for data verification. And due to the immediate update of the above verification structure, data verification can be performed on the latest updated data in the data query scenario, providing better performance and response speed.

[0017] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] FIG. 1 is a schematic diagram of a scenario provided by an exemplary embodiment.

[0019] FIG. 2 is a flowchart of a method for updating a verification structure provided by an exemplary embodiment.

[0020] FIG. 3 is a flowchart of a data verification method based on a verification structure provided by an exemplary embodiment.

[0021] FIG. 4 is a schematic diagram before and after the update of a verification structure provided by an exemplary embodiment.

[0022] FIG. 5 is a schematic diagram of a data table based on MVCC provided by an exemplary embodiment.

[0023] FIG. 6 is a schematic diagram of a Merkle B-tree provided by an exemplary embodiment.

[0024] FIG. 7 is a flowchart of another data verification method based on a verification structure provided by an exemplary embodiment.

[0025] FIG. 8 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of apparatuses and methods that are consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims. It should be noted that: In other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0028] In the related art, database service providers can provide various database services for different users based on the databases they hold, such as data query services, data update services, etc. In order to enable the client to effectively verify the correctness of the data returned by the service end of the database service provider, when the service end responds to the data query request of the client, it will return the query result and the query result proof to the client. The client can verify the correctness of the query result based on the query result proof and the verification certificate obtained in advance.

[0029] In one example, the data owner can entrust the database management to an outsourced database service provider, so that the outsourced database service provider can provide various database services for different users. Or, the database service provider can provide various database services for different users based on the database established by itself. This embodiment does not impose any restrictions on this.

[0030] Among them, the verification certificate can be obtained based on the ADS (Authenticated Data Structure) structure maintained by the service end. When the service end updates the data in the database it manages based on the data update request sent by the client, it also needs to update the ADS structure accordingly; then the service end synchronizes the updated verification certificate to all clients based on the updated ADS structure, and the client can verify the latest updated data queried based on the updated verification certificate.

[0031] Taking Merkle Tree, Merkle B-Tree, etc. as examples of the ADS structure for illustration.

[0032] The Merkle Tree is used to record the current data state of the database table. Generally speaking, it is a way to hash a large number of aggregated data units. Each leaf node corresponds to the hash of a data unit (i.e., a tuple in the data table), and non-leaf nodes are hashed and aggregated by their child nodes to calculate the hash value of this node. Repeat the same process until the hash value of the root node is calculated.

[0033] The B-Tree (B-tree) is the most commonly used index structure for database table indexes and is a self-balancing tree that can keep data in order. This data structure enables actions such as finding data, sequential access, inserting data, and deleting to be completed in logarithmic time.

[0034] MBT (Merkle B-Tree) is a data structure that combines Merkle Tree and B-Tree. It can not only record and represent the current data state but also serve as a query index to replace the traditional B-Tree index.

[0035] Each leaf node in the Merkle Tree or Merkle B-Tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf nodes in the tree is aggregated from the hash values recorded by their child nodes. Repeat the same process until the root hash value of the root node is calculated. The verification certificate obtained by each client is the root hash value RootHash of the root node.

[0036] When the server updates the data in the database it manages based on the data update request sent by the client, for example, when updating a certain tuple in the database table, it is necessary to recalculate the hash value of this tuple, use the hash value of this tuple to update the leaf node corresponding to this tuple in the Merkle Tree, and then update the hash value of the parent node of this leaf node. Repeat the same process until the root hash value of the root node is updated, and the updated root hash value RootHash is synchronized as the latest verification certificate to each client that has a query requirement for this data table.

[0037] When the server responds to the client's data query request, it will return the query result and the proof of the query result to the client. For example, if the query result is a certain tuple, the proof of the query result is the proof path ProofPath from the leaf node corresponding to the tuple in the Merkle tree to the root node. After the client receives the query result and the proof path ProofPath, it calculates the hash value according to the query result, and then combines the proof path ProofPath to perform hash aggregation and reverse deduction to obtain the final calculation result. If the final calculation result is the same as the root hash value RootHash of the root node held by the client, it means that the data is trustworthy and has not been modified; otherwise, it means that the data has been modified.

[0038] Since the database query scenario has relatively high latency requirements for update queries, and the resources required to update the ADS structure (such as the above-mentioned Merkle tree and Merkle B-tree) in real time are relatively large. To ensure that the data update performance and data query performance in the database are not affected, the ADS structure is usually updated in a bypass manner. However, the problem caused by this method is that there is no way to ensure the real-time nature of the verification certificate determined based on the ADS structure, and there is no way to perform trustworthy verification on the latest updated data queried. Obviously, for some sensitive and important data scenarios (such as: financial asset data, platform user permission data), it does not meet the requirements if real-time verification cannot be achieved.

[0039] Based on this, the embodiments of this specification provide a verification structure update method and a data verification method based on the verification structure, aiming to solve the problem of real-time verification of data accuracy in the database query scenario.

[0040] First, a verification structure is provided. The verification structure is used to perform data verification on the data table in the database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is aggregated from the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to the second client that has a query requirement for the data table as verification certificates.

[0041] By using the root hash value and the log as verification certificates, in the process of updating the verification structure, after receiving the update instruction, the log in the verification structure can be updated immediately and synchronized to the client that has a query requirement for the data table, thus ensuring the real-time nature of data verification; and only when the update condition is met will all the hash values recorded in the log be updated to the tree at one time, avoiding the resource consumption problem caused by real-time updating of the tree, and being able to provide better performance and response speed in the data query scenario, while maintaining the high efficiency of data update.

[0042] When performing data query, the log can be accessed first and then the tree. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency. For the target tuple not recorded in the log, the proof path can be obtained by using the tree in the verification structure and returned to the second client for data verification. And due to the immediate update of the above verification structure, data verification can be performed on the newly updated data in the data query scenario, and better performance and response speed can be provided in the data query scenario.

[0043] The embodiments of this specification provide a verification structure update method and a data verification method based on the verification structure, which can be executed by the server. Exemplarily, the server assumes the role of managing the database, is responsible for processing requests from the client and performing corresponding operations; and processing various operations on the database, such as data insertion, update, deletion, and query. The verification structure update method and the data verification method based on the verification structure are applied in the server to enhance the server's data management ability for the database, ensure that the data in the database is always in a correct and complete state, and provide a reliable data verification service.

[0044] Exemplarily, please refer to FIG. 1. The server 10 is used to manage the database, and the server 10 is communicatively connected to the first client 20 and the second client 30. The first client 20 refers to the client with the need to update data; the second client 30 refers to the client with the need to query data.

[0045] In some possible application scenarios, the first client 20 and the second client 30 are different clients. For example, the first client 20 is the client corresponding to the data provider, and the second client 30 is the client corresponding to the data user. In some other possible application scenarios, the first client 20 and the second client 30 are the same client.

[0046] In some embodiments, referring to FIG. 2, FIG. 2 is a schematic flowchart of a method for verifying structure update provided by an embodiment of this specification. This method can be executed by a server responsible for managing a database. The verification structure is used to perform data verification on data tables in the database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by a non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to a second client that has a query requirement for the data table as verification vouchers; this method includes S101 to S102.

[0047] In S101, in response to an update instruction of a tuple in the data table by a first client, obtain the hash value of the updated tuple, record the hash value of the updated tuple in the log, and synchronize the log to a second client that has a query requirement for the data table.

[0048] Exemplarily, since the hash value of the updated tuple is recorded in the log, that is, the log has changed, it is necessary to synchronize the log to a second client that has a query requirement for the data table after recording the hash value of the updated tuple in the log to ensure the accuracy of data verification in the data query stage.

[0049] In S102, in response to the log meeting the update condition, update the leaf nodes in the tree corresponding to each tuple with the hash values of each tuple recorded in the log, and update the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes to obtain an updated tree and an updated log with the recorded hash values updated to the tree. Synchronize the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table.

[0050] Exemplarily, when the log meets the update condition, the server will batch-update the hash values of each tuple recorded in the log to the tree, and then release the storage space corresponding to the log. That is, both the tree and the log have changed, so the server needs to synchronize the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table to ensure the accuracy of data verification in the data query stage.

[0051] In this embodiment, by using the root hash value and the log as verification vouchers, after receiving an update instruction, the log in the verification structure can be updated immediately and synchronized to the client that needs to query the data table, thereby ensuring the real-time nature of data verification; and all the hash values recorded in the log will be updated to the tree at one time only when the update condition is met, avoiding the resource consumption problem caused by real-time tree update, providing better performance and response speed in the data query scenario, and maintaining the high efficiency of data update. It should be noted that each data table in the database has a corresponding verification structure, which includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded in the non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; the log is used to record the hash values of at least one tuple to be updated to the tree. Among them, a tuple is a basic concept in a relational database, a relationship is a table, each row in the table (i.e., each record in the database) is a tuple, and each column is an attribute.

[0053] Exemplarily, the tree in the verification structure includes a Merkle tree.

[0054] Exemplarily, the above update conditions include that the number of hash values recorded in the log reaches a preset threshold; or, the storage capacity of the log is full; or, a preset update time point is reached, etc. This embodiment does not make any restrictions on this.

[0055] In some embodiments, please refer to FIG. 3. The embodiment of the present specification also provides a data verification method based on a verification structure, and the verification structure is obtained based on the above verification structure update method; the method includes S201 to S203.

[0056] In S201, in response to a query instruction from a second client for a data table in the database, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table.

[0057] In S202, if so, return the hash value of the target tuple recorded in the log to the second client, so that the second client can perform data verification on the target tuple based on the hash value of the target tuple and the log stored locally.

[0058] In S203, if not, query the tree corresponding to the data table, determine the proof path of the leaf node corresponding to the target tuple in the tree and return it to the second client, so that the second client can perform data verification based on the proof path and the root hash value stored locally.

[0059] In this embodiment, when performing data query, the log can be accessed first and then the tree. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency. For the target tuple not recorded in the log, the proof path can be obtained by using the tree in the verification structure and returned to the second client for data verification. And due to the immediate update of the above verification structure, data verification can be performed on the newly updated data in the data query scenario, and better performance and response speed can be provided in the data query scenario.

[0060] Exemplarily, when the server returns the hash value of the target tuple or the proof path corresponding to the target tuple, it also returns the target tuple queried from the database.

[0061] When the server returns the target tuple and the hash value of the target tuple recorded in the log to the second client, the second client can query whether the hash value of the target tuple recorded in the local log stored by it is consistent with the hash value of the target tuple returned by the server. If so, the data verification passes, and the target tuple returned by the server together is trusted and not modified. If not, the data verification fails, indicating that the target tuple returned by the server together has been modified.

[0062] When the server returns the target tuple and the proof path corresponding to the target tuple to the second client, the second client can calculate the hash value based on the target tuple, and perform hash aggregation reverse inference based on the calculated hash value and the proof path to obtain the final calculation result. If the final calculation result is the same as the root hash value held by the client, the data verification passes, and the target tuple returned by the server together is trusted and not modified. Otherwise, the data verification fails, indicating that the target tuple returned by the server together has been modified.

[0063] In some embodiments, in order to accelerate the traversal of the log when performing data query, a Bloom filter is built into the log. BF (Bloom Filter) is a probabilistic data structure used to determine whether an element is in a set. Its core idea is to use multiple hash functions and a binary vector to represent the elements in the set. The characteristic of the Bloom filter is that it can tell you that a certain element "definitely is not in the filter" or "may be in the filter". The advantage of the Bloom filter is that both the space efficiency and the query time are relatively efficient, and it is especially suitable for dealing with the existence detection problem of large-scale data.

[0064] When updating the verification structure, after recording the hash value of the updated tuple in the log, the server can update the Bloom filter with the hash value of the updated tuple. Then, when performing data query, the server can use the Bloom filter in the log to query whether the hash value of the target tuple is recorded in the log; if the Bloom filter feedbacks that the hash value of the target tuple is not recorded in the log, execute the steps of the query tree; otherwise, traverse the hash values recorded in the log until the hash value of the target tuple is found. This embodiment improves the data query efficiency by using the Bloom filter.

[0065] Exemplarily, in order to further accelerate the traversal, the log includes at least two storage units, and a Bloom filter can be built in each storage unit. When updating the verification structure, after recording the hash value of the updated tuple in one of the storage units of the log, the Bloom filter of this storage unit can be updated with the hash value of the updated tuple. Then, when performing data query, the server first traverses the Bloom filters of each storage unit in the log until the Bloom filter of one of the storage units feedbacks that the hash value of the target tuple may be stored in this storage unit, or all Bloom filters feedback that the hash value of the target tuple is not recorded in this storage unit.

[0066] In some embodiments, for S102 in Figure 2, when the log meets the update condition, the server will batch update the hash values of each tuple recorded in the log to the tree, and then release the storage space corresponding to the log so that the log can continue to record new hash values.

[0067] Exemplarily, please refer to FIG. 4. Assume that there is a data table, which includes 7 rows of records, namely 7 tuples, consisting of v1, v2, v3, v4, v5, v6, and v7. The data table corresponds to a verification structure, including a tree and a log. The leaf nodes of the tree record the hash values h1~h7 corresponding to v1~v7 respectively. During the data update process, the first client updates tuples v2 and v4, assuming that a certain attribute value in tuples v2 and v4 is updated; the log records the hash value h2 corresponding to the updated tuple v2 and the hash value h4 corresponding to the updated tuple v4. When the number of hash values recorded in the log reaches a preset threshold (for example, the preset threshold is 2), the hash value h2 corresponding to the updated tuple v2 and the hash value h4 corresponding to the updated tuple v4 recorded in the log are batch-updated to the tree, and the relevant non-leaf nodes need to be updated accordingly, so as to obtain an updated verification structure. The gray nodes in the tree in the updated verification structure are the nodes whose hash values need to be updated.

[0068] Exemplarily, in order to reduce the overall update overhead of the tree, during the process of updating the hash values recorded in the log to the tree, if there are multiple hash values recorded in the log for the same tuple, the server can use the latest recorded hash value among the multiple hash values to update the leaf node corresponding to the tuple in the tree. That is to say, only the latest recorded hash value needs to be written back to the leaf node corresponding to the tuple in the tree, reducing the update overhead caused by multiple hash values of the same tuple being repeatedly written into the same leaf node, and also being beneficial to improving the update efficiency.

[0069] In some embodiments, the database adopts the MVCC (Multi-Version Concurrency Control) mechanism to support transactions; each transaction corresponds to a unique transaction identifier. Transactions include query transactions and update transactions. Update transactions include update instructions for the data table; query transactions include query instructions for the data table.

[0070] MVCC (Multi-Version Concurrency Control) is a database concurrency control mechanism that supports the execution of concurrent transactions by maintaining multiple versions of data in the database. When each transaction reads data, it can see a visible and consistent database state without being affected by other transactions.

[0071] The core idea of MVCC is to create an independent transaction identifier for each transaction and use the transaction identifier to determine the visibility of transactions. The transaction identifier is an incrementing number. Specifically, when a transaction starts, it is assigned a unique transaction identifier. When reading data, a transaction can only see the data committed before its transaction identifier. If other transactions are modifying or deleting a data item and the transaction identifier of that transaction is greater than the transaction identifier of the current transaction, then the current transaction will not be able to see the results of these modification or deletion operations.

[72] The database creates a new version for each modification operation and retains the old version of the data in the history. Each version is associated with the transaction identifier that executed the modification operation. That is, each tuple in the data table corresponds to at least one version, and each version has a transaction identifier range for the transaction served by the tuple representing that version. The transaction identifier range is jointly determined by the transaction identifiers of the update transaction for the tuple used to update this version and the update transaction for the tuple used to update the next version. In other words, the minimum transaction identifier served by each version is the transaction identifier of the update transaction for the tuple used to update this version, and the maximum transaction identifier served is less than the transaction identifier of the update transaction for the tuple used to update the next version.

[73] For each operation on the data table in the database, there will be a corresponding transaction identifier. Different versions of the same tuple will have two items, xmin (the minimum transaction identifier served) and xmax (the maximum transaction identifier served), to mark the transaction identifier range for the tuple of that version. In an example, refer to Figure 5. In a data table, when the tuple with id = 2 is updated, a new updated tuple with id = 2' is generated. The update transaction ID is 150, so the xmax of the old data row (id = 2) is set to 150, and the xmin of the new data row (id = 2') is set to 150. Operations with transaction identifiers in the range [100, 150) will access the data row with id = 2; operations with transaction IDs greater than or equal to 150 will access the data row with id = 2'.

[74] To adapt the verification structure to the MVCC multi-version concurrency control mechanism, each data table in the database corresponds to at least one version of the verification structure, and the transaction identification ranges served by the verification structures of each version are different. After the server responds to the log meeting the update condition, batches the hash values of each tuple recorded in the log to update them in the tree and releases the storage space corresponding to the log, it can obtain the updated tree and the updated log with the hash values recorded updated in the tree, and then determine the updated tree and the updated log as the verification structure of the latest version; and determine the transaction identification range served by the verification structure of the latest version according to the transaction identification ranges served by the tuples corresponding to each leaf node in the updated tree. In this embodiment, by determining the transaction identification range served by the verification structure of the latest version, the transaction identification ranges served by the verification structures of each version are clarified, so that the verification results of the corresponding versions can be accurately utilized for data verification in subsequent query processes.

[75] Please refer to FIG. 4. For example, the verification structure before update shown in FIG. 4 is the verification structure of the first version. After updating the hash value of the log in the verification structure shown in FIG. 4 to the tree, the updated verification structure can be determined as the verification structure of the second version. The transaction identification ranges served by the verification structures of the two versions are different. Taking the tuple with id = 2 in FIG. 5 as an example, assume that the verification structure of the first version is used to perform data verification on the tuple with id = 2, and the served transaction identification range is [100, 150), and the verification structure of the second version is used to perform data verification on the tuple with id = 2, and the served transaction identification range is [150, +∞). o

[76] Then when performing a data query, in response to a query instruction from the second client for a data table in the database, the server first obtains the transaction identification of the query transaction carried by the query instruction; according to the transaction identification of the query transaction carried by the query instruction Identify the transaction identification range served by the verification structures of different versions corresponding to the data table, and determine the target verification structure for serving the query transaction to which the query instruction belongs. Then, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log in the target verification structure; if so, return the hash value of the target tuple recorded in the log to the second client, so that the second client can perform data verification on the target tuple based on the hash value of the target tuple and the log stored locally. If not, query the tree in the target verification structure, determine the proof path of the leaf node corresponding to the target tuple in the tree, and return it to the second client, so that the second client can perform data verification based on the proof path and the root hash value stored locally. In this embodiment, the target verification structure is determined according to the transaction identification of the query transaction, and the verification structure serving the query transaction can be directly located, which can reduce unnecessary traversal and verification processes, and improve the query efficiency and response speed.

[0077] In some embodiments, the server can determine the minimum transaction identification in the currently ongoing transactions in the database in real time or periodically, and then compare the minimum transaction identification with the transaction identification ranges served by the verification structures of different versions respectively, to determine the verification structures to be deleted that meet the deletion conditions in the verification structures of different versions and delete them; wherein, the deletion condition includes that the maximum transaction identification served by the verification structure to be deleted is less than the minimum transaction identification in the currently ongoing transactions in the database. In this embodiment, by deleting the verification structures that are no longer used, the effective recycling of the verification structures is realized, the storage space can be released, and the overall performance and efficiency of the database can be improved.

[0078] In some embodiments, in order to further improve the update query performance of the database, the verification function and the index function are combined. The above verification structure is also used for data indexing; each leaf node in the tree is also used to record the index value and storage address of each tuple in the data table; each non-leaf node in the tree also records the index value of the tuple used to represent the search order; the log is also used to record the index value and storage address of at least one tuple to be updated in the tree; wherein, the index value of each tuple includes the primary key of the tuple or the attribute value of the tuple other than the primary key. In this embodiment, the verification structure is used for data indexing, which can speed up the query speed. The tree in the verification structure can record the index value and storage address of the tuple. By searching the index value, the storage location of the target tuple can be quickly located, reducing the overhead of traversing the entire data table and improving the query efficiency. In addition, by recording the index value of the tuple in the non-leaf node, the ordered access to the tuple can be achieved. In the query or range search operation, the traversal can be performed according to the order of the index value, reducing the number of disks 10 and improving the efficiency of data access. By using the verification structure for data indexing, the consistency and integrity of the index can also be guaranteed. During the update operation, the verification structure is updated first, and then the data table is updated to ensure the consistency of the index and data. This can avoid index corruption or data anomalies and improve the reliability of the database.

[0079] Exemplarily, the tree in the verification structure includes a Merkle B-tree obtained by combining a Merkle tree structure and a B-tree structure. The Merkle B-tree combines the Merkle tree and the B-tree structure to implement indexing and verification functions; in the Merkle B-tree, Each leaf node stores the index value and storage address of the tuple in the data table, and calculates the hash value of the tuple data. Non-leaf nodes store the index value of the tuple used to represent the search order, and calculate the hash value of the child node. Through the hash value mechanism of the Merkle tree, the integrity and consistency of the data in the tree can be verified. The Merkle B-tree makes indexing and verification operations in the verification structure more efficient and reliable. It not only utilizes the balance and multi-way search characteristics of the B-tree to reduce the number of disk 10, but also uses the hash value mechanism of the Merkle tree to ensure the integrity and consistency of the data, which can improve the performance and efficiency of the verification structure while ensuring the security and reliability of the data.

[0080] In one example, refer to FIG. 6, which shows a Merkle B-tree for data verification and indexing. Assume there is a data table that includes 7 rows of records, namely 7 tuples, which are v1, v2, v3, v4, v5, v6, and v7. Each leaf node in the Merkle B-tree records <index value, hash value, storage address of the tuple> corresponding to v1~v7 respectively. The non-leaf nodes of the Merkle B-tree are used to record the index value of the tuple representing the search order and the hash value aggregated from the hash values of its child nodes. For example, the index value corresponding to v4 in the root node indicates to search for <index value, hash value, storage address of the tuple> corresponding to v1~v4 from the left subtree, and the index value corresponding to v5 in the root node indicates to search for <index value, hash value, storage address of the tuple> corresponding to v5~v7 from the right subtree. The meanings indicated by the index values in the child nodes of the root node are similar and will not be elaborated here.

[0081] When performing a data query, the server responds to the query instruction of the second client for the data table in the database, and according to the index value of the target tuple carried in the query instruction, queries whether the storage address and hash value of the target tuple are recorded in the log corresponding to the data table. If so, reads the target tuple from the storage address of the target tuple recorded in the log, and returns the target tuple and the hash value of the target tuple recorded in the log to the second client. If not, queries the tree to determine the leaf node corresponding to the target tuple, reads the target tuple from the storage address recorded in the leaf node corresponding to the target tuple, and determines the proof path based on the leaf node corresponding to the target tuple, and returns the target tuple and the proof path of the leaf node corresponding to the target tuple in the tree to the second client. In this embodiment, by querying the storage address and hash value of the target tuple in the log record, the target tuple can be quickly located and returned to the second client, thereby improving the query speed and efficiency. And by combining the index function and the verification function, the hash value for data verification can be obtained together when indexing the storage address of the target tuple, which is beneficial to improving the performance of update query and data verification.

[0082] Correspondingly, refer to FIG. 7. An embodiment of the present specification also provides a data verification method based on a verification structure, which is applied to the second client. The verification certificate held by the second client for data verification is obtained based on the above verification structure update method. The verification certificate includes the root hash value corresponding to the data table in the database and the log, and the log records the hash values of at least one tuple in the data table. The method includes S301 to S3O3.

[0083] In S301, a query instruction for the data table is sent to the database.

[0084] In S302, if the database returns the hash value of the target tuple indicated by the query instruction, data verification of the target tuple is performed based on the hash value of the target tuple and the log stored locally.

[0085] In S3O3, if the database returns the proof path of the target tuple indicated by the query instruction, data verification is performed based on the proof path and the root hash value stored locally.

[0086] In this embodiment, the second client holds the root hash value and the log as verification vouchers, so that data verification can be performed based on the root hash value and the log.

[0087] Exemplarily, when the server returns the hash value of the target tuple or the proof path corresponding to the target tuple, the server also returns the target tuple queried from the database.

[0088] When the server returns the target tuple and the hash value of the target tuple recorded in the log to the second client, the second client can query whether the hash value of the target tuple recorded in the log stored locally is the same as the hash value of the target tuple returned by the server. If so, the data verification passes, and the target tuple returned by the server is trusted and not modified; if not, the data verification fails, indicating that the target tuple returned by the server has been modified.

[0089] When the server returns the target tuple and the proof path corresponding to the target tuple to the second client, the second client can calculate the hash value based on the target tuple, and perform hash aggregation reverse inference based on the calculated hash value and the proof path to obtain the final calculation result. If the final calculation result is the same as the root hash value held by the client, the data verification passes, and the target tuple returned by the server is trusted and not modified; otherwise, the data verification fails, indicating that the target tuple returned by the server has been modified.

[0090] The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the features. However, due to space limitations, they are not described one by one. Therefore, any arbitrary combination of the various technical features in the above embodiments also belongs to the scope disclosed in this specification.

[0091] ​In some embodiments, the embodiments of this specification also provide an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor realizes the method described in any one of the above by running the executable instructions.

[0092] Exemplarily, FIG. 8 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Referring to FIG. 8, at the hardware level, the device includes a processor 802, an internal bus 804, a network interface 806, a memory 808, and a non-volatile memory 810. Of course, it may also include other hardware required by other scenarios. One or more embodiments of this specification can be implemented in a software manner. For example, the processor 802 reads the corresponding computer program from the non-volatile memory 810 into the memory 808 and then runs it. Of course, in addition to the software implementation manner, one or more embodiments of this specification do not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0093] In some embodiments, the embodiments of this specification also provide a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any one of the above are realized.

[0094] In some embodiments, the embodiments of this specification also provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the above are realized. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this specification are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0096] The systems, devices, modules or units illustrated in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0097] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0098] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0099] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.

[0100] It should also be noted that the term "comprises", "comprising", or any other variation thereof is intended to cover non-exclusive The inclusion of, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or apparatus. Without more limitations, an element limited by the statement "including an ....." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0101] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] The terms used in one or more embodiments of this specification are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "that" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0103] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "upon determining".

[0104] The above is only the preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope protected by one or more embodiments of this specification.

Claims

Claims 1. A method for verifying structure update, where the verification structure is used to perform data verification on data tables in a database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by a non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; where, The root hash value recorded by the root node of the tree and the log are synchronized to a second client that has a query requirement for the data table; the method includes: in response to an update instruction of a tuple in the data table by a first client, obtaining the hash value of the updated tuple, recording the hash value of the updated tuple in the log, and synchronizing the log to a second client that has a query requirement for the data table; in response to the log meeting the update condition, updating the leaf nodes in the tree corresponding to the respective tuples with the hash values of the respective tuples recorded in the log, and updating the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes, to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronizing the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table.

2. The method according to claim 1, wherein a Bloom filter is built in the log; after recording the hash value of the updated tuple in the log, the method further comprises: Updating the Bloom filter with the hash value of the updated tuple; and / or the update condition includes: the number of hash values recorded in the log reaches a preset threshold.

3. The method according to claim 1, wherein updating the leaf nodes in the tree corresponding to the respective tuples by using the hash values of the respective tuples recorded in the log comprises: If there are multiple hash values recorded in the log for the same tuple, updating the leaf node in the tree corresponding to the tuple with the latest recorded hash value among the multiple hash values.

4. The method according to claim 1, wherein the database adopts an MVCC multi-version concurrent control mechanism to support transactions; each transaction corresponds to a unique transaction identifier, the transactions include query transactions and update transactions, the update transactions include the update instructions; each tuple in the data table corresponds to at least one version, and each version has a transaction identifier range for indicating the transactions served by the tuple of this version, and the transaction identifier range is determined according to the transaction identifiers of the update transaction for updating the tuple of this version and the update transaction for updating the tuple of the next version; the data table corresponds to at least one version of verification structure, and the transaction identifier ranges served by the verification structures of each version are different; Before synchronizing the hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table, the method further includes: Determining the updated tree and the updated log as the verification structure of the latest version; and determining the transaction identifier range served by the verification structure of the latest version according to the transaction identifier ranges served by the tuples corresponding to the respective leaf nodes in the updated tree.

5. The method according to claim 4, further comprising: Determining the minimum transaction identifier in the transaction currently being performed by the database; Comparing the minimum transaction identifier with the transaction identifier ranges served by the verification structures of different versions respectively, to determine the verification structures to be deleted that meet the deletion condition in the verification structures of different versions and deleting them; wherein the deletion condition includes that the maximum transaction identifier served by the verification structure to be deleted is less than the minimum transaction identifier.

6. The verification structure according to claim 1 is further used for data indexing; each leaf node in the tree is further used to record the index value and storage address of each tuple in the data table; each non-leaf node in the tree further records the index value of the tuple representing the search order; the log is further used to record the index value and storage address of at least one tuple to be updated to the tree; wherein, the index value of each tuple includes the primary key of the tuple or the attribute value other than the primary key in the tuple.

7. A data verification method based on a verification structure, where the verification structure is obtained by updating according to the verification structure updating method described in any one of claims 1 to 6; the method includes: In response to a query instruction of a second client for a data table in the database, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table; If so, return the hash value of the target tuple recorded in the log to the second client, so that the second client performs data verification on the target tuple based on the hash value of the target tuple and the log stored locally; if not, query the tree corresponding to the data table, determine the proof path of the leaf node corresponding to the target tuple in the tree and return it to the second client, so that the second client performs data verification based on the proof path and the root hash value stored locally.

8. The method according to claim 7, wherein a Bloom filter is built in the log; The query of whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table includes: using the Bloom filter in the log to query whether the hash value of the target tuple is recorded in the log; if the Bloom filter feedbacks that the hash value of the target tuple is not recorded in the log, execute the step of querying the tree; otherwise, traverse the hash values recorded in the log until the hash value of the target tuple is queried.

9. The method according to claim 7, wherein the database adopts a multi-version concurrency control (MVCC) mechanism to support transactions; each transaction corresponds to a unique transaction identifier, and the transactions include query transactions and update transactions, and the query transactions include the query instructions. Before querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table, it further includes: obtaining the transaction identifier of the query transaction carried by the query instruction; determining the target verification structure for serving the query transaction to which the query instruction belongs according to the transaction identifier of the query transaction carried by the query instruction and the transaction identifier range served by the verification structures of different versions corresponding to the data table; the query of whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table includes: querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log in the target verification structure; the query of the tree corresponding to the data table includes: querying the tree in the target verification structure.

10. The method according to claim 7, wherein the verification structure is further used for data indexing; querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table, including: According to the index value of the target tuple carried by the query instruction, query whether the storage address and hash value of the target tuple are recorded in the log corresponding to the data table; The method further includes: if so, reading the target tuple from the storage address of the target tuple in the log record, and returning the target tuple and the hash value of the target tuple in the log record to the second client; if not, querying the tree to determine the leaf node corresponding to the target tuple; reading the target tuple from the storage address recorded in the leaf node corresponding to the target tuple, and returning the target tuple and the proof path of the leaf node corresponding to the target tuple in the tree to the second client.

11. A data verification method based on a verification structure, which is applied to a second client. The verification certificate held by the second client is obtained based on the verification structure update method described in any one of claims 1 to 6. The verification certificate includes a root hash value corresponding to a data table in the database and a log, and the log records the hash values of at least one tuple in the data table; the method includes: 19 Sending a query instruction for the data table to the database; if the database returns the hash value of the target tuple indicated by the query instruction, performing data verification on the target tuple based on the hash value of the target tuple and the log stored locally; if the database returns the proof path of the target tuple indicated by the query instruction, performing data verification based on the proof path and the root hash value stored locally.

12. An electronic device, comprising: Processor; A memory for storing executable instructions of the processor; wherein, the processor realizes the method described in any one of claims 1 to 11 by running the executable instructions.

13. A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in any one of claims 1 to 11 are realized.

14. A computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 11 are realized. 20

Images