Verification structure updating method, data verification method, equipment, medium and product

Abstract

One or more embodiments of the invention provide a verification structure updating method, a data verification method, electronic equipment, a storage medium and a product. The verification structure is used for performing data verification on the data table in the database; the verification structure comprises a tree and a log; the verification structure updating method comprises the following steps: in response to an updating instruction of a first client for a tuple in a data table, obtaining a hash value of the updated tuple, recording the hash value of the updated tuple in a log, and synchronizing the log to a second client having a query demand for the data table; in response to the fact that the log meets the updating condition, using the hash value of each tuple recorded in the log to update leaf nodes corresponding to each tuple in the tree, and updating non-leaf nodes in the tree based on the hash value recorded by the updated leaf nodes, and synchronizing the root hash value recorded by the root node of the updated tree and the updated log to a second client having a query demand on the data table. And the high efficiency of data updating can be maintained.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of database technology, and in particular, to a method for verifying structure update, a data verification method based on a verification structure, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the related art, database service providers can provide various database services for different users based on the databases they hold, such as data query services, data update services, etc. In order to enable the client to effectively verify the correctness of the data returned by the server of the database service provider, when the server responds to the client's data query request, it will return the query result and the query result proof to the client. The client can verify the correctness of the query result based on the query result proof and the verification certificate obtained in advance.

[0003] Among them, the verification certificate can be obtained based on the ADS (authenticated data structure) structure maintained by the server. When the server updates the data in the database it manages based on the data update request sent by the client, it also needs to update the ADS structure accordingly; the server synchronizes the updated verification certificate to all clients based on the updated ADS structure, and the client can verify the latest updated data queried based on the updated verification certificate.

[0004] However, the resources consumed for updating the ADS structure in the related art are relatively large. In order to ensure that the data update performance and data query performance in the database are not affected, the ADS structure is usually updated in a bypass manner. However, this method will result in the inability to ensure the real-time nature of the verification certificate, and thus the inability to perform a trustworthy verification on the latest updated data queried. Summary of the Invention

[0005] In view of this, one or more embodiments of this specification provide a method for verifying structure update, a data verification method based on a verification structure, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] To achieve the above object, one or more embodiments of this specification provide the following technical solutions:

[0007] According to the first aspect of one or more embodiments of the present specification, a method for verifying structure update is proposed. The verification structure is used to perform data verification on data tables in a database. The verification structure includes a tree and a log. Each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by a non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes. Among them, the root hash value recorded by the root node of the tree and the log are synchronized to a second client that has a query requirement for the data table as verification vouchers. The method includes:

[0008] In response to an update instruction for a tuple in the data table from a first client, obtain the hash value of the updated tuple, record the hash value of the updated tuple in the log, and synchronize the log to a second client that has a query requirement for the data table.

[0009] In response to the log satisfying the update condition, use the hash values of each tuple recorded in the log to update the leaf nodes in the tree corresponding to each tuple, and update the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes, to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronize the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table.

[0010] According to the second aspect of one or more embodiments of the present specification, a data verification method based on a verification structure is proposed. The verification structure is obtained based on the verification structure update method described in the first aspect. The method includes:

[0011] In response to a query instruction for a data table in the database from a second client, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table.

[0012] If so, return the hash value of the target tuple recorded in the log to the second client, so that the second client performs data verification on the target tuple based on the hash value of the target tuple and the log stored locally.

[0013] If not, query the tree corresponding to the data table, determine the proof path of the leaf node corresponding to the target tuple in the tree and return it to the second client, so that the second client performs data verification based on the proof path and the root hash value stored locally.

[0014] According to a third aspect of the embodiments of the present specification, a data verification method based on a verification structure is provided, which is applied to a second client. The verification certificate held by the second client is obtained based on the verification structure update method described in the first aspect. The verification certificate includes a root hash value corresponding to a data table in the database and a log, and the log records the hash values of at least one tuple in the data table.

[0015] The method includes:

[0016] Send a query instruction for the data table to the database;

[0017] If the database returns the hash value of the target tuple indicated by the query instruction, perform data verification on the target tuple based on the hash value of the target tuple and the log stored locally;

[0018] If the database returns the proof path of the target tuple indicated by the query instruction, perform data verification based on the proof path and the root hash value stored locally.

[0019] According to a fourth aspect of the embodiments of the present specification, an electronic device is provided, including:

[0020] A processor;

[0021] A memory for storing instructions executable by the processor;

[0022] Wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0023] According to a fifth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the steps of the method described in any one of the above.

[0024] According to a sixth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in any one of the above.

[0025] The technical solutions provided by the embodiments of the present specification may include the following beneficial effects:

[0026] In the embodiments of the present specification, first, a verification structure is provided, which is used to perform data verification on a data table in a database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by a non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to the second client that has a query requirement for the data table as verification certificates.

[0027] The verification structure update method provided in this embodiment uses the root hash value and the log as verification vouchers. During the verification structure update process, when an update instruction is received, the log in the verification structure can be updated immediately and synchronized to the client that has a query requirement for the data table, thus ensuring the real-time nature of data verification. And only when the update condition is met, all the hash values recorded in the log will be updated to the tree at one time, avoiding the resource consumption problem caused by real-time tree updates, providing better performance and response speed in the data query scenario, and maintaining the high efficiency of data updates.

[0028] The data verification method provided in this embodiment can first access the log and then access the tree when performing data queries. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency. For the target tuple not recorded in the log, a proof path can be obtained using the tree in the verification structure and returned to the second client for data verification. And due to the immediate update of the above verification structure, data verification can be performed on the latest updated data in the data query scenario, providing better performance and response speed in the data query scenario.

[0029] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. Description of the Drawings

[0030] Figure 1 is a schematic diagram of a scenario provided by an exemplary embodiment.

[0031] Figure 2 is a flowchart of a method for updating a verification structure provided by an exemplary embodiment.

[0032] Figure 3 is a flowchart of a data verification method based on a verification structure provided by an exemplary embodiment.

[0033] Figure 4 is a schematic diagram before and after the update of a verification structure provided by an exemplary embodiment.

[0034] Figure 5 is a schematic diagram of a data table based on MVCC provided by an exemplary embodiment.

[0035] Figure 6 is a schematic diagram of a Merkle B-tree provided by an exemplary embodiment.

[0036] Figure 7 is a flowchart of another data verification method based on a verification structure provided by an exemplary embodiment.

[0037] Figure 8 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Detailed implementation manners

[0038] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all the implementation manners consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0039] It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0040] In the related art, database service providers can provide various database services for different users based on the databases they hold, such as data query services, data update services, etc. In order to enable the client to effectively verify the correctness of the data returned by the server of the database service provider, when the server responds to the data query request of the client, it will return the query result and the query result proof to the client. The client can verify the correctness of the query result based on the query result proof and the verification certificate obtained in advance.

[0041] In one example, the data owner can entrust the database management to an outsourced database service provider, so that the outsourced database service provider can provide various database services for different users. Or, the database service provider can provide various database services for different users based on the database established by itself. This embodiment does not make any restrictions on this.

[0042] Among them, the verification certificate can be obtained based on the ADS (authenticated data structure) structure maintained by the server. When the server updates the data in the database it manages based on the data update request sent by the client, it also needs to update the ADS structure accordingly; then the server synchronizes the updated verification certificate to all clients based on the updated ADS structure, and the client can verify the latest updated data queried based on the updated verification certificate.

[0043] Taking Merkle Tree, Merkle B-Tree, etc. as examples of the ADS structure for illustration:

[0044] The Merkle Tree is used to record the current data state of a database table. Generally speaking, it is a way to hash a large number of aggregated data units. Each leaf node corresponds to the hash of a data unit (i.e., a tuple in the data table), and non-leaf nodes are hashed and aggregated by their child nodes to calculate the hash value of that node. Repeat the same process until the hash value of the root node is calculated.

[0045] The B-Tree (B-tree) is the most commonly used index structure for database table indexes and is a self-balancing tree that can keep data in order. This data structure enables actions such as searching for data, sequential access, inserting data, and deleting to be completed in logarithmic time.

[0046] The MBT (Merkle B-Tree) is a data structure that combines the Merkle Tree and the B-Tree. It can not only record the current data state but also serve as a query index to replace the traditional B-Tree index.

[0047] Each leaf node in the Merkle Tree or Merkle B-Tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf nodes in the tree is aggregated from the hash values recorded by their child nodes. Repeat the same process until the root hash value of the root node is calculated. The verification certificate obtained by each client is the root hash value RootHash of the root node.

[0048] When the server updates the data in the database it manages based on the data update request sent by the client, for example, when a tuple in the database table is updated, it is necessary to recalculate the hash value of that tuple, use the hash value of that tuple to update the leaf node corresponding to that tuple in the Merkle Tree, and then update the hash value of the parent node of that leaf node. Repeat the same process until the root hash value of the root node is updated, and the updated root hash value RootHash of the root node is synchronized as the latest verification certificate to each client that has a query requirement for this data table.

[0049] When the server responds to the client's data query request, it will return the query result and the proof of the query result to the client. For example, if the query result is a certain tuple, the proof of the query result is the proof path ProofPath from the leaf node corresponding to the tuple in the Merkle tree to the root node. After receiving the query result and the proof path ProofPath, the client calculates the hash value based on the query result, and then performs hash aggregation and reverse deduction in combination with the proof path ProofPath to obtain the final calculation result. If the final calculation result is the same as the root hash value RootHash of the root node held by the client, it means that the data is trustworthy and has not been modified; otherwise, it means that the data has been modified.

[0050] Since the database query scenario has relatively high latency requirements for update queries, and the resources required to update the ADS structure (such as the above-mentioned Merkle tree and Merkle B-tree) in real time are relatively large. In order to ensure that the data update performance and data query performance in the database are not affected, the ADS structure is usually updated in a bypass manner. However, the problem caused by this method is that there is no way to ensure the real-time nature of the verification certificate determined based on the ADS structure, and there is no way to perform trustworthy verification on the latest updated data queried. Obviously, for some sensitive and important data scenarios (such as: financial asset data, platform user permission data), the inability to perform real-time verification does not meet the requirements.

[0051] Based on this, the embodiments of this specification provide a verification structure update method and a data verification method based on the verification structure, aiming to solve the problem of real-time verification of data accuracy in the database query scenario.

[0052] First, a verification structure is provided. The verification structure is used to perform data verification on the data table in the database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is aggregated from the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to the second client that has a query requirement for the data table as verification certificates.

[0053] By using the root hash value and the log as verification certificates, during the update process of the verification structure, after receiving the update instruction, the log in the verification structure can be updated immediately and synchronized to the client that has a query requirement for the data table, thus ensuring the real-time nature of data verification; and only when the update condition is met, all the hash values recorded in the log will be updated to the tree at one time, avoiding the resource consumption problem caused by real-time updating of the tree, providing better performance and response speed in the data query scenario, and maintaining the high efficiency of data update at the same time.

[0054] When performing data query, the log can be accessed first and then the tree. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency. For the target tuple not recorded in the log, the proof path can be obtained by using the tree in the verification structure and returned to the second client for data verification. And due to the immediate update of the above verification structure, data verification can be performed on the newly updated data in the data query scenario, providing better performance and response speed in the data query scenario.

[0055] The embodiments of this specification provide a verification structure update method and a data verification method based on the verification structure, which can be executed by the server. Exemplarily, the server undertakes the role of managing the database, is responsible for processing the requests of the client and performing corresponding operations; and processing various operations of the database, such as data insertion, update, deletion, and query, etc. The verification structure update method and the data verification method based on the verification structure are applied to the server to enhance the server's data management ability for the database, ensure that the data in the database is always in a correct and complete state, and provide a reliable data verification service.

[0056] Exemplarily, please refer to Figure 1 , the server 10 is used to manage the database, and the server 10 is communicatively connected to the first client 20 and the second client 30. The first client 20 refers to the client with the need to update data; the second client 30 refers to the client with the need to query data.

[0057] In some possible application scenarios, the first client 20 and the second client 30 are different clients. For example, the first client 20 is the client corresponding to the data provider, and the second client 30 is the client corresponding to the data user. In some other possible application scenarios, the first client 20 and the second client 30 are the same client.

[0058] In some embodiments, please refer to Figure 2 , Figure 2 is a schematic flowchart of a verification structure update method provided by the embodiments of this specification. This method can be executed by the server responsible for managing the database. The verification structure is used to perform data verification on the data table in the database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is aggregated from the hash values recorded by its child nodes; among them, the root hash value recorded by the root node of the tree and the log are synchronized to the second client with the need to query the data table as verification vouchers; this method includes:

[0059] In S101, in response to an update instruction from a first client for a tuple in a data table, obtain the hash value of the updated tuple, record the hash value of the updated tuple in a log, and synchronize the log to a second client that has a query requirement for the data table.

[0060] Exemplarily, since the hash value of the updated tuple is recorded in the log, that is, the log has changed, it is necessary to synchronize the log to the second client that has a query requirement for the data table after recording the hash value of the updated tuple in the log, so as to ensure the accuracy of data verification during the data query phase.

[0061] In S102, in response to the log meeting the update condition, use the hash values of each tuple recorded in the log to update the leaf nodes in the tree corresponding to each tuple, and update the non-leaf nodes in the tree based on the hash values recorded by the updated leaf nodes, to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronize the root hash value recorded by the root node of the updated tree and the updated log to the second client that has a query requirement for the data table.

[0062] Exemplarily, when the log meets the update condition, the server will batch-update the hash values of each tuple recorded in the log to the tree, and then release the storage space corresponding to the log. That is, both the tree and the log have changed, so the server needs to synchronize the root hash value recorded by the root node of the updated tree and the updated log to the second client that has a query requirement for the data table, so as to ensure the accuracy of data verification during the data query phase.

[0063] In this embodiment, by using the root hash value and the log as verification vouchers, the log in the verification structure can be updated immediately after receiving an update instruction and synchronized to the client that has a query requirement for the data table, thus ensuring the real-time nature of data verification; and all the hash values recorded in the log will be updated to the tree at one time only when the update condition is met, avoiding the resource consumption problem caused by real-time tree updates, providing better performance and response speed in the data query scenario, and maintaining the high efficiency of data updates.

[0064] It should be noted that each data table in the database has a corresponding verification structure, which includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; the log is used to record the hash values of at least one tuple to be updated to the tree. Among them, a tuple is a basic concept in a relational database, a relationship is a table, each row in the table (i.e., each record in the database) is a tuple, and each column is an attribute.

[0065] Exemplarily, the tree in the verification structure includes a Merkle tree.

[0066] Exemplarily, the above update conditions include that the number of hash values of the log records reaches a preset threshold; or, the storage capacity of the log is full; or, a preset update time point is reached, etc. This embodiment does not impose any restrictions on this.

[0067] In some embodiments, referring to Figure 3 , this embodiment of the present specification also provides a data verification method based on a verification structure, where the verification structure is obtained based on the above verification structure update method; the method includes:

[0068] In S201, in response to a query instruction from a second client for a data table in the database, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table.

[0069] In S202, if so, return the hash value of the target tuple recorded in the log to the second client, so that the second client can perform data verification on the target tuple based on the hash value of the target tuple and the log stored locally.

[0070] In S203, if not, query the tree corresponding to the data table, determine the proof path of the leaf node corresponding to the target tuple in the tree and return it to the second client, so that the second client can perform data verification based on the proof path and the root hash value stored locally.

[0071] In this embodiment, when performing data query, the log can be accessed first and then the tree. If the hash value of the target tuple indicated by the query instruction is recorded in the log, it can be directly returned to the second client, which is beneficial to improving the query efficiency; for the target tuple not recorded in the log, the proof path can be obtained by using the tree in the verification structure and returned to the second client for data verification. And due to the instant update of the above verification structure, data verification can be performed on the latest updated data in the data query scenario, and better performance and response speed can be provided in the data query scenario.

[0072] Exemplarily, when the server returns the hash value of the target tuple or the proof path corresponding to the target tuple, it also returns the target tuple queried from the database.

[0073] When the server returns the target tuple and the hash value of the target tuple recorded in the log to the second client, the second client can query whether the hash value of the target tuple recorded in the log stored locally is consistent with the hash value of the target tuple returned by the server. If so, the data verification passes, and the target tuple returned by the server together is trusted and not modified; if not, the data verification fails, indicating that the target tuple returned by the server together has been modified.

[0074] When the server returns the target tuple and the corresponding proof path to the second client, the second client can calculate the hash value based on the target tuple, and perform hash aggregation inversion based on the calculated hash value and the proof path to obtain the final calculation result. If the final calculation result is the same as the root hash value held by the client, the data verification passes, indicating that the target tuple returned by the server is trustworthy and has not been modified; otherwise, the data verification fails, indicating that the target tuple returned by the server has been modified.

[0075] In some embodiments, to accelerate the traversal of the log during data query, a Bloom filter is built into the log. BF (Bloom Filter) is a probabilistic data structure used to determine whether an element is in a set. Its core idea is to use multiple hash functions and a binary vector to represent the elements in the set. The characteristic of the Bloom filter is that it can tell you that a certain element "is definitely not in the filter" or "may be in the filter". The advantage of the Bloom filter is that both the space efficiency and the query time are relatively efficient, and it is particularly suitable for dealing with the existence detection problem of large-scale data.

[0076] When updating the verification structure, after recording the hash value of the updated tuple in the log, the server can update the Bloom filter using the hash value of the updated tuple. Then, when performing data query, the server can use the Bloom filter in the log to query whether the hash value of the target tuple is recorded in the log; if the Bloom filter feedbacks that the hash value of the target tuple is not recorded in the log, perform the steps of querying the tree; otherwise, traverse the hash values recorded in the log until the hash value of the target tuple is found. This embodiment improves the data query efficiency by using the Bloom filter.

[0077] Exemplarily, to further accelerate the traversal, the log includes at least two storage units, and a Bloom filter can be built into each storage unit. When updating the verification structure, after recording the hash value of the updated tuple in one of the storage units of the log, the Bloom filter of this storage unit can be updated using the hash value of the updated tuple. Then, when performing data query, the server first traverses the Bloom filters of each storage unit in the log until the Bloom filter of one of the storage units feedbacks that the hash value of the target tuple may be stored in this storage unit, or all Bloom filters feedback that the hash value of the target tuple is not recorded in this storage unit.

[0078] In some embodiments, for Figure 2 in S102, when the log meets the update condition, the server will batch update the hash values of each tuple recorded in the log to the tree, and then release the storage space corresponding to the log so that the log can continue to record new hash values.

[0079] Exemplarily, refer to Figure 4 , assuming there is a data table, which includes a total of 7 rows of records, namely 7 tuples, v1, v2, v3, v4, v5, v6, and v7. The data table corresponds to a verification structure, including a tree and a log. The hash values h1 to h7 corresponding to v1 to v7 are recorded in the leaf nodes of the tree. During the data update process, the first client updates tuples v2 and v4. Suppose a certain attribute value in tuples v2 and v4 is updated; the hash value h2' corresponding to the updated tuple v2 and the hash value h4' corresponding to the updated tuple v4 are recorded in the log. When the number of hash values recorded in the log reaches a preset threshold (for example, the preset threshold is 2), the hash value h2' corresponding to the updated tuple v2 and the hash value h4' corresponding to the updated tuple v4 recorded in the log are batch-updated to the tree, and the relevant non-leaf nodes need to be updated accordingly, so as to obtain the updated verification structure. The gray nodes in the tree in the updated verification structure are the nodes that need to update the hash values.

[0080] Exemplarily, in order to reduce the overall update overhead of the tree, during the process of updating the hash values recorded in the log to the tree, if there are multiple hash values recorded in the log for the same tuple, the server can use the latest recorded hash value among the multiple hash values to update the leaf node corresponding to the tuple in the tree. That is to say, only the latest recorded hash value needs to be written back to the leaf node corresponding to the tuple in the tree, reducing the update overhead caused by multiple hash values of the same tuple being repeatedly written to the same leaf node and also being beneficial to improving the update efficiency.

[0081] In some embodiments, the database adopts the MVCC multi-version concurrency control mechanism to support transactions; each transaction corresponds to a unique transaction identifier. Transactions include query transactions and update transactions. Update transactions include update instructions for the data table; query transactions include query instructions for the data table.

[0082] MVCC (Multi-Version Concurrency Control) is a database concurrency control mechanism that supports the execution of concurrent transactions by maintaining multiple versions of data in the database. Each transaction can see a visible and consistent database state when reading data, without being affected by other transactions.

[0083] The core idea of MVCC is to create an independent transaction identifier for each transaction and use the transaction identifier to determine the visibility of the transaction. The transaction identifier is an incrementing number. Specifically, when a transaction starts, it is assigned a unique transaction identifier. When reading data, a transaction can only see the data that has been committed before its transaction identifier. If other transactions are modifying or deleting a data item and the transaction identifier of that transaction is greater than the transaction identifier of the current transaction, then the current transaction will not be able to see the results of these modification or deletion operations.

[0084] The database creates a new version for each modification operation and retains the old version of the data in the history. Each version is associated with the transaction identifier that executed the modification operation. That is, each tuple in the data table corresponds to at least one version, and each version has a transaction identifier range for the tuple representing that version. The transaction identifier range is jointly determined by the transaction identifiers of the update transaction for the tuple that updates this version and the update transaction for the tuple that updates the next version. In other words, the minimum transaction identifier served by each version is the transaction identifier of the update transaction for the tuple that updates this version, and the maximum transaction identifier served is less than the transaction identifier of the update transaction for the tuple that updates the next version.

[0085] For each operation on the data table in the database, there will be a corresponding transaction identifier. Different versions of the same tuple will have two items, xmin (the minimum transaction identifier served) and xmax (the maximum transaction identifier served), to mark the transaction identifier range for the tuple of that version. In an example, see Figure 5 , in a data table, the tuple with id = 2 is updated to produce an updated tuple with id = 2'. The update transaction ID is 150, so the xmax of the old data row (id = 2) is set to 150, and the xmin of the new data row (id = 2') is set to 150. Operations with transaction identifiers in the range [100, 150) will access the data row with id = 2; operations with transaction IDs greater than or equal to 150 will access the data row with id = 2'.

[0086] To adapt the verification structure to the MVCC multi-version concurrency control mechanism, each data table in the database corresponds to at least one version of the verification structure, and the transaction identification ranges served by the verification structures of each version are different. After the server responds to the log meeting the update condition, batches the hash values of each tuple recorded in the log to update them into the tree and releases the storage space corresponding to the log, it can obtain the updated tree and the updated log with the recorded hash values updated into the tree, and then determine the updated tree and the updated log as the verification structure of the latest version; and determine the transaction identification range served by the verification structure of the latest version according to the transaction identification ranges served by the tuples corresponding to each leaf node in the updated tree. In this embodiment, by determining the transaction identification range served by the verification structure of the latest version, the transaction identification ranges served by the verification structures of each version are clarified, so that the verification results of the corresponding versions can be accurately utilized for data verification in the subsequent query process.

[0087] Please refer to Figure 4 , for example Figure 4 The verification structure before the update shown is the verification structure of the first version. After updating the hash value of the log in the verification structure shown in Figure 4 into the tree, the updated verification structure can be determined as the verification structure of the second version. The transaction identification ranges served by the verification structures of the two versions are different. Taking the tuple with id = 2 in Figure 5 as an example, assume that the verification structure of the first version is used to perform data verification on the tuple with id = 2, and the served transaction identification range is [100, 150), and the verification structure of the second version is used to perform data verification on the tuple with id = 2', and the served transaction identification range is [150, +∞).

[0088] When performing data query, in response to a query instruction from a second client for a data table in the database, the server first obtains the transaction identifier of the query transaction carried in the query instruction; based on the transaction identifier of the query transaction carried in the query instruction and the transaction identifier range served by the verification structures of different versions corresponding to the data table, it determines the target verification structure for serving the query transaction to which the query instruction belongs. Then it queries whether the hash value of the target tuple indicated by the query instruction is recorded in the log of the target verification structure; if so, it returns the hash value of the target tuple recorded in the log to the second client, so that the second client can perform data verification on the target tuple based on the hash value of the target tuple and the log stored locally. If not, it queries the tree in the target verification structure, determines the proof path of the leaf node corresponding to the target tuple in the tree and returns it to the second client, so that the second client can perform data verification based on the proof path and the root hash value stored locally. In this embodiment, the target verification structure is determined according to the transaction identifier of the query transaction, which can directly locate the verification structure serving the query transaction, thus reducing unnecessary traversal and verification processes and improving the query efficiency and response speed.

[0089] In some embodiments, the server can determine the minimum transaction identifier in the current transactions being performed in the database in real time or periodically, and then compare the minimum transaction identifier with the transaction identifier ranges served by the verification structures of different versions respectively to determine the verification structures to be deleted that meet the deletion conditions among the verification structures of different versions and delete them; wherein, the deletion condition includes that the maximum transaction identifier served by the verification structure to be deleted is less than the minimum transaction identifier in the current transactions being performed in the database. In this embodiment, by deleting the verification structures that are no longer in use, the effective recycling of the verification structures is realized, which can release the storage space and improve the overall performance and efficiency of the database.

[0090] In some embodiments, to further improve the update query performance of the database, the verification function and the index function are combined. The above verification structure is also used for data indexing; each leaf node in the tree is also used to record the index value and storage address of each tuple in the data table; each non-leaf node in the tree also records the index value of the tuple representing the search order; the log is also used to record the index value and storage address of at least one tuple to be updated into the tree; wherein, the index value of each tuple includes the primary key of the tuple or the attribute value other than the primary key in the tuple. In this embodiment, the verification structure is used for data indexing, which can speed up the query. The tree in the verification structure can record the index value and storage address of the tuple. Through the search of the index value, the storage location of the target tuple can be quickly located, reducing the overhead of traversing the entire data table and improving the query efficiency. And by recording the index value of the tuple in the non-leaf node, ordered access to the tuple can be realized. In the query or range search operation, traversal can be performed according to the order of the index values, reducing the number of disk I / Os and improving the data access efficiency. By using the verification structure for data indexing, the consistency and integrity of the index can also be guaranteed. During the update operation, the verification structure is updated first, and then the data table is updated, which can ensure the consistency of the index and the data, thus avoiding the situation of index corruption or data anomaly and enhancing the reliability of the database.

[0091] Exemplarily, the tree in the verification structure includes a Merkle B-tree obtained by combining a Merkle tree structure and a B-tree structure. The Merkle B-tree combines the Merkle tree and the B-tree structure to implement the index and verification functions; in the Merkle B-tree, each leaf node stores the index value and storage address of the tuple in the data table and calculates the hash value of the tuple data. The non-leaf node stores the index value of the tuple representing the search order and calculates the hash value of the child nodes. Through the hash value mechanism of the Merkle tree, the integrity and consistency of the data in the tree can be verified. The Merkle B-tree makes the index and verification operations in the verification structure more efficient and reliable. It utilizes both the balance and multi-way search characteristics of the B-tree to reduce the number of disk I / Os and also relies on the hash value mechanism of the Merkle tree to ensure the integrity and consistency of the data, thereby improving the performance and efficiency of the verification structure and ensuring the security and reliability of the data.

[0092] In one example, please refer to Figure 6, which shows a Merkle B-tree for data verification and indexing. Suppose there is a data table that includes a total of 7 rows of records, namely 7 tuples, v1, v2, v3, v4, v5, v6, and v7; each leaf node in the Merkle B-tree records <index value, hash value, storage address of the tuple> corresponding to v1 to v7 respectively. The non-leaf nodes of the Merkle B-tree are used to record the index value of the tuple representing the search order and the hash value aggregated from the hash values of its child nodes. For example, the index value corresponding to v4 in the root node indicates to search for <index value, hash value, storage address of the tuple> corresponding to v1 to v4 from the left subtree, and the index value corresponding to v5 in the root node indicates to search for <index value, hash value, storage address of the tuple> corresponding to v5 to v7 from the right subtree; the meanings represented by the index values in the child nodes of the root node are similar and will not be elaborated here.

[0093] Then, when performing data query, the server responds to the query instruction of the second client for the data table in the database, and according to the index value of the target tuple carried in the query instruction, queries whether the storage address and hash value of the target tuple are recorded in the log corresponding to the data table; if so, reads the target tuple from the storage address of the target tuple recorded in the log and returns the target tuple and the hash value of the target tuple recorded in the log to the second client; if not, queries the tree to determine the leaf node corresponding to the target tuple, reads the target tuple from the storage address recorded in the leaf node corresponding to the target tuple, and determines the proof path based on the leaf node corresponding to the target tuple, and returns the target tuple and the proof path of the leaf node corresponding to the target tuple in the tree to the second client. In this embodiment, by querying the storage address and hash value of the target tuple in the log record, the target tuple can be quickly located and returned to the second client, thereby improving the query speed and efficiency; and by combining the index function and the verification function, the hash value for data verification can be obtained together when indexing the storage address of the target tuple, which is beneficial to improving the performance of update query and data verification.

[0094] Correspondingly, please refer to Figure 7 , this embodiment of the present specification also provides a data verification method based on a verification structure, which is applied to the second client. The verification certificate held by the second client for data verification is obtained based on the above verification structure update method. The verification certificate includes the root hash value corresponding to the data table in the database and the log, and the log records the hash values of at least one tuple in the data table. The method includes:

[0095] In S301, send a query instruction for the data table to the database.

[0096] In S302, if the database returns the hash value of the target tuple indicated by the query instruction, data verification of the target tuple is performed based on the hash value of the target tuple and the log stored locally thereof.

[0097] In S303, if the database returns the proof path of the target tuple indicated by the query instruction, data verification is performed based on the proof path and the root hash value stored locally.

[0098] In this embodiment, the second client holds the root hash value and the log as verification vouchers, so that data verification can be performed based on the root hash value and the log.

[0099] Exemplarily, when the server returns the hash value of the target tuple or the proof path corresponding to the target tuple, the target tuple queried from the database is also returned.

[0100] When the server returns the target tuple and the hash value of the target tuple recorded in the log to the second client, the second client can query whether the hash value of the target tuple recorded in the log stored locally is consistent with the hash value of the target tuple returned by the server. If so, the data verification passes, and the target tuple returned by the server together is trusted and not modified. If not, the data verification fails, indicating that the target tuple returned by the server together has been modified.

[0101] When the server returns the target tuple and the proof path corresponding to the target tuple to the second client, the second client can calculate the hash value based on the target tuple, and perform hash aggregation inversion based on the calculated hash value and the proof path to obtain the final calculation result. If the final calculation result is the same as the root hash value held by the client, the data verification passes, and the target tuple returned by the server together is trusted and not modified. Otherwise, the data verification fails, indicating that the target tuple returned by the server together has been modified.

[0102] The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also belongs to the scope disclosed in this specification.

[0103] In some embodiments, the embodiments of this specification also provide an electronic device, including: a processor; a memory for storing executable instructions that can be executed by the processor; wherein, the processor realizes the method described in any one of the above by running the executable instructions.

[0104] Exemplarily, Figure 8 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Please refer to Figure 8, at the hardware level, the device includes a processor 802, an internal bus 804, a network interface 806, a memory 808, and a non-volatile memory 810. Of course, it may also include other hardware required for other scenarios. One or more embodiments of this specification can be implemented in software. For example, the processor 802 reads the corresponding computer program from the non-volatile memory 810 into the memory 808 and then runs it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0105] In some embodiments, embodiments of this specification also provide a computer-readable storage medium, on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above are implemented.

[0106] In some embodiments, embodiments of this specification also provide a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method described in any of the above are implemented.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this specification are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0108] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or a combination of any several of these devices.

[0109] In a typical configuration, a computer includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0110] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0111] A computer-readable medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transitory media that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0112] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0113] The specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The terms used in one or more embodiments of this specification are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the" and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0115] It should be understood that although the terms first, second, third, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0116] The foregoing are only preferred embodiments of one or more embodiments of this specification and are not intended to limit one or more embodiments of this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included within the scope protected by one or more embodiments of this specification.

Claims

1. A method for verifying structure update, where the verification structure is used to perform data verification on a data table in a database; the verification structure includes a tree and a log; each leaf node in the tree is used to record the hash value of each tuple in the data table, and the hash value recorded by the non-leaf node in the tree is obtained by aggregating the hash values recorded by its child nodes; among them, The root hash value recorded by the root node of the tree and the log are synchronized to a second client that has a query requirement for the data table; the method includes: In response to an update instruction of a tuple in the data table by a first client, obtain the hash value of the updated tuple, record the hash value of the updated tuple in the log, and synchronize the log to a second client that has a query requirement for the data table; In response to the log meeting the update condition, use the hash values of the respective tuples recorded in the log to update the leaf nodes in the tree corresponding to the respective tuples, and based on the hash values recorded by the updated leaf nodes, update the non-leaf nodes in the tree to obtain an updated tree and an updated log with the recorded hash values updated to the tree, and synchronize the root hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table.

2. The method according to claim 1, wherein a Bloom filter is built in the log; After recording the hash value of the updated tuple in the log, further include: updating the Bloom filter with the hash value of the updated tuple; and / or The update condition includes: the number of hash values recorded in the log reaches a preset threshold.

3. The method according to claim 1, wherein the step of using the hash values of the respective tuples recorded in the log to update the leaf nodes in the tree corresponding to the respective tuples includes: If multiple hash values for the same tuple are recorded in the log, use the latest recorded hash value among the multiple hash values to update the leaf node in the tree corresponding to the tuple.

4. The method according to claim 1, wherein the database adopts an MVCC multi-version concurrency control mechanism to support transactions; each transaction corresponds to a unique transaction identifier, the transactions include query transactions and update transactions, and the update transaction includes the update instruction; Each tuple in the data table corresponds to at least one version, and each version has a transaction identifier range for indicating the transaction that the tuple of this version serves, and the transaction identifier range is determined according to the transaction identifiers of the update transaction for updating the tuple of this version and the update transaction for updating the tuple of the next version; The data table corresponds to at least one version of the verification structure, and the transaction identifier ranges served by the verification structures of each version are different; Before synchronizing the hash value recorded by the root node of the updated tree and the updated log to a second client that has a query requirement for the data table, further include: Determine the updated tree and the updated log as the verification structure of the latest version; and According to the transaction identifier range served by the tuple corresponding to each leaf node in the updated tree, determine the transaction identifier range served by the verification structure of the latest version.

5. The method according to claim 4, further include: Determine the minimum transaction identifier in the transaction currently being performed by the database; Compare the minimum transaction identifier with the ranges of transaction identifiers served by verification structures of different versions respectively to determine the verification structures to be deleted that meet the deletion conditions in the verification structures of different versions and delete them; wherein, the deletion conditions include that the maximum transaction identifier served by the verification structure to be deleted is less than the minimum transaction identifier.

6. The method according to claim 1, wherein the verification structure is further used for data indexing; each leaf node in the tree is further used to record the index value and storage address of each tuple in the data table; each non-leaf node in the tree further records the index value of the tuple used to represent the search order; the log is further used to record the index value and storage address of at least one tuple to be updated to the tree; Among them, The index value of each tuple includes the primary key of the tuple or the attribute value of the tuple other than the primary key.

7. A data verification method based on a verification structure, wherein the verification structure is obtained based on the verification structure update method according to any one of claims 1 to 6; the method includes: In response to a query instruction of a second client for a data table in the database, query whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table; If so, return the hash value of the target tuple recorded in the log to the second client, so that the second client performs data verification on the target tuple based on the hash value of the target tuple and the log stored locally; If not, query the tree corresponding to the data table, determine the proof path of the leaf node corresponding to the target tuple in the tree and return it to the second client, so that the second client performs data verification based on the proof path and the root hash value stored locally.

8. The method according to claim 7, wherein a Bloom filter is built in the log; The querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table includes: Use the Bloom filter in the log to query whether the hash value of the target tuple is recorded in the log; If the Bloom filter feedbacks that the hash value of the target tuple is not recorded in the log, execute the step of querying the tree; otherwise, traverse the hash values recorded in the log until the hash value of the target tuple is queried.

9. The method according to claim 7, wherein the database adopts the MVCC multi-version concurrent control mechanism to support transactions; each transaction corresponds to a unique transaction identifier, the transaction includes a query transaction and an update transaction, and the query transaction includes the query instruction; Before querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table, it further includes: Obtain the transaction identifier of the query transaction carried by the query instruction; Determine the target verification structure for serving the query transaction to which the query instruction belongs according to the transaction identifier of the query transaction carried by the query instruction and the ranges of transaction identifiers served by different versions of the verification structure corresponding to the data table; Querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table includes: Querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log in the target verification structure; The querying the tree corresponding to the data table includes: Querying the tree in the target verification structure.

10. According to the method of claim 7, the verification structure is further used for data indexing; Querying whether the hash value of the target tuple indicated by the query instruction is recorded in the log corresponding to the data table includes: According to the index value of the target tuple carried in the query instruction, querying whether the storage address and hash value of the target tuple are recorded in the log corresponding to the data table; The method further includes: If so, reading the target tuple from the storage address of the target tuple recorded in the log, and returning the target tuple and the hash value of the target tuple recorded in the log to the second client; If not, querying the tree to determine the leaf node corresponding to the target tuple; reading the target tuple from the storage address recorded in the leaf node corresponding to the target tuple, and returning the target tuple and the proof path of the leaf node corresponding to the target tuple in the tree to the second client.

11. A data verification method based on a verification structure, applied to a second client, where the verification certificate held by the second client is obtained based on the verification structure update method according to any one of claims 1 to 6, and the verification certificate includes a root hash value corresponding to a data table in the database and a log, and the log records the hash values of at least one tuple in the data table; The method includes: Sending a query instruction for the data table to the database; If the database returns the hash value of the target tuple indicated by the query instruction, performing data verification on the target tuple based on the hash value of the target tuple and the log stored locally; If the database returns the proof path of the target tuple indicated by the query instruction, performing data verification based on the proof path and the root hash value stored locally.

12. An electronic device, including: A processor; A memory for storing processor-executable instructions; Wherein, the processor realizes the method according to any one of claims 1 to 11 by running the executable instructions.

13. A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 11 are realized.

14. A computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are realized.

Images