Merkle Tree Data Integrity Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Establishing trust in the integrity of data stored in database systems is challenging due to the ability of high-privileged users to tamper with data and erase traces of their actions, especially as data moves to the cloud, expanding the trust boundary.
Innovation Solution
A compute and storage-efficient technique for generating a tree-based data structure, such as a Merkle tree, where each data item's update creates a leaf node with a hash value, and parent nodes store hash values of child nodes, with the root node representing the entire dataset, allowing for efficient verification and remediation if modifications occur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional data storage methods are used in database systems, then data can be stored and accessed, but data integrity cannot be verified and high-privileged users can tamper with data without traces
Solution Approach 1:
The patent segments the data set into individual data items, each represented by a leaf node in a tree data structure. Each leaf node contains a hash of a data item, and parent nodes contain hashes derived from their children. This segmentation allows verification of individual data items without requiring trust in the entire system, resolving the contradiction by enabling integrity verification while maintaining manageable system complexity.
Solution Approach 2:
The patent introduces hash values as intermediary elements between the raw data and the verification mechanism. The root hash value acts as a mediator that represents the entire data set's integrity state. This intermediary approach enables verification of data integrity without requiring direct trust in storage systems or high-privileged users, as any tampering would change the root hash value.
2Quantity of substance
If cloud storage is used to expand data capacity, then storage flexibility increases, but the trust boundary expands and data integrity verification becomes more difficult
Solution Approach 1:
The patent implements a feedback mechanism where the root hash value is continuously updated as data items are added or modified. This feedback loop allows the system to detect any changes in the data set by comparing the current root hash with previous values. The feedback mechanism works independently of the storage location, enabling integrity verification in cloud storage environments without expanding the trust boundary.
3Measurement precision
If complete data sets are stored for verification purposes, then verification accuracy is maintained, but storage requirements increase significantly
Solution Approach 1:
The patent extracts only the essential verification information (hash values) from the complete data set. Instead of storing entire data items for verification, the system stores only hash values in the tree structure. The root hash value provides complete verification capability for the entire data set while occupying minimal storage space, thus maintaining verification accuracy while dramatically reducing storage requirements.
Data Source
AI summary
Embodiments described herein are directed to generating a tree-based data structure representative of a data set and the verification thereof. As each data item of a data set is updated, a leaf node is generated that stores a hash value therefor. For every even leaf node generated, a parent node storing a hash value based on the hash values of its child nodes is generated. For each level of the tree, the hash value of the last odd node generated therefor is maintained. The foregoing process is performed recursively at each level of the tree. During verification, a new root hash value is determined for a new tree-based data structure generated for the data set to be verified. The old and new root hash values are compared. If the hash values do not match, a remediation is performed to restore the data set.