Signature Generator for Content Addressable Storage Data Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for comparing sets of data in content addressable storage systems are inefficient, particularly in distributed environments, which undermines the performance of data replication and migration processes.
Innovation Solution
Generating signatures for sets of data using content-based page identifiers and page addresses, and comparing these signatures to verify equivalence between data sets, thereby improving efficiency in data verification processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional methods are used to compare sets of data in distributed content addressable storage systems, then data verification can be performed, but the process is inefficient and undermines overall system performance
Solution Approach 1:
The patent divides the data verification process into segments by generating signatures for individual pages or blocks of data rather than comparing entire data sets. Each page signature is generated independently using a hash function applied to the page identifier and data, allowing parallel processing across multiple storage nodes in the distributed system.
Solution Approach 2:
The patent introduces signatures as intermediary representations of the actual data. Instead of directly comparing large volumes of replicated data, the system compares compact signature values that serve as proxies for the underlying data sets, dramatically reducing the computational burden of verification.
2Reliability
If complete data sets are compared for verification, then data integrity can be confirmed, but the computational complexity and time required increase significantly
Solution Approach 1:
The patent extracts essential verification information from the complete data sets by computing signatures based on page identifiers and selected data portions. This extraction process isolates the critical elements needed for verification while discarding redundant information, simplifying the comparison process without compromising integrity assurance.
Solution Approach 2:
The patent transforms the verification problem by changing the parameters being compared. Instead of comparing raw data bytes, the system computes and compares hash-based signature values derived from page identifiers and data content. This parameter transformation reduces the complexity of verification while maintaining reliability.
3Reliability
If data replication is performed across multiple storage nodes, then system reliability is improved, but the overhead of verifying replicated data increases
Solution Approach 1:
The patent segments the verification workload across multiple storage nodes by assigning different page signature generation tasks to different nodes. Each node independently generates signatures for its assigned pages, enabling parallel verification processing that scales with the number of storage nodes in the distributed system.
Solution Approach 2:
Each storage node in the distributed system performs self-service verification by generating its own page signatures and participating in the collective verification process. Nodes autonomously compute signatures for their stored pages without requiring centralized verification, reducing the overhead burden on any single node while maintaining overall system reliability.
Data Source
AI summary
An apparatus in one embodiment comprises a storage system having a plurality of storage devices configured to store at least a first set of data. The storage system is configured to generate a first signature for the first set of data and to verify equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data. The first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address, and the first and second signatures are generated as respective functions of the page identifiers and page addresses of the respective first and second sets of pages. The first and second sets of data may comprise respective first and second storage volumes, respective first and second sets of multiple storage volumes, or other sets of data.