Hash-Based Data Recovery in Content-Addressable Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Content addressable storage systems face challenges in recovering data pages with errors, particularly when synchronous replication is absent, as errors in data pages or metadata can make it difficult to restore data efficiently.
Innovation Solution
The implementation of hash-based data recovery techniques, where a processing device detects errors and sends a hash handle to additional storage systems to retrieve a recovery data page with a matching hash handle, verifying its accuracy through hash digest computation and address comparison.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication is implemented to enable data recovery, then data reliability is improved, but device complexity and storage overhead increase
Solution Approach 1:
The patent extracts only the essential recovery information (hash handle) from the full data page and stores it in metadata. This allows the system to recover data without maintaining full synchronous replicas, reducing complexity while preserving recovery capability. The hash handle serves as a compact identifier that can be used to locate and verify recovery data pages in remote storage systems.
Solution Approach 2:
The system performs preliminary hashing of data pages to generate hash handles that are stored in metadata before any errors occur. When errors are detected, these pre-computed hash handles enable rapid identification and recovery of data pages from remote storage systems without requiring complex real-time replication protocols.
2Measurement precision
If full hash digest is stored and transmitted for data verification, then measurement precision is improved, but loss of time and communication overhead increase
Solution Approach 1:
The patent extracts a shortened hash handle from the full hash digest and uses this compact representation for metadata storage and communication. This extraction maintains sufficient verification precision while dramatically reducing the time and bandwidth required for transmitting verification data between storage systems.
3Device complexity
If synchronous replication is avoided to reduce complexity, then device complexity is reduced, but data recovery capability deteriorates
Solution Approach 1:
The patent introduces hash handles as an intermediary mechanism that enables data recovery without requiring full synchronous replication. The hash handle serves as a mediator that allows the primary storage system to locate and verify data pages in remote storage systems using content-addressable storage principles, thus maintaining recovery capability while avoiding the complexity of synchronous replication protocols.
4Quantity of substance
If content addressable storage with deduplication is implemented, then storage efficiency is improved, but vulnerability to hash collisions and metadata errors increases
Solution Approach 1:
The patent uses a shortened hash handle (partial hash) rather than the full hash digest for metadata storage and comparison. This partial action approach maintains deduplication efficiency while the system performs additional verification steps when recovering data, balancing storage efficiency with data integrity requirements.
Data Source
AI summary
An apparatus in one embodiment comprises at least one processing device comprising a processor coupled to a memory. The processing device is configured to detect an error relating to storage of a given data page in a first storage system, and to send a message to at least one additional storage system, the message containing a hash handle of the given data page with the hash handle having a length that is shorter than that of a corresponding hash digest of the given data page. The processing device is further configured to receive from the additional storage system responsive to the message a recovery data page identified by the additional storage system as having the same hash handle as the given data page. Separate instances of the message containing the hash handle of the given data page may be sent to each of a plurality of additional storage systems.