Block Attribute Parity for Sparse Snapshot Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual storage area networks, the combination of features like snapshots, checksums, encryption, and deduplication complicates data integrity and compatibility, particularly with RAID systems, due to issues with distinguishing data from holes in sparse disks and managing encryption keys for deduplicated data.
Innovation Solution
The solution involves calculating block attribute parity to maintain hole boundary information, providing end-to-end checksum within a distributed virtual storage area network, and storing and verifying encryption keys to ensure data integrity and compatibility across multiple storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If snapshots are created as sparse disks to conserve resources, then storage efficiency is improved, but the ability to rebuild snapshots from parity is lost
Solution Approach 1:
The system performs preliminary actions by calculating and storing block attribute parity values alongside data parity values during the initial snapshot creation. This preliminary calculation of hole boundary information enables future reconstruction of sparse disk snapshots from parity without requiring the original snapshot data, thus resolving the contradiction between storage efficiency and rebuild capability
Solution Approach 2:
Block attribute parity values serve as an intermediary that bridges the gap between sparse disk representation and RAID parity reconstruction. These metadata values encode hole boundary information that allows the system to distinguish between actual zero data and holes during parity calculation, enabling snapshot rebuild from parity while maintaining sparse disk storage efficiency
2Reliability
If end-to-end checksums are implemented to detect data errors, then data integrity is improved, but I/O processing speed deteriorates
Solution Approach 1:
The system applies partial checksum verification by focusing checksum calculations on only the modified blocks within a stripe rather than the entire stripe. This partial action approach maintains data integrity verification for changed data while significantly reducing the computational overhead compared to full stripe checksumming, thus improving I/O processing speed while preserving data integrity
Solution Approach 2:
The system implements local quality by applying checksum verification selectively to specific blocks that have been modified, rather than uniformly processing all blocks. This localized approach concentrates verification efforts where data changes occur, maintaining high data integrity for modified blocks while minimizing the impact on overall I/O processing speed
3Loss of substance
If data-specific encryption keys are generated to enable deduplication of encrypted data, then storage efficiency is improved, but key management complexity increases
Solution Approach 1:
The system uses copying by storing references or pointers to encryption keys rather than duplicating the actual key material across multiple locations. Each encrypted block stores a reference to its encryption key, allowing deduplication of encrypted data while simplifying key management. The master key remains stored in a single secure location, and references enable efficient retrieval without complex key distribution
Solution Approach 2:
The encryption key management system achieves universality by using a single master encryption key that can encrypt multiple different data blocks. Data-specific encryption is achieved not through multiple keys but through deterministic encryption where the same key produces different ciphertexts for different plaintexts, or through key derivation functions that generate block-specific keys from a master key, reducing key management complexity while maintaining storage efficiency
4Reliability
If block attribute parity is calculated and stored to maintain hole boundary information, then snapshot rebuild capability from parity is improved, but storage space requirements increase
Solution Approach 1:
The system changes parameters by using compact bit-level representation for block attribute parity values instead of storing full block data. Each block attribute parity value requires only minimal bits to indicate hole boundaries, transforming the storage requirement from full block-sized data to compact metadata, thus enabling snapshot rebuild capability while minimizing additional storage space requirements
Data Source
AI summary
Exemplary methods, apparatuses, and systems maintain hole boundary information by calculating a block attribute parity value. For example, a request is received to write to a first block of a stripe of data. A block attribute of a second block is determined. The block attribute of the second block indicates whether the second block includes written data or is a hole. A block attribute parity value is calculated based upon both the block attribute of the first block and the block attribute of the second block. The block attribute of the first block indicates the first block includes written data based upon the received request. The block attribute parity value and the data parity value are stored on one of the physical storage devices in response to the received write request. As a result, if a disk is lost, holes can be recovered using the block attribute parity value.