Targetless Snapshot Metadata Compression for Cold Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-capacity data storage systems face resource utilization challenges in maintaining frequent snapshots, as standard snapshot creation requires writing data to snap volumes, consuming memory, storage space, and network bandwidth, whereas targetless snapshots aim to reduce resource usage by referencing original data through metadata without writing to snap volumes.
Innovation Solution
Generating targetless snapshots that represent different points in time using replication data structures with metadata mapping original data, characterizing snapshots by access likelihood, and compressing metadata for resource-efficient storage and access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If standard snapshots are created by writing data to a snap volume, then snapshots can be easily accessed by mounting like other storage devices, but this requires generation of volume metadata and use of memory, non-volatile storage space, and network bandwidth
Solution Approach 1:
The patent extracts the essential snapshot functionality from the traditional volume-based approach. Instead of creating full snap volumes with complete metadata structures, the invention extracts only the necessary metadata elements (pointers, mapping information) to create lightweight targetless snapshots. This extraction eliminates the need for extensive memory allocation, large non-volatile storage spaces, and significant network bandwidth while preserving snapshot accessibility through alternative access methods.
Solution Approach 2:
The patent uses metadata copying rather than full data copying. Instead of writing actual snapshot data to snap volumes, the system creates metadata copies (pointers and mapping structures) that reference the original data in situ. This metadata copying approach dramatically reduces resource consumption while maintaining the ability to access snapshot data, as the metadata is much smaller than the actual data being referenced.
2Quantity of substance
If targetless snapshots are created by generating metadata that references original data in situ, then fewer resources are required to create and maintain snapshots, but metadata must be efficiently stored and accessed
Solution Approach 1:
The patent applies parameter changes by compressing metadata storage. The system transforms metadata from an uncompressed state to a compressed state, changing the storage parameters to reduce space requirements. This compression approach allows efficient storage of metadata on cold storage volumes while maintaining the ability to access and decompress metadata when needed, thereby reducing resource consumption without excessive complexity.
Solution Approach 2:
The patent implements dynamic metadata management where metadata is compressed for storage but can be dynamically decompressed and accessed when needed. The system transitions metadata between compressed and uncompressed states based on access requirements, creating a dynamic storage solution that balances resource efficiency with accessibility. This dynamic approach manages complexity by only maintaining full metadata structures temporarily during access operations.
3Reliability
If frequent snapshots are generated to represent changes since the most recent snapshot, then data loss is avoided and storage object state can be restored to earlier points in time, but this consumes memory, storage space, and network bandwidth
Solution Approach 1:
The patent extracts the essential snapshot functionality from the traditional volume-based approach. Instead of creating full snap volumes with complete metadata structures, the invention extracts only the necessary metadata elements (pointers, mapping information) to create lightweight targetless snapshots. This extraction eliminates the need for extensive memory allocation, large non-volatile storage spaces, and significant network bandwidth while preserving snapshot accessibility through alternative access methods.
Solution Approach 2:
The patent uses metadata copying rather than full data copying. Instead of writing actual snapshot data to snap volumes, the system creates metadata copies (pointers and mapping structures) that reference the original data in situ. This metadata copying approach dramatically reduces resource consumption while maintaining the ability to access snapshot data, as the metadata is much smaller than the actual data being referenced.
Data Source
AI summary
Targetless snapshots of a storage object are characterized in terms of likelihood of access using time-series analysis. Metadata of replication data structures of individual targetless snapshots is maintained in either uncompressed or compressed form based on the characterization of the targetless snapshot. Metadata is compressed at the page level, with same-pages of all replication data structures of cold snapshots of a storage object being compressed together. Compressed pages of targetless snapshot metadata are maintained of storage devices selected based on storage device performance and the time-series characterization of the targetless snapshot.