Cloud Snapshot Archiving via Chunk-Based Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems face inefficiencies in archiving and retrieving snapshots due to the inability to partially update or read objects, leading to high costs and resource usage when performing periodic archival of snapshots.
Innovation Solution
A method that divides storage objects into chunks, organizes them into families with full and incremental snapshots, and creates synthesized full snapshots by logically incorporating cloud objects, reducing the need for network transfers and optimizing metadata management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If cloud storage objects are used for snapshot archiving, then wide-area availability and internet connectivity are improved, but the inability to partially update or read objects increases storage costs and resource usage
Solution Approach 1:
The patent divides storage objects into fixed-size chunks (e.g., 5 MB each) that can be independently managed. This segmentation allows the system to transfer only the necessary chunks to cloud storage rather than entire objects, reducing bandwidth consumption and storage costs while maintaining the ability to access specific data portions through chunk-level operations.
Solution Approach 2:
The patent introduces a new dimensional approach by organizing data hierarchically into chunks, objects, and snapshot families. This multi-dimensional structure enables efficient retrieval and archiving operations by allowing the system to navigate through different levels of organization, selecting only the necessary data units for transfer or access.
2Reliability
If full snapshots are archived to cloud storage, then data integrity and recoverability are improved, but network bandwidth usage and transfer time increase significantly
Solution Approach 1:
The patent extracts only the changed or necessary chunks from full snapshots and stores them separately in the cloud. By identifying and isolating the specific data portions that need archiving (rather than transferring entire snapshots), the system maintains data integrity for critical information while dramatically reducing network transfer time and bandwidth consumption.
Solution Approach 2:
The patent applies partial action by archiving only the necessary portions of snapshots (changed chunks, incremental changes) rather than complete snapshots. This selective archiving approach provides sufficient data integrity for recovery purposes while avoiding the excessive time and resource costs of transferring entire snapshot datasets.
3Loss of energy
If incremental snapshots are used for archiving, then bandwidth usage is reduced, but the complexity of tracking and managing chunk changes across multiple snapshots increases
Solution Approach 1:
The patent uses copying mechanisms where chunk metadata and allocation information are replicated and tracked across snapshot families. By maintaining copies of chunk allocation maps and using reference counting, the system can track chunk changes across multiple incremental snapshots without requiring complex real-time analysis, thereby reducing management complexity while enabling efficient incremental archiving.
4Quantity of substance
If cloud storage is used for periodic archival, then storage capacity and accessibility are improved, but the cost per GB and API call charges increase operational expenses
Solution Approach 1:
The patent segments storage operations into chunk-level transactions rather than object-level operations. This segmentation allows the system to minimize the number of cloud API calls by batching chunk uploads and using efficient metadata management, thereby reducing API call charges.同时，通过只上传必要的chunk而非整个对象，降低了存储成本。
Solution Approach 2:
The patent changes the operational parameters of cloud storage usage by transitioning from object-based to chunk-based operations. This parameter change enables more efficient use of cloud storage resources, allowing the system to optimize transfer sizes, leverage compression at the chunk level, and implement smarter retention policies, all of which reduce operational expenses while maintaining adequate storage capacity.
Data Source
AI summary
A data processing system archives local snapshots of a primary storage object to cloud storage by dividing an address space of the snapshots into sequential chunks, and organizing the cloud storage into archived snapshots including sets of cloud objects created from respective modified chunks of local snapshots. The archived snapshots are organized into families each having a full snapshot and incremental snapshots. A new family is started by creating a synthesized full snapshot from the archived snapshots of a preceding family, which includes (1) for chunks whose data is contained within one cloud object of the preceding family, logically incorporating the cloud object into the synthesized full snapshot, and (2) for chunks whose data is contained within multiple cloud objects of the preceding family, coalescing the data of the cloud objects into a new cloud object and logically incorporating the new cloud object into the synthesized full snapshot.