Cloud Snapshot Archiving via Chunk-Based Synthesized Full Backups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems face inefficiencies in archiving and retrieving snapshots due to the inability to partially update or read objects, leading to high costs and resource usage when performing periodic archival of snapshots.
Innovation Solution
A method that divides snapshot address spaces into chunks, organizes them into families with full and incremental snapshots, and creates synthesized full backups without requiring the entire storage object to be transferred, using chunk-based archiving and metadata optimization to promote efficient use of cloud resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full snapshots are archived to cloud storage, then data reliability is improved, but storage costs and bandwidth usage increase
Solution Approach 1:
The patent segments snapshots into chunksets, where each chunkset represents a portion of the snapshot data. Instead of treating entire snapshots as indivisible units, the system divides them into manageable chunks that can be independently stored, tracked, and retrieved. This segmentation enables incremental archiving where only changed chunksets are transferred to cloud storage, reducing overall storage costs while maintaining data reliability through versioned chunkset management.
Solution Approach 2:
The patent implements a lifecycle management approach where archived chunksets can be discarded from cloud storage after a retention period while maintaining the ability to recover them when needed. The system tracks chunkset versions and allows recovery of specific chunksets without requiring all archived data to be permanently retained, optimizing storage costs while preserving data reliability for the required retention period.
2Reliability
If full snapshots are transferred to cloud storage, then data completeness is improved, but bandwidth usage and transfer time increase
Solution Approach 1:
The patent extracts only the necessary chunksets from source storage for archiving to cloud storage, rather than transferring complete snapshots. The system identifies which chunksets have changed since the last archive and transfers only those specific chunks, significantly reducing transfer time and bandwidth usage while ensuring data completeness for the archived version.
Solution Approach 2:
The system performs preliminary identification of changed chunksets before initiating the archiving transfer. By pre-processing to determine which chunksets require archiving based on change detection, the system avoids unnecessary data transfer and reduces overall archiving time while maintaining complete data representation for the snapshot version being archived.
3Loss of energy
If incremental snapshots are used, then bandwidth usage is reduced, but retrieval complexity increases
Solution Approach 1:
The patent creates synthetic full snapshots by copying and assembling the latest chunksets from incremental archives. When a full snapshot is needed for retrieval, the system automatically copies the necessary chunksets from the incremental archive and assembles them into a complete snapshot representation, simplifying the retrieval process while maintaining the bandwidth efficiency of incremental archiving.
Solution Approach 2:
The system merges incremental chunkset updates with the base snapshot to reconstruct full snapshots when needed. By combining the base snapshot data with incremental changes stored in the archive, the system retrieves complete snapshot information without transferring all data, reducing bandwidth usage while managing retrieval complexity through automated merging operations.
4Quantity of substance
If cloud objects are deleted during aging out, then storage costs are reduced, but data integrity may be compromised
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors which cloud objects are still referenced by active or retained snapshots before deleting them during aging out. The aging-out process checks reference counts and retention policies, only deleting objects that are no longer needed, thereby reducing storage costs while preventing data integrity issues from premature deletion.
Solution Approach 2:
The system performs preliminary verification of object references and retention status before executing deletion during the aging-out process. By pre-checking whether cloud objects are still required by any snapshots that must be retained, the system safely deletes only those objects that can be removed without compromising data integrity, optimizing storage costs while maintaining reliability.
Data Source
AI summary
A data processing system archives local snapshots of a primary storage object to cloud storage by dividing an address space of the snapshots into sequential chunks, and organizing the cloud storage into archived snapshots including sets of cloud objects created from respective modified chunks of local snapshots. The archived snapshots are organized into families each having a full snapshot and incremental snapshots. A new family is started by creating a synthesized full snapshot from the archived snapshots of a preceding family, which includes (1) for chunks whose data is contained within one cloud object of the preceding family, logically incorporating the cloud object into the synthesized full snapshot, and (2) for chunks whose data is contained within multiple cloud objects of the preceding family, coalescing the data of the cloud objects into a new cloud object and logically incorporating the new cloud object into the synthesized full snapshot.