Cloud Snapshot Archiving via Chunk-Object Mapping and Synthetic Full Backups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud storage systems face inefficiencies in archiving and retrieving snapshots due to the need for full object copies, which are costly in terms of bandwidth and storage, and cannot handle partial updates or reads, making periodic archival of snapshots challenging.
Innovation Solution
The technique involves dividing snapshot address spaces into chunks, organizing them into families with full and incremental snapshots, and creating synthesized full backups by logically incorporating cloud objects, reducing the need for network transfers and optimizing resource use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full object copies are used for cloud archiving, then data integrity and completeness are ensured, but bandwidth consumption and storage costs increase significantly
Solution Approach 1:
The patent divides storage objects into fixed-size chunks (e.g., 5 MB each) and stores only the modified chunks in cloud storage. This segmentation allows incremental backups where only changed portions are transferred, dramatically reducing bandwidth consumption while maintaining data integrity through chunk-level tracking and synthesis capabilities.
Solution Approach 2:
The system performs preliminary synthesis of full object copies locally by combining base objects with incremental chunk updates before cloud transfer. This preliminary action ensures data integrity is maintained locally while only essential incremental changes are uploaded to the cloud, reducing bandwidth consumption.
2Reliability
If full object copies are stored in cloud, then complete data availability is achieved, but cloud storage costs and upload times increase
Solution Approach 1:
By segmenting objects into chunks and storing only modified chunks in the cloud, the system maintains complete data availability through local synthesis capabilities while minimizing cloud storage usage to only the essential incremental changes.
Solution Approach 2:
The system creates synthetic full copies locally by combining base objects with incremental chunk updates, avoiding the need to store complete object copies in the cloud. This copying approach ensures data availability while reducing cloud storage requirements to minimal chunk-level increments.
3Ease of manufacture
If cloud storage uses fixed-size objects, then storage management is simplified, but partial updates and reads become impossible
Solution Approach 1:
The patent segments storage objects into fixed-size chunks with unique identifiers, enabling partial updates by allowing individual chunk modifications while maintaining simplified cloud storage management through the fixed-size chunk structure. This segmentation provides both ease of management and adaptability for partial updates.
Solution Approach 2:
The system introduces a chunk management layer as an intermediary between the application and cloud storage. This intermediary handles chunk-level operations, tracking, and synthesis, enabling partial updates and reads while maintaining simplified fixed-size object storage in the cloud.
4Loss of energy
If incremental snapshots are used, then bandwidth efficiency improves, but retrieval complexity increases
Solution Approach 1:
By segmenting snapshots into manageable chunks with systematic naming and tracking, the system enables efficient incremental backups while simplifying retrieval through organized chunk assembly processes. The chunk-level organization reduces retrieval complexity compared to traditional incremental snapshot approaches.
Solution Approach 2:
The system uses synthetic full copies created by combining base objects with incremental chunks, providing simple full data retrieval without requiring complex incremental snapshot assembly. This copying approach maintains bandwidth efficiency of incremental backups while simplifying the retrieval process.
Data Source
AI summary
A data processing system archives local snapshots of a primary storage object to cloud storage by dividing an address space of the snapshots into sequential chunks, and organizing the cloud storage into archived snapshots including sets of cloud objects created from respective modified chunks of local snapshots. The archived snapshots are organized into families each having a full snapshot and incremental snapshots. A new family is started by creating a synthesized full snapshot from the archived snapshots of a preceding family, which includes (1) for chunks whose data is contained within one cloud object of the preceding family, logically incorporating the cloud object into the synthesized full snapshot, and (2) for chunks whose data is contained within multiple cloud objects of the preceding family, coalescing the data of the cloud objects into a new cloud object and logically incorporating the new cloud object into the synthesized full snapshot.