Synthesizing Hydrated Cloud Snapshots for Data Utility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based data storage systems primarily utilize proprietary formats for backup data, limiting the utility of stored objects to only restore operations, and fail to efficiently support additional services like disaster recovery and test/development workloads.
Innovation Solution
The method involves synthesizing fully hydrated cloud snapshots from local array snapshots by converting and hydrating raw disk image files into block devices, using incremental changes (snapdiffs) to update cloud snapshots, allowing for efficient generation and use of cloud snapshots suitable for disaster recovery, testing, and development.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in proprietary cloud storage format, then storage cost is reduced and data can be offloaded to cloud, but the data loses utility and can only be used for restore operations
Solution Approach 1:
The patent segments cloud snapshots into two types: archive snapshots for backup purposes and synthesized hydrated snapshots for operational use. This segmentation allows data to serve multiple purposes - cost-effective archival storage and versatile operational workloads - by separating the storage function from the access function.
Solution Approach 2:
The patent introduces an intermediary process that synthesizes hydrated snapshots from archive snapshots when needed. This intermediary mechanism allows the system to maintain data in a compact proprietary format for storage while providing fully hydrated data formats for operational use, bridging the gap between storage efficiency and data utility.
2Reliability
If full snapshots are transferred to cloud storage, then data completeness is ensured, but transfer time and bandwidth consumption increase significantly
Solution Approach 1:
The patent extracts only the necessary metadata and change information from full snapshots for cloud transfer, rather than transferring complete snapshot data. The synthesis process extracts the essential information needed to reconstruct hydrated snapshots on-demand, significantly reducing transfer time while maintaining data completeness.
Solution Approach 2:
The system performs preliminary actions by maintaining local archive snapshots in hydrated format and pre-processing change information (snapdiffs). This preliminary preparation enables rapid synthesis of hydrated snapshots in the cloud without requiring time-consuming data transfers, as the essential data remains locally available.
3Ease of operation
If cloud snapshots are kept in hydrated format, then immediate use for various workloads is enabled, but storage efficiency and cost increase
Solution Approach 1:
The patent applies local quality by maintaining hydrated format only where needed (local archive snapshots and synthesized cloud snapshots for active use) while keeping cloud archive snapshots in compact proprietary format. This localized application of hydration ensures operational readiness for active workloads while maintaining storage efficiency for archival purposes.
Solution Approach 2:
The system dynamically changes the hydration parameter of snapshots based on their usage state. Archive snapshots in the cloud remain in a dehydrated compact format, while synthesized snapshots for active use are converted to hydrated format. This parameter change allows the same data to occupy different storage states based on operational requirements.
4Loss of energy
If incremental changes (snapdiffs) are used to update cloud snapshots, then bandwidth consumption is reduced, but the synthesis process becomes more complex
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing incremental change information (snapdiffs) locally. This preliminary preparation of change data simplifies the cloud synthesis process, as the cloud只需要 needs to apply these pre-computed diffs to base snapshots rather than managing complex incremental updates from scratch.
Data Source
AI summary
Systems and methods for synthesizing cloud snapshots of production data. A client system includes data and snapshots may be taken of that data. These snapshots are transferred to the cloud and hydrated to block devices that are similar to the devices on which the production data exists. Snapshots are then taken of the block devices. The cloud snapshots are the same as and correspond to the locally generated snapshots.