Ephemeral Storage Snapshotting via Hybrid Cloud Sync
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional local storage systems face data loss during outages and lack data portability, while cloud-based systems offer portability but at higher costs and slower access speeds.
Innovation Solution
A hybrid storage system that combines a local ephemeral storage system with a cloud-based storage system, using a storage manager to synchronize data changes and generate snapshots, enabling data portability and recovery while maintaining low costs and high access speeds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is stored in a cloud-based storage system, then data portability and recovery capabilities are improved, but access speed and cost efficiency deteriorate
Solution Approach 1:
The storage system is segmented into two parts: local ephemeral storage for high-speed data access and cloud-based storage for data portability and recovery. The system divides data management functions between local storage manager and cloud storage manager, allowing simultaneous optimization of both speed and portability by operating on different storage layers
Solution Approach 2:
A storage manager acts as an intermediary between the local ephemeral storage system and the cloud-based storage system. It coordinates data synchronization, manages snapshots, and handles data portability operations, enabling the system to achieve both fast local access and cloud-based portability without direct user interaction with either system
2Adaptability or versatility
If data is stored in a cloud-based storage system, then data portability is improved, but cost efficiency deteriorates
Solution Approach 1:
The storage architecture segments data between cost-effective local ephemeral storage for primary operations and cloud storage only when portability or backup is needed, significantly reducing overall storage costs while maintaining data portability capabilities through selective cloud synchronization
Solution Approach 2:
The system dynamically changes the storage parameter (local vs. cloud) based on operational needs. Data resides locally by default for cost efficiency, and the system transitions to cloud storage only when portability or recovery operations are required, optimizing the cost-benefit ratio
3Reliability
If snapshots are generated frequently to ensure data recovery, then data safety is improved, but system performance and resource usage deteriorate
Solution Approach 1:
The system performs preliminary data synchronization to the cloud before potential failure events. By continuously maintaining updated cloud copies and snapshots in the background, the system ensures data safety is already established before any disaster occurs, eliminating the need for time-consuming snapshot operations during critical recovery scenarios
Solution Approach 2:
The storage manager implements periodic synchronization and snapshot creation at optimized intervals, balancing data safety requirements with system performance. This periodic approach ensures data is captured at regular intervals without continuously impacting system productivity
Data Source
AI summary
Systems, methods, and computer readable media for snapshotting ephemeral and/or volatile storage systems are provided. These techniques may include synchronizing that state of the ephemeral storage system to a cloud-based storage system and capturing a cloud snapshot of the cloud-based storage system. To reduce the time to capture a cloud snapshot, techniques disclosed herein include obtaining sets of pending data changes to the ephemeral storage system that have not been synchronized to the cloud-based storage system. When the pending data changes satisfy a threshold metric, the pending data changes may then be synchronized to the ephemeral storage system on a rolling basis.