Hybrid Storage System for Data Loss Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Local storage systems face data loss during outages due to volatile storage and lack of snapshot capabilities, while cloud-based systems offer data portability but with slower access and higher costs.
Innovation Solution
A hybrid storage system combining a local transactional storage system with a cloud-based snapshot storage system, where a storage manager synchronizes data changes to the cloud, captures incremental snapshots, and uses transaction logs to restore the local system to a previous state, ensuring data integrity and portability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in a local storage system with volatile storage, then high-speed data access and low cost are achieved, but data loss occurs during outages or power failures
Solution Approach 1:
The storage system is segmented into a local volatile storage component for high-speed access and a cloud-based persistent storage component for data safety. The hybrid storage manager coordinates between these segments, allowing the local storage to operate at high speed while the cloud storage provides a safety net against data loss.
Solution Approach 2:
The system performs preliminary actions by continuously synchronizing data from local storage to cloud storage before failures occur. Incremental snapshots are captured and stored in the cloud, ensuring that a recent state of the data is already preserved and ready for recovery if needed.
2Reliability
If data is stored in a cloud-based storage system, then data portability and recovery capability are improved, but data access speed and cost efficiency deteriorate
Solution Approach 1:
The system separates data storage into two segments: local volatile storage for frequently accessed data requiring high speed, and cloud persistent storage for backup and portability. This segmentation allows each component to optimize for its specific function without compromising overall system performance.
Solution Approach 2:
The hybrid storage manager acts as an intermediary between the local storage system and the cloud storage system. It manages data synchronization, snapshot creation, and recovery operations, allowing the local storage to operate independently at high speed while maintaining connection to cloud storage for portability and recovery.
3Adaptability or versatility
If incremental snapshots are captured and stored in cloud storage, then data portability is improved, but storage cost and access time increase
Solution Approach 1:
Instead of continuously storing complete snapshots, the system uses incremental snapshots that only store changes since the previous snapshot. This partial action approach reduces the total amount of data stored in the cloud, lowering storage costs while still providing adequate portability and recovery capability.
Solution Approach 2:
The system discards redundant data by storing only incremental changes rather than complete snapshots repeatedly. Older snapshots can be discarded once their data is incorporated into newer snapshots, reducing storage requirements while maintaining the ability to recover data when needed.
Data Source
AI summary
Systems, methods, and computer readable media for preventing data loss at ephemeral and/or volatile storage of a local storage system are provided. These techniques may include synchronizing that state of the ephemeral storage system to a cloud-based storage system and capturing a cloud snapshot of the cloud-based storage system. In the event of a failure at the volatile storage, the cloud-based snapshot can be used as a restore point for the cloud-based storage system, the state of which can then be synchronized back to the volatile storage. Additionally, the local storage system includes durable storage for storing transaction logs. After synchronizing the state of the cloud-based storage system to the volatile storage, the local storage system can playback transactions in the transaction log to restore the volatile segment to the state at the time of the failure.