Virtual Storage Journal Flushing for Ephemeral Storage Data Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual storage systems in cloud environments lack non-volatile memory or persistent storage with NVRAM-like performance, leading to data loss during shutdown or reboot scenarios due to the unreliability of ephemeral storage.
Innovation Solution
A virtual storage system maintains an operation log in vNVRAM and flushes it to both ephemeral and persistent storage during shutdown to minimize data loss, ensuring data durability and low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If ephemeral storage is used as backing storage for journaling, then write performance and latency are improved, but data durability during shutdown or reboot is worsened
Solution Approach 1:
The system segments the backing storage into two distinct components: ephemeral storage for high-speed journaling operations and persistent storage for durability. This segmentation allows each component to serve its specific function optimally without compromising the other.
Solution Approach 2:
The system introduces a mediator mechanism that monitors the ephemeral storage and automatically flushes journal entries to persistent storage when shutdown or reboot is detected. This intermediary protection layer prevents data loss while maintaining the performance benefits of ephemeral storage.
2Reliability
If persistent storage is used as backing storage for journaling, then data durability is improved, but write performance and latency are worsened
Solution Approach 1:
The system divides the storage function into two segments: ephemeral storage handles high-speed write operations, while persistent storage ensures durability. Each segment performs its specialized function without interfering with the other's performance characteristics.
3Loss of time
If vNVRAM is used for operation log, then low latency is achieved, but data loss occurs during rehosting scenarios
Solution Approach 1:
The system performs preliminary action by flushing the operation log from vNVRAM to persistent storage before shutdown or reboot occurs. This advance preparation ensures that critical data is preserved even if rehosting happens, preventing information loss while maintaining low latency during normal operation.
Data Source
AI summary
Systems and methods for flushing an operation log journal to both ephemeral storage and persistent storage during a shutdown sequence of a virtual storage system to minimize data-loss scenarios are provided. According to one embodiment, the shutdown or reboot scenarios that result in loss of data are minimized by using persistent storage as a backup to ephemeral storage when the scenario results in rehosting of virtual storage system. For example, responsive to an event indicative of an imminent shutdown or reboot of the virtual storage system, vNVRAM memory may be flushed to both ephemeral storage and persistent storage (e.g., a boot disk). In this manner, when the virtual storage system is rehosted after an unplanned shutdown or reboot resulting from an unrecoverable host error (other than an unrecoverable hardware failure), the operation log journal may be recovered from persistent storage to facilitate vNVRAM replay and avoid data loss.