Snapshot Availability via Writeback Log Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional snapshot systems in high-availability environments face delays due to network latency, as snapshot data is stored on a data node separate from the compute node, slowing down the availability of snapshots.
Innovation Solution
Enabling other nodes to request snapshot data directly from the writeback log on the compute node that originated the snapshot, allowing for instantaneous marking of snapshots as available and reducing dependency on data node transfer completion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshot data is stored on a separate data node, then data redundancy and security are improved, but network latency slows down snapshot availability
Solution Approach 1:
The patent segments snapshot data into two parts: hot data (recently written data) remains on the compute node in the writeback log, while cold data (older data) is stored on the separate data node. This segmentation allows read requests to be served from the compute node without network latency, while still maintaining the benefits of separate storage for data security and redundancy.
Solution Approach 2:
The patent performs preliminary action by pre-positioning snapshot data in the writeback log on the compute node before it is needed for rapid access. By keeping recently written data locally on the compute node, the system prepares the data in advance for potential snapshot operations, eliminating network latency when snapshots are created.
2Stability of the object's composition
If all snapshot data is transferred to the data node before marking as available, then data consistency is ensured, but snapshot availability is delayed
Solution Approach 1:
The patent applies partial action by marking snapshots as available incrementally rather than waiting for complete data transfer. Once a portion of data is written to the writeback log, the snapshot can be marked as partially available, allowing read operations to proceed with the available data while the rest continues to be transferred in the background.
Solution Approach 2:
The system performs preliminary action by pre-marking snapshots as available in the metadata as soon as data is written to the writeback log, even before complete transfer to the data node. This allows the snapshot to be immediately accessible while the remaining data transfer occurs asynchronously.
3Productivity
If snapshots are marked as available before complete data transfer, then snapshot availability is improved, but data completeness may be compromised
Solution Approach 1:
The patent implements feedback through metadata that tracks the status of snapshot data transfer. The metadata records which portions of data have been written to the writeback log and which are still being transferred, providing continuous feedback about data completeness. This allows the system to manage read requests appropriately based on the current state of data availability.
Solution Approach 2:
The patent segments snapshot data into portions that have been transferred to the writeback log and portions still being transferred to the data node. By tracking and managing these segments separately through metadata, the system can serve read requests from available segments while indicating which segments are still being populated.
Data Source
AI summary
The disclosed computer-implemented method for making snapshots available may include (i) identifying a writeback log that records input/output operations of a compute node within a high-availability environment, (ii) placing, in the writeback log, a marker that indicates a start of a snapshot to be stored on a data node, (iii) marking, after placing the marker and before all data within the snapshot has been transferred to the data node, the snapshot as available, (iv) receiving, from an additional compute node, a request to read from the snapshot, and (v) sending, from the compute node to the additional compute node, metadata indicating which portion of data from the snapshot is stored on the data node and which portion of the data from the snapshot is not stored on the data node but is stored in the writeback log. Various other methods, systems, and computer-readable media are also disclosed.