Distributed Storage Snapshot Creation via Segment Finalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating snapshots of database storage volumes are inefficient in managing changes and rolling back to previous states, particularly in large-scale deployments with distributed data, as they lack effective mechanisms for quiescing write requests and managing snapshot hierarchies.
Innovation Solution
The solution involves a network environment with a storage manager that coordinates snapshot creation by quiescing compute nodes, creating snapshots by finalizing segments, and maintaining a snapshot hierarchy, allowing for efficient rollback and data retrieval across multiple storage nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional snapshot methods are used in distributed storage systems, then snapshot creation can be performed, but the process is inefficient and lacks effective mechanisms for quiescing write requests and managing snapshot hierarchies
Solution Approach 1:
The system divides the distributed storage volume into multiple segments distributed across different storage nodes. Each segment can be independently snapshot, allowing parallel snapshot creation across the distributed system without requiring centralized coordination of the entire volume, thus improving efficiency while managing complexity through modular segmentation.
Solution Approach 2:
The storage manager sends quiesce instructions to compute nodes before creating snapshots, temporarily suspending write requests to ensure data consistency. This preliminary action prevents write conflicts during snapshot creation, ensuring that snapshots capture a consistent state without requiring complex post-processing or validation mechanisms.
2Reliability
If snapshots are created without quiescing write requests, then continuous data access is maintained, but data integrity and consistency of the snapshot are compromised
Solution Approach 1:
The system performs a quiesce operation before snapshot creation, temporarily pausing write requests to ensure data consistency. This preliminary action guarantees that the snapshot captures a consistent state of the storage volume, preventing data corruption while minimizing disruption to data access operations.
Solution Approach 2:
The system implements periodic snapshot creation with controlled quiesce periods. By scheduling snapshots at appropriate intervals and using brief quiesce windows, the system maintains data integrity while minimizing the impact on continuous data access operations, allowing the system to resume normal write operations immediately after snapshot completion.
Data Source
AI summary
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots. A non-snapshot volume may be converted to a snapshot volume at any point after creation.