Storage Manager Coordinates Snapshot Creation in Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack an efficient method for creating and managing snapshots of databases in distributed storage and computation systems, making it difficult to revert to previous states without risking critical data corruption during software or configuration changes.
Innovation Solution
A method involving a storage manager that coordinates snapshot creation by quiescing compute nodes, creating snapshots of storage volumes, and managing storage nodes to capture the state of data, allowing for efficient rollback and cloning of snapshots, while maintaining data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional snapshot methods are used in distributed systems, then data can be captured for rollback, but the process is complex and risks data corruption during software or configuration changes
Solution Approach 1:
The patent introduces a storage manager as an intermediary component that coordinates between compute nodes and storage nodes. The storage manager receives snapshot requests, directs compute nodes to quiesce (pause) their operations, and coordinates with storage nodes to create consistent snapshots of storage volumes. This intermediary layer simplifies the complexity for users while ensuring data integrity through coordinated quiescing and snapshot creation.
Solution Approach 2:
The system performs preliminary quiescing of compute nodes before creating snapshots. By pausing compute node operations in advance, the system ensures that no write operations are in progress when the snapshot is taken, preventing data corruption. This preliminary action guarantees consistency without requiring complex post-snapshot validation.
2Reliability
If compute nodes continue operating during snapshot creation, then system availability is maintained, but data consistency cannot be guaranteed
Solution Approach 1:
The system dynamically adjusts compute node operation states based on snapshot requirements. During snapshot creation, compute nodes are temporarily quiesced (paused) to ensure consistency, then resumed after the snapshot is complete. This dynamic state change ensures data consistency during the critical snapshot window while maintaining high availability overall, as the quiescing is temporary and coordinated.
3Ease of operation
If manual snapshot coordination is used, then fine-grained control is achieved, but the process is time-consuming and error-prone
Solution Approach 1:
The storage manager implements automated self-service coordination for snapshot creation. When a snapshot request is received, the storage manager automatically directs compute nodes to quiesce, coordinates with storage nodes to create snapshots, and manages the entire process without manual intervention. This automation eliminates human errors and reduces the time required for snapshot coordination while maintaining fine-grained control over the process.
Data Source
AI summary
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a bundled application that is provisioned with virtualized storage and computation resources. A snapshot of the application may be created and used to rollback or clone the application. The amount of processing cores, memory, and containers of the bundled application may be increased or decreased based on usage. Components of the bundled application may be assigned to nodes to satisfy affinity and anti-affinity rules.