Snapshot Quiesce Coordination for Data Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack an efficient method for creating and managing snapshots of databases in distributed storage and computation systems, making it difficult to revert to previous states without risking critical data corruption during software or configuration changes.
Innovation Solution
The implementation of a method that involves a storage manager coordinating snapshot creation across compute nodes and storage nodes, using quiesce instructions to suppress writes, and managing snapshots through segment maps and block maps to ensure data integrity and allow for seamless rollback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshots are created to allow rollback to previous states, then reliability is improved, but device complexity increases due to the need for coordination between storage manager, compute nodes, and storage nodes
Solution Approach 1:
The snapshot management process is divided into distinct phases: quiesce phase (suppressing writes), snapshot creation phase (creating snapshots at storage nodes), and rollback phase (restoring to previous snapshots). This segmentation allows each phase to be managed independently, reducing overall system complexity while maintaining reliability
Solution Approach 2:
The storage manager acts as an intermediary that coordinates between compute nodes and storage nodes. It receives quiesce instructions from compute nodes, manages snapshot creation at storage nodes, and orchestrates rollback operations. This intermediary role simplifies the interaction between multiple components and centralizes snapshot management logic
2Reliability
If quiesce instructions are used to suppress writes during snapshot creation, then data integrity is improved, but productivity decreases due to temporary write suppression
Solution Approach 1:
The quiesce instruction is sent to compute nodes before snapshot creation begins. This preliminary action ensures that no new writes are generated during the snapshot capture process, guaranteeing data integrity. The quiesce state is temporary and automatically lifted after snapshot creation, restoring normal productivity
Solution Approach 2:
The quiesce mechanism operates periodically - quiesce is activated during snapshot creation, then deactivated to allow normal write operations to resume. This periodic activation ensures data integrity during critical snapshot moments while maintaining high productivity during normal operation
3Adaptability or versatility
If snapshots are created for testing configuration changes, then adaptability is improved, but loss of time increases due to potential rollback operations
Solution Approach 1:
Multiple snapshots are created and stored in advance before configuration changes are applied. This allows rapid rollback to any previous known good state without waiting for snapshot creation. The preliminary creation of snapshots enables quick adaptation when issues arise
Solution Approach 2:
Snapshots create copies of the database state at different points in time. These copies can be rapidly restored without affecting the current operational database. The copying mechanism enables safe experimentation and quick rollback to previous states, improving adaptability while minimizing time loss
Data Source
AI summary
A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. An orchestration layer implements a bundled application that is provisioned with storage volumes and containers. A snapshot of the application may be created and used to rollback or clone the application. The amount of processing cores, memory, and containers may be increased or decreased based on usage. Components of the bundled application may be assigned to nodes to satisfy affinity and anti-affinity rules. Device sets may be generated according to these rules and used to plan for the assignment of storage volumes and containers of the bundled application.