Storage Manager Snapshot Coordination for Distributed Data Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack an efficient method for creating snapshots of distributed databases and applications, making it difficult to revert to previous states without risking data corruption during testing or configuration changes.
Innovation Solution
A network environment with a storage manager coordinating snapshot creation across compute and storage nodes, involving quiesce instructions, snapshot creation, garbage collection, and data management to ensure data integrity and facilitate rolling back to previous states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If snapshots are created to enable safe testing and configuration changes, then data integrity is improved, but system complexity increases due to coordination requirements across distributed nodes
Solution Approach 1:
The patent introduces a storage manager as an intermediary component that coordinates snapshot creation across multiple storage nodes. The storage manager receives snapshot requests, manages the creation process, and ensures proper coordination, thereby isolating the complexity from the client applications and simplifying the overall system interface while maintaining data integrity across the distributed system
Solution Approach 2:
The patent implements a quiesce mechanism that pauses data writes to storage volumes before creating snapshots. This preliminary action ensures that the snapshot captures a consistent state of the data by preventing new writes during the snapshot creation process, thereby guaranteeing data integrity without requiring complex post-snapshot validation mechanisms
2Reliability
If distributed application state is frozen for snapshot creation, then data consistency is improved, but operation time increases due to quiesce instructions
Solution Approach 1:
The patent applies quiesce instructions selectively only to the specific storage volumes that need to be snapshotted, rather than freezing the entire distributed application state. This partial action approach minimizes the impact on application operations while still ensuring data consistency for the targeted volumes, reducing the overall operation time compared to a full system freeze
Solution Approach 2:
The patent divides the snapshot creation process into independent operations on individual storage volumes, each coordinated by the storage manager. This segmentation allows different volumes to be snapshotted independently and potentially in parallel, reducing the total operation time while maintaining data consistency for each volume through targeted quiesce instructions
Data Source
AI summary
A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. An orchestration layer implements a bundled application that is provisioned with virtualized storage and computation resources. A snapshot of the bundled application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot. Chaos testing of the bundled application may be performed and storage volumes may be created, expanded, and/or moved based on usage of the bundled application.