Federated Coordinator for Consistent Database Snapshots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in taking application consistent snapshots of sharded relational databases that span across multiple storage clusters, as existing backup utilities lack the ability to ensure application consistency across disparate clusters, leading to potential data loss in mission-critical systems.
Innovation Solution
A distributed and federated protection domains model is implemented, using a federated consistency coordination master coordinator to coordinate simultaneous storage array side snapshots across multiple storage clusters, ensuring application consistent snapshots by freezing and thawing data shards in a coordinated manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If database sharding is used to scale data storage across multiple clusters, then data capacity and IO performance are improved, but the ability to perform application consistent snapshots across clusters deteriorates
Solution Approach 1:
The patent introduces a federated consistency coordination master coordinator as an intermediary component that spans multiple storage clusters. This coordinator receives snapshot requests, coordinates with local coordinators across clusters, manages freeze/thaw operations, and ensures application consistency. The intermediary resolves the contradiction by enabling cross-cluster snapshot coordination without requiring changes to the underlying sharding architecture.
2Adaptability or versatility
If data is distributed across multiple storage clusters, then storage scalability is improved, but coordination of simultaneous snapshots across clusters becomes more difficult
Solution Approach 1:
The patent segments the snapshot coordination function into hierarchical levels: a federated master coordinator that handles cross-cluster coordination and local coordinators that handle cluster-specific operations. This segmentation allows each component to focus on a manageable scope, reducing overall system complexity while maintaining scalability across multiple clusters.
Solution Approach 2:
The federated master coordinator acts as an intermediary that abstracts the complexity of cross-cluster coordination from individual clusters. It manages the freeze/thaw sequencing, timing synchronization, and failure recovery across clusters, allowing local clusters to operate independently while maintaining global consistency.
3Measurement precision
If clock synchronization is used across clusters, then timing coordination is improved, but nanosecond-level miscoordination can still cause critical transaction loss
Solution Approach 1:
The patent implements preliminary freeze actions across all clusters before the actual snapshot operation. The federated coordinator sends freeze requests to all local coordinators, waits for confirmation that all data shards are frozen, and only then proceeds with the snapshot. This preliminary action ensures that no transactions are lost during the snapshot process, regardless of clock synchronization precision.
Solution Approach 2:
The system implements feedback mechanisms where local coordinators report back to the federated coordinator about the status of freeze operations and snapshot completion. This feedback loop allows the coordinator to verify that all clusters are ready before proceeding, ensuring transaction consistency even when clock synchronization is not perfect.
Data Source
AI summary
A method and product for implementing application consistent snapshots of a sharded relational database across multiple storage arrays using a distributed and federated protection domains model across two or more storage clusters.