Time-Synchronized Snapshot Aggregation for Distributed Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual computing systems face challenges in capturing data consistent snapshots across distributed services due to lack of synchronized time between nodes, leading to increased latency and scale issues with high-frequency snapshot schedules.
Innovation Solution
Implementing a system and method for time-synchronized snapshots by providing a synchronized clock across services, using a snapshot aggregator to stitch snapshots across multiple clusters, and leveraging a time synchronization service via APIs to manage I/O operations and avoid remote procedure calls, allowing for high-frequency snapshot capture without stalling writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If high-frequency snapshot schedules are implemented across distributed services, then snapshot consistency is improved, but latency increases and system performance deteriorates
Solution Approach 1:
The system performs preliminary time synchronization across all nodes before snapshot operations. Each node obtains synchronized time from a time synchronization service, allowing them to independently determine when to capture snapshots without waiting for coordination signals, thus eliminating latency while maintaining consistency.
Solution Approach 2:
Each distributed service independently captures snapshots based on its own local clock that is synchronized to a common time source. Services autonomously determine snapshot timing and execution without requiring remote procedure calls or coordination with other services, eliminating inter-service latency while ensuring all snapshots represent the same point in time.
2Reliability
If synchronized time is implemented across distributed services, then snapshot consistency is improved, but system complexity increases
Solution Approach 1:
A centralized time synchronization service acts as an intermediary that provides synchronized time to all distributed services. Each service simply queries this service for the current time and uses it for snapshot operations, avoiding the complexity of implementing distributed consensus algorithms or peer-to-peer time synchronization while ensuring all nodes operate with consistent time references.
3Reliability
If remote procedure calls are used for coordination, then snapshot consistency is improved, but performance deteriorates
Solution Approach 1:
Each distributed service independently determines snapshot timing and execution based on its locally synchronized clock, eliminating the need for remote procedure calls for coordination. Services autonomously capture snapshots when their local time matches the target snapshot time, achieving both consistency and high performance without inter-service communication overhead.
Data Source
AI summary
An illustrative embodiment disclosed herein is an apparatus including a processor having programmed instructions to request, from each of a plurality of services spanning multiple clusters, a plurality of time instances. A plurality of snapshots were taken during the plurality of time instances. The processor has programmed instructions to present, to a user, the plurality of time instances and a plurality of identifiers. Each of the plurality of identifiers corresponds to one of the plurality of services. The processor has programmed instructions to receive, from the user, a first selection of a first time instance of the plurality of time instances and a second selection of a subset of the plurality of identifiers. Each of the subset of identifiers corresponds to one of a subset of the plurality of services. The processor has programmed instructions to apply, to a target, a subset of the plurality of snapshots.