High-Throughput Synchronous Replication for Streaming Data Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional asynchronous replication processes in cloud data storage leave a window of vulnerability where data loss can occur if a failure happens before data is replicated to secondary storage locations, leading to irrecoverable data loss.
Innovation Solution
A system for synchronous replication of streaming data that writes data blocks simultaneously to primary and secondary storage locations, with failure handling mechanisms to ensure data availability by synchronously writing to a third location upon unrecoverable failures at the secondary location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If asynchronous replication is used to store data at multiple storage locations, then data robustness is improved, but data loss can occur during the replication window
Solution Approach 1:
The patent implements synchronous replication where data is written to both primary and secondary storage locations simultaneously, eliminating the replication window vulnerability. This preliminary action ensures that backup data is created at the same time as the primary data, preventing data loss during the replication process.
Solution Approach 2:
The system prepares multiple storage locations in advance and maintains data copies at these locations before failures occur. By having pre-positioned backup storage locations ready and synchronized, the system cushions against potential data loss from failures during the replication window.
2Reliability
If synchronous replication is implemented to prevent data loss, then data integrity is improved, but system complexity increases
Solution Approach 1:
The patent segments the replication process into distinct phases: initial synchronous replication to primary and secondary locations, failure detection, and recovery operations. This segmentation manages complexity by breaking down the complex replication and recovery process into manageable, independent components.
Solution Approach 2:
The system introduces intermediary components such as replication controllers and failure detection mechanisms that mediate between storage locations. These intermediaries simplify the overall system complexity by centralizing control logic and providing standardized interfaces for data replication and recovery operations.
3Reliability
If data is written to multiple storage locations simultaneously, then data availability is improved, but writing time increases
Solution Approach 1:
The patent merges the write operations to multiple storage locations into a single synchronous transaction. By combining the replication operations and using shared storage infrastructure, the system reduces the total writing time compared to sequential writes while maintaining data availability across multiple locations.
Data Source
AI summary
A method for synchronous replication of stream data includes receiving a stream of data blocks for storage at a first storage location associated with a first geographical region and at a second storage location associated with a second geographical region. The method also includes synchronously writing the stream of data blocks to the first storage location and to the second storage location. While synchronously writing the stream of data blocks, the method includes determining an unrecoverable failure at the second storage location. The method also includes determining a failure point in the writing of the stream of data blocks that demarcates data blocks that were successfully written and not successfully written to the second storage location. The method also includes synchronously writing, starting at the failure point, the stream of data blocks to the first storage location and to a third storage location associated with a third geographical region.