High-Throughput Synchronous Replication With Failure-Point Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional asynchronous replication processes in cloud data storage leave a window of vulnerability where data loss can occur if a failure happens before replication to secondary storage locations, especially for mission-critical applications.
Innovation Solution
Implementing a synchronous replication method that writes data blocks simultaneously to primary and secondary storage locations, with failure handling mechanisms to ensure data availability by synchronously writing to a third location if an unrecoverable failure occurs at the secondary location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If asynchronous replication is used to store data at multiple storage locations, then data robustness is improved, but data loss can occur during the replication window
Solution Approach 1:
The system performs preliminary actions by initiating replication to multiple storage locations before the primary write is complete. The data is written to both the primary storage location and secondary storage locations in parallel, ensuring that replication occurs before the transaction is considered complete, thereby eliminating the vulnerability window of asynchronous replication.
Solution Approach 2:
The system provides beforehand cushioning by maintaining multiple copies of data across different geographical zones simultaneously. This redundancy acts as a cushion against data loss, ensuring that if one storage location becomes inaccessible, data can be recovered from other locations without loss.
2Loss of information
If synchronous replication is implemented to prevent data loss, then data integrity is improved, but system complexity increases
Solution Approach 1:
The system segments the replication process by dividing data into data blocks and handling replication for each block independently. This segmentation allows the system to manage complexity by processing smaller units of data across multiple storage locations rather than handling entire datasets as single units.
Solution Approach 2:
The data processing hardware performs multiple functions simultaneously - it writes data to primary storage locations, replicates to secondary storage locations, determines failures, identifies failure points, and initiates recovery operations all through a unified synchronous replication system, reducing overall system complexity.
3Reliability
If data is written to multiple geographical regions simultaneously, then data availability is improved, but writing time increases
Solution Approach 1:
The system maintains continuity of useful action by performing data writing and replication operations in parallel rather than sequentially. Data blocks are written to primary and secondary storage locations simultaneously, ensuring continuous data flow without interruption, thereby minimizing total writing time while maintaining data availability across multiple regions.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method (500) for synchronous replication of stream data includes receiving a stream of data blocks (22) for storage at a first storage location (210) associated with a first geographical region and at a second storage location associated with a second geographical region. The method also includes synchronously writing the stream of data blocks to the first storage location and to the second storage location and determining an unrecoverable failure at the second storage location. The method also includes determining a failure point (220) in the writing of the stream of data blocks that demarcates data blocks that were successfully written and not successfully written to the second storage location. The method also includes synchronously writing, starting at the failure point, the stream of data blocks to the first storage location and to a third storage location associated with a third geographical region.