Synchronous Distributed Storage Replication with Dual-Store Commit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed computing environments face challenges in maintaining data consistency and availability across geographically separate locations, particularly in the context of natural disasters or business disruptions, while ensuring compliance with local regulations and reducing latency.
Innovation Solution
A method for synchronously replicating data between primary and secondary data stores using a strong consistency approach, where data is annotated with unique identifiers and acknowledged at both stores before committing, ensuring data is written to both locations before providing an acknowledgment to the client.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If synchronous replication with strong consistency is implemented, then data availability and consistency are improved, but latency increases due to waiting for acknowledgment from secondary data store
Solution Approach 1:
The system performs preliminary actions by writing data to the log at the secondary data store before committing at the primary data store. The unique identifier is assigned and communicated to the secondary data store in advance, allowing it to prepare for replication without waiting for the complete commit process, thus reducing latency while maintaining strong consistency.
Solution Approach 2:
The system implements feedback mechanisms where the secondary data store sends acknowledgment messages back to the primary data store. This feedback loop allows the primary data store to know when the secondary has successfully written and committed the data, enabling proper synchronization while managing latency through selective acknowledgment timing.
2Reliability
If data is replicated to multiple geographical locations, then disaster recovery capability is improved, but system complexity increases
Solution Approach 1:
The system segments the replication process into distinct components: a primary data store handling write operations, a secondary data store handling replication reception, and a unique identifier mechanism tracking replication status. This segmentation simplifies the overall system architecture by dividing complex disaster recovery functions into manageable, independent modules that can be implemented and maintained more easily.
Solution Approach 2:
The unique identifier acts as an intermediary mechanism that coordinates between the primary and secondary data stores. It facilitates the replication process without requiring complex communication protocols or coordination systems, thereby reducing overall system complexity while enabling multi-location replication for disaster recovery.
3Reliability
If data is written to log before commitment, then data durability is improved, but availability of uncommitted data is reduced
Solution Approach 1:
The system performs the preliminary action of writing data to the log at the secondary data store before the primary data store commits the data. This ensures that even if the primary data store fails before committing, the secondary data store has already preserved the data in its log, improving data durability without requiring the primary store to wait for commit completion.
Solution Approach 2:
The system implements beforehand cushioning by maintaining a log at the secondary data store that captures data before final commitment. This log acts as a cushion or safety net, preserving data durability in case of failures. The unique identifier ensures that only properly tracked and validated data is replicated to the secondary store's log, balancing durability with availability.
Data Source
AI summary
Embodiments of the present invention relate to synchronously replicating data in a distributed computing environment. To achieve synchronous replication both an eventual consistency approach and a strong consistency approach are contemplated. Received data may be written to a log of a primary data store for eventual committal. The data may then be annotated with a record, such as a unique identifier, which facilitates the replay of the data at a secondary data store. Upon receiving an acknowledgment that the secondary data store has written the data to a log, the primary data store may commit the data and communicate an acknowledgment of success back to the client. In a strong consistency approach, the primary data store may wait to send an acknowledgement of success to the client until it receives an acknowledgment that the secondary has not only written, but also committed, the data.