Write Order Fidelity in Distributed Storage via Delta Sets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in maintaining write order fidelity across distributed networks, especially in geographically separated locations, where asynchronous data transmission can lead to data loss and consistency issues, and existing active-passive approaches are inefficient for totally-active operations.
Innovation Solution
The implementation of a distributed cache coherency mechanism combined with delta sets to ensure write order fidelity, allowing multiple nodes to read and write concurrently while maintaining data coherence, using a multi-stage pipeline for delta sets and a system-wide barrier mechanism to advance the pipeline simultaneously across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If asynchronous data transmission is used to improve write performance and minimize bandwidth requirements, then write performance is improved and bandwidth usage is reduced, but data consistency and write order fidelity are compromised
Solution Approach 1:
The system performs preliminary actions by recording write operations in a write order log before actual data transmission occurs. This allows the system to acknowledge writes immediately to the local node while maintaining a record of the write order, enabling later reconstruction of consistent data images without waiting for remote nodes to receive and apply the writes.
Solution Approach 2:
A write order log acts as an intermediary mechanism between multiple distributed nodes. This log captures the sequential order of writes and enables remote nodes to reconstruct their data images in the correct order, mediating between the asynchronous transmission needs and the consistency requirements without requiring synchronous coordination.
2Speed
If data is cached at access nodes to improve access speed, then data access speed is improved, but data coherence across nodes becomes difficult to maintain
Solution Approach 1:
The system implements feedback mechanisms where access nodes periodically receive updates about write operations from the write order log. This feedback allows cached data to be refreshed or invalidated based on the actual write order that occurred, maintaining coherence between cached copies and the authoritative data image without requiring continuous synchronization.
Solution Approach 2:
Remote nodes create copies of data blocks from the authoritative data image at appropriate points in the write order log. These copies are then cached locally, allowing fast access while maintaining coherence because the copies are created from a known consistent state and can be refreshed when new writes occur.
3Reliability
If strict write order fidelity is enforced to ensure data consistency, then data consistency is improved, but system complexity and performance overhead increase significantly
Solution Approach 1:
The system extracts the write order tracking function from the complex distributed coordination protocol and places it in a separate write order log. This separation allows the main data transmission and caching operations to proceed asynchronously with minimal complexity, while the write order log independently maintains the sequential record needed for consistency reconstruction.
4Reliability
If synchronous write replication is used to maintain write order fidelity, then data consistency is improved, but write performance deteriorates due to network latency
Solution Approach 1:
The system performs preliminary recording of write operations in the write order log before initiating data transmission to remote nodes. This preliminary action allows the local node to immediately acknowledge the write to the application, decoupling the write acknowledgment from the slower network transmission and replication processes.
Solution Approach 2:
The write operation is segmented into independent phases: local recording in the write order log, asynchronous data transmission to remote nodes, and later reconstruction of data images. This segmentation allows each phase to proceed independently without blocking the others, improving overall write performance while maintaining consistency through the sequential record in the log.
Data Source
AI summary
Write order fidelity (WOF) is maintained for totally-active implementations wherein a plurality of access nodes at geographically separated sites can concurrently read and/or write data in a “totally active” fashion on a distributed data system. From the hosts' perspective at diverse geographic locations, a synchronous, cache-coherent view of data is provided. Data transfer is asynchronous. A time ordered data image is created and maintained so operations can be restarted after a partial system failure that causes loss of data not yet asynchronously transferred across the network, but that has been write-acknowledged to the originating host. Time ordered asynchronous data transfer is implemented as a pipeline of changes that reflect contributions from all nodes. WOF also improves network performance and lowers bandwidth consumption. Extensions can provide, in a totally-active context, features such as point-in-time snapshots, time firewalls, on-demand backend storage allocation, synchronous/asynchronous distribution of data, and continuous data protection.