Data Replication Gateway Arbitrator for Multi-Site Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication systems face challenges in maintaining a single consistent view of the file system across mirrored sites, especially in the presence of network and node failures, and require different methodologies for single-site and multi-site data protection.
Innovation Solution
A method for replicating a content-addressable object store between multiple sites, ensuring bi-directional data synchronization, read and write ordering, and locking guarantees, with a gateway node acting as an arbitrator to manage file system operations and handle site additions or removals, and asynchronous data object transmission to maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data replication is implemented between multiple sites, then data protection and fault tolerance are improved, but system complexity increases
Solution Approach 1:
The system segments data into objects with unique signatures, allowing independent replication of each object across sites. The gateway node segments control functions by electing an arbitrator to manage ordering and locking, dividing the complexity of multi-site coordination into manageable parts.
Solution Approach 2:
The gateway node acts as an intermediary between sites, managing replication coordination, ordering, and locking protocols. The arbitrator elected at the gateway mediates conflicts and ensures consistent ordering across distributed sites, simplifying the complexity of direct peer-to-peer coordination.
2Ease of operation
If bi-directional replication is implemented to allow simultaneous access, then accessibility is improved, but data consistency and ordering become more difficult to maintain
Solution Approach 1:
The system implements feedback mechanisms where the arbitrator receives notifications of data changes from both sites and coordinates replication accordingly. This feedback loop ensures that changes are propagated in the correct order and that consistency is maintained despite bi-directional access.
Solution Approach 2:
The system performs preliminary actions by establishing ordering rules and locking mechanisms before data replication occurs. The arbitrator pre-determines the sequence of operations and sets up locking protocols to prevent conflicts, ensuring data consistency is maintained from the outset of bi-directional replication.
3Productivity
If asynchronous replication is used to improve performance, then replication speed is improved, but data integrity and ordering guarantees are weakened
Solution Approach 1:
The system segments data into discrete objects with unique signatures, allowing asynchronous replication of individual objects without affecting others. This segmentation enables parallel processing and improved throughput while maintaining integrity through content-based addressing and verification.
Solution Approach 2:
The system replaces synchronous mechanical replication with asynchronous content-addressable replication using cryptographic signatures. This substitution allows non-blocking replication where data integrity is verified through signature matching rather than synchronous coordination, improving performance while maintaining reliability.
4Reliability
If gateway nodes cooperate to elect an arbitrator, then coordination and ordering are improved, but node selection and management complexity increases
Solution Approach 1:
The gateway nodes perform self-service by automatically electing an arbitrator based on predefined criteria without external intervention. This self-election mechanism simplifies management by eliminating the need for centralized control or manual configuration of arbitrator selection.
Solution Approach 2:
The system uses parameter changes in node status and gateway roles to determine arbitrator election. By dynamically changing state parameters such as node availability and gateway status, the system enables automatic arbitrator selection while maintaining simple management through state-based decision logic.
Data Source
AI summary
Computer-implemented methods and systems for replication of data between mirrored data sites are provided. An exemplary method may comprise receiving a data object at a mirrored data site, the mirrored data site including one or more data nodes. The data objects are replicated to the one or more data nodes internally. Then, a data object reference associated with the data object is generated. The reference referring to the data object is queued for transmission to all other mirrored data sites. The data object reference associated with the data object is transmitted to the other mirrored data sites. When the data object is received at each mirrored data site, it is replicated to one or more data nodes within the site. After transmitting the data object, the data object reference is discarded.