Cloud Data Replication with Replica ID Reassignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, particularly in semi-connected environments, web applications face challenges in sharing data among end-user devices due to intermittent, low-bandwidth, high-latency network connectivity, which limits data replication and increases battery consumption.
Innovation Solution
A multi-master cloud-based data replication service that facilitates near real-time data replication between personal computing devices using an incremental and state-based replication protocol, allowing devices to join existing data sharing groups without loss of non-conflicting changes, and supporting disconnected updates with reduced metadata usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If web applications store and share data among end-user devices in semi-connected environments, then data availability and accessibility are improved, but network communication overhead and battery consumption increase
Solution Approach 1:
The system performs preliminary actions by establishing a data replication model where changes are buffered locally on devices and queued for asynchronous replication. This allows devices to continue operating independently without immediate network communication, reducing battery consumption while maintaining data availability through eventual synchronization.
Solution Approach 2:
The replication service maintains continuous operation by buffering changes locally and asynchronously replicating them when network conditions permit. This continuous buffering and eventual replication ensures data availability across devices without requiring constant network communication, thus reducing energy consumption.
2Reliability
If web applications implement real-time data replication among devices, then data consistency is improved, but network bandwidth and communication overhead increase
Solution Approach 1:
The system extracts only the essential change information from full data replication by implementing an incremental replication model. Instead of transferring complete datasets, only the deltas (changes) are replicated, significantly reducing network communication volume while maintaining data consistency across devices.
Solution Approach 2:
The system changes the replication parameters from synchronous real-time replication to asynchronous event-driven replication. This parameter change allows the system to maintain data consistency by replicating changes as they occur, but only when network conditions are favorable, reducing overall communication overhead.
3Reliability
If web applications use traditional replication protocols, then data synchronization is improved, but metadata storage requirements and network traffic increase
Solution Approach 1:
The system uses disposable, lightweight change event objects that contain only the essential information needed for replication (change type, affected records, timestamps). These lightweight objects replace traditional heavy metadata structures, reducing both storage requirements and network traffic while maintaining effective data synchronization.
Solution Approach 2:
Instead of copying entire datasets and metadata structures, the system creates and replicates only the minimal change event representations. This selective copying approach maintains synchronization by tracking only what has changed, significantly reducing metadata volume and network traffic.
4Adaptability or versatility
If web applications operate in semi-connected environments with intermittent connectivity, then adaptability to diverse networks is improved, but data replication reliability and speed decrease
Solution Approach 1:
The system performs preliminary actions by buffering all data changes locally in a queue, ensuring that even if network connectivity is intermittent or lost, no data is lost. The replication service continuously attempts to deliver buffered changes when connectivity is restored, maintaining replication reliability despite network conditions.
Solution Approach 2:
The system implements feedback mechanisms where devices report their replication status and network conditions to the replication service. This feedback allows the service to adjust its behavior, requeuing failed deliveries and optimizing replication timing based on actual network conditions, thereby maintaining reliability in semi-connected environments.
Data Source
AI summary
A multi-master cloud-based data replication service designed to support the needs of a broad class of web applications is disclosed. Generally, the replication service serves the purposes of facilitating sharing of web application data among personal computing devices and increasing the availability of that data. In one embodiment, the replication service includes a replica identifier reassignment features that allows a personal computing device to begin sharing application data changes with an existing group of personal computing devices that are already sharing application data changes between them even if the joining personal computing device has application data changes to share with the other personal computing devices of the group at the time the joining personal computing device requests to join the group.