Tree-Based Server Architecture for Data Replication Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication methods face scalability issues due to increased latency and unreliable write operations, especially when clients are geographically distant from primary servers, and existing scalable methods like peer-to-peer networks suffer from unpredictable data transfer times.
Innovation Solution
Implementing a distributed computing system with a tree-based architecture that combines synchronous and asynchronous replication, where data is written to a sync replica set for immediate acknowledgment and then asynchronously replicated to an async tree, minimizing write latency while ensuring high data reliability and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is replicated synchronously to multiple secondary servers to increase data availability and reliability, then data reliability is improved, but write latency increases significantly
Solution Approach 1:
The patent segments the replication process into two distinct phases: a synchronous phase where data is replicated to a primary server and a limited number of secondary servers to ensure reliability, and an asynchronous phase where data is propagated to additional follower servers. This segmentation allows the system to achieve both high reliability (through synchronous replication to a quorum) and low latency (by not waiting for all followers), thus resolving the contradiction between data reliability and write latency.
2Reliability
If the number of secondary servers is increased to improve data availability, then data availability is improved, but write delay increases
Solution Approach 1:
The patent implements dynamic replication groups where the system can adaptively manage different sets of servers for synchronous and asynchronous replication. The synchronous replica set is dynamically configured to include only the minimum necessary servers to achieve quorum, while additional follower servers are added to the asynchronous group. This dynamic configuration allows the system to maintain high data availability across many servers without increasing write delay, as the synchronous phase completes quickly with a limited quorum set.
3Reliability
If synchronous replication is used to ensure data consistency, then data consistency is improved, but client experience deteriorates due to additional delays
Solution Approach 1:
The patent applies partial action by implementing synchronous replication only to the extent necessary to achieve data consistency - specifically, to a quorum of the synchronous replica set rather than to all possible replica servers. This partial synchronous approach ensures data consistency for the critical majority of servers while avoiding the excessive delays that would result from waiting for all servers to confirm. The remaining servers receive data asynchronously, providing redundancy without impacting client experience.
Data Source
AI summary
The disclosure is directed to data replication in a distributed computing system. The data, e.g., received from a client, is replicated to a first set of servers (“sync replica set”) synchronously and to a second set of servers asynchronously (“async tree”). A server can be a primary, secondary or a follower server. A sync replica set includes a primary server and one or more secondary servers. The async tree includes multiple follower servers deployed in a tree structure. A primary server can replicate the data to the secondary servers synchronously, and the secondary servers can replicate the data to one or more follower servers, e.g., a root node of the async tree, asynchronously. The root node then replicates the data to children of the root node, which then replicate to their children and so on until the leaf nodes of the async tree receive the data.