Data Replication Protocol with Dynamic Quorum Formation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data replication systems face challenges in maintaining steady state data distribution and efficient quorum formation, especially in failure situations, leading to potential data loss and inefficiencies in updating replica machines.
Innovation Solution
A high-throughput data replication protocol that enables steady state data distribution and a three-phase commit quorum formation protocol, combined with enhanced B+ trees and operation logs for efficient updates, allowing multiple outstanding prepare operations and minimizing data copying, while maintaining data integrity and consistency across replica machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data replication protocols are used, then data consistency is maintained, but system throughput decreases and update efficiency is reduced
Solution Approach 1:
The patent segments the data replication process into distinct phases (prepare, commit, update) that can operate independently and in parallel. Multiple prepare operations can be outstanding simultaneously, allowing the system to process multiple replication requests concurrently without compromising consistency, as each phase completes before the next begins.
Solution Approach 2:
The patent implements preliminary actions by performing prepare operations ahead of actual data updates. Replica machines prepare their data structures in advance, validating consistency requirements before committing changes. This allows throughput optimization without sacrificing reliability, as consistency checks are completed beforehand.
2Reliability
If frequent quorum elections are performed to maintain data distribution, then data consistency is ensured, but system overhead and update time increase
Solution Approach 1:
The patent implements dynamic quorum formation where the quorum composition can change without requiring full re-elections. When replica machines are added or removed, the system dynamically adjusts the quorum configuration, maintaining consistency while minimizing disruption and time loss compared to traditional fixed quorum requirements.
Solution Approach 2:
The patent changes the parameter of quorum size and composition dynamically based on system state. Rather than requiring fixed majority votes for every update, the system adapts quorum parameters to current replica availability, reducing the time overhead while maintaining sufficient consistency guarantees for data distribution.
3Reliability
If comprehensive data copying is performed during updates, then data completeness is maintained, but network bandwidth consumption and update efficiency decrease
Solution Approach 1:
The patent extracts only the necessary data portions that need to be copied during updates, rather than performing comprehensive data copying. By identifying and transferring only the specific data segments that changed or are required for consistency, the system maintains data completeness while significantly reducing network bandwidth consumption and improving update efficiency.
Solution Approach 2:
The patent applies partial action by performing selective data copying only where needed for consistency, rather than copying all data. The system determines the minimum necessary data transfer required to maintain completeness, avoiding excessive copying operations that would consume bandwidth without providing additional consistency benefits.
4Productivity
If data structures are updated to handle multiple prepare operations, then replication throughput increases, but data structure complexity increases
Solution Approach 1:
The patent uses nested data structures where prepare operations are organized in hierarchical containers that can hold multiple operations. The nested structure allows efficient grouping and management of multiple prepare operations, enabling high throughput processing while keeping the overall data structure organization manageable and systematic rather than chaotic.
Solution Approach 2:
The patent adds another dimension to the data structure by introducing temporal or sequential ordering of prepare operations. Rather than managing multiple operations in a single flat structure, the system organizes them along an additional dimension (such as operation sequence or phase completion status), which enables efficient throughput processing while maintaining structural clarity.
Data Source
AI summary
Steady state data distribution is provided between a client application, a leader machine, and a plurality of replica machines. The distribution comprises the leader machine receiving an operation request from the client application, the leader machine sending the prepare message to each of the plurality of replica machines, the replica machines recording in their logs information on the operation, the replica machines sending acknowledgement messages to the leader machine, and the leader machine sending commit command messages to the replica machines. A new quorum of the replica machines is created by using log information. Replica machines that become part of the new quorum are updated in an efficient manner.