Free World Replication Protocol for Key-Value Store Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing consensus protocols for distributed key-value stores rely solely on replication nodes for determining consensus, leading to increased latency and inefficiency in data replication, as clients are not utilized for transaction transmission, resulting in suboptimal performance and resource utilization.
Innovation Solution
The free world replication protocol leverages client computing resources by allowing write clients to transmit transactions directly to multiple nodes, utilizing a master node for quorum detection and transaction commitment, thereby reducing latency and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If clients are not utilized for transaction transmission and only replication nodes are used, then data consistency is maintained, but transaction latency increases and resource utilization decreases
Solution Approach 1:
The patent applies self-service by enabling write clients to directly transmit transactions to multiple replication nodes without requiring coordination through a single leader node. Each client independently performs the transmission task that would otherwise be handled by the replication layer, reducing latency while maintaining consistency through the underlying consensus protocol
Solution Approach 2:
The patent implements preliminary action by having the write client proactively send transactions to multiple replication nodes in advance, before the transaction is fully committed. This pre-distribution of transactions reduces the overall transaction latency while the consensus protocol ensures eventual consistency
2Device complexity
If clients are not utilized for transaction transmission, then protocol simplicity is maintained, but computing resources such as processor cycles and network bandwidth are not optimized
Solution Approach 1:
The patent applies universality by enabling write clients to perform multiple functions: acting as both application clients and transaction distributors. This multi-functionality allows clients to directly transmit transactions to multiple replication nodes, optimizing network bandwidth and processor utilization without adding complex protocol layers
3Productivity
If multiple write clients are allowed without quorum detection, then system availability is improved, but data consistency cannot be ensured
Solution Approach 1:
The patent implements feedback by introducing quorum detection mechanisms that monitor transaction acknowledgment status. Write clients receive feedback from replication nodes about transaction receipt status, and the system adjusts its behavior based on this feedback to ensure that a sufficient quorum of nodes have received and committed the transaction before considering it complete, thereby maintaining consistency while allowing multiple write clients
Data Source
AI summary
The “free world replication protocol” makes use of client computing resources, wherein the clients are not part of the replicated key-value store, but instead reside in the “free world” outside of the dedicated resources of the nodes of the replicated key-value store. In the free world replication protocol, only a single “write” client is authorized to modify the key-value store at any time but any number of clients may be authorized to read data from the key-value store. The write client sends its transactions to multiple nodes in the replicated key-value store. As a result, the latency between the transaction being sent from the client and the transaction being received by the multiple nodes is reduced. A consensus protocol, driven by a master node, is used to periodically ensure consistency, but the data transactions themselves do not make use of a master node.