Heartbeat Communication Randomization for Distributed Database Partition Groups
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-partitioned distributed databases, the increasing number of heartbeat communications due to the Raft consensus algorithm can lead to congestion and unnecessary leader elections, particularly as the system scales up, affecting the reliability and performance of the system.
Innovation Solution
Randomizing the timing of heartbeat communications across partition groups based on a maximum duration heartbeat communication interval, which is partly derived from the heartbeat timeout interval, to minimize the number of concurrent heartbeat communications and prevent unnecessary leader elections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of computing nodes and partitions is increased to achieve horizontal scalability, then data access performance is improved, but the number of heartbeat communications increases causing network congestion
Solution Approach 1:
The patent applies dynamics by making the heartbeat communication interval variable rather than fixed. Each partition group dynamically adjusts its heartbeat interval based on its position in a randomized sequence, transforming the static periodic communication into a dynamic adaptive system that distributes communication load over time
Solution Approach 2:
The patent changes the parameter of heartbeat communication interval from a fixed value to a variable value drawn from a probability distribution. This parameter change allows the system to maintain the necessary heartbeat functionality while varying the timing to avoid synchronized communications across all partition groups
2Reliability
If periodic heartbeat communications are sent according to a pre-determined schedule, then leader detection is maintained, but unnecessary leader elections are triggered due to congestion
Solution Approach 1:
The patent maintains periodic action by ensuring heartbeats are sent at regular intervals within each partition group, but applies a twist by randomizing the phase offset of these periodic actions across different partition groups. This preserves the reliability of leader detection within each group while preventing system-wide synchronization issues
Solution Approach 2:
The patent applies preliminary action by pre-assigning randomized heartbeat intervals to partition groups before the system operates. This preliminary randomization prevents congestion and unnecessary elections from occurring in the first place, rather than reacting to them after they happen
3Reliability
If the number of heartbeat communications is increased to maintain consistency across more partitions, then data consistency is improved, but computational resource consumption increases
Solution Approach 1:
The patent applies segmentation by dividing the system into partition groups that communicate heartbeats independently with randomized intervals. This segmentation allows each group to maintain consistency without contributing to system-wide communication storms, reducing overall computational resource consumption while preserving data consistency within each segment
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In some examples, a first computing device of a plurality of computing devices may be a member of a first partition group. For example, the first partition group may correspond to a first partition of multiple partitions of a metadata database. The first partition group may include at least one other computing device. Furthermore, each of the plurality of computing devices may maintain at least one partition of the metadata database so that the metadata database is partitioned into multiple partitions distributed across the computing devices. The first computing device may determine a first time threshold for a heartbeat communication to be sent to the other computing device(s) in the first partition group, and may select a random time before the first time threshold. The first computing device may send, based on the selected random time, the heartbeat communication to the other computing device(s) in the first partition group.