Hybrid Cloud Cluster Heartbeat Segmentation for Lower WAN Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clustered systems using periodic heartbeat communications for monitoring resources do not scale effectively across wide area networks like the Internet, leading to issues with latency and network interference.
Innovation Solution
Implementing a hybrid cloud structure with metadata partition groups that restrict heartbeat communications to the same geographic location, using separate monitoring for data and metadata, and employing a coordination service to split and migrate partitions based on load balancing criteria across multiple geographic locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic heartbeat communications are used for monitoring cluster resources, then resource monitoring capability is improved, but network latency and interference increase across wide area networks
Solution Approach 1:
The patent segments the cluster into geographically dispersed data centers, each maintaining local partitions. Heartbeat communications are confined to local data centers rather than spanning the entire WAN, reducing latency while maintaining monitoring capability. The coordination service segments the monitoring function by receiving heartbeats locally and selectively forwarding only necessary information across WAN boundaries.
Solution Approach 2:
The patent implements local quality by allowing each data center to independently monitor its own partitions through local heartbeat communications. This ensures high reliability for local resource monitoring without the penalty of WAN latency. The system adapts monitoring behavior to local conditions, with each data center operating autonomously within its geographic boundary.
2Adaptability or versatility
If data is stored across multiple geographic locations, then data accessibility and redundancy are improved, but system complexity increases
Solution Approach 1:
The coordination service acts as a universal component that handles multiple functions: receiving heartbeats from all data centers, determining partition migration targets, and managing the migration process itself. This multi-functional approach simplifies the overall system architecture by providing centralized intelligence rather than requiring complex peer-to-peer coordination between multiple data centers.
Solution Approach 2:
The system dynamically adjusts partition locations based on load conditions. The coordination service continuously receives heartbeat information, determines which partitions should migrate based on current load balances, and executes migrations as needed. This dynamic behavior allows the system to adapt to changing conditions without requiring manual intervention or complex static configuration.
3Loss of information
If heartbeats are exchanged across wide area networks, then cluster-wide monitoring is improved, but network interference and latency worsen
Solution Approach 1:
The patent extracts the heartbeat communication function from the global cluster level and implements it at the local data center level. Each data center independently exchanges heartbeats with other data centers rather than all nodes exchanging heartbeats with all other nodes. This extraction reduces the total volume of heartbeat traffic across the WAN, minimizing network interference while preserving cluster-wide monitoring capability through the coordination service.
4Productivity
If partitions are migrated based on load balancing, then resource utilization is improved, but migration overhead increases
Solution Approach 1:
The system implements partial action by migrating only those specific partitions that need to move based on load balance requirements, rather than migrating all partitions. The coordination service selectively identifies partitions for migration based on heartbeat load information, executing migrations only where necessary. This partial approach optimizes resource utilization while minimizing migration overhead by avoiding unnecessary data movement.
Data Source
AI summary
In some examples, a computing device of a plurality of computing devices at a first geographic location may divide data into a plurality of partitions. For example, at least two computing devices at the first geographic location may maintain a copy of the data of a first partition of the plurality of partitions and may exchange periodic heartbeat communications related to the first partition. The computing device may determine that a computing device at the second geographic location has a lower frequency of access load than the computing devices at the first location that maintain the first partition. The computing device may migrate the data of the first partition to a computing device at the second geographic location to cause at least two computing devices at the second location to maintain the data of the first partition and exchange periodic heartbeat communications related to the first partition.