Distributed Group Membership Self-Discovery via Consistent Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, managing group membership and resource distribution becomes complex as the system scales, leading to challenges in isolating issues, troubleshooting, and ensuring high availability, especially with central management servers acting as bottlenecks and single points of failure.
Innovation Solution
Implementing self-discovery of group membership among host computer systems using configuration information, such as hashing algorithms like consistent hashing, to distribute hosts evenly across groups, allowing for parallel execution and robust task processing, while maintaining the option for central management system oversight.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If centralized management servers are used to manage group membership, then system control and coordination are improved, but system availability and fault tolerance deteriorate due to single points of failure
Solution Approach 1:
The patent extracts the group membership management function from the centralized management server and distributes it to individual hosts. Each host independently determines its own group membership using configuration information and hashing algorithms, eliminating the single point of failure while maintaining systematic control through distributed autonomy.
Solution Approach 2:
The centralized management function is segmented into individual host-level decisions. Instead of one central authority managing all hosts, each host independently manages its own group membership determination, distributing the control function across multiple autonomous units that collectively maintain system coordination.
2Ease of operation
If centralized management servers manage all hosts, then coordination is simplified, but system complexity increases due to bottleneck and failure propagation risks
Solution Approach 1:
Hosts perform self-service by independently determining their own group membership using configuration information and hashing algorithms. This eliminates the need for hosts to communicate with a central management server for membership determination, reducing coordination complexity and eliminating failure propagation while maintaining systematic organization.
3Loss of information
If hosts communicate frequently with central management server, then membership information is kept current, but system efficiency deteriorates due to communication overhead and bottlenecks
Solution Approach 1:
Configuration information including group definitions and hashing algorithms is provided to hosts in advance. Hosts use this pre-provided information to independently determine group membership without needing to communicate with the management server during operation, eliminating communication overhead while ensuring accurate and current membership information.
Data Source
AI summary
In large distributed computing environments, application execution may be distributed between a plurality of groups, the plurality of groups containing a set of host computer systems responsible for the execution of one or more operations of the application. Group membership may be determined by generating configuration information based at least in part on the plurality of groups. The configuration information may be provided to a plurality of host computer systems and each host computer system of the plurality of host computer systems may determine membership to a particular group of the plurality of groups based at least in part on the configuration information.