Inter-cluster Communication via Leader Node Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster computing systems face scalability issues due to the large number of open network connections required for inter-cluster communication, leading to resource drain and performance degradation, especially when communicating with external clusters through cloud platforms with limited infrastructure capabilities.
Innovation Solution
Implementing a system where only designated leader nodes in a cluster establish and maintain inter-cluster connections, reducing the overall number of simultaneous open connections and allowing non-leader nodes to communicate through these leader nodes, thereby optimizing resource usage and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If each node in a cluster establishes connections with all nodes in external clusters, then inter-cluster communication capability is improved, but the number of open connections increases exponentially causing resource drain
Solution Approach 1:
The patent segments the cluster nodes into two functional groups: leader nodes that establish external connections and non-leader nodes that communicate through leaders. This segmentation reduces the number of external connections from O(N*M) to O(L*M) where L is the number of leaders, significantly reducing connection quantity while maintaining communication capability.
Solution Approach 2:
Leader nodes act as intermediaries between non-leader nodes and external clusters. Non-leader nodes send messages to their designated leader, which then forwards them to the appropriate external cluster. This intermediary mechanism enables indirect communication without requiring direct connections from each node to every external node.
2Adaptability or versatility
If the number of open connections is increased to improve communication coverage, then more nodes can communicate with external clusters, but memory and processor resources are depleted
Solution Approach 1:
By segmenting nodes into leaders and non-leaders, the patent concentrates connection management resources on leader nodes only. Each non-leader node maintains minimal connection state (one to its leader), while leader nodes handle the actual external connections. This segmentation prevents resource depletion by limiting the number of connections each node must manage.
Solution Approach 2:
Leader nodes perform multiple functions: they maintain external cluster connections, receive messages from non-leader nodes, forward messages to external clusters, and manage connection state. This multi-functionality allows the system to achieve broad communication coverage without each node requiring full connection capabilities, thus conserving resources.
3Adaptability or versatility
If cloud provider connection limits are increased to allow more inter-cluster connections, then more nodes can communicate externally, but infrastructure costs increase and performance degrades
Solution Approach 1:
The patent implements partial action by having only leader nodes establish external connections rather than all nodes. This partial approach is sufficient to achieve full communication coverage since non-leader nodes can reach external clusters through their leaders. This avoids the excessive action of opening all possible connections, thereby respecting cloud provider limits and avoiding infrastructure complexity.
Data Source
AI summary
Various embodiments set forth a computer-readable media storing program instructions that, when executed by one or more processors, cause the processors to perform steps of maintaining, by a first node of a first cluster, a respective first open transport control protocol (TCP) connection with each of a plurality of second nodes in a second cluster; maintaining, by a third node of the first cluster, a second open TCP connection with the first node, wherein the third node is prevented from establishing a TCP connection with any of the plurality of second nodes; and sending, by the third node, a message to the second cluster by sending the message to the first node via the second open TCP connection, wherein the first node is configured to forward the message to one of the second nodes via a corresponding one of the respective first open TCP connections.