Container Group Leader Election with Runner-Watcher Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing leader election processes in distributed computing systems involving multiple applications across nodes are inefficient, leading to increased computing resource utilization, bandwidth consumption, and latency due to query-based mechanisms, and result in multiple leaders within the same pod, increasing architectural complexity.
Innovation Solution
Implement an election runner process and an election watcher process within each pod to manage leader status changes, using a bidirectional RPC service for communication, allowing instances on the same node to respond timely and reduce the need for source code modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If query-based leader election mechanism is used, then leader status can be determined, but computing resource utilization increases and latency increases
Solution Approach 1:
Instead of having instances query the leader status periodically (pull mechanism), the system inverts the approach by having the leader actively push status updates to instances. The leader broadcaster process publishes leader status changes to a topic, and instance watcher processes subscribe to receive notifications, thereby eliminating the need for continuous querying and reducing computing resource utilization.
Solution Approach 2:
The system introduces a message broker (Kafka) as an intermediary between the leader broadcaster and instance watchers. The leader status changes are published to a Kafka topic, and instances subscribe to receive updates asynchronously. This intermediary mechanism decouples the leader and followers, reducing direct communication overhead and latency while maintaining reliable leader status determination.
2Reliability
If query-based leader election mechanism is used, then leader status can be determined, but bandwidth consumption increases
Solution Approach 1:
The system inverts the communication pattern from pull-based queries to push-based notifications. The leader broadcaster actively publishes status changes to the Kafka topic, and instances subscribe to receive only when changes occur. This eliminates continuous bandwidth consumption from periodic queries and reduces overall bandwidth usage while maintaining reliable leader status determination.
3Adaptability or versatility
If multiple leaders exist within the same pod, then leader election can occur across distributed nodes, but architectural complexity increases
Solution Approach 1:
The system segments the leader election functionality into distinct processes: leader broadcaster process that manages leader status, instance watcher processes that monitor status changes, and separate election candidate applications. This segmentation allows multiple leaders to exist across different pods while maintaining clear architectural boundaries and reducing overall system complexity through modular design.
Data Source
AI summary
A computer-implemented method for electing a leader in a computing system is provided. In one aspect, a method includes identifying a computing resource for multiple container groups that each include one or more containers. A determination is made, from applications running in containers of the container groups, of multiple election candidate applications. Each election candidate application has an instance deployed in a corresponding container in each container group. For each container group, an election runner process is established within the container group. For each instance of each of the election candidate applications, a corresponding election watcher process is established. A communication link is established between the election runner process and each election watcher process. A request for leader election is transmitted from the election runner process to the computing resource. A response received from the computing resource. The response is transmitted to each election watcher process via the communication link.