Batch Job Scheduler Election Using Shared Memory Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed systems, particularly microservices architectures, face limitations in handling batch jobs during traffic spikes due to single instance requirements, leading to clogging and manual vertical scaling, which hinders horizontal scaling and resilience.
Innovation Solution
Implementing multiple scheduler nodes in a scheduler system that write their host network address and port number into a shared memory, designating the last writer as the primary scheduler to manage batch job execution, allowing for resilient operation without manual scaling or maintenance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single scheduler instance is used to manage batch jobs, then the system structure is simple, but the system cannot handle traffic spikes and requires manual vertical scaling
Solution Approach 1:
The patent divides the single scheduler into multiple scheduler nodes that can operate independently. Each node can pick up and execute batch jobs from the queue, allowing the system to handle traffic spikes by distributing the load across multiple nodes rather than relying on a single point of failure.
Solution Approach 2:
The patent implements dynamic election of a primary scheduler among multiple nodes. The system automatically selects which node acts as primary based on current conditions, enabling flexible adaptation to traffic patterns without manual intervention or fixed architectural constraints.
2Reliability
If multiple scheduler nodes are implemented, then horizontal scaling and resilience are improved, but the system complexity increases
Solution Approach 1:
The scheduler nodes automatically perform election of the primary scheduler and coordinate job execution without external management. The nodes monitor their own state and the shared memory, automatically adapting to failures and load conditions, which reduces the need for complex external control mechanisms.
Solution Approach 2:
The patent introduces a shared memory as an intermediary mechanism that all scheduler nodes access to coordinate their operations. This shared memory simplifies coordination by providing a common communication channel for electing primary schedulers and tracking job status, eliminating the need for complex inter-node communication protocols.
3Reliability
If manual vertical scaling is used to resolve stale state, then the system can recover from clogging, but the system requires manual intervention and maintenance
Solution Approach 1:
The patent implements automatic recovery mechanisms where scheduler nodes continuously monitor the shared memory and election state. When a node becomes stale or fails, other nodes automatically detect this condition and re-elect a new primary scheduler without manual intervention, enabling the system to self-heal and maintain operational reliability.
Data Source
AI summary
A system includes a task queue that stores multiple tasks of a batch job, and a scheduler. The scheduler includes multiple scheduler nodes which, on a first-come, first-served basis during a first interval, write their network address and port number to a same memory location of a shared memory storage, while overwriting any previous network address and port number written to the same memory location. A last one of the multiple scheduler nodes to write its network address and port number to the same memory location during the first interval is designated as a first primary scheduler and commands an execution system to perform task execution upon a first task of the batch job stored in the task queue.