Node-Local-Unscheduler for Container Scheduling Remediation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing container scheduling techniques often result in improper scheduling due to outdated resource information and scheduling policies, leading to containers becoming stuck in a pending state, inefficient resource allocation, and potential system crashes, especially in high-demand environments, where multiple schedulers can exacerbate issues and reduce system integrity.
Innovation Solution
Implementing a node-local-unscheduler (NLU) that annotates and stops improperly scheduled containers, allowing the scheduler to reschedule them with up-to-date information, thereby preventing resource overconsumption and ensuring efficient resource allocation without the overhead of multiple schedulers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple schedulers are deployed to handle high scheduling loads, then scheduling capacity increases, but system integrity deteriorates and scheduling violations increase
Solution Approach 1:
The patent segments the scheduling function into two distinct components: a primary scheduler that handles the main scheduling load, and a node-local-unscheduler that operates independently at each node to validate and remediate scheduling violations. This segmentation allows the system to maintain high scheduling capacity through the primary scheduler while ensuring system integrity through the distributed validation function of the node-local-unscheduler, thereby resolving the contradiction between productivity and reliability.
2Speed
If containers are scheduled quickly without validation, then scheduling speed increases, but scheduling violations increase causing containers to get stuck in pending state
Solution Approach 1:
The node-local-unscheduler performs preliminary validation actions by continuously monitoring scheduled containers and checking them against current node conditions and scheduling policies. When a scheduling violation is detected, the unscheduler proactively annotates the container with remediation instructions before the container can get stuck in a pending state. This preliminary action ensures scheduling correctness is maintained without compromising the speed of the primary scheduler.
3Reliability
If scheduling validation is performed thoroughly, then scheduling correctness improves, but scheduling time increases
Solution Approach 1:
The node-local-unscheduler acts as an intermediary that continuously monitors and validates container scheduling in the background, independent of the primary scheduler's operation. Instead of blocking the scheduling process to perform validation, the unscheduler asynchronously checks scheduled containers against current node conditions and policies, and only intervenes when violations are detected. This intermediary approach maintains scheduling correctness while minimizing impact on scheduling time.
Data Source
AI summary
A system for scheduling remediation includes a memory, a processor in communication with the memory, a container scheduled on a first node, a scheduler executing on the processor, and a node-local-unscheduler (“NLU”). The scheduler has a watch module. The NLU executes on the processor to determine a status of the container as failing validation. The NLU has access to scheduling policies corresponding to the container and the first node. Responsive to determining the status of the container as failing validation, the NLU annotates the container and stops execution of the container. The watch module executes on the processor to detect the annotation associated with the container. Responsive to detecting the annotation, the container is rescheduled to a second node.