Distributed Task Scheduler for Dynamic Workload Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing task scheduling systems face challenges in efficiently managing tasks across multiple processors, particularly when the number of available processors changes, leading to bottlenecks and difficulties in real-time rescheduling.
Innovation Solution
A distributed scheduling architecture where each processing node has its own task scheduler that periodically checks the number of active nodes and adjusts its workload range, allowing for dynamic reassignment of tasks based on changes in processor availability, ensuring balanced workload distribution and tolerance to individual node failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a centralized task scheduler is used to manage tasks across multiple processors, then task assignment can be coordinated, but the system becomes complex and difficult to reschedule in real-time when processor availability changes
Solution Approach 1:
The centralized task scheduler is segmented into distributed task schedulers, with each processing node having its own independent scheduler. This divides the monolithic scheduling function into smaller, autonomous units that can make local decisions, reducing overall system complexity while improving adaptability to real-time changes in processor availability
Solution Approach 2:
The system implements dynamic workload range adjustment where each task scheduler periodically determines the number of active processing nodes and adjusts its assigned workload range accordingly. This dynamic adaptation allows the system to automatically reschedule tasks in real-time when processor availability changes, without requiring complex centralized coordination
2Productivity
If the number of parallel processors increases to meet increased demand, then system capacity improves, but the complexity of task rescheduling increases
Solution Approach 1:
Each task scheduler dynamically adjusts its workload range based on the current number of active processing nodes. When processors are added or removed, schedulers automatically recalculate and redistribute tasks within their assigned ranges, enabling seamless scaling without increasing rescheduling complexity
Solution Approach 2:
Task schedulers autonomously manage their own workload assignments by periodically determining active node counts and adjusting their ranges independently. This self-service mechanism eliminates the need for complex external coordination when system capacity changes, allowing the system to scale while maintaining simple rescheduling logic
3Productivity
If tasks are assigned to specific processors, then workload distribution can be optimized, but the system becomes vulnerable to bottlenecks when processor availability changes
Solution Approach 1:
The system implements periodic workload range determination where each task scheduler adjusts its assigned task range based on current processor availability. This dynamic recalibration prevents bottlenecks by automatically redistributing tasks when processors become unavailable, maintaining both workload efficiency and system reliability
Solution Approach 2:
The system changes the parameter of workload range assignment dynamically. Instead of fixed processor-task mappings, schedulers adjust their workload ranges as a parameter based on real-time processor availability, allowing optimized workload distribution while maintaining flexibility to prevent bottlenecks
Data Source
AI summary
In one embodiment, a method for scheduling tasks comprises, at a task scheduler of a processing node of a plurality of processing nodes: retrieving a first task descriptor from a local memory of the task scheduler, the task descriptor corresponding to a task scheduled for execution at the current time and comprising at least a task execution time, a frequency for performing the task, and a task identifier; determining whether the task descriptor is assigned to the task scheduler for execution; if it is determined that the task descriptor is assigned to the task scheduler for execution: executing the task; updating the task execution time based on the current task execution time and the frequency for performing the task; and re-queuing the task descriptor in the local memory.