Pluggable Scheduler Architecture for Thread Pool Fairness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scheduling methods for multi-threaded enterprise applications face inefficiencies in resource utilization and fairness across applications, with approaches either underutilizing resources or leading to resource starvation due to inadequate scheduling strategies.
Innovation Solution
A pluggable scheduler architecture that integrates with existing schedulers or operates standalone, employing a single thread pool with customizable scheduling policies at both inter-application and intra-application levels, using fairness algorithms and APIs to manage thread allocation and task execution based on weight-based, priority-based, round-robin, and other strategies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single thread pool is used for the whole system with first-come first-served scheduling, then resource utilization efficiency is improved, but fairness between applications deteriorates leading to resource starvation
Solution Approach 1:
The patent segments the scheduling system into multiple application-specific thread pools, where each application has its own dedicated thread pool. This segmentation allows each application to have guaranteed resource allocation while maintaining overall system efficiency, resolving the contradiction between resource utilization and fairness by preventing any single application from monopolizing the shared thread pool.
Solution Approach 2:
The patent changes the scheduling parameters by introducing configurable thread pool sizes for each application based on their priority and resource requirements. By dynamically adjusting these parameters, the system can optimize resource utilization while ensuring fair distribution, preventing resource starvation for lower-priority applications while maintaining high efficiency for critical applications.
2Reliability
If separate thread pools are created per application, then fairness in thread sharing is improved, but resource utilization efficiency deteriorates
Solution Approach 1:
The patent merges the benefits of separate thread pools with dynamic resource sharing by implementing a hierarchical structure where application-specific thread pools can borrow threads from a global pool when needed. This combining approach maintains fairness through dedicated pools while improving resource utilization through shared access, resolving the contradiction between fair sharing and efficient utilization.
Solution Approach 2:
The patent introduces dynamic thread pool management where the size and allocation of thread pools can change based on real-time system conditions and application priorities. This dynamic adjustment allows the system to optimize resource distribution continuously, improving utilization efficiency while maintaining fairness through adaptive resource allocation rather than static separate pools.
3Productivity
If continuous jobs are created from one application, then resource utilization is improved, but other applications suffer from resource starvation
Solution Approach 1:
The patent segments the system into isolated application thread pools that prevent any single application from continuously consuming all available threads. Each application's thread pool acts as an independent resource container, ensuring that even if one application continuously creates jobs, it cannot starve other applications of their allocated thread resources, thus preventing resource starvation while maintaining efficient utilization within each segment.
Solution Approach 2:
The patent implements feedback mechanisms that monitor thread pool utilization and job creation rates across applications. When one application approaches its thread pool limits or creates jobs at an excessive rate, the system adjusts resource allocation dynamically, providing feedback control that prevents resource starvation of other applications while maintaining high overall utilization through balanced resource distribution.
Data Source
AI summary
A method of scheduling tasks for a group of applications. Each application is associated with a separate task queue for identifying application tasks that are ready for execution. The method receives a request to place a task in an application task queue. The method places the task in the application task queue based on a set of intra application scheduling policies defined for the application. The method receives a request to identify the next task to execute. The method identifies an application queue from which from which the next task is to be executed. The application queue is identified based on a set of inter application scheduling policies defined for the plurality of applications. The method schedules a task from the identified application queue for execution based on the intra application scheduling policies.