Dynamic Thread Pool Sizing via Task Throughput Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for determining the optimal size of a thread pool in a server environment are inefficient, leading to suboptimal performance due to excessive thread creation and scheduling overhead, especially in fast-changing web environments, and result in uninterrupted increases in thread and task queue sizes even at high CPU usage.
Innovation Solution
A method for dynamically managing thread pool size by calculating representative task throughputs during different time sections and adjusting the maximum thread value based on predefined thresholds, ensuring optimal thread allocation and preventing thread queue saturation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a thread is created for each request in a conventional server, then the server can handle service requests effectively, but the time required for creating and discarding threads exceeds the time for processing requests, causing lower system performance
Solution Approach 1:
The patent pre-creates a pool of threads in advance before requests arrive, so that when requests come in, threads are already available and do not need to be created at that moment. This eliminates the time loss associated with thread creation during request processing.
Solution Approach 2:
The patent makes threads reusable by having them return to the thread pool after completing a task, allowing the same thread to handle multiple different requests over time. This multi-functionality of threads significantly reduces the frequency of thread creation and destruction operations.
2Productivity
If threads are created for each request when requests rise abruptly, then all requests can be processed, but the number of threads rises excessively, necessitating scheduling which results in system overhead
Solution Approach 1:
The patent implements dynamic adjustment of the thread pool size based on the current workload and system state. The thread pool can expand when requests increase and contract when requests decrease, optimizing the balance between handling capacity and management overhead.
Solution Approach 2:
The patent uses feedback mechanisms to monitor task queue depth, CPU usage, and thread pool performance, adjusting the thread pool size accordingly. This feedback loop prevents excessive thread creation by responding to actual system conditions rather than blindly creating threads for every request.
3Productivity
If the maximum thread value is increased to handle more requests, then request processing capacity improves, but thread pool size management becomes complex and difficult to optimize
Solution Approach 1:
The patent implements self-service mechanisms where the thread pool automatically adjusts its own size based on monitored performance metrics and system conditions. This eliminates the need for manual configuration and optimization of thread pool parameters, reducing management complexity while maintaining optimal performance.
Solution Approach 2:
The patent dynamically changes thread pool parameters such as maximum thread value, minimum thread value, and adjustment thresholds based on observed system behavior and performance data. This allows the system to adapt to changing conditions without requiring manual reconfiguration.
4Device complexity
If thread pool size is fixed in advance, then thread management is simple, but the system cannot adapt to fast-changing web environments and performance optimization is difficult
Solution Approach 1:
The patent transforms the static, fixed thread pool into a dynamic structure that automatically adjusts its size based on real-time system conditions and workload characteristics. This dynamic behavior enables the system to adapt to changing environments while maintaining relatively simple management through automated control.
Data Source
AI summary
A method for managing a thread pool size dynamically using elemental task throughputs is provided. The method includes steps of: a computing device (a) calculating a first representative task throughput corresponding to a first group including elemental task throughputs during the first time section, and then increasing a maximum thread value for the first time section and setting it as a maximum thread value for the second time section; (b) calculating a second representative task throughput corresponding to a second group including elemental task throughputs during the second time section; and (c) comparing the first and the second representative task throughput and then, if the latter is greater than the former by an amount equal to or greater than a preset first threshold value, increasing the maximum thread value for the second time section and setting it as a maximum thread value for a third time section.