Distributed Application Server Thread Management for Adaptive Batch Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing batch processing in distributed application servers is inefficient due to limited number of batch servers and processing threads, leading to resource inefficiencies, time delays, and missed service level agreements (SLAs).
Innovation Solution
A computing platform with a batch queue thread manager initiates multiple process threads to handle batch servicing requests, dynamically throttling the number of threads based on CPU power, memory, and data volume to optimize processing, while simultaneously handling individual requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed number of batch servers and processing threads are configured, then system stability is maintained, but processing efficiency and resource utilization deteriorate
Solution Approach 1:
The patent implements dynamic thread management where the batch queue thread manager continuously monitors system conditions (CPU utilization, memory availability, queue depth) and adjusts the number of active processing threads in real-time. This allows the system to transition from a static fixed-thread configuration to a dynamic adaptive configuration, maximizing processing efficiency during high-load periods while reducing resource consumption during low-load periods.
Solution Approach 2:
The system incorporates feedback mechanisms through performance monitors that track processing metrics and feed this information back to the batch queue thread manager. This feedback loop enables the manager to make informed decisions about thread creation and termination, adjusting the thread pool size based on actual system performance and workload conditions rather than relying on predetermined fixed configurations.
2Speed
If more processing threads are initiated to handle batch requests, then processing speed increases, but resource consumption and system overload increase
Solution Approach 1:
The batch queue thread manager dynamically adjusts the number of active processing threads based on real-time system conditions. When CPU utilization and memory consumption are within acceptable thresholds, the manager initiates additional threads to increase processing speed. When resource consumption approaches limits, the manager terminates excess threads to prevent system overload, creating a dynamic balance between processing speed and resource usage.
Solution Approach 2:
The system changes operational parameters (number of active threads) based on system state. The batch queue thread manager monitors resource utilization parameters and adjusts the thread count parameter accordingly, increasing threads when resources are abundant and decreasing threads when resources are constrained, thereby optimizing the balance between processing speed and resource consumption.
3Reliability
If batch processing uses a limited number of threads, then system stability is maintained, but service level agreements (SLAs) are missed
Solution Approach 1:
The dynamic thread management system responds to increasing workload by activating additional processing threads, allowing the system to maintain SLA compliance during peak periods. The batch queue thread manager monitors queue depth and processing delays, and when thresholds indicate potential SLA violations, it initiates more threads to reduce processing time and meet service level requirements.
4Adaptability or versatility
If individual service requests are processed separately from batch requests, then request handling flexibility increases, but system resource fragmentation increases
Solution Approach 1:
The batch queue thread manager serves multiple functions: it manages both batch processing threads and individual request threads, monitors system resources, and dynamically adjusts thread allocation for different request types. This universal thread management approach allows the system to handle diverse request types (batch and individual) through a single coordinated mechanism, maintaining flexibility while avoiding resource fragmentation through unified resource control.
Data Source
AI summary
Aspects of the disclosure relate to batch processing at distributed servers. A computing platform may receive a first web service request, which may be a request for batch servicing. The computing platform may initiate a batch queue thread manager to process the first web service request, which may initiate a plurality of process threads, each configured to process, at any one time, a single job of the batch servicing. The computing platform may process, using the batch queue thread manager, the first web service request, which may include, by each of the plurality of process threads: 1) processing a first job of the batch servicing, and 2) processing, after completion of the first job, at least a second job of the batch servicing. After completion of the processing of the first web service request, the computing platform may output a response to the first web service request.