Application-Level Thread Dispatcher for Burst Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing operating system-level threads are limited by operating system constraints, restricting application flexibility in partitioning work and requiring excessive system thread usage, which hinders efficient application-level processing and throughput.
Innovation Solution
An application-level thread dispatcher dynamically manages application-level pseudo threads and operating system threads across multiple CPUs, allowing for granular thread dispatching and concurrent processing, thereby avoiding excessive system thread usage and optimizing throughput during burst periods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If operating system-level threads are used to perform application processing, then CPU processing cycles can be shared among multiple threads, but the operating system constraints limit application flexibility in partitioning work and require excessive system thread usage
Solution Approach 1:
The patent segments thread management into two distinct levels: application-level pseudo-threads for fine-grained work partitioning and operating system-level threads for resource management. This segmentation allows the application to independently divide work into smaller units without being constrained by OS thread limits, while the OS manages only the necessary system threads, thereby improving flexibility and reducing overall thread complexity.
Solution Approach 2:
The patent introduces an intermediary layer called 'application-level pseudo-threads' that mediates between the application's work partitioning needs and the operating system's thread management. These pseudo-threads act as virtual representations of work units that can be mapped to actual OS threads dynamically, allowing flexible work partitioning without requiring proportional increases in system thread usage.
2Productivity
If excessive system thread usage is employed to handle burst periods, then throughput can be improved, but system costs increase
Solution Approach 1:
The patent implements dynamic thread allocation where the number and type of threads are adjusted based on workload characteristics. During normal operation, fewer system threads are used, but the system can dynamically create additional pseudo-threads and map them to system threads during burst periods to maintain high throughput. This dynamic adaptation allows the system to achieve high productivity only when needed, reducing overall system resource consumption and costs.
Solution Approach 2:
The patent changes the parameter of thread weight classification into three categories (full-weight, lighter-weight, and pseudo-threads) to optimize resource allocation. By adjusting which type of thread is created based on workload intensity, the system can maintain high throughput during burst periods without proportionally increasing all thread types, thereby reducing overall system costs while preserving productivity capability.
3Ease of operation
If application-level pseudo threads are implemented within the main full-weight thread, then application-controlled thread management is achieved, but the main thread must manage multiple concurrent execution contexts
Solution Approach 1:
The patent applies nesting by placing application-level pseudo-threads within the context of the main full-weight thread. The main thread acts as a container that manages multiple pseudo-threads as nested execution contexts. This nesting structure allows the application to control thread management at a higher level while the main thread handles the complexity of concurrent execution, effectively managing ease of operation without proportionally increasing visible complexity.
Solution Approach 2:
The patent segments thread management responsibilities: the main full-weight thread handles OS-level scheduling and resource management, while application-level pseudo-threads handle work partitioning and execution control. This segmentation of management functions allows the main thread to maintain application-controlled thread management without bearing the full burden of managing all concurrent execution contexts directly, thereby reducing effective complexity.
Data Source
AI summary
An application-level thread dispatcher that operates in a main full-weight thread allocated to an application is established. The application-level thread dispatcher initializes a group of application-level pseudo threads that operate as application-controlled threads within the main full-weight thread allocated to the application. The application-level thread dispatcher evaluates run-time performance of the application.