Dynamic Thread Allocation via Runtime Buffer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for automatically controlling run-time parallelization of software applications on multi-core processors are inefficient, particularly in over-subscribed systems, as they often require costly system queries to determine the optimal number of threads, leading to performance limitations.
Innovation Solution
A system and method that utilize a buffer to store demand information and allocation information, allowing the software application or runtime library to dynamically adjust the number of threads for parallel execution, bypassing expensive system calls by using asynchronous load and store instructions, and a supervising entity to compute and store allocation information for later access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the application queries the operating system to determine the optimal number of threads, then the thread count can be optimized for performance, but the execution time increases due to costly system calls
Solution Approach 1:
The runtime library performs preliminary actions by pre-computing thread allocation decisions and caching them in a buffer before the parallel region executes. The supervising entity prepares allocation information in advance based on demand information stored in the buffer, eliminating the need for costly real-time OS queries during parallel execution.
Solution Approach 2:
A buffer acts as an intermediary between the application and the operating system. The buffer stores demand information from the application and allocation information from the supervising entity, mediating communication and eliminating direct system calls during parallel region execution.
2Productivity
If gang scheduling is used to schedule all threads as a single group, then performance is improved on multi-core systems, but the system becomes over-subscribed when there are more ready threads than available hardware threads
Solution Approach 1:
The system dynamically adjusts the number of threads allocated to a parallel region based on real-time conditions. The runtime library can modify the thread count dynamically by updating allocation information in the buffer, allowing the system to adapt between gang scheduling (for performance) and flexible thread allocation (for system resource management).
Data Source
AI summary
A system and method for automatically controlling run-time parallelization of a software application. A buffer is allocated during execution of program code of an application. When a point in program code near a parallelized region is reached, demand information is stored in the buffer in response to reaching a predetermined first checkpoint. Subsequently, the demand information is read from the buffer in response to reaching a predetermined second checkpoint. Allocation information corresponding to the read demand information is computed and stored the in the buffer for the application to later access. The allocation information is read from the buffer in response to reaching a predetermined third checkpoint, and the parallelized region of code is executed in a manner corresponding to the allocation information.