Workload Adjuster for Speculative Loop Parallelization Startup Cost
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face inefficiencies in multithreaded operations due to significant startup costs for non-main threads in speculative automatic parallelization, which negatively impact overall performance.
Innovation Solution
A workload adjuster analyzes loops to determine if they can be parallelized and estimates startup costs for non-main threads, migrating workload to reduce total execution time by distributing iterations between main and non-main threads based on calculated metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If speculative automatic parallelization is used to parallelize loop execution, then productivity is improved, but loss of time increases due to significant startup costs for non-main threads
Solution Approach 1:
The patent applies preliminary action by pre-warming hardware structures (private cache and TLB) with necessary data before non-main threads begin execution. The main thread performs preliminary cache warming by executing loop iterations first, so that when non-main threads start, the required data is already in the cache, reducing their startup cost and improving overall parallelization efficiency
Solution Approach 2:
The patent merges the workload of the main thread and non-main threads by allowing the main thread to contribute to warming the cache for all threads. Instead of each thread independently incurring full startup costs, the main thread's preliminary execution benefits all threads by pre-loading data into the private cache and TLB, effectively sharing the warmup burden
2Reliability
If the workload for speculative loop threads is kept small to avoid transaction failure, then reliability is improved, but productivity decreases due to insufficient work granularity
Solution Approach 1:
The patent applies dynamics by making the parallelization strategy adaptive based on detected dependencies. The system dynamically determines whether to use speculative parallelization, traditional parallelization, or sequential execution based on runtime analysis of loop trip counts and dependency detection, allowing the workload size and parallelization approach to adjust according to actual program behavior rather than being fixed at compile time
Data Source
AI summary
Methods and apparatus provide for a workload adjuster to estimate the startup cost of one or more non-main threads of loop execution and to estimate the amount of workload to be migrated between different threads. Upon deciding to parallelize the execution of a loop, the workload adjuster creates a scheduling policy with a workload for a main thread and workloads for respective non-main threads. The scheduling policy distributes iterations of a parallelized loop to the workload of the main thread and iterations of the parallelized loop to the workloads of the non-main threads. The workload adjuster evaluates a start-up cost of the workload of a non-main thread and, based on the start-up cost, migrates a portion of the workload for that non-main thread to the main thread's workload.