Interprocedural Prefetching for Shared Cache Warming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speculative automatic parallelization in transactional memory systems faces performance issues due to significant startup costs for non-main threads, which are exacerbated by hardware limitations, leading to inefficiencies in workload distribution and cache warming.
Innovation Solution
The implementation of interprocedural prefetching, where prefetch instructions are inserted into the main thread to pre-load data into a shared cache for both the main and non-main threads, reducing cache misses and startup costs by warming up the private cache and translation lookahead buffer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If speculative automatic parallelization is implemented with small workload granularity to avoid transaction failure, then reliability is improved, but startup cost increases significantly due to cold cache and TLB
Solution Approach 1:
The system performs preliminary actions by inserting prefetch instructions into the main thread before the parallelized loop executes. These prefetch instructions load data into the shared cache in advance, so when non-main threads start executing, the data is already available in the cache, eliminating the cold cache penalty and reducing startup cost.
Solution Approach 2:
The shared cache acts as an intermediary between main memory and the private caches of multiple threads. The main thread prefetches data into the shared cache, which then serves as a warm buffer for non-main threads, mediating the data access and reducing the impact of cold starts.
2Productivity
If the granularity of parallel region is increased to reduce startup cost impact, then productivity is improved, but transaction failure risk increases due to hardware store limits
Solution Approach 1:
Data is prefetched into the shared cache before the parallelized loop begins, so when non-main threads access memory, they benefit from warm cache hits rather than cold cache misses. This reduces the effective workload time and allows larger parallel regions to complete within transaction store limits, improving both productivity and reliability.
3Productivity
If non-main threads are created to share workload in parallelized loops, then productivity is improved, but cache warming overhead increases due to cold private cache and TLB
Solution Approach 1:
The main thread performs preliminary data loading into the shared cache before non-main threads are created or before they begin execution. This preliminary action ensures that when non-main threads access data, it is already present in the shared cache, eliminating the need for them to warm their private caches and TLBs from cold state, thus reducing energy overhead.
Solution Approach 2:
The system merges the data loading function into the main thread's prefetch operations, combining the benefits of parallel execution with the efficiency of shared cache utilization. Instead of each thread independently warming its cache, the main thread prefetches data that benefits all threads, reducing redundant cache warming operations.
Data Source
AI summary
A computing system has an amount of shared cache, and performs runtime automatic parallelization wherein when a parallelized loop is encountered, a main thread shares the workload with at least one other non-main thread. A method for providing interprocedural prefetching includes compiling source code to produce compiled code having a main thread including a parallelized loop. Prior to the parallelized loop in the main thread, the main thread includes prefetching instructions for the at least one other non-main thread that shares the workload of the parallelized loop. As a result, the main thread prefetches data into the shared cache for use by the at least one other non-main thread.