Compiler Data Streaming for Many Integrated Core Processors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Achieving high performance on manycore accelerators like Intel Xeon Phi coprocessors is challenging due to high data transfer overhead and inefficient handling of irregular memory accesses, leading to performance gaps compared to multicore CPUs.
Innovation Solution
The implementation of source-to-source compiler optimizations, including data streaming to overlap data transfer with computation, regularization of irregular memory accesses, and a shared memory mechanism for efficient transfer of large pointer-based data structures, to reduce memory footprint and optimize buffer usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data transfer is performed sequentially before computation, then data is fully available for processing, but data transfer overhead dominates execution time
Solution Approach 1:
The compiler performs preliminary actions by pre-computing and preparing data structures (such as computing inverse transforms) before the main computation loop. This allows the actual data transfer and computation to overlap, reducing the dominant data transfer overhead while ensuring data is fully available when needed.
Solution Approach 2:
The patent implements continuous useful action through overlapping data transfer with computation. The data transfer operation continues in the background while computation proceeds, eliminating idle waiting time and ensuring both data availability and time efficiency are improved simultaneously.
2Productivity
If buffers are allocated for each array, then data transfer can be optimized, but memory footprint on MIC increases
Solution Approach 1:
The patent applies universality by using a single shared buffer that serves multiple arrays sequentially. Instead of allocating dedicated buffers for each array, the same buffer is reused across different arrays, reducing the total memory footprint on the MIC while maintaining efficient data transfer through proper buffer management and timing.
3Loss of time
If kernel is launched for each data transfer, then data transfer can be asynchronous, but thread reuse is prevented and overhead increases
Solution Approach 1:
The patent implements self-service by having threads reuse themselves across multiple kernel launches. Instead of creating new threads for each asynchronous data transfer operation, existing threads are reused, which reduces the complexity of thread management and eliminates repeated kernel launch overhead while maintaining asynchronous data transfer capability.
Data Source
AI summary
Systems and methods for source-to-source transformation for compiler optimization for many integrated core (MIC) coprocessors, including identifying data dependencies in candidate loops and data elements used in each iteration for arrays, profiling candidate loops to find a proper number m, wherein data transfer and computation for m iterations take an equal amount of time, and creating an outer loop outside the candidate loop, with each iteration of the outer loop executing m iterations of the candidate loop. Data streaming is performed by determining optimum buffer size for one or more arrays and inserting code before the outer loop to create optimum sized buffers, overlapping data transfer between central processing units (CPUs) and MICs with the computation; reusing buffers to reduce memory employed on the MICs, and reusing threads on MICs to repeatedly launch kernels on the MICs for asynchronous data transfer.