Polyhedral Compiler Optimization for Heterogeneous Memory Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current compilers struggle to efficiently optimize source code for complex heterogeneous architectures with multiple types of memories, leading to suboptimal performance and increased communication costs due to the tradeoff between parallelism and locality, and the inability to effectively exploit memory reuse opportunities.
Innovation Solution
A method and apparatus that optimize source code by replacing main memory copies with local memory copies, utilizing a polyhedral representation to transform the code and generate tiled variants, and parameterizing data movement operations to reduce communication costs and memory traffic, thereby exploiting communication reuse and improving data locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallelism is increased to execute more concurrent operations, then computational throughput is improved, but memory locality deteriorates causing increased communication costs
Solution Approach 1:
The patent applies segmentation by dividing the computation into independent tiles that can be executed in parallel. Each tile operates on localized data subsets, enabling concurrent execution while maintaining data locality within each tile. This resolves the contradiction by organizing parallel work units to minimize cross-tile communication.
Solution Approach 2:
The patent introduces a tile dimension to the computation space, transforming the problem from a flat parallel execution model to a multi-dimensional tiled structure. This additional dimension allows data to be reused across multiple iterations within a tile, improving locality while maintaining parallelism across different tiles.
2Speed
If data is kept in local memory to improve access speed, then memory access time is reduced, but memory usage increases
Solution Approach 1:
The patent applies partial action by loading only the necessary portion of data into local memory - specifically, one tile's worth of data at a time rather than the entire dataset. This partial loading strategy improves access speed for active computations while keeping memory usage bounded and manageable.
Solution Approach 2:
The patent uses preliminary action by pre-loading data into local memory before it is needed for computation. The tiling mechanism ensures that data is transferred to local memory in advance of when it will be accessed, reducing memory access time during the actual computation phase.
3Ease of operation
If communication operations are performed frequently to maintain data in local memory, then data availability is improved, but communication overhead increases
Solution Approach 1:
The patent achieves continuity of useful action by organizing computations into continuous tile iterations that reuse the same local data. Once data is loaded into local memory for one tile, it remains available for subsequent tiles that access the same data, eliminating the need for repeated communication operations and reducing overhead.
Data Source
AI summary
Methods, apparatus and computer software product for source code optimization are provided. In an exemplary embodiment, a first custom computing apparatus is used to optimize the execution of source code on a second computing apparatus. In this embodiment, the first custom computing apparatus contains a memory, a storage medium and at least one processor with at least one multi-stage execution unit. The second computing apparatus contains at least one local memory unit that allows for data reuse opportunities. The first custom computing apparatus optimizes the code for reduced communication execution on the second computing apparatus. This Abstract is provided for the sole purpose of complying with the Abstract requirement rules. This Abstract is submitted with the explicit understanding that it will not be used to interpret or to limit the scope or the meaning of the claims.