Compiler Tile Size Optimization via Build Cost Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tiling methods in software optimization lack effective heuristics for dynamic memory allocation, particularly in managing large loop nests and multi-dimensional arrays, leading to inefficient data transfers and cache overflow issues.
Innovation Solution
A method is introduced that formulates tiling as an optimization problem using a build cost model to minimize dynamic memory allocation transfers, allowing for flexible tile shapes and accommodating both static and dynamic loop bounds and data shapes, with a focus on minimizing host-device and device-host data transfers within cache constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If tiling is performed to improve data access locality, then data locality is improved, but device complexity increases due to manual DMA insertion requirements
Solution Approach 1:
The system performs self-service by automatically inserting DMA operations and computing tile sizes without manual intervention. The build cost model autonomously evaluates computation configurations and determines optimal tiling strategies, eliminating the need for manual DMA insertion while maintaining improved data locality.
Solution Approach 2:
The system changes parameters by dynamically computing tile sizes based on the build cost model rather than using fixed or manually specified tile sizes. The model evaluates multiple computation configurations with varying tile dimensions and selects the optimal configuration that minimizes data transfer costs while improving data locality.
2Loss of energy
If build cost model evaluation is performed to optimize data transfer cost, then data transfer cost is reduced, but computation time increases
Solution Approach 1:
The system performs preliminary action by pre-computing and caching the build cost model results during the compilation phase. The tile size computation and build cost evaluation are performed ahead of runtime execution, allowing the optimized computation configuration to be applied without incurring the evaluation overhead during actual data processing.
Solution Approach 2:
The system applies partial action by evaluating only the necessary computation configurations required for the specific computation graph rather than exhaustively searching all possible tile size combinations. The build cost model selectively evaluates configurations based on the specific data access patterns and computational requirements, reducing unnecessary computation time.
3Productivity
If tile size is increased to reduce number of DMA instructions, then DMA instruction count is reduced, but cache overflow occurs
Solution Approach 1:
The system implements feedback by using the build cost model to evaluate whether a proposed tile size configuration would exceed cache capacity. The model incorporates cache size constraints and provides feedback to adjust tile sizes accordingly, ensuring that larger tiles reduce DMA instructions without causing cache overflow by continuously monitoring and adapting to cache capacity limitations.
Data Source
AI summary
There is provided methods and devices for computing a tile size for software optimization. A method includes receiving, by a computing device, information indicative of one or more of a set of loop bounds and a set of data shapes; processing, by the computing device, the information to determine a computation configuration based on the obtained information, the computation configuration implementable by a compiler, said processing including evaluating at least the computation configuration based on a build cost model, the build cost model representative of a data transfer cost and a data efficiency of the computation configuration; and transmitting, by the computing device, instructions directing the compiler to implement the computation configuration.