Compiler GEMM Tiling Configuration Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning frameworks rely on hardware-specific operator libraries for General Matrix Multiplication (GEMM) optimization, which is mostly achieved through manual tuning, limiting performance on diverse hardware platforms and requiring efficient compiler-level optimization to harmonize AI algorithms with hardware.
Innovation Solution
The use of Markov Decision Process (MDP) based algorithms, specifically Greedy Best-First Search (G-BFS) and Neighborhood Actor Advantage Critic (N-A2C) methods, to explore the configuration space and find optimal tiling configurations for GEMM, reducing the search space and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual tuning is used for GEMM optimization in hardware-specific libraries, then performance on particular hardware can be optimized, but adaptability to diverse hardware platforms deteriorates
Solution Approach 1:
The patent introduces a configuration space as an intermediary layer between the high-level AI algorithm and the low-level hardware implementation. This configuration space includes parameters like tiling sizes, blocking factors, and loop fusion patterns that can be automatically adjusted to bridge the gap between algorithmic requirements and hardware capabilities, enabling portable optimization across different hardware platforms without manual tuning for each device
Solution Approach 2:
The patent systematically varies multiple configuration parameters (tiling dimensions, blocking factors, loop fusion patterns) to explore different execution strategies. By changing these parameters dynamically based on hardware characteristics, the system achieves optimal performance on diverse hardware platforms while maintaining a unified optimization framework, eliminating the need for hardware-specific manual tuning
2Reliability
If exhaustive search of configuration space is performed, then optimal GEMM configuration can be found, but search time and computational cost increase exponentially
Solution Approach 1:
The patent segments the configuration space into multiple levels by introducing hierarchical tiling and blocking factors. Instead of searching all possible configurations at once, the system divides the search into manageable levels (e.g., coarse-grained blocking factors and fine-grained tile sizes), allowing efficient exploration of the configuration space while maintaining optimality. This segmentation reduces the exponential search complexity to a manageable level
3Adaptability or versatility
If multiple tiling strategies are used for GEMM, then performance on different hardware can be optimized, but complexity of configuration space increases
Solution Approach 1:
The patent designs a universal configuration space framework that can accommodate multiple tiling strategies (e.g., block tiling, row-major tiling, column-major tiling) within a single unified structure. The same configuration parameters and search mechanisms work across different tiling approaches, allowing the system to explore diverse hardware-optimized strategies without increasing overall configuration space complexity. This multi-functionality enables portable optimization across hardware platforms while managing complexity
Data Source
AI summary
A system and method is provided for optimizing general matrix multiplication (GEMM) on target hardware by splitting matrices to be multiplied into tiles and formulating a tiling configuration search problem for matrices to be multiplied that explores a configuration search space to identify an optimal tiling configuration that minimizes running time on the target hardware for multiplication of matrices A (m×k) and B (k×n) on the target hardware for respective configuration states as a function of matrix parameters m, k, and n, and numbers of respective nested loops for each dimension m, k, and n, respectively. The optimal tiling configuration for the target hardware is obtained by implementing a Greedy Best-First-Search (GBFS) algorithm or a Neighborhood Actor Advantage Critic (N-A2C) algorithm that optimizes the running time for multiplication of the matrices on the target hardware, and the target hardware is configured and computations are run accordingly.