Tensor Compiler Loop Scheduling With ML-Guided Transformation Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing microkernel-based JIT tensor compilers face challenges in identifying an efficient sequence of loop transformations that minimize data movement and maximize parallelism within a limited compile time budget, due to the complexity of modern architectures and the impracticality of existing optimization methods like polyhedral compilation, autotuning, and AI models.
Innovation Solution
Employing machine learning techniques to predict processor performance and memory misses, using trained models to generate and select optimal loop transformation sequences in tensor compilers, leveraging existing schedule trees and continuous training to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If polyhedral compilation or autotuning methods are used to optimize loop transformations, then manufacturing precision of the optimization sequence is improved, but loss of time increases due to excessive compile time
Solution Approach 1:
The system performs preliminary actions by pre-generating schedule trees that represent possible loop transformation sequences before actual compilation. These pre-computed schedules are stored and can be quickly retrieved and applied, avoiding the need to perform exhaustive polyhedral compilation or autotuning during the actual compilation process. This preliminary preparation resolves the contradiction by providing accurate optimization sequences without the excessive compile time penalty.
Solution Approach 2:
The optimization process is segmented into distinct phases: generating schedule trees representing different transformation sequences, evaluating these schedules using performance models, and selecting the optimal schedule. This segmentation allows the system to avoid the computationally expensive monolithic optimization approaches while maintaining precision through systematic evaluation of segmented transformation options.
2Productivity
If exhaustive loop transformation sequences are explored to maximize parallelism and minimize data movement, then productivity of the tensor computation is improved, but loss of time increases due to limited compile time budget
Solution Approach 1:
The system applies partial action by generating and evaluating only a subset of possible loop transformation sequences through schedule trees, rather than exhaustively exploring all possible transformations. The schedule tree generation incorporates performance modeling to prioritize and evaluate only the most promising transformation sequences, achieving high tensor computation efficiency without the prohibitive compile time cost of exhaustive exploration.
Solution Approach 2:
Schedule trees are pre-generated to represent likely optimal transformation sequences based on performance models. This preliminary action allows the compiler to quickly select efficient transformation sequences without performing exhaustive exploration during the actual compilation, thus maintaining high productivity while respecting the compile time budget.
3Manufacturing precision
If complex optimization methods are applied to handle modern architecture intricacies, then manufacturing precision of the transformation sequence is improved, but device complexity increases
Solution Approach 1:
The system introduces an intermediary representation called schedule trees that mediates between the source code and the final optimized code. These schedule trees encapsulate the complex transformation sequences in a structured, manageable form that is easier to generate, evaluate, and apply. This intermediary simplifies the compiler's task while maintaining the ability to handle modern architecture intricacies, thus improving transformation optimality without proportionally increasing device complexity.
Solution Approach 2:
The complex optimization problem is segmented into manageable components through schedule trees, where each node represents a specific transformation sequence. This segmentation allows the compiler to handle complexity in a systematic way, evaluating and selecting optimal sequences without being overwhelmed by the full complexity of modern architectures, thus improving precision while controlling device complexity.
Data Source
AI summary
A tensor compiler for DNNs can use trained models for optimizing loop nests in IRs. A loop nest may include loops. A loop may be nested within another loop. A loop specifies a tensor operation to be repeatedly executed by a processor. The tensor compiler generates a schedule tree for an IR. The schedule tree includes schedules arranged based on hierarchies. The tensor compiler may select a schedule from the schedule tree by using a trained model that can predict performances of the processor executing the tensor operation in accordance with the IR transformed using the schedules. The tensor compiler then transforms the loop nest with the selected schedule and generates an implementation to be run by the processor. The tensor compiler may instrument the implementation for facilitating receipt of runtime performance information of the processor. The tensor compiler may use the runtime performance information to further train the model.