Compiler Logic for Tensor Memory Layout Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in optimally determining and resolving conflicts in tensor memory layouts across different processors, leading to suboptimal application performance and incompatibilities in kernel-based compilation flows.
Innovation Solution
A compiler analysis and transformation pass that automatically determines required physical layouts for tensors by processing a dataflow graph, using memory layout functions to generate producer and consumer layouts, and inserting conversion operations to resolve layout conflicts, ensuring compatibility and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a compiler analysis and transformation pass is implemented to automatically determine tensor memory layouts, then memory layout compatibility and application performance are improved, but device complexity increases
Solution Approach 1:
The patent applies preliminary action by performing compile-time analysis to determine memory layouts before execution. The compiler analyzes the dataflow graph and operation kernels in advance to compute expected producer and consumer memory layouts, storing them in lookup tables. This pre-computation eliminates the need for runtime layout determination, improving reliability while the complexity is confined to the compile phase rather than execution time.
Solution Approach 2:
The patent introduces memory layout conversion operations as intermediaries to resolve conflicts between expected and current memory layouts. When a conflict is detected, the system inserts conversion operations (such as transposes or reshapes) between the producer and consumer operations. These intermediary operations mediate the incompatibility, ensuring data can be correctly transformed between different memory layouts without requiring complex runtime analysis.
2Reliability
If memory layout conflicts are resolved by inserting conversion operations, then layout compatibility is improved, but device complexity and execution time increase
Solution Approach 1:
The patent extracts the memory layout conflict resolution logic from the main execution flow by performing it during the compile time analysis phase. The expected producer and consumer memory layouts are pre-computed and stored in lookup tables indexed by operation type. During execution, the system simply retrieves these pre-determined layouts, avoiding the need to re-analyze conflicts at runtime. This extraction of complex analysis to the compile phase reduces execution-time complexity.
Solution Approach 2:
The patent changes the parameter of when memory layout determination occurs from runtime to compile time. By shifting this critical analysis to the compilation phase, the system can perform exhaustive analysis of the dataflow graph and operation kernels to determine optimal memory layouts. The execution phase then simply uses these pre-determined layouts, reducing the computational burden during runtime while maintaining compatibility.
3Productivity
If compile time analysis is performed to determine memory layouts, then execution efficiency is improved, but compilation time and processing overhead increase
Solution Approach 1:
The patent applies preliminary action by performing memory layout analysis during the compile phase before execution. The compiler computes expected producer and consumer memory layouts for all operations in the dataflow graph and stores them in lookup tables. This pre-computation enables efficient execution by avoiding runtime analysis, and while it does increase compilation time, the trade-off is achieved through optimized analysis algorithms that process the dataflow graph systematically.
Data Source
AI summary
The technology disclosed relates to automatically assigning and optimizing the physical memory layouts of all intermediate dense tensor data in a program. The technology disclosed is an implementation of a compiler analysis and transformation pass which automatically determines required physical layouts in light of kernel operation and performance requirements. The proposed solution also inserts physical layout conversion operations where necessary in cases of unresolvable incompatibilities. The pass takes as input a program acyclic dataflow graph and a set of physical layout constraints for every known operation.