Tensor IR Highering for Reconstructing Optimized RNN Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The unidirectional nature of the 'lowering' process in intermediate representation (IR) systems, particularly in compilers like LLVM IR, restricts the ability to convert from higher-level IRs to lower-level IRs, leading to inefficiencies in optimizing computations, especially for complex operations like RNN layers, preventing the use of optimized implementations from libraries like CUDNN or DNNL.
Innovation Solution
A method for intermediate representation 'highering' that detects and replaces low-level sub-graphs with higher-level layers within computational graphs, allowing for bidirectional transformation and enabling high-level optimizations, including the use of specialized hardware and libraries like CUDNN or DNNL.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unidirectional lowering process is used in intermediate representation systems, then implementation details and optimization strategies can be added progressively, but the ability to convert from lower-level IRs back to higher-level IRs is lost, preventing use of optimized library implementations
Solution Approach 1:
The patent inverts the traditional unidirectional lowering process by implementing bidirectional conversion capabilities. Specifically, it enables conversion from lower-level IRs (like LLVM IR) back to higher-level IRs (like MLIR), allowing optimized library implementations to be utilized. This is achieved through graph extraction and matching techniques that identify patterns in lower-level IR and reconstruct corresponding higher-level operations.
2Device complexity
If low-level IRs like LLVM IR are used, then detailed implementation control is achieved, but high-level mathematical optimizations become difficult to implement
Solution Approach 1:
The patent introduces an intermediary conversion process that bridges low-level and high-level IRs. By extracting computational graphs from low-level IR and matching them against patterns to reconstruct higher-level operations, the system enables mathematical optimizations at higher levels while maintaining the implementation control of low-level IRs. This intermediary step allows optimizations like ReLU→MaxPooling transformations that would be difficult to implement directly on low-level IR.
3Productivity
If bidirectional transformation between IR levels is enabled, then optimized library implementations can be utilized, but compilation complexity increases
Solution Approach 1:
The patent segments the bidirectional transformation process into distinct, manageable components: graph extraction from lower-level IR, pattern matching to identify higher-level operations, and reconstruction of higher-level computational graphs. This segmentation manages compilation system complexity by breaking down the transformation into modular steps, enabling optimized library implementations to be utilized effectively.
Data Source
AI summary
A method for intermediate representation highering includes detecting one or more types of higherable operations associated with one or more extracted sub-graphs or one or more hyperparameters within the one or more extracted sub-graphs. The one or more extracted sub-graphs are part of a computational graph associated with a first intermediate representation (IR) during a compiling process for converting source code to machine code. The method further includes replacing the one or more extracted sub-graphs with one or more higher level layers indicated by the higherable operations to generate a new computational graph.