Lazy Compilation and Kernel Fusion for Dynamic Computation Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques do not provide effective optimization for dynamic computation graphs, where operations are determined dynamically during processing, leading to inefficiencies in execution.
Innovation Solution
A lazy compilation and kernel fusion system that queues operations until needed, optimizes them by combining, reducing, or deleting operations, and generates just-in-time kernels for improved processing speed, particularly utilizing a framework that translates user code into expression graphs and applies optimization rules to create a transformation graph for efficient GPU or CPU processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If operations are determined dynamically during processing in dynamic computation graphs, then the system can handle variable operations based on data contents, but the execution speed and efficiency deteriorate due to lack of optimization opportunities
Solution Approach 1:
The system performs preliminary actions by translating user code into an expression graph and queuing operations before execution. By preparing the computation graph structure in advance and identifying optimization opportunities beforehand, the system can apply optimizations without sacrificing dynamic operation determination during actual execution
Solution Approach 2:
The system dynamically determines which operations to perform based on data contents while maintaining optimization capabilities through lazy compilation. The computation graph is built dynamically but optimized using iterative application of optimization rules and kernel fusion techniques that adapt to the specific operation sequence generated
2Productivity
If operations are optimized prior to execution in static computation graphs, then execution speed improves, but the system loses the ability to handle dynamic operations determined during processing
Solution Approach 1:
The system uses lazy compilation to dynamically generate and optimize computation graphs at runtime based on actual operation sequences. This allows the system to maintain high execution speeds through optimization while adapting to different operation sequences determined during processing, combining benefits of both static and dynamic approaches
Solution Approach 2:
The system changes the parameter of compilation timing from static (pre-compilation) to dynamic (lazy compilation at runtime). This allows optimization to be applied adaptively based on the actual operation sequence generated during execution, maintaining both speed and flexibility
3Manufacturing precision
If operations are optimized iteratively with multiple rules applied, then optimization quality improves, but the time required for optimization increases
Solution Approach 1:
The system performs preliminary translation of user code into expression graphs and queues operations before applying optimization rules. This preliminary structuring of operations enables more efficient optimization by organizing the computation flow in advance, reducing the time required for iterative rule application while maintaining high optimization quality
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for lazy compilation and kernel fusion in dynamic computation graphs. One of the operations is performed by generating an input graph based on translation of user code into an expression graph. The expression graph represents control flow dependencies of operations of the generated input graph. Optimization of the input graph is then performed by iterative application of optimization rules to the input graph. An optimized version of the input graph results from the application of the optimization rules. A transformation graph then is generated by comparing changes made from the original input graph to the final optimized version of the input graph. The transformation graph provides a blueprint such that the system may recreate the optimization of a similarly structured later generated input graph without having to reapply the optimization rules. During the optimization of the input graph, the system may also generate fused blocks of just-in-time operations which the system may later optimize for parallel processing on one or more graphic processing units, and/or hardware accelerated computation units.