Neural Network Subgraph Compilation Time Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network compilation techniques lead to significant increases in compilation times due to optimizations that have negligible effects on runtime performance metrics, necessitating new methods to improve compilation efficiency.
Innovation Solution
Classifying neural network subgraphs as memory bound or compute bound and selectively suppressing optimizations that have negligible effects on runtime performance, using a compiler to determine the compute-to-load ratio and apply appropriate optimizations based on the classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If optimizations are applied to neural network subgraphs, then runtime performance is improved, but compilation time increases significantly
Solution Approach 1:
The patent segments the neural network into subgraphs and classifies them as memory-bound or compute-bound. This segmentation allows different optimization strategies to be applied selectively to each subgraph type, avoiding unnecessary optimizations on subgraphs where they would have negligible impact, thereby reducing overall compilation time while maintaining runtime performance.
Solution Approach 2:
The patent applies optimizations partially rather than universally. By suppressing optimizations on memory-bound subgraphs (where they have negligible effect) and applying them only to compute-bound subgraphs (where they provide significant benefit), the patent achieves the desired runtime performance improvement with minimal compilation time overhead.
2Productivity
If all optimizations are applied to improve runtime performance, then execution efficiency increases, but compilation complexity increases
Solution Approach 1:
The patent applies different optimization qualities to different parts of the neural network based on their characteristics. Compute-bound subgraphs receive aggressive optimizations to maximize execution efficiency, while memory-bound subgraphs receive minimal or no optimizations. This local differentiation reduces compilation complexity by avoiding uniform application of complex optimization logic across all subgraphs.
Solution Approach 2:
The patent changes the optimization parameter (whether to apply optimization) based on the subgraph type. By using the compute-to-load ratio as a parameter to determine optimization application, the patent simplifies the compilation process into clear decision rules rather than requiring complex analysis for every possible optimization scenario.
3Measurement precision
If compute-to-load ratio analysis is performed for each subgraph, then optimization accuracy improves, but compilation time increases
Solution Approach 1:
The patent performs compute-to-load ratio analysis partially, only for subgraphs where it will directly influence optimization decisions. By suppressing optimizations on memory-bound subgraphs regardless of detailed analysis, the patent reduces the number of subgraphs requiring intensive analysis, thereby maintaining optimization accuracy where needed while reducing overall compilation time.
Data Source
AI summary
Techniques for reducing a compilation time for compiling a neural network are disclosed. A description of a neural network is received by a compiler. A plurality of operators are identified based on the description of the neural network. A plurality of subgraphs are formed, each including one or more operators. For each subgraph, a performance factor is calculated based on a compute usage and a memory usage associated with the operators included in the subgraph. The performance factor is compared to a threshold. Based on the comparison, either the subgraph is classified as a compute bound subgraph and a set of memory optimizations are suppressed or the subgraph is classified as a memory bound subgraph and a set of compute optimizations are suppressed.