Computation Graph Optimization via Partial Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI frameworks require costly and repetitive memory layout conversions to achieve peak performance, leading to increased execution times due to their inability to optimize data layouts according to hardware specifications, especially during training where these conversions are executed thousands of times.
Innovation Solution
The method involves identifying and splitting the computation graph into pre-evaluation and computation parts, with a wrapper performing transparent data layout mapping between default AI framework layouts and compute library layouts, allowing for pre-evaluation of input-independent parameters and dynamic reconfiguration of neural network parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data layout conversion is performed to achieve peak hardware performance, then computational efficiency is improved, but execution time increases due to repetitive conversions during training
Solution Approach 1:
The patent performs data layout conversion in advance during the graph optimization phase, before the actual training execution. By pre-converting the computation graph to use optimized memory layouts (such as blocking patterns for convolution operations), the system eliminates the need for repetitive layout conversions during training iterations, thus achieving peak hardware performance without the time penalty of repeated conversions.
2Productivity
If computation graph is optimized with hardware-specific layouts, then performance is improved, but system complexity increases due to multiple layout mappings
Solution Approach 1:
The patent segments the computation graph into different parts that can be independently optimized for specific hardware layouts. By dividing the graph into subgraphs that can be processed with hardware-specific optimizations (such as separate handling for convolution layers requiring blocked layouts versus other layers), the system achieves performance optimization while managing complexity through modular organization of layout mappings.
Data Source
AI summary
A method for optimizing a neural network includes identifying parameters of a computation graph of the neural network that depend on input data as a computation part, and parameters of the computation graph that are independent of the input data as a pre-evaluation part. The method splits the computation graph into the pre-evaluation part and the computation part, and generates and applies a wrapper that performs a transparent mapping of data layouts of the pre-evaluation part.