Neural Network Training Time Prediction via Execution Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for predicting the training time of neural network models are inefficient due to their reliance on heuristic approaches and oversimplified models, which fail to account for various dynamic and static parameters involved in the training process.
Innovation Solution
The proposed solution involves generating a first execution graph representing a distributed learning process of a neural network model, and then creating a second execution graph based on the first graph and a target operator selected for a distributed learning process. This allows for the prediction of training time by simulating the execution time of hardware kernels performing the target operator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If heuristic approaches and oversimplified models are used for predicting training time, then the prediction process is simple and fast, but the prediction accuracy is poor
Solution Approach 1:
The patent segments the training process into discrete computation nodes and communication nodes, representing the training graph as a directed acyclic graph where each node corresponds to a specific operation. This segmentation allows complex training time prediction to be broken down into manageable components, each can be analyzed and predicted independently, then aggregated to obtain the total training time prediction.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and storing execution time characteristics of hardware kernels before actual training. The system profiles and measures execution times of various operators and stores them in a database, so that during training time prediction, these pre-measured values can be directly retrieved and used without performing time-consuming measurements during the actual prediction process.
2Adaptability or versatility
If existing prediction methods are used, then the system is simple to implement, but it fails to account for multiple dynamic and static parameters
Solution Approach 1:
The patent creates a universal prediction framework that can handle multiple types of parameters (static and dynamic) through a single execution graph representation. The system can accommodate different hardware configurations, network topologies, and training settings by using the same graph-based model, making it universally applicable to various training scenarios without requiring separate specialized models for each case.
Solution Approach 2:
The patent incorporates dynamic parameters into the prediction model by allowing the execution graph to adapt to different training conditions. The system can dynamically adjust the graph structure and parameter values based on actual training configurations, enabling accurate predictions across varying hardware setups, data sizes, and model complexities without requiring manual reconfiguration of the prediction system.
3Measurement precision
If full profiling of all operators is performed, then prediction accuracy is improved, but profiling overhead increases significantly
Solution Approach 1:
The patent extracts only the essential execution time information needed for prediction by selectively profiling and measuring only the critical operators and computation nodes in the training graph. Instead of profiling every single operator exhaustively, the system identifies and measures only the key operations that have the most significant impact on overall training time, extracting the necessary information to achieve accurate predictions without the overhead of complete profiling.
Solution Approach 2:
The patent creates a simplified copy or representation of the training process in the form of an execution graph that captures the essential structure and dependencies without requiring full execution of all operations. This graph copy allows the system to analyze and predict training time by working with the abstracted representation rather than performing complete profiling of all operators, significantly reducing the overhead while maintaining prediction accuracy.
Data Source
AI summary
Disclosed is a method and apparatus for predicting a training time. The method includes generating a first execution graph representing a distributed learning process of a neural network model, wherein the first execution graph includes a computation node representing a repeated computation of the neural network model based on a parallelism technique; generating a second execution graph based on the first execution graph and a target operator corresponding to the computation node, wherein the second execution graph includes an execution time of a hardware kernel performing the target operator; and predicting a training time of the neural network model based on the second execution graph.