Compiler-Aware Neural Network Models for Hardware-Aligned Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models lack accurate performance prediction during training due to limited knowledge of compilation optimization information, leading to suboptimal deployment on target hardware.
Innovation Solution
Obtain compilation optimization information based on target hardware characteristics and use it to modify, retrain, or calibrate the neural network model, incorporating tiling and fusion configurations to create a compiler-aware model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the neural network model is trained without compilation optimization information, then the training process is simpler and faster, but the deployment performance on target hardware is suboptimal
Solution Approach 1:
The patent applies preliminary action by obtaining compilation optimization information from the target hardware compiler before training the neural network model. This information about hardware-specific optimizations (such as tiling, fusion, quantization) is used to pre-configure the model architecture and training parameters, so that when the model is deployed, it already aligns with the hardware's optimal execution patterns, thereby improving deployment performance without requiring complex post-processing
Solution Approach 2:
The patent implements feedback by using the compilation optimization information returned by the target hardware compiler to iteratively adjust the neural network model during training. The compiler's optimization suggestions feed back into the training process, allowing the model to be refined based on actual hardware performance characteristics, creating a closed-loop system that continuously improves deployment efficiency
2Productivity
If compilation optimization is applied to the neural network model, then deployment efficiency improves, but the model performance prediction during training becomes less accurate
Solution Approach 1:
The patent addresses this contradiction by dynamically changing parameters based on the training stage. During early training stages, the model uses original parameters for accurate performance prediction and learning. As training progresses, compilation optimization information is integrated to adjust parameters such as computation order, tiling factors, and fusion configurations, allowing the model to transition from theoretical accuracy to practical deployment efficiency while maintaining reasonable performance predictions
3Reliability
If the neural network model is modified based on compilation optimization information, then hardware alignment improves, but the training time increases
Solution Approach 1:
The patent applies partial action by selectively modifying only those portions of the neural network model that benefit most from compilation optimization. Instead of comprehensively redesigning the entire model, the approach focuses on key components such as specific operation layers, computation graphs, or critical paths that will yield the greatest hardware alignment improvement, thereby reducing the overall training time while still achieving significant hardware optimization
Data Source
AI summary
This disclosure provides a method and a system for constructing a neural network. Processing circuitry of the system obtains compilation optimization information of a compilation of a neural network model. The compilation optimization information indicates one or more modifications to the neural network model during the compilation of the neural network model. The one or more modifications are based on hardware information of a target hardware that the neural network model is to be deployed onto. The processing circuitry modifies the neural network model based on the one or more modifications indicated by the compilation optimization information, compiles the modified neural network model into a compiled neural network model, and deploys the compiled neural network model onto the target hardware.