Neural Network Configuration Optimization via Hybrid Policy Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for optimizing neural networks are manual, time-consuming, and inefficient, often focusing on a single objective and struggling to find optimal configurations due to the large number of possible settings for tensor tiling and layer fusion.
Innovation Solution
A neural network optimization method that uses a combination of learning-based and population-based algorithms to automatically find optimal configurations for different neural networks, generating multiple configuration candidates and selecting the best one for compilation into an optimized neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If grid searching is used to find optimal configuration among multiple configurations for tensor tiling and layer fusion, then the optimal configuration can be found, but the process becomes quite difficult and time-consuming due to the large number of configurations
Solution Approach 1:
The patent replaces the mechanical grid searching approach with a learning-based algorithm that uses neural networks to predict optimal configurations. Instead of systematically iterating through all possible configurations, the system trains a neural network model to learn the mapping between neural network structures and optimal tensor tiling/layer fusion settings, significantly reducing optimization time while maintaining accuracy.
Solution Approach 2:
The patent creates a virtual model (neural network) that copies and learns from the complex configuration space. The neural network model serves as a surrogate that can quickly predict optimal configurations without actually evaluating all possible settings, effectively copying the optimization task into a more manageable form.
2Ease of operation
If manual setting of tensor tiling and layer fusion is performed, then configuration can be adjusted, but the process is quite time-consuming and inefficient
Solution Approach 1:
The system performs self-optimization by automatically generating and selecting optimal configurations through the learning-based algorithm. Instead of requiring manual intervention for each configuration adjustment, the neural network model autonomously determines the best tensor tiling and layer fusion settings based on the neural network structure and performance requirements.
Solution Approach 2:
The patent replaces manual configuration adjustment with an automated learning-based system that uses neural networks to predict and generate optimal settings. This substitution eliminates the time-consuming manual process while maintaining the ability to adjust configurations for different neural network architectures.
3Device complexity
If existing optimization methods focus on a single objective such as low latency, then the optimization process is simplified, but the method can only optimize for one objective at a time
Solution Approach 1:
The patent introduces dynamic multi-objective optimization where the learning-based algorithm can adapt to different optimization goals. The system dynamically adjusts its evaluation criteria and can optimize for multiple objectives simultaneously (e.g., latency, accuracy, resource consumption) depending on the specific requirements, making the optimization process flexible and adaptable rather than fixed to a single objective.
Data Source
AI summary
A neural network optimization method includes: executing a population-based algorithm to tune and evaluate a policy group, in order to generate one or more evaluation results, wherein the policy group comprises one or more policies, and each of the one or more policies is related to a neural network; executing a learning-based algorithm to tune the one or more policies according to the one or more evaluation results, to generate one or more tuned policies; performing an inference operation according to a target neural network and the one or more tuned policies, to generate multiple configuration candidates; and performing a selection operation upon the multiple configuration candidates to generate an optimal configuration, for outputting to a compiler and generating an optimized neural network, wherein the optimized neural network is an optimized version of the target neural network.