AI Model Operator Replacement for Hardware-Aware Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence (AI) models face challenges in achieving optimal performance across different hardware platforms due to varying hardware capabilities, requiring extensive knowledge and resources to determine suitable models and hardware configurations.
Innovation Solution
A method for optimizing AI models by grouping operators into blocks, identifying unsupported operators, and replacing them with compatible alternatives, using change algorithms and filtering criteria to enhance model performance on target devices, along with quantization and retraining for improved efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI models are optimized for specific hardware platforms, then performance and efficiency are improved, but the complexity of determining suitable model-hardware configurations increases
Solution Approach 1:
The system performs self-service by automatically analyzing hardware capabilities, selecting appropriate operators, and optimizing model configurations without requiring external expert intervention. The hardware-aware operator selection process autonomously matches model operators with hardware-supported operations, eliminating the need for manual configuration expertise.
Solution Approach 2:
The system changes parameters by dynamically adjusting model configuration parameters based on hardware capabilities. It analyzes hardware features and modifies operator selection, precision settings, and optimization strategies to match the target hardware, enabling the same model to adapt to different hardware platforms through parameter adjustment rather than structural redesign.
2Productivity
If hardware-specific optimizations are applied, then inference efficiency is improved, but the model may not be supported on certain hardware platforms
Solution Approach 1:
The system achieves universality by creating a hardware-agnostic optimization framework that can adapt to multiple hardware platforms. The hardware-aware operator selection process identifies operators that are either natively supported or can be efficiently simulated on the target hardware, enabling the optimized model to run across diverse hardware environments including CPUs, GPUs, and specialized AI accelerators.
Solution Approach 2:
The system applies dynamics by making the optimization strategy adaptive rather than static. The hardware capability analysis dynamically determines the appropriate optimization level and operator substitution strategy based on the target hardware's features, allowing the same optimization framework to produce hardware-specific configurations on demand without requiring separate optimization pipelines for each platform.
3Productivity
If extensive hardware knowledge is required for model deployment, then optimal performance can be achieved, but the ease of operation decreases
Solution Approach 1:
The system performs self-service by automatically analyzing hardware capabilities, selecting appropriate operators, and optimizing model configurations without requiring external expert intervention. The hardware-aware operator selection process autonomously matches model operators with hardware-supported operations, eliminating the need for manual configuration expertise.
Solution Approach 2:
The system introduces an intermediary optimization layer that translates between model requirements and hardware capabilities. This intermediary process automatically handles the complex mapping between AI model operators and hardware-supported operations, shielding users from hardware complexity while delivering hardware-optimized performance.
4Adaptability or versatility
If operator replacement is performed to match hardware capabilities, then compatibility is improved, but computational accuracy may be affected
Solution Approach 1:
The system implements feedback by evaluating the impact of operator replacements on model accuracy and iteratively adjusting the optimization strategy. The hardware-aware operator selection process monitors performance metrics and adjusts operator substitution decisions to maintain accuracy thresholds, ensuring that compatibility improvements do not come at the cost of computational precision.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Disclosed is a method for optimizing an artificial intelligence based model, which is performed by a computing device. The method may include: obtaining model information corresponding to an artificial intelligence based model, and obtaining target device information corresponding to a target device in which the model is to be executed; obtaining first blocks corresponding to the model by grouping first operators corresponding to the model in units of a block corresponding to a set of one or more operators based on the model information; obtaining second operators which are not supportable by the target device based on the target device information; changing the first blocks corresponding to the model to second blocks corresponding to the model and the target device by changing an operator included in the second operators among the first operators to a replacement operator supportable by the target device; and performing optimization of the model to correspond to the target device based on the second blocks.