AI Model Conversion for Node-Specific Inference Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-based models often exhibit varying inference performance across different hardware platforms, necessitating extensive knowledge and resources to determine suitable models and hardware for specific applications, with potential mismatches leading to suboptimal performance.
Innovation Solution
A method for providing an AI-based model involves creating a candidate model list with detailed information, selecting a target model and node, and converting the model to be compatible with the target node, using user inputs and predefined node information to ensure compatibility and optimal performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extensive knowledge and resources are used to determine suitable models and hardware, then model-hardware compatibility is improved, but device complexity and resource consumption increase
Solution Approach 1:
The patent introduces a model conversion service as an intermediary between the AI model and the target hardware node. This service automatically handles the compatibility matching and conversion process, eliminating the need for users to possess extensive knowledge about model-hardware compatibility while reducing the complexity burden from the user side.
Solution Approach 2:
The system enables self-service by automatically determining compatible models and converting them to hardware-specific formats without requiring manual intervention. The conversion service autonomously selects appropriate models based on node capabilities and performs the conversion process, reducing the need for user expertise and resource consumption.
2Reliability
If extensive knowledge and resources are used to determine suitable models and hardware, then model-hardware compatibility is improved, but ease of operation deteriorates
Solution Approach 1:
The model conversion service acts as a mediator that handles the complex compatibility matching and conversion processes automatically. Users simply need to specify their needs, and the service manages the technical details of model selection and hardware compatibility, significantly improving ease of operation.
Solution Approach 2:
The patent replaces manual mechanical processes of model selection and compatibility checking with an automated digital conversion service. The system uses algorithmic determination and automatic conversion instead of requiring users to manually assess compatibility, thereby simplifying operation while maintaining reliability.
3Reliability
If AI-based models are adapted to specific hardware nodes, then inference performance is improved, but conversion time and processing resources increase
Solution Approach 1:
The system performs preliminary actions by pre-identifying compatible models and pre-converting them to hardware-specific formats before the actual inference process. This advance preparation ensures optimal inference performance while minimizing conversion time during deployment, as the conversion has already been performed in advance.
Data Source
AI summary
According to an embodiment of the present disclosure, a method for providing an artificial intelligence-based model, performed by a computing device, is disclosed. The method includes providing a candidate model list comprising a plurality of candidate models which are artificial intelligence-based models. The method includes determining a target model to be converted or benchmarked, based on a first user input on the candidate model list. The method includes providing a candidate node list comprising a plurality of candidate nodes. The method includes determining a target node to be converted or benchmarked, based on a second user input on the candidate node list. The method includes converting the target model into a model supportable by the target node, based on information of the determined target model and information preset on the target node.