AI Model Performance Estimation Using Device-Specific Predictors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge is to efficiently provide an artificial intelligence-based model suitable for a specific device and accurately estimate the performance of such a model considering the device's characteristics.
Innovation Solution
A method is disclosed that involves receiving an AI-based target model and target device information, determining a target workload set, extracting a target performance predictor related to the device, and calculating the estimated performance of the model when executed on the device. The performance predictor is generated by combining pre-stored characteristics of other devices when necessary.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive background knowledge and vast resources are required to determine suitable AI model and hardware combinations, then accuracy of performance estimation is improved, but system complexity and resource requirements increase
Solution Approach 1:
The patent segments the performance estimation process into distinct components: workload decomposition into N-dimensional vectors representing different model aspects (accuracy, speed, resource usage), device characteristic extraction into separate performance predictors, and systematic combination of workload-performance pairs. This segmentation transforms the complex overall estimation into manageable modular components that can be independently analyzed and combined.
Solution Approach 2:
The patent changes the parameter representation from qualitative expert knowledge to quantitative N-dimensional vectors with specific features (accuracy, inference speed, memory usage, compute operations). By parameterizing both workload characteristics and device performance predictors in consistent mathematical forms, the system enables automated computation rather than requiring extensive human expertise.
2Reliability
If extensive resources and background knowledge are allocated for AI and hardware technology understanding, then model-hardware matching quality is improved, but resource consumption increases
Solution Approach 1:
The patent creates simplified copies of device performance characteristics through performance predictors that capture essential device behaviors without requiring complete hardware specifications or extensive testing. These predictor copies enable rapid estimation by substituting detailed hardware analysis with condensed performance profiles that can be stored and reused across multiple estimations.
Solution Approach 2:
The patent performs preliminary extraction and storage of device performance predictors before actual model deployment decisions. By pre-characterizing devices with performance predictors based on benchmark workloads, the system eliminates the need for resource-intensive real-time analysis when matching models to devices, having already performed the heavy lifting in advance.
3Measurement precision
If detailed device characteristics and workload analysis are performed, then performance estimation accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies partial action by selecting and weighting only the most relevant workload features and device characteristics for each estimation scenario, rather than uniformly processing all possible parameters. The N-dimensional vector approach allows the system to focus computational effort on the most impactful dimensions while reducing or eliminating less significant factors, achieving good estimates with reduced processing.
Data Source
AI summary
According to an embodiment of the present disclosure, a method for providing estimated performance of an artificial intelligence (AI)-based model considering characteristic of a device, performed by a computing device, is disclosed. The method includes receiving at least one of an AI-based target model or target model information corresponding to the target model. The method includes receiving target device information corresponding to a target device. The method includes determining a target workload set including a plurality of workloads constituting the target model, based on at least one of the target model or the target model information. The method includes extracting a target performance predictor corresponding to the target device. The target performance predictor includes characteristic of the target device related to a workload. The method includes determining estimated performance when the target model is executed on the target device based on the target workload set and the target performance predictor.