AI Model Benchmark Prediction via Modular Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in artificial intelligence technology is determining the suitable artificial intelligence-based model and hardware for edge devices, as performance varies significantly across different hardware environments, requiring extensive knowledge and resources to match models with appropriate hardware for optimal performance.
Innovation Solution
A method for providing a benchmark prediction result by identifying target blocks within pre-stored blocks based on a benchmark query, using node and edge identifiers to determine performance information, and combining benchmark results from multiple blocks to generate accurate predictions for specific hardware environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a model with excellent performance in a specific domain is selected, then the model performance is improved, but the inference performance varies for each hardware environment
Solution Approach 1:
The patent segments the model benchmarking process into modular blocks that can be independently evaluated and recombined. Each block represents a specific computational unit or operation within the model, allowing the system to analyze and predict performance on different hardware environments by composing results from individual block evaluations.
Solution Approach 2:
The patent changes the parameter representation by using intermediate representation (IR) code to describe model operations in a hardware-agnostic manner. This allows the same model to be evaluated across different hardware platforms by translating the IR parameters into hardware-specific instructions, thereby maintaining model performance while adapting to various hardware environments.
2Measurement precision
If extensive knowledge and resources are invested to match models with appropriate hardware, then the matching accuracy is improved, but the complexity and resource requirements increase
Solution Approach 1:
The patent performs preliminary actions by pre-compiling model blocks into intermediate representation code and pre-evaluating them on target hardware. This advance preparation creates a library of benchmarked blocks that can be quickly queried and composed, eliminating the need for extensive real-time analysis and reducing the complexity of the matching process.
Solution Approach 2:
The patent introduces an intermediary layer in the form of intermediate representation (IR) code that acts as a mediator between the model definition and hardware implementation. This IR layer simplifies the matching process by providing a standardized interface that can be translated to any target hardware, reducing the complexity of directly mapping models to specific hardware platforms.
3Measurement precision
If the entire model is benchmarked on target hardware, then the accuracy of performance prediction is improved, but the time and resource consumption increase significantly
Solution Approach 1:
The patent segments the model into smaller computational blocks that can be independently benchmarked. By evaluating individual blocks rather than the entire model, the system achieves sufficient prediction accuracy while dramatically reducing the time and resources required, as each block can be processed separately and results composited.
Solution Approach 2:
The patent applies partial action by benchmarking only the critical or representative blocks of a model rather than every single operation. This selective approach provides adequate performance prediction for the overall model while avoiding the excessive time and resource consumption that would result from comprehensive full-model benchmarking.
Data Source
AI summary
According to an embodiment of the present disclosure, a method for providing a benchmark prediction result, performed by a computing apparatus is disclosed. The method includes obtaining a benchmark query specifying a target of a benchmark. The method includes determining at least one target block to be used to obtain the benchmark prediction result corresponding to the benchmark query among pre-stored blocks based on the benchmark query. The blocks comprise a node identifying a function or an operation constituting a model, and an edge connecting nodes. The method includes obtaining the benchmark prediction result corresponding to the benchmark query, using a benchmark result related to the determined at least one target block.