Terminal AI Model Evaluation for Adaptive Communication Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication systems lack effective methods to ensure optimal performance of AI/ML models or functions by monitoring and selecting suitable models or functions, failing to provide effective performance evaluation, leading to communication performance degradation, and existing methods fail to address the need for better model or function selection.
Innovation Solution
A method for performance evaluation that involves determining a set of models or functions to be evaluated, where a model in the set of models or functions to be evaluated is deployed on the terminal, and a function in the set of functions to be evaluated is a function supported by the terminal, and includes performance evaluation on these models or functions to obtain a performance evaluation result, which is transmitted to the network side.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If performance monitoring is performed on the currently activated model or function, then the availability of the current model can be determined, but the terminal cannot select a better model or function and communication performance cannot be guaranteed
Solution Approach 1:
The terminal performs performance evaluation on multiple models or functions in advance before actual communication occurs. By pre-evaluating the performance of different models under various conditions and storing the evaluation results, the terminal can select the optimal model for communication without waiting until communication starts to determine availability. This preliminary evaluation enables both reliable communication performance and adaptable model selection.
2Adaptability or versatility
If multiple models or functions are evaluated, then better model selection can be achieved, but the complexity of evaluation and resource consumption increase
Solution Approach 1:
The performance evaluation is conducted locally at the terminal using locally available resources and pre-stored model information, rather than requiring complex centralized evaluation systems. The terminal evaluates models based on local performance indicators and communicates only necessary evaluation results to the network side, reducing overall system complexity while maintaining comprehensive model selection capability.
3Reliability
If performance evaluation is performed on deployed models, then communication performance can be improved, but time and computational resources are consumed
Solution Approach 1:
Models are evaluated in advance before actual communication occurs, and evaluation results are stored for later reference. This allows the terminal to quickly select optimal models during communication without performing time-consuming evaluations in real-time, thus improving communication performance while minimizing time loss during actual data transmission.
Solution Approach 2:
Instead of continuously performing full performance evaluations, the terminal uses pre-computed evaluation results and model information copies to make selection decisions during communication. This approach maintains communication performance improvement while significantly reducing the time and computational resources consumed during actual communication operations.
Data Source
Figure 1~2
Figure 3~4
Figure 5~7
AI summary
Provided in the embodiments of the present disclosure are a performance evaluation method and apparatus, and a terminal and a network-side device. The performance evaluation method comprises: determining a model set or function set to be evaluated, wherein models or functions in said model set or function set are deployed in a terminal; performing a performance evaluation on the models or functions in said model set or function set, so as to obtain a performance evaluation result; and sending the performance evaluation result to a network side. By means of performing a performance evaluation on models or functions that are deployed in a terminal, a performance evaluation result is obtained to serve as a reference for determining a model or function suitable for an environment where the terminal is located, such that the communication performance of the terminal is improved.