AI/ML Model Monitoring for Two-Sided Model Fallbacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a lack of clear solutions for handling AI/ML model performance issues when they do not meet requirements, particularly in two-sided AI/ML models deployed at different nodes, such as user equipment and network sides, which are crucial for tasks like CSI compression and decompression.
Innovation Solution
A method for AI/ML model monitoring that involves a first communication device transmitting information to a second device when performance requirements are not met, including various indication and identity information to facilitate model changes or fallbacks, and utilizing priority orders, mapping relationships, or random selection to manage AI/ML models effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If AI/ML model performance monitoring is implemented, then model performance can be detected, but there is no clear handling solution when performance requirements are not met
Solution Approach 1:
The patent implements a feedback mechanism where monitoring results are transmitted back to the communication device, enabling closed-loop control. The first communication device sends monitoring information to the second communication device, which then provides handling instructions based on the detected performance status, creating a continuous feedback cycle that resolves the ambiguity in handling solutions.
Solution Approach 2:
The system enables self-service by allowing the monitoring system to automatically trigger handling actions. When performance requirements are not met, the system automatically initiates model switching or fallback procedures without requiring external intervention, making the handling solution clear and autonomous.
2Productivity
If two-sided AI/ML models are deployed at different nodes, then task completion capability is improved, but model management complexity increases
Solution Approach 1:
The patent introduces a monitoring system as an intermediary between the two-sided AI/ML models at different nodes. This intermediary collects performance data from both sides, processes it centrally, and coordinates the switching or fallback actions, thereby simplifying the management complexity while maintaining the distributed deployment architecture.
Solution Approach 2:
The monitoring system performs multiple functions including performance detection, threshold comparison, decision making, and coordination of switching actions. By consolidating these diverse functions into a single multi-functional system, the patent reduces overall management complexity while supporting the two-sided model deployment for enhanced task completion.
3Reliability
If AI/ML model switching is implemented, then performance requirements can be maintained, but system operation time increases due to switching delays
Solution Approach 1:
The patent implements preliminary action by pre-configuring multiple AI/ML models and preparing fallback options in advance. When performance degradation is detected, the system can immediately switch to a pre-prepared alternative model rather than needing to train or deploy a new model, significantly reducing switching delays while maintaining performance requirements.
Solution Approach 2:
The system provides beforehand cushioning by maintaining backup AI/ML models that are ready to be activated. This cushioning mechanism ensures that when the primary model fails to meet performance requirements, a pre-prepared alternative is immediately available, minimizing the time loss during transitions while ensuring continuous performance compliance.
Data Source
Figure 1~3
Figure 4~6
AI summary
Provided in the embodiments of the present disclosure are an AI/ML model monitoring method, a device, an apparatus, and a storage medium. The method comprises: when detecting that the performance of at least one AI/ML model among AI/ML models on the two sides or of an AI/ML model pair on the two sides does not meet a model performance requirement, a first communication device sends first information to a second communication device, the first information comprising information related to monitoring of the AI/ML models on the two sides.