ML Model Accuracy Monitoring via Consumer Feedback in 5G NWDAF
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The 5G telecommunication system's network data analytics function (NWDAF) faces challenges in accurately monitoring and improving the performance of machine learning (ML) models used for network automation, as existing methods lack effective mechanisms for feedback-based retraining and model evaluation.
Innovation Solution
A method and apparatus that utilize feedback information from ML model consumers to monitor and evaluate ML model accuracy, allowing for retraining or selection of new models, with the feedback information being transmitted through a network exposure function, enabling continuous improvement of analytics accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ML models are used for network automation analytics, then productivity is improved, but reliability deteriorates due to lack of accuracy monitoring and feedback mechanisms
Solution Approach 1:
The patent implements a feedback mechanism where consumers of ML model outputs provide accuracy feedback information back to the system. This feedback loop enables continuous monitoring and evaluation of ML model performance, allowing the system to maintain high productivity while improving reliability through iterative model refinement based on real-world performance data.
Solution Approach 2:
The system enables self-service by automatically collecting feedback information from consumers, computing accuracy metrics, and triggering retraining processes without manual intervention. This automated self-monitoring and self-improving mechanism ensures continuous reliability enhancement while maintaining high automation productivity.
2Reliability
If feedback information collection and accuracy monitoring are implemented, then reliability is improved, but device complexity increases
Solution Approach 1:
The patent creates a multi-functional feedback processing system that handles multiple tasks: collecting feedback from various consumers, computing different accuracy metrics, storing feedback information, and triggering retraining processes. This universal system consolidates what would otherwise be separate complex functions into a single integrated mechanism, reducing overall system complexity while improving reliability.
Solution Approach 2:
The patent introduces a feedback information management system as an intermediary layer between ML model consumers and the model training process. This intermediary handles the complexity of feedback collection, validation, and processing, shielding the core ML system from complexity while enabling comprehensive accuracy monitoring and improvement.
3Manufacturing precision
If consumer feedback is registered and processed, then manufacturing precision is improved, but loss of time increases due to feedback collection and processing
Solution Approach 1:
The patent implements preliminary action by pre-registering consumers and their feedback capabilities before ML model deployment. This advance preparation ensures that feedback collection is already structured and ready, eliminating the need for complex real-time setup and reducing processing delays while maintaining high analytics accuracy through systematic feedback accumulation.
Solution Approach 2:
The system employs periodic action by collecting and processing feedback information at optimized intervals rather than continuously. This periodic feedback processing reduces time loss compared to real-time processing while still maintaining high manufacturing precision through regular accuracy monitoring and targeted model retraining based on accumulated feedback.
Data Source
AI summary
A method of using analytics feedback information for analytics accuracy of network data and apparatuses for performing the same are provided. The method of using analytics feedback information includes requesting a machine learning (ML) model, receiving feedback information of an information consumer provided with information generated through the ML model, monitoring accuracy of the ML model, and providing at least one of the feedback information and information on the accuracy.