NWDAF Analytics Accuracy Monitoring Against ML Model Drift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy of analytics provided by Network Data Analytics Functions (NWDAFs) in 3GPP wireless communications networks deteriorates over time due to ML model drift, leading to unstable and erroneous predictions.
Innovation Solution
Implement methods for determining the accuracy of ML models by comparing historical data used for training with real-time data, and retraining the models when accuracy falls below a threshold, triggered by significant network behavior changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If ML models are used to derive analytics in NWDAF, then analytics can be automatically generated, but accuracy deteriorates over time due to model drift
Solution Approach 1:
The system performs preliminary actions by collecting historical data in advance and training ML models before they are needed for analytics derivation. The models are pre-trained on historical datasets and stored for later use, allowing the system to quickly generate analytics when needed while maintaining accuracy through periodic retraining.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring the performance of ML models and comparing their outputs against actual network data. When accuracy deterioration is detected through this feedback loop, the system automatically triggers model retraining with updated historical data, thereby maintaining analytics accuracy over time.
2Reliability
If ML model accuracy is continuously monitored and models are retrained, then analytics accuracy is maintained, but system complexity increases
Solution Approach 1:
The system performs self-service by automatically monitoring its own model performance, detecting accuracy deterioration, and triggering retraining operations without external intervention. The NWDAF autonomously manages the ML model lifecycle including performance monitoring, accuracy assessment, and retraining initiation, reducing the need for complex external management systems.
Solution Approach 2:
The system manages complexity by changing parameters selectively - it monitors specific accuracy thresholds and performance metrics rather than continuously analyzing all model outputs. The retraining process adjusts model parameters based on updated historical data, maintaining accuracy while avoiding unnecessary complexity through targeted parameter optimization.
3Reliability
If historical data is collected and stored for model training, then model accuracy can be maintained, but data storage requirements increase
Solution Approach 1:
The system extracts only the essential and relevant features from historical network data for ML model training, rather than storing and processing complete raw datasets. By extracting key performance indicators and critical data patterns, the system maintains model training effectiveness while significantly reducing data storage requirements.
Solution Approach 2:
The system applies partial action by collecting and storing only the portion of historical data that is most relevant for model training and accuracy maintenance. Rather than archiving all historical network data, the system selectively retains data that has proven value for analytics derivation, optimizing the balance between model accuracy and storage consumption.
Data Source
AI summary
There is provided a method comprising determining that identifying an accuracy of a ML model for deriving analytics for an analytic ID is required based on feedback received from an analytics consumer, receiving, from a first network function, a first set of data, wherein the first set of data was collected in the past and used to train the ML model, receiving, from a second network function, a second set of data corresponding the analytic ID, wherein the second set of data is real time data, and determining the accuracy of the ML model by comparing the first set of data and the second set of data.