AI/ML CSI Prediction Accuracy Test Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating AI/ML-based CSI prediction accuracy face challenges due to the need for ground truth data, which may not be readily available, and simultaneous active CSI measurements and predictions, complicating the extraction of accurate prediction metrics.
Innovation Solution
A test framework is proposed that collects ground truth data independently during Phase I using non-AI/ML functionality and compares it with AI/ML predicted values during Phase II, enabling evaluation of CSI prediction accuracy without requiring the DUT to send ground truth to the test equipment, and allowing for the determination of prediction accuracy by comparing measured and predicted CSI values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ground truth data is collected from the DUT during CSI measurements, then prediction accuracy evaluation is enabled, but device complexity and measurement complications increase
Solution Approach 1:
The patent segments the evaluation process into two independent phases: Phase I collects ground truth data using non-AI/ML functionality, while Phase II performs AI/ML predictions. This segmentation allows ground truth collection to be separated from the prediction process, reducing measurement complexity while maintaining evaluation accuracy.
Solution Approach 2:
The patent performs preliminary action by collecting ground truth data during Phase I before conducting AI/ML predictions in Phase II. This preliminary collection of reference data enables accurate prediction evaluation without requiring simultaneous measurements, thereby reducing device complexity and measurement complications.
2Productivity
If simultaneous CSI measurements and predictions are performed, then real-time evaluation is achieved, but extraction of accurate prediction metrics becomes difficult
Solution Approach 1:
The patent divides the evaluation process into distinct Phase I (ground truth collection) and Phase II (prediction) segments. This segmentation prevents the interference that occurs in simultaneous measurements, allowing accurate extraction of prediction metrics by comparing Phase II predictions against Phase I ground truth data.
Solution Approach 2:
The patent uses ground truth data collected in Phase I as an intermediary reference to evaluate prediction accuracy in Phase II. This intermediary dataset serves as a stable benchmark that enables accurate metric extraction without the complications of simultaneous real-time measurements.
3Ease of operation
If non-AI/ML functionality is used to collect ground truth data independently, then DUT ground truth transmission is eliminated, but additional testing phase is required
Solution Approach 1:
The patent performs preliminary action by collecting ground truth data during Phase I using non-AI/ML functionality before the actual prediction testing. This eliminates the need for the DUT to transmit ground truth data during prediction operations, simplifying the operational process despite adding an initial data collection phase.
Data Source
AI summary
Embodiments of the present disclosure relate to a test framework to evaluate the CSI prediction accuracy for AI/ML based CSI prediction use case without the need for the DUT to send the ground truth to the test equipment separately. Further, embodiments of the present disclosure propose a procedure to obtain the parameter values predicted by the DUT and the corresponding ground truth at the TE side independently during the conformance testing.