AI Confidence Level for Network Prediction Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI learning systems in network management may inaccurately predict network situations due to the absence of specific classifiers in testing data, affecting the confidence and accuracy of AI models used in communication network configurations.
Innovation Solution
A system and method for evaluating the confidence level of predicting network situations in communication networks using AI, which involves determining a detection entity with a classifier based on event log data and predicting network situations, along with a confidence level representing the probability of the prediction, to facilitate timely and effective network reconfiguration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If AI learning systems use traditional accuracy metrics (number of accurate, false positive, and false negative results) to evaluate predictions, then the evaluation process is simple and straightforward, but the accuracy and confidence of predictions are compromised when specific classifiers are absent from testing data
Solution Approach 1:
The patent introduces a confidence level as an intermediary metric between traditional accuracy measurements and prediction reliability. This confidence level specifically accounts for the presence or absence of classifiers in testing data, providing a nuanced evaluation that bridges the gap between simple accuracy counts and the actual reliability of AI predictions in network situation detection
Solution Approach 2:
The patent changes the evaluation parameter from simple accuracy counts to a confidence level that incorporates classifier presence information. By transforming the evaluation metric to include this additional dimension, the system can maintain high prediction accuracy while also ensuring reliability even when certain classifiers are missing from testing datasets
2Loss of time
If AI models are trained to detect classifier patterns preceding network situations, then the system can identify potential issues earlier, but the predictions become less reliable when those specific classifiers are not present in testing data
Solution Approach 1:
The patent applies preliminary action by training the AI model to detect classifier patterns that precede network situations, enabling early detection. The confidence level mechanism then ensures that even when specific classifiers are absent during testing, the system maintains reliable predictions by adjusting confidence scores based on the detected patterns and available data
Solution Approach 2:
The confidence level serves as a feedback mechanism that continuously monitors the reliability of predictions. When classifiers are missing from testing data, the feedback loop adjusts the confidence level accordingly, allowing the system to maintain early detection capabilities while compensating for the reduced reliability through transparent confidence scoring
3Productivity
If network reconfiguration is performed based on AI predictions, then network management efficiency improves, but incorrect predictions due to missing classifiers lead to wasted resources and operational expenditures
Solution Approach 1:
The confidence level provides critical feedback before network reconfiguration actions are executed. By evaluating the confidence score, the system can distinguish between high-confidence predictions that warrant reconfiguration and low-confidence predictions that would likely result in wasted resources, thereby improving network management efficiency while reducing unnecessary operational expenditures
Solution Approach 2:
The patent applies partial action by performing network reconfiguration only when the confidence level threshold is met. Instead of acting on all predictions regardless of reliability, the system selectively executes reconfiguration actions based on the confidence evaluation, avoiding excessive actions that would waste resources while still capturing the benefits of AI-driven network management
Data Source
AI summary
A system, method, and computer program product are provided for evaluating confidence level of predicting a network situation in a communication network managed using artificial intelligence. In use, for a configuration of a communication network, at least one network situation is determined requiring a change of the configuration of the communication network. A minimal configuration time period is determined required to implement the change of the configuration of the communication network. Additionally, a detection entity including a first classifier is determined that includes one or more event log data associated with the configuration of the communication network, and that further includes a prediction of an occurrence of a particular network situation of the at least one network situation. Further, a first confidence level of the detection entity is determined, the first confidence level representing, at least in part, a probability of the prediction.