KPI Backfill Automation for Cellular Network Data Anomalies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cellular network providers face challenges in accurately aggregating key performance indicators (KPIs) due to delayed or unavailable data from operational support systems, leading to inaccurate or incomplete KPIs, which are difficult to detect manually and can harm reputation and require recalculating after the fact.
Innovation Solution
An anomaly detection and backfill engine (ADBE) using artificial intelligence/machine learning to detect deviations in data quality metrics, initiating a backfill operation to reaggregate KPIs when anomalies are detected, allowing parallel processing with standard aggregation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If manual detection methods are used for data anomalies, then system complexity is reduced, but detection precision and timeliness deteriorate
Solution Approach 1:
The patent replaces manual detection methods with an automated machine learning-based anomaly detection system. The system uses trained models to automatically identify data anomalies in KPI streams, substituting human manual inspection with computational algorithms that provide higher precision and timeliness while operating independently without human intervention.
2Loss of time
If data anomalies are not detected timely, then processing time is reduced, but data integrity and reliability deteriorate
Solution Approach 1:
The system performs preliminary anomaly detection on incoming data streams before the data is fully processed and aggregated into KPIs. By detecting anomalies early in the data flow, the system can trigger immediate backfill operations to retrieve missing data, preventing corrupted or incomplete data from affecting downstream processing and maintaining data integrity throughout the pipeline.
Solution Approach 2:
The system implements a feedback mechanism where anomaly detection results trigger automatic backfill operations. When an anomaly is detected, the system sends a backfill request to retrieve the missing or corrupted data, then reprocesses the affected KPIs with the corrected data, creating a closed-loop feedback system that continuously maintains data quality.
3Extent of automation
If backfill operations are performed manually, then automation level is reduced, but operational complexity is simplified
Solution Approach 1:
The system implements self-service automation where the anomaly detection system automatically triggers backfill operations without human intervention. When data anomalies are detected, the system autonomously initiates backfill requests, retrieves missing data, and reprocesses affected KPIs, creating a self-correcting system that handles data quality issues independently.
Solution Approach 2:
The patent merges the anomaly detection function with the backfill operation function into an integrated system. The anomaly detection module and backfill execution module work as a unified automated pipeline, where detection automatically triggers correction actions, eliminating the need for separate manual operations and reducing overall operational complexity despite increased automation.
4Reliability
If complete data streams are waited for before processing, then data completeness is improved, but processing speed and productivity deteriorate
Solution Approach 1:
The system performs preliminary processing of available data streams without waiting for complete data arrival. KPIs are calculated and made available as soon as sufficient data is received, enabling downstream consumers to access partial results immediately. Simultaneously, the system continues to monitor for additional data and performs backfill operations to complete the picture, thus maintaining both processing speed and data completeness.
Data Source
AI summary
A method of initiating a KPI backfill operation for a cellular network based on detecting a network data anomaly, where the method includes receiving, by an anomaly detection and backfill engine (ADBE) executed by a computing device, a data quality metric that is based on a KPI of the cellular network; detecting, by the ADBE, the network data anomaly based on the data quality metric being more than a threshold amount different than a predicted value for the data quality metric, where the network data anomaly indicates that at least a portion of a data stream from which the KPI is calculated was unavailable for a previous iteration of the KPI; and providing, by the ADBE and based on detecting the network data anomaly, a backfill command to a backfill processing pipeline to perform the backfill operation by reaggregating the KPI when the portion of the data stream becomes available.