Polling Manager Anomaly Data Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems face challenges in accurately separating anomaly data from normal data during device polling, leading to skewed results and analysis, especially when anomaly data significantly deviates from normal ranges, causing distortion in graphical representations and statistical averages.
Innovation Solution
A system employing a polling manager that filters out anomaly data from normal data using criteria such as range, threshold, and directional transition-based methods, storing them separately in metric and event databases, and merging them for graphical representation without using anomaly data points, while indicating anomalies with markers for notification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly data is included in graphical representation and statistical calculations, then complete data coverage is achieved, but data accuracy and reliability deteriorate due to skewing and distortion
Solution Approach 1:
The patent segments anomaly data and normal data into separate collections, processing and visualizing them through different pathways. Normal data is used for graphical representations and statistical calculations to maintain accuracy, while anomaly data is stored and analyzed separately to preserve completeness without compromising reliability.
Solution Approach 2:
The patent extracts anomaly data points from the main data stream used for graphical representation and statistical calculations. By removing outlier values that fall outside defined thresholds or exhibit abnormal patterns, the system prevents these values from skewing averages and distorting visualizations, while still maintaining them in a separate anomaly collection for later analysis.
2Measurement precision
If anomaly data is filtered out completely, then data accuracy improves, but loss of information increases due to removal of potentially valuable anomaly insights
Solution Approach 1:
The patent implements segmentation by creating distinct data collections: one for normal data used in statistical calculations and graphical representations, and another for anomaly data. This allows precise measurement and analysis of normal operational patterns while preserving anomaly information separately for specialized analysis, preventing contamination of statistical metrics.
Solution Approach 2:
The patent introduces an intermediary anomaly detection and filtering mechanism that sits between raw data collection and statistical processing. This intermediary layer identifies and separates anomaly data points using threshold-based and pattern-based detection methods, allowing normal data to flow smoothly into statistical calculations while routing anomaly data to a separate collection for later analysis.
3Loss of information
If all polled data is used in graphical representation, then data completeness is maintained, but visualization clarity deteriorates due to distortion from anomaly points
Solution Approach 1:
The patent segments the data visualization process by using only normal data points for graphical representations such as line graphs and charts. Anomaly data points are excluded from these visualizations to prevent distortion and maintain clarity, while the system maintains complete data coverage by storing anomaly points separately and potentially indicating their occurrence through separate anomaly logs or alerts.
4Reliability
If anomaly detection and separate storage is implemented, then data processing complexity increases, but data quality improves through accurate separation of normal and anomaly data
Solution Approach 1:
The patent applies preliminary action by performing anomaly detection and data classification immediately during the data collection phase, before statistical calculations or graphical representations are generated. By identifying and separating anomaly data points upfront using threshold comparisons and pattern recognition, the system establishes clean, segmented data collections that simplify subsequent processing steps and improve overall data quality.
Data Source
AI summary
For each of a plurality of time intervals over which metric data is collected from periodic polling of a network of devices, a determination is made of whether metric data of the time interval satisfies a criterion for normal data values defined for a metric corresponding to the metric data. The metric data is marked as anomaly data, in response to determining that the metric data does not satisfy the criterion defined for the metric corresponding to the metric data of the time interval. The metric data is marked as normal data, in response to determining that the metric data satisfies the criterion defined for the metric corresponding to the metric data of the time interval. The metric data marked as normal data is stored in a first database, and the metric data marked as anomaly data and corresponding time intervals in a second database.