IoT Sensor Anomaly Detection via KDE Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the IoT environment, anomaly detection in sensor data from millions or billions of devices is challenging due to sensor failures, environmental changes, and limited resource capabilities, requiring efficient and accurate methods to differentiate normal from abnormal behavior.
Innovation Solution
The use of Kernel Density Estimation (KDE) and Jensen-Shannon (JS) divergence to determine a threshold for anomaly detection in sensor data, where data points are segmented and evaluated to identify deviations from normal behavior, allowing for automatic filtering of anomalies in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional anomaly detection methods are used in IoT sensor networks, then the system can identify abnormal behavior, but the computational complexity and resource consumption increase significantly when handling data from millions or billions of devices
Solution Approach 1:
The patent segments the sensor data into fixed-size windows and further divides each window into segments for computing JS divergence. This segmentation allows the system to process large volumes of data from billions of devices in manageable chunks, reducing computational complexity while maintaining anomaly detection accuracy through localized statistical analysis.
Solution Approach 2:
The patent transforms the anomaly detection problem by changing parameters from direct data comparison to JS divergence calculation based on probability density functions. By converting raw sensor data into statistical parameters (mean, variance, KDE curves) and then computing JS divergence between segments, the system achieves efficient anomaly detection with reduced computational burden compared to traditional methods.
2Reliability
If more sensors are deployed to monitor assets in the IoT network, then the coverage and detection capability improve, but the amount of data to be processed and the resource requirements increase
Solution Approach 1:
The patent divides data from multiple sensors into time-based windows and further segments each window into smaller units for JS divergence calculation. This hierarchical segmentation enables the system to efficiently process data from millions of sensors by analyzing local patterns in small segments rather than processing all sensor data globally, significantly reducing energy consumption while maintaining comprehensive coverage.
Solution Approach 2:
The patent replaces computationally intensive traditional anomaly detection mechanisms with a statistical approach using Kernel Density Estimation and JS divergence. This substitution transforms the problem from direct data comparison (mechanical processing) to probability-based statistical analysis, reducing the computational resources and energy required to handle data from expanded sensor networks.
3Measurement precision
If the anomaly detection threshold is set to be highly sensitive, then more anomalies are detected, but the number of false positives increases
Solution Approach 1:
The patent changes the detection parameter from simple threshold comparison to JS divergence measurement between probability density segments. By computing the statistical divergence between segments and comparing against a dynamically determined threshold (based on the distribution of JS divergence values), the system achieves high sensitivity in detecting true anomalies while maintaining low false positive rates through statistically rigorous threshold selection.
Data Source
AI summary
Embodiments determine anomalies in sensor data generated by a plurality of sensors that correspond to a single asset. Embodiments receive a first time window of clean sensor input data generated by the sensors, the clean sensor data including anomaly free data comprised of clean data points. Embodiments divide the clean data points into training data points and evaluation data points, and divide the training data points into a pre-defined number of plurality of segments of equal length. Embodiments convert each of the plurality of segments into corresponding segment curves using Kernel Density Estimation (“KDE”) and determine a Jensen-Shannon (“JS”) divergence value for each of the plurality of segments using the segment curves to generate a plurality of JS divergence values. Embodiments then assign the maximum value of the plurality of JS divergence values as a threshold value and validate the threshold value using the evaluation data points.