Deterministic Device Partitioning for Accurate Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection systems face challenges in determining optimal device subpopulations for accurately representing normal behavior, leading to false positives and negatives due to subjective human judgments and the wide variety of device attributes, which increases network vulnerability.
Innovation Solution
A method and system for recursively partitioning device activity data based on deterministic characteristics, using a split density metric to create partitions that meet a threshold, establishing baselines for each partition, and detecting anomalies based on device behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If larger subpopulations are selected for anomaly detection, then statistical significance is improved, but the ability to represent specific device types accurately deteriorates
Solution Approach 1:
The patent segments the device population into multiple subpopulations based on deterministic characteristics (device type, manufacturer, model). Each subpopulation is analyzed separately to establish baseline behavior specific to that device type, allowing both statistical significance through adequate sample sizes within each segment and precise representation of specific device characteristics.
Solution Approach 2:
The patent applies local quality by creating device-specific baselines for each subpopulation rather than using a single universal baseline. This allows anomaly detection to be tailored to the specific characteristics of each device type, improving measurement precision for each local group while maintaining statistical reliability through aggregated analysis across all subpopulations.
2Measurement precision
If manual human input is used for subpopulation selection, then subjective judgment and human bias are reduced, but operational complexity and time consumption increase
Solution Approach 1:
The patent implements self-service by enabling the anomaly detection system to automatically select and partition device subpopulations based on deterministic characteristics without requiring manual human input. The system autonomously analyzes device attributes, identifies appropriate subpopulations, and establishes baselines, thereby eliminating human bias while managing complexity through automated algorithms that process device metadata systematically.
3Measurement precision
If more device attributes are considered for partitioning, then the accuracy of anomaly detection is improved, but the computational processing requirements increase
Solution Approach 1:
The patent applies preliminary action by pre-processing and organizing device activity data into structured subpopulations based on deterministic characteristics before anomaly detection occurs. Device attributes are pre-analyzed and devices are pre-grouped into appropriate subpopulations, so that during actual anomaly detection, the system only needs to compare current behavior against pre-established baselines for the relevant subpopulation, significantly reducing real-time computational requirements while maintaining high detection accuracy.
Data Source
AI summary
A system and method for anomaly detection. A method includes recursively partitioning a sample of device activity data including deterministic characteristics of a population of devices over iterations in order to create partitions. Each iteration includes determining a split density metric for a candidate subpopulation created by splitting a portion of the population with respect to a corresponding type of deterministic characteristic. The split density metric for the candidate subpopulation is determined based on a density value of the candidate subpopulation and a coverage value of the corresponding type of deterministic characteristic. The partitions include each candidate subpopulation meeting a split density metric threshold. A baseline for each of the partitions is established based on device activity for devices represented in device activity data of the partition. An anomaly is detected based on behavior of a device and the baseline established for a partition corresponding to the device.


