Multi-Granularity Aggregation for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional anomaly detection systems fail to effectively identify both non-stealthy and stealthy anomalies in information handling systems, as they either detect significant attacks easily but miss stealthy attacks that spread over multiple data points with minimal variance, or vice versa, due to large variance in normal behavior data.
Innovation Solution
The approach involves creating multiple aggregation layers with varying granularities to detect anomalies by training machine learning models on data sets aggregated at different levels, allowing for the detection of both fine and coarse anomalies, and applying these models in parallel to real-time data sets to flag anomalous behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection systems focus on single data point analysis, then significant attacks are readily detectable, but stealthy attacks spreading over multiple data points cannot be detected
Solution Approach 1:
The patent segments the anomaly detection task into multiple aggregation layers, each operating at a different granularity level. Fine-grained layers detect anomalies in individual data points, while coarse-grained layers detect anomalies across aggregated groups of data points. This segmentation allows the system to simultaneously detect both significant attacks (fine anomalies) and stealthy attacks (coarse anomalies) without compromising detection precision for either type.
Solution Approach 2:
The patent introduces a new dimension to anomaly detection by adding aggregation layers that operate at different granularities. Instead of analyzing data at a single level, the system creates multiple layers of aggregation (fine, medium, coarse) that add a dimensional aspect to the detection process. This allows the system to detect anomalies that manifest differently across various aggregation levels, thereby improving both precision and adaptability.
2Reliability
If data aggregation is applied to reduce variance, then stealthy anomalies become detectable, but information about single point deviations is lost
Solution Approach 1:
The patent segments the aggregation process into multiple distinct layers with different aggregation factors. The fine-grained layer maintains low aggregation to preserve information about single data point deviations, while the coarse-grained layer applies high aggregation to reduce variance and detect stealthy anomalies. By segmenting the aggregation process, the system achieves both reliability through variance reduction and preserves information about single point deviations through the fine-grained layer.
Solution Approach 2:
The patent applies local quality by having different aggregation layers serve different detection purposes. The fine-grained aggregation layer focuses on detecting anomalies in individual or small groups of data points, preserving local information. The coarse-grained aggregation layer focuses on detecting anomalies across large aggregates, reducing variance. Each layer has optimized quality characteristics suited to its specific detection task, allowing the system to maintain both information fidelity and detection reliability.
3Adaptability or versatility
If multiple aggregation layers with varying granularities are created, then both fine and coarse anomalies can be detected, but system complexity increases
Solution Approach 1:
The patent segments the anomaly detection system into multiple independent aggregation layers, each with a specific granularity level. This segmentation allows each layer to be independently configured and optimized for its specific detection task. The fine-grained layer handles significant attacks, while coarse-grained layers handle stealthy attacks. This modular segmentation improves detection versatility while managing system complexity through clear separation of concerns.
Solution Approach 2:
The patent implements multi-functionality by designing the aggregation layer framework to handle multiple types of anomalies through a unified architecture. The same basic aggregation and detection mechanism is applied across multiple layers with different granularities, allowing the system to detect both fine and coarse anomalies using a consistent approach. This universal framework reduces system complexity compared to implementing separate detection systems for different anomaly types.
Data Source
AI summary
Embodiments address the problem of detecting anomalies in data sets with respect to well-defined normal behavior. Deviations of data collected in real-time are detected using a previously observed distribution of data known to be benign. Embodiments provide techniques to detect varying types of anomalies by creating multiple aggregation layers having varying granularities on top of the lowest level of data collection. This allows detection of fine anomalies that strongly impact single data points, as well as coarse anomalies that detect multiple data points less strongly. Machine learning models are trained and used to compare real-time data sets against behavior of a benign data set in order to detect differences and to flag anomalous behavior.