Anomaly Detection Using 3D Score Diversity Diagrams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Enterprise threat detection systems face challenges in detecting anomalies and visualizing threats due to the periodic purging of log data, which limits the ability to correlate events and identify suspicious behavior effectively.
Innovation Solution
The implementation of a computer-implemented method for anomaly detection using statistical methods, including defining data types, generating reference data, grouping evaluations into patterns, and visualizing data in a 3D score diversity diagram to identify outliers and critical anomalies within the enterprise threat detection system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If log data is periodically purged to conserve storage and computing resources, then storage space is saved, but the ability to detect anomalies through correlation with past behavior is reduced
Solution Approach 1:
The patent uses rolling windows of varying lengths (shorter for recent data, longer for historical data) to create temporary analysis datasets. These windows are continuously updated and discarded after use, allowing the system to maintain anomaly detection capabilities without permanently storing large amounts of log data. The rolling window approach enables the system to consume only the necessary portion of log data for each analysis cycle.
Solution Approach 2:
The patent implements dynamic rolling windows that automatically adjust their temporal scope based on the analysis needs. The system uses different window lengths for different evaluation metrics, with more recent events weighted in shorter windows and historical patterns captured in longer windows. This dynamic adjustment allows optimal anomaly detection while minimizing data retention requirements.
2Ease of manufacture
If standard statistical analysis is used to identify anomalies, then implementation is simple, but false positives increase and individual behavior patterns cannot be built
Solution Approach 1:
The patent applies different statistical analysis methods to different entities and evaluation metrics based on their specific characteristics. Instead of using a uniform approach, the system tailors the analysis methodology to match the local properties of each data type, entity behavior pattern, and evaluation metric, thereby improving detection precision while maintaining implementation feasibility through modular design.
Solution Approach 2:
The patent dynamically adjusts statistical parameters such as rolling window lengths, standard deviation thresholds, and evaluation weights based on the specific entity being analyzed and the type of anomaly being detected. This parameter adaptation allows the system to optimize detection precision for different scenarios while maintaining a relatively simple underlying statistical framework.
3Measurement precision
If multiple evaluation metrics are used to assess entity behavior, then detection accuracy improves, but visualization and interpretation become more complex
Solution Approach 1:
The patent transforms multi-dimensional evaluation metrics into a unified anomaly score by projecting multiple evaluation results onto a single dimensional scale. This dimensionality reduction technique allows the system to maintain the benefits of multiple evaluation metrics while presenting the results in a simplified, easily interpretable format that security analysts can quickly understand and act upon.
Data Source
AI summary
A selection of data types is defined from available log data for an evaluation of events associated with an entity. One or more evaluations associated with the entity are defined and reference data is generated from the selection of data types based on the one or more defined evaluations. The one or more evaluations are grouped into a pattern. A three dimensional (3D) score diversity diagram visualization is initialized for display in a graphical user interface, where a point representing the entity in the visualization is localized in 3D space at a coordinate based on two-dimensional (2D) coordinates in a 2D coordinate system of a centroid of the calculated area of a polygon placed to into the 2D coordinate system and defined by the values of each evaluation associated with the entity.