Anomaly Detection in Networked Entities via SVD
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods struggle to effectively identify anomalous entities in a networked environment, particularly in detecting incongruent behavior across multiple devices and time periods, leading to high noise levels and false detections.
Innovation Solution
A multi-phase anomaly detection approach that assigns observation anomaly scores to time-stamped data from networked devices, re-casts these scores into time bins, and applies a two-stage analysis using singular value decomposition (SVD) to identify anomalous time bins and extract causal information about the entities contributing to these anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing anomaly detection methods are applied to networked entities, then anomaly detection is performed, but noise levels are high and false detections occur
Solution Approach 1:
The patent segments the anomaly detection process into two distinct phases: (1) computing observation anomaly scores for individual time-stamped observations, and (2) re-casting these scores into time bins and applying two-stage analysis to identify anomalous time bins and causal entities. This segmentation allows the system to handle temporal correlations and reduce noise by analyzing patterns across multiple observations grouped by time periods, thereby improving detection accuracy while reducing false positives.
2Measurement precision
If multi-phase anomaly detection with time binning is applied, then noise is reduced and anomaly accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary actions by first computing observation anomaly scores for all time-stamped observations before grouping them into time bins. This preliminary computation of individual observation scores enables subsequent aggregation and analysis at the time bin level, improving anomaly detection accuracy while managing computational complexity through structured processing stages.
Solution Approach 2:
The patent introduces a temporal dimension by organizing observations into time bins, transforming the data structure from individual observations to aggregated time-period data. This dimensional change enables the system to capture temporal patterns and correlations between entities, reducing noise and improving anomaly detection accuracy while the systematic approach to data organization helps manage computational complexity.
3Loss of information
If two-stage analysis with SVD is applied to identify causal entities, then causal information is extracted, but processing time increases
Solution Approach 1:
The patent segments the causal analysis into two stages: (1) identifying anomalous time bins using aggregated anomaly scores, and (2) extracting causal information by analyzing the relationship between time bins and entities using SVD on the residuals matrix. This segmentation allows the system to first narrow down to potentially anomalous time periods before performing computationally intensive causal analysis, thereby reducing overall processing time while maintaining complete causal information extraction.
Solution Approach 2:
The patent performs preliminary identification of anomalous time bins before extracting causal information. By first determining which time bins are anomalous based on aggregated observation scores, the system can then focus the computationally intensive SVD analysis only on the residuals related to these anomalous periods, reducing processing time while preserving complete causal information for the identified anomalies.
Data Source
AI summary
In a two-phase anomaly detection applied to observations collected from networked entities, observation anomaly scores are obtained in phase 1. In phase 2, the observation anomaly scores are structured as a data matrix, each row of the data matrix corresponding to a time period and each column corresponding to an entity device of the set of networked entities. At least one time period of the data matrix as anomalous. Causal information about the at least one anomalous time period is extracted based on an angular relationship between a second-pass coordinate vector of the at least one time period and a second-pass coordinate vector of at least one entity of the set of networked entities, the second-pass coordinate vectors determined by applying a second-pass singular value decomposition (SVD) to a residuals matrix computed between the data matrix and an approximation of the data matrix via truncated SVD.