Anomaly Cause Identification Using Diffusion Map Gradients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods fail to identify the specific parameters responsible for anomalous behavior in multidimensional data, hindering the identification of the root cause of undesirable events.
Innovation Solution
A method using diffusion maps to calculate gradients for each parameter of an anomaly, rating them based on their contribution to abnormality, thereby identifying the parameter(s) most responsible for the anomaly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If dimensionality reduction methods are applied to detect anomalies in multidimensional data, then anomaly detection capability is improved, but the ability to identify specific causing parameters is lost
Solution Approach 1:
The patent segments the anomaly detection process into two distinct stages: first, dimensionality reduction using diffusion maps to identify anomalous datapoints; second, gradient-based parameter ranking to identify specific causing parameters. This segmentation allows each stage to optimize for its specific function without compromising the other, resolving the contradiction between anomaly detection and parameter identification.
Solution Approach 2:
The patent introduces a new dimensional aspect by computing gradients in the parameter space alongside the existing dimensional reduction in data space. By adding this gradient dimension, the system simultaneously achieves anomaly detection in reduced dimensionality while preserving parameter contribution information through the gradient vectors, thus resolving the information loss contradiction.
2Productivity
If nonparametric anomaly detection methods are used, then computational efficiency is improved, but parameter contribution analysis becomes impossible
Solution Approach 1:
The patent changes the approach by computing gradients of the diffusion map embedding with respect to the original parameters. This parameter change allows the system to maintain the computational efficiency of nonparametric methods while adding the capability to analyze parameter contributions, as the gradients directly indicate which parameters drive the anomaly detection score changes.
3Measurement precision
If gradient-based parameter ranking is added to diffusion maps, then parameter identification capability is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-computing the diffusion map embedding and its gradients during normal operation. When an anomaly is detected, the system only needs to evaluate the gradient-based ranking for the specific anomalous datapoint, rather than recomputing everything from scratch. This preliminary preparation significantly reduces the computational complexity of real-time parameter identification.
Data Source
AI summary
Methods that rate (i.e., ranks) the parameters (features) that cause a specific abnormal behavior to occur. It is assumed that the anomalies have already been detected by other anomaly detection methods. A method uses an algorithm based on the underlying potential of the diffusion process that is used in Diffusion Maps to reduce the dimensionality of the data. The gradient of this potential indicates the direction from an anomaly to a cluster that represents a normal behavior. This direction is used to rate the parameters that cause an abnormal behavior to occur.