Anomaly Detection Analysis for Identifying Anomalous Variables
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods, such as PCA, fail to identify specific anomalous variables causing anomalies in data samples, leading to misidentification and performance issues due to dimensionality reduction and limited applicability to other models.
Innovation Solution
A computer-implemented method using imputation models to process subsets of variables, applying anomaly detection models to obtain imputed samples, and selecting variables based on lowest anomaly scores through exhaustive or greedy algorithms to identify anomalous variables.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If PCA model is used for anomaly detection, then anomaly detection capability is achieved, but identification of specific anomalous variables is lost due to dimensionality reduction
Solution Approach 1:
The patent segments the anomaly detection process into two distinct phases: first using PCA for anomaly detection, then applying contribution analysis to identify specific anomalous variables. This segmentation allows each method to perform its specialized function without compromise - PCA for reliable anomaly detection and contribution analysis for variable identification.
Solution Approach 2:
The patent introduces contribution analysis as an intermediary method that bridges the gap between PCA anomaly detection and variable identification. The contribution analysis decomposes the anomaly score into individual variable contributions, serving as a mediator that translates the dimensionality-reduced PCA results back into identifiable variable-level insights.
2Loss of information
If contribution analysis method is used to identify anomalous variables, then variable identification is attempted, but result accuracy is reduced due to ambiguous decomposition and cross-variable influence
Solution Approach 1:
The patent applies dynamic thresholding to the contribution analysis results, where the threshold for identifying anomalous variables is adjusted based on the distribution of contribution values and statistical properties of the data. This dynamic approach adapts to different data characteristics and anomaly types, improving identification accuracy compared to fixed thresholds.
Solution Approach 2:
The patent changes the parameter representation by transforming contribution values through normalization and ranking operations. By converting raw contribution values into standardized scores and ranked positions, the method enhances the precision of variable identification while reducing the impact of ambiguous decomposition.
3Productivity
If PCA model is used for dimensionality reduction, then computational efficiency is improved, but misidentification of anomalous variables occurs due to loss of variable-specific information
Solution Approach 1:
The patent segments the analysis into efficient PCA-based anomaly detection followed by precise contribution analysis for variable identification. This segmentation maintains computational efficiency by using PCA for the computationally intensive anomaly detection phase while reserving the more precise but computationally heavier contribution analysis for the subsequent identification phase.
Solution Approach 2:
The patent applies partial action by using contribution analysis selectively - only for variables that show significant contribution to the anomaly score. Rather than analyzing all variables equally, the method focuses computational resources on the most likely anomalous variables, improving accuracy while maintaining efficiency.
Data Source
AI summary
A computer implemented method of analyzing measurement results of a target system, such as an industrial process or communication network. The method includes receiving a data sample with a plurality of variables representing the measurement results, detecting that the data sample is an anomalous sample using an anomaly detection model, processing the data sample by applying an imputation model to selected subsets of variables of the data sample to obtain imputed samples, and applying the anomaly detection model to the imputed samples, determining anomalous variables of the data sample based on results from the processing of the data sample, and outputting the anomalous variables of the data sample for management operations in the target system.