Anomaly Threshold Refinement for Multivariate Detection Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection models in data analytics platforms often produce overly-aggressive or conservative anomaly thresholds due to reliance on non-anomalous training data, leading to false positives or false negatives, and lack data-driven accuracy.
Innovation Solution
A process that refines anomaly thresholds by using training data to establish starting values and then employing test data to differentiate between non-anomalous and anomalous variables, iteratively adjusting thresholds to optimize detection, using a range of multiplier values to balance over-detection and under-detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly thresholds are established using only non-anomalous training data with predefined tolerances, then false positives are reduced, but false negatives increase and detection accuracy decreases
Solution Approach 1:
The patent segments the threshold establishment process into two distinct phases: (1) establishing initial thresholds using only non-anomalous training data with predefined tolerances to minimize false positives, and (2) refining those thresholds using test data that may contain anomalies to improve detection accuracy. This segmentation allows each phase to optimize for its specific goal without compromising the other.
Solution Approach 2:
The patent performs preliminary threshold establishment using conservative predefined tolerances before final refinement. By first setting thresholds that are guaranteed to minimize false positives, then iteratively refining them with test data, the system ensures a reliable starting point that prevents excessive false alarms while allowing subsequent optimization for detection accuracy.
2Ease of operation
If conservative anomaly thresholds are established using predefined tolerances, then false positives are minimized, but detection sensitivity decreases leading to false negatives
Solution Approach 1:
The patent implements feedback by using test data to evaluate the performance of initially established thresholds and iteratively refine them. The system monitors detection results and adjusts thresholds based on actual performance metrics, allowing optimization of both false positive rate and detection reliability through continuous feedback loops.
Solution Approach 2:
The patent transforms the static threshold establishment process into a dynamic iterative refinement process. Thresholds evolve from initial conservative values to optimized values through multiple iterations of testing and adjustment, allowing the system to adapt and optimize for both ease of operation and reliability based on actual data patterns.
3Reliability
If anomaly thresholds are set to maximize detection sensitivity, then fewer false negatives occur, but false positives increase due to overly-aggressive thresholds
Solution Approach 1:
The patent segments the threshold optimization process into distinct stages: initial threshold establishment focused on minimizing false positives, followed by iterative refinement stages that balance sensitivity improvement with false positive control. Each stage has specific optimization goals that prevent the harmful effects of overly-aggressive thresholds.
Solution Approach 2:
The patent applies partial action by iteratively adjusting thresholds in controlled increments rather than making large aggressive changes. This allows the system to progressively improve detection sensitivity while monitoring and controlling false positive rates, avoiding the harmful effects of excessive threshold adjustments.
Data Source
AI summary
A computing system may create an anomaly detection model to detect anomalies in multivariate data originating from a given data source by extracting a model object for the anomaly detection model using a first set of training data originating from the given data source, establishing starting values of a set of anomaly thresholds for the anomaly detection model using the extracted model object and a second set of training data originating from the given data source, and refining the starting values of the set of anomaly thresholds for at least a subset of the variables included in the multivariate data using the extracted model object and a set of test data. In turn, the computing system may use the anomaly detection model to monitor for anomalies in observation data originating from the given data source.