Anomaly Threshold Selection With Quantitative Explanations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Zero trust systems face challenges with high false alert rates from anomaly detection, hindering effective remediation and impacting system performance due to the lack of justifiable anomaly identification.
Innovation Solution
A framework for anomaly detection that incorporates frequent pattern mining to identify feature-outlier combinations, computes risk ratios, and generates quantitative explanations using feature-value ranges to improve detection accuracy and reduce false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If anomaly detection techniques are used to identify abnormal events in zero trust systems, then security detection capability is improved, but false alert rate increases
Solution Approach 1:
The patent segments the anomaly detection process into multiple phases: (1) computing outlier scores for each feature, (2) determining Boolean transactions based on outlier scores, (3) performing frequent pattern mining to generate item sets, (4) sorting item sets by support and risk ratios, and (5) selecting thresholds. This segmentation allows systematic control over false alert generation while maintaining detection capability.
Solution Approach 2:
The patent performs preliminary actions by computing outlier scores and determining Boolean transactions before final anomaly identification. This preliminary processing creates a structured foundation (item sets with support and risk ratio metrics) that enables more accurate threshold selection and reduces false alerts in the final detection phase.
Solution Approach 3:
The patent implements feedback mechanisms by computing risk ratios for item sets and using these ratios to sort and select thresholds. The risk ratio metric provides feedback on the quality of anomaly detections, allowing the system to adjust threshold selection to minimize false alerts while maintaining detection effectiveness.
2Productivity
If traditional anomaly detection is used without justification, then detection speed is improved, but remediation effectiveness deteriorates
Solution Approach 1:
The patent introduces an intermediary explanation layer that connects anomaly detection results to remediation actions. By generating explanations that identify specific feature-outlier combinations and their risk ratios, the system bridges the gap between automated detection and human remediation, making the detection process both fast and actionable.
Solution Approach 2:
The patent changes the parameter representation from raw anomaly scores to structured explanations involving feature-outlier combinations, support values, and risk ratios. This parameter transformation maintains detection speed while improving remediation effectiveness by providing actionable insights about which features contribute most to detected anomalies.
3Loss of information
If frequent pattern mining is performed to generate item sets, then explanation quality is improved, but computational complexity increases
Solution Approach 1:
The patent extracts only the most relevant information from frequent pattern mining results by sorting item sets based on support and risk ratio metrics, then selecting only the top item sets for threshold determination. This extraction approach maintains explanation quality by focusing on the most significant patterns while reducing computational complexity by discarding less relevant item sets.
Solution Approach 2:
The patent applies local quality by differentiating the treatment of item sets based on their local characteristics (support and risk ratio). High-support, high-risk-ratio item sets receive detailed analysis and threshold selection, while lower-priority item sets are processed more simply. This localized approach optimizes computational resources while maintaining explanation quality where it matters most.
Data Source
AI summary
A framework for extracting quantitative and comparable explanations in terms of feature-value ranges for anomaly detection models based on outlier scores is disclosed. In a first phase, outlier scores are computed for a data set. In a second phase, thresholds per feature, which are used to identify abnormal entries or records in the data set, are extracted. In a third phase, a map between feature-outlier scores and feature-value ranges is generated. The feature-value ranges represent explanations. The explanations may include extracting quantitative metrics to explain a model.