Fidelity Metric for Attribution-Based Anomaly Detection Explainers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) explainability techniques face challenges in evaluating the quality of attribution-based methods, particularly in complex models like deep learning, where human-centered evaluations are time-consuming and unreliable, and functionally-grounded methods lack direct empirical measurement of explanation quality, especially for opaque models.
Innovation Solution
A novel fidelity metric for explainers based on feature attribution that is model-agnostic and explainer-agnostic, allowing for empirical evaluation of explanation quality by artificially creating anomalous datapoints through perturbation, which assesses the capability to identify anomalous features and provides a quantifiable measure of explanation quality without requiring prior labels or scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If human-centered evaluation methods are used to assess explanation quality, then reliability of evaluation may be improved, but time consumption and cost increase significantly
Solution Approach 1:
The patent creates artificial anomalous datapoints by perturbing features of existing data points, generating synthetic test cases that copy the structure and characteristics of real anomalies. This allows automated evaluation without requiring actual human analysts, thus resolving the contradiction between reliable evaluation and time consumption.
Solution Approach 2:
The evaluation method uses the anomaly detection model itself and automated fidelity calculations to assess explanation quality, rather than requiring external human evaluators. The system self-evaluates by comparing attribution explanations against known perturbed features, eliminating the time and cost burden of human-centered evaluation while maintaining evaluation reliability.
2Measurement precision
If axiomatic evaluation methods are used, then theoretical properties are satisfied, but direct empirical measurement of explanation quality is lost
Solution Approach 1:
The patent segments the evaluation process into distinct components: (1) generating artificial anomalies through feature perturbation, (2) obtaining explanations from the anomaly detection model, and (3) calculating fidelity metrics by comparing explanations against known perturbed features. This segmentation enables direct empirical measurement while managing complexity through modular, systematic evaluation steps.
3Adaptability or versatility
If evaluation methods for opaque models are developed, then applicability to deep learning models is improved, but interpretation of model operation becomes more difficult
Solution Approach 1:
The patent introduces artificial anomalous datapoints as intermediaries between the opaque anomaly detection model and the evaluation process. These synthetic test cases with known ground truth (the perturbed features) serve as a mediator that enables measurement of explanation quality without requiring interpretation of the model's internal operations, thus resolving the contradiction between model applicability and assessment difficulty.
Data Source
AI summary
Herein are machine learning (ML) explainability (MLX) techniques for calculating and using a novel fidelity metric for assessing and comparing explainers that are based on feature attribution. In an embodiment, a computer generates many anomalous tuples from many non-anomalous tuples. Each anomalous tuple contains a perturbed value of a respective perturbed feature. For each anomalous tuple, a respective explanation is generated that identifies a respective identified feature as a cause of the anomalous tuple being anomalous. A fidelity metric is calculated by counting correct explanations for the anomalous tuples whose identified feature is the perturbed feature. Tuples may represent entries in an activity log such as structured query language (SQL) statements in a console output log of a database server. This approach herein may gauge the quality of a set of MLX explanations for why log entries or network packets are characterized as anomalous by an intrusion detector or other anomaly detector.