XAI Committee Auditing for Adversarially Robust Model Explanations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing explainable artificial intelligence (XAI) techniques are vulnerable to adversarial attacks that manipulate explanations to conceal biases in machine learning models, undermining their reliability and fairness, particularly in critical decision-making scenarios.
Innovation Solution
Implementing a committee of diverse XAI techniques to audit ML models, which combines different XAI methods such as gradient, perturbation, and class activation maps, to detect and thwart adversarial attacks by exploiting the specificity of each technique's vulnerabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single XAI technique is used to explain ML model results, then the explanation process is simple and fast, but the system becomes vulnerable to adversarial attacks that can manipulate explanations to hide biases
Solution Approach 1:
The patent divides the XAI system into multiple independent techniques (committee of XAI techniques), where each technique generates its own explanation separately. This segmentation allows the system to maintain speed for each individual explanation while the collective system provides robustness against adversarial attacks through cross-verification of multiple explanations.
Solution Approach 2:
The patent combines multiple XAI techniques into a unified auditing framework where explanations from different techniques are integrated and evaluated together. The committee approach merges the strengths of individual techniques while mitigating their individual vulnerabilities, achieving both efficiency and reliability.
2Reliability
If multiple diverse XAI techniques are combined in a committee, then the system becomes more robust against adversarial attacks, but the system complexity and computational cost increase
Solution Approach 1:
The patent segments the complex auditing task into independent modules, where each XAI technique operates as a separate component that can be independently selected and executed. This modular segmentation manages system complexity by allowing dynamic configuration of the committee size and composition based on specific audit requirements.
Solution Approach 2:
The patent creates a universal auditing framework that can accommodate multiple XAI techniques through a common interface and evaluation protocol. This multi-functional design allows the same committee structure to handle different ML models and explanation types, reducing overall system complexity through standardization.
3Reliability
If XAI techniques are used to audit ML models for biases, then fairness and transparency are improved, but the models can be manipulated to generate deceptive explanations
Solution Approach 1:
The patent applies preliminary anti-action by having the committee of XAI techniques proactively detect and flag potential adversarial manipulations before they can deceive the auditing process. The system anticipates attacks by comparing explanations across multiple techniques and identifying inconsistencies that indicate manipulation attempts.
Solution Approach 2:
The patent implements feedback mechanisms where the committee continuously monitors and evaluates explanations from all techniques, using the results to adjust the auditing process. This feedback loop enables the system to adapt to adversarial strategies and maintain reliable fairness assessments despite manipulation attempts.
Data Source
AI summary
A method includes assembling an explainable artificial intelligence committee comprising two or more explainable artificial intelligence techniques, performing the explainable artificial intelligence techniques on results generated by a machine learning model, as a result of the performing, obtaining respective explanations, generated by each of the explainable artificial intelligence techniques, for the results generated by the machine learning model, and determining that one of the explanations was compromised by an attacker.