Explainability Ensembles for Neural Networks in Regulated Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial intelligence systems, particularly in decision-making processes like misappropriation detection, lack explainability and interpretability, which is critical for reliability and compliance with regulatory requirements, and existing tools are insufficient for providing clear explanations of their decision-making processes.
Innovation Solution
A system utilizing an array of shadow models to analyze machine learning-derived misappropriation types, which includes a controller with a processing device to input interaction data into a machine learning model, identify data features, construct shadow models to extract logical constructs, and consolidate these constructs to provide a final explanation output, enhancing explainability through relevance visualization and reason codes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a machine learning model is used to improve decision-making accuracy in misappropriation detection, then the accuracy and productivity are improved, but the explainability and interpretability of the decision-making process deteriorate
Solution Approach 1:
The patent introduces shadow models as intermediary components that replicate the behavior of the main machine learning model while providing interpretable explanations. These shadow models act as mediators between the complex neural network and the end users, translating opaque predictions into understandable logical constructs without affecting the original model's decision-making accuracy
Solution Approach 2:
The system creates copies of the main machine learning model in the form of shadow models that mimic its predictive behavior. These copies are trained to reproduce the same decisions while being structured to provide human-readable explanations, thus preserving accuracy while improving interpretability
2Loss of information
If an array of shadow models is constructed to extract logical constructs and improve explainability, then the explainability is improved, but the device complexity increases
Solution Approach 1:
The patent segments the explanation generation task across multiple shadow models, where each model focuses on specific aspects or features of the input data. This segmentation allows the system to provide comprehensive explanations while keeping individual shadow models relatively simple and manageable
Solution Approach 2:
The shadow models are designed to serve multiple functions: they replicate the predictive behavior of the main model, extract logical constructs from different feature subsets, and generate human-readable explanations. This multi-functionality reduces the need for separate components and manages system complexity
3Reliability
If multiple shadow models are used to provide comprehensive explanations, then the reliability and explainability are improved, but the processing time and productivity are reduced
Solution Approach 1:
The system implements partial action by allowing users to select which shadow models to execute based on their specific needs. Not all shadow models need to run for every query - users can choose a subset appropriate for their explainability requirements, thus reducing processing time while maintaining reliability when needed
Solution Approach 2:
The shadow models are pre-trained offline to replicate the main model's behavior and learn logical constructs from historical data. This preliminary training allows the online system to quickly generate explanations without extensive real-time computation, improving processing speed while maintaining reliability
Data Source
AI summary
A system for analyzing machine learning-derived misappropriation types with an array of shadow models is provided. The system comprises: a controller configured for analyzing an output of a machine learning model, the controller being further configured to: input interaction data into a machine learning model, wherein the interaction data is analyzed using the machine learning model to determine a misappropriation type output associated with the interaction data; identify data features in the interaction data associated with the misappropriation type output; construct an array of shadow models based on the data features, wherein each individual model in the array of shadow models is configured to extract logical constructs from a portion of the data features; and consolidate the logical constructs output by the array of shadow models, wherein consolidating the logical constructs determines a final explanation output for the misappropriation type output determined by the machine learning model.