ML Model Explanation Generation Without Direct Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) model explanation techniques require access to the model, which is often unavailable due to confidentiality, privacy, or technical feasibility issues, making it difficult to generate explanations without querying the model or using model predictions on new data points.
Innovation Solution
A method and apparatus that generate ML explanations without using the ML model by receiving inputs and outputs from a remote compute device, using precomputed coalitions and their attributions stored in a database, and employing k-nearest neighbors regressors to predict feature attributions, allowing for instantaneous explanation generation without model ingestion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If known techniques to generate post-hoc explanations for an opaque model are used, then explanations can be generated to understand the rationale behind predictions, but access to the model is required which is sometimes unavailable due to business confidentiality, data privacy, and technical feasibility
Solution Approach 1:
The patent extracts only the necessary input-output pairs from the ML model system, separating the explanation generation process from the actual model. This allows explanations to be generated using only observed inputs and outputs without requiring access to the model's internal structure, weights, or architecture, thereby resolving the contradiction between maintaining model confidentiality and enabling explanation generation.
Solution Approach 2:
The patent introduces an intermediary explanation generation system that mediates between the ML model and the user. This intermediary observes inputs and outputs and generates explanations without directly accessing the model, acting as a bridge that enables transparency while preserving model security and confidentiality.
2Reliability
If model access is required for explanation generation, then comprehensive model understanding can be achieved, but this creates barriers due to business confidentiality, data privacy, and technical viability constraints
Solution Approach 1:
The patent segments the explanation generation process into independent components that operate without model access. By dividing the explanation task into observable input-output analysis and interpretation generation, the system achieves both reliable explanations and flexible deployment in environments where model access is restricted.
3Loss of information
If traditional explanation methods are used that require model access, then detailed model insights can be obtained, but latency and storage requirements increase
Solution Approach 1:
The patent performs preliminary analysis of input-output patterns to generate explanations without requiring real-time model access. By preparing and storing explanation-based insights in advance based on observed data patterns, the system reduces explanation generation latency while maintaining meaningful insights.
Data Source
AI summary
In an embodiment, a method includes receiving, via a processor of a first compute device, a representation of a set of inputs and a set of outputs that were generated by inputting the set of inputs into a machine learning (ML) model by a set of compute devices not including the first compute device to generate the set of outputs. The method further includes receiving, via the processor, a request for a machine learning (ML) explanation associated with the ML model and at least one explicand. The method further includes generating, via the processor and without using the ML model, a representation of the ML explanation based on the at least one explicand, the set of inputs, and the set of outputs.