Feature Group Influence Analysis for AI Model Interpretability

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current machine learning models lack interpretability, making it difficult for organizations to understand and justify decisions, such as loan rejections, which is crucial for compliance and improving model performance.

Innovation Solution

The system identifies and analyzes groups of features used by machine learning models to determine their influence on model outputs, using techniques like Shapley values and similarity metrics to provide insights into feature importance and model behavior.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If machine learning models are used to make decisions, then decision accuracy and automation are improved, but interpretability and compliance are worsened

Engineering Contradiction:
Improvedecision automationVSAvoidinterpretability
Core Design Contradiction:
Extent of automationVSLoss of information

Solution Approach 1:

The patent introduces Shapley value analysis as an intermediary mechanism that bridges the gap between complex machine learning models and human interpreters. By computing feature importance scores through Shapley values, the system provides a mediator layer that translates model decisions into comprehensible explanations without altering the underlying model architecture or decision-making process.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces traditional mechanical explanation methods (such as rule-based systems or feature importance plots) with a game-theoretic approach using Shapley values. This substitution provides a mathematically rigorous framework for attribute importance analysis that better handles complex interactions between features while maintaining interpretability.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Loss of information

If feature groups are analyzed using Shapley values, then model interpretability is improved, but computational complexity and time are worsened

Engineering Contradiction:
ImproveinterpretabilityVSAvoidcomputational time
Core Design Contradiction:
Loss of informationVSLoss of time

Solution Approach 1:

The patent segments the feature set into multiple groups or subsets, allowing Shapley value computations to be performed on smaller groups rather than all features simultaneously. This segmentation reduces the computational burden of calculating Shapley values while still providing comprehensive interpretability across the entire feature set through aggregated results.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent computes Shapley values for a selected subset of features or feature groups rather than exhaustively computing them for all possible feature combinations. This partial action approach provides sufficient interpretability for practical purposes while significantly reducing computational time and resources required.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS20230097940A1System and method for extracting and using groups of features for interpretability analysis
Publication Date: 2023.03.30 SNOWFLAKE INC
  • US20230097940A1 patent drawing
  • US20230097940A1 patent drawing
  • US20230097940A1 patent drawing

AI summary

In some implementations, a computing machine accesses an artificial intelligence model and a dataset for the artificial intelligence model, the dataset comprising at least one datapoint. The computing machine identifies a feature group used by the artificial intelligence model, the feature group comprising at least two features having a similarity with one another exceeding a similarity threshold, wherein the feature group comprises a subset of the features used by the artificial intelligence model. The computing machine determines an overall influence value for the feature group on an output of the artificial intelligence model applied to the dataset. The computing machine provides an output representing the overall influence value.