Influence Score Normalization for Interpretable ML Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning algorithms (MLAs) face challenges in providing interpretable explanations for their predictions, as modern black box models output obscure results, making it difficult for users to trust and understand the decision-making process, with existing influence functions often highlighting outliers rather than representative data points.
Innovation Solution
A method and system that apply a normalizing function to influence scores determined by the influence function, specifically using cosine similarity and magnitude reduction, to identify and display relevant training data points that are more closely related to the prediction, thereby providing a more useful explanation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an influence function is used to identify influential training data points, then the most influential points can be determined, but the identified points are typically outliers that are not representative of the training data distribution
Solution Approach 1:
The patent applies a normalization transformation to the influence scores, changing the parameter scale from raw influence values to normalized scores that account for data point probability. This transformation converts the measurement from absolute influence to relative influence weighted by data representativeness, thereby resolving the contradiction between identifying highly influential points and providing interpretable, representative explanations
2Reliability
If uncommon training data points (outliers, mislabeled examples, low probability points) are identified as most influential, then robustness analysis can be performed, but these points do not provide useful explanations for specific predictions
Solution Approach 1:
The patent introduces a probability-based weighting that creates local quality differentiation in the influence measurement. Instead of treating all data points uniformly, the normalization function applies local adjustments based on each point's probability in the training distribution, thereby preserving robustness analysis capability while enhancing explanatory quality for representative points
3Productivity
If training data points with very broad impact on model behavior are identified, then the overall model sensitivity can be analyzed, but the impact is not specific to a particular prediction
Solution Approach 1:
The patent segments the influence measurement into prediction-specific components by normalizing scores individually for each prediction query. This segmentation allows the system to maintain overall model sensitivity analysis while providing prediction-specific explanatory information, as each normalized score reflects the influence of training points on a particular prediction rather than aggregate model behavior
Data Source
AI summary
There is disclosed a method and system for identifying relevant training data points for a prediction by a machine learning algorithm (MLA). The method comprises determining indicators of influence for training data points used to train the MLA. A normalizing function is applied to the indicators of influence to determine normalized indicators of influence. Training data points having the highest and lowest normalized indicators of influence are output for display.