Feature Importance Confidence Assessment for ML Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning algorithms lack interpretability and struggle with confidence estimation and uncertainty in predictions, especially when extrapolating beyond their training data domain, and existing methods for feature importance and outlier detection are complex and costly.
Innovation Solution
Performing principal component analysis on model features to obtain uncorrelated inputs, fitting a proxy model, computing feature importance weights, and relaxing data distributions according to these weights to identify in- or out-of-distribution samples using weighted probabilities and visual cues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning algorithms are used for predictions, then prediction capability is improved, but interpretability and confidence estimation deteriorate
Solution Approach 1:
The patent introduces feature importance scores as an intermediary mechanism that bridges the gap between complex ML predictions and interpretable confidence assessment. These scores act as mediators that translate model internals into meaningful confidence metrics without requiring simplification of the underlying ML algorithms.
Solution Approach 2:
The patent replaces traditional mechanical/outlier detection methods with a data-driven approach using feature importance weights. Instead of relying on statistical thresholds or distance-based outlier detection, the system uses learned feature importances to dynamically assess confidence, substituting mechanical rules with adaptive statistical learning.
2Measurement precision
If traditional outlier detection methods are used, then outlier identification is improved, but complexity and cost increase
Solution Approach 1:
The patent changes the parameters used for outlier detection from traditional statistical measures (Z-scores, percentiles) to feature importance-based confidence scores. This parameter transformation allows the system to leverage existing ML model knowledge rather than requiring separate complex detection mechanisms.
Solution Approach 2:
The patent makes the feature importance computation serve multiple functions: it provides interpretability for model predictions, enables confidence estimation, and simultaneously performs outlier detection. This multi-functionality eliminates the need for separate outlier detection systems, reducing overall complexity.
3Ease of manufacture
If feature importance methods interrogate features independently, then computational simplicity is improved, but accuracy deteriorates for correlated features
Solution Approach 1:
The patent merges the assessment of correlated features by computing confidence scores in the feature space rather than treating each feature independently. This combining approach naturally accounts for feature correlations by evaluating the joint contribution of features to prediction confidence.
Data Source
AI summary
Embodiments presented provide for a method for establishing a confidence assessment for data. Data may be segregated by features importance during the confidence assessment, allowing evaluators the ability to determine the quality of data being processed by the method.