ML Prediction Interpretation via Dynamic Surrogate Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interpreting machine learning models is challenging due to their dynamic and complex mathematical design, especially in real-time data analysis, as existing techniques rely on static mathematical functions and do not accurately consider the pre-defined thresholds or training data, leading to potential economic risks in business applications.
Innovation Solution
A method and system that compute a contribution factor for each predictor in a machine learning model by creating duplicate data sets with random values, using pre-trained statistics, and calculating final predictions based on a pre-defined threshold, to determine the importance of predictors during predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If existing techniques use static mathematical functions learnt during model training to interpret ML models, then the interpretation process is simplified, but the reliability for real-time data decreases
Solution Approach 1:
The patent applies dynamics by transitioning from static mathematical functions to dynamic sampling-based interpretation. Instead of using fixed functions learned during training, the system dynamically generates multiple duplicate datasets by sampling from the original dataset, allowing the interpretation to adapt to real-time data characteristics and capture the actual behavior of the ML model under varying conditions.
Solution Approach 2:
The patent uses copying by creating multiple duplicate datasets that replicate the structure and characteristics of the original training data. These copied datasets are then used to train surrogate models, enabling the interpretation process to reflect the actual data distribution and model behavior without directly exposing the proprietary ML model internals.
2Ease of operation
If white box models are created to interpret ML models, then the interpretation is simplified, but the accuracy decreases because thresholds and training data are not considered
Solution Approach 1:
The patent applies preliminary action by pre-processing the original dataset to create duplicate datasets that preserve the statistical properties, feature distributions, and threshold characteristics of the training data. This preliminary preparation ensures that subsequent surrogate models are trained on data that accurately reflects the conditions under which the ML model operates, thereby maintaining measurement precision.
Solution Approach 2:
The patent uses parameter changes by systematically varying the sampling parameters and data characteristics when generating duplicate datasets. This allows the interpretation process to capture the sensitivity of the ML model to different input conditions and threshold values, improving the accuracy of the interpretation while maintaining operational simplicity through automated parameter adjustment.
3Productivity
If ML models use dynamic complex mathematical design for predictive analysis, then the predictive performance is improved, but the interpretability of predictions deteriorates
Solution Approach 1:
The patent introduces an intermediary approach by using surrogate models trained on duplicate datasets as mediators between the complex ML model and the interpretation process. These surrogate models capture the essential input-output relationships of the ML model without exposing its complex internal mathematics, thereby preserving predictive performance information while making it interpretable through simpler, more transparent models.
Data Source
AI summary
This disclosure relates to field of machine learning. The outcome of the ML model translates to economic damage for a business. While the risk associated with outcome of ML model can be mitigated by interpreting/explanation each prediction of the ML model, interpreting ML models is challenging as ML models are built in dynamically changing complex mathematical design. The disclosure is a technique for interpreting machine learning model's prediction by computing a percentage contribution of each of the predictors. The percentage contribution indicates an importance of each of the predictors during prediction by the ML model. The percentage contribution is computed in several steps using several parameters associated with the ML Model including a pre-defined threshold of the ML model, an input feature vector comprising a plurality of predictors, an original prediction (N) for each predictor, a pre-determined duplication factor, and a plurality of pre-trained data statistics.