Grouped PDP Ranking for Adverse Action Reason Codes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Financial services companies face challenges in generating accurate Adverse Action Reason Codes (AARCs) due to the complexity of machine learning models, making it difficult to adhere to regulations like the Equal Credit Opportunity Act, as the predictive capabilities of these models are hard to analyze and the contribution of input variables to output decisions is not immediately apparent.
Innovation Solution
A method utilizing grouped partial dependence plots (GPDP) framework to analyze machine learning models by clustering input variables based on dependencies, generating partial dependence plot tables for groups, and creating a ranking vector to determine the contribution of each group to the model output, enabling the generation of AARCs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are used to improve prediction accuracy and efficiency, then productivity and reliability are improved, but device complexity increases and measurement precision becomes difficult to achieve
Solution Approach 1:
The patent segments the complex machine learning model into interpretable components by dividing input variables into groups based on their dependencies and relationships. This segmentation allows the model to maintain high prediction accuracy while becoming more manageable and analyzable, directly addressing the contradiction between productivity and device complexity.
Solution Approach 2:
The patent introduces partial dependence plots as an intermediary mechanism between the complex model and the decision-maker. These plots serve as a mediator that translates the model's internal complexity into visual representations that show the relationship between input variables and output predictions, making the model's behavior transparent and interpretable.
2Productivity
If machine learning models are used to automate credit decisions, then productivity is improved, but ease of operation deteriorates due to difficulty in analyzing predictive capabilities
Solution Approach 1:
The patent uses partial dependence plots as an intermediary layer that bridges the gap between the complex machine learning model and the user. These plots provide intuitive visualizations that show how input variables affect predictions, making the model easy to operate and interpret while maintaining the productivity benefits of automation.
Solution Approach 2:
The patent employs visualizations with color coding and graphical representations to communicate model behavior. By using color changes and visual patterns in the partial dependence plots, the patent makes complex relationships between variables and predictions immediately understandable, greatly improving ease of operation.
3Ease of operation
If traditional statistical techniques are used to generate AARCs, then ease of operation is maintained, but manufacturing precision deteriorates due to inability to handle complex ML models
Solution Approach 1:
The patent segments the analysis process into two parts: using machine learning for accurate prediction and using partial dependence plots for interpretable analysis. This segmentation allows the system to achieve both high manufacturing precision through ML and ease of operation through visual analysis tools, overcoming the limitations of traditional statistical techniques.
Solution Approach 2:
The patent replaces manual mechanical analysis with automated machine learning models for prediction, while using visual partial dependence plots for interpretation. This substitution allows the system to handle complex models with the same ease of operation as traditional methods while achieving superior manufacturing precision through ML's advanced predictive capabilities.
4Measurement precision
If input variables are analyzed individually, then measurement precision is improved, but device complexity increases due to strong dependencies between variables
Solution Approach 1:
The patent segments input variables into groups based on their dependencies and relationships. This segmentation allows the system to analyze variables in a structured manner that accounts for their interdependencies, achieving measurement precision while managing device complexity through organized grouping rather than treating all variables independently.
Data Source
AI summary
A framework for interpreting machine learning models is proposed that utilizes interpretability methods to determine the contribution of groups of input variables to the output of the model. Input variables are grouped based on dependencies with other input variables. The groups are identified by processing a training data set with a clustering algorithm. Once the groups of input variables are defined, scores related to each group of input variables for a given instance of the input vector processed by the model are calculated according to one or more algorithms. The algorithms can utilize group Partial Dependence Plot (PDP) values, Shapley Additive Explanations (SHAP) values, and Banzhaf values, and their extensions among others, and a score for each group can be calculated for a given instance of an input vector per group. These scores can then be sorted, ranked, and then combined into one hybrid ranking.