Categorical Input Machine Learning Models for Interpretable Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive data analysis systems face challenges in handling categorical input data due to issues like noise introduction from imposing ordinality, overfitting, and the lack of interpretability, which affects the accuracy and reliability of predictions, especially in high-impact business contexts where explanations are required.
Innovation Solution
The development of methods that utilize categorical level merging, mutual-information-based feature filtering, feature-correlation-based feature filtering, and training categorical input machine learning models to generate interpretable predictions by refining features based on predictive correlations and noise reduction, ensuring both accuracy and explainability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional machine learning models process categorical data by imposing ordinality, then the data can be processed mathematically, but noise is introduced and prediction accuracy deteriorates
Solution Approach 1:
The patent changes the parameter representation of categorical data by introducing target-encoded features that capture the relationship between categorical levels and target variables. Instead of using arbitrary ordinal codes, the system transforms categorical values into continuous features representing their predictive power, thereby maintaining mathematical processability while eliminating noise from imposed ordinality
Solution Approach 2:
The patent introduces target encoding as an intermediary transformation layer between raw categorical data and the machine learning model. This intermediary process computes statistical relationships (mean, median, mode) between categorical levels and target variables, creating new features that mediate the connection without imposing false ordinal relationships
2Productivity
If machine learning models use all available categorical features, then comprehensive analysis is achieved, but overfitting occurs and model reliability decreases
Solution Approach 1:
The patent extracts only the most relevant categorical features by computing target encoding statistics and selecting features based on their predictive strength. Instead of using all categorical features, the system extracts and retains only those that show significant correlation with the target variable, thereby preventing overfitting while maintaining comprehensive analysis of important factors
Solution Approach 2:
The patent applies partial action by selectively encoding only certain categorical features rather than all of them. The system identifies and encodes features that meet specific criteria (such as minimum frequency or maximum information gain), performing encoding on a subset of features that provides sufficient predictive power without the noise introduced by encoding all features
3Measurement precision
If machine learning models provide detailed predictions, then accuracy is improved, but interpretability and explainability are lost
Solution Approach 1:
The patent creates a parallel interpretable representation by copying the target encoding process into human-readable explanations. The system generates explanations that mirror the mathematical computations, showing which categorical features and their encoded values contributed to predictions, thereby providing both accurate predictions and understandable reasoning in a parallel format
Solution Approach 2:
The patent implements feedback loops that generate explanations based on the model's internal computations. The target encoding statistics and feature importances are fed back into the explanation generation process, creating interpretable outputs that directly reflect the model's decision-making logic and allow users to understand and verify the reasoning behind predictions
4Speed
If categorical data is processed without target encoding, then processing speed is maintained, but prediction accuracy and reliability deteriorate
Solution Approach 1:
The patent performs target encoding as a preliminary action during the training phase, pre-computing the statistical relationships between categorical features and target variables. This preliminary encoding creates ready-to-use numerical features that can be quickly applied during prediction, maintaining fast processing speed while incorporating the accuracy benefits of target-aware transformation
Data Source
AI summary
There is a need for more effective and efficient predictive data analysis based at least in part on categorical input data. This need can be addressed by, for example, solutions for performing predictive data analysis that utilize at least one of categorical level merging, mutual-information-based feature filtering, feature-correlation-based feature filtering to generate training data feature value arrangements, as well as training and using categorical input machine learning models trained using the training data feature value arrangements.