Categorical Machine Learning Models with Statistical Feature Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive data analysis systems struggle with categorical input data, leading to inaccurate predictions, overfitting, and a lack of interpretability, especially in high-impact business contexts where explanations are required.
Innovation Solution
The use of categorical level merging, mutual-information-based feature filtering, and feature-correlation-based feature filtering to preprocess training data, followed by training categorical input machine learning models that generate interpretable predictions through statistically significant feature value arrangements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional predictive data analysis systems are used with categorical input data, then processing speed is maintained, but prediction accuracy deteriorates and overfitting occurs
Solution Approach 1:
The patent segments categorical features by creating separate binary indicator variables for each categorical level. This segmentation allows the model to process categorical data without imposing artificial ordinal relationships, thereby improving prediction accuracy while maintaining model reliability through proper statistical handling of each segment independently.
Solution Approach 2:
The patent changes the parameter representation of categorical data from traditional encoded formats to binary indicator variables. This parameter change enables the model to accurately capture relationships between categorical features and target variables without the distortions introduced by traditional encoding methods, resolving the contradiction between accuracy and reliability.
2Measurement precision
If detailed categorical features are used in the model, then prediction accuracy improves, but model complexity increases leading to overfitting
Solution Approach 1:
The patent extracts only the statistically significant categorical levels from the full set of categorical features. By identifying and removing irrelevant or redundant categorical levels through statistical testing, the model maintains high prediction accuracy while reducing complexity and preventing overfitting.
Solution Approach 2:
The patent applies partial action by including only those categorical levels that demonstrate statistical significance in predicting the target variable. This selective inclusion approach avoids the overfitting that would result from using all possible categorical levels, while still capturing the necessary predictive information.
3Ease of operation
If traditional machine learning models are used, then processing speed is maintained, but interpretability of predictions deteriorates
Solution Approach 1:
The patent creates a simplified copy of the categorical data structure using binary indicator variables that directly correspond to interpretable categorical levels. This copying approach maintains the essential predictive information while creating a model structure that is easily interpretable, as each binary variable can be directly associated with a specific categorical level without requiring complex decoding.
Data Source
AI summary
There is a need for more effective and efficient predictive data analysis based at least in part on categorical input data. This need can be addressed by, for example, solutions for performing predictive data analysis that utilize at least one of categorical level merging, mutual-information-based feature filtering, feature-correlation-based feature filtering to generate training data feature value arrangements, as well as training and using categorical input machine learning models trained using the training data feature value arrangements.