Random Feature Transformation Forests for Automated Feature Engineering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated feature engineering methods for predictive modeling incur high computational costs due to the need for extensive feature space exploration and explicit feature expansion, which is impractical with large datasets.
Innovation Solution
The implementation of random feature transformation forests (RFTF) classifiers that apply selected transformations to a dataset to create new features on-the-fly, selecting the most discriminative transform features and storing them at each decision tree node for improved feature engineering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If transformed feature space exploration through evaluation-guided search is used, then feature engineering performance is improved, but computational runtime cost increases
Solution Approach 1:
The patent pre-generates a large set of transformed features before the actual model training process. By performing feature transformation in advance and storing the results, the system avoids repeated computational work during evaluation-guided search, thus improving runtime efficiency while maintaining feature engineering performance
Solution Approach 2:
The patent extracts and stores only the most promising transformed features from the full feature space, separating the feature generation process from the feature selection process. This allows the system to work with a reduced feature set during model training, reducing computational runtime cost while preserving the benefits of extensive feature exploration
2Measurement precision
If explicit expansion of datasets with all transformed features is used, then feature engineering performance is improved, but memory cost increases
Solution Approach 1:
The patent extracts and retains only the most discriminative transformed features from the complete transformed feature set, discarding redundant features. This selective extraction reduces the memory footprint while preserving the essential information needed for high-performance feature engineering
Solution Approach 2:
Instead of selecting features to keep from the expanded feature set, the patent inverts the approach by selecting features to discard. This inversion allows the system to maintain high feature engineering performance with a reduced feature subset, thereby reducing memory cost
Data Source
AI summary
Embodiments for automated feature engineering by one or more processors are described. One or more selected transformations may be applied to a set of features in a dataset to create a set of transform features using random feature transformation forest (RFTF) classifiers. A transform feature may be selected from the set of transform features having a highest discriminative power as compared to other features of the set of transform features. At each node in a decision tree, store the selected feature, a split value, and the one or more selected transformations for the transform feature.