Target-Driven Feature Engineering for Predictive Dimensionality Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data science methods face challenges in effectively extracting predictive signals from large datasets with numerous synthesized features, leading to inefficient model training and resource-intensive processing.
Innovation Solution
A method involving deep feature synthesis followed by dimensionality reduction using techniques like LDA and binary search to identify a dataset with an explained variance threshold, reducing the number of features while retaining predictive power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deep feature synthesis is performed to synthesize features from existing features, then the predictive capability of the dataset is improved, but the dimensionality and computational complexity increase
Solution Approach 1:
The patent extracts only the necessary features from the synthesized feature set by applying dimensionality reduction techniques. It identifies and retains the most informative features while discarding redundant ones, thereby maintaining predictive capability while reducing dimensionality.
Solution Approach 2:
The patent introduces an intermediary dimensionality reduction process between feature synthesis and model training. This intermediary step uses techniques like PCA or feature selection to bridge the gap between high-dimensional synthesized features and the practical requirements of machine learning models.
2Productivity
If multiple dimensionality reductions are run on the dataset, then the computational efficiency is improved, but the processing time and resource consumption increase
Solution Approach 1:
The patent applies partial action by running multiple dimensionality reduction techniques but only fully processing the most promising results. It uses a staged approach where initial reductions filter the data, and subsequent reductions are applied only to subsets, avoiding the need to fully process all possible reductions.
Solution Approach 2:
The patent performs preliminary dimensionality reduction to create a reduced feature set before applying more computationally intensive analysis. This preliminary step filters out obviously redundant features early, preventing wasted computational resources on subsequent processing of unnecessary features.
3Reliability
If the explained variance threshold is set high to ensure predictive accuracy, then the model reliability is improved, but the number of retained features increases reducing efficiency
Solution Approach 1:
The patent systematically varies the explained variance threshold parameter to find the optimal balance between accuracy and efficiency. By testing different threshold values and evaluating model performance at each level, it identifies the point where marginal gains in accuracy no longer justify the increased computational cost.
Solution Approach 2:
The patent implements feedback loops where model performance is continuously evaluated against the explained variance threshold. The system adjusts the threshold based on performance metrics, retaining features only when they contribute meaningfully to predictive accuracy, thereby avoiding retention of redundant features.
Data Source
AI summary
In some embodiments, an exemplary method may include receiving a first dataset having a first plurality of features, performing a deep feature synthesis to synthesize a second plurality of features from the first plurality of features, separating the first plurality of features from the second plurality of features to form a third plurality of features, generating a second dataset based on the third plurality of features, running a plurality of dimensionality reductions on the second dataset to generate a plurality of reduced datasets, wherein each dimensionality reduction produces a different dimension less than a dimension of the second dataset, calculating an explained variance (EV) of each of the plurality of reduced datasets to generate a plurality of EVs, identifying a particular EV from the plurality of EVs that is a smallest EV above a threshold, and selecting a particular reduced dataset corresponding to the particular EV as a target dataset.