Synthesized Feature Reduction Using Correlation Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in managing synthesized features effectively, leading to repetitive data and increased dimensionality without improving predictive capability, resulting in computational inefficiencies and resource waste.
Innovation Solution
Implementing feature extraction and dimension-reducing operations on datasets with synthesized features, using correlation matrices to identify and remove redundant features, and generating a modified dataset based on a user-defined correlation threshold.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If deep feature synthesis algorithms are applied to generate new features from a dataset, then insights into correlations or patterns may be discovered, but the synthesized features become repetitive and noisy without increasing predictive capability
Solution Approach 1:
The patent extracts and removes redundant synthesized features from the feature set by calculating correlation matrices and identifying features that exceed a threshold correlation with existing features. This extraction process eliminates repetitive features while preserving informative ones, directly resolving the contradiction between discovering patterns and avoiding redundancy.
Solution Approach 2:
The patent changes the parameter of feature selection by introducing a correlation threshold parameter that controls which synthesized features are retained. By adjusting this parameter, the system can control the balance between preserving informative features and removing redundant ones, thereby managing the quantity of features while maintaining information quality.
2Productivity
If feature generation operations are performed without managing synthesized features, then a plethora of additional data is generated, but computational efficiency decreases and resources are wasted
Solution Approach 1:
The patent performs preliminary dimensionality reduction by removing redundant features before the features are used in downstream machine learning operations. This preliminary action prevents the waste of computational resources on processing redundant features, thereby improving overall productivity while reducing energy consumption.
Solution Approach 2:
The patent discards redundant synthesized features that do not provide additional predictive value, thereby recovering computational resources that would have been wasted on processing these features. This selective discarding maintains productivity by preserving useful features while eliminating waste.
Data Source
AI summary
A method includes obtaining a first dataset comprising a first set of features and generating a second set of features based on the first set of features set by providing the first dataset to feature primitive stacks that respectively corresponds to features of the second set of features. The method further includes determining a reduced feature set based on the second set of features and a count of correlation values between features of the second set of features, wherein the correlation values satisfy a correlation threshold. The method further includes storing the reduced feature set in a database in association with the first set of features based on a determination that a second dataset comprising the reduced feature set satisfies a set of criteria.