Automated Feature Selection for Missing Data Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In intelligent enterprises, predicting missing attribute values in large datasets is a time-consuming and computationally expensive task due to the complexity of determining relevant attributes for prediction.
Innovation Solution
The proposed solution involves preprocessing structured data using cleaning policies, filtering out highly correlated features through correlation-based filtering, and applying machine learning algorithms to select a subset of features based on relative importance for predicting missing values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all available attributes are used to predict missing attribute values, then prediction accuracy is improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent extracts and removes irrelevant or redundant attributes from the full attribute set through automated feature selection. The system identifies and retains only the most relevant attributes for prediction, eliminating unnecessary computational processing while maintaining prediction accuracy.
Solution Approach 2:
The patent applies different processing treatments to different attributes based on their individual characteristics and relevance to the target attribute. Each attribute is evaluated and processed according to its specific quality and contribution to prediction, rather than applying uniform processing to all attributes.
2Measurement precision
If comprehensive feature analysis is performed to identify relevant attributes, then prediction quality is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary feature selection and filtering before the actual prediction process. By pre-identifying and ranking relevant attributes using correlation analysis and feature importance metrics, the system prepares the optimal feature set in advance, reducing processing time during actual prediction operations.
Solution Approach 2:
The patent dynamically adjusts processing parameters such as correlation thresholds and feature selection criteria based on data characteristics. This allows the system to optimize the balance between comprehensive analysis and processing efficiency, adapting to different datasets and prediction scenarios.
3Productivity
If manual attribute selection is performed to reduce computational load, then processing efficiency is improved, but operational complexity increases
Solution Approach 1:
The patent implements automated feature selection that performs attribute analysis, ranking, and selection without manual intervention. The system self-determines the optimal feature set through algorithmic processes including correlation analysis and machine learning-based feature importance evaluation, eliminating the need for manual attribute selection while maintaining processing efficiency.
Solution Approach 2:
The patent replaces manual mechanical attribute selection processes with automated computational algorithms. Machine learning models and statistical analysis automatically identify and select relevant features, substituting human operational complexity with systematic algorithmic processing that is both efficient and reproducible.
Data Source
AI summary
Technologies are described for identifying features that can be used to predict missing attribute values. For example, a set of structured data can be received comprising a plurality of features and one or more labels. The set of structured data can be pre-processed, comprise applying one or more cleaning policies to produce a set of pre-processed features. The set of pre-processed features can be filtered using correlation-based filtering that uses one or more correlation estimation techniques to remove at least some highly correlated features. The correlation-based filtering can produce a set of filtered features. Feature subset selection can be performed comprising applying machine learning algorithms to the set of filtered features to determine relative importance among the set of filtered features. Based on the relative importance, a subset of the set of filtered features can be determined. The subset of the set of filtered features can be output.