Automated Predictive Model Generation via Feature Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data analysis methodologies for generating predictive models are time-consuming, prone to errors, and require extensive expertise, often necessitating a large pool of data scientists to choose and apply appropriate features and models, especially when dealing with large volumes of noisy data.
Innovation Solution
A system and method that automatically generates predictive models by receiving data, extracting predictor features using a modified mutual information technique and Pearson co-efficient, identifying data types, and applying appropriate models such as classification or regression models based on data types, utilizing a distributed framework for high-performance output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data analysis methodology is used with multiple data scientists, then model accuracy can be improved through expert selection, but time consumption and operational complexity increase significantly
Solution Approach 1:
The system enables self-service automated model generation where the computational system automatically performs feature selection, data type identification, and model generation without requiring manual intervention by data scientists, thereby reducing time consumption while maintaining accuracy through algorithmic optimization
Solution Approach 2:
The system changes the parameters of the analysis process by using automated algorithms with configurable parameters (such as mutual information thresholds, Pearson co-efficient values, and model selection criteria) to replace manual expert judgment, achieving both speed and accuracy through optimized parameter settings
2Reliability
If extensive feature selection and model choice analysis is performed manually, then model reliability improves, but device complexity and operational difficulty increase
Solution Approach 1:
The system segments the complex model generation process into distinct automated modules: data reception, feature selection (using mutual information and Pearson co-efficient), data type identification, and model generation. Each module handles a specific task reliably without requiring overall process complexity to be managed manually
Solution Approach 2:
The system introduces an automated intermediary framework that mediates between raw data and final predictive models, using intermediate computational steps (feature extraction, data type classification) to bridge the gap and ensure reliability without exposing the complexity to users
3Productivity
If automated feature selection using mutual information and Pearson co-efficient is implemented, then productivity increases, but measurement precision may be affected by algorithmic limitations
Solution Approach 1:
The system merges multiple feature selection techniques (mutual information for non-linear relationships and Pearson co-efficient for linear relationships) into a unified automated process, combining their strengths to maintain measurement precision while achieving high productivity through systematic evaluation of multiple correlation metrics
4Adaptability or versatility
If multiple data types require different model approaches, then adaptability improves, but ease of operation deteriorates due to increased complexity in model selection
Solution Approach 1:
The system performs self-service automated data type identification and model selection, automatically detecting whether data is time-series or non-time-series and selecting appropriate models (ARIMA, Holt-Winters for time-series; regression, classification for non-time-series) without requiring user knowledge of the complexities involved
Data Source
AI summary
The present disclosure relates to a system(s) and method(s) for generating a predictive model, the method comprises receiving data and extracting one or more predicator features from the data based on a feature selection methodology. In one example, the feature selection methodology comprises computing a degree connectedness for each of the plurality of features using a modified mutual information technique and a Pearson co-efficient and identifying the one or more predicator features on a comparison of degree of connectedness and a predefined threshold. Further, the method comprises identifying a data type associated with the data, and generating a predictive model to be applied on the data based on the data type and the one or more predicator features.