Mixed Initiative Feature Engineering via Interactive Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Feature engineering in data analysis and machine learning is often effort-intensive and lacks a scientific approach, making it difficult to efficiently create new features from existing ones, especially in tasks like classification, regression, clustering, and data imputation.
Innovation Solution
A mixed initiative feature engineering approach that combines computerized interactive feature visualization with user feedback to rank and transform features, automatically constructing new features through a system that employs transformation functions based on user selection and ranking metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual feature engineering is performed, then feature quality can be controlled, but the process is effort-intensive and time-consuming
Solution Approach 1:
The system enables automated feature engineering where the computer automatically generates, ranks, and transforms features based on visualizations and user feedback, reducing manual effort while maintaining quality control through iterative user interaction
Solution Approach 2:
The system implements a feedback loop where user interactions with feature visualizations guide automated feature transformation, allowing the system to learn from user preferences and continuously improve feature quality while reducing manual intervention
2Productivity
If automated feature engineering is implemented, then productivity increases, but feature quality and relevance may deteriorate
Solution Approach 1:
The system introduces an intermediary layer of automated analysis and visualization that bridges manual user input and automated feature generation, allowing users to guide the process through feedback on visualizations while the system handles the computationally intensive automated feature engineering
Solution Approach 2:
The system performs preliminary automated analysis and generates feature visualizations before final feature transformation, allowing users to review and provide feedback on feature relevance before the automated system commits to specific transformations, ensuring quality control
3Measurement precision
If comprehensive feature analysis is performed, then feature relevance improves, but system complexity increases
Solution Approach 1:
The system segments the complex feature engineering process into distinct automated components: data analysis, visualization generation, user feedback collection, and feature transformation. Each component handles a specific aspect, reducing overall system complexity while maintaining comprehensive analysis capabilities
Data Source
AI summary
Computerized interactive feature visualization is carried out on a data set—a plurality of insight classes rank a plurality of features of the data set. Via a computerized user interface, user feedback is obtained based on the interactive feature visualization—a user selects and ranks a subset of the features. At least one transformation function is applied to at least one feature of the subset of features selected by the user, to automatically construct, with a computer, at least one additional feature for the data set. The data set with the at least one additional feature is a transformed data set. In some cases, a supervised task is carried out on the final data set; accuracy of a machine learning system implementing the at least one supervised task can be enhanced by the at least one additional feature, and/or a physical system can be controlled based on results of the at least one supervised task.