Automated Feature Generation for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Feature engineering in machine learning applications is time-consuming and inefficient, requiring data scientists to manually develop hundreds of features, which consumes valuable expert time and limits the ability to test and develop predictive models effectively.
Innovation Solution
The implementation of an automated feature generation method using ensemble learning, which involves identifying preliminary features through aggregation functions, calculating their quality based on optimization criteria, and incrementally adjusting conditions to select the best features, thereby reducing the computational burden and increasing scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual feature engineering is performed by data scientists, then feature quality and model accuracy are improved, but time consumption and expert resource requirements increase significantly
Solution Approach 1:
The system enables automated feature engineering where the computational system performs feature generation, selection, and optimization automatically using machine learning algorithms, eliminating the need for manual data scientist intervention in the feature engineering process while maintaining feature quality through algorithmic optimization
Solution Approach 2:
The patent replaces the manual mechanical process of feature engineering with an automated computational system that uses machine learning models, optimization algorithms, and automated data processing to generate and select features, substituting human expert labor with an autonomous computational mechanism
2Measurement precision
If hundreds of features are hand-crafted by data scientists, then model predictive accuracy is improved, but the time available for model testing and development decreases
Solution Approach 1:
The system performs preliminary automated feature engineering and feature selection before model development, pre-processing and optimizing features in advance so that data scientists can directly proceed to model testing and development without spending extensive time on manual feature creation, thereby increasing model development throughput
Solution Approach 2:
The automated feature engineering system independently generates and optimizes features without requiring data scientist intervention, allowing the team to allocate more time to model testing and development activities, thus improving overall model development productivity
3Measurement precision
If expert time is allocated to manual feature engineering, then feature quality is improved, but the cost of the project increases due to expensive expert resources
Solution Approach 1:
The system performs feature engineering autonomously using automated algorithms and machine learning models, eliminating the need to allocate expensive expert data scientist time to manual feature creation, thereby reducing project costs while maintaining feature quality through computational optimization
Solution Approach 2:
The patent replaces expensive manual expert labor with an automated computational system that uses machine learning and optimization algorithms to generate features, substituting high-cost human resources with a scalable computational solution that reduces project expenses
4Measurement precision
If manual feature engineering processes are used, then feature selection accuracy is improved, but scalability and efficiency of the machine learning process deteriorate
Solution Approach 1:
The system automatically performs feature engineering and selection through automated algorithms that can process and evaluate numerous potential features systematically, maintaining selection accuracy through algorithmic optimization while enabling the process to scale efficiently to large datasets and complex models
Solution Approach 2:
The patent replaces manual feature selection with an automated computational system that uses machine learning models and optimization algorithms to evaluate and select features automatically, enabling scalable processing of large feature sets without the limitations of manual expert review
Data Source
AI summary
Various implementations include approaches for automating feature generation. The underlying intellectual paradigm to the approach is ensemble learning. That is, each generated feature is an element in an ensemble. Ensemble learning is a very successful paradigm in classical machine learning and dominates real-world predictive analytics projects through tools such as xgboost ( . . . ) or lightgbm ( . . . ). It is also appropriate, because of its ease-of-use compared to other successful paradigms such as deep learning. Moreover, it is possible to generate human-readable SQL code, which is very difficult with deep learning approaches. The various implementations described herein provide for increased scalability and efficiency as compared with conventional approaches.