Metalearner for Unsupervised AutoML Pipeline Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Automated Machine Learning (AutoML) systems are limited in their ability to perform unsupervised machine learning tasks due to the lack of labeled data sets, which hinders their capability to automate pipeline generation and selection for tasks like outlier detection and clustering, and they are slow due to large search spaces.
Innovation Solution
A metalearner system that generates data subsets and unsupervised machine learning pipelines from labeled data sets, trains a metalearner using these subsets, and applies it to identify optimal pipelines for unsupervised tasks, leveraging supervised data sets to enhance unsupervised learning efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If current AutoML systems are used for unsupervised machine learning tasks, then they can perform basic pipeline execution, but they are slow due to large search spaces and cannot effectively automate pipeline generation and selection
Solution Approach 1:
The system performs preliminary actions by generating multiple data subsets from the labeled dataset before the actual pipeline selection process. These pre-generated subsets are used to train a metalearner that captures patterns and relationships, enabling faster pipeline identification without exhaustively searching the entire large search space during the actual unsupervised task execution.
Solution Approach 2:
A metalearner is introduced as an intermediary component between the labeled dataset and the unsupervised pipeline selection process. The metalearner learns from the generated data subsets and acts as a mediator to predict optimal pipelines for unsupervised tasks, avoiding the need to directly search through the entire large search space for each unsupervised task.
2Productivity
If labeled data sets are used to train metalearner, then unsupervised learning efficiency is improved, but additional data processing steps are required
Solution Approach 1:
The labeled dataset is segmented into multiple data subsets, which are then used to train the metalearner. This segmentation allows the system to process the labeled data in manageable portions, capturing diverse patterns while reducing the computational burden compared to processing the entire dataset at once. The segmented subsets enable efficient metalearner training that improves unsupervised learning performance.
Data Source
AI summary
A method, system, and computer program product for a metalearner for automated machine learning are provided. The method receives a labeled data set. A set of data subsets is generated from the labeled data set. A set of unsupervised machine learning pipelines is generated. A training set is generated from the set of data subsets and the set of unsupervised machine learning pipelines. The method trains a metalearner for unsupervised tasks based on the training set.