ML Pipeline Recommendation via K-NN and Bayesian Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning pipeline exploration is time-consuming due to its combinatorial nature, and existing methods lack efficient recommendations for new data sets based on historical performance data.
Innovation Solution
A computer-implemented method using K-nearest neighbor and Bayesian optimization techniques to recommend machine learning pipelines by computing similarities between new and historical data sets, selecting top-performing pipelines, and determining their accuracy for recommendation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning pipeline exploration is performed exhaustively to ensure optimal pipeline selection, then pipeline accuracy and reliability are improved, but the time required for exploration increases significantly due to combinatorial nature
Solution Approach 1:
The system pre-computes and stores performance matrices for multiple pipelines on multiple training datasets before actual use. When a new testing dataset arrives, the system uses this pre-computed information to make rapid recommendations without needing to exhaustively evaluate all pipelines from scratch, thus reducing exploration time while maintaining accuracy.
Solution Approach 2:
The system introduces an intermediary recommendation mechanism that uses K-nearest neighbor algorithms and Bayesian optimization to select pipelines. Instead of directly testing all possible pipelines on the new dataset, the system uses historical performance data as an intermediary to guide the selection process, significantly reducing the number of pipelines that need to be evaluated.
2Measurement precision
If multiple pipelines are executed on testing datasets to determine optimal recommendations, then recommendation accuracy is improved, but the number of pipeline executions and computational resources increase
Solution Approach 1:
The system does not need to execute all possible pipelines on the testing dataset to achieve accurate recommendations. By using K-nearest neighbor algorithms with a defined K value and Bayesian optimization, the system selectively executes only the necessary number of pipelines (partial action) to achieve sufficient recommendation accuracy, avoiding unnecessary computational resources.
Solution Approach 2:
The system uses feedback from historical performance data stored in the performance matrix to guide future pipeline selections. The K-nearest neighbor algorithm uses feedback from similar historical datasets to predict which pipelines will perform best on the new testing dataset, reducing the number of executions needed while maintaining high recommendation accuracy.
Data Source
AI summary
Methods, computer program products and/or systems are provided that perform the following operations: obtaining a performance matrix representing accuracies obtained by executing a plurality of pipelines on a plurality of training data sets, wherein a pipeline comprises a series of operations performed on a data set; selecting a defined number of top pipelines as potential pipelines for a testing data set based, at least in part, on a similarity between the testing data set and each of the plurality of training data sets represented in the performance matrix; storing results from executing each of the potential pipelines as a new data set; determining a pipeline accuracy for each of the potential pipelines when executed against the testing data set; and providing a recommended pipeline for use with the testing data set based, at least in part, on the pipeline accuracy for each potential pipeline.