Submodular Optimization for Machine Learning Algorithm Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current methods for selecting machine learning algorithms are often inefficient and resource-intensive, relying on trial and error, grid searches, random searches, and meta-learning approaches that do not effectively utilize historical data to identify optimal algorithms for new datasets.
Innovation Solution
The proposed solution involves using historical performance data to inform the selection of machine learning algorithms. A computing system generates a set of algorithms that have been previously tried on benchmark datasets, and a submodular function is used to select the next candidate algorithm to test against a candidate dataset, optimizing the search for the best or optimum ML algorithm.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional trial and error methods are used to select machine learning algorithms, then model selection can be performed, but time and computational resources are excessively consumed
Solution Approach 1:
The system performs preliminary actions by pre-computing compatibility scores between algorithms and datasets using historical performance data before actual model selection is needed. This allows the system to quickly retrieve pre-evaluated algorithm recommendations rather than performing exhaustive trials at selection time, thereby reducing time loss while maintaining selection accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism - a compatibility scoring system that mediates between the algorithm and dataset. This intermediary computes and stores compatibility metrics based on historical data, serving as a bridge that eliminates the need for direct trial-and-error testing between algorithms and new datasets, thus reducing computational time while preserving selection reliability.
2Reliability
If exhaustive testing of multiple algorithms is performed, then optimal model selection can be achieved, but computational resources are excessively consumed
Solution Approach 1:
The system applies partial action by testing and evaluating only a subset of algorithms rather than exhaustively testing all possible algorithms. The compatibility scoring mechanism identifies and prioritizes the most promising algorithms based on historical data, allowing the system to achieve satisfactory model selection accuracy without consuming computational resources required for exhaustive testing of every algorithm.
Solution Approach 2:
The patent uses copying by leveraging historical performance data from previous algorithm executions on similar datasets. Instead of performing complete exhaustive testing on new datasets, the system copies and applies knowledge from historical results through compatibility scoring, significantly reducing computational resource consumption while maintaining reliable model selection.
3Productivity
If historical performance data is utilized to guide algorithm selection, then selection efficiency is improved, but system complexity increases
Solution Approach 1:
The system applies self-service by automatically computing and maintaining compatibility scores using its own historical performance data without requiring external intervention. The compatibility scoring mechanism serves itself by leveraging accumulated historical results to guide future algorithm selections, improving productivity while managing complexity through automated self-evaluation rather than requiring complex external optimization systems.
Data Source
AI summary
Systems and methods of selecting machine learning models/algorithms for a candidate dataset are disclosed. A computer system may access historical data of a set of algorithms applied to a set of benchmark datasets; select a first algorithm of the set of algorithms; apply the first algorithm to an input dataset to create a model of the input dataset; evaluate and store results of the applying; and add the first algorithm to a set of tried algorithms. The computer system may select a next algorithm of the algorithm set via submodular optimization based on the historical data and the set of tried algorithms; apply the next algorithm to the input dataset; capture a next result based on the applying; add the next result to update the set of tried algorithms; and repeat the submodular optimization. The procedure may continue until a termination condition is reached.