Multi-objective Automated Machine Learning Pipeline Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems are inadequate in optimizing multiple objectives simultaneously, often prioritizing one objective at the expense of others, leading to sub-optimal solutions in multi-objective problems due to the conflict between objectives and the inability to search over entire machine learning pipelines.
Innovation Solution
A multi-objective automated machine learning system that employs a joint optimization engine to select objectives, inject them into a machine learning pipeline, and generate Pareto-optimal solutions by optimizing data transformation stages and modeling stages with appropriate algorithms and hyperparameters, considering the entire pipeline search space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If single objective optimization is used, then one objective is sufficiently optimized, but other objectives may not be optimized and may conflict with each other
Solution Approach 1:
The patent segments the optimization problem into multiple independent objective functions that can be optimized simultaneously. Each objective is treated as a separate function to be maximized or minimized, allowing the system to handle multiple competing goals without compromising the optimization precision of any single objective.
Solution Approach 2:
The patent transforms the multi-objective optimization problem into a single-objective optimization problem by combining multiple objectives into a unified objective function using weighted sums or other aggregation methods. This parameter transformation allows standard optimization algorithms to be applied while still considering multiple objectives.
2Adaptability or versatility
If multiple objectives are optimized simultaneously, then multiple objectives can be balanced, but the complexity of the optimization problem increases
Solution Approach 1:
The patent introduces an intermediary objective function that mediates between multiple conflicting objectives. This intermediary function aggregates multiple objectives into a single composite function that can be optimized using standard algorithms, thereby reducing the apparent complexity of the multi-objective problem.
Solution Approach 2:
The patent changes the parameter representation by transforming multiple objective parameters into a single aggregated parameter through mathematical operations such as weighted sums, Pareto optimization, or hierarchical aggregation. This parameter transformation simplifies the optimization landscape while preserving the essential characteristics of multiple objectives.
3Productivity
If only part of the machine learning pipeline is optimized, then optimization resources are conserved, but the overall pipeline performance may not be fully optimized
Solution Approach 1:
The patent creates a universal optimization framework that can optimize any or all components of the machine learning pipeline. The same optimization engine can be applied to data preprocessing, feature engineering, model selection, and hyperparameter tuning, providing consistent multi-objective optimization across the entire pipeline while allowing selective optimization of specific stages when resources are constrained.
Data Source
AI summary
A system, program product, and method for performing multi-objective automated machine learning. The method includes selecting two or more objectives from a plurality of objectives to be optimized and injecting data and the objectives into a first machine learning (ML) pipeline. The first ML pipeline includes one or more data transformation stages in communication with a modeling stage. The method also includes executing, subject to the injecting, optimization of the two or objectives. Such executing includes selecting a respective algorithm for each of the data transformation stages and the modeling stage. Each respective algorithm is associated with a first set of respective hyperparameters. The executing also includes generating a plurality of second ML pipelines. Each second ML pipeline defines a Pareto-optimal solution of the two or more objectives, thereby defining a plurality of Pareto-optimal solutions, The executing also includes selecting one Pareto-optimal solution from the plurality of Pareto-optimal solutions.