Bayesian Optimization Support for Heteroscedastic Noise
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Bayesian optimization methods cannot effectively handle heteroscedastic observation noise, preventing the use of parallel Bayesian optimization.
Innovation Solution
A support method and system that integrate predictive distributions from two machine learning models to account for heteroscedastic observation noise, enabling parallel Bayesian optimization through a third predictive distribution and multiple acquisition functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If heteroscedasticity Bayesian optimization is used to account for observation noise, then measurement precision is improved, but device complexity increases and parallel processing becomes impossible
Solution Approach 1:
The patent segments the predictive distribution into two separate machine learning models: one for the expected value of the response variable and another for the variance of the response variable. This segmentation allows each model to specialize in one aspect, simplifying the overall system while enabling parallel processing of multiple recommended values through the acquisition function.
2Productivity
If parallel Bayesian optimization is used to explore multiple recommended values, then productivity is improved, but the ability to handle heteroscedastic observation noise is lost
Solution Approach 1:
The patent merges the capabilities of heteroscedasticity Bayesian optimization and parallel Bayesian optimization by integrating the predictive distribution of the expected value with the predictive distribution of the variance into a unified framework. This combination allows the system to maintain the ability to handle heteroscedastic observation noise while enabling parallel exploration of multiple recommended values through the acquisition function.
3Device complexity
If a single machine learning model outputs both predictive distribution of expected value and variance, then device complexity is reduced, but measurement precision for handling heteroscedastic noise deteriorates
Solution Approach 1:
The patent divides the predictive distribution task into two separate machine learning models: one dedicated to predicting the expected value of the response variable and another dedicated to predicting the variance. This segmentation improves measurement precision by allowing each model to specialize in its specific task, thereby achieving more accurate predictive distributions for handling heteroscedastic observation noise.
Data Source
AI summary
A support method supports exploration of a value of an explanatory variable that maximizes or minimizes an expected value of a response variable, and includes: outputting, from a first machine learning model, a first predictive distribution which is a predictive distribution of the expected value of the response variable; outputting, from a second machine learning model, a second predictive distribution which is a predictive distribution of a variance of the response variable; constructing a third predictive distribution that integrates the first predictive distribution and the second predictive distribution; and a recommended value acquisition process of executing parallel Bayesian optimization based on the third predictive distribution, at least one acquisition function, and an exploration range, and acquiring at least one recommended value of the explanatory variable that maximizes the acquisition function from within the exploration range.