Method and system for bayesian optimization

Abstract

The present invention relates to a method and system for Bayesian optimization. More specifically, the present invention relates to a method and system for Bayesian optimization using individual parameter kernel density estimation (KDE) in a mixed variable environment.

Description

Bayesian Optimization Method and System

[0001] The present invention relates to a Bayesian optimization method and system. More specifically, the present invention relates to a Bayesian optimization method and system using kernel density estimation (KDE) of individual parameters in a mixed variable environment.

[0002] Bayesian optimization is a technique that utilizes prior information to efficiently search for an optimal solution while minimizing high-cost function evaluations. This type of optimization is applied in various fields, including hyperparameter tuning, process control, and autonomous driving system optimization.

[0003] However, in real-world industrial environments, optimization problems frequently occur in environments involving a mixture of continuous and discrete variables (or parameters). For example, in mixed-variable environments where continuous and discrete variables are combined, accurately estimating the joint probability distribution of each variable becomes difficult as the number of variables increases and their ranges widen, and optimization performance can degrade rapidly due to the curse of dimensionality.

[0004] In this regard, conventional Bayesian optimization methods have limitations in that they use a method of modeling the joint distribution by integrating all variables, which increases computational costs and makes it difficult to finely reflect the characteristics of each variable.

[0005] Furthermore, in high-dimensional and mixed-variable environments, the distribution characteristics of each variable differ, so applying the same estimation technique or parameters can reduce optimization efficiency. Conventional Bayesian optimization methods are difficult to apply to real-time processing or large-scale system optimization due to the rapid increase in computational load, and often fail to reflect the characteristics of different variable types, which may limit the accuracy and interpretability of optimization results.

[0006] Accordingly, there is still a need for a method that enables the precise estimation of the individual distributions of each variable, even in environments where continuous and discrete variables are mixed, and allows for stable and efficient Bayesian optimization based on this.

[0007] The present invention provides a method and system for performing Bayesian optimization in an environment where continuous and discrete variables (or parameters) are mixed.

[0008] More specifically, the present invention aims to provide a Bayesian optimization method and system that operates effectively even in high-dimensional environments where continuous and discrete variables are mixed.

[0009] In particular, the present invention aims to provide a Bayesian optimization method and system capable of improving optimization performance by estimating the individual distribution of each variable in a mixed variable environment.

[0010] Furthermore, the present invention aims to provide a Bayesian optimization method and system capable of efficiently handling cases where the number of variables is large and the optimization range of each variable is wide.

[0011] Furthermore, the present invention aims to provide a method and system capable of accurately estimating a distribution by individually considering the characteristics of each variable and performing Bayesian optimization at minimum cost.

[0012] To solve the problem described above, a computerized Bayesian optimization method according to the present invention, comprising the following steps, may include: specifying a plurality of parameters having different characteristics; estimating a prior distribution for each of the plurality of parameters using a pre-set probability distribution estimation technique; constructing an initial sample set based on the prior distributions estimated for each of the plurality of parameters; and training a prediction model using the initial sample set to produce an optimal solution for the plurality of parameters.

[0013] In an embodiment, in the step of estimating the prior distribution, the prior distribution for each of the plurality of parameters is estimated by applying the pre-set probability distribution estimation technique to each of the plurality of parameters, and the initial sample set may be constructed by performing sampling from the prior distribution estimated for each of the plurality of parameters.

[0014] In an embodiment, the step of constructing the initial sample set may include sampling the value of each of the plurality of parameters from a prior distribution for each of the plurality of parameters, generating a plurality of initial samples using the sampled value of each of the plurality of parameters, and constructing the initial sample set using the plurality of initial samples to train the prediction model.

[0015] In an embodiment, the step of constructing the initial sample set may involve performing an evaluation on each of the plurality of initial samples sampled from a prior distribution for each of the plurality of parameters, obtaining an evaluation result for each of the plurality of initial samples based on the evaluation, and constructing the initial sample set using the plurality of initial samples and the evaluation result for each of the plurality of initial samples.

[0016] In an embodiment, in the step of training the prediction model, the prediction model can be trained using the plurality of initial samples included in the initial sample set and the evaluation results.

[0017] In an embodiment, the prediction model can learn an approximation of the optimization target function in an environment where the plurality of parameters are mixed, by using a prior distribution for each of the plurality of parameters.

[0018] In an embodiment, the learning of the approximation of the optimization target function may be a process of learning to approximate the plurality of initial samples and the evaluation result for each of the plurality of initial samples.

[0019] In an embodiment, the step of generating the plurality of initial samples may be a step of generating the plurality of initial samples including different parameter combinations using the value of each of the plurality of parameters sampled from a prior distribution for each of the plurality of parameters.

[0020] In an embodiment, the prediction model can select an optimization evaluation point by learning the interaction between a plurality of parameters having different characteristics.

[0021] In an embodiment, the method may further include the step of sampling candidate evaluation points from a prior distribution for each of the plurality of parameters, and the step of selecting at least one specific evaluation point among the candidate evaluation points to be evaluated using the learned prediction model and a specific function.

[0022] In an embodiment, the step of sampling the candidate evaluation points may be a step of sampling the candidate evaluation points including different parameter combinations from a prior distribution for each of the plurality of parameters.

[0023] In an embodiment, in the step of selecting the specific evaluation point, the predicted value for each of the candidate evaluation points is predicted using the learned prediction model, and the specific evaluation point can be selected using the predicted value for each of the candidate evaluation points and the specific function.

[0024] In an embodiment, in the step of selecting the specific evaluation point, the specific function value is calculated for each of the candidate evaluation points using at least one of the predicted value and the optimal value used as a criterion for selecting the specific evaluation point, and among the candidate evaluation points, at least one evaluation point in which the calculated specific function value satisfies a preset criterion can be selected as the specific evaluation point.

[0025] In an embodiment, the specific evaluation point may include a point that maximizes the specific function value.

[0026] In an embodiment, the learned prediction model may be configured to predict at least one of a predicted value and a predicted variance for each of the candidate evaluation points.

[0027] In an embodiment, the method may further include the steps of performing an evaluation on the selected specific evaluation point, updating the learned prediction model based on the evaluation result for the specific evaluation point, and calculating an optimal solution for the plurality of parameters based on the update to the learned prediction model.

[0028] In an embodiment, the optimal solution may include an optimal combination of parameters calculated from the plurality of parameters.

[0029] In an embodiment, the plurality of parameters having different characteristics include at least one of a first parameter having a first characteristic and a second parameter having a second characteristic, and in the step of estimating the prior distribution, the pre-set probability distribution estimation technique is applied to each of the first parameter and the second parameter to estimate the prior distribution for each of the first parameter and the second parameter, and the prediction model can be trained based on the prior distribution estimated for each of the first parameter and the second parameter using the pre-set probability distribution estimation technique.

[0030] A Bayesian optimization system according to the present invention, comprising a memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, can specify a plurality of parameters having different characteristics, estimate a prior distribution for each of the plurality of parameters using a pre-set probability distribution estimation technique, construct an initial sample set based on the prior distribution estimated for each of the plurality of parameters, and train a prediction model using the initial sample set to calculate an optimal solution for the plurality of parameters.

[0031] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored on a computer-readable recording medium, and may include instructions for performing the steps of: specifying a plurality of parameters having different characteristics; estimating a prior distribution for each of the plurality of parameters using a pre-set probability distribution estimation technique; constructing an initial sample set based on the prior distribution estimated for each of the plurality of parameters; and training a prediction model using the initial sample set to calculate an optimal solution for the plurality of parameters.

[0032] As discussed above, the Bayesian optimization method and system according to the present invention can improve optimization performance in an environment where multiple parameters having different characteristics are mixed, by estimating the individual distribution for each of the multiple parameters. In other words, by processing each parameter individually, the present invention can prevent performance degradation in high-dimensional problems and mitigate the curse of dimensionality.

[0033] Furthermore, according to the Bayesian optimization method and system of the present invention, a prior distribution for each of a plurality of parameters having different characteristics can be estimated, and Bayesian optimization can be performed based on the estimated distribution. Through this, the present invention minimizes the total search space and concentrates search resources on a valid and high-performance space, thereby reducing unnecessary evaluations and improving the optimization convergence speed.

[0034] In addition, according to the Bayesian optimization method and system of the present invention, computational costs are reduced by estimating the prior distribution for each of a plurality of parameters having different characteristics, and individual learning rates or bandwidths can be adjusted according to the characteristics of each parameter.

[0035] Furthermore, according to the Bayesian optimization method and system of the present invention, each parameter can be processed simultaneously according to the characteristics of each of the multiple parameters, and accurate estimation is possible even within a wide optimization range of each parameter.

[0036] Furthermore, according to the Bayesian optimization method and system of the present invention, when only some parameters among a plurality of parameters having different characteristics are changed, only the prior distribution of the changed parameters can be selectively updated. Through this, the present invention can improve the efficiency of incremental learning and can apply different bandwidths to each parameter, thereby increasing the accuracy of distribution estimation according to the characteristics of each variable. In addition, distribution information of some parameters can be reused from similar optimization problems to be quickly applied to new problems, and the distribution of related parameters can be immediately updated whenever new data is received, thereby enabling real-time response to environmental changes. Accordingly, since the present invention only needs to store the distribution information of each parameter, memory can be used efficiently to save storage space.

[0037] As such, the present invention efficiently processes complex mixed variables and enables accurate estimation even within a wide optimization range for each variable. This method can provide improved performance in high-dimensional problems, effective processing of mixed variables, and increased computational efficiency. In particular, the present invention can be usefully applied to real industrial environments where interactions between variables are complex and the range of influence of each variable is wide, such as factory automation or complex system optimization.

[0038] According to the present invention, the total search space can be reduced by probabilistically optimizing the initial search space using the individual distributions of each parameter. This reduces computational complexity and improves the optimization convergence speed. In particular, by mitigating the curse of dimensionality problem in mixed-variable environments, learning stability and prediction accuracy can be improved compared to conventional joint distribution-based Bayesian optimization.

[0039] Meanwhile, existing technologies have limitations in that they estimate the joint distribution of all variables as a single model, leading to a rapid increase in computational complexity and difficulty in reflecting variable characteristics. The present invention solves these problems by individually estimating the prior distribution for each parameter. This improves computational efficiency during the optimization process and ensures prediction precision that reflects interactions between variables. Therefore, the present invention can derive a highly reliable optimal solution while efficiently utilizing computational resources, even in high-dimensional mixed-variable environments.

[0040] Furthermore, the present invention can be applied not only to manufacturing process optimization but also to various industrial fields where complex interactions between variables exist. For example, the present invention can be usefully applied to tuning control parameters in autonomous driving systems, optimizing operating conditions of semiconductor process equipment, adjusting efficiency parameters in energy management systems, or the calibration process of robot control algorithms. Since continuous and discrete variables exist simultaneously in such application environments, the individual parameter-based probability distribution estimation and sampling structure according to the present invention can flexibly respond to the data characteristics of each industrial environment.

[0041] FIG. 1 is a conceptual diagram illustrating a Bayesian optimization system according to the present invention.

[0042] FIGS. 2, FIGS. 3 and FIGS. 4 are flowcharts illustrating a Bayesian optimization method according to the present invention.

[0043] Figures 5 and 6 are formulas related to the Bayesian optimization method according to the present invention.

[0044] FIG. 7 shows an example of an algorithm of a Bayesian optimization method according to the present invention.

[0045] FIG. 8 is a table showing an example of experimental results of a Bayesian optimization method and system according to the present invention.

[0046] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0047] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0048] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0049] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0050] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0051] The present invention relates to a method and system for performing Bayesian optimization in an environment where continuous and discrete variables (or parameters) are mixed. More specifically, the present invention relates to a Bayesian optimization method and system that operates effectively even in a high-dimensional environment (or mixed-variable environment) where continuous and discrete variables are mixed.

[0052] The Bayesian optimization method according to the present invention can be applied to various industrial environments, including not only manufacturing process control but also tuning of control parameters in autonomous driving systems, optimization of semiconductor process conditions, efficiency improvement of energy management systems, and adjustment of robot control parameters. In particular, stable and rapid optimization is possible even in complex system environments where the number of parameters is large and continuous and discrete variables are mixed.

[0053] A mixed-variable environment refers to a variable space where continuous and discrete variables coexist and interact within the same optimization problem. For example, in a mixed-variable environment, the data types, ranges, and resolutions of each variable differ, making it complex or inefficient to process all variables using a single joint probability model (or module). This tendency can be exacerbated, particularly when the number of variables is large or the optimization range of each variable is wide. Such mixed-variable environments can appear in various industrial fields, including factory automation, complex system parameter tuning, and production process optimization.

[0054] Here, a continuous variable can mean a variable that can take any real value (or infinitely many values) within a given specific interval. A continuous variable has a set of values ​​that belongs to a continuous interval or a subset of the entire set of real numbers, and between two adjacent values, there can be infinitely many different values ​​(or always different values). For example, a continuous variable can have a continuity of values ​​in which there are infinitely many intermediate values ​​between two values ​​(e.g., between 1.2 and 1.3, there exist 1.25, 1.1251, 1.2511, etc.). Examples of such continuous variables may include variables that numerically express physical / quantitative characteristics, such as length (e.g., 1.2 m, 1.25 m, 1.257 m, etc.), temperature (e.g., 25.3℃, 25.31℃, 25.314℃, etc.), pressure (e.g., 1.01 MPa, 1.011 MPa, etc.), time (e.g., 1.5 sec, 1.53 sec, 1.534 sec, etc.), lighting intensity, voltage / current (e.g., 3.3V, 3.31V, 3.314V, etc.), speed (e.g., 50.2 km / h, 50.23 km / h, etc.). In this specification, the term continuous variable may be used interchangeably with “first parameter,” “continuous parameter,” “first variable,” “continuous variable,” or “continuous parameter.”

[0055] Additionally, a discrete variable may refer to a variable capable of taking only a finite or countable infinite number of distinct values. Discrete variables are expressed solely by distinct individual values ​​rather than a continuous range of real numbers, and there may be no other value (or continuous value) between the two values. Such discrete variables may include at least one of integer values ​​(e.g., production process step number, equipment mode setting, etc.), nominal values ​​(e.g., color, category, etc.), ordinal values ​​(e.g., grade, level, etc.), or boolean values ​​(e.g., ON / OFF, etc.). For example, examples of discrete variables may include process steps (e.g., {Step 1: 0.4, Step 2: 0.35, Step 3: 0.25}), algorithm types (e.g., {A: 0.5, B: 0.3, C: 0.2}), number of product defects (e.g., 0, 1, 2, ...), machine operation modes (e.g., {Standby, Running, Stopped}), quality grades (e.g., High, Medium, Low), color classifications (e.g., {Red, Blue, Green}), production line numbers (e.g., No. 1, No. 2, No. 3...), part specification codes (e.g., {A1, A2, B1, B2}), etc. In this specification, the term discrete variable may be used interchangeably with “second parameter,” “discrete parameter,” “second variable,” “discrete variable,” “discrete parameter,” etc.

[0056] However, the examples included in continuous and discrete variables are not necessarily limited to those mentioned above, and it is obvious that various other examples may be included in addition to those mentioned.

[0057] Furthermore, Bayesian optimization is a stochastic optimization technique for efficiently searching for the optimal value of an objective function that is costly or computationally complex. Bayesian optimization is a technique (or algorithm) that efficiently explores a given solution space using a surrogate model and / or a probabilistic model (e.g., a Gaussian Process (GP) model). Bayesian optimization can induce rapid convergence by proceeding with the search based on probabilistic predictions from given data. For example, instead of directly iteratively evaluating the objective function, Bayesian optimization can estimate the distribution of the objective function using a surrogate model and / or a probabilistic model, and select the next search point through an acquisition function. Such Bayesian optimization may be one of the algorithms included in an optimization algorithm, and in the present invention, it may also be referred to as a “pre-configured optimization algorithm.”

[0058] Meanwhile, the present invention can be usefully utilized in various situations. More specifically, the Bayesian optimization method and system according to the present invention can be usefully utilized by applying them to various industries and services. For example, as shown in FIG. 1, the present invention can be usefully utilized by applying it to a vision inspection system that determines (or predicts) whether a product to be manufactured (20, or an image of the product to be manufactured (20)) is abnormal (e.g., normal (21) or defective (22)) when the product to be manufactured (20, or an image of the product to be manufactured (20)) is received (or input). However, the fields to which the present invention is applied are not necessarily limited to the fields mentioned above, and can be usefully applied to actual industrial environments where the interaction between variables is complex and the range of influence of each variable is wide (e.g., factory automation or complex system optimization).

[0059] Hereinafter, the present invention will be examined in more detail with reference to the attached drawings. FIG. 1 is a conceptual diagram illustrating a Bayesian optimization system according to the present invention. FIGS. 2, FIGS. 3, and FIGS. 4 are flowcharts illustrating a Bayesian optimization method according to the present invention. FIGS. 5 and FIGS. 6 are formulas related to a Bayesian optimization method according to the present invention. Furthermore, FIG. 7 shows an example of an algorithm of a Bayesian optimization method according to the present invention, and FIG. 8 is a table showing an example of experimental results of a Bayesian optimization method and system according to the present invention.

[0060] Meanwhile, as illustrated in FIG. 1, the Bayesian optimization system (100) according to the present invention may include at least one of an input unit (110), an output unit (120), a communication unit (130), a storage unit (140), a probability distribution estimation unit (150), a data processing unit (160), a prediction model (170), and a control unit (180).

[0061] A Bayesian optimization system (100) according to the present invention may include at least one processor and at least one memory containing computer program code. In this case, the memory may perform the role of the storage unit (140) above. In the present invention, the memory and the program code may cooperate with the processor to perform a series of processes described below.

[0062] Although not illustrated, the Bayesian optimization system (100) according to the present invention may include one or more processors, and such processors may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processor, tensor processing unit (TPU), graphics processing unit (GPU), neural network processing unit (NPU), application integrated circuit, application semiconductor (ASIC), field programmable gate array (FPGA), quantum processing unit (or quantum processor, QPU), etc.). One or more processors may be configured to execute instructions, computer-readable instructions, and / or other instructions described herein that are stored (or included) in the storage unit (140). The Bayesian optimization method and system according to the present invention may perform data processing described below in cooperation with memory and at least one processor. The processor may perform a series of operations and data processing using data and information stored in memory. In this case, memory may be a component of the storage unit (140).

[0063] In addition, the Bayesian optimization system (100) according to the present invention can perform data processing and computation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into account implementation in a quantum computer environment. For example, the present invention can perform parallel computations based on qubits, and such quantum computations can operate complementarily with existing classical computers.

[0064] Such quantum computers may include parallel computation using qubits and high-speed data processing devices utilizing quantum entanglement, and hardware-based computational optimization using FPGAs and ASICs is possible. In addition, quantum computers may utilize quantum processors capable of qubit-based parallel computation, and data processing efficiency can be improved through a hybrid structure with existing classical computers.

[0065] Meanwhile, the input unit (110) can be configured in various ways as a means of data input. For example, the input unit (110) can be configured to receive user input. The input unit (110) can be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on input made by the user through the input unit configuration provided in the user terminal (10).

[0066] Here, the user terminal (10) may include at least one of a mobile phone, a smartphone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head-mounted display).

[0067] In addition, the input unit (110) in the present invention does not necessarily mean a hardware means, but can be understood as a channel for receiving input from a user.

[0068] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, a computer mouse, a keyboard, a keypad, a touchpad, a trackball, a joystick, a voice recognition module, or other similar devices. However, the present invention does not limit the type of input unit (110).

[0069] Here, user input may include documents, text, images (or videos), voice, etc. In this case, the Bayesian optimization system (100) may further include a module that converts voice into text.

[0070] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked to the Bayesian optimization system (100) according to the present invention. For example, the output unit (120) can output a page (or service page) linked to the Bayesian optimization system (100) according to the present invention to the display unit of the user terminal. In addition, the output unit (120) does not necessarily mean a hardware means, but can be understood as a channel for outputting results to a user.

[0071] Next, the communication unit (130) may be connected via a wireless or wired network to a user terminal, a server (e.g., a central server, an external server, etc.), a device, and at least one network, etc., to receive or transmit overall data and information necessary for the operation of the Bayesian optimization system (100) according to the present invention.

[0072] The communication unit (130) can support various communication methods depending on the communication standard of the communicating device.

[0073] For example, the communication unit (130) may be configured to communicate with a communication target using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ RFID (Radio Frequency Identification), Infrared Communication (Infrared Data Association; IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus).

[0074] Next, the storage unit (140, or memory) serves to store various data related to the present invention and may include one or more non-transient computer-readable storage media that can be read and / or accessed by at least one of one or more processors.

[0075] One or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage unit (140) may be implemented using a single physical device (e.g., one optical, magnetic, organic, or other memory or disk storage device), whereas in other examples, the storage unit (140) may be implemented using two or more physical devices.

[0076] The storage unit (140) may include computer-readable instructions and additional data. The storage unit (140) may include a storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the device and network.

[0077] Furthermore, at least a portion of the storage unit (140) may be a cloud storage or a cloud server. The storage unit (140) may store at least some of the data corresponding to user input received from the input unit (110) (e.g., an image (20) of a product to be manufactured, discrete variables and continuous variables, etc.) and training data.

[0078] That is, the storage unit (140) is sufficient as a space where information necessary for the operation of the Bayesian optimization system (100) according to the present invention is stored, and it can be understood that there are no restrictions on the physical space.

[0079] Furthermore, the storage unit (140) may store a computer program including computer program instructions. Furthermore, the storage unit (140) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (180) when loaded into the processor of the system (100).

[0080] Next, the probability distribution estimation unit (150) may be configured to perform the role of estimating a prior distribution for each of a plurality of parameters having different characteristics (or data characteristics) using a pre-set probability distribution estimation technique. Here, the pre-set probability distribution estimation technique (or probability density function estimation technique) may include a Kernel Density Estimation (KDE) technique.

[0081] Kernel density estimation is a non-parametric method for estimating the probability density function (PDF) of data from given data samples. It approximates the distribution of data within a specific interval as a smooth curve. For example, kernel density estimation can be performed by placing kernel functions around observed samples and summing the total density functions, without assuming the shape of the data distribution. This method smoothly represents the structure of the data and allows for the application of various kernels tailored to the characteristics of the variables (e.g., continuous or discrete variables).

[0082] The probability distribution estimation unit (150) can estimate the individual distribution (or prior distribution, probability distribution, etc.) of each parameter by performing kernel density estimation for each of a plurality of parameters having different ranges and characteristics. In this case, the probability distribution estimation unit (150) can estimate the prior distribution by applying different probability distribution estimation techniques to each of the plurality of parameters. For example, the probability distribution estimation unit (150) can model the prior distribution by applying a Gaussian kernel density estimation (Gaussian KDE) technique to the first parameters (continuous variables) and a categorical probability distribution estimation (KDE for discrete variables) technique to the second parameters (discrete variables). That is, the pre-set probability distribution estimation technique may include at least one of the Gaussian kernel density estimation technique and the categorical probability distribution estimation technique.

[0083] The Gaussian probability distribution estimation technique is a nonparametric method for estimating the probability density function of a first parameter. It involves transforming each data sample (or data point) into a normal distribution (or normal kernel) where the mean is the corresponding sample value and the variance (or standard deviation) is the bandwidth (h), and then summing all of these to estimate the overall probability density function (or approximate the overall distribution). In this specification, the Gaussian probability distribution estimation technique may also be referred to as the “pre-established first probability distribution estimation technique.”

[0084] In addition, the categorical probability distribution estimation technique is a KDE method that estimates a probability mass function based on the frequency of occurrence (or frequency of occurrence) for each category (or category) to estimate the distribution of categorical data (or second parameter). Unlike the Gaussian probability distribution estimation technique, which uses kernels in a continuous space, the categorical probability distribution estimation technique can estimate probabilities by applying a categorical kernel (e.g., Dirichlet kernel, based on a delta function) to each category. That is, unlike continuous KDEs, the kernel function of the categorical probability distribution estimation technique can be defined based on whether the categorical data is identical. For example, an index function that assigns 1 if the categories are the same and 0 if they are different, or other kernels that reflect similarity, can be used. In this specification, the categorical probability distribution estimation technique may also be referred to as a “pre-configured second probability distribution estimation technique.”

[0085] That is, Gaussian probability distribution estimation generates a probability density function (PDF) in the form of a smooth curve in real space, and categorical probability distribution estimation can estimate the probability mass for each category in discrete space. However, in the present invention, different techniques are not necessarily applied to each parameter having different characteristics, and the Gaussian probability distribution estimation technique may be used for each of the first parameter and the second parameter.

[0086] The “pre-configured probability distribution estimation technique” used in the present invention may include not only probability density estimation based on kernel density estimation (KDE), but also probability estimation based on Gaussian processes, non-parametric parametric models, or distribution fitting techniques based on prior data. These techniques may be performed in hardware by a numerical computation module on a processor, or in software by a code block that implements a probability density estimation algorithm. Accordingly, the present invention can be configured to be applicable to various probability estimation frameworks.

[0087] Next, the data processing unit (160) may be configured to perform the role of generating (or configuring, constructing, etc.) at least one initial sample or candidate evaluation point by performing sampling from the estimated prior distribution for each of a plurality of parameters having different characteristics.

[0088] For example, the data processing unit (160) may generate multiple initial samples based on the prior distribution of each parameter estimated for each of the multiple parameters. Alternatively, the data processing unit (160) may extract initial samples from individual prior distributions estimated by the probability distribution estimation unit (150) using different probability distribution estimation techniques according to the data type and / or distribution characteristics of each of the multiple parameters. In this case, the data processing unit (160) may sample multiple initial samples from each of the first prior distribution estimated by performing Gaussian kernel density estimation (Gaussian KDE) on the first parameter and the second prior distribution estimated by performing categorical probability distribution estimation on the second parameter, and construct an initial sample set to be used for optimization using the multiple initial samples.

[0089] Next, the prediction model (170) can be trained based on prior distributions estimated for multiple parameters having different characteristics. For example, the data processing unit (160) can construct an initial sample set using multiple initial samples, and the prediction model (170) can be trained using the initial sample set. In the present invention, the prediction model (170) may also be named a “probability model,” a “surrogate model,” a “proxy model,” or an “artificial intelligence model,” etc.

[0090] Specifically, the prediction model (170) can learn an approximation of the optimization target function in an environment where multiple parameters are mixed, by using a prior distribution for each of the multiple parameters. Here, learning an approximation of the optimization target function can be understood as the prediction model (170) learning to approximate multiple initial samples and the evaluation results for each of the multiple initial samples.

[0091] That is, the prediction model (170) can learn the interaction between multiple parameters having different characteristics, and the learned prediction model (170) can select the next optimization evaluation point among the candidate evaluation points. More specific details regarding this will be explained later.

[0092] Meanwhile, the prediction model (170) may be a single Gaussian process (GP) model, or it may be composed of an ensemble structure including a Random Forest or a Bayesian Neural Network. This configuration can improve the approximation accuracy of the function to be optimized by compensating for the prediction uncertainty based on the distribution characteristics of each parameter. Additionally, the prediction model may be trained to simultaneously calculate the expected function value and the predicted variance by performing probability distribution-based regression.

[0093] Next, the control unit (180) can perform the role of controlling the overall operation of the Bayesian optimization system (100) related to the present invention. The control unit (180) can process signals, data, information, etc. that are input or output through the components of the Bayesian optimization system (100) described above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (180) can be physically implemented by the processor described above.

[0094] Meanwhile, the present invention aims to provide a method and system for performing Bayesian optimization in an environment where continuous and discrete variables (or parameters) are mixed. More specifically, the present invention aims to provide a Bayesian optimization method and system that operates effectively even in a high-dimensional environment where continuous and discrete variables are mixed. Below, we will examine the Bayesian optimization method according to the present invention in more detail.

[0095] In the present invention, a plurality of parameters having different characteristics are specified (S210), and a process of estimating a prior distribution for each of the plurality of parameters using a pre-set probability distribution estimation technique may be performed (S220, see FIG. 2).

[0096] In the present invention, estimating a prior distribution for each of a plurality of parameters may mean estimating a prior distribution for each of the plurality of parameters according to the characteristics of each of the plurality of parameters through a parameter-wise method. Alternatively, it may mean estimating a prior distribution independently for each of the plurality of parameters.

[0097] Here, multiple parameters having different characteristics may be included in a Bayesian optimization system (100) or a target system to which Bayesian optimization is applied (e.g., a simulator, a simulation system). As an example, the simulator may perform the role of determining (or predicting) whether a product to be manufactured is normal (21) or defective (22) (e.g., normal (21) or defective (22)). In the present invention, the term “parameter” may be used interchangeably with “variable.”

[0098] In this case, the simulator may be at least one component included in the Bayesian optimization system (100) itself. Alternatively, the Bayesian optimization system (100) may also be named a “simulation system” or “simulator” that performs Bayesian optimization. The present invention is not limited to any one of these.

[0099] That is, multiple parameters having different characteristics may refer to input variables (parameters) used in a Bayesian optimization system (100, or simulator). For example, when a Bayesian optimization system (100) receives various parameters (e.g., temperature, pressure, speed, lighting conditions, etc.) for process optimization, the control unit (180) may identify them as multiple parameters having different characteristics.

[0100] Additionally, multiple parameters having different characteristics may include parameters received from a user terminal (10) in addition to parameters set (or defined, input) within the Bayesian optimization system (100, or simulator). For example, when the control unit (180) receives input from the user terminal (10) regarding a first parameter (continuous variable) and a second parameter (discrete variable) to be optimized, the first parameter and the second parameter may be identified as multiple parameters having different characteristics.

[0101] However, in addition to the cases mentioned above, there may be various ways in which multiple parameters having different characteristics are specified, and the present invention is not limited to any one of these.

[0102] Meanwhile, the control unit (180) can estimate a prior distribution for each of a plurality of parameters having different characteristics by using the probability distribution estimation unit (150). In the present invention, the prior distribution may also be referred to as a “probability distribution” or “prior knowledge.”

[0103] The probability distribution estimation unit (150) can estimate individual probability density functions (PDFs) by applying a probability distribution estimation technique set for each of the multiple parameters in a mixed parameter (or variable) environment. A probability density function is a function that indicates the probability that a value of a continuous random variable exists in a specific interval, and can be understood as a mathematical expression of a probability distribution. The probability density function (p(x)) has a value greater than or equal to 0 for all possible values, and the integral value over the entire domain can be 1. For any interval ([a, b]), the probability that a random variable exists in that interval is given by the value obtained by integrating the probability density function over that interval. In the present invention, the probability density function of each parameter is estimated through a set probability distribution estimation technique, and this can be utilized as a functional expression of the prior distribution.

[0104] The probability distribution estimation unit (150) can estimate a prior distribution for each of a plurality of parameters having specific different characteristics using a pre-set probability distribution estimation technique.

[0105] Here, the prior distribution may refer to prior knowledge regarding the probability distribution of a parameter under analysis prior to the observed data in Bayesian statistics or Bayesian optimization. Alternatively, the prior distribution may refer to the probability distribution established for each parameter prior to the observation data in Bayesian statistics or Bayesian optimization. For example, in the present invention, the prior distribution may include probability density functions estimated individually (independently) by applying a pre-established probability distribution technique to each of the first and second parameters having different characteristics. This prior distribution is utilized for initial sampling and training of the prediction model by reflecting the characteristics and range of each variable, and can be updated in real-time or incrementally during the optimization process.

[0106] Specifically, as illustrated in FIG. 3, the probability distribution estimation unit (150) can estimate a prior distribution for each of the plurality of parameters by applying a pre-set probability distribution estimation technique to each of the plurality of parameters (S301). Here, the plurality of parameters may include at least one of a first parameter (or a first type of parameter) having a first characteristic and a second parameter (or a second type of parameter) having a second characteristic. The parameter having the first characteristic may include a first parameter having the characteristics of a continuous variable, and the parameter having the second characteristic may include a second parameter having the characteristics of a discrete variable.

[0107] The probability distribution estimation unit (150) can estimate a prior distribution for each of the first parameter and the second parameter by applying a pre-set probability distribution estimation technique to each of the first parameter and the second parameter having a first characteristic. In this case, the probability distribution estimation unit (150) can estimate a prior distribution for each of the first parameter and the second parameter by reflecting the characteristics of each of the first parameter and the second parameter, applying a pre-set first probability distribution estimation technique to the first parameter and applying a pre-set second probability distribution estimation technique to the second parameter. That is, the first parameter can be modeled through the first probability distribution estimation technique, and the second parameter can be modeled through the second probability distribution estimation technique.

[0108] In the process of constructing an initial sample set based on the results of prior distribution estimation, independent or interdependent sampling may be performed depending on the prior distribution of each parameter. For example, if correlations exist between parameters, complex sampling may be performed using a multivariate probability distribution including a covariance matrix. Additionally, the initial sample set may be constructed using random sampling, importance sampling, or Latin hypercube sampling techniques.

[0109] In one embodiment, let us assume that among a plurality of parameters, a first parameter and a third parameter exist among the parameters having a first characteristic, and a second parameter and a fourth parameter exist among the parameters having a second characteristic. The probability distribution estimation unit (150) can estimate a prior distribution for each of the first parameter and the third parameter by applying a first probability distribution estimation technique to each of the first parameter and the third parameter having the first characteristic. Additionally, the probability distribution estimation unit (150) can estimate a prior distribution for each of the second parameter and the fourth parameter by applying a second probability distribution estimation technique to each of the second parameter and the fourth parameter having the second characteristic.

[0110] That is, the probability distribution estimation unit (150) can estimate the prior distribution for each parameter by performing kernel density estimation (KDE) for each parameter having different characteristics (see FIG. 5 (a)). This prior distribution estimation process can be represented as FIG. 5 (b). Here, the prior distribution (or probability density function) estimated for each parameter can be represented as FIG. 5 (c), and the sample size, bandwidth, and kernel function (e.g., first probability distribution estimation technique or second probability distribution estimation technique, etc.) can be represented as FIG. 5 (d) to (f), respectively.

[0111] Meanwhile, a process of constructing an initial sample set based on the prior distribution estimated for each of the multiple parameters may be carried out (S230, see FIG. 2).

[0112] Once the prior distribution estimation for individual parameters is complete, the control unit (180) can use the data processing unit (160) to construct an initial sample set to be used for training the prediction model (170).

[0113] In this case, the prior distribution for each of the multiple parameters estimated in the S220 process can be used as a weight to directly determine the selection probability of the corresponding parameter value during the sampling process. More specifically, for a specific interval (or region, part, zone, etc.) where the probability value (or probability density value, probability density function value, prior distribution value, etc.) is high (or large) in the probability density function (i.e., prior distribution) estimated (or calculated, obtained, etc.) through kernel density estimation (KDE), the number of samplings (or frequency) of said specific interval may be set to increase relatively based on (or proportionally to) the probability value. For example, among the prior distributions estimated for each of the multiple parameters, the interval with a large probability density value may have a high weight in the sampling probability.

[0114] In this way, by performing random sampling or importance sampling based on set weights, intervals with high prior distributions can have more samples selected relative to the same sample size. Conversely, intervals with low prior distributions are selected less frequently, which can prevent inefficient searching within the entire search space.

[0115] This sampling method is performed independently for each of the multiple parameters, and the sampling results (multiple initial samples) performed independently for each of the multiple parameters can be combined to form an initial sample set.

[0116] For example, an interval with a prior distribution value of 0.8 can be selected with a frequency approximately four times greater than an interval with a prior distribution value of 0.2. This allows for the concentration of search resources on intervals with high probability, thereby inducing faster convergence. As a result, the present invention reduces the number of unnecessary evaluations and enables the attainment of a higher-quality optimal solution within a given evaluation budget. In other words, the present invention provides the effect of reducing unnecessary evaluations and improving the speed of optimization convergence by concentrating search resources on intervals with high validity within the entire search space. Here, the search space may refer to the set of all candidate solutions (e.g., possible states, parameter combinations, paths, etc.) that can be explored to solve a given specific problem (or instance). This is expressed as all possible combinations of variables, parameters, states, or input values ​​that constitute the problem, and each point (or point) within that space may represent a single candidate solution. That is, the search space may refer to the set of all possible combinations generated according to predefined variables and the range of values ​​of those variables to derive an optimal solution for the problem.

[0117] In this regard, the data processing unit (160) can generate initial samples based on the estimated prior distribution for each of the plurality of parameters. More specifically, as illustrated in FIG. 3, the data processing unit (160) samples the value of each of the plurality of parameters (or the value for each of the plurality of parameters (or parameter value), and the value corresponding to each of the plurality of parameters (or parameter value) from the prior distribution for each of the plurality of parameters (S303), and can generate a plurality of initial samples using the sampled value of each of the plurality of parameters (S305).

[0118] In one embodiment, let us assume that there exists a first prior distribution (e.g., a Gaussian KDE with a mean of 75°C and a standard deviation of 5°C) estimated from a first parameter (e.g., temperature) having a first characteristic and a third prior distribution (e.g., a Gaussian KDE with a mean of 1.0 bar and a standard deviation of 0.1 bar) estimated from a third parameter (e.g., pressure), and there exists a second prior distribution (e.g., {A1, A2, B1, B2}) estimated from a second parameter (e.g., part specification code) having a second characteristic and a fourth prior distribution (e.g., {Step 1: 0.4, Step 2: 0.35, Step 3: 0.25}) estimated from a fourth parameter (e.g., process step). The data processing unit (160) can perform sampling (e.g., random sampling) from estimated prior distributions for each of a plurality of parameters having different characteristics (e.g., first parameter, second parameter, third parameter, fourth parameter, etc.).

[0119] In this case, sampling is performed independently for the prior distributions estimated for multiple parameters a predetermined number of times (N times), and the results of such sampling are combined to generate N initial samples. That is, sampling is performed the same number of times N for the prior distribution of each parameter, and N multidimensional initial samples can be generated by combining identical indices. Here, the meaning of performing N times independently implies that the process of extracting initial samples from the prior distribution of each parameter is repeated N times through different random number extraction processes without interference (i.e., N samples are repeatedly extracted randomly from the same distribution). This means that when sampling the first parameter, the values ​​of the second, third, and fourth parameters are not affected, and sampling for each parameter can be performed from independent probability distributions (prior distributions). In addition, when N=3, three values ​​are sampled from the prior distribution of each parameter (e.g., three values ​​from the prior distribution of the first parameter and five values ​​from the prior distribution of the second parameter), and each sample can be newly randomly based on extraction regardless of the previously extracted value.

[0120] In one embodiment, the data processing unit (160) may perform a first sampling of the estimated prior distribution for each of a plurality of parameters (e.g., a first parameter, a second parameter, a third parameter, a fourth parameter, etc.) to obtain the values ​​of each of the plurality of parameters corresponding to the first sampling result (e.g., 72.8°C, A1, 1.05 bar, 2nd step, etc.). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the first sampling result to construct (or generate) a first initial sample (e.g., [72.8°C, A1, 1.05 bar, 2nd step]).

[0121] In another embodiment, the data processing unit (160) may perform a second sampling of the estimated prior distribution for each of the plurality of parameters to obtain the values ​​of each of the plurality of parameters corresponding to the second sampling result (e.g., 76.3°C, B2, 0.98 bar, Step 1). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the second sampling result to form a second initial sample (e.g., [76.3°C, B2, 0.98 bar, Step 1]).

[0122] In another embodiment, the data processing unit (160) may perform a third sampling of the estimated prior distribution for each of the plurality of parameters to obtain the values ​​of each of the plurality of parameters corresponding to the third sampling result (e.g., 74.1°C, A2, 1.02 bar, step 1, etc.). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the third sampling result to form a third initial sample (e.g., [74.1°C, A2, 1.02 bar, step 1]).

[0123] Additionally, the data processing unit (160) can weight the sampling probability based on the probability density function value in the prior distribution for each of the plurality of parameters (i.e., adjust or set the weight of the sampling probability). The data processing unit (160) can specify a specific interval with high probability density according to (or based on) the prior distribution estimated for each of the plurality of parameters, and perform sampling so that the probability of a sample being selected increases in the interval specified as having high probability density (or so that the frequency of sample selection increases, so that the sample is preferentially selected).

[0124] In one embodiment, the data processing unit (160) can identify a specific interval with high probability density based on a first prior distribution (e.g., a Gaussian KDE with a mean of 75°C and a standard deviation of 5°C) estimated from a first parameter (e.g., temperature). In the case of the first parameter, based on the fact that a high probability density is formed in the average 75°C interval (e.g., the 70-80°C interval), the data processing unit (160) can perform N samplings by adjusting the weights so that the sampling probability in the specific interval (e.g., the 70-80°C interval) increases (or becomes higher).

[0125] Alternatively, the data processing unit (160) may specify a section with low probability density according to (or based on) the prior distribution estimated for each of the multiple parameters, and perform sampling so that the probability of a sample being selected in the section specified as having low probability density is reduced (or lowered).

[0126] In one embodiment, let us assume that the Nth prior distribution of the Nth parameter (e.g., speed) is estimated to have a mean of 1.5 m / s and a standard deviation of 0.2 m / s. The data processing unit (160) can identify a high probability density section and a low probability density section, respectively, based on the Nth prior distribution of the Nth parameter (e.g., speed). In this case, the 1.4 to 1.6 m / s section has a high probability density, but the 0.8 m / s or lower or 2.2 m / s or higher section may have a low probability density. The data processing unit (160) can identify the section having a low probability density and perform sampling so that the sampling probability in the identified section is reduced (or lowered). At this time, when N is set to 5, speed values ​​of 0.8 m / s or less and 2.2 m / s or more can be selected with equal probability when uniformly sampling the entire distribution, but the data processing unit (160) can set the sampling probability in the low probability density range to be close to 0 so that most of the initial samples sampled from the Nth parameter are intensively sampled in the high probability density range (e.g., 1.4 to 1.6 m / s).

[0127] Accordingly, regions with a relatively low probability of occurrence within the entire search space are naturally excluded, and samples are generated by concentrating them in regions with a high probability of occurrence. Compared to the case where initial samples are randomly and uniformly distributed, this improves optimization efficiency and reduces unnecessary evaluations by allowing promising regions to be explored with higher frequency. This approach contributes to increasing search efficiency in the subsequent Bayesian optimization process by constructing initial samples that reflect the prior distribution characteristics of each parameter. In other words, since the initial samples generated through this method are structured to reflect the prior distribution characteristics of each parameter while concentrating in regions with high probability density, they improve optimization efficiency during the subsequent Bayesian optimization process and enable the derivation of high-performance candidate solutions with fewer iterations.

[0128] In this case, the process of generating multiple initial samples may be a process of generating multiple initial samples including different parameter combinations (e.g., a first parameter combination, a second parameter combination, a third parameter combination, etc.) by using the values ​​of each of the multiple parameters sampled from the prior distribution for each of the multiple parameters. The initial samples may include parameter combinations sampled from the prior distribution estimated for each of the multiple parameters. That is, in the present invention, after estimating the prior distribution of individual parameters, the combination of parameters sampled from the prior distributions may be named an “initial sample.”

[0129] Alternatively, the process of generating (or constructing) a plurality of initial samples in the present invention may also be understood as a process of constructing different parameter vectors (e.g., [x1, x2, ... xn]) using values ​​sampled from a prior distribution for each of the plurality of parameters (e.g., values ​​of each of the plurality of parameters). In the present invention, the initial samples may also be referred to as “initial data,” “initial input sample,” “initial input data,” “initial parameter combination,” “initial sample vector,” “sample vector,” “training sample,” “training sample vector,” “input sample,” “input sample vector,” “input vector,” “initial sampling data (sampled data),” “initial parameter set,” “initial set,” “initial parameter pair,” or “training data,” etc.

[0130] In addition, the data processing unit (160) may construct (or generate, build, etc.) an initial sample set using a plurality of initial samples to train a prediction model. For example, the data processing unit (160) may construct an initial sample set including at least one of a first initial sample (e.g., [72.8°C, A1, 1.05 bar, Step 2]), a second initial sample (e.g., [76.3°C, B2, 0.98 bar, Step 1]), a third initial sample (e.g., [74.1°C, A2, 1.02 bar, Step 1]), and a Nth initial sample.

[0131] In this case, during the process of constructing the initial sample set, as illustrated in FIG. 3, the Bayesian optimization system (100) may perform an evaluation on each of the multiple initial samples sampled from the prior distribution for each of the multiple parameters (S307). The Bayesian optimization system (100) may perform an evaluation on each of the multiple initial samples through a pre-set evaluation method (e.g., simulation), and based on the evaluation results, obtain (or collect) the evaluation results for each of the multiple initial samples (e.g., the objective (or target) function value or evaluation score (or score) actually evaluated for the parameter combination included in each of the multiple initial samples) (S309). Here, the objective function value evaluated for the initial samples may also be understood as a performance indicator (or performance value) calculated by inputting each initial sample into the Bayesian optimization system (100, or simulator) in the data processing unit (160). As an example, the form of the value may be numeric (e.g., score, time cost, etc.), and may be a single value or, in the case of multi-objective optimization, a vector form.

[0132] Specifically, the Bayesian optimization system (100) can perform a simulation for each of a plurality of initial samples containing different parameter combinations to obtain an evaluation result for each of the plurality of initial samples.

[0133] For example, a Bayesian optimization system (100) can perform a simulation on a first parameter combination (e.g., [72.8°C, A1, 1.05 bar, 2 steps]) included in a first initial sample to obtain an evaluation result (e.g., 92.5) for the first parameter combination.

[0134] As another example, the Bayesian optimization system (100) can perform a simulation on a second parameter combination (e.g., [76.3°C, B2, 0.98 bar, step 1]) included in a second initial sample to obtain an evaluation result (e.g., 88.3) for the second parameter combination.

[0135] As another example, the Bayesian optimization system (100) can perform a simulation on a third parameter combination (e.g., [74.1°C, A2, 1.02 bar, 1 step]) included in a third initial sample to obtain an evaluation result (e.g., 95.7) for the third parameter combination.

[0136] Furthermore, as illustrated in FIG. 3, the data processing unit (160) can construct an initial sample set using a plurality of initial samples and an evaluation result for each of the plurality of initial samples (S311). For example, the data processing unit (160) can construct an initial sample set by having a plurality of initial samples and an evaluation result for each of the plurality of initial samples form a pair. In this case, the initial sample set may be configured to include at least one of a first initial sample-first evaluation result pair in which a first initial sample (X1) and a first evaluation result for said first initial sample form a pair, a second initial sample-second evaluation result pair in which a second initial sample (X2) and a second evaluation result for said second initial sample form a pair, a third initial sample-third evaluation result pair in which a third initial sample (X3) and a third evaluation result for said third initial sample form a pair, and an Nth initial sample-Nth evaluation result pair in which an Nth initial sample (XN) and a Nth evaluation result for said Nth initial sample form a pair (e.g., D = {(X1, 92.5), (X2, 88.3), (X3, 95.7)..., (XN, ...)}). In the present invention, the initial sample set may also be named “initial sample vector set,” “training sample set,” “training sample vector set,” “input sample set,” “input sample vector set,” or “training data set,” etc.

[0137] Meanwhile, in the present invention, a process of training a prediction model using an initial sample set may be performed to calculate an optimal solution for a plurality of parameters (S240, see FIG. 2).

[0138] The control unit (180) can train the prediction model (170) using an initial sample set constructed by performing sampling from a prior distribution estimated for each of a plurality of parameters. As illustrated in FIG. 4, the control unit (180) can train the prediction model (170) using a plurality of initial samples included in the initial sample set and evaluation results for each of the plurality of initial samples (S401). In the present invention, the types of models that can be included in the prediction model (170) may vary. For example, the types of the prediction model (170) may include at least one of a Gaussian Process (GP) model and / or a Random Forest model and / or a Bayesian Neural Network. However, this is merely an example, and the types of models included in the prediction model (170) are not necessarily limited thereto. Additionally, the prediction model (170) may include one or a plurality of models. When multiple models are included, the multiple models can be combined into an ensemble (e.g., combining a Gaussian process model and a random forest model) and used in the optimization process.

[0139] This prediction model (170) can learn an approximation of the target function (f(x)) in an environment where multiple parameters are mixed, by utilizing the prior distribution for each of the multiple parameters. More specifically, instead of directly calculating the actual target function, the prediction model (170) can approximate the target function by learning an initial set of samples (pairs of initial samples and evaluation results for those samples). That is, learning an approximation of the target function can also be understood as the prediction model (170) being trained to approximate multiple initial samples and evaluation results for each of the multiple initial samples. Alternatively, the prediction model (170) can also be understood as being trained in a direction that approximates the target function (i.e., the actual objective function).

[0140] That is, in the present invention, the prediction model (170) can perform the role of providing an approximation of the optimization target function by learning a plurality of initial samples included in an initial sample set and the evaluation results for each of the plurality of initial samples.

[0141] When the training of the prediction model (170) is completed, the control unit (180) can obtain the trained model prediction model (170) based on the training. Then, the control unit (180) can sample candidate evaluation points using the data processing unit (160). The data processing unit (160) can sample candidate evaluation points from the prior distribution for each of the plurality of parameters. The method of sampling candidate evaluation points in the present invention may be the same as the method of sampling the initial sample described above. Accordingly, to avoid duplication of explanation, the method of sampling candidate evaluation points will be described only briefly below.

[0142] As illustrated in FIG. 4, the data processing unit (160) can sample candidate evaluation points based on the estimated prior distribution for each of the plurality of parameters (S403). Alternatively, the data processing unit (160) can generate candidate evaluation points from the prior distribution for each of the plurality of parameters.

[0143] In one embodiment, the data processing unit (160) may perform a first sampling of the estimated prior distribution for each of the plurality of parameters (e.g., a first parameter (e.g., temperature), a second parameter (e.g., part specification code), a third parameter (e.g., pressure), a fourth parameter (e.g., process step), etc.) to obtain the values ​​of each of the plurality of parameters corresponding to the first sampling result (e.g., 73.8°C, A2, 1.05 bar, step 2, etc.). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the first sampling result to generate (or configure) a first candidate evaluation point (e.g., [73.8°C, A2, 1.05 bar, step 2]).

[0144] In another embodiment, the data processing unit (160) may perform a second sampling of the estimated prior distribution for each of the plurality of parameters to obtain the values ​​of each of the plurality of parameters corresponding to the second sampling result (e.g., 74.2°C, B1, 0.96 bar, 2nd stage). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the second sampling result to form a second candidate evaluation point (e.g., [74.2°C, B1, 0.96 bar, 2nd stage]).

[0145] In another embodiment, the data processing unit (160) may perform a third sampling of the estimated prior distribution for each of the plurality of parameters to obtain the values ​​of each of the plurality of parameters corresponding to the third sampling result (e.g., 74.4°C, A2, 1.04 bar, level 1, etc.). In this case, the data processing unit (160) may combine (or combine) the values ​​of each of the plurality of parameters obtained as the third sampling result to form a third candidate evaluation point (e.g., [74.4°C, A2, 1.04 bar, level 1]).

[0146] In this case, the process of generating candidate evaluation points may be a process of generating candidate evaluation points containing different parameter combinations by using the values ​​of each of the multiple parameters sampled from the prior distribution for each of the multiple parameters. In this case, the candidate evaluation points may include new multiple parameter combinations that are not included in the initial sample set used in the training process of the prediction model (170). That is, the data processing unit (160) may generate candidate evaluation points containing different parameter combinations (or corresponding to different parameter combinations) by sampling parameter combinations that are not included in the initial sample set.

[0147] Candidate evaluation points may include combinations of parameters sampled from prior distributions estimated for each of the plurality of parameters. That is, in the present invention, after the prediction model (170) is trained, combinations of parameters sampled from prior distributions of individual parameters may be named “candidate evaluation points.”

[0148] Alternatively, the process of generating candidate evaluation points in the present invention may also be understood as a process of constructing different parameter vectors (e.g., [x1, x2, ... xn]) using values ​​sampled from a prior distribution for each of a plurality of parameters. In the present invention, a candidate evaluation point may also be referred to as a “candidate point,” “candidate evaluation data,” “candidate evaluation sample,” “candidate parameter combination,” “candidate sample vector,” “candidate sample,” “candidate input sample,” “candidate input sample vector,” “candidate input data,” “candidate vector,” “candidate input vector,” “candidate sampling data (sampled data),” “candidate parameter set,” “candidate evaluation set,” or “candidate parameter pair,” etc.

[0149] And, as illustrated in FIG. 4, the control unit (180) can select at least one specific evaluation point among the candidate evaluation points to be evaluated using a learned prediction model (170) and a specific function (S405).

[0150] The learned prediction model (170) can be configured to predict at least one of the predicted value and the predicted variance for each of the candidate evaluation points.

[0151] Here, the predicted value (or function value) may be a predicted average value calculated by the learned prediction model (170) by approximating the value of the objective (or goal) function for a specific input (candidate evaluation point (x)). Instead of directly calculating or experimenting with the actual objective function, it can be understood as a result value estimated through an initial sample set and the prediction model (170) learned from said initial sample set. Such a predicted value can predict the performance of the candidate evaluation point in advance without actual function evaluation. The predicted value of such a learned prediction model can be represented as shown in (c) of FIG. 6.

[0152] Additionally, the prediction variance may be a variance representing the degree of uncertainty regarding the predicted value when the learned prediction model (170) predicts the objective function value at a specific input. The higher the prediction variance, the greater the uncertainty, as there is a lack of data in the corresponding input range or little experience with model training; the higher the exploration value, the greater the likelihood of acquiring new information (however, the model's predicted value may differ significantly from the actual value). On the other hand, the lower the prediction variance, the lower the uncertainty, as there is sufficient data in the corresponding input range and the model has learned well; the higher the utility value, the more advantageous it may be for fine-tuning around an already known optimal point (the model's predicted value is likely to approximate the actual value).

[0153] The control unit (180) can predict a predicted value for each of the candidate evaluation points using the learned prediction model (170). More specifically, the control unit (180) can input each of the candidate evaluation points into the learned prediction model (170) and obtain a predicted value for each of the candidate evaluation points from the learned prediction model (170). For example, when the learned prediction model (170) receives an input for a candidate evaluation point composed of multiple parameters having different characteristics, it can estimate a conditional probability distribution for the input using parameters (e.g., weights, kernel parameters, etc.) obtained through learning through an initial sample set. In this case, the learned prediction model (170) can finally output a predicted value (or predicted mean value) and a predicted variance for the candidate evaluation point. As seen above, the predicted value represents an approximation of the objective function at the candidate evaluation point, and the predicted variance can represent the degree of uncertainty of the approximation.

[0154] In one embodiment, the control unit (180) may input a first candidate evaluation point (e.g., [73.8°C, A2, 1.05 bar, 2nd step]) among candidate evaluation points into the learned prediction model (170). Then, the control unit (180) may obtain from the learned prediction model (170) at least one of the predicted value (e.g., 0.85) and the predicted variance (e.g., 0.05) predicted by the learned prediction model for the first candidate evaluation point.

[0155] In another embodiment, the control unit (180) may input a second candidate evaluation point (e.g., [74.2°C, B1, 0.96 bar, 2nd step]) among the candidate evaluation points into the learned prediction model (170). Then, the control unit (180) may obtain from the learned prediction model (170) at least one of the predicted value (e.g., 0.78) and the predicted variance (e.g., 0.10) predicted by the learned prediction model for the second candidate evaluation point.

[0156] In another embodiment, the control unit (180) may input a third candidate evaluation point (e.g., [74.4°C, A2, 1.04 bar, 1st step]) among the candidate evaluation points into the learned prediction model (170). Then, the control unit (180) may obtain from the learned prediction model (170) at least one of the predicted value (e.g., 0.90) and predicted variance (e.g., 0.02) predicted by the learned prediction model for the second candidate evaluation point.

[0157] Next, the control unit (180) can select a specific evaluation point using a predicted value and a specific function for each of the candidate evaluation points.

[0158] Specifically, the control unit (180) calculates a specific function value for each of the candidate evaluation points using at least one of the predicted value for each of the candidate evaluation points and the optimal value that serves as a criterion for selecting a specific evaluation point, and among the candidate evaluation points, selects (or specifies) at least one evaluation point where the calculated specific function value satisfies a preset criterion as a specific evaluation point. Here, the optimal value that serves as a criterion for selecting a specific evaluation point may refer to the function value of the point with the best target function value among all evaluation points obtained so far in the optimization process. This may be a value measured through experiments and / or simulations, or may correspond to at least one of the highest values ​​among the predicted values ​​predicted by the learned prediction model (170).

[0159] In the present invention, a specific function may include an Expected Improvement (EI) function. The Expected Improvement (EI) function may be an acquisition function that probabilistically estimates the degree of expected performance improvement at a new candidate evaluation point compared to the optimal value observed so far during the Bayesian optimization process. This specific function may be represented as shown in FIG. 6 (a). Here, the expected value may be represented as shown in FIG. 6 (b), the predicted value of the prediction model (170) for each of the candidate evaluation points may be represented as shown in FIG. 6 (c), and the optimal value observed so far (or the optimal value serving as a criterion for selecting a specific evaluation point) may be represented as shown in FIG. 6 (d). At this time, FIG. 6 (e) may mean that it is reflected only when there is improvement.

[0160] The control unit (180) can calculate a specific function value for each of the multiple candidate evaluation points using the mathematical formula (i.e., a specific function) shown in (a) of FIG. 6.

[0161] In one embodiment, the control unit (180) can calculate a specific function value (e.g., 0.0542) for a first candidate evaluation point (e.g., [73.8°C, A2, 1.05 bar, 2nd step]) by using at least one of a predicted value (e.g., 0.85), a predicted variance (e.g., 0.05), and an optimal value (e.g., 0.80) for a first candidate evaluation point.

[0162] In another embodiment, the control unit (180) can calculate a specific function value (e.g., 0.0307) for a second candidate evaluation point (e.g., [74.2°C, B1, 0.96 bar, 2nd step]) using at least one of a predicted value (e.g., 0.78), a predicted variance (e.g., 0.10), and an optimal value (e.g., 0.80) for the second candidate evaluation point.

[0163] In another embodiment, the control unit (180) can calculate a specific function value (e.g., 0.1) for a third candidate evaluation point (e.g., [74.4°C, A2, 1.04 bar, 1st step]) using at least one of a predicted value (e.g., 0.90), a predicted variance (e.g., 0.02), and an optimal value (e.g., 0.80) for the third candidate evaluation point.

[0164] Furthermore, as a result of calculating a specific function value for each of the multiple candidate evaluation points, the control unit (180) may select at least one evaluation point among the multiple candidate evaluation points as a specific evaluation point, wherein the calculated specific function value satisfies a preset criterion. For example, the preset criterion (or preset selection criterion) may be related to a point among the multiple candidate evaluation points that maximizes the specific function value.

[0165] The control unit (180) may select a point that maximizes a specific function value among a plurality of candidate evaluation points as a specific evaluation point. Such a specific evaluation point may include a point that maximizes a specific function value. Here, selecting a point that maximizes a specific function value (or an acquired function value) may mean selecting a point with the highest improvement expectation, rather than simply a point with a high predicted value. The acquired function described above is a function that numerically expresses the expectation of being better than the current optimal value by considering both the predicted value and uncertainty predicted by the prediction model (170). In this case, a high improvement expectation may include at least one of the following: i) high prediction performance, ii) high possibility of improvement due to high uncertainty (prediction variance), or iii) both. That is, selecting a point that maximizes a specific function value can be understood as selecting the point with the most expected performance improvement in the next evaluation.

[0166] For example, the control unit (180) may select the third candidate evaluation point, which has the maximum (or highest) specific function value among the multiple candidate evaluation points, as the next specific evaluation point to proceed with evaluation, based on the specific function value for the first candidate evaluation point (e.g., 0.0542), the specific function value for the second candidate evaluation point (e.g., 0.0307), and the specific function value for the third candidate evaluation point (e.g., 0.1). However, the number of specific evaluation points selected is not necessarily limited to one. For example, if there are multiple candidate evaluation points with the same specific function value, multiple specific evaluation points may be selected.

[0167] As seen above, the prediction model (170) may refer to a model that predicts the value of an objective function by performing learning based on initial data and an estimated prior distribution, instead of directly evaluating the actual objective function during the optimization process. This prediction model (170) learns the interaction between multiple parameters having different characteristics to select the next optimization evaluation point. That is, the prediction model (170) can reduce the total number of evaluations and improve optimization efficiency by using the learned interaction information to select the next optimization evaluation point through an acquisition function (e.g., Expected Improvement (EI) function). Below, we will examine the process of updating the prediction model (170) based on the selected evaluation point.

[0168] Meanwhile, as illustrated in FIG. 4, the present invention can perform an evaluation on a selected specific evaluation point (S407) and update a learned prediction model based on the evaluation result for the specific evaluation point (S409).

[0169] As seen above, based on the learned prediction model (170) and a specific function, candidate evaluation points that are most likely to improve the optimization objective function can be determined (or selected).

[0170] The control unit (180) can input a combination of parameters of a selected specific evaluation point into a Bayesian optimization system (100, or a simulator, experimental device, etc.) to calculate or measure the actual objective function value. This process may be a process of obtaining an actual measured value rather than a process of obtaining a predicted value of the model for the selected specific evaluation point. For example, the control unit (180) can input an objective function for the selected specific evaluation point into a Bayesian optimization system (100, or a simulator, experimental device, etc.) and obtain an actual measured value (or actual performance value) from the Bayesian optimization system (100). In the present invention, such an actual measured value may correspond to an evaluation result for the specific evaluation point.

[0171] Additionally, the control unit (180) can expand the initial sample set (or training data set) used in the learning process of the prediction model (170) by adding a newly acquired specific evaluation point (new parameter combination) and an evaluation result for said specific evaluation point to the initial sample set (or training data set). This can also be understood as a process of updating (or renewing) the initial sample set.

[0172] Furthermore, the updated initial sample set can be used in the update process of the learned prediction model (170). The control unit (180) can update the learned prediction model (170) using the updated initial sample set. Alternatively, the control unit (180) can retrain the learned prediction model (170) using the updated initial sample set. In this case, during retraining, the estimated prior distribution for each parameter is reflected, and the density function of the KDE can also be adaptively updated according to the newly added data. That is, in the present invention, during the retraining process, the newly added evaluation point reduces the uncertainty region of the model and improves the overall prediction accuracy.

[0173] Finally, as illustrated in FIG. 4, the control unit (180) can calculate an optimal solution for a plurality of parameters (or an optimal solution for a plurality of parameters, an optimal solution for a combination of different plurality of parameters, an optimal solution for a combination of different plurality of parameters, etc.) based on an update of the prediction model (S411). Here, the optimal solution may include an optimal combination of parameters calculated from a plurality of parameters (or from a combination of different plurality of parameters). In the present invention, the optimal solution may also be referred to as an “optimal solution” or “optimal combination of parameters,” etc. That is, the control unit (180) can perform Bayesian optimization based on the learned prediction model to optimize for a plurality of parameters, and calculate an optimal solution for a plurality of parameters based on the optimization result (or based on the optimization result).

[0174] The present invention can adaptively update the prior distribution of each parameter in response to real-time data inflow. For example, when new measurement data or environmental changes are detected, the kernel density function (KDE) of the corresponding parameter can be configured to be automatically corrected to reflect the latest distribution. Through this, the system can immediately respond to optimization conditions that change over time and achieve high efficiency in applications such as real-time quality control or online tuning.

[0175] Meanwhile, the Bayesian optimization method (or process) according to the present invention described above can be performed iteratively depending on whether a pre-set convergence criterion is satisfied.

[0176] Here, the pre-set convergence criteria may be related to determining whether the search has progressed sufficiently enough to yield an optimal solution. For example, the pre-set convergence criteria may include at least one of the following: i) an improvement limit criterion (e.g., if the improvement range of the objective function value during the last k iterations is below a certain value, the optimal solution candidate is no longer updated and the process is terminated), ii) an acquisition function criterion (e.g., if the maximum value of the acquisition function falls below a certain threshold, it is determined that there is no expected gain to be obtained through further evaluation and the process is terminated), iii) an evaluation budget criterion (e.g., if the maximum allowable number of evaluations (number of experiments / simulations)) is reached, the optimal candidate up to that point is calculated as the optimal solution), and iv) a prediction variance (or uncertainty) criterion of the prediction model (170) (e.g., if the prediction variance decreases below a threshold in the entire design space, it is determined that sufficient information has been obtained for the entire search space and the process is terminated).

[0177] If the control unit (180) determines that the preset convergence criteria are not satisfied (or are not satisfied), it may repeatedly perform the process of estimating individual prior distributions by performing KDE on each parameter, as in the optimization algorithm according to the present invention shown in FIG. 7, sampling from the estimated prior distribution, learning a prediction model using the sampled data, selecting the next evaluation point using an acquisition function, performing an evaluation on the selected evaluation point and updating the model, and calculating the optimal solution. In the present invention, this process may be performed N times (or N cycles, N iterations, etc.) until the preset convergence criteria are satisfied.

[0178] Meanwhile, in the present invention, the estimated prior distribution for each parameter can be selectively maintained and / or updated (or updated).

[0179] The control unit (180) can selectively update only the prior distribution of the changed parameters when only some of the parameters among a plurality of parameters having different characteristics are changed. Here, selectively updating only the KDE of the changed parameters means a method of updating (or updating) only the probability density estimation (i.e., prior distribution estimation) of the changed parameters without needing to recalculate the distribution of all parameters by using a structure that independently models the prior distribution (or probability distribution) of each parameter in the Bayesian optimization process according to the present invention using an individual kernel density estimation (KDE) technique (or method).

[0180] Specifically, the control unit (180) can re-perform kernel density estimation (KDE) only for specific parameters that have changed values ​​when only some of the parameters (or sets of parameters) having different characteristics are changed during the Bayesian optimization process according to the present invention. That is, for parameters that have not changed, the prior distribution estimation results calculated in the S220 process are reused as is, and for specific parameters that have changed, the prior distribution can be re-estimated by selectively reflecting newly added data (or samples). Here, change may mean a case where there is a possibility that the distribution estimation of specific parameter values ​​may change due to new evaluation results (e.g., evaluation results for selected specific evaluation points), experimental data, or external input (input of user terminal (10)).

[0181] In one embodiment, the prior distribution (or information about the prior distribution) estimated for each of a plurality of parameters using a pre-set probability distribution estimation technique may be stored in a storage unit (140, or memory). Each of the plurality of parameters and the prior distribution for each of the plurality of parameters may be matched with each other and stored in the storage unit (140). The control unit (180) may specify (or identify) the changed parameter among the plurality of parameters stored in the storage unit (140) and re-perform kernel density estimation (or probability distribution estimation) only for the specified parameter. The kernel density re-estimation is performed based on existing data (initial sample) + new data (selected specific evaluation point + evaluation result for the specific evaluation point), and the bandwidth (h) and kernel function (K) may be applied according to the characteristics of the corresponding parameter. For the remaining parameters excluding the specified parameter, the existing prior distribution estimation results may be used as is. Through this, the computational cost required to recalculate the prior distribution (KDE) for all parameters from scratch is reduced, and computational efficiency can be maximized by avoiding unnecessary redundant calculations. The process of selectively updating such prior distributions can be performed during the process of updating the prediction model (170) based on the evaluation results for a specific evaluation point (e.g., process S409 of FIG. 4). Afterwards, the partially updated KDE results and the unchanged KDE results can be integrated and reflected in the sampling process and the model training process.

[0182] In this regard, when a Bayesian optimization process according to the present invention is repeatedly performed after the prediction model (170) is updated based on the evaluation results for a specific evaluation point, the control unit (180) may perform sampling from each of the prior distribution for the updated specific parameter and the prior distribution for the remaining parameters different from the specific parameter among a plurality of parameters stored in the storage unit (140), and learn and update the prediction model (170) using the sampled values.

[0183] In one embodiment, let us assume that after the Bayesian optimization process according to the present invention up to a specific iteration is terminated, the Nth iteration of the Bayesian optimization process according to the present invention is carried out. The control unit (180) can obtain the value of the updated specific parameter and the value of each of the remaining parameters different from the specific parameter by performing sampling from the prior distribution for the updated specific parameter and the prior distribution for the remaining parameters different from the specific parameter, respectively, among the plurality of parameters stored in the storage unit (140). Then, the control unit (180) can generate candidate evaluation points using the value of the updated specific parameter and the value of each of the remaining parameters different from the specific parameter. After the candidate evaluation points are generated, the control unit (180) can select at least one specific evaluation point among the candidate evaluation points to be evaluated using a learned prediction model and a specific function, and perform an evaluation on the selected specific evaluation point. Finally, the control unit (180) can update the learned prediction model based on the evaluation result for the specific evaluation point and calculate the optimal solution for the plurality of parameters based on the update to the learned prediction model.

[0184] As such, the present invention can significantly reduce the amount of computation compared to re-estimating the entire joint distribution by selectively updating (or recalculating) only the prior distribution of the changed parameters. In particular, the iteration speed can be accelerated by reducing unnecessary calculations in high-dimensional problems. Furthermore, in cases of frequent environmental changes or data inflows, real-time optimization is possible by immediately updating only the relevant parameters (e.g., immediately reflecting changes in process conditions to variables to immediately search for improved candidates in the next iteration). Additionally, since the distribution estimation of the changed parameters is maintained in an up-to-date state, the subsequent sampling process better reflects the actual data distribution. Consequently, unnecessary searches are reduced and search efficiency is increased, which can lead to a faster speed in reaching the global optimal value (optimal solution).

[0185] Meanwhile, as illustrated in FIG. 8, according to one embodiment of the experimental results of the Bayesian optimization method and system according to the present invention, when KDE sampling is applied, the number of simulations is 595, which is reduced by up to 45% compared to the conventional method, and it can be confirmed that time and / or cost are reduced. In this structure, even when only some KDEs are updated, there is an advantage of further reducing update costs while maintaining the same effect of reducing the number of simulations. That is, partially updating KDEs in the present invention can provide major advantages in terms of optimization response speed, computational efficiency, memory reduction, and maintenance of model stability.

[0186] Meanwhile, according to an additional embodiment of the Bayesian optimization method and system of the present invention, the present invention may utilize an adaptive KDE or a clustering-based KDE. That is, the present invention performs KDE for each parameter to reflect the characteristics of each parameter, and in this case, it is possible not only to use a fixed kernel density width for all parameters but also to dynamically adjust the bandwidth according to the characteristics of the parameters. For example, by additionally applying a clustering-based KDE, the present invention can achieve the effect of increasing computational efficiency by forming clusters of parameters with similar characteristics to use the same bandwidth.

[0187] Furthermore, the present invention can contribute to more accurately reflecting the combined distribution of multiple parameters by reinforcing interaction information between parameters within a cluster. Moreover, once the initial distribution of each parameter is set, the present invention sets an optimal bandwidth tailored to the characteristics of each parameter using an adaptive KDE; thereby, a wide bandwidth is set for parameters within a wide range to stably estimate the distribution, while a relatively small bandwidth is used for parameters within a narrow range to more accurately represent the dense distribution.

[0188] Meanwhile, the present invention can apply multi-objective optimization. That is, not only can Bayesian optimization be performed according to individual parameter characteristics even in high dimensions and wide ranges, but it is also possible to consider various objective functions simultaneously. For example, by introducing multi-objective optimization, multiple objectives, such as maximizing performance while minimizing cost, can be optimized simultaneously, and accordingly, Pareto optimization techniques can be utilized. Furthermore, the present invention, which includes a multi-objective optimization function, can enable optimization to reflect various goals by applying Expected Hypervolume Improvement (EHVI), a multi-objective gain function, rather than EI, which considers only a single evaluation point.

[0189] Furthermore, the present invention enables the incorporation of real-time data through online learning. While the present invention estimates a prior distribution using KDE and performs optimization based thereon, it is not limited to a method of setting the prior distribution using fixed data, but can flexibly incorporate a modified distribution derived from real-time data. In particular, by introducing online learning and an incremental KDE update method, it is possible to update the KDE of each parameter according to new data input in real time. This approach can be advantageous in systems with frequent changes, such as factory automation. For example, when a new data point is collected, the point is immediately reflected in the existing distribution to update the KDE, thereby enabling real-time adaptation.

[0190] Meanwhile, the present invention may introduce ensemble learning of surrogate models. That is, while the present invention focuses on improving optimization performance by reflecting the interaction between each parameter, it may be configured to combine multiple models to provide an average prediction or select a model suitable for a specific goal in order to reduce the possibility of overfitting of a single surrogate model.

[0191] In particular, the present invention enables a method of simultaneously training Gaussian Process and Random Forest-based models to perform optimization and determining the optimal evaluation point by averaging the results of the two models.

[0192] Furthermore, the present invention can add a model reliability evaluation function through the quantification of uncertainty. That is, the present invention enables a quantitative evaluation of model uncertainty by including a model reliability evaluation when estimating the distribution of each parameter during the Bayesian optimization process. To this end, the present invention enables a method of estimating uncertainty together with model prediction by applying Bayesian Neural Networks (BNNs), and thereby contributes to increasing the reliability of the optimization process by reflecting uncertainty regarding specific parameters.

[0193] Meanwhile, the present invention can complement the vision inspection system of production equipment. In particular, the present invention can help detect defects and inspect quality quickly and accurately in production processes, even in complex environments with various variables. For example, in the case of detecting irregular defects, the present invention can flexibly respond to various types of defects by precisely estimating the distribution of each variable. In particular, when inspecting defects that deviate from specifications, such as fine scratches on metal surfaces or asymmetrical geometric changes, inspection accuracy can be improved by establishing optimal criteria for each variable.

[0194] Since vision inspection is sensitive to changes in lighting or the surrounding environment, it can be optimized by using adaptive KDEs and surrogate models that reflect environmental variations. This invention incorporates lighting conditions as variables to learn inspection criteria differently for each lighting environment, and maintains the stability of defect detection even when lighting changes by applying an appropriate bandwidth. For example, it enables consistent quality inspection of product surfaces by adaptively considering differences in reflectivity according to angle when lighting intensity changes.

[0195] As discussed above, the Bayesian optimization method and system according to the present invention can improve optimization performance in an environment where multiple parameters having different characteristics are mixed, by estimating the individual distribution for each of the multiple parameters. In other words, by processing each parameter individually, the present invention can prevent performance degradation in high-dimensional problems and mitigate the curse of dimensionality.

[0196] Furthermore, according to the Bayesian optimization method and system of the present invention, a prior distribution for each of a plurality of parameters having different characteristics can be estimated, and Bayesian optimization can be performed based on the estimated distribution. Through this, the present invention minimizes the total search space and concentrates search resources on a valid and high-performance space, thereby reducing unnecessary evaluations and improving the optimization convergence speed.

[0197] In addition, according to the Bayesian optimization method and system of the present invention, computational costs are reduced by estimating the prior distribution for each of a plurality of parameters having different characteristics, and individual learning rates or bandwidths can be adjusted according to the characteristics of each parameter.

[0198] Furthermore, according to the Bayesian optimization method and system of the present invention, each parameter can be processed simultaneously according to the characteristics of each of the multiple parameters, and accurate estimation is possible even within a wide optimization range of each parameter.

[0199] Furthermore, according to the Bayesian optimization method and system of the present invention, when only some parameters among a plurality of parameters having different characteristics are changed, only the prior distribution of the changed parameters can be selectively updated. Through this, the present invention can improve the efficiency of incremental learning and can apply different bandwidths to each parameter, thereby increasing the accuracy of distribution estimation according to the characteristics of each variable. In addition, distribution information of some parameters can be reused from similar optimization problems to be quickly applied to new problems, and the distribution of related parameters can be immediately updated whenever new data is received, thereby enabling real-time response to environmental changes. Accordingly, since the present invention only needs to store the distribution information of each parameter, memory can be used efficiently to save storage space.

[0200] As such, the present invention efficiently processes complex mixed variables and enables accurate estimation even within a wide optimization range for each variable. This method can provide improved performance in high-dimensional problems, effective processing of mixed variables, and increased computational efficiency. In particular, the present invention can be usefully applied to real industrial environments where interactions between variables are complex and the range of influence of each variable is wide, such as factory automation or complex system optimization.

[0201] Furthermore, the Bayesian optimization structure of the present invention is not limited to a single system but can be applied in a network environment where multiple optimization modules are distributed. For example, it can be configured so that multiple devices or processors independently estimate the prior distribution of parameters, and a central control node integrates each distribution information to perform global optimization. This structure can be extended and applied to cloud-based optimization servers, cooperative optimization systems between edge devices, or quantum-classical hybrid computing systems. Therefore, the present invention can be implemented with the same probabilistic search mechanism not only in a single device but also in a distributed system architecture.

[0202] Therefore, the present invention can be immediately utilized in actual industrial fields such as industrial process control, robot motion parameter adjustment, semiconductor production line tuning, medical image analysis system optimization, and energy efficiency control. In particular, compared to conventional single-distribution-based optimization, the present invention reduces the number of evaluations and concentrates search resources in an effective space, thereby simultaneously achieving process cost reduction and quality stability. Furthermore, this structure can be extended into an AI-based autonomous optimization engine, allowing it to be applied to future intelligent control systems that require complex parameter combinations.

[0203] As described above, the present invention aims to solve the problem of reduced optimization efficiency when performing Bayesian optimization in a high-dimensional environment where continuous and discrete variables are mixed, in which it is difficult to accurately estimate the joint distribution between variables, computational costs increase rapidly due to the curse of dimensionality, and individual estimation tailored to variable characteristics (continuous / discrete) is impossible.

[0204] To address this, the present invention provides a Bayesian optimization method and system that independently estimates a prior distribution by applying different probability distribution estimation techniques according to the characteristics of each parameter, and constructs an initial sample set based thereon. Specifically, Kernel Density Estimation (KDE) or categorical distribution estimation techniques are applied for each parameter type, and a prediction model is trained using the sampling results.

[0205] In addition, the learned prediction model can be configured to progressively produce an optimal solution by selecting candidate evaluation points reflecting the interaction between each parameter and updating the model by reflecting the selected evaluation results.

[0206] Through this configuration, the present invention solves the computational complexity problem of conventional technology that modeled all variables as a single joint distribution, and simultaneously improves optimization accuracy and convergence speed by accurately reflecting the statistical characteristics of each variable. In particular, it can mitigate the curse of dimensionality even in high-dimensional mixed variable environments and reduce the number of evaluations by automatically excluding unnecessary search intervals. Furthermore, since only the prior distribution of specific parameters can be selectively updated when only some parameters are changed, it can efficiently respond to real-time environmental changes.

[0207] Therefore, the present invention can expand the scope of application of existing Bayesian optimization and provide fast and accurate search performance required in industrial settings.

[0208] Meanwhile, the present invention described above can be implemented based on a quantum computer. The present invention implemented based on a quantum computer may include a qubit-based quantum processor and quantum memory, and may include software and hardware interfaces optimized for quantum computation.

[0209] Quantum processors in quantum computers utilize qubits to efficiently process complex operations through parallel computation, quantum entanglement, and quantum superposition, which cannot be performed by the binary bits of classical computers. Quantum processors process data using quantum gates and can provide exponential speed improvements for specific problems.

[0210] Meanwhile, the present invention described above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a computer-readable medium (or recording medium).

[0211] Furthermore, the present invention described above can be implemented as computer-readable code or instructions on a medium on which a program is recorded. That is, the present invention can be provided in the form of a program.

[0212] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0213] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.

[0214] A computer program may reach the system (100) through various suitable transmission mechanisms. The transmission mechanism may be, for example, a computer-readable storage medium, a computer program product, a memory device, a recording medium such as a CD-ROM or DVD, or a product that tangibly embodies the computer program. The transmission mechanism may be a signal configured to reliably transmit the computer program through air or an electrical connection. The system (100) may propagate or transmit the computer program as a computer data signal.

[0215] Furthermore, references to 'computer-readable storage media,' 'computer program products,' 'computer programs embodied in a tangible form,' etc., or to 'controller,' 'computer,' 'processor,' etc., should be understood to include not only computers with various architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures, but also specialized circuits such as Field-Programmable Gate Arrays (FPGAs), Application Specific Circuits (ASICs), signal processing units, and other devices. References to computer programs, instructions, code, etc., should be understood to include software for programmable processors or firmware, such as programmable content for hardware devices, whether it is instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device.

[0216] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.

[0217] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

1. In a computerized method comprising the following, A step of specifying multiple parameters having different characteristics; A step of estimating a prior distribution for each of the plurality of parameters using a pre-established probability distribution estimation technique; A step of constructing an initial sample set based on the prior distribution estimated for each of the plurality of parameters; and A Bayesian optimization method characterized by including the step of training a prediction model using the initial sample set to produce an optimal solution for the plurality of parameters.

2. In Paragraph 1, In the step of estimating the prior distribution mentioned above, For each of the plurality of parameters, the pre-set probability distribution estimation technique is applied to estimate the prior distribution for each of the plurality of parameters, and The above initial sample set is, A Bayesian optimization method characterized by being constructed by performing sampling from the estimated prior distribution for each of the plurality of parameters.

3. In Paragraph 2, The step of configuring the above initial sample set is, A step of sampling the value of each of the plurality of parameters from a prior distribution for each of the plurality of parameters; A step of generating a plurality of initial samples using the values ​​of each of the sampled plurality of parameters; and A Bayesian optimization method characterized by including the step of constructing an initial sample set using a plurality of initial samples to train the prediction model.

4. In Paragraph 3, In the step of configuring the above initial sample set, An evaluation is performed on each of the plurality of initial samples sampled from the prior distribution for each of the plurality of parameters, and Based on the above evaluation, an evaluation result for each of the plurality of initial samples is obtained, and A Bayesian optimization method characterized by constructing the initial sample set using the plurality of initial samples and the evaluation results for each of the plurality of initial samples.

5. In Paragraph 4, In the step of training the above prediction model, A Bayesian optimization method characterized by training the prediction model using the plurality of initial samples included in the initial sample set and the evaluation results.

6. In Paragraph 4, The above prediction model is, A Bayesian optimization method characterized by learning an approximation of a function to be optimized in an environment where the plurality of parameters are mixed, using a prior distribution for each of the plurality of parameters.

7. In Paragraph 6, A Bayesian optimization method characterized in that the learning of the approximation of the function to be optimized is a process of learning to approximate the plurality of initial samples and the evaluation results for each of the plurality of initial samples.

8. In Paragraph 3, The step of generating the above plurality of initial samples is, A Bayesian optimization method characterized by the step of generating a plurality of initial samples including different parameter combinations using the values ​​of each of the plurality of parameters sampled from a prior distribution for each of the plurality of parameters.

9. In Paragraph 1, The above prediction model is, A Bayesian optimization method characterized by selecting an optimization evaluation point by learning the interaction between multiple parameters having different characteristics.

10. In Paragraph 1, A step of sampling candidate evaluation points from a prior distribution for each of the plurality of parameters; and A Bayesian optimization method characterized by further including the step of selecting at least one specific evaluation point among the candidate evaluation points to be evaluated using the above-mentioned learned prediction model and specific function.

11. In Paragraph 10, The step of sampling the above candidate evaluation points is, A Bayesian optimization method characterized by the step of sampling candidate evaluation points containing different parameter combinations from a prior distribution for each of the plurality of parameters.

12. In Paragraph 10, In the step of selecting the specific evaluation point mentioned above, Using the above-mentioned learned prediction model, predict a predicted value for each of the above-mentioned candidate evaluation points, and A Bayesian optimization method characterized by selecting a specific evaluation point using a predicted value for each of the above candidate evaluation points and a specific function.

13. In Paragraph 12, In the step of selecting the specific evaluation point mentioned above, Calculate the specific function value for each of the candidate evaluation points using at least one of the optimal values ​​serving as a criterion for selecting the above predicted value and the above specific evaluation point, and A Bayesian optimization method characterized by selecting, among the above candidate evaluation points, at least one evaluation point in which the calculated specific function value satisfies a preset criterion as the specific evaluation point.

14. In Paragraph 13, A Bayesian optimization method characterized in that the above-mentioned specific evaluation point includes a point that maximizes the above-mentioned specific function value.

15. In Paragraph 10, The above-mentioned learned prediction model is, A Bayesian optimization method characterized by predicting at least one of a predicted value and a predicted variance for each of the above-mentioned candidate evaluation points.

16. In Paragraph 10, A step of performing an evaluation on the specific evaluation point selected above; A step of updating the learned prediction model based on the evaluation results for the specific evaluation point; and A Bayesian optimization method characterized by further including the step of calculating an optimal solution for a plurality of parameters based on the update to the learned prediction model.

17. In Paragraph 16, A Bayesian optimization method characterized in that the optimal solution includes an optimal combination of parameters calculated from the plurality of parameters.

18. In Paragraph 1, The plurality of parameters having the above different characteristics are, It includes at least one of a first parameter having a first characteristic and a second parameter having a second characteristic, and In the step of estimating the prior distribution mentioned above, By applying the pre-set probability distribution estimation technique to each of the first parameter and the second parameter, a prior distribution for each of the first parameter and the second parameter is estimated, and The above prediction model A Bayesian optimization method characterized by being learned based on the prior distribution estimated for each of the first parameter and the second parameter using the above-mentioned probability distribution estimation technique.

19. A system comprising memory configured to store executable instructions and one or more processors configured to perform operations by executing one or more instructions, The above system is, Specifying multiple parameters having different characteristics, and Using a pre-established probability distribution estimation technique, a prior distribution for each of the plurality of parameters is estimated, and Based on the prior distribution estimated for each of the above multiple parameters, an initial sample set is constructed, and A Bayesian optimization system characterized by training a prediction model using the initial sample set to produce an optimal solution for the plurality of parameters.

20. A program that is executed by one or more processes in an electronic device and stored on a computer-readable recording medium, The above program is, A step of specifying multiple parameters having different characteristics; A step of estimating a prior distribution for each of the plurality of parameters using a pre-established probability distribution estimation technique; A step of constructing an initial sample set based on the prior distribution estimated for each of the plurality of parameters; and A program stored on a computer-readable recording medium characterized by including instructions for performing a step of training a prediction model using an initial set of samples to calculate an optimal solution for the plurality of parameters.

Images