Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program that generate an accurate estimation model and efficiently optimize a setting value.SOLUTION: In an information processing system 10, an information processing device includes a processing unit that generates an estimation model on the basis of change direction information and one or more data sets each including n setting values and one or more evaluation values representing evaluations of an experiment or a simulation performed using the n setting values, and calculates, on the basis of the estimation model, n recommended values that are recommended as n setting values to be used in the experiment or the simulation. The change direction information indicates, for any one or more combinations of n parameters corresponding to n setting values and one or more evaluation values, the direction of change of a target evaluation value among one or more evaluation values in response to a change in a target parameter among the n parameters.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In various fields, optimization based on simulation is utilized to improve the set values configured in a system. For example, the set values configured in a manufacturing system for manufacturing a product can be calculated by optimization based on simulation. In this case, for example, an optimal set value is calculated by an optimization process that maximizes or minimizes an evaluation function for evaluating the system for manufacturing the product.

[0003] Also, constrained optimization based on simulation is known. Constrained optimization based on simulation calculates an optimal set value by an optimization process that maximizes or minimizes an evaluation function under the condition that a constraint function representing the constraints of the system is equal to or greater than a predetermined lower threshold value and equal to or less than a predetermined upper threshold value. As such constrained optimization based on simulation, constrained Bayesian optimization is known.

[0004] By the way, constrained Bayesian optimization repeats the process of calculating a recommended set value, executing an experiment or simulation based on the recommended set value to calculate an evaluation value, generating an estimation model based on the pair of the recommended set value and the evaluation value, and calculating a new recommended set value using the generated estimation model. Such constrained Bayesian optimization can reduce the number of simulation times and improve the calculation efficiency if an accurate estimation model can be generated. However, it is very difficult to generate an accurate estimation model.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Non-Patent Literature

[0006]

Non-Patent Literature 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] The problem to be solved by the present invention is to generate an accurate estimation model and efficiently optimize set values.

Means for Solving the Problems

[0008] The information processing apparatus according to the embodiment includes a processing unit. The processing unit generates an estimation model based on one or more data sets each including n set values (n is an integer of 1 or more) and one or more evaluation values representing evaluations of experiments or simulations executed using the n set values, and change direction information. The processing unit calculates n recommended values recommended as the n set values to be used in the experiment or the simulation based on the estimation model. The change direction information indicates the direction of change of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0011] FIG. 1 is a diagram showing the configuration of an information processing system 10 according to the embodiment.

[0012] The information processing system 10 calculates and outputs optimal n set values used in the manufacture of a product in order to improve productivity, yield, and reliability in a manufacturing system for manufacturing products such as semiconductors. Note that n represents an integer of 1 or more.

[0013] The n set values are values input to the manufacturing system, such as processing time, dimensions, resistance, voltage, and charge. Each of the n set values is independent of each other and is an individual value. Each of the n set values may be any of a continuous value, a discrete value, and a logical value (categorical variable). That is, the type of each of the n set values is not particularly limited. For example, each of the n set values may represent a physical value such as temperature and pressure, or may represent a value related to the operation of the system such as processing time and processing conditions.

[0014] The objective function is a function that includes n parameters, and is a function for calculating an objective function value that represents an evaluation of a manufacturing system, such as quality characteristics, defect rate, manufacturing time, and manufacturing cost of a manufactured product. Each of the n parameters is a variable representing a set value input to the manufacturing system, such as processing time, dimensions, resistance, voltage, and charge. The n parameters correspond one-to-one to the n set values. The constraint function is a function that includes n parameters and calculates a constraint function value representing the constraints of the manufacturing system.

[0015] When the information processing system 10 does not include a constraint function, it calculates and outputs the values of the n parameters that optimize (e.g., minimize or maximize) the objective function value as the optimal n set values. When the information processing system 10 includes one or more constraint functions, it calculates and outputs the values of the n parameters that optimize (e.g., minimize or maximize) the objective function value as the optimal n set values under the condition that each of the one or more constraint function values is not less than a separately determined lower threshold value and not more than a predetermined upper threshold value. Also, when the information processing system 10 includes a plurality of objective functions, it calculates and outputs the values of the n parameters in a plurality of sets that result in non-dominated solutions for the plurality of objective function values as a plurality of sets of optimal n set values. Then, the user sets the optimal n set values output from the information processing system 10 in the manufacturing system. Thereby, the manufacturing system can improve, for example, the productivity, yield, and reliability of products such as semiconductors.

[0016] In the present embodiment, each of the objective function and the constraint function is referred to as an evaluation function. Also, in the present embodiment, each of the objective function value and the constraint function value is referred to as an evaluation value.

[0017] Note that the information processing system 10 is not limited to such a manufacturing system, and may output the optimal n set values set for any system, experiment, information processing, etc. For example, the information processing system 10 may output the optimal set values of each of the n parameters used in a power generation plant, and the optimal set values of each of the n hyperparameters used in machine learning.

[0018] Further, the optimal n set values are n values regarded as optimal by the information processing system 10, regardless of whether they are actually optimal. Also, the information processing system 10 may calculate multiple sets of optimal n set values.

[0019] Also, in this specification, the number of n parameters may be referred to as the number of items or dimensions of the model. Also, in this specification, for example, the nth (n is an integer of 2 or more) parameter among the n parameters may be referred to as the parameter of the nth dimension.

[0020] The information processing system 10 performs black box optimization to search for the optimal n set values by repeatedly calculating the n set values based on the results of experiments or simulations. The information processing system 10 executes a simulation using a simulation model including n parameters. The information processing system 10 repeatedly generates n set values set for the n parameters included in the simulation model. Instead of the simulation, an experiment may be executed, or the information processing system 10 may obtain the results of an experiment conducted by a user or the like. In this case, the information processing system 10 outputs n set values regarding, for example, the amount of a sample used in the experiment as a parameter. Note that hereinafter, unless otherwise particularly described, the term "experiment" shall include simulations.

[0021] Also, in this embodiment, the information processing system 10 calculates and outputs optimal n set values using Bayesian optimization, which is an example of black box optimization. In particular, in this embodiment, the information processing system 10 uses a Gaussian process regression model considering monotonicity as an estimation model for calculating the estimated values and estimated standard deviations of one or more evaluation values based on n parameters. The Gaussian process regression model considering monotonicity is a Gaussian process regression model that uses information for identifying whether, for each of the n parameters, the target evaluation value among one or more evaluation values monotonically increases as the target parameter increases, whether the target evaluation value monotonically decreases as the target parameter increases, or whether the target evaluation value neither monotonically increases nor monotonically decreases with respect to the target parameter.

[0022] Such a Gaussian process regression model considering monotonicity is shown in Non-Patent Document 1. Further details of the Gaussian process regression model considering monotonicity will be described later.

[0023] The information processing system 10 includes an information processing device 20 and an evaluation device 30.

[0024] The information processing device 20 outputs n recommended values as n set values to be set for n parameters used in the experiment.

[0025] The evaluation device 30 evaluates the results of the experiment based on the n recommended values output from the information processing device 20 and generates information regarding the evaluation. The information regarding the evaluation includes one or more evaluation values. Each of the one or more evaluation values represents an evaluation of the results of the experiment executed using the n recommended values output from the information processing device 20. Note that the evaluation device 30 may perform a simulation based on the n recommended values and generate information regarding the evaluation based on the execution results of the simulation. When the user conducts an experiment based on the n recommended values, the evaluation device 30 may acquire the results of the experiment and generate information regarding the evaluation based on the acquired results of the experiment.

[0026] Further, the information processing apparatus 20 acquires information regarding evaluation from the evaluation apparatus 30, and recalculates n recommended values based on the acquired information regarding evaluation. In other words, the information processing apparatus 20 outputs n recommended values to be used in the next experiment. In this way, the information processing system 10 alternately repeats the output of n recommended values and the experiment, and after reaching a predetermined end condition, calculates and outputs optimal n set values. In the present embodiment, the optimal n set values may sometimes be referred to as n optimal values.

[0027] In the example of FIG. 1, the information processing apparatus 20 outputs n set values to the outside and acquires information regarding the results of the experiment from the outside. However, the information processing apparatus 20 may include a simulator that executes a simulation. That is, the information processing apparatus 20 may also serve as a simulator.

[0028] Here, consider the set X ⊂ R D of the above evaluation function group {f(x)}.

[0029] The evaluation function group {f(x)} is represented as in Equation (1).

Equation

[0030] m is an integer of 1 or more. x1,..., x n each of which is a parameter. f1(x1,..., x n ) is the first evaluation function that calculates the first evaluation value among one or more evaluation values. f2(x1,..., x n ) is the second evaluation function that calculates the second evaluation value among one or more evaluation values. f m (x1,..., x n ) is the m-th evaluation function that calculates the m-th evaluation value among one or more evaluation values.

[0031] The optimization problem includes only the first evaluation function {f1(x1,..., x n )} in the evaluation function group {f(x)}. The optimization problem is f1(x1,..., xn ) to minimize or maximize the values of n parameters (x1, …, x n ).

[0032] In a constrained optimization problem, the set of evaluation functions {f(x)} includes two or more evaluation functions. In a constrained optimization problem, under the condition that f2(x1, …, x n ) is greater than or equal to the second lower threshold and less than or equal to the second upper threshold, f3(x1, …, x n ) is greater than or equal to the third lower threshold and less than or equal to the third upper threshold, …, f m (x1, …, x n ) is greater than or equal to the m-th lower threshold and less than or equal to the m-th upper threshold, the problem is to calculate the values of n parameters (x1, …, x n ) to minimize or maximize f1(x1, …, x n ).

[0033] In a multi-objective optimization problem, the set of evaluation functions {f(x)} includes two or more evaluation functions. A multi-objective optimization problem is a problem of calculating the Pareto solutions of f1(x1, …, x n ), f2(x1, …, x n ), …, f m (x1, …, x n ).

[0034] The evaluation device 30 calculates one or more evaluation values represented by such a set of evaluation functions {f(x)}. The information processing device 20 solves an optimization problem, a constrained optimization problem, or a multi-objective optimization problem represented using such a set of evaluation functions {f(x)}.

[0035] The information processing device 20 includes a storage unit 40 and a processing unit 50.

[0036] The storage unit 40 is composed of any commonly used storage medium such as a flash memory, a memory card, a RAM (Random Access Memory), an HDD (Hard Disk Drive), and an optical disk.

[0037] The storage unit 40 stores data used for the processing of the information processing apparatus 20. The storage unit 40 stores at least the setting range information, the dataset information, and the change direction information. The storage unit 40 may store information other than these. For example, the storage unit 40 may store the processing results of each component of the information processing apparatus 20 and the like.

[0038] Prior to the optimization process, for example, the change direction information is input by the user to the processing unit 50. In this case, prior to the optimization process, the processing unit 50 causes the input change direction information to be stored in the storage unit 40. Also, during the optimization process, the processing unit 50 may update the change direction information. In this case, the processing unit 50 rewrites the change direction information stored in the storage unit 40 with the updated change direction information. Further, during the optimization process, the change direction information may be input by the user to the processing unit 50. In this case, the processing unit 50 rewrites the change direction information stored in the storage unit 40 with the change direction information input by the user.

[0039] Also, during the optimization process, the processing unit 50 repeatedly acquires one or more evaluation values from the evaluation device 30. Also, during the optimization process, the processing unit 50 repeatedly outputs n recommended values. Then, at the end of the optimization process, the processing unit 50 outputs n optimal values. Note that further details of the processing unit 50 will be described with reference to FIG. 5.

[0040] Note that the components shown in FIG. 1 are components for performing the process of calculating and outputting n recommended values and n optimal values, and other components are omitted. Also, each component may be subdivided or combined. For example, the storage unit 40 may be divided into a plurality of storage devices (for example, a plurality of storage media) according to the files to be stored and the like. Also, the components other than the storage unit 40 may be regarded as one. Further, the processing results of each component may be sent to the component where the next process is performed, or may be stored in the storage unit 40. In the latter case, the component where the next process is performed accesses the storage unit 40 to acquire the processing result.

[0041] FIG. 2 is a diagram showing an example of setting range information.

[0042] The setting range information includes the setting range of each of the n parameters. The setting range represents the range within which the setting value set for the corresponding parameter can be taken. In other words, the setting range means the range of values that can be set for the corresponding parameter, and it becomes the search range when searching for the recommended value. The setting range information is input, for example, by a user or the like and stored in the storage unit 40 in advance.

[0043] As shown in FIG. 2, for example, it is assumed that there are four parameters from x1 to x4. The setting range information represents the setting range for each of these parameters. The setting range is a range from a lower limit value or more to an upper limit value or less. For example, in the example of FIG. 2, the setting range of x1 is a range from 10 or more to 100 or less.

[0044] Note that the method of representing the setting range is not particularly limited. For example, the setting range may be represented using an inequality. For example, when A is a matrix, B is a vector, and W is a parameter, the setting range may be expressed as "satisfying AW + B < 0". Also, when A is a vector, R is a real number, and W is a parameter, the setting range may be expressed as "satisfying |W - A| < R". Here, |W - A| represents the magnitude of the vector "W - A". In addition, the setting range may be represented using an inequality including various forms of parameters.

[0045] FIG. 3 is a diagram showing an example of dataset information.

[0046] The dataset information includes one or more datasets. Each of the one or more datasets includes n setting values and one or more evaluation values. Each of the one or more evaluation values represents the evaluation of an experiment executed using the n setting values included in the same dataset. That is, one dataset associates the n recommended values (n setting values) output by the information processing device 20 with one or more evaluation values obtained by evaluating the experiment executed using the n recommended values (n setting values).

[0047] In the example of FIG. 3, a data set including four setting values corresponding to four parameters from x1 to x4 and two evaluation values corresponding to f1 and f2 is shown. In the example of FIG. 3, one row of the table represents one data set.

[0048] Each time the information processing apparatus 20 acquires one or more evaluation values from the evaluation apparatus 30, a new data set including the acquired one or more evaluation values is added to the data set information. The new data set includes n recommended values used in the experiment on which the included one or more evaluation values are based as n setting values.

[0049] Note that the data set information may include a data set including test data generated in advance by a user, for example. That is, the data set information may include a data set including n setting values that have not actually been used in the experiment and one or more evaluation values that have not actually been generated by conducting the experiment.

[0050] Further, when the evaluation apparatus 30 generates a plurality of evaluation values, the data set may include a comprehensive evaluation value calculated based on the plurality of evaluation values. Also, the data set may include output data for which a value can be calculated as an evaluation value. The output data is, for example, data output from the evaluation apparatus 30. The output data is composed of a plurality of items like the parameters. Each of the plurality of items represents an individual value. For example, the output data may be detection data of various sensors used in the experiment, or physical characteristic values and measurement values of experimental results or simulation results.

[0051] FIG. 4 is a diagram showing an example of change direction information.

[0052] The change direction information indicates the change direction of the target evaluation value among one or more evaluation values with respect to the change of the target parameter among each combination of each of the n parameters and each of one or more evaluation values. The change direction information may indicate the change direction of the target evaluation value among one or more evaluation values with respect to the change of the target parameter for each combination of each of the n parameters and each of one or more evaluation values.

[0053] The change direction information may indicate that the change direction of the target evaluation value with respect to the change of the target parameter is uncorrelated for any one or more combinations among the combinations of each of the n parameters and each of one or more evaluation values. Also, the change direction information may indicate that the change direction is unclear for any one or more combinations among the combinations of each of the n parameters and each of one or more evaluation values.

[0054] In this embodiment, as shown in FIG. 4, the change direction information includes monotonicity information for each combination of each of the n parameters and each of one or more evaluation values.

[0055] The monotonicity information indicates any one of monotonic increase, monotonic decrease, non-monotonicity, unclear direction, or unclear monotonicity.

[0056] Monotonic increase represents that the target evaluation value monotonically increases in response to the increase of the target parameter. Monotonic decrease represents that the target evaluation value monotonically decreases in response to the increase of the target parameter. Non-monotonicity represents that neither monotonic increase nor monotonic decrease occurs. Unclear direction represents that it monotonically increases or decreases, but it is unclear whether it is monotonic increase or monotonic decrease. Unclear monotonicity represents that it is unclear whether it is monotonic increase, monotonic decrease, or non-monotonicity. Note that unclear monotonicity may also represent that it is unclear whether it is monotonic increase, monotonic decrease, non-monotonicity, or unclear direction.

[0057] For example, when the target evaluation value (f(x1, …, x i , …, x n )) is monotonically increasing with respect to the target parameter (x i ) in the i-th dimension, it means that for any value within the range that each of the parameters in all dimensions other than x i can take, when x i < x i ´, f(x1, …, x i , …, x n ) ≤ f(x1, …, x i ´, …, x n ) holds. Also, when the target evaluation value (f(x1, …, x i , …, x n )) is monotonically decreasing with respect to the target parameter (x i ) in the i-th dimension, it means that for any value within the range that each of the parameters in all dimensions other than x i can take, when x i < x i ´, f(x1, …, x i , …, x n ) ≥ f(x1, …, x i ´, …, x n ) holds.

[0058] Also, non-monotonicity means that the target evaluation value (f(x1, …, x i , …, x n )) is neither monotonically increasing nor monotonically decreasing with respect to the target parameter (x i ) in the i-th dimension.

[0059] Note that when it is said to be monotonically increasing, it does not mean for any value within the range that each of the parameters in all dimensions other than x i can take, but rather for a part within the range of values that can be taken, for example, for most of the values within the range. When x i < x i ´, it may mean that f(x1, …, x i , …, x n ) ≤ f(x1, …, x i ´, …, x n ) holds. Also, when it is said to be monotonically decreasing, when x iNot any arbitrary value within the range of possible values of each parameter in all other dimensions, but rather a part within the range of possible values, for example, for the majority of values within the range, x i <x i ´, if so, f(x1,…,x i ,…x n )≧f(x1,…,x i ´,…x n ) may hold. That is, monotonicity may hold in a broader sense than the monotonicity used in mathematics. Also, in this case, non-monotonicity means that neither broad monotonic increase nor broad monotonic decrease holds.

[0060] For example, the change direction information shown in FIG. 4 shows the monotonicity information for each combination when using four-dimensional parameters of x1, x2, x3, and x4 and two evaluation values of f1 and f2.

[0061] For example, the monotonicity information in the second row of the table in FIG. 4 shows that f1 has monotonic increase with respect to x1. For example, the monotonicity information in the third row of the table in FIG. 4 shows that f1 has monotonic decrease with respect to x2. The monotonicity information in the fourth row of the table in FIG. 4 shows that f1 has monotonicity with respect to x3, but the direction is unclear. The monotonicity information in the fifth row of the table in FIG. 4 shows that f1 has non-monotonicity with respect to x4.

[0062] Also, the monotonicity information in the sixth row of the table in FIG. 4 shows that the monotonicity of f2 with respect to x1 is unclear. For example, the monotonicity information in the seventh row of the table in FIG. 4 shows that the direction of f2 with respect to x2 is unclear. The monotonicity information in the eighth row of the table in FIG. 4 shows that f2 has monotonic increase with respect to x3. The monotonicity information in the ninth row of the table in FIG. 4 shows that f2 has non-monotonicity with respect to x4.

[0063] FIG. 5 is a diagram showing the configuration of the processing unit 50 together with the storage unit 40.

[0064] The processing unit 50 includes a change direction information input unit 62, an acquisition unit 64, an end determination unit 66, a model generation unit 68, a monotonicity update unit 70, a recommendation unit 72, and an output unit 74.

[0065] The change direction information input unit 62 acquires change direction information from the outside. For example, the change direction information input unit 62 acquires the change direction information input by the user. The change direction information input unit 62 stores the acquired change direction information in the storage unit 40. The change direction information input unit 62 may receive, for example, data in the format shown in FIG. 4, or may convert the received data in a format different from the format shown in FIG. 4 into the data in the format shown in FIG. 4.

[0066] The change direction information input unit 62 may acquire the change direction information prior to the optimization process and may not acquire the change direction information thereafter. Further, after the change direction information input unit 62 acquires the change direction information prior to the optimization process, during the optimization process, it may receive and rewrite a part of the monotonicity information included in the change direction information.

[0067] The acquisition unit 64 acquires the input of information necessary for the processing of the information processing apparatus 20 from the outside. For example, the acquisition unit 64 acquires one or more evaluation values based on n recommended values (n set values) from the evaluation apparatus 30. When the acquisition unit 64 acquires one or more evaluation values from the evaluation apparatus 30, the acquisition unit 64 generates a new data set including the acquired one or more evaluation values. In this case, the new data set includes the n recommended values used in the experiment that is the basis of the included one or more evaluation values as n set values. Then, the acquisition unit 64 adds the new data set to one or more data sets included in the data set information stored in the storage unit 40

[0068] The acquisition unit 64 may further acquire, prior to the optimization process, setting range information, information indicating an end condition for determining whether to output n optimal values, and the like. When acquiring the setting range information, the acquisition unit 64 causes the storage unit 40 to store the acquired setting range information. Further, when acquiring the information indicating the end condition, the acquisition unit 64 provides the acquired information indicating the end condition to the end determination unit 66.

[0069] The end determination unit 66 determines whether a predetermined end condition has been reached. The end condition is, for example, the number of executions of the experiment or the elapsed time being a predetermined set time or the like.

[0070] When the end condition is not reached, the end determination unit 66 gives an output instruction to, for example, the recommendation unit 72 to continue causing the recommendation unit 72 to output n recommendation values. When the end condition is reached, the end determination unit 66 gives an output stop instruction to, for example, the recommendation unit 72, notifies the output unit 74 that the end condition has been reached, and causes the output unit 74 to select n optimal values (optimal n set values) based on a plurality of sets of n recommendation values (a plurality of sets of n set values) generated so far and output the selected n optimal values.

[0071] The model generation unit 68 generates an estimation model based on the change direction information and part or all of one or more data sets included in the data set information. The model generation unit 68 generates an estimation model every time a new data set is added to one or more data sets included in the data set information by the acquisition unit 64 during the optimization process.

[0072] The estimation model is a model that calculates the estimated value and estimated standard deviation of each of one or more evaluation values based on n parameters. In the present embodiment, the estimation model is represented by a mathematical formula including n parameters.

[0073] In the present embodiment, the model generation unit 68 generates an estimation model using Gaussian process regression considering monotonicity.

[0074] For example, when outputting the n recommended values for the Nth time, the dataset information includes the (N - 1) datasets generated so far. In this case, the model generation unit 68 generates an estimation model using the dataset shown in Equation (2). N is an integer of 2 or more. k is an integer of 1 or more.

Number

[0075] x (k) is an array including the n set values generated in the k-th iteration process. y (k) is an array including one or more evaluation values based on the n recommended values {x (k)} generated in the k-th iteration process.

[0076] Furthermore, when the model generation unit 68 generates an estimation model using Gaussian process regression considering monotonicity, it generates a dataset of M pseudo partial derivative values and generates an estimation model based on the dataset of M pseudo partial derivative values. Note that M is an integer of 1 or more.

[0077] For example, when outputting the n set values for the Nth time, the model generation unit 68 generates M pseudo datasets shown in Equation (3). j is an integer of 1 or more.

Number

[0078] x´ (j) is a value having the same number of dimensions as the number of dimensions of the n parameters, and is an array including the values of the n pseudo parameters included in the j-th pseudo dataset among the M pseudo datasets.

[0079] y d ´ (j)is an array including an estimated partial differential value of any one of one or more evaluation values corresponding to the parameter of the d-th dimension included in the j-th pseudo data set among the M pseudo data sets. d is an integer of 1 or more and n or less. For example, when the evaluation value corresponding to the d-dimensional parameter monotonically increases, the model generation unit 68 sets y d ´ (j) = 1, and when the evaluation value corresponding to the d-dimensional parameter monotonically decreases, y d ´ (j) = -1.

[0080] The model generation unit 68 may generate M pseudo data sets using all of the (N - 1) x (k) . Further, the model generation unit 68 may select a part of all of the (N - 1) x (k) and generate M pseudo data sets using the selected part. Further, the model generation unit 68 may generate M pseudo data sets using a part randomly selected from the parameters within the search range. Further, the model generation unit 68 may generate grid points based on the search range, select all or a part of the generated grid points, and generate M pseudo data sets using the selected part. Further, when the model generation unit 68 selects a part from all of the (N - 1) x (k) , it may randomly select a parameter set, or select it so that the D-optimality criterion of the selected part of the parameter sets becomes larger.

[0081] Also, after once generating the estimation model, the model generation unit 68 calculates the estimated partial differential value with a candidate of the pseudo parameter using the generated estimation model. Subsequently, for a parameter in which the monotonicity information indicates monotonic increase, if the estimated partial differential value is positive, the model generation unit 68 determines that the estimation is correct. Also, for a parameter in which the monotonicity information indicates monotonic decrease, if the estimated partial differential value is negative, the model generation unit 68 determines that the estimation is correct. Then, the model generation unit 68 may add all or part of the candidates of the pseudo parameters that are not correctly estimated to the pseudo parameters. Further, the model generation unit 68 may repeat such generation, determination, and addition of the pseudo parameters of the estimation model.

[0082] Note that the model generation unit 68 is not limited to the above method, and may generate the M pseudo data sets by any method.

[0083] For example, the model generation unit 68 generates the model shown in Equation (4) as a model for calculating the estimated value of the evaluation value (f).

Equation

[0084] Also, the model generation unit 68 generates the model shown in Equation (5) as a model for calculating the estimated standard deviation of the evaluation value (f).

Equation

[0085] Equations (4) and (5) are the equations shown in Non-Patent Document 1. The constants and variables shown in Equations (4) and (5) are shown in Non-Patent Document 1 and are as follows. μ0 f (x) represents an arbitrary function. X is a matrix arranged with x (k) and X´ is a matrix arranged with x´ (j) and y is a vector arranged with y (k) Using an arbitrary kernel function k, K(X,X) has the (i,j) component as k(x(i) , x (j) ) is described as a matrix. Similarly, for example, K(x, X) is a matrix whose (1, j) component is k(x, x (j) ). σ is an arbitrary real number, Σ ~ is an arbitrary diagonal matrix, μ ~ represents an arbitrary vector. Note that Σ~ is synonymous with the symbol with ~ attached to Σ, and μ ~ is synonymous with the symbol with ~ attached to μ. The average of y and μ ~ is represented as m.

[0086] Note that the model generation unit 68 may generate an estimation model by a method other than Gaussian process regression considering monotonicity.

[0087] When there is one evaluation value, that is, when y (k) is one-dimensional, the model generation unit 68 calculates, for example, the sum of squared errors between the evaluation values included in one or more data sets and the estimated values. Then, the model generation unit 68 calculates the estimated values of the evaluation values so that the calculated sum of squared errors is minimized for the estimation model {μ n f (x1,..., x n )} and may adjust the parameters included therein by a predetermined regression method.

[0088] As the predetermined regression method, the model generation unit 68 may use, for example, linear regression, Lasso regression, elastic net regression, random forest regression considering monotonicity, neural network considering monotonicity, etc. Also, the model generation unit 68 calculates an estimated standard deviation based on the regression result of the estimation model that calculates the estimated values of the evaluation values for the estimation model {σ n f (x1,..., x n )} and may generate it. Also, for example, the model generation unit 68 may use the confidence interval of Bayesian linear regression, the variance of the outputs of a plurality of learned decision trees, or the variance of the outputs when dropout is probabilistically performed a plurality of times in a neural network as the estimation model for calculating the estimated standard deviation.

[0089] When there are multiple evaluation values, that is, when y(k) When it is two - dimensional or higher, the model generation unit 68 generates an estimation model, for example, for each of a plurality of evaluation values, by a method used when there is one evaluation value. That is, the model generation unit 68 generates f(x) = {f1(x1,…,x n ), f2(x1,…,x n ),…, f m (x1,…,x n )}, and calculates an estimation model {μ n n 1 (x1,…,x (x1,…,x n )} that calculates an estimated value corresponding to f1(x1,…,x n ) and an estimation model {σ n 1 (x1,…,x n )} that calculates an estimated standard deviation, and calculates an estimation model {μ n n 2 (x1,…,x (x1,…,x n )} that calculates an estimated value corresponding to f2(x1,…,x n ) and an estimation model {σ n 2 (x1,…,x n )} that calculates an estimated standard deviation, and repeats this process m times. Also, when there are a plurality of evaluation values, the model generation unit 68 may calculate the estimated values and estimated standard deviations of the plurality of evaluation values simultaneously for each of the plurality of evaluation values using a multi - output regression method.

[0090] The monotonicity update unit 70 updates the change direction information stored in the storage unit 40 as necessary. The monotonicity update unit 70 may output the updated change direction information to the outside via the output unit 74.

[0091] For example, the monotonicity update unit 70 updates the change direction information when the change direction information includes monotonicity information indicating unclear monotonicity or unclear direction.

[0092] Further, the monotonicity update unit 70 may calculate an estimation error, and update the change direction information when the calculated estimation error is greater than a preset threshold value. The estimation error represents the error between the estimated value of one or more evaluation values calculated based on the estimation model generated by the model generation unit 68 and one or more evaluation values obtained by evaluating an experiment performed using n recommended values generated using the estimation model.

[0093] Further, the monotonicity update unit 70 acquires, from the model generation unit 68, the estimated partial differential value of each of one or more evaluation values with respect to a parameter corresponding to the monotonicity information indicating monotonic increase or monotonic decrease in the change direction information. The estimated partial differential value is calculated by the model generation unit 68 based on the generated estimation model. Then, when the monotonicity information indicating monotonic increase or monotonic decrease in the change direction information is different from the change direction specified by the estimated partial differential value for the corresponding parameter, the monotonicity update unit 70 may update the change direction information. For example, when the estimated partial differential value of the corresponding parameter is negative for the monotonicity information indicating monotonic increase, the monotonicity update unit 70 may update the change direction information. Also, for example, when the estimated partial differential value of the corresponding parameter is positive for the monotonicity information indicating monotonic decrease, the monotonicity update unit 70 may update the change direction information.

[0094] For example, when the change direction information includes monotonicity information indicating unclear monotonicity or unclear direction, the monotonicity update unit 70 updates the change direction information as follows.

[0095] The monotonicity update unit 70 generates all combination patterns in which all of the monotonicity information indicating unclear monotonicity in the change direction information is replaced with monotonic increasing, monotonic decreasing, or non-monotonic, and all of the monotonicity information indicating unclear direction is replaced with monotonic increasing or monotonic decreasing, or a plurality of hypothetical change direction information corresponding to some of all the combination patterns. Subsequently, for each of the plurality of hypothetical change direction information, the monotonicity update unit 70 causes the model generation unit 68 to generate an estimation model, and specifies an estimation model in which the estimation error is minimized, the length scale in the learned Gaussian process regression is maximized, or the change direction specified by the estimated partial differential value with respect to the parameter corresponding to the monotonicity information matches. Then, the monotonicity update unit 70 updates the change direction information to the content of the hypothetical change direction information that is the basis for the generation of the specified estimation model.

[0096] More specifically, for example, when the change direction information includes only one piece of monotonicity information indicating unclear monotonicity, the monotonicity update unit 70 generates three patterns of hypothetical change direction information in which the monotonicity information indicating unclear monotonicity is replaced with monotonic increasing, monotonic decreasing, or non-monotonic. Also, when the change direction information includes two or more pieces of monotonicity information indicating unclear monotonicity, the monotonicity update unit 70 generates all combination patterns in which each of the two or more pieces of monotonicity information indicating unclear monotonicity is replaced with monotonic increasing, monotonic decreasing, or non-monotonic, or a plurality of hypothetical change direction information corresponding to some of all the combination patterns.

[0097] Also, for example, when the change direction information includes only one piece of monotonicity information indicating unclear direction, the monotonicity update unit 70 generates two patterns of hypothetical change direction information in which the monotonicity information indicating unclear direction is replaced with monotonic increasing or monotonic decreasing. Also, when the change direction information includes two or more pieces of monotonicity information indicating unclear direction, the monotonicity update unit 70 generates all combination patterns in which each of the two or more pieces of monotonicity information indicating unclear direction is replaced with monotonic increasing or monotonic decreasing, or a plurality of hypothetical change direction information corresponding to some of all the combination patterns.

[0098] Further, when the monotonicity update unit 70 includes two or more pieces of monotonicity information indicating unclear monotonicity and monotonicity information indicating unclear direction in the change direction information, the monotonicity update unit 70 replaces the monotonicity information indicating unclear monotonicity with monotonic increasing, monotonic decreasing, or non-monotonic, and replaces the monotonicity information indicating unclear direction with monotonic increasing or monotonic decreasing, and generates all combination patterns, or a plurality of hypothetical change direction information corresponding to some of all combination patterns.

[0099] Further, when the monotonicity update unit 70 does not include monotonicity information indicating unclear monotonicity or unclear direction in the change direction information, the monotonicity update unit 70 selects one or more pieces of monotonicity information indicating monotonic increasing or monotonic decreasing. In this case, for example, the monotonicity update unit 70 may select monotonicity information indicating monotonic increasing or monotonic decreasing that is different from the change direction specified by the estimated partial derivative value. The monotonicity update unit 70 generates all combination patterns in which the monotonic increasing and monotonic decreasing in each of the selected one or more pieces of monotonicity information are interchanged, or one or more pieces of hypothetical change direction information corresponding to some of all combination patterns. The monotonicity update unit 70 specifies an estimated model generated based on each of the one or more pieces of hypothetical change direction information and an estimated model generated based on the original change direction information, and specifies an estimated model with the minimum estimation error, the maximum length scale in the learned Gaussian process regression, or the change direction specified by the estimated partial derivative value for the parameter corresponding to the monotonicity information being consistent. Then, the monotonicity update unit 70 may update the change direction information to the content of the change direction information that is the basis for generating the specified estimated model.

[0100] The recommendation unit 72 calculates and outputs n recommended values recommended as n set values to be used in the experiment based on the estimated model. The recommendation unit 72 calculates and outputs n recommended values based on the generated estimated model every time the estimated model is generated by the model generation unit 68. The recommendation unit 72 continues to calculate and output n recommended values every time the estimated model is generated until it receives an output stop instruction from the end determination unit 66, that is, until a preset end condition is reached.

[0101] Such a recommendation unit 72 can determine and output n set values to be used in the next experiment from within the set range. The recommendation unit 72 determines the next n recommended values using black box optimization. In the present embodiment, the recommendation unit 72 determines the next n recommended values using Bayesian optimization. The recommendation unit 72 may determine the next n recommended values using a genetic algorithm, an evolutionary strategy, or CMA-ES.

[0102] For example, the recommendation unit 72 may calculate an acquisition function based on an estimation model that calculates an estimated value and an estimated standard deviation of an evaluation value, and use, as the next n recommended values to be output, the parameters at which the acquisition function is maximized. As the acquisition function, the recommendation unit 72 may use, for example, PI (improvement probability) or EI (expected improvement measure). As the acquisition function, the recommendation unit 72 may use UCB (upper confidence bound), Thompson Sampling (TS), Entropy Search (ES), and Mutual Information (MI).

[0103] For example, in the case of UCB, the recommendation unit 72 calculates an acquisition function αn(z) as shown in Equation (6) using β, which is an arbitrary constant. n using it.

Equation

[0104] Also, for example, in the case of EI, the recommendation unit 72 calculates EI(x), which is an acquisition function, as shown in Equation (7). n as shown in Equation (7).

Equation

[0105] Z in Equation (7) n is expressed as shown in Equation (8).

Equation

[0106] In equation (7), φ is the probability density function of the standard normal distribution. n + is the minimum parameter of the current evaluation value.

[0107] The recommendation unit 72 may maximize the acquisition function using any optimization method. For example, the recommendation unit 72 may maximize the acquisition function using a full search, a random search, a grid search, a gradient method, an L-BFGS, a DIRECT, a CMA-ES, or a multi-start local improvement method.

[0108] In the case of a constrained optimization problem, the recommendation unit 72 calculates an evaluation function (f1(x1, . . . , x n ) is the minimum or maximum, and the other evaluation functions {f2(x1,…,x n ),…,f m (x1,…,x n )} is greater than or equal to the lower threshold and less than or equal to the upper threshold for n parameters (x1,…,x n ) (n recommended values). In this case, the recommendation unit 72 detects n estimated values ​​that provide the best objective function value within a range in which the constraint function value satisfies the constraint. For example, the recommendation unit 72 may search for and output new n estimated values ​​that are estimated to have a smaller objective function value than the previous n estimated values ​​within a set range. For example, the recommendation unit 72 can find a value that provides a smaller evaluation value of the objective function using various optimization methods such as full search, random search, grid search, gradient method, L-BFGS, DIRECT, CMA-ES, and multi-start local improvement method.

[0109] The recommendation unit 72 may also determine n recommendation values ​​to be output based on the product of an acquisition function based on the estimated value of the evaluation value and the estimated standard deviation and the constraint satisfaction rate. For example, the estimation value of the constraint function is n c (x), the estimated standard deviation is σ n c (x), the recommendation unit 72 calculates a constraint fulfillment rate PF n(x) is calculated as in equation (9).

number

[0110] In formula (9), Φ represents the cumulative distribution function of the standard normal distribution. For example, the recommendation unit 72 calculates EI n (x) and PF n (x) is multiplied by the acquisition function {EIC n (x)=EI n (x) x PF n Then, the recommendation unit 72 calculates the point x where this acquisition function is maximized. ~ n is calculated using formula (10). ~ is synonymous with the symbol 〜 after x in formula (10).

number

[0111] Furthermore, the recommendation unit 72 may fix one or more setting values ​​among the n recommended values ​​as the best setting values ​​at the current time and determine the n recommended values ​​to be output next. That is, the recommendation unit 72 sets one or more setting values ​​selected from the n recommended values ​​as the best values ​​at the current time, and optimizes the remaining unselected recommended values ​​by the above-mentioned method.

[0112] For example, the recommendation unit 72 selects all or some of the parameters whose monotonicity information is not unknown and fixes them to the best value. Alternatively, the recommendation unit 72 may randomly select some of the parameters whose monotonicity information is not unknown and fix them to the best value. In this way, the recommendation unit 72 can optimize the setting values ​​corresponding to the parameters whose monotonicity information is incorrect or whose monotonicity information is not known or whose monotonicity information is not known, i.e., the parameters that are the cause of the large estimated standard deviation, and efficiently determine the n recommended values ​​to be output next.

[0113] Note that the recommendation unit 72 determines the following plurality of output values on the premise that at least one dataset is included in the dataset information. However, for example, when a user conducts an experiment for the first time, etc., there is a situation where the dataset information does not include any dataset. Even in such a situation, the user may wish to conduct an experiment using the n recommended values output from the information processing apparatus 20. In such a case, the recommendation unit 72 may select and output the initial n recommended values from among a plurality of predetermined values. Further, the recommendation unit 72 may output the initial n recommended values determined according to a predetermined rule. The predetermined rule is, for example, a rule for determining a plurality of values using any one of a random number, a Latin square, and a Sobol sequence.

[0114] When it is determined by the end determination unit 66 that the end condition has been reached, the output unit 74 selects n optimal values (optimal n set values) based on the n set values output so far included in the dataset information stored in the storage unit 40.

[0115] For example, the output unit 74 selects, as the n optimal values, the n set values of the set in which the evaluation value is the minimum or maximum among the plurality of sets of n set values output so far included in the dataset information stored in the storage unit 40. Further, the output unit 74 may select, as the set of n optimal values, the set of non-dominated solutions among the plurality of sets of n set values output so far included in the dataset information stored in the storage unit 40. Further, the output unit 74 may select, as the n optimal values, the n set values in which the evaluation value corresponding to the constraint function value is equal to or greater than the lower threshold value and equal to or less than the upper threshold value, and the evaluation value corresponding to the objective function value is the minimum or maximum among the plurality of sets of n set values output so far included in the dataset information stored in the storage unit 40. Then, the output unit 74 outputs the selected n optimal values.

[0116] Further, the output unit 74 outputs the processing results of each component. For example, the output unit 74 may output the n recommended values to be output. Also, the output unit 74 may receive an instruction via the acquisition unit 64 and output the data stored in the storage unit 40 such as the dataset information.

[0117] The output format of the output unit 74 is not particularly limited, and for example, it may be a table or an image. For example, the output unit 74 may generate and output a graph based on data such as the dataset information.

[0118] Note that when the information processing apparatus 20 includes a simulator, the simulator sets the n recommended values to the model parameters, executes the simulation, and calculates one or more evaluation values based on the simulation results. In this case, the calculation formula for the one or more evaluation values is predetermined.

[0119] FIG. 6 is a flowchart showing the processing flow of the information processing apparatus 20 according to the embodiment. The information processing apparatus 20 executes the processing in the flow shown in FIG. 6.

[0120] First, in S101, the recommendation unit 72 determines the initial n recommended values (n set values). For example, the recommendation unit 72 may select and output the initial n recommended values from among a plurality of predetermined values. Also, the recommendation unit 72 may output the initial n recommended values determined according to a predetermined rule. Then, the recommendation unit 72 outputs the determined initial n recommended values.

[0121] When n recommended values are output, for example, the user conducts an experiment using the n output recommended values. The evaluation device 30 acquires the result of the experiment and generates one or more evaluation values representing the evaluation of the experiment with the n recommended values. Alternatively, a simulator executed by the evaluation device 30 or the information processing device 20 executes a simulation based on the n output recommended values. The simulator generates one or more evaluation values representing the evaluation of the simulation based on the result of the simulation. Note that the evaluation device 30 may acquire the result of the simulation from the simulator and generate one or more evaluation values representing the evaluation of the simulation instead of the simulator.

[0122] Subsequently, in S102, the acquisition unit 64 acquires one or more evaluation values generated by the evaluation device 30 or the simulator. The acquisition unit 64 generates a new data set including the acquired one or more evaluation values, and adds the generated new data set to the data set information stored in the storage unit 40. In this case, the acquisition unit 64 includes the n recommended values used in the experiment of the acquired one or more evaluation values as n set values in the new data set.

[0123] Subsequently, in S103, the end determination unit 66 determines whether a predetermined end condition has been reached. When the end determination unit 66 determines that the end condition has been reached (Yes in S103), the process proceeds to S111. When the end determination unit 66 determines that the end condition has not been reached (No in S103), the process proceeds to S104.

[0124] In S104, the monotonicity update unit 70 determines whether the change direction information stored in the storage unit 40 includes monotonicity information indicating unknown monotonicity or monotonicity information indicating unknown direction. If it is included (Yes in S104), the monotonicity update unit 70 advances the process to S107. If it is not included (No in S104), the monotonicity update unit 70 advances the process to S105.

[0125] In S105, based on the change direction information and some or all of the one or more data sets included in the data set information, the model generation unit 68 generates an estimation model that calculates the estimated value and the estimated standard deviation of each of one or more evaluation values based on n parameters.

[0126] Subsequently, in S106, the monotonicity update unit 70 calculates the estimation error of the estimation model and determines whether the calculated estimation error is greater than a predetermined threshold. If the estimation error is greater than the threshold (Yes in S106), the monotonicity update unit 70 proceeds with the process to S107. If the estimation error is not greater than the threshold (No in S106), the monotonicity update unit 70 proceeds with the process to S110. Note that in S106, the monotonicity update unit 70 may determine whether the monotonicity information indicating monotonic increase or monotonic decrease in the change direction information is different from the change direction specified by the estimated partial derivative value for the corresponding parameter. If they are different, the monotonicity update unit 70 proceeds with the process to S107; if they are not different, the monotonicity update unit 70 proceeds with the process to S110.

[0127] In S107, based on the change direction information stored in the storage unit 40, the monotonicity update unit 70 generates a plurality of assumed change direction information. For example, the monotonicity update unit 70 replaces all of the monotonicity information indicating unclear monotonicity in the change direction information with monotonic increase, monotonic decrease, or non-monotonicity, and replaces all of the monotonicity information indicating unclear direction with monotonic increase or monotonic decrease, and generates all combination patterns, or a plurality of assumed change direction information corresponding to some of the all combination patterns. Or, the monotonicity update unit 70 selects one or more monotonicity information indicating monotonic increase or monotonic decrease in the change direction information, and generates all combination patterns in which each of the selected monotonicity information is replaced with a different one of monotonic increase or monotonic decrease, or a plurality of assumed change direction information corresponding to some of the all combination patterns.

[0128] Subsequently, in S108, the monotonicity update unit 70 and the model generation unit 68 generate estimation models for each of the plurality of assumed change direction information.

[0129] Subsequently, in S109, the monotonicity update unit 70 selects one of the estimation models for each of the plurality of assumed change direction information as the correct estimation model. For example, the monotonicity update unit 70 specifies, as the correct estimation model, an estimation model with the minimum estimation error, the maximum length scale in the learned Gaussian process regression, or the one whose change direction specified by the estimated partial derivative value for the parameter corresponding to the monotonicity information matches. Then, the monotonicity update unit 70 updates the change direction information stored in the storage unit 40 to the content of the assumed change direction information used for generating the correct estimation model. When the processing of S109 is completed, the monotonicity update unit 70 proceeds to S110.

[0130] In S110, the recommendation unit 72 calculates and outputs n recommended values recommended as the n set values to be used in the experiment based on the estimation model. For example, the recommendation unit 72 may use the product of the constraint satisfaction probability and the expected improvement amount as the acquisition function, and use the value that maximizes the acquisition function as the next n recommended values to be output. Note that when the change direction information is updated in S109, the recommendation unit 72 calculates n recommended values based on the estimation model selected as correct. When the recommendation unit 72 outputs the n recommended values, the process returns to S102.

[0131] Then, in S111, the output unit 74 selects and outputs n optimal values from the plurality of sets of n set values output so far included in the dataset information stored in the storage unit 40. When the processing of S111 is completed, the information processing apparatus 20 ends this flow.

[0132] As described above, the information processing apparatus 20 according to the present embodiment generates an estimation model based on one or more data sets each including n set values and one or more evaluation values, and the change direction information, and calculates n recommended values based on the estimation model. Further, the information processing apparatus 20 according to the present embodiment acquires one or more evaluation values representing the evaluation of an experiment or simulation using the calculated n recommended values, adds a new data set to the one or more data sets, and generates an estimation model based on the one or more data sets to which the new data set is added and the change direction information, and calculates the next n recommended values based on the estimation model, and repeats the above process. Then, the information processing apparatus 20 according to the present embodiment repeats the process until a predetermined end condition is reached, and then outputs n optimum values based on the one or more data sets. Thereby, the information processing apparatus 20 according to the present embodiment can output n set values with good evaluation values as n optimum values.

[0133] Furthermore, since the information processing apparatus 20 according to the present embodiment calculates n estimated values using an estimation model considering monotonicity, the n optimum values can be output with a small number of repetitions. In addition, by using the change direction information, the information processing apparatus 20 according to the present embodiment can confirm whether the optimization is correctly performed, and if the optimization is not correctly performed, update the change direction information to perform the correct optimization.

[0134] As described above, according to the information processing apparatus 20 of the present embodiment, an estimation model with high accuracy can be generated, and n set values can be efficiently optimized.

[0135] FIG. 7 is a diagram showing a hardware configuration example of the information processing apparatus 20 according to the embodiment.

[0136] The information processing apparatus 20 according to the embodiment includes a control device such as a CPU 201, a storage device such as a ROM 202 (Read Only Memory) and a RAM 203, a communication I / F 204 that connects to a network and performs communication, and a bus 211 that connects each part.

[0137] The program executed by the information processing apparatus 20 according to the embodiment is provided by being pre-installed in the ROM 202 or the like.

[0138] The program executed by the information processing apparatus 20 according to the embodiment may be recorded on a computer-readable recording medium such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), a DVD (Digital Versatile Disk) in an installable format or an executable format file, and provided as a computer program product.

[0139] The program executed by such an information processing apparatus 20 includes, for example, a change direction information input module, an acquisition module, an end determination module, a model generation module, a monotonicity update module, a recommendation module, and an output module.

[0140] This program is expanded and executed on the RAM 203 by the CPU 201 (processor), causing the computer to function as a change direction information input unit 62, an acquisition unit 64, an end determination unit 66, a model generation unit 68, a monotonicity update unit 70, a recommendation unit 72, and an output unit 74. Note that part or all of the change direction information input unit 62, the acquisition unit 64, the end determination unit 66, the model generation unit 68, the monotonicity update unit 70, the recommendation unit 72, and the output unit 74 may be configured as a hardware circuit. Also, the RAM 203 functions as a storage unit 40.

[0141] Also, the program executed by the computer is recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk, a CD-R, a DVD (Digital Versatile Disk) in an installable format or an executable format file and provided.

[0142] Alternatively, the program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Further, the program may be configured to be provided or distributed via a network such as the Internet. Further, the program may be configured to be provided by being pre-installed in a ROM 202 or the like.

[0143] Although some embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

[0144] (Supplementary Note) Incidentally, the above embodiments can be summarized into the following technical proposals.

[0145] [Technical Proposal 1] Based on one or more data sets each including n set values (n is an integer of 1 or more) and one or more evaluation values representing the evaluation of an experiment or simulation executed using the n set values, and change direction information, generate an estimation model, Based on the estimation model, calculate n recommended values recommended as the n set values to be used in the experiment or the simulation Comprising a processing unit, The change direction information indicates the change direction of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. An information processing apparatus.

[0146] [Technical Proposal 2] The estimation model is a model that calculates respective estimated values and estimated standard deviations of the one or more evaluation values based on n parameters corresponding to the n set values, Each of the n parameters is a variable into which the corresponding set value among the n set values is input The information processing apparatus according to Technical Proposal 1.

[0147] [Technical Proposal 3] The change direction information indicates the direction of change of the target evaluation value with respect to the change of the target parameter for each combination of each of the n parameters and each of the one or more evaluation values. The information processing apparatus according to claim 1 or 2.

[0148] [Technical Proposal 4] The processing unit Every time the n recommended values are calculated, obtain the one or more evaluation values representing the evaluation of the experiment or the simulation using the n recommended values, Every time the one or more evaluation values are obtained, add a new data set including the obtained one or more evaluation values to the one or more data sets, and the new data set includes the n recommended values as the n set values, Every time the new data set is added to the one or more data sets, generate the estimation model, Every time the estimation model is generated, calculate the n recommended values based on the generated estimation model Repeat the process The information processing apparatus according to Technical Proposal 2 or 3.

[0149] [Technical Proposal 5] The processing unit When the one or more data sets do not exist, output the n recommended values selected from a plurality of predetermined values or the n recommended values generated based on a predetermined rule. The information processing apparatus according to Technical Proposal 4.

[0150] [Technical Solution 6] After repeating the processing until a predetermined end condition is reached, the processing unit outputs the n set values included in the data set among the one or more data sets in which the one or more evaluation values are the best. The information processing apparatus according to Technical Solution 5.

[0151] [Technical Solution 7] The estimation model is a model considering monotonicity using information for identifying, for each of the n parameters, whether the target evaluation value among the one or more evaluation values monotonically increases as the target parameter increases, whether the target evaluation value monotonically decreases as the target parameter increases, or whether the target evaluation value neither monotonically increases nor monotonically decreases with respect to the target parameter. The information processing apparatus according to any one of Technical Solutions 2 to 6.

[0152] [Technical Solution 8] The estimation model is a Gaussian process regression model considering monotonicity, the estimated value is represented by Equation (4), and the estimated standard deviation is represented by Equation (5). The information processing apparatus according to Technical Solution 7.

[0153] [Technical Solution 9] The processing unit further uses one or more pseudo data sets each including n values corresponding to n pseudo parameters and the respective estimated partial derivative values of the one or more evaluation values with respect to the parameter of the d-th dimension (d is 1 or more and n or less) among the n parameters to generate the estimation model. The information processing apparatus according to Technical Solution 7 or 8.

[0154] [Technical Solution 10] The change direction information includes monotonicity information for each combination of each of the n parameters and each of the one or more evaluation values. The monotonicity information is The monotonic increase property in which the target evaluation value monotonically increases as the target parameter increases. A monotonic decreasing property in which the target evaluation value monotonically decreases in response to an increase in the target parameter, A non-monotonic property in which the target evaluation value neither monotonically increases nor monotonically decreases with respect to the target parameter, A direction-unknown property in which the target evaluation value either monotonically increases or monotonically decreases with respect to the target parameter, but it is unknown whether it is a monotonic increase or a monotonic decrease, or, A monotonicity-unknown property in which it is unknown whether the target evaluation value is the monotonic increasing property, the monotonic decreasing property, or the non-monotonic property with respect to the target parameter indicating any one of the above An information processing apparatus according to any one of Technical Proposals 7 to 9.

[0155] [Technical Proposal 11] The processing unit, when the monotonicity information indicating the monotonicity-unknown or the direction-unknown is included in the change direction information, generates all combination patterns in which all of the monotonicity information indicating the monotonicity-unknown in the change direction information is replaced with the monotonic increasing property, the monotonic decreasing property, or the non-monotonic property, and all of the monotonicity information indicating the direction-unknown is replaced with the monotonic increasing property or the monotonic decreasing property, or generates a plurality of hypothetical change direction information corresponding to some of the all combination patterns, for each of the plurality of hypothetical change direction information, generates the estimation model based on the corresponding hypothetical change direction information and the one or more data sets, selects an estimation model having the minimum estimation error or the maximum length scale among the estimation models generated for each of the plurality of hypothetical change direction information, calculates the n recommended values based on the selected estimation model An information processing apparatus according to Technical Proposal 10.

[0156] [Technical Proposal 12] The processing unit updates the change direction information to the hypothetical change direction information used for generating the selected estimation model. An information processing apparatus according to Technical Proposal 11.

[0157] [Technical Solution 13] The processing unit updates the change direction information when the estimation error is greater than a predetermined threshold value The information processing apparatus according to Technical Solution 12

[0158] [Technical Solution 14] The processing unit calculates, based on the estimation model, the respective estimated partial differential values of the one or more evaluation values with respect to the parameter corresponding to the monotonicity information indicating the monotonic increase or the monotonic decrease in the change direction information, updates the change direction information when the monotonicity information indicating the monotonic increase or the monotonic decrease in the change direction information is different from the change direction of each of the one or more evaluation values specified by the estimated partial differential value for the corresponding parameter The information processing apparatus according to Technical Solution 12 or 13

[0159] [Technical Solution 15] The processing unit calculates the n recommended values based on the estimation model using Bayesian optimization The information processing apparatus according to any one of Technical Solutions 10 to 14

[0160] [Technical Solution 16] The processing unit calculates, in the Bayesian optimization, the n recommended values that maximize the product of the expected improvement amount and the constraint satisfaction probability The information processing apparatus according to Technical Solution 15

[0161] [Technical Solution 17] The processing unit selects the data set in which the one or more evaluation values among the one or more data sets are the best, selects some of the n parameters in the estimation model Fix the values of the selected part of the parameters to the corresponding values among the n set values included in the best data set, and calculate the n recommended values based on the estimation model. The information processing apparatus according to Technical Proposal 15 or 16.

[0162] [Technical Proposal 16] The processing unit selects the parameter for which the monotonic information is not the monotonicity unknown or the direction unknown, Fix the values of the selected part of the parameters to the corresponding values among the n set values included in the best data set, and calculate the n recommended values for which the acquisition function becomes maximum based on the estimation model. The information processing apparatus according to Technical Proposal 17.

[0163] [Technical Proposal 20] Based on one or more data sets each including n set values (n is an integer of 1 or more) and one or more evaluation values representing the evaluation of the experiment or simulation executed using the n set values, and change direction information, an estimation model is generated. The information processing apparatus calculates n recommended values recommended as the n set values to be used in the experiment or the simulation based on the estimation model. The change direction information indicates the change direction of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. Information processing method.

[0164] [Technical Proposal 20] An information processing apparatus, Based on one or more data sets each including n set values (n is an integer of 1 or more) and one or more evaluation values representing the evaluation of the experiment or simulation executed using the n set values, and change direction information, an estimation model is generated. Based on the estimation model, calculate n recommended values recommended as the n set values to be used in the experiment or the simulation. Function as a processing unit, The change direction information indicates the change direction of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. Program.

Explanation of Signs

[0165] 10 Information processing system 20 Information processing device 30 Evaluation device 40 Storage unit 50 Processing unit 62 Change direction information input unit 64 Acquisition unit 66 End determination unit 68 Model generation unit 70 Monotonicity update unit 72 Recommendation unit 74 Output unit

Claims

1. Based on one or more data sets each including n set values (n is an integer of 1 or more) and one or more evaluation values representing evaluations of experiments or simulations executed using the n set values, and change direction information, generate an estimation model, Based on the estimation model, calculate n recommended values recommended as the n set values to be used in the experiment or the simulation Comprising a processing unit, The change direction information indicates the direction of change of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. An information processing apparatus.

2. The estimation model is a model that calculates an estimated value and an estimated standard deviation of each of the one or more evaluation values based on n parameters corresponding to the n set values, Each of the n parameters is a variable into which the corresponding set value among the n set values is input. The information processing apparatus according to claim 1.

3. The change direction information indicates the direction of change of the target evaluation value with respect to the change of the target parameter for each combination of each of the n parameters and each of the one or more evaluation values. The information processing apparatus according to claim 1.

4. The processing unit, Every time the n recommended values are calculated, obtain the one or more evaluation values representing the evaluation of the experiment or the simulation using the n recommended values, Every time the one or more evaluation values are obtained, add a new data set including the obtained one or more evaluation values to the one or more data sets, and the new data set includes the n recommended values as the n set values, Every time the new data set is added to the one or more data sets, generate the estimation model, Every time the estimation model is generated, calculate the n recommended values based on the generated estimation model Repeat the process The information processing apparatus according to claim 2.

5. The processing unit, When the one or more data sets do not exist, output the n recommended values selected from a plurality of predetermined values or the n recommended values generated based on a predetermined rule. The information processing apparatus according to claim 4.

6. After repeating the processing until a predetermined end condition is reached, the processing unit outputs the n set values included in the data set among the one or more data sets in which the one or more evaluation values are the best. The information processing apparatus according to claim 5.

7. The estimation model is a model that takes into account monotonicity, using information for identifying, for each of the n parameters, whether the target evaluation value among the one or more evaluation values monotonically increases as the target parameter increases, whether the target evaluation value monotonically decreases as the target parameter increases, or whether the target evaluation value neither monotonically increases nor monotonically decreases with respect to the target parameter. The information processing apparatus according to claim 2.

8. The estimation model is a Gaussian process regression model that takes into account monotonicity, the estimated value is represented by Equation (101), and the estimated standard deviation is represented by Equation (102). 【Number 1】 The information processing apparatus according to claim 7.

9. The processing unit further uses one or more pseudo data sets each including n values corresponding to n pseudo parameters and respective estimated partial differential values of the one or more evaluation values with respect to the parameter of the d-th dimension (d is 1 or more and n or less) among the n parameters to generate the estimation model. The information processing apparatus according to claim 7.

10. The change direction information includes monotonicity information for each combination of each of the n parameters and each of the one or more evaluation values. The monotonicity information is Monotonic increasing property in which the target evaluation value monotonically increases as the target parameter increases, Monotonic decreasing property in which the target evaluation value monotonically decreases as the target parameter increases, Non-monotonicity in which the target evaluation value neither monotonically increases nor monotonically decreases with respect to the target parameter, Direction uncertainty in which the target evaluation value monotonically increases or decreases with respect to the target parameter, but it is unknown whether it monotonically increases or decreases, or Monotonicity uncertainty in which it is unknown whether the target evaluation value has the monotonic increasing property, the monotonic decreasing property, or the non-monotonicity with respect to the target parameter indicating any one of them. The information processing apparatus according to claim 7.

11. The processing unit when the change direction information includes the monotonicity information indicating the monotonicity uncertainty or the direction uncertainty Generate all combination patterns in which all of the monotonicity information indicating the unknown monotonicity in the change direction information is replaced with the monotonic increasing property, the monotonic decreasing property, or the non-monotonic property, and all of the monotonicity information indicating the unknown direction is replaced with the monotonic increasing property or the monotonic decreasing property, or generate a plurality of hypothetical change direction information corresponding to some of the all combination patterns. For each of the plurality of hypothetical change direction information, generate the estimation model based on the corresponding hypothetical change direction information and the one or more data sets. Among the estimation models generated for each of the plurality of hypothetical change direction information, select an estimation model with the minimum estimation error or the maximum length scale. Calculate the n recommended values based on the selected estimation model. The information processing apparatus according to claim 10.

12. The processing unit updates the change direction information to the hypothetical change direction information used for generating the selected estimation model. The information processing apparatus according to claim 11.

13. The processing unit If the estimation error is greater than a predetermined threshold value, update the change direction information. The information processing apparatus according to claim 12.

14. The processing unit Based on the estimation model, calculate the estimated partial derivative value of each of the one or more evaluation values with respect to the parameter corresponding to the monotonicity information indicating the monotonic increase or the monotonic decrease in the change direction information. If the monotonicity information indicating the monotonic increase or the monotonic decrease in the change direction information is different from the change direction of each of the one or more evaluation values specified by the estimated partial derivative value for the corresponding parameter, update the change direction information. The information processing apparatus according to claim 12.

15. The processing unit calculates the n recommended values based on the estimation model using Bayesian optimization. The information processing apparatus according to claim 10.

16. The processing unit calculates the n recommended values that maximize the product of the expected improvement amount and the constraint satisfaction probability in the Bayesian optimization. The information processing apparatus according to claim 15.

17. The processing unit Select the data set in which the one or more evaluation values among the one or more data sets are the best. Select some of the n parameters in the estimation model. Fix the values of the selected part of the parameters to the corresponding values among the n set values included in the best data set, and calculate the n recommended values based on the estimation model. The information processing apparatus according to claim 15.

18. The processing unit selects a parameter for which the monotonic information is not the monotonicity unknown or the direction unknown, Fix the values of the selected part of the parameters to the corresponding values among the n set values included in the best data set, and calculate the n recommended values for which the acquisition function becomes maximum based on the estimation model. The information processing apparatus according to claim 17.

19. Based on one or more data sets each including n set values (n is an integer of 1 or more), one or more evaluation values representing the evaluation of an experiment or simulation executed using the n set values, and change direction information, an estimation model is generated. The information processing apparatus calculates n recommended values recommended as the n set values to be used in the experiment or the simulation based on the estimation model. The change direction information indicates the change direction of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. Information processing method.

20. An information processing apparatus, Based on one or more data sets each including n set values (n is an integer of 1 or more), one or more evaluation values representing the evaluation of an experiment or simulation executed using the n set values, and change direction information, an estimation model is generated. Based on the estimation model, n recommended values recommended as the n set values to be used in the experiment or the simulation are calculated. Function as a processing unit, The change direction information indicates the change direction of the target evaluation value among the one or more evaluation values with respect to the change of the target parameter among the n parameters for any one or more combinations of each of the n parameters corresponding to the n set values and each of the one or more evaluation values. Program.

Images