Parameter optimization device, method, and program

Abstract

To efficiently search for an optimum parameter value.SOLUTION: A decision section determines a search space of a second number of dimensions smaller than a first number of dimensions including a predetermined parameter value. An acquisition section calculates a distance between a parameter value and the search space for each of a plurality of data sets and acquires one or more data sets each having a parameter value within a distance of a threshold or less. A search section searches the search space for a parameter value that may optimize an objective function using a surrogate model of the objective function based on one or more data sets. A generation section generates a data set including an observed value of the objective function corresponding to the searched parameter value, and the searched parameter value. A storage section further stores the data set generated by the generation section. A determination section determines whether or not a predetermined condition is satisfied after the generation section generates the data set. When the predetermined condition is not satisfied, processing by at least the acquisition, search, and generation sections is repeated.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] An embodiment of the present invention relates to a parameter optimization device, method, and program. [Background technology]

[0002] In society, there are various devices that are composed of various parts. These devices and parts are designed, manufactured, and utilized.

[0003] In the design stage, devices or parts whose characteristics satisfy the specifications may be designed. In this case, parameters that can be adjusted during design are changed to various values, and simulations, experiments, and surveys are conducted to obtain the characteristics when designed with those parameter values, and parameter values ​​that satisfy the specifications are determined. Here, the characteristics are, for example, the performance, manufacturing cost, and customer satisfaction of the devices or parts. The better the performance of the devices or parts, the better the manufacturing cost, and the higher the customer satisfaction. When the larger the value of the characteristic, the better, it is required to determine the parameter value that maximizes the characteristic with less time and effort. When the smaller the value of the characteristic, the better, it is required to determine the parameter value that minimizes the characteristic with less time and effort.

[0004] Finding parameter values ​​that maximize or minimize a characteristic is called parameter optimization. The characteristic that changes depending on the parameter value is called the objective function. Simulations, experiments, and surveys are means of observing the value of the objective function.

[0005] Parameter optimization may also be used during the manufacturing stage, for example to find parameter values ​​that maximize the yield during manufacturing or minimize the failure rate after shipment.

[0006] Parameter optimization may also be used in the utilization phase. For example, when a device or part is delivered to a user, the user may need to determine parameter values ​​that will provide performance suited to the user's usage environment during initial setup.

[0007] There are often multiple parameters. If the number of parameters to be adjusted is represented by D, then the multiple parameters can be expressed as a D-dimensional vector. A parameter that can be expressed as a D-dimensional vector is called a D-dimensional parameter, and the value of a D-dimensional parameter is called a D-dimensional parameter value. The space in which the optimal D-dimensional parameter values ​​are searched is a D-dimensional space. The larger D is, the wider the search space is, and therefore the more difficult optimization is.

[0008] The method described in Non-Patent Document 1 is known as a parameter optimization method for searching for D-dimensional parameter values ​​that maximize or minimize an objective function in a D-dimensional space (D≧2). In this method, the search space is limited to a one-dimensional space in the D-dimensional space, and parameter values ​​are searched for by repeating Bayesian optimization while switching between the one-dimensional search spaces. This makes the search for D-dimensional parameters more efficient. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2020-27370 A [Non-patent literature]

[0010] [Non-Patent Document 1] J. Kirschner, M. Mutny, N. Hiller, R. Ischebeck, and A. Krause, “Adaptive and safe Bayesian optimization in high dimensions via one-dimensional subspaces,” in Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 3429-3438, PMLR, 2019. Summary of the Invention [Problem to be solved by the invention]

[0011] The Bayesian optimization used in the method of Non-Patent Document 1 requires the calculation of an inverse matrix, and the amount of calculation increases according to the number of iterations of the parameter search. Even though it takes a long time to observe the value of the objective function, such as in a simulation, an experiment, or a questionnaire, the amount of calculation may become significant when the number of iterations of the parameter search increases.

[0012] An object of the present invention is to provide a parameter optimization device, method, and program capable of efficiently searching for optimal parameter values. [Means for solving the problem]

[0013] A parameter optimization device according to an embodiment includes a storage unit, a search space determination unit, a first acquisition unit, and a parameter search unit. The storage unit stores a plurality of data sets including a first parameter value of a first dimension and an observation value of an objective function corresponding to the first parameter value. The search space determination unit determines a search space of a second dimension smaller than the first dimension including a predetermined first parameter value. The first acquisition unit acquires one or more data sets having a first parameter value included within a predetermined distance from the search space or a data set having a second parameter value corresponding to the data set based on the plurality of data sets. The parameter search unit searches for a first parameter value or a second parameter value that can optimize the objective function within the search space using a proxy model of the objective function based on the acquired one or more data sets. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an example of the functional configuration of a parameter optimization system according to a first embodiment; [Diagram 2] FIG. 13 is a diagram showing an example of a data set stored in a data storage unit; [Diagram 3] A diagram showing an example of a one-dimensional search space in a two-dimensional parameter space. [Figure 4] Diagram showing the concept of a data set around a 1D search space in a 2D parameter space [Diagram 5] A diagram showing an example of how to convert a 20-dimensional dataset into a 1-dimensional dataset. [Figure 6] A diagram showing an example of a method for converting a 1-dimensional parameter value into a 20-dimensional parameter value. [Figure 7] A flowchart showing an example of the processing operation of the parameter optimization system according to the first embodiment. [Figure 8] A diagram showing an example of a one-dimensional affine subspace. [Figure 9] FIG. 13 is a diagram showing an example of the functional configuration of a parameter optimization system according to a second embodiment. [Figure 10] A flowchart showing an example of a processing operation of the parameter optimization system according to the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of a dimension acquisition screen. [Figure 12] FIG. 13 is a diagram showing an example of a distance type input screen. [Figure 13] FIG. 13 is a diagram showing an example of a threshold input screen. [Figure 14] FIG. 1 is a diagram showing an example of a hardware configuration of a parameter optimization device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The terms used in this embodiment are defined below. The number of parameters to be adjusted is represented by D. The D parameters can be expressed by a D-dimensional vector. A parameter that can be expressed by a D-dimensional vector is called a D-dimensional parameter, and a value that a D-dimensional parameter has is called a D-dimensional parameter value. The D-dimensional parameter value is represented by x. The objective function to be optimized is represented by f(·), and the observed value of the objective function is represented by y. An r-dimensional parameter value is represented by x. r Let D>r.

[0016] If you want to find the D-dimensional parameter values ​​that maximize the objective function, you can rewrite it into an equivalent minimization problem by inverting the sign of the objective function. Therefore, below we will explain an example of finding the D-dimensional parameter values ​​that minimize the objective function f(·), and omit the explanation of the maximization problem.

[0017] Hereinafter, a parameter optimization device, method, and program according to the present embodiment will be described with reference to the drawings. In the following embodiment, the characteristic configuration and operation of the parameter optimization device will be mainly described, but the parameter optimization device may have configurations and operations that are omitted in the following description.

[0018] (First embodiment) FIG. 1 is a diagram showing an example of a functional configuration of a parameter optimization system 1 according to the first embodiment. As shown in FIG. 1, the parameter optimization system 1 is a computer system having a parameter optimization device 2A and an observation value output device 3. The parameter optimization device 2A and the observation value output device 3 are connected to each other so as to be able to communicate with each other via wired or wireless communication. The parameter optimization device 2A is a computer that searches for parameter values ​​that may optimize an objective function based on the observation values ​​from the observation value output device 3 and the parameter values ​​corresponding thereto. The observation value output device 3 is a computer that performs a simulation, an experiment, or a questionnaire based on the parameter values, and observes the values ​​of the objective function, in other words, outputs the observation values ​​of the objective function. The parameter optimization system 1 repeats the search for parameter values ​​that may optimize the objective function by the parameter optimization device 2A, and the output of the observation values ​​of the objective function using the observation value output device 3 corresponding to the searched parameter values.

[0019] As shown in FIG. 1, a parameter optimization device 2A includes, for example, a data storage unit 11, a search space determination unit 12, a data acquisition unit 13, a parameter search unit 14, a data generation unit 15, and an optimal parameter value acquisition unit 16.

[0020] The data storage unit 11 stores, for example, a set of D-dimensional parameter values ​​and observed values ​​of an objective function corresponding to the D-dimensional parameter values ​​as data. The data set of D-dimensional parameter values ​​and observed values ​​is called a data set. The data storage unit 11 stores one or more data sets. The observed values ​​of the objective function are calculated by the observed value output device 3 based on the D-dimensional parameter values.

[0021] The data storage unit 11 stores N0 data sets {(x n ,y n )|n=0,1,…,N0-1}. x n represents the n-th D-dimensional parameter value. y n x n represents the observed value of the objective function f with respect to y n =f(x n)+ε n Here, ε n is the observed value y n The number of data sets N0 can be increased by storing data sets generated by the data generating unit 15 in the data storage unit 11.

[0022] Observed value y n is observed based on simulations, experiments, questionnaires, etc. by the observation value output device 3. In experiments, simulations, questionnaires, etc., characteristic values ​​for one or more items are obtained. When there is one item, the obtained characteristic value is a scalar, and the characteristic value is used as the observation value of the objective function. When there are multiple characteristic value items obtained in experiments, simulations, questionnaires, etc., the obtained characteristic value is expressed as a vector. The characteristic value expressed as a vector is converted to a scalar and used as the observation value of the objective function. For conversion to a scalar, for example, a linear weighted sum method is used.

[0023] FIG. 2 is a diagram showing an example of a data set stored in the data storage unit 11. FIG. 2 is an example of a data set of D=20. The 20-dimensional parameters are 20 variables x (0) , x (1) , …, x (19) where each variable has a unique value. Each variable may be a design parameter of a filter or device, or a setting parameter of a manufacturing device. The value of each variable may be a continuous value, a discrete value, or a categorical variable. The variable of the observed value of the objective function calculated from the 20-dimensional parameter value is denoted as y. Each row in FIG. 2 is a data set having the 20-dimensional parameter value and the observed value of the objective function corresponding to the 20-dimensional parameter value.

[0024] The search space determination unit 12 selects, from the data set stored in the data storage unit 11, the D-dimensional parameter value x expThen, r-dimensional parameters less than D dimensions are selected from the D-dimensional parameters, and a search space of r dimensions including the obtained D-dimensional parameter values ​​is determined. If there is no data set stored in the data storage unit 11, the D-dimensional parameter value x exp may be set randomly, or may be repurposed from past adjustment results. The search space is an affine subspace. The value of the number of dimensions r may be a predetermined value, or may be a randomly determined value.

[0025] FIG. 3 is a diagram showing an example of a one-dimensional search space in a two-dimensional parameter space. That is, FIG. 3 shows an example in the case where D=2 and r=1. The two-dimensional parameter space shown in FIG. 3 is a space defined by coordinate axes x0 and x1. A contour map 50 represents a two-dimensional objective function in the two-dimensional parameter space. The center 51 of the contour map 50 corresponds to the minimum value of the objective function. The four points represent four two-dimensional parameter values. Of these two-dimensional parameter values, the two-dimensional parameter value with the smallest observed value of the objective function is x exp The two-dimensional parameter value x exp is a reference point of the one-dimensional search space 52. When determining the one-dimensional search space 52, it is necessary to select a dimension of interest. In FIG. 3, x0 is selected as the dimension of interest. The search space determination unit 12 calculates the two-dimensional parameter value x exp A one-dimensional space along x0 including is determined as one-dimensional search space 52. By selecting dimensions in this way, it becomes possible to solve the optimization problem of high-dimensional parameters by dividing it into subproblems of optimization problems of low-dimensional parameters.

[0026] The data acquisition unit 13 acquires a data set having a D-dimensional parameter value that is within a predetermined distance from the search space acquired from the search space determination unit 12, based on a plurality of data sets stored in the data storage unit 11. The data set having the D-dimensional parameter value that is within a predetermined distance from the search space is called a data set near the search space. The distance between the search space and the D-dimensional parameter value may be arbitrarily selected from Euclidean distance, Manhattan distance, cosine distance, and Mahalanobis distance. The cosine distance is obtained by subtracting the cosine similarity from 1. Although the cosine distance does not satisfy the mathematical axiom of distance, it can be used as a measure for measuring the distance from the search space. The data set acquired by the data acquisition unit 13 may be a set of a D-dimensional parameter value and an observation value, or a set of an r-dimensional parameter value and an observation value.

[0027] FIG. 4 is a diagram showing the concept of a data set in the vicinity of a one-dimensional search space in a two-dimensional parameter space. That is, FIG. 4 shows an example in the case where D=2 and r=1. The two-dimensional parameter values ​​in the vicinity of the one-dimensional search space are two-dimensional parameter values ​​whose distance from the one-dimensional search space 52 is equal to or less than a threshold value d. In other words, the two-dimensional parameter values ​​x exp and the two-dimensional parameter value x oth The two-dimensional parameter value x oth is included in the space 53 but not included in the one-dimensional search space 52. When d=0, the two-dimensional parameter value near the one-dimensional search space means the two-dimensional parameter value on the one-dimensional search space 51. Among the four points in FIG. 4, the two-dimensional parameter value on the one-dimensional search space 51 is x exp For d=0, the two-dimensional parameter value x oth is not acquired by the data acquisition unit 13.

[0028] The data acquisition unit 13 may select a data set near the search space from a plurality of data sets stored in the data storage unit 11, or may generate the data set based on a plurality of data sets stored in the data storage unit 11. The data acquisition unit 13 may convert a data set having D-dimensional parameter values ​​near the search space acquired from the data storage unit 11 into a data set of r-dimensional parameter values ​​and observed values ​​of the objective function corresponding to the r-dimensional parameter values. Alternatively, the data set may be generated randomly from D-dimensional parameter values ​​or r-dimensional parameter values ​​near the search space, and a data set may be created based on the parameter values.

[0029] FIG. 5 is a diagram showing an example of a method for converting a 20-dimensional data set into a one-dimensional data set. That is, FIG. 5 shows an example in which D=20 and r=1. FIG. 5 shows a case in which the one-dimensional parameters selected by the search space determination unit 12 are x (2) 5 is an example of the case where the left side of FIG. 5 is a 20-dimensional data set near the one-dimensional search space acquired from the data storage unit 11. The conversion of this 20-dimensional data set into a one-dimensional data set can be performed, for example, by using the one-dimensional parameter x (2) This is done by converting the value of and the observed value y of the objective function into a set of data.

[0030] The parameter search unit 14 uses a surrogate model of the objective function based on one or more data sets acquired by the data acquisition unit 13 to find D-dimensional parameter values ​​x that can optimize the objective function within the search space determined by the search space determination unit 12. next or r-dimensional parameter value x r next Specifically, the parameter search unit 14 generates a surrogate model of the objective function based on the acquired data set, and determines D-dimensional parameter values ​​or r-dimensional parameters in the r-dimensional search space that optimize the surrogate model as D-dimensional parameter values ​​x next or r-dimensional parameter value x r next When the parameter value of the data set acquired from the data acquisition unit 13 is D-dimensional, the D-dimensional parameter value xnext On the other hand, if the parameter value of the data set acquired from the data acquisition unit 13 is r-dimensional, the r-dimensional parameter value x r next The parameter values ​​may be searched for using Bayesian optimization or the like.

[0031] The data generator 15 calculates the D-dimensional parameter value x next or r-dimensional parameter value x r next The observed value y of the objective function corresponding to next and the D-dimensional parameter value x next or r-dimensional parameter value x r next Generate a dataset containing:

[0032] Specifically, the data generator 15 calculates the D-dimensional parameter value x next is searched, the D-dimensional parameter value x next is input to the observation output device 3. The observation output device 3 performs simulations, experiments, and questionnaires to obtain the D-dimensional parameter value x next From the observed value y next The data generating unit 15 outputs the observed value y next Obtain the D-dimensional parameter value x next and the observed value y next and stores the data set in the data storage unit 11. The data generation unit 15 generates a data set including the r-dimensional parameter value x r next is searched for, the r-dimensional parameter value x r next Let x be the dimensional parameter value. next Convert to.

[0033] Fig. 6 is a diagram showing an example of a method for converting a one-dimensional parameter value into a twenty-dimensional parameter value. That is, Fig. 6 shows an example in which D=20 and r=1. Fig. 6 shows a case in which the one-dimensional parameter selected by the search space determination unit 12 is the one-dimensional parameter value x(2) This is an example of the case where the 20-dimensional parameter value x exp The 1-dimensional parameter value x of the third element of (2) , the one-dimensional parameter value x 1 next By embedding the 20-dimensional parameter value x exp may be converted to

[0034] Then, the data generator 15 converts the converted D-dimensional parameter value x next is input to the observation output device 3. The observation output device 3 performs simulations, experiments, and questionnaires to obtain the D-dimensional parameter value x next From the observed value y next The data generating unit 15 outputs the observed value y next Obtain the D-dimensional parameter value x next and the observed value y next and stores the data set in the data storage unit 11.

[0035] The optimal parameter value acquisition unit 16 obtains the D-dimensional parameter value corresponding to the smallest observed value among the multiple data sets stored in the data storage unit 11 as the optimal parameter value x best Select as.

[0036] Next, the processing operation of the parameter optimization system 1 having the parameter optimization device 2A according to the first embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the processing operation of the parameter optimization system 1 according to the first embodiment.

[0037] As shown in FIG. 7, first, the search space determination unit 12 determines a parameter value x exp (Step SA1) The initial parameter values ​​may be selected randomly, or may be diverted from the results of past adjustments.

[0038] After step SA1 is performed, the search space determination unit 12 selects r-dimensional parameters with dimensions less than D (step SA2). In addition, if there is a data set stored in the data storage unit 11, the search space determination unit 12 selects the D-dimensional parameter value x exp If there is no stored data set, the parameter value x set in step SA1 is obtained. exp The r-dimension parameter may be selected, for example, by dividing the D-dimension parameters into groups each consisting of a plurality of r-dimension parameters in advance, and one group may be selected from the plurality of groups, or may be selected randomly. It is preferable that a group is selected in order from the plurality of groups each time a loop related to the second condition described below is repeated a predetermined number of times. For example, x0 to x4 are the first group, x5 to x9 are the second group, and x 10 From x 14 is the third group, x 15 From x 19 may be set to the fourth group, and the first group, the second group, the third group, and the fourth group may be selected in order for each iteration of the loop related to the second condition. Alternatively, the r-dimensional parameters may be selected randomly from among the D-dimensional parameters without grouping.

[0039] When step SA2 is performed, the search space determination unit 12 calculates the D-dimensional parameter value x exp The search space determination unit 12 determines, as a search space, an r-dimensional affine subspace including the parameter value x exp Determine the r-dimensional affine subspace that passes through. Here, the r-dimensional affine subspace is denoted by A. A is defined as A=x exp +U={x exp +u|u∈U}. exp is the position vector of A. U is an r-dimensional linear subspace associated with A. The r-dimensional affine subspace may be changed for each iteration of the loop regarding the second condition.

[0040] FIG. 8 is a diagram showing an example of a one-dimensional affine subspace 61. That is, FIG. 8 shows an example in the case where D=2 and r=1. FIG. 8 shows an example of a one-dimensional affine subspace 61 set in a two-dimensional space with coordinate axes x0 and x1. A one-dimensional linear subspace 62 associated with the one-dimensional affine subspace 61 is set. The one-dimensional linear subspace 62 passes through the origin of the two-dimensional coordinate space. The one-dimensional affine subspace 61 is a space between x exp Within a range passing through, the one-dimensional linear subspace 62 can be changed according to the number of iterations of the loop related to the second condition. For example, the one-dimensional affine subspace 61 is changed so as to be parallel to each coordinate axis of the two-dimensional space according to the number of iterations of the loop related to the second condition. In the example of FIG. 8, the one-dimensional affine subspace 61 before the change is parallel to the x0 axis corresponding to the one-dimensional x0 selected in step SA2, but when the loop related to the second condition is repeated, it may be changed so as to be parallel to the x1 axis corresponding to the one-dimensional x1 selected in step SA2. In this way, the one-dimensional linear subspace 62 associated with the one-dimensional affine subspace 61 may be changed alternately between the x0 axis and the x1 axis each time the loop related to the second condition is repeated. The search space determination unit 12 sets the changed one-dimensional affine subspace 61 in the one-dimensional search space.

[0041] If D>r≧2 and the dimension of the affine subspace is 2 or more, then for each iteration of the loop for the second condition, the r-dimensional linear portion associated with the r-dimensional affine subspace may be set to include any r of the D coordinate axes defining the D-dimensional space. exp If is not a zero vector, then the r-dimensional affine subspace is parallel to each of its r coordinate axes.

[0042] When step SA3 is performed, the data acquisition unit 13 acquires a data set within a predetermined distance (threshold) from the r-dimensional search space (step SA4). Specifically, the data acquisition unit 13 selects and acquires a data set having a D-dimensional parameter value (D-dimensional parameter value near the search space) that is included within a predetermined distance from the r-dimensional search space from among a plurality of data sets included in the data storage unit 11. Specifically, the data acquisition unit 13 calculates the distance between the D-dimensional parameter value and the r-dimensional search space for each of the N0 data sets, and identifies a D-dimensional parameter value whose calculated distance is equal to or less than the predetermined distance. A data set including the identified D-dimensional parameter value and an observation value corresponding to the D-dimensional parameter value is acquired from the data storage unit 11. Here, the data acquisition unit 13 may acquire an r-dimensional parameter value from among the identified D-dimensional parameter values. In this case, the data acquisition unit 13 may convert the data set related to the identified D-dimensional parameter value into a data set related to the r-dimensional parameter value and the r-dimensional parameter value, as shown in FIG. 6. The distance threshold does not need to be a constant value and may be variable. The distance threshold may be changed, for example, each time in a loop relating to steps SA11 and SA12 described later, or according to the number of iterations of the loop. The threshold may be set smaller as the number of iterations increases.

[0043] The number of data sets acquired by the data acquisition unit 13 is represented as N. The N data sets are expressed as {(x n ,y n )|n=0, 1, ..., N-1}. The number N0 of data sets stored in the data storage unit 11 increases by repeating the loop related to the first condition and the loop related to the second condition, so the value of the number N changes according to the repetition of the loop related to the first condition and the loop related to the second condition.

[0044] When step SA4 is performed, the data acquisition unit 13 determines whether or not one or more data sets have been acquired from the data storage unit 11 in step SA4 (step SA5).

[0045] If it is determined in step SA5 that one or more data sets have been acquired (step SA5: YES), the parameter search unit 14 generates a proxy model of the objective function based on the one or more data sets (step SA6). As described above, in step SA6, the parameter search unit 14 models the objective function by Gaussian process regression, for example. If the parameter values ​​of the data sets acquired in step SA4 are D-dimensional parameter values, the objective function is modeled as a function of the D-dimensional parameters. If the parameter values ​​of the data sets acquired in step SA4 are r-dimensional parameter values, the objective function is modeled as a function of the r-dimensional parameters.

[0046] When step SA6 is performed, the parameter search unit 14 uses the surrogate model to find D-dimensional parameter values ​​x that are likely to minimize the observed value of the objective function in the r-dimensional search space. next or r-dimensional parameter value x r next (Step SA7). When the objective function is modeled as a function of D-dimensional parameters, the D-dimensional parameter values ​​x next is searched for, and if the objective function is modeled as a function of r-dimensional parameters, then the r-dimensional parameter values ​​x r next is explored.

[0047] Here, steps SA6 and SA7 will be described in detail. To generate a surrogate model of the objective function, Gaussian process regression may be used, or other surrogate models such as TPE (Tree-structured Parzen Estimator) and random forest regression may be used.

[0048] In the case of Bayesian optimization, Gaussian process regression, which is a function of D-dimensional parameters, may be used. In this case, the parameter search unit 14 uses the N data sets {(x n ,y n)|n=0,1,...,N-1}, a Gaussian process regression model is generated as a surrogate model for the objective function. Next, the parameter search unit 14 uses the Gaussian process regression model to search for an arbitrary D-dimensional parameter value x * The observed value y of the objective function corresponding to * The mean m(x * ) and standard deviation σ(x * ) The parameter search unit 14 calculates the average value m(x * ) and standard deviation σ(x * ) and based on the obtained function a(x * ) is calculated. The parameter search unit 14 calculates the acquisition function a(x * ) is the D-dimensional parameter value x that maximizes next is determined within an r-dimensional search space. next is determined as the D-dimensional parameter values ​​that are likely to minimize the observed value of the objective function.

[0049] When the parameter values ​​acquired from the data acquisition unit 13 are r-dimensional, the Gaussian process regression model can be used to calculate an arbitrary r-dimensional parameter value x r * The observed value y of the objective function corresponding to * The mean value m of the predictive distribution of r (x r * ) and standard deviation σ r (x r * ) can be calculated. Then, the average value m r (x r * ) and standard deviation σ r (x r * ) is calculated using the acquisition function a r (x r * ) in the r-dimensional search space. r next Calculate the r-dimensional parameter value x r nextis set as an r-dimensional parameter value that is likely to minimize the observed value of the objective function. Compared with the formula when the parameter value acquired from the data acquisition unit 13 is D-dimensional, there is a difference in whether or not there is a subscript r. Since the formula for the r-dimensional case can be obtained by adding the subscript r to the formula for the D-dimensional case, hereinafter, some explanations for the case when the parameter value is r-dimensional may be omitted.

[0050] The acquisition function may be, for example, PI (Probability of Improvement) or EI (Expected Improvement). Alternatively, UCB (Upper Confidential Bound), TS (Thompson Sampling), ES (Entropy Search), or MI (Mutual Information) may be used. In addition, the D-dimensional parameter value x that maximizes the acquisition function may be next Known methods for obtaining are, for example, full search, random search, grid search, Newton's method, L-BFGS, DIRECT, CMA-ES, multi-start local search, etc. The method for maximizing the acquisition function is not limited to these, and any method may be used.

[0051] Here, Gaussian process regression will be described in detail. N data sets acquired by the data acquisition unit 13 are expressed as {(x n ,y n )|n=0,1,…,N-1}. Gaussian process regression is a linear regression model for a vector of input parameter values ​​{x n A vector of observed values ​​of the objective function corresponding to |n=0,1,…,N-1} {y n |n=0,1,…,N-1} is a Gaussian distribution Norm(m, K+s 2 I), m is an N-dimensional mean vector. The i-th component of m is the mean function m0(x i ) where K is the sum of all input parameter values ​​x n is an N×N matrix that represents the covariance between the K-th (x i ,x j ) component is the kernel function k(x i ,x j) where the kernel function can be, for example, a Gaussian kernel, a Matern kernel, or a linear kernel. s is the standard deviation of the noise component when observing the objective function f(·). I is an N×N unit matrix. In addition, m0 is often treated as a 0 vector by appropriately processing the data. Therefore, in the following, we will use the Gaussian distribution Norm(0,K+s 2 The following description will be given assuming that the model is based on I).

[0052] Use a Gaussian process regression model to find any D-dimensional parameter value x * The observed value y of the objective function corresponding to * If you want to predict the distribution of observed value y * The mean m(x * ) is m(x * )=k(x * ,x 0:N-1 ) T (K+s 2 I) -1 y 0:N-1 It is calculated by and has a variance σ 2 (x * ) is σ 2 (x * )=k(x * ,x * )-k(x * ,x 0:N-1 ) T (K+s 2 I) -1 k(x * ,x 0:N-1 ) is calculated as follows: * K+s used in calculating the predictive distribution of 2 Inverse matrix of I(K+s 2 I) -1 is the inverse matrix of N rows and N columns, and the order of computation is O(N 3 ) is known. Similarly, when the parameter value is r-dimensional, the order of computational complexity is O(N 3 On the other hand, when using all of the N0 data sets stored in the data storage unit 11, rather than the N data sets acquired by the data acquisition unit 13, the order of the amount of calculation of the corresponding inverse matrix is ​​O(N0 3 ).

[0053] As with the data acquisition unit 13 according to this embodiment, by selecting N data sets to be used in the Gaussian process regression from among the N0 data sets stored in the data storage unit 11, the number of data sets to be used in the Gaussian process regression is reduced to K+s 2 Inverse matrix of I(K+s 2 I) -1 This reduces the amount of calculation for K+s 2 Inverse matrix of I(K+s 2 I) -1 This can solve the problem of the computational complexity. In addition, using a dataset near the r-dimensional search space as a dataset selection policy leads to preventing a deterioration in search efficiency. In other words, this embodiment can achieve both a reduction in the computational complexity and suppression of deterioration in search efficiency.

[0054] When using TPE instead of Gaussian process regression, in step SA2, the search space determination unit 12 sets r = 1. In step SA6, the parameter search unit 14 divides the data set into two groups, one where the observed value of the objective function is less than the threshold and the other where the observed value is equal to or greater than the threshold, based on a predetermined threshold, and calculates the r-dimensional parameter value x r The distribution of is modeled using kernel density estimation. The distribution of the group whose observed value of the objective function is smaller than the threshold is defined as l(x r ), and the distribution of the group with the observed value of the objective function greater than the threshold is g(x r ) Distribution l(x r ) and distribution (x r ) corresponds to the surrogate model of the objective function.

[0055] Next, in step SA7, the parameter search unit 14 calculates g(x r ) / l(x r ) is the r-dimensional parameter value x that minimizes r next is determined as the r-dimensional parameter value that is likely to minimize the observed value of the objective function. Even when TPE is used, the data set used is limited to the data set near the search space, so g(x r ) / l(xr ) can be reduced, which leads to preventing deterioration of search efficiency. Therefore, according to this embodiment, it is possible to achieve both reduction in the amount of calculation and suppression of deterioration of search efficiency.

[0056] If it is determined in step SA5 that one or more data sets have not been acquired (step SA5: NO), the data acquisition unit 13 calculates the D-dimensional parameter value x next In step SA8, the data acquisition unit 13 acquires, for example, the D-dimensional parameter value x exp The D-dimensional parameter values ​​x for calculating the observed values ​​of the objective function next The data acquisition unit 13 acquires the D-dimensional parameter value x next may be generated randomly from within the r-dimensional search space. next Using the same method, we can find the r-dimensional parameter value x r next may be obtained.

[0057] When step SA7 or SA8 is performed, the observation value output device 3 outputs the D-dimensional parameter value x next Specifically, the data generator 15 first outputs the observed value of the objective function based on the D-dimensional parameter value x next The data generator 15 notifies the observation value output device 3 of the r-dimensional parameter value x r next When searching or obtaining, the r-dimensional parameter value x r next Let x be the dimensional parameter value. next and notifies the observation value output device 3. The observation value output device 3 converts the D-dimensional parameter value x next Using this, a simulation, experiment, or survey is performed to obtain the D-dimensional parameter value x next The observed value of the objective function corresponding to is output.

[0058] For example, the observation value output device 3 includes a simulator / experimental device and a conversion device. The simulator / experimental device and the conversion device are realized by a computer. The simulator / experimental device converts D-dimensional parameter values ​​x next A simulation, an experiment, or a questionnaire is carried out at the observation output device 3, or the results of the simulation, the experiment, or the questionnaire are obtained outside the observation output device 3, and the D-dimensional parameter value x next The converter obtains the characteristic values ​​from the simulator / experimental device and converts the obtained characteristic values ​​into scalars. The characteristic values ​​after conversion into scalars are expressed as D-dimensional parameter values ​​x next The observed value of the objective function for y next The data is then transmitted to the data generating unit 15.

[0059] The simulator / experimental device and the conversion device may be realized by a single computer. In addition, either the simulator / experimental device or the conversion device may be incorporated in the parameter optimization device 2A, for example, included in the data generation unit 15.

[0060] When step SA9 is performed, the data storage unit 11 stores the D-dimensional parameter value x next and the observed values ​​output in step SA9 in the data storage unit 11 (step SA10). next and the observed value y output in step SA9 next and combine them to create a dataset (x next ,y next ) is generated. The data storage unit 11 then stores the generated data set (x next ,y next ) to remember.

[0061] When step SA10 is performed, the data acquisition unit 13 determines whether or not the first condition is satisfied (step SA11). The first condition specifies the number of times to repeat the parameter search in the r-dimensional search space determined by the search space determination unit 12. Therefore, the r-dimensional search space does not change during this repetition. The first condition may be satisfied, for example, when the number of times exceeds a predetermined number of times, when the elapsed time exceeds a predetermined time, or when the improvement amount of the observed value of the objective function falls below a predetermined value. Alternatively, it may be a combination of these conditions.

[0062] If it is determined that the first condition is not satisfied (step SA11: NO), data acquisition unit 13 executes the process of step SA4 for the same r-dimensional search space. Then, steps SA5 to SA11 are performed. Until it is determined in step SA11 that the first condition is satisfied, the processes of steps SA4 to SA11 of data storage unit 11, data acquisition unit 13, parameter search unit 14, and data generation unit 15 are repeated, and the D-dimensional parameter value x next Searching for or obtaining the D-dimensional parameter value x next and the observed value y next A dataset containing (x next ,y next The generation and storage of the data set (x, next ,y next ) is accumulated.

[0063] If it is determined in step SA11 that the first condition is satisfied (step SA11: YES), the search space determination unit 12 determines whether or not the second condition is satisfied (step SA12). The second condition specifies the number of times the search space determination unit 12 determines the r-dimensional search space. The value of r may be a predetermined value or a randomly determined value. The second condition may be satisfied, for example, when the number of times exceeds a predetermined number of times, when the elapsed time exceeds a predetermined time, or when the improvement amount of the observed value of the objective function falls below a predetermined value. Alternatively, it may be a combination of these conditions.

[0064] If it is determined that the second condition is not satisfied (step SA12: NO), the search space determination unit 12 selects a different r-dimensional parameter from the D-dimensional parameters (step SA2). In addition, the search space determination unit 12 selects a D-dimensional parameter value x that minimizes the observed value of the objective function from the data set stored in the data storage unit 11. exp As described above, when the D-dimensional parameters are divided into a plurality of groups of r-dimensional parameters in advance, the r-dimensional parameter may be selected from the plurality of groups in order or may be selected randomly. Alternatively, the r-dimensional parameter may be selected randomly from the D-dimensional parameters. The value of r may be set randomly within a range less than D. Thereafter, steps SA3 to SA12 are performed. Until it is determined in step SA12 that the second condition is satisfied, the processes from steps SA2 to SA12 of the data storage unit 11, the search space determination unit 12, the data acquisition unit 13, the parameter search unit 14 and the data generation unit 15 are repeated, and the D-dimensional parameter value x is calculated while changing the r-dimensional parameter, i.e., while changing the r-dimensional search space. next Searching for or obtaining the D-dimensional parameter value x next and the observed value y next A dataset containing (x next ,y nextThe generation and storage of a set of data (x, y, z) for a corresponding task is repeated. The series of processes from step SA2 to step SA12 may be executed in parallel or serially by a plurality of tasks. In this case, a data storage unit 11 is provided for each task, and each data storage unit 11 stores a plurality of data sets (x, y, z) for the corresponding task. next ,y next ) is accumulated.

[0065] When a series of processes from step SA4 to step SA11 are executed in parallel or serially by multiple tasks, after it is determined that the first condition is satisfied and before the process of step SA2 is performed, data generator 15 may acquire D-dimension parameter values ​​that minimize the observed value of the objective function from among multiple data sets stored in data storage unit 11 for each task. The acquired D-dimension parameter values ​​and the data set of observed values ​​corresponding to the D-dimension parameter values ​​are stored in data storage unit 11 for each task.

[0066] If it is determined in step SA12 that the second condition is satisfied (step SA12: YES), the optimal parameter value acquisition unit 16 selects the optimal parameter value x best (Step SA13) The optimal parameter value x best is output to an external device. In the manufacturing stage, the optimal parameter value x best For example, the parameter value that maximizes the yield during manufacturing or the parameter value that minimizes the failure rate after shipment. In the utilization stage, the optimal parameter value x best For example, the optimal parameter value x means the parameter value that allows the device or part delivered to the user to perform in a way that suits the user's usage environment. best is used for various purposes in the external device of the output destination. Note that the optimal parameter value x best may be stored in the data storage unit 11 or displayed on a display device.

[0067] In addition, when a series of processes from step SA2 to step SA12 is executed in parallel or in series by a plurality of tasks, after it is determined that the second condition is satisfied, the optimal parameter value acquisition unit 16 selects an optimal parameter value x that minimizes the observed value of the objective function from among a plurality of data sets stored in the data storage unit 11 for each task. best may be obtained.

[0068] This completes the processing operation of the parameter optimization system 1 according to the first embodiment.

[0069] As described above, according to the first embodiment, the data set used for calculating the surrogate model is limited to the data set near the search space, so that the amount of calculation of the surrogate model can be reduced compared to the conventional method. In addition, by utilizing a data set that is relatively close to the search space as the data set used for calculating the surrogate model, the parameter value x next In other words, it is possible to reduce the amount of calculation while suppressing the deterioration of the search efficiency, and the optimal parameter value x best This improves the search efficiency, in other words, the efficiency of parameter optimization.

[0070] Second embodiment Next, a parameter optimization system 1 and a parameter optimization device 2B according to the second embodiment will be described. In the following description, components having substantially the same functions as those in the first embodiment will be denoted by the same reference numerals, and will be described only when necessary.

[0071] 9 is a diagram showing an example of a functional configuration of a parameter optimization system 1 according to the second embodiment. As shown in FIG. 9, the parameter optimization system 1 according to the second embodiment has a parameter optimization device 2B and an observation value output device 3.

[0072] The parameter optimization device 2B has a data memory unit 11, a search space determination unit 12, a data acquisition unit 13, a parameter search unit 14, a data generation unit 15, an optimal parameter value acquisition unit 16, a dimension acquisition unit 17, a distance acquisition unit 18, a threshold acquisition unit 19, a display unit 20 and an external data acquisition unit 21.

[0073] The dimension acquiring unit 17 acquires the number of dimensions r input by a user via an input device etc. For example, the dimension acquiring unit 17 acquires the number of dimensions r inputted into a GUI (Graphical User Interface) screen (hereinafter referred to as a dimension input screen) for inputting the number of dimensions r displayed on the display unit 20.

[0074] The distance acquisition unit 18 acquires the type of distance input by the user via an input device, etc. For example, the distance acquisition unit 18 acquires the type of distance input on a GUI screen for inputting the type of distance displayed on the display unit 20 (hereinafter referred to as a distance type input screen).

[0075] The threshold acquisition unit 19 acquires a threshold for a distance from the r-dimensional search space, which is input by a user via an input device, etc. For example, the threshold acquisition unit 19 acquires a threshold input to a GUI screen for inputting a threshold displayed on the display unit 20 (hereinafter, referred to as a threshold input screen).

[0076] The display unit 20 displays a dimension input screen, a distance type input screen, a threshold input screen, etc. on a display device. The display device may be realized by, for example, a display or a projector.

[0077] The external data acquisition unit 21 acquires a plurality of data sets from a device external to the parameter optimization device 1B.

[0078] Next, a processing operation of the parameter optimization system 1 having the parameter optimization device 2B according to the second embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the processing operation of the parameter optimization system 1 according to the second embodiment.

[0079] 10, the dimension acquisition unit 17, the distance acquisition unit 18, and / or the threshold acquisition unit 19 acquire various setting values ​​according to instructions from a user via an input device (step SB1). The setting values ​​are a collective term for the number of dimensions r, the type of distance, and the threshold value.

[0080] In step SB1, the dimension acquisition unit 17 receives an instruction regarding the number of dimensions from the user and supplies it to the search space determination unit 12 and the display unit 20. For example, the dimension acquisition unit 17 receives an input instruction regarding the number of dimensions from the user on a dimension acquisition screen displayed on the display unit 20.

[0081] FIG. 11 is a diagram showing an example of the dimension acquisition screen I1. As shown in FIG. 11, the dimension acquisition screen I1 displays a message such as "Please input the number of dimensions r of the search space" that prompts input of the number of dimensions r, a display field I11 of the number of dimensions r, and an up-down button I12. The display field I11 displays the value of the number of dimensions r specified by the user. The up-down button I12 is pressed by the user via an input device to increase or decrease the value of the number of dimensions r. When the up-down button I12 is pressed, the display unit 20 increases or decreases the value of the number of dimensions r displayed in the display field I11 by a predetermined value unit. The value displayed on the display unit 20 is accepted by the dimension acquisition unit 17 as an input instruction for the number of dimensions.

[0082] Note that the dimension acquisition screen I1 shown in FIG. 11 is an example and can be modified in various ways. For example, the dimension acquisition screen I1 may display a text box for the user to directly input the number of dimensions r via an input device. When the user inputs a number representing the number of dimensions into the text box, an instruction for the number of dimensions to be accepted by the dimension acquisition unit 17 is output. Since r assumed in this embodiment is an integer value of 1 or more and less than D, the dimension acquisition unit 17 may restrict the input value so that a number or decimal less than 1 or equal to or greater than D cannot be input. In addition, the dimension acquisition unit 17 may display an error on the display unit 20 when a number or decimal less than 1 or equal to or greater than D is input.

[0083] The user does not necessarily have to input the number of dimensions. If there is no input instruction for the number of dimensions from the user, the dimension acquisition unit 17 supplies a default value for the number of dimensions to the search space determination unit 12 and the display unit 20. The default value may be set to, for example, "1". The default value may also be displayed in a text box.

[0084] In step SB1, distance acquisition unit 18 receives an instruction regarding the type of distance from the user, and supplies the instruction to search space determination unit 12 and display unit 20. For example, distance acquisition unit 18 receives an input instruction regarding the type of distance from the user on a distance type input screen displayed on display unit 20.

[0085] FIG. 12 is a diagram showing an example of a distance type input screen I2. As shown in FIG. 12, the distance type input screen I2 displays a message such as "Please select a distance type" that prompts input of a distance type, a distance type candidate display field I21, and a selection button I22. In the candidate display field I21, for example, Euclidean distance, Manhattan distance, cosine distance, Mahalanobis distance, etc. are displayed. The selection button I22 is provided corresponding to each of the multiple candidates displayed in the candidate display field I21. When the user presses the selection button I22 via an input device such as a mouse, the corresponding distance type is accepted by the distance acquisition unit 18 as an input instruction of the distance type.

[0086] 12 is an example and may be modified in various ways. For example, the distance type input screen I2 may display a text box for the user to directly input the type of distance via an input device. When the user inputs a character string representing the type of distance into the text box, an instruction for the type of distance to be accepted by the distance acquisition unit 18 is output.

[0087] The user does not necessarily have to input a distance type. If the user does not input a distance type, the distance acquisition unit 18 supplies a default value of the distance type to the data acquisition unit 13 and the display unit 20. The default value may be set to, for example, Euclidean distance. However, the default value may be set to Manhattan distance, cosine distance, or Mahalanobis distance. When the units, scales, and domains of definition are different for each of the D types of parameters, the Mahalanobis distance may be effective. As for the distance scale of the kernel function used in the Gaussian process regression, the Manhattan distance, cosine distance, or Mahalanobis distance may be adopted instead of the Euclidean distance.

[0088] In step SB1, the threshold acquisition unit 19 receives an instruction regarding the threshold from the user and supplies it to the data acquisition unit 13 and the display unit 20. The threshold acquired by the threshold acquisition unit 19 is a threshold for the distance from the search space. For example, the threshold acquisition unit 19 receives an instruction from the user to input a threshold on a threshold input screen displayed on the display unit 20.

[0089] FIG. 13 is a diagram showing an example of a threshold input screen I3. As shown in FIG. 13, the threshold input screen I3 displays a message prompting input of a threshold such as "Please input a threshold for the distance from the search space", a threshold display field I31, and an up-down button I32. The display field I31 displays the value of the threshold d specified by the user. The up-down button I32 is pressed by the user via an input device to increase or decrease the value of the threshold d. When the up-down button I32 is pressed, the display unit 20 increases or decreases the value of the threshold d displayed in the display field I31 by a predetermined value unit. The value displayed on the display unit 20 is accepted by the dimension acquisition unit 17 as an input instruction for the threshold d.

[0090] Note that the threshold input screen I3 shown in FIG. 13 is an example and can be modified in various ways. For example, the threshold input screen I3 may display a text box for the user to directly input the numerical value of the threshold d via an input device. When the user inputs a numerical value representing the threshold d into the text box, an instruction for the threshold d to be accepted by the threshold acquisition unit 19 is output. Since the threshold d assumed in this embodiment is a decimal number equal to or greater than 0.0, the threshold acquisition unit 19 may restrict the input value so that a negative value cannot be input. Furthermore, the threshold acquisition unit 19 may display an error on the display unit 20 when a negative value is input.

[0091] The user does not necessarily have to input a threshold value. If there is no instruction to input a threshold value from the user, the threshold acquisition unit 19 supplies the default value of the threshold value to the search space determination unit 12 and the display unit 20. The default value may be set to, for example, "0.0" or any positive value. The default value may also be displayed in a text box. If there is no instruction to input a threshold value from the user, the threshold value does not have to be a constant value as the default value, and may be changed automatically. The threshold value may be changed, for example, each time in a loop related to steps SB10, SB11, or SB12 described later. It may be changed according to the number of iterations of the loop. The threshold value may be set smaller as the number of iterations increases.

[0092] When step SB1 is performed, the external data acquisition unit 21 acquires N0 data sets {(x n ,y n )|n=0, 1, ..., N0-1} is acquired (step SB2). This allows the results of past simulations, experiments, questionnaires, etc. to be acquired. The acquired N0 data sets are stored in the data storage unit 11.

[0093] When step SB2 is performed, the search space determination unit 12 calculates the parameter value x exp (Step SB3) Set the parameter value x expis set to the D-dimensional parameter value corresponding to the smallest observed value among the N0 data sets stored in the data storage unit 11.

[0094] After step SB3 is performed, the search space determination unit 12 selects r-dimensional parameters that are less than D dimensions (step SB4). Step SB4 is substantially the same as step SA3. Note that the number of dimensions r is the one acquired by the dimension acquisition unit 17 in step SB1.

[0095] After step SB4 is performed, the data acquisition unit 13 acquires a data set within a predetermined distance from the r-dimensional search space (step SB5). Step SB5 is substantially the same as step SA4. The type of distance used is that acquired by the distance acquisition unit 18 in step SB1. The threshold value used is that acquired by the threshold acquisition unit 19 in step SB1. The acquired N data sets are divided into {(x n ,y n )|n=0,1,…,N-1}, where the value of N changes depending on the iteration of the loop for the first condition.

[0096] After step SB5 is performed, the parameter search unit 14 generates a surrogate model of the objective function based on one or more data sets (step SB6). Step SB6 is substantially the same as step SA6.

[0097] When step SB6 is performed, the parameter search unit 14 searches the r-dimensional search space using the surrogate model to find parameter values ​​x that are likely to minimize the observed value of the objective function. next Step SB7 is substantially the same as step SA7.

[0098] When step SB7 is performed, the observation value output device 3 outputs the D-dimensional parameter value x next Based on the objective function observation y next (Step SB8). Step SB8 is substantially the same as step SA9.

[0099] When step SB8 is performed, the data storage unit 11 stores the D-dimensional parameter value x next and the observed value y output in step SB8 next A dataset containing (x next ,y next ) is stored in data storage unit 11 (step SB9). Step SB9 is substantially the same as step SA10.

[0100] When step SB9 is performed, the data acquiring unit 13 determines whether or not the first condition is satisfied (step SB10). The first condition related to the second embodiment is defined as the number of iterations of the loop related to the first condition reaching a predetermined number.

[0101] If it is determined that the first condition is not satisfied (step SB10: NO), steps SB5 to SB10 are performed. Until it is determined in step SB10 that the first condition is satisfied, the processes from step SB5 to step SB10 of the data storage unit 11, data acquisition unit 13, parameter search unit 14, and data generation unit 15 are repeated, and the D-dimensional parameter value x next Searching for or obtaining the D-dimensional parameter value x next and the observed value y next A dataset containing (x next ,y next ) is generated and the process is repeated.

[0102] If it is determined in step SB10 that the first condition is satisfied (step SB10: YES), the search space determination unit 12 determines whether or not the second condition is satisfied (step SB11). The second condition related to the second embodiment is defined as the number of iterations of the loop related to the second condition reaching a predetermined number.

[0103] If it is determined that the second condition is not satisfied (step SB11: NO), steps SB3 to SB11 are performed. Until it is determined in step SB11 that the second condition is satisfied, the processes from step SB3 to step SB11 of the data storage unit 11, the search space determination unit 12, the data acquisition unit 13, the parameter search unit 14, and the data generation unit 15 are repeated. As a result, the D-dimensional parameter value x exp While changing the r-dimensional search space including the parameter value, we search for the D-dimensional parameter value x next Searching for or obtaining the D-dimensional parameter value x next and the observed value y next A dataset containing (x next ,y next ) is generated and the process is repeated.

[0104] If it is determined in step SB11 that the second condition is satisfied (step SB11: YES), the search space determination unit 12 determines whether or not the third condition is satisfied (step SB12). The third condition is, for example, defined as a condition on the number of iterations in steps SB3 to SB12. Specifically, it is defined as the number of iterations reaching a predetermined number. The predetermined number is set to an integer equal to or greater than 1.

[0105] If it is determined that the third condition is not satisfied (step SB12: NO), steps SB3 to SB12 are performed. Until it is determined in step SB12 that the third condition is satisfied, the processes from step SB3 to step SB12 of the data storage unit 11, the search space determination unit 12, the data acquisition unit 13, the parameter search unit 14, and the data generation unit 15 are repeated. As a result, the D-dimensional parameter value x exp While changing the r-dimensional search space including the parameter value, we search for the D-dimensional parameter value x next Searching for or obtaining the D-dimensional parameter value x next and the observed value y next A dataset containing (x next ,y next ) is generated and the process is repeated.

[0106] If it is determined in step SB12 that the third condition is satisfied (step SB12: YES), the optimal parameter value acquisition unit 16 selects, from the data set stored in the data storage unit 11, the optimal parameter value x best (Step SB13) The optimal parameter value x best is output to an external device.

[0107] This completes the processing operation of the parameter optimization system 1 according to the second embodiment.

[0108] As described above, according to the second embodiment, as in the first embodiment, the data set used for calculating the surrogate model is limited to the data set near the search space, so that the amount of calculation of the surrogate model can be reduced compared to the conventional method. In addition, the data set used for calculating the surrogate model is a data set that is relatively close to the search space, so that the parameter value x next In other words, it is possible to reduce the amount of calculation while suppressing the deterioration of the search efficiency, and the optimal parameter value x best This improves the search efficiency, in other words, the efficiency of parameter optimization.

[0109] Furthermore, according to the second embodiment, it becomes possible to set items such as the number of dimensions, the type of distance, and the threshold value according to user instructions using a GUI, etc. Therefore, it becomes possible to efficiently perform parameter optimization by utilizing the knowledge of the user.

[0110] (Modification) Hereinafter, some modified examples of the first embodiment and / or the second embodiment will be described. In the following description, components having substantially the same functions as those in the present embodiment will be given the same reference numerals, and will be described repeatedly only when necessary.

[0111] (Variation 1) In the above embodiment, the data acquisition unit 13 acquires a data set in the vicinity of the search space from a database stored in the data storage unit 11. The data acquisition unit 13 according to the first modification randomly generates D-dimensional parameter values ​​in the vicinity of the search space. The generated D-dimensional parameter values ​​are supplied to the data generation unit 15. The number of randomly generated D-dimensional parameter values ​​is represented by R, where R is an integer equal to or greater than 1. The generated R random D-dimensional parameter values ​​are summed up as {x rand,i It is expressed as |i=0,1,…,R-1}.

[0112] The data generator 15 calculates the D-dimensional parameter values ​​{x rand,i |i=0,1,...,R-1} is sent to the observation value output device 3.

[0113] Next, for example, the simulator / experimental device, which is an example of a component of the observation value output device 3, generates R D-dimensional parameter values ​​{x rand,i Conduct a simulation, experiment, or survey for each of |i=0,1,…,R-1} and {x rand,i For each of |i=0,1,...,R-1}, the device outputs characteristic values ​​for one or more items. The conversion device, which is an example of a component of the observation value output device 3, acquires one or more characteristic values ​​and converts them into scalars. The characteristic values ​​converted into scalars are expressed as D-dimensional parameter values ​​{x rand,i The observed values ​​of the objective function corresponding to |i=0,1,…,R-1} {y rand,i The observed values ​​{y rand,i |i=0, 1, . . . , R−1} is transmitted to the data generator 15.

[0114] The data generator 15 generates a data set {(x rand,i ,y rand,i )|i=0,1,...,R-1} to a data store 11, which stores the data set {(x rand,i ,y rand,i ) |i=0,1,…,R-1}.

[0115] Finally, the data acquisition unit 13 acquires, from the data stored in the data storage unit 11, a data set having, as elements, D-dimensional parameter values ​​whose distances to the search space are equal to or less than a threshold value.

[0116] The N data sets thus obtained are {(x n ,y n )|n=0,1,…,N-1} contains {(x rand,i ,y rand,i )|i=0, 1, ..., R-1} will be included. That is, the data acquiring unit 13 may acquire a data set having as elements D-dimensional parameter values ​​whose distance to the search space is equal to or less than a threshold from the data stored in the data storage unit 11, and may also randomly generate D-dimensional parameter values ​​whose distance to the search space is equal to or less than a threshold, and acquire a data set including the D-dimensional parameter values ​​and the corresponding observed values ​​of the objective function.

[0117] This allows us to easily add D-dimensional parameter values ​​whose distance to the search space is less than or equal to a threshold without Gaussian process regression, which also eliminates the need for matrix inversion.

[0118] (Variation 2) Regarding the second embodiment, the dimension acquisition unit 17, the distance acquisition unit 18, and the threshold acquisition unit 19 may operate at a timing different from that of step SB1.

[0119] The number of dimensions r acquired by the dimension acquisition unit 17 is used by the search space determination unit 12. That is, it is sufficient that the dimension acquisition unit 17 operates before the search space determination unit 12 operates.

[0120] The type of distance acquired by distance acquisition unit 18 is used by data acquisition unit 13. That is, distance acquisition unit 18 only needs to operate before data acquisition unit 13 operates.

[0121] The threshold value acquired by the threshold value acquiring unit 19 is used by the data acquiring unit 13. In other words, it is sufficient that the threshold value acquiring unit 19 operates before the data acquiring unit 13 operates.

[0122] The dimension acquisition unit 17, distance acquisition unit 18, and threshold acquisition unit 19 may be operated at a timing that allows the number of dimensions, type of distance, and threshold to be changed for each iteration of the loop related to the first condition, the second condition, and the third condition.

[0123] This makes it possible to accept instructions from the user at various times, and to change the number of dimensions, type of distance, and threshold value at various times.

[0124] (Variation 3) In the above-mentioned first and second embodiments, in FIGS. 3, 4, 5, 6, and 8, the r-dimensional affine subspace set in the search space is x exp has been shown an example in which the one-dimensional affine subspace passing through is controlled to be parallel to each different coordinate axis of the D-dimensional space depending on the number of iterations of the loop related to the second condition and / or the third condition. In this example, r is limited to 1, and the r-dimensional linear subspace associated with the r-dimensional affine subspace is controlled to include one of the coordinate axes. However, the method of controlling the r-dimensional affine subspace set in the search space is not limited to this.

[0125] The r-dimensional affine subspace, which is the search space, is determined by the number of dimensions r and an r-dimensional linear subspace associated with the r-dimensional affine subspace. Therefore, the r-dimensional affine subspace set in the search space can be controlled by controlling r and the r-dimensional linear subspace. In this modification, r is not limited to 1, and r may be changed within the range of integers equal to or greater than 1 in the loop related to the second and third conditions. r may be changed according to a rule according to a predetermined order or number of iterations in the loop related to the second and third conditions, or may be changed randomly. In addition, the r-dimensional linear subspace associated with the r-dimensional affine subspace is not limited to including any of the coordinate axes, and may be changed in the loop related to the second and third conditions. This r-dimensional linear subspace may be changed according to a rule according to a predetermined order or number of iterations in the loop related to the second and third conditions, or may be changed randomly.

[0126] For example, if you set r to 1 at some point, you can control the r(=1)-dimensional affine subspace to a straight line, and if you set r to 2 at some other point, you can control the r(=2)-dimensional affine subspace to a plane. If r is 1 and the r(=1)-dimensional linear subspace associated with the r(=1)-dimensional affine subspace is set so that it does not include any coordinate axis, then the r(=1)-dimensional affine subspace is x exp If r is 1, and an r(=1)-dimensional linear subspace associated with the r(=1)-dimensional affine subspace is set to include a certain coordinate axis, then the r(=1)-dimensional affine subspace is x exp If r is 2, and the r(=2)-dimensional linear subspace associated with the r(=2)-dimensional affine subspace is set so that it does not include any of the coordinate axes, then the r(=2)-dimensional affine subspace is x exp If r is 2, then if the r(=2)-dimensional linear subspace associated with the r(=2)-dimensional affine subspace contains only one coordinate axis, then the r(=2)-dimensional affine subspace is x exp If r is 2, then if we set the r(=2)-dimensional linear subspace associated with the r(=2)-dimensional affine subspace to include two coordinate axes, then the r(=2)-dimensional affine subspace is x exp and can be controlled to a plane parallel to those two coordinate axes.

[0127] If the r-dimensional linear subspace associated with the r-dimensional affine subspace does not include r coordinate axes, in step SA3 or step SB4, the search space determination unit 12 determines the r-dimensional affine subspace as described above, rather than determining the r-dimensional affine subspace by selecting an r-dimensional parameter having dimensions less than D. Once the r-dimensional affine subspace to be used as the search space has been determined, in the following step SA4 or step SB5, the data acquisition unit 13 calculates the projection length of the D-dimensional parameter value onto the r-dimensional affine subspace as the distance between the D-dimensional parameter value and the r-dimensional affine subspace.

[0128] (Variation 4) The D-dimensional parameter values ​​may be normalized or de-normalized by the parameter optimization devices 2A and 2B.

[0129] The D types of parameters to be adjusted often have different units, scales, and domains. Thus, the parameter optimization devices 2A and 2B may normalize the parameter values ​​so that the domain is, for example, from 0 to 1, or may denormalize the normalized parameters to return them to the original domain. Normalization may be performed by a normalization unit, and denormalization may be performed by an inverse normalization unit. The parameter optimization devices 2A and 2B may be provided with the normalization unit when performing normalization and the inverse normalization unit when performing denormalization. This makes it easier to simultaneously handle D types of parameters with different units, scales, and domains.

[0130] It should be noted that the normalization and / or denormalization may be performed outside the parameter optimization devices 2A, 2B, such as by the observation value output device 3 or other devices.

[0131] (Variation 5) The second condition according to the first embodiment and the third condition according to the second embodiment may be changed to a condition as to whether or not the absolute value of the difference (improvement amount) between the observed value of the objective function corresponding to the optimal parameter value acquired in the current iterative process by the optimal parameter value acquisition unit 16 and the observed value of the objective function corresponding to the optimal parameter value acquired in the immediately preceding iterative process is equal to or less than a predetermined threshold value. The observed value of the objective function corresponding to the optimal parameter value may be acquired from those stored in the data storage unit 11.

[0132] This makes it possible to terminate the process when the amount of improvement in the optimal parameter values ​​due to the iterative process becomes small, thereby enabling an efficient search for the optimal parameter values.

[0133] (Variation 6) In this embodiment, in order to optimize the D-dimensional parameter value, the search space is set to an r-dimensional affine subspace. Any D-dimensional parameter value belonging to the r-dimensional affine subspace can be expressed by an r-dimensional vector, and the r-dimensional vector can be inversely transformed to the original D-dimensional vector without degradation. When the distance threshold is 0, the D-dimensional parameter value acquired by the data acquisition unit 13 belongs to the r-dimensional affine subspace. Therefore, when the threshold is 0, the parameter search unit 15 according to the sixth modification executes the following processes as steps SA6 and SA7 and steps SB6 and SB7.

[0134] The parameter search unit 15 according to the sixth modification searches for a D-dimensional parameter value x n (n=0,1,…,N-1) is the r-dimensional parameter value x r n (n=0,1,…,N-1) to convert N datasets {(x r n ,y n )|n=0,1,…,N-1}. Then, we use the Gaussian process regression model to estimate the r-dimensional parameter values ​​x r The objective function value h(x r ) average m r (x r ) and standard deviation σ r (x r Here, the objective function h(·) is the r-dimensional parameter value x obtained by converting all D-dimensional parameter values ​​x belonging to the r-dimensional affine subspace into an r-dimensional vector. r With respect to h(x r )=f(x). r (x r ) and σ r (x r ) to find the r-dimensional parameter value x r Acquisition function a r (x r ) is the largest x r Any method can be used to maximize the acquisition function or the acquisition function. r Converted into D-dimensional parameter value x nextLet x be the next belongs to the r-dimensional affine subspace, i.e., the search space. next is supplied to the data generating unit 15.

[0135] As a result, the dimension of the parameters handled in the Gaussian process regression is reduced from D to r. Also, the dimension of the parameters of the kernel function used in the Gaussian process regression and the dimension of the parameters handled in the acquisition function are reduced from D to r. This makes it possible to reduce the amount of calculation. Note that when the distance threshold is a positive value, the D-dimensional parameter value acquired by the data acquisition unit 13 does not belong to the r-dimensional affine subspace, but the distance is equal to or less than the threshold and is not far. Therefore, the D-dimensional parameter value acquired by the data acquisition unit 13 may be projected into the r-dimensional affine subspace and processed in the same way as when the threshold is 0. Although there is degradation during inverse transformation, the degradation error is not large, so the same effect as when the threshold is 0 can be obtained.

[0136] (Variation 7) In the first and second embodiments, the parameter optimization devices 2A, 2B and the observation value output device 3 are separate computers. The observation value output device 3 according to the seventh modification is incorporated into the parameter optimization devices 2A, 2B. This makes it possible to automatically optimize parameters by a single device.

[0137] (Variation 8) The first embodiment, the second embodiment, Modification 1, Modification 2, Modification 3, Modification 4, Modification 5, Modification 6, and Modification 7 can be combined in any manner.

[0138] (Hardware configuration) Fig. 14 is a diagram showing an example of a hardware configuration of the parameter optimization devices 2A and 2B. As shown in Fig. 14, the parameter optimization devices 2A and 2B include a processing circuit 91, a main storage device 92, an auxiliary storage device 93, a display device 94, an input device 95, and a communication device 96. The processing circuit 91, the main storage device 92, the auxiliary storage device 93, the display device 94, the input device 95, and the communication device 96 are connected via a bus.

[0139] The processing circuit 91 executes a parameter optimization program read from the auxiliary storage device 93 to the main storage device 92, and functions as a search space determination unit 12, a data acquisition unit 13, a parameter search unit 14, a data generation unit 15, an optimal parameter value acquisition unit 16, a dimension acquisition unit 17, a distance acquisition unit 18, a threshold acquisition unit 19, and an external data acquisition unit 21. The main storage device 92 is a memory such as a RAM (Random Access Memory). The auxiliary storage device 13 is a hard disk drive (HDD), a solid state drive (SSD), a memory card, and the like. The main storage device 92 and the auxiliary storage device 93 function as a data storage unit 11.

[0140] The display device 94 displays various display information. The display device 14 is, for example, a display, a projector, etc. The display device 94 functions as a display unit 20.

[0141] The input device 95 is an interface for operating a computer. The input device 95 is, for example, a keyboard or a mouse. The display device 94 and the input device 95 may be configured with a touch panel. The communication device 96 is an interface for communicating with other devices such as the observation value output device 3.

[0142] The programs executed by the computer are provided as computer program products, recorded in an installable or executable format on a computer-readable storage medium such as a CD-ROM, memory card, CD-R, or DVD (Digital Versatile Disc).

[0143] The program executed by the computer may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the computer may be configured to be provided via a network such as the Internet without being downloaded.

[0144] The program executed by the computer may be provided by being pre-installed in a ROM or the like. The program executed by the computer has a modular configuration including functional blocks that can be realized by the program, among the functional configurations (functional blocks) of the parameter optimization devices 2A, 2B. As for each functional block, as actual hardware, the processing circuit 91 reads out the program from a storage medium and executes it, whereby each of the functional blocks is loaded onto the main storage device 92. In other words, each of the functional blocks is generated on the main storage device 92.

[0145] Some or all of the above-described functional blocks may be realized by hardware such as an IC (Integrated Circuit) instead of by software. When the functions are realized by using multiple processors, each processor may realize one of the functions, or two or more of the functions.

[0146] The operation form of the computer that realizes the parameter optimization devices 2A and 2B may be arbitrary. For example, the parameter optimization devices 2A and 2B may be realized by a single computer. In addition, for example, the parameter optimization devices 2A and 2B may be operated as a cloud system on a network.

[0147] (Additional remarks) As described above, the parameter optimization devices 2A and 2B include a data storage unit 11, a search space determination unit 12, a data acquisition unit 13, and a parameter search unit 14. The data storage unit 11 stores a plurality of data sets including D-dimensional parameter values ​​of a first dimension D and observed values ​​y of an objective function corresponding to the D-dimensional parameter values. The search space determination unit 12 determines an r-dimensional search space smaller than the D dimension including a predetermined D-dimensional parameter value. The data acquisition unit 13 acquires one or more data sets having D-dimensional parameter values ​​included in a predetermined distance d from the r-dimensional search space or a data set having r-dimensional parameter values ​​corresponding to the data sets based on the plurality of data sets. The parameter search unit 14 searches for D-dimensional parameter values ​​or r-dimensional parameter values ​​that can optimize the objective function in the r-dimensional search space using a proxy model of the objective function based on the acquired one or more data sets.

[0148] In the first embodiment, the predetermined D-dimension parameter value is a default value or a random value set in step SA1 in the first round, and in the second round and thereafter, is a D-dimension parameter value acquired in step SA2 and corresponding to the minimum observed value among the multiple data sets stored in data storage unit 11. In the second embodiment, it is a D-dimension parameter value set in step SB3 and corresponding to the minimum observed value among the multiple data sets stored in data storage unit 11.

[0149] According to the above configuration, in a search for D-dimensional parameter values ​​that can optimize the objective function, all data sets in the data storage unit 11 are not used, and a data set having D-dimensional parameter values ​​included at a distance d from the r-dimensional search space or a data set having r-dimensional parameter values ​​corresponding to the data set is used, thereby making it possible to reduce the amount of calculations involved in the search compared to a case where the data acquisition unit 13 is not provided. In addition, by using a data set close to the r-dimensional search space as described above, it is possible to suppress deterioration in search efficiency due to reduction in the number of data sets. In other words, according to this embodiment, it is possible to achieve both reduction in the amount of calculations and suppression of deterioration in search efficiency.

[0150] Thus, according to this embodiment, it becomes possible to efficiently search for optimal parameters.

[0151] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0152] 1...parameter optimization system, 2A, 2B...parameter optimization device, 3...observation value output device, 11...data storage unit, 12...search space determination unit, 13...data acquisition unit 13...parameter search unit, 15...data generation unit, 16...optimum parameter value acquisition unit, 17...dimension acquisition unit, 18...distance acquisition unit, 19...threshold acquisition unit, 20...display unit, 21...external data acquisition unit.

Claims

1. a storage unit configured to store a plurality of data sets including first parameter values ​​of a first dimension and observed values ​​of an objective function corresponding to the first parameter values; a search space determination unit that determines a search space having a second dimensionality smaller than the first dimensionality and including a predetermined first parameter value; an acquisition unit that calculates a distance between a first parameter value and the search space for each of the plurality of data sets, and acquires one or more data sets having a first parameter value for which the distance is equal to or smaller than a threshold, or a second parameter value of the second dimension corresponding to the first parameter value; a parameter search unit that searches for a first parameter value or a second parameter value that can optimize the objective function within the search space using a surrogate model of the objective function based on the acquired one or more data sets; a generator that generates a data set including an observed value of the objective function corresponding to the searched first parameter value or an observed value of the objective function corresponding to the searched second parameter value, and the searched first parameter value or the searched second parameter value; A determination unit that determines whether a predetermined condition is satisfied, The storage unit further stores the dataset generated by the generation unit; The determination unit determines whether or not the predetermined condition is satisfied after the data set is generated by the generation unit, When the predetermined condition is not satisfied, at least the processes by the acquisition unit, the parameter search unit, and the generation unit are repeated. Parameter optimizer.

2. The determination unit determines whether or not a first condition is satisfied as the predetermined condition after the data set is generated by the generation unit, if the first condition is satisfied, storing a minimum or maximum observed value among the observed values ​​and the first parameter values ​​stored in the storage unit, and the first parameter value corresponding to the observed value; The parameter optimization device according to claim 1.

3. The determination unit determines whether a second condition is satisfied as the predetermined condition after the data set is generated by the generation unit, if the second condition is not satisfied, storing a minimum or maximum observed value among the observed values ​​and the first parameter values ​​stored in the storage unit, and the first parameter value corresponding to the observed value; The parameter optimization device according to claim 1.

4. the determination unit, after the data set is generated by the generation unit, determines whether a first condition is satisfied as the predetermined condition, and if the first condition is satisfied, determines whether a second condition is satisfied as the predetermined condition; if the second condition is not satisfied, storing a minimum or maximum observed value among the observed values ​​and the first parameter values ​​stored in the storage unit, and the first parameter value corresponding to the observed value; The parameter optimization device according to claim 1.

5. The computer storing a plurality of data sets in a storage device, the data sets including first parameter values ​​of a first dimensionality and observed values ​​of an objective function corresponding to the first parameter values; a search space determination step of determining a search space of a second dimensionality less than the first dimensionality that includes a predetermined first parameter value; an acquisition step of calculating a distance between a first parameter value and the search space for each of the plurality of data sets, and acquiring one or more data sets having a first parameter value for which the distance is equal to or less than a threshold value or a second parameter value of the second dimension corresponding to the first parameter value; a parameter search step of searching for a first parameter value or a second parameter value that can optimize the objective function within the search space using a surrogate model of the objective function based on the acquired one or more data sets; a generating step of generating a data set including observed values ​​of the objective function corresponding to the searched first parameter values ​​or observed values ​​of the objective function corresponding to the searched second parameter values ​​and the searched first parameter values ​​or second parameter values; A determination step of determining whether a predetermined condition is satisfied, The storage step further stores the data set generated by the generation step; The determination step determines whether or not the predetermined condition is satisfied after the data set is generated by the generation step, If the predetermined condition is not satisfied, at least the obtaining step, the parameter searching step, and the generating step are repeated. Parameter optimization methods.

6. On the computer, a storage function for storing a plurality of data sets in a storage device, the data sets including first parameter values ​​of a first dimension and observed values ​​of an objective function corresponding to the first parameter values; a search space determination function for determining a search space having a second dimension smaller than the first dimension and including a predetermined first parameter value; an acquisition function that calculates a distance between a first parameter value and the search space for each of the plurality of data sets, and acquires one or more data sets having a first parameter value for which the distance is equal to or less than a threshold value or a second parameter value of the second dimension corresponding to the first parameter value; a parameter search function that searches for a first parameter value or a second parameter value that can optimize the objective function within the search space using a surrogate model of the objective function based on the acquired one or more data sets; a generation function for generating a data set including an observed value of the objective function corresponding to the searched first parameter value or an observed value of the objective function corresponding to the searched second parameter value, and the searched first parameter value or the searched second parameter value; A determination function for determining whether a predetermined condition is satisfied; The storage function further stores the data set generated by the generation function; the determination function determines whether or not the predetermined condition is satisfied after the data set is generated by the generation function; If the predetermined condition is not met, at least the obtaining function, the parameter searching function and the generating function are repeated. Parameter optimization program.

Images