Parameter vector value proposal device, parameter vector value proposal method, parameter optimization method, and parameter vector value proposal program

Abstract

To improve the efficiency of retrieving parameter vector values and reduce calculation costs.SOLUTION: A storage unit stores observation data which is the aggregate of a set of a parameter vector value that represents a point in a D(D≥2) dimension space and an observed value of the value of an objective function at the point. A search space determination unit determines an R(1≤R<D) dimension affine subspace passing through a point in the D dimension space that is represented by a prescribed parameter vector value, as a low-dimension search space. An extraction unit extracts the aggregate of a set corresponding to the one or more points that are included in the D dimension space represented by one or more parameter vector values included in the observation data that is stored in the storage unit, and the similarity of which to points included in the low-dimension search space is greater than or equal to a prescribed value, as extraction data. A proposition unit proposes the parameter vector value that represents a next point at which to observe the value of the objective function, on the basis of the extraction data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a parameter vector value proposing device, a parameter vector value proposing method, a parameter optimization method, and a parameter vector value proposing program. [Background technology]

[0002] In society, there are various devices, equipment, and application software that are composed of various parts. These devices, equipment, application software, and parts are designed, manufactured, and utilized.

[0003] In the design stage, devices, equipment, application software, and parts whose characteristics satisfy the specifications may be designed. In this case, a parameter vector having elements of one or more parameters that can be adjusted at the time of design is changed to various values, and simulations, experiments, and surveys are conducted to obtain characteristic values ​​that are numerically expressed as characteristics when designed with those parameter vector values, and a parameter vector value that satisfies the specifications is obtained. Here, the characteristics are, for example, the performance, manufacturing cost, and customer satisfaction of devices, equipment, application software, and parts. The better the performance of the equipment and parts, the better the manufacturing cost, and the higher the customer satisfaction. When the larger the characteristic value, the better, it is required to find a parameter vector value that maximizes the characteristic value with less time, effort, and cost. When the smaller the characteristic value, the better, it is required to find a parameter vector value that minimizes the characteristic value with less time, effort, and cost.

[0004] Finding the parameter vector values ​​that maximize or minimize the characteristic value is called parameter optimization. The characteristic value that changes according to the parameter vector value is called the objective function. Simulations, experiments, and surveys are means of observing the value of the objective function, i.e., the characteristic value. The characteristic value for each parameter vector value is not known until the characteristic value is observed by performing a simulation, experiment, or survey, and the objective function is unknown. In many cases, noise is added when observing the characteristic value, i.e., the value of the objective function.

[0005] Parameter optimization may also be used during the manufacturing stage, for example to find parameter vector values ​​that maximize the yield during manufacturing or minimize the failure rate after shipment.

[0006] Parameter optimization may also be used in the utilization stage. For example, a parameter vector value that maximizes the performance of a device, equipment, application software, or part delivered to a user in the user's environment may be calculated during initial setup by the user.

[0007] If the number of parameters to be adjusted is represented by D, the dimension of the parameter vector is D. A certain D-dimensional parameter vector value can be regarded as one point in the D-dimensional space. Therefore, the space in which the optimal D-dimensional parameter vector value is searched is the D-dimensional space. If no upper or lower limit is set for the D-dimensional parameter vector, the range in which the optimal D-dimensional parameter vector value is searched is the entire D-dimensional space. If an upper or lower limit is set for the D-dimensional parameter vector, that is, if the D-dimensional parameter vector has a domain, the range in which the optimal D-dimensional parameter vector value is searched is that domain in the D-dimensional space. The larger D is, the wider the search space and the search range become, making optimization more difficult. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent application No. 2020-185291 [Non-patent literature]

[0009] [Non-Patent Document 1] J. Kirschner, M. Mutny, N. Hiller, R. Ischebeck, and A. Krause, “Adaptive and safe Bayesian optimization in high dimensions via one-dimensional subspaces,” in Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 3429-3438, PMLR, 2019. Summary of the Invention [Problem to be solved by the invention]

[0010] Hereinafter, the D-dimensional parameter vector value may be abbreviated to simply the parameter vector value. Also, the description of the domain is omitted. Even if the description of the domain is omitted, the search range is limited to within the domain.

[0011] As a parameter optimization method, there is a method described in Non-Patent Document 1. This method is a Bayesian optimization method for cases where D is an integer equal to or greater than 2, and is known for its good search efficiency when D is large. In this method, the search space is limited to a one-dimensional space in a D-dimensional space, and while switching the one-dimensional search space, it iteratively proposes a point at which the value of the objective function should be observed next and observes the value of the objective function at the proposed point. Here, as described above, the point is a D-dimensional parameter vector value. The point at which the value of the objective function is observed is called the observation point.

[0012] In the proposal of observation points, an acquisition function is generated instead of the unknown objective function, and the point with the maximum value of the acquisition function is proposed as a candidate point that may minimize the value of the objective function. The acquisition function is calculated based on Gaussian process regression. In the following, Gaussian process regression is abbreviated as GP regression.

[0013] In GP regression, the objective function value at an unobserved point is predicted using one or more points where the objective function value has been observed and the observed values ​​of the objective function at those one or more points. In this case, it is necessary to calculate the inverse matrix.

[0014] The computation order of the inverse matrix is ​​O(N 3 ), where N represents the number of points at which the objective function value has been observed. As the number of proposal and observation iterations increases and N increases, the computational cost of the inverse matrix increases.

[0015] In contrast, the method of Patent Document 1 limits the points to be used for GP regression among the N points at which the objective function values ​​have been observed to those whose distance to a low-dimensional search space whose spatial dimension is lower than D is less than a predetermined threshold.

[0016] If the number of points in the limited result is represented as N', the computation order of the inverse matrix is ​​O(N' 3 ) This reduces the computational cost of matrix inversion.

[0017] However, the method of Patent Document 1 does not disclose a method for determining a predetermined threshold value for the distance to the low-dimensional search space. The prediction accuracy of the objective function value at each point by GP regression varies depending on which of the N points at which the objective function value has been observed is used. If points that have a large effect on the prediction accuracy are not used, the prediction accuracy will deteriorate. Therefore, if the points to be used for GP regression are determined by a uniform threshold value with a predetermined threshold value for the distance to the low-dimensional search space, the prediction accuracy may deteriorate. If the prediction accuracy deteriorates, the search efficiency of the parameter vector value is likely to deteriorate. It is putting the cart before the horse to deteriorate the search efficiency in order to reduce the calculation cost of the inverse matrix.

[0018] The problem that the present invention aims to solve is to provide a parameter vector value proposal device, a parameter vector value proposal method, a parameter optimization method, and a parameter vector value proposal program that improve the search efficiency of parameter vector values ​​and reduce calculation costs. [Means for solving the problem]

[0019] A parameter vector value proposal device according to an embodiment includes a memory unit that stores observation data, which is a set of pairs of parameter vector values ​​representing a point in a D (D is an integer equal to or greater than 2)-dimensional space and observed values ​​of an objective function value at the point; a search space determination unit that determines an R (R is an integer equal to or greater than 1 and less than D)-dimensional affine subspace that passes through a point represented by a predetermined parameter vector value in the D-dimensional space as a low-dimensional search space; an extraction unit that extracts, as extracted data, a set of pairs corresponding to one or more points, among one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the memory unit, whose similarity to a point included in the low-dimensional search space is equal to or greater than a predetermined value; and a proposal unit that proposes a parameter vector value representing a point at which the value of the objective function will next be observed, based on the extracted data. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram showing an example of the functional configuration of a parameter optimization system according to an embodiment of the present invention; [Diagram 2] FIG. 1 is a diagram showing the flow of a parameter optimization process performed by the parameter optimization system according to the present embodiment. [Diagram 3] FIG. 3 is a diagram showing a pseudo program code of the process by the parameter vector value proposing device in the parameter optimization process shown in FIG. 2. [Figure 4] A diagram showing the observed states of the objective function at seven observation points in a D-dimensional space when D=2. [Diagram 5] FIG. 23 is a diagram showing an example of a functional configuration of a parameter optimization system according to a ninth modification. [Figure 6]FIG. 3 is a diagram showing a pseudo program code according to a tenth modification, which corresponds to the parameter optimization process shown in FIG. [Figure 7] FIG. 1 is a diagram showing an example of the hardware configuration of a parameter vector value proposing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, a parameter vector value proposing device, a parameter vector value proposing method, a parameter optimization method, and a parameter vector value proposing program according to the present embodiment will be described with reference to the drawings.

[0022] FIG. 1 is a diagram showing an example of a functional configuration of a parameter optimization system 1 according to the present embodiment. As shown in FIG. 1, the parameter optimization system 1 is a computer system having a parameter vector value proposal device 100 and an observation device 200. The parameter vector value proposal device 100 and the observation device 200 are connected to each other so as to be able to communicate with each other via wire or wirelessly. The parameter vector value proposal device 100 is a computer that proposes a parameter vector value (proposal point) at which the value of the objective function should be observed next. The observation device 200 obtains an observation value of the objective function at the proposal point by observing the value of the objective function at the proposal point. Specifically, the observation is performed by a simulation based on the parameter value, an experiment, a questionnaire, or the like. The parameter optimization system 1 repeats the proposal of a proposal point by the parameter vector value proposal device 100 and the acquisition of an observation value of the objective function at the proposal point by the observation device 200, and outputs a parameter vector value (observation point) corresponding to the minimum observation value to the outside as an optimal point. Hereinafter, the observation value of the objective function may be abbreviated to simply an observation value.

[0023] Parameter optimization may involve maximizing or minimizing the value of an objective function. Maximization is equivalent to a minimization problem by multiplying the value of the objective function by -1. For ease of explanation, the following description will be given of a case in which a parameter vector value that minimizes the value of the objective function is obtained. However, the parameter optimization of this embodiment is not limited to the case of minimization. The parameter optimization of this embodiment can also be applied to a problem of maximizing the value of an objective function.

[0024] As shown in FIG. 1, a parameter vector value proposing device 100 includes a storage unit 101, a search space determining unit 102, an extracting unit 103, a proposing unit 104, and a control unit 105.

[0025] The storage unit 101 stores a set of pairs of D-dimensional parameter vector values ​​and observed values ​​of an objective function corresponding to the D-dimensional parameter vector values. The data of the set is called observed data. The parameter vector values ​​represent points in a D-dimensional space (D is a natural number equal to or greater than 2). The observed values ​​are obtained by the observation device 200 using simulations, experiments, questionnaires, etc., based on the corresponding D-dimensional parameter vector values.

[0026] The search space determination unit 102 determines an R-dimensional (R is an integer between 1 and D) affine subspace that passes through a point represented by a predetermined parameter vector value in the D-dimensional space as a low-dimensional search space. Observation data The best observation among the observations, e.g., the minimum The parameter vector value corresponding to This observed value is called the best observed value.

[0027] The extraction unit 103 extracts, as extracted data, a set of pairs corresponding to one or more points in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the memory unit 101, the set of pairs corresponding to one or more points whose similarity to a point included in the low-dimensional search space is equal to or greater than a predetermined value.

[0028] The proposing unit 104 proposes a parameter vector value representing a point (parameter vector value) at which the value of the objective function will be observed next based on the extracted data. This point is called a proposed point. The value of the objective function at the proposed point is observed by the observation device 200, and an observed value corresponding to the proposed point is obtained. A pair of the proposed point (parameter vector value) and the observed value corresponding to the proposed point is stored in the storage unit 101.

[0029] The control unit 105 performs overall control of the parameter vector value proposal device 100. Specifically, the control unit 105 controls the storage of observation data by the storage unit 101, the determination of the search space by the search space determination unit 102, the extraction of extracted data by the extraction unit 103, and the proposal of the proposal point by the proposal unit 104, so as to be repeated until a termination condition is satisfied, in response to the acquisition of the observation value by the observation device 200. In order to perform control in response to the acquisition of the observation value by the observation device 200, the control unit 105 has a function of monitoring the acceptance of the observation value by the storage unit 101 and a function of monitoring the transmission of the proposal point by the proposal unit 104 to the outside of the parameter vector value proposal device 100. The optimal point (parameter vector value) at the end of the repetition is called the optimal point. The optimal point is provided by the control unit 105 to an external device different from the parameter vector value proposal device 100.

[0030] Fig. 2 is a diagram showing a flow of a parameter optimization process by the parameter optimization system 1 according to this embodiment. Fig. 3 is a diagram showing a pseudo program code of a process by the parameter vector value proposing device in the parameter optimization process shown in Fig. 2. The parameter optimization shown in Figs. 2 and 3 is executed under the control of the control unit 105 over the storage unit 101, the search space determination unit 102, the extraction unit 103, and the proposing unit 104.

[0031] As shown in Fig. 2, first, the control unit 105 executes initialization of the parameter vector value proposal device 100 (S201). At the start of S201, the control unit 105 sets time t to 0 as shown in S301 in Fig. 3, and at the end of S201, sets time t to 1. Time t is a time used for parameter optimization processing, and indicates the number of times the processing is performed in the processing loop in Fig. 2.

[0032] Also, in S201, the control unit 105 initializes the storage unit 101 and sends observation data, which will be described later, to the extraction unit 103. As the initialization, at least one pair of a D-dimensional parameter vector value and an observation value of an objective function corresponding to the D-dimensional parameter vector value is stored in the storage unit 101.

[0033] Not limited to S201, the number of pairs stored in the storage unit 101 as a result of performing the process of storing in the storage unit 101 at time t is N t From this definition, the number of pairs stored in the storage unit 101 after one or more pairs are stored in S201 at time 0 is N0, which is an integer equal to or greater than 1.

[0034] At time 0, N0 D-dimensional parameter vector values ​​stored in the storage unit 101 are expressed as x n (n=0,1,…,N0-1), and x n The observed values ​​of the objective function for y n (n=0,1,…,N0-1). Note that the D-dimensional parameter vector value x n is a vector, and the observations y n is a scalar. The observed value y of the objective function for the D-dimensional parameter vector value x is expressed as y = f(x) + ε. Here, f represents the objective function, and ε represents the noise component when observing the value of the objective function. ε follows, for example, a Gaussian distribution with a mean of 0 and a standard deviation of σ. When considering the case where there is no noise component, σ can be considered to be 0.

[0035] Not only S201, but also N stored in the storage unit 101 at time t t A set of pairs D t is expressed by the following formula (1). t is called the observed data at time t.

[0036]

number

[0037] The observed data D0 stored in the storage unit 101 after initialization in S201 at time 0 is represented by the following equation (2), as shown in S302 of FIG. 3. In S301, the observed data D0 is supplied from the storage unit 101 to the extraction unit 103.

[0038]

Number

[0039] The observed data D0 may be composed only of data that has been observed before S201, or each x may be randomly determined within the domain for S201 n (n = 0, 1, …, N0 - 1), and y corresponding to each x n may be observed by the observation device 200, or they may be a mixture of them. n As shown in S303 of FIG. 3, for times t = 1, 2, …, T, the subsequent S202 to S207 are repeated. T is a preset upper limit value of the time t.

[0040] When S201 is performed, the search space determination unit 102 determines the search space (S202). Specifically, in S202, the search space determination unit 102 determines the low-dimensional search space S

[0041] t and supplies it to the extraction unit 103. The number of dimensions of the low-dimensional search space S t is represented by R t . R t is an integer having a number of dimensions of 1 or more and less than D dimensions, and is an integer satisfying 1 ≤ R t < D. The value of R t may be changed according to the time t, or may be constant regardless of the time t. The value of R t may be a predetermined value or a randomly determined value.

[0042] To determine the low-dimensional search space S t , the search space determination unit 102 acquires the best observation point from the storage unit 101. Here, the best observation point is the observed data D stored in the storage unit 101t-1 The set of observations in {y n |n=0,1,…,N t-1 −1}. This minimum observation is called y bt-1 and y bt-1 The best observation point corresponding to x bt-1 It is expressed as b t-1 As shown in S304 of FIG. 3, is an index expressed by the following formula (3).

[0043]

number

[0044] In S202, the search space determination unit 102 determines the best observation point x bt-1 A low-dimensional search space S through t Here, we determine the low-dimensional search space S t is R t dimensional affine subspace. t is expressed by the following formula (4). x bt-1 is S t is the position vector of U t is R t dimensional affine subspace S t R t It is a dimensional linear subspace.

[0045]

number

[0046] low-dimensional search space S t , i.e., R t dimensional affine subspace S t changes according to time t in the loop of Figure 2. x bt-1 or U t changes depending on time t, so S t As described later, the observation data is updated as time progresses, so the best observation point x bt-1 may also change depending on time t.t is R t changes depending on time t. R t Even if U is a constant value regardless of time t, t changes by changing the direction of the linear subspace according to time t.

[0047] After S202, the extraction unit 103 executes the extraction process (S203). In S203, the extraction unit 103 extracts the observation data D t-1 The set of observation points {x n |n=0,1,…,N t-1 -1}, the low-dimensional search space S t The similarity to a given point x' in t A pair corresponding to one or more points that satisfy the above conditions is extracted from the storage unit 101. The extracted pair is supplied to the proposing unit 104. Note that a query sent by the extraction unit 103 to the storage unit 101 is not shown in FIG.

[0048] low-dimensional search space S t For example, the observation point x n (n=0,1,…,N t-1 Regardless of -1) S t The same point in is used. Observation point x n The similarity of k(x n,x´), where k(·,·) is a kernel function that evaluates the similarity between two points. Known kernel functions include the linear kernel, squared exponential kernel, exponential kernel, Matern 3 / 2 kernel, Matern 5 / 2 kernel, rational 1uadratic kernel, ARD squared exponential kernel, ARD exponential kernel, ARD Matern 3 / 2 kernel, ARD Matern 5 / 2 kernel, and ARD Rational Quadratic kernel. Any of these kernel functions may be used as the kernel function, or a different kernel function may be used. The kernel function may include hyperparameters. The values ​​of the hyperparameters may be predetermined values ​​or may be estimated from observed data or extracted data, which will be described later.

[0049] As shown in S307 of FIG. 3, at time t, the extraction unit 103 extracts the observed data D t-1 A set of pairs to be extracted from E t Extract E t is called the extracted data. As shown in S307, the extracted data E t is expressed by the following equation (5).

[0050]

number

[0051] Extracted Data E t The number of elements in t It is expressed as T t Depending on the value of N´ t varies from 0 to N t-1 It takes the following values: T t As for N' t Use a value that is 1 or more. For example, N t-1 k(x n ,x´)(n=0,1,…,N t-1 -1), and then calculate one of them, k(x n,x´) to T t If you set it to N´ t is guaranteed to be greater than or equal to 1. t N t-1 When it is equal to E t D t-1 From now on, N´ t is 1 or more N t-1 The following description will be given assuming the above.

[0052] The given point x' is the observation point x n (n=0,1,…,N t-1 Regardless of -1) S t Instead of taking the same point in n For example, different points may be used for each observation point x. n Let us consider the low-dimensional search space S t The point orthogonally projected onto the target point may be used as the specified point x'. In this case, the specified point x' is expressed by the following equation (6). Here, P St is a method for dividing points in the D-dimensional space into a low-dimensional search space S t This is a function that returns the orthogonal projection of a point onto a point.

[0053]

number

[0054] Function P St is expressed by the following equation (7). D represents a unit matrix with D rows and D columns. P Ut is R t dimensional linear subspace U t Represents the orthogonal projection matrix onto

[0055]

number

[0056] Even in this case, N´ t T where is 1 or more t In this case, the extraction unit 103 sets the observation data D t-1 Extracted from N´ tA set of observed data E t is expressed by the following equation (8). In this case, E t Then, E shown in S307 of Figure 3 t It is possible to replace

[0057]

number

[0058] For a given point x', we define a low-dimensional search space S t We do not explicitly define the position in the low-dimensional search space S t It is also possible to define a point x' in N'. t T where is 1 or more t We can set the low-dimensional search space S t For point x' included in n ,x´) is T t This means that the low-dimensional search space S t The similarity k(x n ,x´), that is, the maximum similarity is T t Since this is equivalent to the above, the following equation (9) is established. In this case, E t Then, E shown in S307 of Figure 3 t That is, the extraction unit 103 can replace the observation data D t-1 One or more points in the D-dimensional space represented by one or more parameter vector values ​​in x n Of these, the low-dimensional search space S t All points in x´ Similarity to k(x n , x´) is the maximum similarity that is the maximum value of a given value T t Extract the set of pairs corresponding to one or more points that are equal to or greater than E t Extract as.

[0059]

number

[0060] Depending on the kernel function, such as when the kernel function is a squared exponential kernel function or an ARD squared exponential kernel function, the following equation (10) may hold. In this case, E t Then, E shown in S307 of Figure 3 t It is possible to replace

[0061]

number

[0062] 2 points x i and x j The squared exponential kernel function k(x i ,x j ) is expressed by the following equation (11). σ ,θ l are hyperparameters called the signal standard deviation and the length scale, respectively. θ σ ,θ l Each must have a value greater than 0.

[0063]

number

[0064] From the definition shown in equation (11), the following equation (12) is established.

[0065]

number

[0066] 2 points x i and x j ARD squared exponential kernel function k(x i ,x j) is expressed by the following equation (13). Here, θ l is a hyperparameter and represents a D-dimensional scale length vector whose elements are the scale lengths for each coordinate axis direction in the D-dimensional space. [d] (d=0,1,…,D-1) represents the dth element of the vector.

[0067]

number

[0068] From the definition shown in equation (13), the following equation (14) is obtained.

[0069]

number

[0070] After S203, the proposing unit 104 executes the proposing process (S204). In S204 at time t, the proposing unit 104 receives the extracted data E t , the point at which the value of the objective function should be observed next is proposed, sent to the storage unit 101, and output to the outside of the parameter vector value proposal device 100. t-1 The observation points included in Nt-1-1 Therefore, the index of the next point to be observed is N t-1 is adopted, and the point proposed by the suggestion unit 104 is x Nt-1 This is expressed as x Nt-1 is called the proposal point.

[0071] As shown in S308 of FIG. 3, the proposing unit 104 constructs an acquisition function from a surrogate model that captures the characteristics of the unknown objective function, and creates a low-dimensional search space S t The point with the largest value of the acquisition function is proposed as x Nt-1 Set the proposed point x Nt-1 is expressed by the following equation (15). t (x|E t ) is the extracted data E tis an acquisition function defined from a surrogate model based on

[0072]

number

[0073] Here, as the surrogate model, for example, a GP regression model or a random forest model is adopted. t For example, Lower Confidence Bound (LCB), Expected Improvement (EI), Probability of Improvement (PI), Mutual Information (MI), Predictive Entropy Search (PES), Max-value Entropy Search (MES), etc. are used.

[0074] low-dimensional search space S t The point with the largest value of the acquisition function in is, for example, S t Set multiple points in and use the acquisition function a t It can be found by selecting the point with the largest value of , or by using an optimization method such as the L-BFGS algorithm.

[0075] After S204, the observation device 200 executes the observation process (S205). In S205 at time t, the observation device 200 receives the proposed point x Nt-1 Get the proposal point x Nt-1 Based on observed value y Nt-1 Observe the observed value y Nt-1 is supplied to the parameter vector value proposal device 100 via the network.

[0076] Observed value y Nt-1 is the proposed point x Nt-1 As shown in S309 of FIG. 3, the observed value y Nt-1 is stored in the storage unit 101. Nt-1 is expressed by the following equation (16).Nt-1 is the observed value y at time t Nt-1 This represents the noise component contained in

[0077]

number

[0078] After S205, the storage unit 101 executes an update process (S206). In S206 at time t, the storage unit 101 updates the observation point x Nt-1 and the observation value y Nt-1 D t-1 Observation data added to D t The observation data D t As shown in S310 of Fig. 3, is expressed by the following formula (17): ∪ represents the union of two sets.

[0079]

number

[0080] By adding this pair, the number of elements of the observation data increases by one. Therefore, the number of pairs stored in the storage unit 101 at time t is N t and the number of pairs N stored in the storage unit 101 at time t-1. t-1 Regarding the above, as shown in S311 of FIG. 3, the following formula (18) holds.

[0081]

number

[0082] When S206 is performed, the control unit 105 executes a judgment process (S207). In S207 at time t, the control unit 105 judges whether the processes from S202 to S206 have been repeated a predetermined number of times T. If the time t is less than T (S207: NO), the control unit 105 increments the time t and returns to S202. Then, until the time t reaches T, the processes from S202 to S207 are repeated as shown in the for statements in S303 and S312 in FIG. 3.

[0083] If the time t has reached T (S207: YES), the control unit 105 executes the output process (S208). The time in S208 is T. In S207 at the time T, the control unit 105 outputs the observation data D T The observation point corresponding to the smallest observation value among them is determined as the optimal point and output to an external device of the parameter vector value proposal device 100. T The index b of the smallest observation in T As shown in S313 of FIG. 3, is expressed by the following equation (19).

[0084]

number

[0085] The optimal points to be output to the outside of the parameter vector value proposing device 100 are set D T The smallest observed value y in bt The corresponding observation point x Nt It is.

[0086] When S208 is performed, the parameter optimization process by the parameter optimization system 1 ends.

[0087] The effects of this embodiment will be described.

[0088] As a preparation, we will explain how to utilize GP regression in standard Bayesian optimization methods and the method in Non-Patent Document 1. Observation data D t-1is utilized, and the expected value E of the predicted value of the objective function f at point x by GP regression is expressed by the following equation (20). T represents the transpose of a vector or matrix. K is the number of N t-1 Row N t-1 is a column matrix, whose element K [r,c] (r,c=0,1,...,N t-1 -1) is k(x r ,x c ). [r,c] represents the element in the rth row and cth column of the matrix. σ represents the standard deviation of the noise component. I is N t-1 Row N t-1 Represents the column identity matrix. -1 represents the inverse matrix of a matrix. y=(y0,y1,...,y Nt-1 -1) T It is.

[0089]

number

[0090] In addition, observation data D t-1 is utilized, and the variance V of the predicted value by GP regression of the objective function f at point x is expressed as follows (21).

[0091]

number

[0092] For example, the acquisition function a t If we adopt LCB as a t is expressed by the following equation (22): κ is a parameter that determines the balance between exploration and exploitation.

[0093]

number

[0094] In this way, in the method of Non-Patent Document 1, the acquisition function used to determine the proposed points is defined based on GP regression. t Even when a method other than LCB is adopted as the method, the acquisition function used to determine the proposal points is defined based on GP regression.

[0095] In the method of Non-Patent Document 1, if the type of acquisition function used is the same, the formula for the acquisition function is the same as that of the standard Bayesian optimization method. However, in the Bayesian optimization method of Non-Patent Document 1, t This differs from standard Bayesian optimization methods in that the search for the point where is maximized is in a one-dimensional search space rather than a D-dimensional space.

[0096] The expected value E of the predicted value can be transformed into the following equation (23). ((K+σ 2 I) -1 y) [n] is the observation data D t-1 is a constant determined by k(x,x n ) depends on the point x.

[0097]

number

[0098] k(x,x) shown in Eq. (23) n ) to a constant ((K+σ 2 I) -1 y) [n] If we interpret it as the weight for k(x,x n ) is the smallest absolute value of the observation point x n It can be seen that k(x,x n ) is a similarity and cannot take a negative value, so the similarity k(x,x n ) is small at observation point x n has a small contribution to the expected value E of the predicted value. Also, the similarity k(x,x n ) is small at observation point x n The contribution of to the variance V of the predicted value shown in equation (21) is also small.

[0099] In this embodiment, the points at which the value of the acquisition function needs to be calculated are the points in the low-dimensional search space S t Therefore, the low-dimensional search space S t Similarity k(x',x n ) is small at observation point x n is the expected value of the forecast value E, the variance of the forecast value V, and the acquisition function a t The contribution to is small.

[0100] Extracted data E of this embodiment t is the observation data D t-1 From the similarity k(x',x n ) is T t More than x n The extracted data E t is expressed by the following equation (24).

[0101]

number

[0102] Therefore, the extracted data E t is the expected value of the forecast value E, the variance of the forecast value V, and the acquisition function a t is a set of pairs corresponding to observation points with a large contribution of

[0103] In this embodiment, the expected value E of the predicted value, the variance V of the predicted value, and the acquisition function a t are expressed as (25), (26), and (27) below, respectively.

[0104]

number

[0105] For example, if LCB is adopted as the acquisition function, the acquisition function a t is expressed by the following equation (28).

[0106]

number

[0107] Even if an acquisition function other than LCB is adopted, the acquisition function a used to determine the proposal point is t is defined based on the expected value E and variance V of the forecast period.

[0108] In this manner, in this embodiment, the expected value E of the predicted value, the variance V of the predicted value, and the acquisition function a t is the similarity k(x,x n ) is large and has a high contribution from observation point x n Extracted data E consisting of pairs corresponding to t Since the approximation is performed using the above, the accuracy of the approximation is high and the degradation due to the approximation is small.

[0109] In the method of Patent Document 1, the expected value E of the predicted value, the variance V of the predicted value, and the acquisition function a t are approximated by (29), (30), and (31), respectively. Here, Ft={(x n ,y n )|dist(S t ,x n )≦A,n=0,1,···,N t-1 -1}, and dist(S,x) is a function that returns the distance between the space S and the point x. A represents a threshold value for the distance.

[0110]

number

[0111] space S t and point x n Distance dist(S t ,x n ) is small and the point x(∈S t ) and point x n The similarity k(x,x n ) is high, it does not necessarily mean that Ft is the observation point x with high contribution. nTherefore, the method of Patent Document 1 does not necessarily suffer from small degradation due to approximation, and does not necessarily have high approximation accuracy. Therefore, the method of Patent Document 1 does not necessarily have good search efficiency.

[0112] In the method of Non-Patent Document 1, the observation data D t-1 In order to use (K+σ 2 I) -1 The calculation order of is D t-1 Number of elements in N t-1 depends on O(N t-1 3 On the other hand, in this embodiment, the extracted data E t-1 In order to utilize ~ +σ 2 I ~ ) -1 The calculation order of E t Number of elements in N t ´, and O(N t ´ 3 ) 1≦N t ´≦N t-1 Therefore, the calculation cost of the inverse matrix in this embodiment is equal to or less than the calculation cost of the inverse matrix in the method of Non-Patent Document 1. t Depending on the value of t ´ <N t-1 In this case, the calculation cost of the inverse matrix in this embodiment is smaller than the calculation cost of the inverse matrix in the method of Non-Patent Document 1.

[0113] In this manner, in this embodiment, the expected value and variance of the predicted value and the acquisition function are approximated with high accuracy, and the calculation cost of the inverse matrix in the GP regression is low. Therefore, this embodiment can reduce the calculation cost of the inverse matrix in the GP regression as much as possible without deteriorating the search efficiency of the parameter vector value as much as possible.

[0114] The effect of this embodiment is not limited to reducing the calculation cost of the inverse matrix in GP regression as much as possible without deteriorating the search efficiency of parameter vector values ​​as much as possible, but also has the effect of improving the search efficiency of parameter optimization in some cases.

[0115] FIG. 4 is a diagram showing a state in which the values ​​of the objective function have been observed at seven observation points in a D-dimensional space when D=2. In the graph 41 on the left side of FIG. 4, the horizontal axis corresponds to the 0th element of the D-dimensional parameter vector, and the vertical axis corresponds to the 1st element. The axis in the depth direction corresponds to the value of the objective function. Each of the seven points represents the position of an observation point in the D-dimensional space. The dashed lines indicate the low-dimensional search space S t The shading of the ellipse represents the value of the unknown objective function f, with black representing a small objective function value and white representing a large objective function value.

[0116] For the graph 42 on the right side of Figure 4, the horizontal axis is the x n For a given point x´(∈S t ) to the distance ||x n -x´||, and the vertical axis is x n Similarity between and x' k(x n ,x´). The solid curves represent squared exponential kernel functions with small scale lengths, and the dashed curves represent squared exponential kernel functions with large scale lengths.

[0117] As shown in Fig. 4, by using a squared exponential kernel function with a large scale length, n The similarity of the observation points x becomes relatively uniform, and by using a squared exponential kernel function with a small scale length, n It can be seen that the degree of similarity varies considerably depending on the

[0118] The objective function f in Fig. 4 takes small values ​​locally, so it is limited to the low-dimensional search space S t An observation point x located farther away from the local minimum n is S t Therefore, in this example of Figure 4, it is preferable to have a small scale length for the squared exponential kernel function.

[0119] As described above, the hyperparameters of the kernel function are either predetermined values ​​or values ​​estimated from observed data or extracted data (described later). Therefore, the optimal values ​​for the shape of the objective function are not necessarily adopted.

[0120] When the scale length is larger than the optimal value, the prediction accuracy of the GP regression is low, and the search efficiency of the parameter optimization is poor. t-1 Extracted data E of t Only the observation points with low similarity are treated, i.e., the similarity of the observation points is forcibly replaced with 0. Therefore, the behavior of the GP regression becomes closer to the behavior when the scale length is reduced and the optimal scale length is adopted. As a result, the observed data D t-1 Extracted data E t In some cases, the efficiency of parameter optimization search may be improved by using only the

[0121] In this way, in this embodiment, there are cases where it is possible to improve the search efficiency of parameter vector values ​​and reduce the calculation cost of the inverse matrix in GP regression. This compatibility is not limited to when the objective function f has many local solutions. This compatibility is possible when the scale length is large compared to the optimal value.

[0122] <Variation 1> The extraction unit 103 according to the first modification extracts, as extraction data, a set of pairs corresponding to one or more points having a predetermined ratio of the highest similarity to the points included in the low-dimensional search space, from one or more points in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit 101. The first modification will be described in detail below.

[0123] Observation point x n (n=0,1,…,N t-1 -1) to the similarity k(x n ,x´) are rearranged in ascending order to give x”n (n=0,1,…,N t-1 -1) and the corresponding observation value is y” n The extraction unit 103 extracts the extracted data E t D t-1 may be extracted from, where r t represents a ratio, 1 / N t-1 The value is greater than or equal to 1. t If is 1, E t D t-1 The function flооr(·) returns the largest integer less than or equal to the argument.

[0124]

number

[0125] In this embodiment, E t and {(x n ,y n )|k(x n ,x´)≧T t ,n=0,1,···,N t-1 -1} is r t and T t Depending on the setting of , it becomes equivalent. In this modified example, E t Number of elements in N t ´ is expressed as r t It can be directly controlled by

[0126]

number

[0127] The computational cost of inverting a GP regression matrix is ​​N t Since it depends on , this modification is advantageous in that it is possible to control the calculation cost.

[0128] The computational cost of the inverse matrix is ​​N t While it increases rapidly as ´ increases, tWhen ´ is small, the computation cost of the inverse matrix is ​​often not a practical problem. t-1 Number of elements in N t-1 While there are few t It is recommended to set E to 1. t D t-1 Match it with N t ´ to N t-1 If the computation cost of the inverse matrix becomes too high, r t It is recommended to set it so that is smaller than 1 and the calculation time is within a desired time. This makes it possible to suppress the calculation cost of the inverse matrix without substantially deteriorating the search efficiency of parameter optimization.

[0129] In this embodiment, a given point x′ is defined in a low-dimensional search space S t If we define it as a certain point in, then the following equation (34) holds.

[0130]

number

[0131] In this case, the maximum similarity maxk(x n ,x´) are rearranged in ascending order to give x” n (n=0,1,…,N t-1 -1) and the corresponding observation value is y” n The extraction unit 103 extracts the extracted data E t D t-1 It may be extracted from.

[0132]

number

[0133] Equation (35) t and E in Eq. (34) t is r t and T t Depending on the setting of , it becomes equivalent. In this modified example, E t Number of elements in N t´ is expressed as r t It can be directly controlled by

[0134]

number

[0135] The computational cost of inverting a GP regression matrix is ​​N t Since it depends on r , this modification is superior in that it can control the calculation cost. t By controlling in the same manner as described above, the calculation cost of the inverse matrix can be suppressed without substantially deteriorating the search efficiency of the parameter optimization.

[0136] <Variation 2> The proposing unit 104 according to the second modification calculates the similarity between two points in a D-dimensional space from components of an orthogonal complement of a linear subspace associated with an R-dimensional affine subspace, which is a low-dimensional search space included in the D-dimensional space. The second modification will be described in detail below.

[0137] In this embodiment, the extracted data E t gave an example expressed by the following equation (37).

[0138]

number

[0139] For example, if the kernel function is a squared exponential kernel function, then the function P St From the definition of, the following equation (38) holds. (I D -P Ut ) is S t R t dimensional linear subspace U t The orthogonal complement of U t ⊥ is the orthogonal projection matrix onto

[0140]

number

[0141] Any point x in D-dimensional space can be expressed by the following equation (39). That is, any point x in D-dimensional space can be decomposed into the first and second terms on the right-hand side of equation (39). The former is the U of point x. t The latter is called the component of U at point x. t ⊥ These are called components. D -P Ut )(x n -x bt-1 ) is the observation point x n and the best observation point x bt-1 The difference vector (x n -x bt-1 )U t ⊥ Ingredients: U t Has no ingredients.

[0142]

number

[0143] R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t If it is an element of (I D -P Ut ) is a diagonal matrix, and among its diagonal elements, R t R corresponding to the coordinate axes t components are 0 and the remaining (DR t ) components are 1. R t Each component is U t The remaining (DR t ) components are U t ⊥ Therefore, (I D -P Ut )(x n -x bt-1 ) is x n and x bt-1 U t ⊥ Corresponding to the component (DR t) components.

[0144] In this modified example, this property is utilized to create R t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t is changed according to time t, and U t ⊥ Corresponding to the component (DR t ) components, n ,P St (x n ) and extract the data E t Extract U t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0145] <Modification 3> In this embodiment, the extraction unit 103 calculates the similarity k(x n ,x´) is a given value T t Extracted data E based on the above criteria t An example of extracting

[0146]

number

[0147] The extraction unit 103 according to the third modification extracts the extracted data E based on a different criterion. t If the kernel function is a squared exponential kernel function, then k(x n ,x´)≧T t The fact that the above expression holds is equivalent to the fact that the following expression (41) holds.

[0148]

number

[0149] Also, T t ≦θσ 2 Assuming that k(x n ,x´)≧T t holds true, the distance ||x n This is equivalent to the following equation (42) being true for -x´||.

[0150]

number

[0151] The extraction unit 103 according to the third modification uses the distance ||x n -x´|| is the scale length θ l T t Extracted data E based on the criteria of ´ times or less t Therefore, E t The following equation (43) holds true.

[0152]

number

[0153] Extracted data E according to modification example 3 t is the similarity k(x n ,x´) is a given value T t The data is the same as the data extracted based on the above criteria, and therefore the same effects as in this embodiment can be obtained.

[0154] T shown in Eq. (42) t From the definition of ´, T t ´, the user is T t By simply giving σ An adaptive value is set according to the distance ||x n -x´ || threshold T t ´θ l is the hyperparameter θ of the kernel function l Therefore, the threshold T t ´θ l is the distance ||x n -x´||, the kernel function hyperparameter θl ,θ σ The value is adaptively set according to the

[0155] T t ' is not limited to that shown in equation (42). In this case, there is no guarantee that the extracted data will be the same as that of this embodiment, and T t ´ is the signal standard deviation θ σ In this case, the calculation of logarithms and square roots is unnecessary, and the calculation cost is reduced. Even in this case, the similarity k(x n The set of pairs with large x' is the extracted data E t is extracted as

[0156] x´=P St (x n ), then R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t is changed according to time t, and U t ⊥ Corresponding to the component (DR t ) components, n You can also calculate -x´||. t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0157] In the third modification, the case where the kernel function is a squared exponential kernel function is exemplified. Even if the kernel function is not a squared exponential kernel function, if the kernel function has a scale length as a hyperparameter, the extraction unit 103 extracts the observation data D t-1 The observation points {x n |n=0,1,…,N t-1 -1)}, the low-dimensional search space S t Distance ||x to a given point x´ contained in n -x´|| is the scale length θ l Extract a set of pairs corresponding to one or more observation points that are less than or equal to the coefficient of Et In this case, although there is no guarantee that the extracted data will be the same as that in this embodiment, the similarity k(x n ,x´) is large at observation point x n The set of pairs corresponding to the extracted data E t is extracted as

[0158] <Variation 4> When the kernel function has a scale length as a hyperparameter, the extraction unit 103 according to the fourth modification extracts, as extracted data, a set of pairs corresponding to one or more points whose D distances in each coordinate axis direction of the D-dimensional space to a point included in the low-dimensional search space are all equal to or less than a coefficient multiple of the scale length, out of one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit 101. Specifically, the extraction unit 103 extracts, for all d=0, 1, ..., D-1, a set of pairs corresponding to the point x n The absolute value of the difference between the d-th component of point x' and the scale length θ l T t Extracted data E based on the criteria of ´´ times or less t Here, T t ´´ is a coefficient set by the user. In this case, E t is expressed by the following equation (44).

[0159]

number

[0160] As can be seen from equation (44), the distance |(x n ) [d] -(x´) [d] For |, an adaptive threshold is set according to the hyperparameter of the kernel function. This modification 4 is not equivalent to modification 3, but is an approximation. Therefore, almost the same effect as modification 3 can be obtained.

[0161] x´=P St (x n ), then R in D-dimensional spacet R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t If we change U according to time t, t ⊥ Corresponding to the component (DR t ) for all d points x n The absolute value of the difference between the d-th component of point x' and the scale length θ l T t Extracted data E based on the criteria of ´´ times or less t You can also extract U t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0162] <Variation 5> When a kernel function for calculating a similarity has a vector of a scale length as a hyperparameter, the extraction unit 103 according to the fifth modification extracts, as extracted data, a set of pairs corresponding to one or more points whose squared normalized Euclidean distance to a point included in a low-dimensional search space is equal to or less than a predetermined value, among one or more points included in a D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit 101. In calculating the squared normalized Euclidean distance, the extraction unit 103 adopts the value of each element of the vector of the scale length as the standard deviation corresponding to each coordinate axis direction in the D-dimensional space. The fifth modification will be described in detail below.

[0163] If the kernel function is an ARD squared exponential kernel function, then k(x n ,x´)≧T t is equivalent to the following equation (45) being true.

[0164]

number

[0165] In addition, in the following equation (46), T tWhen ´´´ is defined, the satisfaction of equation (45) is equivalent to the satisfaction of equation (47) below.

[0166]

number

[0167] From the definition of (46), the user t By simply giving σ An adaptive threshold T t ´´´ is set.

[0168] When the kernel function is an ARD squared exponential kernel function, the extraction unit 103 extracts the point x n The square of the normalized Euclidean distance between x and x is T t Extracted data E based on the criteria below t Here, in the calculation of the square of the normalized Euclidean distance, a D-dimensional scale length vector θ l The values ​​of each element of E are adopted. t The following equation (48) holds true.

[0169]

number

[0170] T t ''' is not limited to the one shown in equation (46). In this case, there is no guarantee that the extracted data will be the same as that in the first embodiment, and T t ´´´ is the signal standard deviation θ σ In this case, the calculation of logarithms is unnecessary, and the calculation cost is reduced. Even in this case, the similarity k(x n The set of pairs with large x' is the extracted data E t is extracted as

[0171] x´=P St (x n), then R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t is changed according to time t, and U t ⊥ Corresponding to the component (DR t ) components to calculate the square of the normalized Euclidean distance. t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0172] In this modification, the case where the kernel function is an ARD squared exponential kernel function is exemplified. Even if the kernel function is not an ARD squared exponential kernel function, when the kernel function has a D-dimensional scale length vector as a hyperparameter, the extraction unit 103 extracts the observed data D t-1 The observation points {x n |n=0,1,…,N t-1 -1)}, the low-dimensional search space S t A set of pairs corresponding to one or more observation points whose squared normalized Euclidean distance to a given point x' included in is less than a given value is extracted as data E t In this case, although there is no guarantee that the extracted data will be the same as that in the first embodiment, the similarity k(x n ,x´) is large at observation point x n The set of pairs corresponding to the extracted data E t is extracted as

[0173] <Variation 6> When the kernel function for calculating the similarity has a scale length vector as a hyperparameter, the extraction unit 103 according to the sixth modification extracts, as extracted data, a set of pairs corresponding to one or more points, among one or more points included in a D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit 101, whose D distances in each coordinate axis direction of the D-dimensional space to a point included in a low-dimensional search space are equal to or less than a coefficient multiple of the corresponding element of the scale length vector. Specifically, when the kernel function has a D-dimensional scale length vector as a hyperparameter, the extraction unit 103 extracts a set of pairs corresponding to one or more points, among one or more points included in a D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit 101, whose D distances in each coordinate axis direction of the D-dimensional space to a point included in a low-dimensional search space are equal to or less than a coefficient multiple of the corresponding element of the scale length vector. n The absolute value of the difference between the d-th component of point x' and the D-dimensional scale length vector θ l The dth element of T t Extracted data E based on the criteria of ´´´´ times or less t Here, T t ´´´´ is a coefficient set by the user. In this case, E t is expressed by the following equation (49).

[0174]

number

[0175] As can be seen from equation (49), the distance |(x n ) [d] -(x´) [d] For |, an adaptive threshold is set according to the hyperparameters of the kernel function. This is not equivalent to the fifth modification, but is an approximation. Therefore, almost the same effect as the fifth modification can be obtained.

[0176] x´=P St (x n ), then R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U tIf we change U according to time t, t ⊥ Corresponding to the component (DR t ) for all d, point x n The absolute value of the difference between the d-th component of point x' and the scale length vector θ l The dth element of T t Extracted data E based on the criteria of ´´´´ times or less t You can also extract U t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0177] <Variation 7> If the kernel function is a squared exponential kernel function, then the observation point x n (n=0,1,…,N t-1 -1) to the distance ||x n -x´|| rearranged in ascending order, x'' n (n=0,1,…,N t-1 -1) and the corresponding observations are expressed as y'' n The extraction unit 103 extracts the extracted data E t D t-1 Here, as in the first modification, r t represents a ratio, 1 / N t-1 The value is greater than or equal to 1. t If is 1, E t D t-1 The function flооr(·) returns the largest integer less than or equal to the argument.

[0178]

number

[0179] E of this modified example t and E of Variation 3 t is r t and T t Depending on the setting of ´, it becomes equivalent. In this modified example, E t Number of elements in N t ´ is expressed as r tThis differs from the fifth modified example in that it can be directly controlled by

[0180]

number

[0181] The computational cost of inverting a GP regression matrix is ​​N t Since it depends on , this modification is advantageous in that it is possible to control the calculation cost.

[0182] The computational cost of the inverse matrix is ​​N t While it increases rapidly as ´ increases, t When ´ is small, the computation cost of the inverse matrix is ​​often not a practical problem. t-1 Number of elements in N t-1 While there are few t It is recommended to set E to 1. t D t-1 Match it with N t ´ to N t-1 If the computation cost of the inverse matrix becomes too high, r t It is recommended to set it so that is smaller than 1 and the calculation time is within a desired time. This makes it possible to suppress the calculation cost of the inverse matrix without substantially deteriorating the search efficiency of parameter optimization.

[0183] x´=P St (x n ), then R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t is changed according to time t, and U t ⊥ Corresponding to the component (DR t ) components to calculate the square of the normalized Euclidean distance. t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0184] In this modification, the case where the kernel function is a squared exponential kernel function is exemplified. Even if the kernel function is not a squared exponential kernel function, if the kernel function has a scale length as a hyperparameter, the extraction unit 103 extracts the observation data D t-1 The observation points {x n |n=0,1,…,N t-1 -1)}, the low-dimensional search space S t Distance ||x to a given point x´ contained in n Extract a set of pairs corresponding to a certain percentage of observation points from the smallest -x´|| data E t Even in this case, the similarity k(x n ,x´) is large at observation point x n The set of pairs corresponding to the extracted data E t is extracted as

[0185] <Variation 8> When a kernel function for calculating a similarity has a scale length vector as a hyperparameter, extraction unit 103 according to Modification 8 extracts, as extracted data, a set of pairs corresponding to one or more points whose squared normalized Euclidean distances to a point included in a low-dimensional search space are equal to or smaller than a predetermined ratio from among one or more points included in a D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in storage unit 101. In calculating the squared normalized Euclidean distance, extraction unit 103 employs the value of each element of the scale length vector as the standard deviation corresponding to each coordinate axis direction in the D-dimensional space.

[0186] If the kernel function is an ARD squared exponential kernel function, then the observation point x n (n=0,1,…,N t-1 -1) are rearranged in ascending order of the squared normalized Euclidean distance to a given point x', and then x * n (n=0,1,…,N t-1 -1) and the corresponding observations are expressed as y* n Here, in the calculation of the square of the normalized Euclidean distance, the D-dimensional scale length vector θ l Therefore, the square of the normalized Euclidean distance is n The square of the normalized Euclidean distance to a given point x' is expressed by the following equation (52).

[0187]

number

[0188] The extraction unit 103 extracts the extracted data E t D t-1 Here, as in the first modification, r t represents a ratio, 1 / N t-1 The value is greater than or equal to 1. t If is 1, E t D t-1 The function flооr(·) returns the largest integer less than or equal to the argument.

[0189]

number

[0190] E of this modified example t and E of Variation 5 t is r t and T t Depending on the setting of ´´´, it can be equivalent. In this modified example, E t Number of elements in N t ´ is expressed as r t This differs from the fifth modified example in that it can be directly controlled by

[0191]

number

[0192] The computational cost of inverting a GP regression matrix is ​​N t Since it depends on , this modification is advantageous in that it is possible to control the calculation cost.

[0193] The computational cost of the inverse matrix is ​​N t While it increases rapidly as ´ increases, t When ´ is small, the computation cost of the inverse matrix is ​​often not a practical problem. t-1 Number of elements in N t-1 While there are few t It is recommended to set E to 1. t D t-1 Match it with N t ´ to N t-1 If the computation cost of the inverse matrix becomes too high, r t It is recommended to set it so that is smaller than 1 and the calculation time is within a desired time. This makes it possible to suppress the calculation cost of the inverse matrix without substantially deteriorating the search efficiency of parameter optimization.

[0194] x´=P St (x n ), then R in D-dimensional space t R along the coordinate axes t All of the vectors are R t dimensional linear subspace U t Under the constraint that U t is changed according to time t, and U t ⊥ Corresponding to the component (DR t ) components to calculate the square of the normalized Euclidean distance. t Since there is no need to refer to the values ​​of the components, calculation costs can be reduced.

[0195] In this modification, the case where the kernel function is the ARD squared exponential kernel function is exemplified. Even if the kernel function is not the ARD squared exponential kernel function, if the kernel function has a scale length as a hyperparameter, the extraction unit 103 extracts the observed data DD t-1 The observation points {x n |n=0,1,…,N t-1 -1)}, the low-dimensional search space S t Extract a set of pairs corresponding to a predetermined proportion of observation points from the set with the smallest square of the normalized Euclidean distance to a given point x' included in the data E t Even in this case, the similarity k(x n ,x´) is large at observation point x n The set of pairs corresponding to the extracted data E t is extracted as

[0196] <Variation 9> In S203 at time t, the extraction unit 103 according to this embodiment extracts the observation data D t-1 Extract data from E t Depending on the kernel function used for this extraction, there are hyperparameters. Extracted data E t If the extraction of depends on the hyperparameters of the kernel function, the hyperparameters must be determined in advance. Note that the hyperparameters are assumed to be scale lengths or vectors of scale lengths.

[0197] If the adopted kernel function has a hyperparameter, the value of the hyperparameter should be determined in advance. The value may be a constant, or may vary according to time t. If the value is a constant, the extraction unit 103 should store the hyperparameter value.

[0198] When the hyperparameters of the kernel function are changed according to time t, in S203 at each time t, the extraction unit 103 t In order to extract t-1, or extracted data E t If the hyperparameter values ​​are estimated from the extracted data E, the calculation cost is high. t Since there is no process for extracting , the calculation cost is preferably small.

[0199] Fig. 5 is a diagram showing an example of a functional configuration of a parameter optimization system 5 according to Modification 9. As shown in Fig. 5, in a parameter vector value proposal device 500 according to Modification 9, the storage unit 101 also stores hyperparameters. Only modified points of the processing of Modification 9 will be described below. In the following description, components having substantially the same functions as those in this embodiment are denoted by the same reference numerals, and will be described repeatedly only when necessary.

[0200] In S203 when the time t is 1, the extraction unit 103 estimates hyperparameter values ​​from the observation data D0, and uses any existing method as the hyperparameter estimation method. t Extract.

[0201] In S204 at time t, the proposal unit 104 extracts the extracted data E t The hyperparameter values ​​are estimated from the acquisition function a used to determine the proposed points. Any existing method is used as the hyperparameter estimation method. The estimated hyperparameter values ​​are supplied to the storage unit 101. The storage unit 101 stores the received hyperparameter values. The estimated hyperparameter values ​​are used as the acquisition function a used to determine the proposed points. t This is also reflected in the definition of

[0202] In S203 after time t is 2, the extraction unit 103 acquires hyperparameter values ​​from the storage unit 101. The acquired hyperparameter values ​​are the hyperparameter values ​​estimated by the proposing unit 104 at time (t-1). The extraction unit 103 uses the hyperparameter values ​​acquired from the extraction unit 103 to generate extracted data E t Extract.

[0203] In this modified example, the proposing unit 104 reuses the hyperparameter values ​​estimated at the immediately preceding time and the extracting unit 103 extracts the extracted data E t , there is an advantage that it is not necessary for the extraction unit 103 to estimate the hyper-parameter values.

[0204] <Modification 10> Fig. 6 is a diagram showing pseudo program code according to Modification 10, which corresponds to the parameter optimization process shown in Fig. 2. Only the differences from Fig. 3 will be explained below.

[0205] S601 is the same as S301, and S602 is the same as S302.

[0206] S603 corresponds to t=1 in the for statement in S303. In Fig. 3, the time t is incremented in the for statement in S303, whereas in Fig. 6, it is incremented in S616, which will be described later.

[0207] S604 is a for statement that is not shown in Fig. 3. In this for statement, the for statement in S605 (described later) is repeated J times. The maximum value of time t is determined by J and G and L (described later).

[0208] S605 is a for statement that is not shown in Figure 3. In this for statement, the processes from S606 to S617 are repeated G times.

[0209] The processing content of S606 is the same as that of S304. The processing timing is different. S304 is processed each time the time advances by 1, whereas S606 is outside the for statement of S610 (described later) that includes S616 (described later) that increments the time t, and therefore is processed each time the time advances by L (described later).

[0210] The processing content of S607 is the same as that of S305. The processing timing is different. S305 is processed each time the time advances by 1, whereas S607 is outside the for statement of S610 (described later) that includes S616 (described later) that increments the time t, and therefore is processed each time the time advances by L (described later).

[0211] The processing content of S608 is the same as that of S306. The processing timing is different. S306 is processed each time the time advances by 1, whereas S608 is outside the for statement of S610 (described later) that includes S616 (described later) that increments the time t, and therefore is processed each time the time advances by L (described later).

[0212] The processing content of S609 is the same as that of S307. The processing timing is different. S307 is processed each time the time advances by 1, whereas S609 is outside the for statement of S610 (described later) that includes S616 (described later) that increments the time t, and therefore is processed each time the time advances by L (described later).

[0213] S610 is a for statement that is not shown in FIG. 3. In this for statement, the processes from S611 to S616 are repeated L times. t Since the processes from S606 to S608 for updating are outside this for statement, t does not change.

[0214] The process content of S611 is the same as that of S308. The process timing is also the same in the sense that each time the time advances by 1.

[0215] The process of S612 is the same as that of S309. The process timing is also the same in that the time advances by 1.

[0216] The process of S613 is the same as that of S310. The process timing is also the same in that the time advances by 1.

[0217] In S614, the extraction unit 103 extracts the extracted data E t-1 and the pair (x Nt-1 ,y Nt-1 ) and extract data E t Generate x Nt-1 The proposing unit 104 searches the low-dimensional search space S t Since this is a proposed point, St Therefore, the proposed point x Nt-1 and the similarity k(x Nt-1 ,x´) is large. For example, if the kernel function is a squared exponential kernel function and x´=P St (x n ), the following formula (55) is established. This is the maximum value that the similarity can take under this condition. Therefore, even if S307 is processed at the timing of S614, the extracted data E will be the same as when it is processed at S614. t Therefore, under certain conditions, S614 is equivalent to S307.

[0218]

number

[0219] The processing content of S615 is the same as that of S311. The processing timing is also the same in the sense that each time the time advances by 1. S616 corresponds to the increment of the time t in the for statement of S303.

[0220] S617 is a for statement not shown in Fig. 3 and corresponds to the above-mentioned S610. S618 is a for statement not shown in Fig. 3 and corresponds to the above-mentioned S605. S619 is a for statement not shown in Fig. 3 and corresponds to the above-mentioned S604.

[0221] In S620, T is defined as the number of times the time t is incremented in S616. This T corresponds to T in S303.

[0222] S621 is the same as S313. S622 is the same as S314.

[0223] In the pseudocode in Figure 6, we consider a low-dimensional search space S t The process from S606 to S608 related to is executed only every time the time advances by L. The process according to this pseudo code is executed for that time only. t Fix S tThe process is iterative: propose, observe, and update.

[0224] In S607, U t may be set according to g. For example, R t Let be 1 regardless of time t, G = D, and let g correspond to each coordinate axis direction in the D-dimensional space. Let U be the one-dimensional linear subspace corresponding to g. t It would be better to say:

[0225] U t can be set according to a different rule. An example is given below. Define G sets whose elements are integers from 0 to D-1, and set h g (g=0,1,...,G-1). g The number of elements in must be 1 or more. For example, h0={0,1},h1={2,3,4,5},h2={6},...,h G-1 = {D-2,D-1}. Corresponding to g in S605, h in D-dimensional space g R has only the direction vectors of each coordinate axis corresponding to the elements of t Let U be the dimensional linear subspace. t In this case, R t =#(h g ), where #(·) is a function that returns the number of elements. This allows U to be calculated according to g. t The basis vectors of U change depending on g. t The dimension and direction of h changes. In this example, g (g=0,1,...,G-1) can be changed in various ways. R t Let be 1 regardless of time t, G = D, and let g correspond to each coordinate axis direction in the D-dimensional space. Let U be the one-dimensional linear subspace corresponding to g. t For example, h0={0},h1={1},...,h G-1 ={D-1}.

[0226] each h g The number of elements of may be constant regardless of j, or may be changed according to j. gWhen the number of elements of is changed according to j, it may be changed randomly or according to a predetermined rule. g The elements of may be constant regardless of j, or may be changed according to j. g When the elements of are changed according to j, they may be changed randomly or according to a predetermined rule.

[0227] U t may be set according to another rule. An example is given below. Define G sets whose elements are D-dimensional vectors, and g (g=0,1,...,G-1). g The number of elements of must be 1 or more. For example, u0={v 0,0 ,v 0,1},u1={v 1,0 ,v 1,1 ,v 1,2 ,v 1,3},u2={v 2,0},···,u G-1= {v G-1,0 ,v G-1,1}. Corresponding to g in S605, u g R with only the D-dimensional vectors that are elements of t Let U be the dimensional linear subspace. t In this case, R t =#(u g ) This means that U t The basis vectors of change. In this case, U t is not necessarily aligned with the coordinate axis direction of the D-dimensional space.

[0228] Each u g The number of elements of may be constant regardless of j, or may be changed according to j. g When the number of elements of is changed according to j, it may be changed randomly or according to a predetermined rule. g The elements of may be constant regardless of j, or may vary according to j. g When the elements of are changed according to j, they may be changed randomly or according to a predetermined rule.

[0229] In S607, R t (<D)-dimensional U t By setting, S t also has a dimension smaller than D, R t dimension, so the search efficiency of parameter optimization is improved.

[0230] The pseudo-code in Figure 6 included three for-loops. However, similar to Figure 3, the for-loops only included those related to time t and could be transformed into pseudo-code with equivalent processing content to Figure 6. For the transformed pseudo-code, it is necessary to increment j, g, and l in Figure 6 separately. Also, for g and l, when they reach G - 1 and L - 1 respectively by incrementing, it is necessary to reset them to 0. The processing flow of the transformed pseudo-code corresponds to Figure 2. Therefore, h g and u g Based on, U t The method of controlling can also be applied to this embodiment. Since this modification example corresponds to Figure 2, it has the same effect as this embodiment.

[0231] <Modification Example 11> The extraction unit 103 according to Modification Example 11 extracts extraction data from the observed data based on the cumulative contribution rate regarding similarity. Hereinafter, Modification Example 11 will be described in detail.

[0232] Similar to Modification Example 1, the observation points x n (n = 0, 1, …, N t-1 - 1) are rearranged in descending order of the similarity k(x n , x´) with respect to a predetermined point x´, and are represented by x” n (n = 0, 1, …, N t-1 - 1), and the corresponding observed values are represented by y” n As shown in the following formula (56), the ratio of the cumulative similarity to the total similarity is called the cumulative contribution rate regarding similarity.

[0233]

Equation

[0234] The extraction unit 103 obtains the minimum N for which the cumulative contribution rate is equal to or greater than a predetermined value, and extracts the extracted data E t D t-1 It may be extracted from.

[0235]

number

[0236] Equation (57) t Extraction of k(x” N-1 ,x´)≧T t >k(x” N ,x´) t Equation (5) t In this embodiment, k(x" N-1 ,x´)≧T t >k(x” N ,x´) t Therefore, this modification has the same effect as the present embodiment. N-1 ,x´)≧T t >k(x” N ,x´) t The approximation from equation (23) to equation (25) in this embodiment where is set corresponds to the cumulative contribution rate regarding the similarity, and therefore has high explanatory power regarding the approximation accuracy.

[0237] <Modification 12> The present embodiment and several modified examples have been described above. These are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are included in the scope of the invention described in the claims.

[0238] <Hardware configuration> Fig. 7 is a diagram showing an example of the hardware configuration of the parameter vector value proposal device 100, 500. As shown in Fig. 7, the parameter vector value proposal device 100, 500 includes a processing circuit 71, a main memory device 72, an auxiliary memory device 73, a display device 74, an input device 75, and a communication device 76. The processing circuit 71, the main memory device 72, the auxiliary memory device 73, the display device 74, the input device 75, and the communication device 76 are connected via a bus.

[0239] The processing circuit 71 executes a parameter vector value proposal program read from the auxiliary storage device 73 to the main storage device 72, and functions as a search space determination unit 102, an extraction unit 103, a proposal unit 104, and a control unit 105. The main storage device 72 is a memory such as a RAM (Random Access Memory). The auxiliary storage device 73 is a HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, etc. The main storage device 72 and the auxiliary storage device 73 function as a memory unit 101.

[0240] The display device 74 displays various types of display information. The display device 74 is, for example, a display or a projector.

[0241] The input device 75 is an interface for operating a computer. The input device 75 is, for example, a keyboard or a mouse. The display device 74 and the input device 75 may be configured with a touch panel. The communication device 76 is an interface for communicating with other devices such as the observation device 200.

[0242] The programs executed by the computer are provided as computer program products, recorded in an installable or executable format on a computer-readable storage medium such as a CD-ROM, memory card, CD-R, or DVD (Digital Versatile Disc).

[0243] The program executed by the computer may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program executed by the computer may be configured to be provided via a network such as the Internet without being downloaded.

[0244] The program executed by the computer may be provided by being pre-installed in a ROM or the like. The program executed by the computer has a modular configuration including functional blocks that can be realized by the program, among the functional configurations (functional blocks) of the parameter vector value proposal devices 100, 500. As for each functional block, as actual hardware, the processing circuit 71 reads out the program from a storage medium and executes it, so that each of the functional blocks is loaded onto the main storage device 72. In other words, each of the functional blocks is generated on the main storage device 72.

[0245] Some or all of the above-described functional blocks may be realized by hardware such as an integrated circuit (IC) instead of by software. When the functions are realized by using multiple processors, each processor may realize one of the functions, or two or more of the functions.

[0246] Any operating form may be used for the computer that realizes the parameter vector value proposal device 100, 50. For example, the parameter vector value proposal device 100, 50 may be realized by a single computer. Also, for example, the parameter vector value proposal device 100, 50 may be operated as a cloud system on a network.

[0247] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0248] 1,5...parameter optimization system, 71...processing circuit, 72...main memory device, 73...auxiliary memory device, 74...display device, 75...input device, 76...communication device, 101...memory unit, 102...search space determination unit, 103...extraction unit, 104...proposal unit, 105...control unit, 100,500...parameter vector value proposal device, 200...observation device.

Claims

1. a storage unit that stores observed data that is a set of pairs of parameter vector values ​​representing points in a D-dimensional space (D is an integer equal to or greater than 2) and observed values ​​of the objective function at the points; a search space determination unit that determines an R (R is an integer equal to or greater than 1 and less than D)-dimensional affine subspace that passes through a point represented by a predetermined parameter vector value in the D-dimensional space as a low-dimensional search space; an extraction unit that extracts, as extracted data, a set of pairs corresponding to one or more points having a similarity to a point included in the low-dimensional search space equal to or greater than a predetermined value, among one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit; a suggestion unit that proposes a parameter vector value representing a point at which a value of the objective function will be next observed based on the extracted data; A parameter vector value proposal device comprising:

2. 2. The parameter vector value proposal device according to claim 1, wherein the extraction unit extracts as the extracted data a set of pairs corresponding to one or more points among the one or more points included in the D-dimensional space represented by the one or more parameter vector values ​​included in the observation data stored in the memory unit, the set corresponding to one or more points whose maximum similarity, which is the maximum value of the similarity with respect to all points included in the low-dimensional search space, is equal to or greater than a predetermined value.

3. 2. The parameter vector value proposal device according to claim 1, wherein the extraction unit extracts as the extracted data a set of the pairs corresponding to one or more points in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the memory unit, the set corresponding to one or more points having a predetermined percentage of the greatest similarity to points included in the low-dimensional search space.

4. 2. The parameter vector value proposal device according to claim 1, wherein the proposal unit calculates the similarity between two points in the D-dimensional space from components of an orthogonal complement of a linear subspace associated with the R-dimensional affine subspace, which is the low-dimensional search space included in the D-dimensional space.

5. The kernel function for calculating the similarity has a scale length as a hyperparameter, the extraction unit extracts, as the extracted data, a set of pairs corresponding to one or more points whose distance to a point included in the low-dimensional search space is equal to or less than a coefficient multiple of the scale length, among one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit; The parameter vector value proposition device according to claim 1 .

6. The kernel function for calculating the similarity has a scale length as a hyperparameter, 2. The parameter vector value proposal device of claim 1, wherein the extraction unit extracts as the extracted data a set of pairs corresponding to one or more points among the one or more points included in the D-dimensional space represented by the one or more parameter vector values ​​included in the observation data stored in the memory unit, the set of pairs corresponding to one or more points whose D distances in each coordinate axis direction of the D-dimensional space to a point included in the low-dimensional search space are all less than a coefficient multiple of the scale length.

7. The kernel function for calculating the similarity has a vector of scale lengths as a hyperparameter, the extraction unit extracts, as the extracted data, a set of pairs corresponding to one or more points whose squared normalized Euclidean distance to a point included in the low-dimensional search space is equal to or less than a predetermined value, among one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit; the extraction unit employs values ​​of elements of the scale length vector as standard deviations corresponding to each coordinate axis direction of the D-dimensional space in calculating the square of the normalized Euclidean distance. The parameter vector value proposition device according to claim 1 .

8. The kernel function for calculating the similarity has a vector of scale lengths as a hyperparameter, 2. The parameter vector value proposal device of claim 1, wherein the extraction unit extracts as the extracted data a set of pairs corresponding to one or more points among the one or more points included in the D-dimensional space represented by the one or more parameter vector values ​​included in the observation data stored in the memory unit, the set of pairs corresponding to one or more points whose all D distances in each coordinate axis direction of the D-dimensional space to a point included in the low-dimensional search space are less than or equal to a coefficient multiple of a corresponding element of the scale length vector.

9. The kernel function for calculating the similarity has a scale length as a hyperparameter, 2. The parameter vector value proposal device of claim 1, wherein the extraction unit extracts as the extracted data a set of pairs corresponding to one or more points included in the D-dimensional space represented by one or more of the parameter vector values ​​included in the observation data stored in the memory unit, the set corresponding to one or more points whose distance to a point included in the low-dimensional search space is from the smallest to a predetermined percentage.

10. The kernel function for calculating the similarity has a vector of scale lengths as a hyperparameter, the extraction unit extracts, as the extracted data, a set of pairs corresponding to one or more points whose squared normalized Euclidean distances to a point included in the low-dimensional search space are equal to or smaller than a predetermined ratio from among one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in the storage unit; the extraction unit employs values ​​of elements of the scale length vector as standard deviations corresponding to each coordinate axis direction of the D-dimensional space in calculating the square of the normalized Euclidean distance. The parameter vector value proposition device according to claim 1 .

11. The kernel function for calculating the similarity has a scale length or a vector of scale lengths as a hyperparameter, The proposing unit estimates the scale length or a vector of the scale lengths from the extracted data; the storage unit stores the scale length or the vector of scale lengths estimated by the proposing unit; The extraction unit extracts the extraction data based on the scale length or the scale length vector stored in the storage unit. The parameter vector value proposition device according to claim 1 .

12. The parameter vector value proposing device according to claim 1 , wherein the extracting unit extracts the extracted data from the observed data based on a cumulative contribution rate related to the similarity.

13. A computer comprising: determining an R (R is an integer equal to or greater than 1 and less than D)-dimensional affine subspace passing through a point represented by a predetermined parameter vector value in a D (D is an integer equal to or greater than 2)-dimensional space as a low-dimensional search space; extracting, as extracted data, a set of pairs corresponding to one or more points having a similarity to a point included in the low-dimensional search space equal to or greater than a predetermined value, from one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in a storage unit that stores the observation data, which is a set of pairs of parameter vector values ​​representing a point in the D-dimensional space and observed values ​​of the objective function at the point; proposing parameter vector values ​​representing points at which to next observe values ​​of the objective function based on the extracted data; A method for proposing parameter vector values ​​comprising:

14. A parameter vector value proposal device determines an R (R is an integer greater than or equal to 1 and less than D)-dimensional affine subspace passing through a point represented by a predetermined parameter vector value in a D (D is an integer greater than or equal to 2)-dimensional space as a low-dimensional search space, the parameter vector value proposal device extracts, as extracted data, a set of pairs corresponding to one or more points in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in a storage unit that stores the observation data, the set of pairs being a set of parameter vector values ​​representing a point in the D-dimensional space and observed values ​​of the objective function at the point; and the parameter vector value proposal device proposes a parameter vector value representing a point at which a value of the objective function will next be observed based on the extracted data; an observation device observes the next observation point based on a parameter vector value representing the next observation point; A parameter optimization method comprising:

15. On the computer, A function of determining an R (R is an integer of 1 or more and less than D)-dimensional affine subspace passing through a point represented by a predetermined parameter vector value in a D (D is an integer of 2 or more)-dimensional space as a low-dimensional search space; a function of extracting, as extracted data, a set of pairs corresponding to one or more points having a similarity to a point included in the low-dimensional search space equal to or greater than a predetermined value, from one or more points included in the D-dimensional space represented by one or more parameter vector values ​​included in the observation data stored in a storage unit that stores the observation data, which is a set of pairs of parameter vector values ​​representing a point in the D-dimensional space and observed values ​​of the objective function at the point; a function of proposing a parameter vector value representing a point at which the value of the objective function will be next observed based on the extracted data; A parameter vector value suggestion program that achieves this.

Images