EVALUATION APPARATUS, EVALUATION METHOD, AND PROGRAM

Abstract

To provide an evaluation device configured to apply Bayesian optimization to an optimization problem in which a constraint condition is given to an objective characteristic having an objective of the optimization problem.SOLUTION: An evaluation device 100 that evaluates, by Bayesian optimization, an unknown characteristic point corresponding to a candidate experimental point based on a known characteristic point corresponding to an experimented experimental point includes: a reception controller 10 that acquires experimental result data 222 indicating the experimented experimental point and the known characteristic point, objective data 212 indicating an optimization objective, and constraint-condition data 213 indicating a constraint condition; an evaluation value calculator 12 that calculates an evaluation value of an unknown characteristic point based on the experimental result data 222, the objective data 212, and the constraint-condition data 213; and an evaluation value output unit 13 that outputs the evaluation value. The evaluation value calculator 13 gives weighting according to a degree of compatibility of the constraint condition to an evaluation value for at least one objective characteristic.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to a technique for evaluating experimental conditions used in general industrial product development or manufacturing process development, etc.

Background Art

[0002] In industrial product development or manufacturing process development, it is necessary to control control factors under optimal conditions so as to meet the requirements of required target characteristics. For example, in the development of a battery, as control factors, the positive electrode thickness, negative electrode thickness, number of separators, electrolyte ion conductivity, etc. are set, and as target characteristics, capacity, life, cost, etc. are set.

[0003] If the relationship between the control factor and the target characteristic can be expressed by a physical formula, the optimal solution of the control factor can be searched by a mathematical optimization method. However, when the relationship is unknown, a combination of values of the control factor (i.e., an experimental point) is selected as an experimental condition, and an actual experiment is conducted. As an experimental result, a combination of values of the target characteristic corresponding to the experimental point (i.e., a characteristic point) is obtained. By repeating this experiment, the optimal solution of the control factor is searched.

[0004] Generally, in complex industrial product development or manufacturing process development, a great deal of financial or time cost is incurred to execute one experiment. Therefore, in order to efficiently perform development work, it is important to search for the optimal solution with as few experiment times as possible.

[0005] Conventionally, approaches using experimental design methods and response surface methods have been used to search for the optimal solution. However, in these methods, trial and error of an analyst is required in the stage of creating a prediction model or searching for the optimal solution, and quantitative evaluation with a consistent procedure is difficult.

[0006] In recent years, data-driven approaches using Bayesian optimization have attracted attention in the field of machine learning (see, for example, Patent Document 1, Non-Patent Document 1, and Non-Patent Document 2). Bayesian optimization is an optimization method that assumes a Gaussian process as a mathematical model to represent the correspondence between inputs and outputs. For each experimental result obtained, the predicted distribution of characteristic points is calculated for each set experimental point. Based on the predicted distribution of each characteristic point, the optimal next experimental condition is selected using an evaluation criterion called an acquisition function. This enables quantitative evaluation that does not depend on the skill of the analyst and can also contribute to the automation of the optimal solution search process. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2019-113985 [Non-patent literature]

[0008] [Non-Patent Document 1] M.Emmerich, A.Deutz, JWKlinkenberg, "The computation of the expected improvement in dominated hypervolume of Pareto front approximations," Report Technique, Leiden University, Vol.34, 2008. [Non-Patent Document 2] M.Abdolshah, A.Shilton, S.Rana, S.Gupta, S.Venkatesh, "Expected Hypervolume Improvement with Constraints," International Conference on Pattern Recognition (ICPR), 2018. [Overview of the project] [Problems that the invention aims to solve]

[0009] However, evaluation devices using the methods described in the above-mentioned literature have the drawback of having limited applicability.

[0010] Therefore, this disclosure provides an evaluation device that applies Bayesian optimization to an optimization problem in which constraints are imposed on the objective characteristics that have the objective of the optimization problem. [Means for solving the problem]

[0011] An evaluation device according to one aspect of the present disclosure is an evaluation device that evaluates unknown characteristic points corresponding to candidate experimental points by Bayesian optimization based on known characteristic points corresponding to experimentally tested points, and comprises: a first receiving means for acquiring experimental result data showing the experimentally tested points and the known characteristic points; a second receiving means for acquiring objective data showing that the unknown characteristic point indicates the value of one or more objective characteristics, and at least one objective characteristic has an optimization objective; a third receiving means for acquiring constraint data showing constraint conditions assigned to the at least one objective characteristic; a calculation means for calculating an evaluation value of the unknown characteristic point based on the experimental result data, the objective data and the constraint data; and an output means for outputting the evaluation value, wherein the calculation means assigns a weight to the evaluation value for the at least one objective characteristic according to the degree of fit of the constraint conditions. [Effects of the Invention]

[0012] According to the evaluation device of this disclosure, Bayesian optimization can be applied to an optimization problem in which constraints are imposed on the objective characteristics that have the objective of the optimization problem. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a diagram illustrating the schematic operation of the evaluation device according to the embodiment. [Figure 2]Figure 2 is a diagram showing an example of how each candidate experimental point and each characteristic point related to the embodiment are represented graphically. [Figure 3] Figure 3 shows the configuration of the evaluation device according to the embodiment. [Figure 4] Figure 4 is a block diagram showing the functional configuration of the arithmetic circuit according to the embodiment. [Figure 5] Figure 5 shows an example of a reception image displayed on the display unit to receive setting information according to the embodiment. [Figure 6] Figure 6 shows an example of control factor data according to the embodiment. [Figure 7] Figure 7 shows an example of objective data and constraint data according to the embodiment. [Figure 8A] Figure 8A shows an example of the standard range according to the embodiment. [Figure 8B] Figure 8B shows another example of the standard range according to the embodiment. [Figure 9] Figure 9 is a flowchart showing the processing operation of the evaluation device according to the embodiment. [Figure 10A] Figure 10A shows an example of candidate experimental point data according to the embodiment. [Figure 10B] Figure 10B shows another example of candidate experimental point data according to the embodiment. [Figure 11] Figure 11 shows an example of experimental result data according to the embodiment. [Figure 12] Figure 12 is a diagram illustrating the processing performed by the evaluation value calculation unit according to the embodiment. [Figure 13] Figure 13 shows an example of predictive distribution data according to the embodiment. [Figure 14A] Figure 14A is a diagram showing an example of an improved area according to the embodiment. [Figure 14B] Figure 14B shows another example of an improved area according to the embodiment. [Figure 15A] Figure 15A is a diagram illustrating a method for calculating the volume of the improvement area according to an embodiment. [Figure 15B] Figure 15B shows an example of dividing the entire characteristic space into multiple sub-regions according to the embodiment. [Figure 15C] Figure 15C shows an example of the lower and upper endpoints of a small region according to the embodiment. [Figure 16] Figure 16 shows an example of evaluation value data according to the embodiment. [Figure 17] Figure 17 shows an example of the standard range and control range according to Modification 1 of the Embodiment. [Figure 18] Figure 18 is a diagram illustrating the processing performed by the evaluation value calculation unit according to a modified example 2 of the embodiment. [Figure 19] Figure 19 shows an example of the minimum distance according to a modified example 2 of the embodiment. [Figure 20] Figure 20 shows an example of minimum distance data according to a modified example 2 of the embodiment. [Figure 21] Figure 21 shows an example of evaluation value data related to a modified example 2 of the embodiment. [Modes for carrying out the invention]

[0014] (Knowledge that formed the basis of this invention) The inventors have found that the following problems arise with respect to Patent Document 1, Non-Patent Document 1, and Non-Patent Document 2 described in the "Background Art" section.

[0015] Several methods have been proposed for multi-objective Bayesian optimization that simultaneously optimize multiple objective characteristics. For example, Non-Patent Document 1 discloses the optimal solution search principle and specific calculation method of EHVI (Expected Hypervolume Improvement), one such method. This makes it possible to quantitatively evaluate the optimal solution search even when there are multiple objective characteristics to be optimized.

[0016] Furthermore, in industrial product development or manufacturing process development, specification ranges are sometimes set as constraints on the values ​​of target characteristics. For example, specification ranges such as "battery capacity should fall within the specification range of 1850 to 1950 [mAh]" or "lifespan should be as long as possible, with a minimum of 3 years (i.e., the minimum value of the lifespan specification range is 3 years, and the maximum value is +∞)" may be used. When a specification range exists, applying conventional Bayesian optimization may lead to computationally inefficient searches or cause the search to proceed to a different domain that is not the optimal solution.

[0017] Therefore, several methods for Bayesian optimization with constraints have been proposed. For example, in the method described in Patent Document 1, for each candidate experimental point, the probability of it falling within the specification range is calculated from the predictive distribution obtained by Gaussian process regression, and only the candidate experimental points whose probability exceeds a certain threshold are extracted, and the acquisition function is evaluated. This addresses the problem of optimization with constraints.

[0018] The method described in Patent Document 1 above can also employ EHVI as the acquisition function. However, while significant improvement from the provisional optimal solution can be expected if the values ​​of multiple candidate experimental points fall within the specification range, candidate experimental points with a low probability of the target characteristic value falling within the specification range are excluded from the evaluation of the acquisition function. Therefore, while the method described in Patent Document 1, which employs EHVI, contributes to reducing computational costs, it does not necessarily guarantee the search for the true optimal solution.

[0019] Furthermore, for example, Non-Patent Document 2 discloses EHVIC (Expected Hypervolume Improvement with Constraints), an extension of EHVI for cases with constraints. The acquisition function design method described in Non-Patent Document 2 comprehensively quantifies the probability of falling within the specification range and the amount of improvement, and evaluates all candidate experimental points. Therefore, there is a high possibility that the search efficiency will improve (i.e., the true optimal solution will be found). However, in the acquisition function design method described in Non-Patent Document 2, the objective characteristic to be maximized or minimized is different from the objective characteristic to be kept within the specification range. Therefore, using the example above, it is applicable to optimization problems such as "the battery capacity is 1850-1950 [mAh] and we want to find an experimental point that maximizes the lifespan." However, the method of Non-Patent Document 2 cannot be applied to optimization problems where constraints such as a specification range are attached to the objective characteristic that has an objective such as maximization or minimization, such as "we want to find an experimental point that maximizes the lifespan while keeping it at 3 years or more."

[0020] Therefore, the evaluation device of this disclosure aims to apply Bayesian optimization to optimization problems in which constraints are imposed on objective characteristics that have the objective of the optimization problem.

[0021] An evaluation device according to one aspect of the present disclosure is an evaluation device that evaluates unknown characteristic points corresponding to candidate experimental points by Bayesian optimization based on known characteristic points corresponding to experimentally tested points, and comprises: a first receiving means for acquiring experimental result data showing the experimentally tested points and the known characteristic points; a second receiving means for acquiring objective data showing that the unknown characteristic point indicates the value of one or more objective characteristics, and at least one objective characteristic has an optimization objective; a third receiving means for acquiring constraint data showing constraint conditions assigned to the at least one objective characteristic; a calculation means for calculating an evaluation value of the unknown characteristic point based on the experimental result data, the objective data and the constraint data; and an output means for outputting the evaluation value, wherein the calculation means assigns a weight to the evaluation value for the at least one objective characteristic according to the degree of fit of the constraint conditions.

[0022] As a result, when the calculation means calculates an evaluation value for an unknown characteristic point based on experimental result data, target data, and constraint data, it assigns a weight to the evaluation value for at least one target characteristic according to the degree of fit of the constraints. This at least one target characteristic has an optimization objective. Therefore, Bayesian optimization can be applied to optimization problems in which constraints are applied to a target characteristic that has an optimization objective. As a result, the range of application can be expanded.

[0023] Furthermore, the constraints are at least one constraint range, and the optimization objectives include a first objective to keep the objective characteristic within one of the at least one constraint ranges, and a second objective to minimize or maximize the objective characteristic. The calculation means may calculate the evaluation value by applying different weighting processes to each of the at least one objective characteristic in the following cases: (i) when the interval of the objective characteristic used to calculate the evaluation value is outside each of the at least one constraint ranges; (ii) when the interval is within one of the at least one constraint ranges and the optimization objective is the first objective; and (iii) when the interval is within one of the at least one constraint ranges and the optimization objective is the second objective. For example, the at least one constraint range may be one standard range, or it may be a standard range and a control range. Furthermore, the interval of the objective characteristic is, for example, an interval in the characteristic space represented by at least one objective characteristic, that is divided by constraints and one or more experimental points (more specifically, Pareto points).

[0024] As a result, evaluation values ​​calculated for each candidate experimental point are output, allowing the user of the evaluation device to select a candidate experimental point as the next experimental point based on these evaluation values, and then use the characteristic points obtained from experiments using that experimental point to calculate the evaluation values ​​for each subsequent candidate experimental point. By repeating this process of experimentation, calculation of evaluation values, and output, it is possible to obtain a solution for candidate experimental points that satisfies the optimization objective of each target characteristic, i.e., the optimal solution.

[0025] Furthermore, in the evaluation apparatus according to one aspect of this disclosure, different weighting processes are performed for each of at least one objective characteristics in cases (i) to (iii). Therefore, whether the optimization objective of the objective characteristic is the first objective or the second objective, the evaluation value of the candidate experimental point can be appropriately calculated based on Bayesian optimization. In other words, whether the optimization objective of the objective characteristic is within a constraint range, or maximization or minimization, the evaluation value of the candidate experimental point can be appropriately calculated based on Bayesian optimization. Also, in case (iii), since the interval of the objective characteristic is within a constraint range and the optimization objective is the second objective, unlike the method in Non-Patent Literature 2, the evaluation value can be quantitatively and appropriately calculated even when the objective characteristic whose optimization objective is maximization or minimization has a constraint range as a constraint condition.

[0026] As a result, it can be applied to optimization problems with constraints, such as specification ranges. In other words, it expands the scope of application and enables quantitative evaluation to improve the efficiency of searching for the optimal solution.

[0027] Furthermore, the system may further include a means for creating candidate experimental points by combining values ​​that satisfy predetermined conditions for each of multiple control factors.

[0028] For example, a predetermined condition is that the sum of the values ​​of the ratio variables of multiple control factors is 1. In a more specific example, the ratio variables are the blending ratios of materials such as compounds corresponding to the control factors. Therefore, for each combination of blending ratios of multiple types of compounds, an evaluation value can be calculated for that combination. As a result, it is possible to appropriately search for the optimal solution for at least one target property of the synthetic material obtained by blending those compounds.

[0029] Furthermore, the calculation means may calculate the evaluation value based on a constraint range of a shape different from a rectangle among the at least one constraint range.

[0030] This allows for the calculation of evaluation values ​​based on constraint ranges such as circles, ellipses, and star shapes in a characteristic space represented by, for example, two objective characteristics. Therefore, the application is not limited to cases where the shape of the constraint range is rectangular, and the range of applications can be further expanded.

[0031] Furthermore, if there are multiple constraint ranges as the at least one constraint range, the calculation means further divides case (ii) into multiple cases, calculates the evaluation value by performing different weighting processes on each of the multiple cases, and in each of the multiple cases, the interval may be included in different constraint ranges among the multiple constraint ranges.

[0032] For example, the multiple constraint ranges are the specification range and the control range included within that specification range. Then, in case (ii), the case is divided into a first case where the interval is within the specification range and outside the control range, and the optimization objective is the first objective, and a second case where the interval is within the control range and the optimization objective is the first objective. Furthermore, for example, a weighting process is performed using a larger weight for the second case than for the first case. In this way, there are multiple constraint ranges, and by further dividing case (ii) into multiple cases, it is possible to apply weights to each of the multiple constraint ranges in a stepwise manner. Therefore, even when the value of the objective characteristic should fall within the specification range and, as much as possible, within the control range, the evaluation value can be calculated appropriately. As a result, the application to optimization problems can be further expanded.

[0033] Furthermore, the calculation means may assign a priority to each of the at least one objective characteristics and calculate the evaluation value using the assigned priority.

[0034] This ensures that at least one objective characteristic is prioritized, allowing the objective characteristic with a higher priority to be optimized more quickly than the objective characteristic with a lower priority.

[0035] Furthermore, the calculation means may further calculate the minimum distance among the distances between the candidate experimental point and each of the one or more experimentally completed experimental points, and the output means may further output the minimum distance corresponding to the candidate experimental point.

[0036] This outputs the minimum distance corresponding to each candidate experimental point, allowing the user of the evaluation device to select the next candidate experimental point based not only on the evaluation value but also on its minimum distance. For example, in the initial stages of optimal solution search, candidate experimental points close to those already used in experiments tend to have higher evaluation values, and selecting such candidate experimental points for the next experiment may not significantly contribute to optimization. Therefore, the user can improve the accuracy of the evaluation values ​​and perform appropriate optimization by, for example, selecting a candidate experimental point for the next experiment that corresponds to an evaluation value that is relatively large and has a relatively long minimum distance.

[0037] Furthermore, the calculation means may calculate the evaluation value using the Monte Carlo method.

[0038] Because the Monte Carlo method is an approximation method, the processing burden for calculating evaluation values ​​can be reduced.

[0039] Furthermore, the calculation means may calculate the evaluation value using at least one of the evaluation methods PI (Probability of Improvement) and EI (Expected Improvement).

[0040] This allows us to calculate the volume within the constraint range in the characteristic space for each candidate experimental point as an improvement in optimization, and then appropriately calculate an evaluation value from that improvement.

[0041] The embodiments will be described in detail below with reference to the drawings.

[0042] The embodiments described below are all general or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept are described as optional components. In addition, each figure is a schematic diagram and is not necessarily a precise representation. Also, the same components are denoted by the same reference numerals in each figure.

[0043] (Embodiment) [overview] Figure 1 is a diagram illustrating the schematic operation of the evaluation device in this embodiment.

[0044] In this embodiment, the evaluation device 100 calculates evaluation values ​​for each of a plurality of candidate experimental points and displays evaluation value data 224 showing these evaluation values. Candidate experimental points are points that are candidates for experimental points. Experimental points represent experimental conditions (combinations of values ​​for each control factor in the experimental space). Evaluation values ​​are values ​​that show the evaluation results of the target characteristics that are predicted to be obtained by the experiment according to the candidate experimental point. For example, the evaluation value indicates the degree to which the target characteristics predicted to be obtained by the experiment match the optimization objective, and the larger the evaluation value, the greater the degree of match.

[0045] The user refers to the evaluation values ​​of each candidate experimental point indicated by the evaluation value data 224 and selects one of those candidate experimental points as the next experimental point. The user then conducts an experiment according to the selected experimental point using the experimental equipment. The experiment yields characteristic points corresponding to that experimental point. These characteristic points indicate, for example, the value of the target characteristic, and if there are multiple target characteristics, they are shown as a combination of the values ​​of the multiple target characteristics. The user associates these obtained characteristic points with the experimental points and inputs them into the evaluation device 100. As a result, the evaluation device 100 uses the characteristic points obtained from the experiment to recalculate the evaluation values ​​for each unselected candidate experimental point and redisplays the evaluation value data 224 showing those evaluation values. In other words, the evaluation value data 224 is updated. The evaluation device 100 searches for the optimal solution for the target characteristic by repeatedly updating the evaluation value data 224 in this way.

[0046] Figure 2 shows an example of a graph representing each candidate experimental point and each characteristic point. Specifically, the graph in Figure 2(a) shows each candidate experimental point placed in the experimental space, and the graph in Figure 2(b) shows each characteristic point placed in the characteristic space.

[0047] Candidate experimental points in the experimental space are arranged on grid points corresponding to combinations of values ​​for the first and second control factors, as shown in Figure 2(a). Characteristic points corresponding to each candidate experimental point shown in Figure 2(a) are arranged in the characteristic space, as shown in Figure 2(b). Specifically, when a candidate experimental point is selected as an experimental point and the values ​​of the first and second objective characteristics are obtained through experiments conducted at that experimental point, the characteristic point corresponding to that experimental point is placed at a position represented by the combination of the values ​​of the first and second objective characteristics. Here, there is a one-to-one correspondence between the candidate experimental points and the characteristic points, but this correspondence (i.e., the function f in Figure 2) is unknown.

[0048] Performing an experiment once can be rephrased as selecting one candidate experimental point and obtaining a pair of correspondences between that selected candidate experimental point and the corresponding characteristic point.

[0049] Furthermore, as shown in Figure 2(b), the characteristic space is divided into an area within the specification range and an area outside the specification range according to the set specification range. For example, the optimization objective of a target characteristic may be to maximize or minimize the value of the target characteristic. In this embodiment, a constraint condition, which is a specification range, may be attached to the target characteristic having an optimization objective. A constraint condition is a condition attached to the target characteristic, and for example, there is a constraint range that specifies the range of values ​​of the target characteristic as a condition. Examples of constraint ranges include a specification range defined by the specification of the target characteristic, and a control range that can be set as appropriate by the user.

[0050] In this embodiment, we will mainly describe examples where there are two control factors, such as the first and second control factors, and two objective characteristics, such as the first and second objective characteristics. However, the number of control factors and objective characteristics is not limited to two. The number of control factors may be one or three or more, and the number of objective characteristics may be one or three or more. Furthermore, the number of control factors and the number of objective characteristics may be equal or different.

[0051] [Hardware configuration] Figure 3 shows the configuration of the evaluation device 100 in this embodiment.

[0052] The evaluation device 100 includes an input unit 101a, a communication unit 101b, an arithmetic circuit 102, a memory 103, a display unit 104, and a storage unit 105.

[0053] The input unit 101a is a Human Machine Interface (HMI) that accepts input operations from the user. The input unit 101a can be, for example, a keyboard, mouse, touch sensor, or touchpad.

[0054] For example, the input unit 101a accepts setting information 210 as input from the user. The setting information 210 includes control factor data 211, objective data 212, and constraint data 213. The control factor data 211 is data that shows the possible values ​​of the control factor, for example, as shown in Figure 2(a). The values ​​of the control factor may be continuous or discrete. The objective data 212 is data that shows the optimization objective of the objective characteristic, for example, minimization / maximization. The constraint data 213 is data that shows constraint conditions, for example, constraint range.

[0055] The communication unit 101b connects to other devices by wire or wireless connection and transmits and receives data with those devices. For example, the communication unit 101b receives characteristic point data 201 from another device (e.g., an experimental apparatus).

[0056] The display unit 104 displays images or text. The display unit 104 is, for example, a liquid crystal display, a plasma display, or an organic EL (Electro-Luminescence) display. The display unit 104 may also be a touch panel integrated with the input unit 101a.

[0057] The storage unit 105 stores a program (i.e., a computer program) 200 in which each instruction to the arithmetic circuit 102 is described, as well as various data. The storage unit 105 is a non-volatile recording medium, such as a magnetic storage device like a hard disk, a semiconductor memory like an SSD (Solid State Drive), or an optical disc. The program 200 and various data may be provided to the evaluation device 100 via the communication unit 101b from the other devices mentioned above, for example, and stored in the storage unit 105. The storage unit 105 stores candidate experimental point data 221, experimental result data 222, predicted distribution data 223, and evaluation value data 224 as various data.

[0058] The candidate experimental point data 221 is data representing each candidate experimental point. In the example in Figure 2(a), each candidate experimental point is represented by a combination of values ​​for the first and second control factors. The candidate experimental point data 221 may also be in a table format in which combinations of values ​​for the first and second control factors are listed. Specific examples of such candidate experimental point data 221 will be explained in detail using Figures 10A and 10B.

[0059] The experimental result data 222 represents data showing one or more experimental points used in the experiment and the characteristic points corresponding to each of those one or more experimental points. For example, the experimental result data 222 shows a combination of experimental points in the experimental space shown in Figure 2(a) and characteristic points in the characteristic space shown in Figure 2(b) obtained from experiments using those experimental points. The experimental points are represented by a combination of values ​​for the first and second control factors, and the characteristic points are represented by a combination of values ​​for the first objective characteristic and the second objective characteristic. The experimental result data 222 may also be in a table format listing the combinations of experimental points and characteristic points. A specific example of this experimental result data 222 will be explained in detail using Figure 11.

[0060] The predictive distribution data 223 represents the predictive distribution of each candidate experimental point that has not been selected as an experimental point, as indicated by the candidate experimental point data 221. The predictive distribution is a distribution obtained by Gaussian process regression, as described above, and is expressed, for example, by its mean and variance. For example, the predictive distribution data 223 may be in a table format that shows the predictive distribution of the first objective characteristic and the predictive distribution of the second objective characteristic in association with each candidate experimental point. A specific example of this predictive distribution data 223 will be explained in detail using Figure 13.

[0061] The evaluation value data 224 is data that shows the evaluation value for each of the multiple candidate experimental points, as shown in Figure 1, for example. For example, the evaluation value data 224 may be in a table format that associates the evaluation value with each of the multiple candidate experimental points. Another specific example of this evaluation value data 224 will be explained in detail with reference to Figure 16.

[0062] The arithmetic circuit 102 is a circuit that reads the program 200 from the storage unit 105 into the memory 103 and executes the expanded program 200. The arithmetic circuit 102 is, for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0063] [Functional Configuration] Figure 4 is a block diagram showing the functional configuration of the arithmetic circuit 102.

[0064] The arithmetic circuit 102 implements multiple functions for generating evaluation value data 224 by executing the program 200. Specifically, the arithmetic circuit 102 includes a reception control unit (first receiving means, second receiving means, and third receiving means) 10, a candidate experimental point creation unit (candidate experimental point creation means) 11, an evaluation value calculation unit (calculation means) 12, and an evaluation value output unit (output means) 13.

[0065] The receiving control unit 10 receives characteristic point data 201, control factor data 211, target data 212, and constraint condition data 213 via the input unit 101a or the communication unit 101b. For example, when characteristic point data 201 is input by a user through an input operation to the input unit 101a, the receiving control unit 10 associates the characteristic points indicated in the characteristic point data 201 with experimental points and writes them to the experimental result data 222 in the storage unit 105. This updates the experimental result data 222. Once the experimental result data 222 is updated, the receiving control unit 10 instructs the evaluation value calculation unit 12 to perform processing using the updated experimental result data 222. In other words, the receiving control unit 10 instructs the evaluation value calculation unit 12 to calculate the evaluation value. At this time, the evaluation value calculation unit 12 performs the evaluation value calculation using candidate experimental point data 221 already stored in the storage unit 105. In this manner, the receiving control unit 10 triggers the evaluation value calculation unit 12 to start calculating the evaluation value upon input of the characteristic point data 201.

[0066] Furthermore, the receiving control unit 10 may cause the evaluation value calculation unit 12 to start calculating evaluation values ​​in response to other triggers. For example, if the experimental result data 222 is already stored in the storage unit 105, the receiving control unit 10 may cause the evaluation value calculation unit 12 to start calculating evaluation values ​​when the user inputs the level of the experimental points as a trigger. The level of the experimental points may be, for example, the minimum value, maximum value, and discrete range of the values ​​that the control factor can take. In other words, when the user inputs the level of the experimental points and candidate experimental point data 221 is generated based on that level, the receiving control unit 10 causes the evaluation value calculation unit 12 to start calculating evaluation values ​​based on the candidate experimental point data 221 and the experimental result data 222.

[0067] Alternatively, if candidate experimental point data 221 is already stored in the storage unit 105, the receiving control unit 10 may trigger the evaluation value calculation unit 12 to start calculating an evaluation value when the user inputs experimental result data 222. When the user inputs experimental result data 222, the receiving control unit 10 causes the evaluation value calculation unit 12 to start calculating an evaluation value based on the experimental result data 222 and the candidate experimental point data 221.

[0068] Alternatively, if candidate experimental point data 221 is already stored in the storage unit 105, the receiving control unit 10 may trigger the evaluation value calculation unit 12 to start calculating an evaluation value upon receiving experimental result data 222 by the communication unit 101b. For example, experimental equipment, experimental apparatus, or manufacturing apparatus transmits experimental result data 222 to the evaluation device 100, and the communication unit 101b receives the experimental result data 222. When the receiving control unit 10 receives the experimental result data 222 by the communication unit 101b, it causes the evaluation value calculation unit 12 to start calculating an evaluation value based on the experimental result data 222 and the candidate experimental point data 221.

[0069] Thus, if candidate experimental point data 221 and experimental result data 222 are available, the receiving control unit 10 instructs the evaluation value calculation unit 12 to start calculating evaluation values ​​based on them. If the experimental result data 222 is already stored in the storage unit 105, the receiving control unit 10 may also initiate the evaluation value calculation unit 12 to start calculating evaluation values ​​when the user inputs candidate experimental point data 221. Alternatively, if the candidate experimental point data 221 and experimental result data 222 are already stored in the storage unit 105, the receiving control unit 10 may also initiate the evaluation value calculation unit 12 to start calculating evaluation values ​​when the user inputs a start command.

[0070] The candidate experimental point creation unit 11 generates candidate experimental point data 221 based on the control factor data 211 acquired by the receiving control unit 10. In other words, the candidate experimental point creation unit 11 creates each of the multiple candidate experimental points using the values ​​of one or more control factors. The candidate experimental point creation unit 11 then stores the generated candidate experimental point data 221 in the storage unit 105.

[0071] The evaluation value calculation unit 12 reads candidate experimental point data 221 and experimental result data 222 from the storage unit 105, generates predicted distribution data 223 based on that data, and stores the predicted distribution data 223 in the storage unit 105. Furthermore, the evaluation value calculation unit 12 generates evaluation value data 224 based on the predicted distribution data 223 and the target data 212 and constraint condition data 213 acquired by the receiving control unit 10, and stores the evaluation value data 224 in the storage unit 105.

[0072] The evaluation value output unit 13 reads evaluation value data 224 from the storage unit 105 and outputs the evaluation value data 224 to the display unit 104. Alternatively, the evaluation value output unit 13 may output the evaluation value data 224 to an external device via the communication unit 101b. In other words, the evaluation value output unit 13 outputs the evaluation value for each candidate experimental point. The evaluation value output unit 13 may also directly acquire evaluation value data 224 from the evaluation value calculation unit 12 and output the evaluation value data 224 to the display unit 104. Similarly, the evaluation value output unit 13 reads predicted distribution data 223 from the storage unit 105 and outputs the predicted distribution data 223 to the display unit 104. The evaluation value output unit 13 may also directly acquire predicted distribution data 223 from the evaluation value calculation unit 12 and output the predicted distribution data 223 to the display unit 104.

[0073] [input] Figure 5 shows an example of a reception image displayed on the display unit 104 to accept the input of setting information 210.

[0074] The reception image 300 includes a control factor region 310 and a target characteristic region 320. The control factor region 310 is a region for receiving input of control factor data 211. The target characteristic region 320 is a region for receiving input of target data 212 and constraint condition data 213.

[0075] The control factor region 310 has input fields 311 to 314. Input field 311 is a field for inputting the name of the first control factor. For example, "X1" is entered in input field 311 as the name of the first control factor. Input field 312 is a field for inputting the value of the first control factor. For example, "-5,-4,-3,-2,-1,0,1,2,3,4,5" is entered in this input field 312 as the value of the first control factor. Similarly, input field 313 is a field for inputting the name of the second control factor. For example, "X2" is entered in input field 313 as the name of the second control factor. Input field 314 is a field for inputting the value of the second control factor. For example, "-5,-4,-3,-2,-1,0,1,2,3,4,5" is entered in this input field 314 as the value of the second control factor.

[0076] Based on the inputs to these input fields 311 to 314, control factor data 211 corresponding to the input results is input to the evaluation device 100.

[0077] The objective characteristic region 320 has input fields 321 to 328. Input fields 321 and 325 are for inputting the names of the first objective characteristic and the second objective characteristic. For example, "Y1" is entered as the name of the first objective characteristic in input field 321, and "Y2" is entered as the name of the second objective characteristic in input field 325. Input fields 322 and 326 are for selecting the optimization objective for the first and second objective characteristics. Specifically, each of the input fields 322 and 326 has three radio buttons for selecting one of the following objectives: "Maximize," "Minimize," and "Within Specification." The objective "Maximize" aims to maximize the value of either the first or second objective characteristic, while "Minimize" aims to minimize the value of either the first or second objective characteristic. "Within Specification" aims to ensure that the value of either the first or second objective characteristic falls within the specification range. For example, if the user selects the radio button indicating "within the specification range" through an input operation to the input unit 101a, the evaluation device 100 selects "within the specification range" as the optimization objective for either the first or second objective characteristic. Input fields 323 and 324 are fields for inputting the minimum and maximum values ​​of the specification range for the first objective characteristic, respectively. For example, if "30" is entered as the minimum value in input field 323 and "40" is entered as the maximum value in input field 324, the evaluation device 100 sets the specification range to 30 to 40. Input fields 327 and 328 are fields for inputting the minimum and maximum values ​​of the specification range for the second objective characteristic, respectively. For example, if "10" is entered as the minimum value of the specification range in input field 327 and input field 328 is left blank, the evaluation device 100 sets the specification range to 0 to +∞. If input field 327 is left blank, the evaluation device 100 sets the minimum value of the specification range to -∞.

[0078] Based on the inputs to input fields 321 to 328, objective data 212 and constraint data 213 corresponding to the input results are input to the evaluation device 100. In other words, the receiving control unit 10 acquires objective data 212 corresponding to the inputs to input fields 322 and 326, and constraint data 213 corresponding to the inputs to input fields 323, 324, 327, and 328. In the example in Figure 5, the objective data 212 indicates that the value of the first objective characteristic falls within the specification range as the optimization objective of the first objective characteristic, and that the value of the second objective characteristic is minimized as the optimization objective of the second objective characteristic. Furthermore, the constraint data 213 indicates that the specification range of the first objective characteristic is 30 to 40, and that the specification range of the second objective characteristic is 10 to +∞.

[0079] Figure 6 shows an example of control factor data 211.

[0080] For example, in the example of control factor data 211 shown in Figure 6(a), the first and second control factors can take discrete values ​​from -5 to 5 in increments of 1. In the example shown in Figure 6(a), the first and second control factors are continuous variables. Continuous variables can take continuous values, but calculations are difficult if they remain continuous. Therefore, it is preferable to discretize the values ​​of each control factor and set a finite number of candidate experimental points. Accordingly, when the control factor is a continuous variable, the user inputs the level of the control factor (minimum value, maximum value, discrete range), and the evaluation device 100 determines the possible values ​​of the control factor. In other words, the receiving control unit 10 receives the level of the control factor in response to the user's input operation to the input unit 101a, and determines the possible values ​​of the control factor based on that level. The receiving control unit 10 then generates control factor data 211 showing the possible values ​​of the determined control factor and displays it, for example, in the input field 312 or 314 included in the control factor region 310 of the reception image 300 in Figure 5.

[0081] Note that variables include discrete variables, which are different from continuous variables. When a control factor is a discrete variable, that discrete variable does not have a relative order or numerical magnitude, like "apple, orange, banana" or "with catalyst, without catalyst".

[0082] In the example in Figure 6(a), the first and second control factors can take the same value, but are not limited to this. For example, as shown in Figure 6(b), the values ​​that the first and second control factors can take may be different. In the example of control factor data 211 shown in Figure 6(b), the first control factor can take discrete values ​​of 10 from 10 to 50. On the other hand, the second control factor can take discrete values ​​of 100 from 100 to 500.

[0083] In the examples shown in Figures 6(a) and 6(b), the values ​​of the control factors are absolute values, but are not limited to this. The values ​​of the control factors may be relative values, such as a ratio to the values ​​of other control factors or to the sum of the values ​​of all control factors. In the example shown in Figure 6(c), the control factor data 211 shows a ratio variable value that is different from the value of a continuous variable. The ratio variable can take the relative value of the ratio as described above. For example, as shown in Figure 6(c), the control factor data 211 may show the value of the continuous variable of the first control factor, the value of the ratio variable of the second control factor, and the value of the ratio variable of the third control factor. Specifically, the value of the continuous variable of the first control factor can take discrete values ​​from 10 to 30 in increments of 10. The ratio variable values ​​for the second control factor are, for example, "0.0, 0.2, 0.4, 0.6, 0.8, 1.0", and the ratio variable values ​​for the third control factor are, for example, "0.0, 0.2, 0.4, 0.6, 0.8, 1.0". The ratio variable indicates, for example, the mixing ratio of the first control factor material or the second control factor material in a synthetic material produced by mixing the material of the first control factor and the material of the second control factor.

[0084] Figure 7(a) shows an example of the target data 212, and Figure 7(b) shows an example of the constraint data 213.

[0085] The objective data 212 input by the objective characteristic region 320 of the reception image 300 in Figure 5 indicates the optimization objective for the first objective characteristic and the optimization objective for the second objective characteristic, as shown in Figure 7(a), for example. Similarly, the constraint data 213 input by the objective characteristic region 320 of the reception image 300 in Figure 5 indicates the specification range for the first objective characteristic and the specification range for the second objective characteristic, as shown in Figure 7(b), for example. Specifically, the objective data 212 indicates "within the specification range" as the optimization objective for the first objective characteristic and "minimize" as the optimization objective for the second objective characteristic. The constraint data 213 indicates a specification range of "30" to "40" for the first objective characteristic and a specification range of "10" to "+∞" for the second objective characteristic. Therefore, the optimization objective for the second objective characteristic is to minimize the value of the second objective characteristic within the specification range of "10" or greater.

[0086] Figure 8A shows an example of the standard range.

[0087] The specification range indicated by the constraint data 213 in Figure 7(b) is represented as a rectangular area in the characteristic space, for example, as shown in Figure 8A. In the example shown in Figure 8A, the shape of the specification range is rectangular, but it may be other shapes. In other words, the shape of the specification range can be any shape as long as the calculation of the evaluation value described later can be implemented.

[0088] Figure 8B shows another example of the standard range.

[0089] The specification range may be circular, as shown in Figure 8B. In one specific example, the specification range in the characteristic space for the first and second objective characteristics is represented by a circle with a center (20,20) and a radius of 10. The shape of the specification range may be other than a circle, such as an ellipse or a star shape.

[0090] Thus, in this embodiment, the evaluation value calculation unit 12 may calculate the evaluation value for each candidate experimental point based on a standard range of a shape other than a rectangle. As a result, since the evaluation value is calculated based on a standard range such as a circle, ellipse, or star shape in the characteristic space, the application can be further expanded without being limited to cases where the shape of the standard range is rectangular.

[0091] [Processing operation] The evaluation device 100 performs processing related to the calculation and output of evaluation values ​​using each of the input data as described above.

[0092] Figure 9 is a flowchart showing the processing operation of the evaluation device 100 in this embodiment.

[0093] First, the candidate experimental point creation unit 11 generates candidate experimental point data 221 using the control factor data 211 (step S1).

[0094] Next, the receiving control unit 10 acquires the target data 212 (step S2). In other words, the receiving control unit 10 performs a second receiving step to acquire the target data indicating the optimization objective. Furthermore, the receiving control unit 10 acquires the constraint data 213 (step S3). In other words, the receiving control unit 10 performs a third receiving step to acquire the constraint data 213 indicating the constraint conditions assigned to at least one target characteristic. Furthermore, the receiving control unit 10 reads the experimental result data 222 from the storage unit 105 (step S4). In other words, the receiving control unit 10 performs a first receiving step to acquire the experimental result data 222 indicating the experimental points that have been experimented on and the known characteristic points. If the experimental result data 222 does not show any characteristic points, the processing of steps S4 to S6, including step S4, is skipped.

[0095] The evaluation value calculation unit 12 then calculates an evaluation value for each candidate experimental point based on the target data 212, constraint data 213, candidate experimental point data 221, and experimental result data 222 (step S5). In other words, the evaluation value calculation unit 12 performs a calculation step to calculate an evaluation value for an unknown characteristic point based on that data. Specifically, the evaluation value calculation unit 12 calculates an evaluation value for each candidate experimental point that has not yet been used in an experiment, from among the multiple candidate experimental points shown in the candidate experimental point data 221. In this calculation step, the evaluation value calculation unit 12 also assigns a weight to the evaluation value for at least one target characteristic, according to the degree of fit of the constraints, as shown in (Equation 4) and (Equation 5) described later. The evaluation value calculation unit 12 then generates evaluation value data 224 showing the calculated evaluation values ​​for each candidate experimental point.

[0096] Next, the evaluation value output unit 13 outputs the evaluation value calculated in step S5, i.e., the evaluation value data 224, to the display unit 104 (step S6). In other words, the evaluation value output unit 13, The output step is executed to output the evaluation value. As a result, the evaluation value data 224 is displayed, for example, on the display unit 104.

[0097] The receiving control unit 10 then acquires an operation signal from the input unit 101a in response to an input operation by the user to the input unit 101a. The operation signal indicates the end of the search for the optimal solution or the continuation of the search for the optimal solution. The search for the optimal solution is a process that calculates and outputs evaluation values ​​for each candidate experimental point based on new experimental results. The receiving control unit 10 determines whether the operation signal indicates the end of the search for the optimal solution or the continuation of the search (step S7).

[0098] Here, when the receiving control unit 10 determines that the operation signal indicates the end of the search for the optimal solution ("end" in step S7), it terminates all processing. On the other hand, when the receiving control unit 10 determines that the operation signal indicates the continuation of the search for the optimal solution ("continue" in step S7), it writes the candidate experimental point selected as the next experimental point into the experimental result data 222 of the storage unit 105. For example, when the user performs an input operation to the input unit 101a, the receiving control unit 10 selects a candidate experimental point as the next experimental point from the evaluation value data 224. The receiving control unit 10 writes the candidate experimental point thus selected into the experimental result data 222. Then, when the characteristic point corresponding to the next experimental point is obtained experimentally, the user performs an input operation to the input unit 101a, inputting the characteristic point data 201 indicating that characteristic point into the evaluation device 100. The receiving control unit 10 acquires the input characteristic point data 201 and writes the characteristic point indicated by the characteristic point data 201 to the experimental result data 222 in the storage unit 105. At this time, the characteristic point is associated with the most recently selected and written experimental point. As a result, the new experimental result is recorded in the experimental result data 222 (step S8). In other words, the experimental result data 222 is updated. When the experimental result data 222 is updated, the evaluation value calculation unit 12 repeatedly executes the process from step S4.

[0099] Through the process outlined above, it is possible to quantitatively analyze past experimental results to determine the optimal experimental conditions (i.e., candidate experimental points) for the next experiment. As a result, a reduction in the development cycle can be expected, regardless of the skill level of the analyst, such as a user.

[0100] Figure 10A shows an example of candidate experimental point data 221.

[0101] The candidate experimental point creation unit 11 generates candidate experimental point data 221 shown in Figure 10A based on the control factor data 211 shown in Figure 6(b), for example. For example, if the values ​​of all the control factors shown in the control factor data 211 are continuous variable values ​​and there are no constraints on those values, the candidate experimental point creation unit 11 creates each of the total combinations of the values ​​of each control factor as a candidate experimental point. In the case of the control factor data 211 shown in Figure 6(b), the control factor data 211 shows the continuous variable values ​​of the first control factor "10, 20, 30, 40, 50" and the continuous variable values ​​of the second control factor "100, 200, 300, 400, 500". Therefore, the candidate experimental point creation unit 11 creates each of the total combinations as a candidate experimental point, such as the combination of the value of the first control factor "10" and the value of the second control factor "100", the combination of the value of the first control factor "10" and the value of the second control factor "200", etc. The candidate experimental point creation unit 11 associates an experimental point number with the created candidate experimental points and generates candidate experimental point data 221 indicating the candidate experimental points associated with that experimental point number.

[0102] In a specific example, candidate experimental point data 221, as shown in Figure 10A, includes candidate experimental points associated with experimental point number "1" (10,100), candidate experimental points associated with experimental point number "2" (10,200), candidate experimental points associated with experimental point number "3" (10,300), and so on. The first component of these candidate experimental points represents the value of the first control factor, and the second component represents the value of the second control factor.

[0103] Here, out of all possible combinations of values, only those that satisfy certain constraints may be created as candidate experimental points. For example, in material development, if a first compound and a second compound are set as the first and second control factors, respectively, and their mixing ratio is set as a value, the candidate experimental point creation unit 11 will select only combinations of values ​​whose sum is 1 as candidate experimental points. An example of this is shown in the candidate experimental point data 221 in Figure 10B.

[0104] Figure 10B shows another example of candidate experimental point data 221.

[0105] The candidate experimental point creation unit 11 generates candidate experimental point data 221 shown in Figure 10B based on the control factor data 211 shown in Figure 6(c), for example. In this case, the control factor data 211 shows "0.0, 0.2, 0.4, 0.6, 0.8, 1.0" as the ratio variable values ​​for the second control factor, and "0.0, 0.2, 0.4, 0.6, 0.8, 1.0" as the ratio variable values ​​for the third control factor. The combination of these ratio variable values ​​corresponds to the mixing ratio of the first compound and the second compound described above. Therefore, the candidate experimental point creation unit 11 creates combinations of the values ​​of the first control factor, the second control factor, and the third control factor as candidate experimental points such that the sum of the ratio variable values ​​of the second control factor and the ratio variable values ​​of the third control factor is 1. For example, the candidate experimental point creation unit 11 creates candidate experimental points using combinations of values ​​where the sum of the ratio variables is 1, such as a combination of the value of the first control factor "10", the value of the second control factor "0.2", and the value of the third control factor "0.8". The candidate experimental point creation unit 11 associates an experimental point number with the created candidate experimental points and generates candidate experimental point data 221 indicating the candidate experimental points associated with that experimental point number.

[0106] In a specific example, candidate experimental point data 221, as shown in Figure 10B, represents candidate experimental points associated with experimental point number "1" (10, 0.0, 1.0), candidate experimental points associated with experimental point number "2" (10, 0.2, 0.8), candidate experimental points associated with experimental point number "3" (10, 0.4, 0.6), and so on. The first component of these candidate experimental points represents the value of the first control factor, the second component represents the value of the second control factor, and the third component represents the value of the third control factor.

[0107] Thus, in this embodiment, when there are multiple control factors, the candidate experimental point creation unit 11 creates each of the multiple candidate experimental points by combining values ​​of each of the multiple control factors that satisfy predetermined conditions. For example, as shown in Figure 10B, the predetermined condition is that the sum of the values ​​of the ratio variables of each of the multiple control factors is 1. In a more specific example, the ratio variable is the blending ratio of materials such as compounds corresponding to the control factors. Therefore, an evaluation value can be calculated for each combination of blending ratios of multiple types of compounds. As a result, the optimal solution for one or more target properties of the synthetic material obtained by blending those compounds can be appropriately searched.

[0108] Figure 11 shows an example of experimental result data 222.

[0109] The evaluation value calculation unit 12 reads the experimental result data 222 stored in the storage unit 105 in order to calculate the evaluation value. As shown in Figure 11, this experimental result data 222 shows, for each experiment number, the experimental points used in the experiment identified by that experiment number, and the characteristic points, which are the experimental results obtained from that experiment. The experimental points are expressed by a combination of values ​​for each control factor. For example, the experimental points are expressed by a combination of values, which is a combination of the value of the first control factor "10" and the value of the second control factor "100". The characteristic points are expressed by a combination of values ​​for each objective characteristic obtained in the experiment. The value of the objective characteristic will also be called the objective characteristic value below. For example, the characteristic point is expressed by a combination of the value of the first objective characteristic "8" and the value of the second objective characteristic "0.0".

[0110] As a specific example, experimental result data 222 shows experimental points (10,100) and characteristic points (8, 0.0) associated with experiment number "1", experimental points (10,500) and characteristic points (40, 1.6) associated with experiment number "2", experimental points (50,100) and characteristic points (40, 1.6) associated with experiment number "3", etc., as shown in Figure 11.

[0111] [Evaluation value calculation process] Figure 12 is a diagram illustrating the processing performed by the evaluation value calculation unit 12. The evaluation value calculation unit 12 generates predicted distribution data 223 based on the candidate experimental point data 221 generated by the candidate experimental point creation unit 11 and the experimental result data 222 stored in the storage unit 105. Then, the evaluation value calculation unit 12 generates evaluation value data 224 based on the objective data 212 indicating the optimization objective of each objective characteristic, the constraint condition data 213 indicating the specification range of each objective characteristic, and the predicted distribution data 223.

[0112] Here, the experimental result data 222 shows one or more experimental points that are already used in experiments among a plurality of candidate experimental points, and characteristic points corresponding to each of those one or more experimental points, which show the experimental results of one or more target characteristics using those experimental points. Therefore, the evaluation value calculation unit 12 in this embodiment calculates the evaluation value of each of the remaining candidate experimental points, excluding one or more experimental points, based on Bayesian optimization, using (a) the optimization objective and specification range for each of the one or more target characteristics, (b) one or more experimental points that are already used in experiments among a plurality of candidate experimental points, and (c) characteristic points corresponding to each of the one or more experimental points, which show the experimental results of one or more target characteristics using those experimental points.

[0113] The evaluation value calculation unit 12 outputs the generated evaluation value data 224 to the evaluation value output unit 13. The evaluation value calculation unit 12 may also output the predicted distribution data 223 to the evaluation value output unit 13. Alternatively, the evaluation value calculation unit 12 may store the predicted distribution data 223 in the storage unit 105, and the evaluation value output unit 13 may read the predicted distribution data 223 from the storage unit 105 in response to an input operation by the user to the input unit 101a.

[0114] The evaluation value calculation unit 12 describes the correspondence between candidate experimental points and characteristic points using a Gaussian process. The Gaussian process involves a vector x of a finite number of candidate experimental points. N The vector f(x) of the corresponding characteristic points Nis a stochastic process assumed to follow an N-dimensional normal distribution. The distance between the experimental point x and the experimental point x’ is defined by a positive definite kernel k(x, x’), and the covariance matrix is expressed using this kernel. Here, N is an integer greater than or equal to 1 and is the number of executed experimental results.

[0115] Also, even if a multi-dimensional normal distribution is conditioned on some of its elements, its normality is preserved. Using this property, consider the joint distribution of the executed experimental results in a known correspondence with the candidate experimental point and the next experimental result in an unknown correspondence with the candidate experimental point, and define the distribution conditioned on the known correspondence as the predictive distribution. The mean of the predictive distribution is calculated for each dimension by the following (Equation 1), and the variance of the predictive distribution is calculated for each dimension by the following (Equation 2).

[0116] [Number]

[0117] In (Equation 1) and (Equation 2), x N =(x (1) , …, x (N) ) T represents a matrix that summarizes past experimental points, and x (N+1) represents a new candidate experimental point. y N =(y (1) , …, y (N) ) T represents a matrix that summarizes characteristic points corresponding to past experimental points. k N+1 represents an N-dimensional vector whose i-th component takes k(x (i) , x (N+1) ), and K N,N represents an N×N Gram matrix whose (i, j)-th component takes k(x (i) , x (j) ). σ 2 represents the observation error and is set to an appropriate value according to the assumed influence degree of the observation error. I represents the N-th identity matrix. The kernel k(·, ·) and its hyperparameters are appropriately set by an analyst such as a user. Here, i and j are integers greater than or equal to 1 and less than or equal to N, respectively. Also, m is called the mean function and is for y with respect to x (N+1) ​(N+1) If the behavior of the function is known to some extent, it will be set to an appropriate function. If its behavior is unknown, it may be set to a constant such as 0.

[0118] The evaluation value calculation unit 12 generates predictive distribution data 223 by performing calculations using (Equation 1) and (Equation 2) on the known experimental results shown in the experimental result data 222 read from the storage unit 105 in step S4.

[0119] Figure 13 shows an example of predictive distribution data 223. Predictive distribution data 223 shows the mean and variance of the predictive distribution at each candidate experimental point. This predictive distribution is a conditional distribution by a Gaussian process for each objective characteristic, calculated using (Equation 1) and (Equation 2). For example, as shown in Figure 13, predictive distribution data 223 shows, for each experimental point number, the mean and variance of the predictive distribution for the first objective characteristic and the mean and variance of the predictive distribution for the second objective characteristic, corresponding to that experimental point number.

[0120] In a specific example, the predictive distribution data 223, as shown in Figure 13, shows a mean of "23.5322" and a variance of "19.4012" for the first objective characteristic corresponding to experimental point number "1," and a mean of "0.77661" and a variance of "0.97006" for the second objective characteristic. Furthermore, the predictive distribution data 223 shows a mean of "30.2536" and a variance of "21.5521" for the first objective characteristic corresponding to experimental point number "2," and a mean of "1.11268" and a variance of "1.07761" for the second objective characteristic. Note that the experimental point numbers are associated with candidate experimental points, as shown in Figure 10A or Figure 10B.

[0121] The evaluation value calculation unit 12 calculates an evaluation value based on an evaluation criterion called the acquisition function in Bayesian optimization. The predictive distribution described above is used to calculate this evaluation value. Furthermore, the acquisition function in this embodiment is an acquisition function in Bayesian optimization with constraints.

[0122] First, before explaining the acquisition function in this embodiment, we will explain the acquisition function for unconstrained Bayesian optimization (i.e., EHVI in Non-Patent Document 1). However, since maximization and minimization are equivalent when the sign of one is reversed, we will explain them using only minimization. In EHVI, the larger the volume of the improvement region (also called the improvement amount), the greater the improvement in the characteristic point obtained from the provisional experimental results. This improvement region is the area enclosed by the Pareto boundary determined from the coordinates of the Pareto point (i.e., non-inferior solution) among at least one characteristic point already obtained from the experiment, and the Pareto boundary newly determined when a new characteristic point is observed. The Pareto point is the characteristic point that is currently provisionally considered to be the Pareto solution. For example, if the optimization objectives of the first objective characteristic and the second objective characteristic are minimization, then there are no other characteristic points where the values ​​of the first objective characteristic and the second objective characteristic are smaller than the Pareto point.

[0123] Figure 14A shows an example of an area for improvement.

[0124] For example, as shown in Figure 14A, a Pareto boundary 31 is determined from four Pareto points 21-24, and one new characteristic point y new The region enclosed by the newly defined Pareto boundary 32 obtained is identified as the improvement region.

[0125] Here, using Gaussian process regression, the behavior of each target characteristic value when each candidate experimental point is selected is represented in the form of a normal distribution, and the amount of improvement also varies depending on the position of the observed characteristic point. EHVI is defined as the expected value of the improvement amount obtained from the predictive distribution for each candidate experimental point, as shown in (Equation 3) below. Candidate experimental points with a larger EHVI value have a larger expected improvement amount and represent the experimental point that should be executed next.

[0126]

number

[0127] In (Equation 3), D represents the number of objective characteristics (i.e., the number of dimensions),

number

[0128] Next, the acquisition function in this embodiment will be described. The acquisition function in this embodiment is the Bayesian optimization acquisition function when constraints are present. Of the D objective properties, y1~y Dminimize D minimize The objective of optimizing individual objective characteristics is minimization, and the remaining y Dminimize+1 ~y D D range (=DD minimize Assume that the optimization objectives of the ) target characteristics are within the specified range. In this case, the acquisition function in this embodiment, i.e., the constrained EHVIC, is defined as follows (Equation 4).

[0129]

number

[0130] In (Equation 4), R minimize The objective characteristic y1~y is the objective characteristic whose optimization goal is to minimize it. Dminimize This represents the region where everything is within the standard range. range The objective characteristic y is one in which the optimization objective is within the specified range. Dminimize+1 ~y D This represents the region where everything is within the standard range. minimize and R rangeEach region is represented by a function that indicates the shape of the standard range corresponding to that region. As shown in Figure 8B, if the shape of the standard range is a circle, then R minimize and R range Each region is represented by a function that defines the circle. Also, if the shape of the standard range is star-shaped, R minimize and R range Each region is represented by a function that indicates its star shape. new,minimize is the characteristic point y new This represents a vector obtained by extracting each dimension of the objective characteristic for which the optimization objective is minimization, from all dimensions of y. new,range is the characteristic point y new This represents a vector obtained by extracting each dimension of the target characteristic for which the optimization objective is within the specification range from all dimensions of IC(y). new ) is the improvement amount when constraints are present, and represents the volume of the region enclosed by the existing Pareto boundary and the newly defined Pareto boundary. The existing Pareto boundary is the boundary determined by at least one Pareto point that lies within the specification range and the coordinates of that specification range. The newly defined Pareto boundary is the boundary determined by the coordinates of the Pareto point that is the new characteristic point and the specification range when a new characteristic point is observed. Pr{A} represents the probability that event A occurs, and can be expressed, for example, using the mean and variance of (Equation 1) and (Equation 2).

[0131] Figure 14B shows another example of the improvement region in this embodiment. The main difference between this embodiment and Non-Patent Literature 2 is that, for objective characteristics whose optimization objective is minimization, the integration range in this embodiment is restricted from the entire characteristic space to the specification range, and the method of measuring the improvement amount changes depending on the specification range. If the maximum and minimum values ​​in the specification range are not specified, the maximum value is set to +∞ and the minimum value is set to -∞. The maximum and minimum values ​​in the specification range of all objective characteristics whose optimization objective is minimization are +∞ and -∞, respectively, and D rangeWhen = 0, the acquisition function EHVIC in this embodiment is reduced to EHVI in Non-Patent Document 1. Furthermore, the maximum and minimum values ​​in the specification range of all objective characteristics for which the optimization objective is minimization are +∞ and -∞, respectively, and D range When >= 1, the acquisition function EHVIC in this embodiment reduces to the EHVIC described in Non-Patent Document 2. Therefore, the evaluation device 100 in this embodiment can calculate evaluation values ​​using conventional methods.

[0132] Furthermore, Non-Patent Document 2 describes an optimization problem in which there is one or more objective characteristics whose optimization objective is minimization, i.e., D minimize Although it is assumed that >= 1, in the acquisition function of this embodiment, D minimize =0 (i.e., D range Even in the case of =D), the formulation can be carried out without any problems. Therefore, the acquisition function in this embodiment can be naturally extended to optimization problems where the optimization objectives of all target characteristics are within the specification range.

[0133] Next, we will explain the specific method for calculating EHVIC, which is the acquisition function in this embodiment.

[0134] Figure 15A is a diagram illustrating the method for calculating the volume of the improvement area. Figure 15A(a) shows the improvement area in the characteristic space, Figure 15A(b) shows the improvement area to be divided, and Figure 15A(c) shows the multiple sub-regions obtained by dividing the improvement area.

[0135] The evaluation value calculation unit 12 calculates the dimension of the objective characteristic for which the optimization objective is minimization as shown in Figure 15A, which is the improvement amount (i.e., IC(y)), which is the volume of the improvement region. new The evaluation value calculation unit 12 divides the improvement region into multiple sub-regions at the coordinates of the Pareto point and the new characteristic point, calculates the expected value of the volume of each sub-region, and then takes the sum of these expected values ​​to calculate the improvement amount (i.e., IC(y). newThe evaluation value calculation unit 12 calculates the probability that each objective characteristic value falls within the specification range for the dimensions of the objective characteristics for which the optimization objective is within the specification range.

[0136] Figure 15B shows an example of dividing the entire characteristic space into multiple sub-regions.

[0137] The evaluation value calculation unit 12 divides the entire characteristic space into multiple sub-regions, as shown in Figure 15B, based on the dimensions of the target characteristic for which the optimization objective is minimization and the dimensions of the target characteristic for which the optimization objective is within the specification range, and calculates the acquisition function uniformly using the following (Equation 5). In other words, the evaluation value calculation unit 12 divides the entire characteristic space into multiple sub-regions at the coordinates of the Pareto point, the new characteristic point, and the specification value, and calculates the volume of each sub-region using the case-by-case calculation shown in the following (Equation 5). The specification value mentioned above is the maximum and minimum value of the specification range. The evaluation value calculation unit 12 then calculates the acquisition function in the case of constraints uniformly by taking the sum of the expected values ​​obtained from the volumes of these sub-regions. This volume is also called the N-dimensional hypervolume.

[0138]

number

[0139] In (Equation 5), y d The lower endpoint of the small region (y1, ..., y D This represents the dth component of ) and y d ' is the upper endpoint of the small region (y1', ..., y D This represents the dth component of ').

[0140] Figure 15C shows an example of the lower and upper endpoints of a small region.

[0141] When D=2, as shown in Figure 15C, (y1,y2) represents the lower endpoint of the subregion, and (y1',y2') represents the upper endpoint of the subregion.

[0142] Furthermore, (i) in (Equation 5) is the interval [y d,y d This applies when '] is outside the standard range with respect to dimension d. (ii) is the interval [y d ,y d This applies when '] is within the specification range with respect to dimension d, and the optimization objective of the objective characteristic of dimension d is within the specification range. (iii) is the interval [y d ,y d This applies when '] is within the specification range with respect to dimension d, and the optimization objective of the objective characteristic of dimension d is minimization. d c is a weighting coefficient, which is set appropriately when assigning search priorities for each dimension d of the target characteristic. For example, a smaller weighting coefficient c is used for higher priority dimensions d. d The weight coefficient c is used, and conversely, the lower the priority of dimension d, the larger the weight coefficient c. d The weight coefficient c is used. d The reciprocal of may be the priority. Unless otherwise specified, i.e., if the priorities of each dimension d are equal, then the c of each dimension d is used. d For example, all of these are set to 1. The above is a description of the acquisition function in this embodiment and the specific method for calculating that acquisition function.

[0143] The above method for calculating the acquisition function is for finding the exact solution, and the computational complexity can become enormous, especially when there are many target characteristics for which the optimization objective is minimization. Therefore, to improve computational efficiency, the acquisition function may be calculated using approximation methods such as the Monte Carlo method. Even in that case, the division of the characteristic space into smaller regions and the improvement region remain the same as described above.

[0144] [output] The evaluation value output unit 13 acquires evaluation value data 224 indicating the evaluation value of each candidate experimental point calculated by the evaluation value calculation unit 12 as described above, and displays the evaluation value data 224 on the display unit 104. The evaluation value output unit 13 may acquire the evaluation value data 224 directly from the evaluation value calculation unit 12, or it may acquire the evaluation value data 224 by reading the evaluation value data 224 stored in the storage unit 105 by the evaluation value calculation unit 12.

[0145] Figure 16 shows an example of evaluation value data 224. As shown in Figure 16, for example, the evaluation value data 224 shows the evaluation value and its rank at each candidate experimental point. Specifically, for each experimental point number, the evaluation value data 224 shows the evaluation value corresponding to that experimental point number and the rank of that evaluation value. Each experimental point number is associated with a candidate experimental point, as shown in Figures 10A and 10B. Therefore, it can be said that the evaluation value data 224 shows, for each candidate experimental point, the evaluation value corresponding to that candidate experimental point and the rank of that evaluation value. Furthermore, the rank is indicated by a smaller number for larger evaluation values ​​and, conversely, by a larger number for smaller evaluation values.

[0146] In a specific example, evaluation data 224 shows, as shown in Figure 16, an evaluation value of "0.00000" and rank of "23" corresponding to experimental point number "1", an evaluation value of "0.87682" and rank of "1" corresponding to experimental point number "2", and an evaluation value of "0.62342" and rank of "4" corresponding to experimental point number "3".

[0147] By displaying this evaluation value data 224 on the display unit 104, the user can decide whether to continue or end the search for the optimal solution. Furthermore, if the user decides to continue the search for the optimal solution, they can select the next candidate experimental point from all the displayed experimental point numbers, i.e., all candidate experimental points, based on each displayed evaluation value and rank. For example, the user selects the candidate experimental point corresponding to the largest evaluation value (i.e., the evaluation value with rank 1). At this time, the user may sort the evaluation values ​​in the evaluation value data 224 in descending order by performing an input operation on the input unit 101a. In other words, the evaluation value output unit 13 sorts the evaluation value data 224 so that each evaluation value is in descending order and each rank is in ascending order. This makes it easier to find the largest evaluation value.

[0148] As described above, the receiving control unit 10 in this embodiment acquires experimental result data 222, target data 212, and constraint data 213. The evaluation value calculation unit 12 calculates evaluation values ​​for unknown characteristic points based on the experimental result data 222, target data 212, and constraint data 213, and the evaluation value output unit 13 outputs these evaluation values. Here, the evaluation value calculation unit 12 assigns weighting to the evaluation value for at least one target characteristic according to the degree of fit of the constraints. Thus, when the evaluation value calculation unit 12 calculates evaluation values ​​for unknown characteristic points based on the experimental result data 222, target data 212, and constraint data 213, it assigns weighting to the evaluation value for at least one target characteristic according to the degree of fit of the constraints. This at least one target characteristic has an optimization objective. Therefore, Bayesian optimization can be applied to optimization problems in which constraints are applied to a target characteristic that has an optimization objective. As a result, the range of application can be expanded.

[0149] Furthermore, in this embodiment, the constraint is the specification range, and the optimization objectives include a first objective to keep the target characteristic within the specification range, and a second objective to minimize or maximize the target characteristic. The evaluation value calculation unit 12 calculates the evaluation value by performing different weighting processes for each of at least one target characteristic, depending on whether (i) the interval of the target characteristic used to calculate the evaluation value is outside the specification range, (ii) the interval is within the specification range and the optimization objective is the first objective, or (iii) the interval is within the specification range and the optimization objective is the second objective. In other words, the evaluation value is calculated based on (Equation 4) and (Equation 5) above. As a result, whether the optimization objective of the target characteristic is the first objective or the second objective, the evaluation value of the candidate experimental point can be appropriately calculated based on Bayesian optimization. Furthermore, in case (iii), since the interval of the target characteristic is within the specification range and the optimization objective is the second objective, unlike the method in Non-Patent Document 2, even when the target characteristic for which the optimization objective is maximization or minimization has a specification range as a constraint, the evaluation value can be calculated quantitatively and appropriately.

[0150] As a result, it can be applied to optimization problems with constraints such as specification ranges. In other words, it expands the range of applications and enables quantitative evaluation to improve the efficiency of searching for optimal solutions.

[0151] Furthermore, the evaluation value calculation unit 12 uses (Equation 5) c d As shown above, a priority is assigned to each of the objective characteristics, and the evaluation value of each candidate experimental point is calculated using the assigned priority. As a result, since one or more objective characteristics are assigned a priority, objective characteristics with a higher priority can be brought closer to the optimization objective faster than objective characteristics with a lower priority.

[0152] Furthermore, the evaluation value calculation unit 12 may calculate the evaluation value for each candidate experimental point using the Monte Carlo method. This reduces the processing burden of calculating the evaluation value because the Monte Carlo method is an approximation method. Specifically, the evaluation value calculation unit 12 takes the sum of the expected values ​​of the volumes of each sub-region determined by (Equation 5) for each sub-region in order to perform the calculation in (Equation 4). Therefore, for example, if there are many sub-regions, the calculation in (Equation 4) requires a great deal of computation. Thus, by using the Monte Carlo method without performing the calculation in (Equation 4) strictly, the processing burden can be reduced. Note that other methods besides the Monte Carlo method may be used as long as they are approximation methods.

[0153] (Variation 1) In the above embodiment, a specification range is set as a constraint. In this modified example, not only is a specification range provided, but a range separate from that specification range is also provided. For example, in practice, it is often required that a control range be set within the specification range to which characteristic points must be at least contained, so that characteristic points can be contained as much as possible. Note that the specification range and the control range are examples of constraint ranges, which are constraint conditions.

[0154] Figure 17 shows an example of the standard range and control range.

[0155] For example, the specification range for the first objective characteristic is a minimum value of "10" to a maximum value of "50," and the specification range for the second objective characteristic is a minimum value of "10" to a maximum value of "50." Furthermore, the control range for the first objective characteristic is a narrower range than its specification range, i.e., a minimum value of "20" to a maximum value of "40," and the control range for the second objective characteristic is a narrower range than its specification range, i.e., a minimum value of "20" to a maximum value of "40." In this way, the control range is included within the specification range.

[0156] In such cases, the evaluation value calculation unit 12 calculates the evaluation value by further setting an intermediate value, such as 0.5, between 0 and 1 for l(yd,yd'), as shown in (Equation 6) below. Note that 0.5 is just an example, and other numerical values ​​may also be used.

[0157]

number

[0158] (i) in (Equation 6) is the interval [y d ,y d This applies when '] is outside the control range and the standard range with respect to dimension d. (ii) is the interval [y d ,y d This applies when '] is within the specification range and outside the control range with respect to dimension d, and the optimization objective of the objective characteristic of dimension d is within the constraint range. (iii) is the interval [y d ,y d This applies when '] is within the control range with respect to dimension d, and the optimization objective of the objective property of dimension d is within the constraint range. (iv) is the interval [y d ,y d This applies when '] is within the control range with respect to dimension d, and the optimization objective of the objective property of dimension d is minimization.

[0159] As described above, in this modified example, when multiple constraint ranges are set as constraint conditions, the number of cases is increased according to those constraint ranges, and different weighting processes are applied to each of those cases. In other words, when there are multiple constraint ranges as constraint conditions, the evaluation value calculation unit 12 further divides case (ii) of (Equation 5) into multiple cases, such as cases (ii) and (iii) of (Equation 6), and calculates the evaluation value by applying different weighting processes to each of those multiple cases. In each of those multiple cases, the interval is included in different constraint ranges among the multiple constraint ranges. For example, as in (ii) and (iii) of (Equation 6), in case (ii), the interval is included in the specification range outside the control range, and in case (iii), the interval is included in the control range. In this way, there are multiple constraint ranges, and by further dividing case (ii) into multiple cases, it is possible to apply weighting to each of the multiple constraint ranges in a stepwise manner. Therefore, even when it is desirable that the value of the target characteristic falls within the specification range and, as far as possible, within the control range, the evaluation value can be calculated appropriately. That is, the acquisition function can be appropriately applied to the practical requirements described above. As a result, the range of applications to optimization problems can be further expanded.

[0160] Furthermore, the standard range used in the conditions for the case distinctions in (i) to (iv) of (Equation 6) above may be replaced with the control range, and conversely, the control range may be replaced with the standard range. Also, in this modified example, the standard range and the control range are used as examples of constraint ranges, but other constraint ranges may also be used. In other words, three or more constraint ranges may be used. In addition, the shapes of the multiple constraint ranges may be the same or they may be different from each other. Furthermore, their shapes may be any shape, such as rectangles, circles, ellipses, or star shapes. Also, in this modified example, there is an inclusion relationship between the multiple constraint ranges, such as the control range being included in the standard range, but the constraint ranges may not be limited to such an inclusion relationship, and they may be far apart from each other, or only a part of each constraint range may overlap.

[0161] Furthermore, as in the embodiments described above and the modified examples here, in this disclosure, there may be at least one constraint range. Therefore, the conditions in cases (i) to (iii) of (Equation 5) above and the standard range used for the first purpose may be replaced with any of the at least one constraint range. Moreover, the conditions in cases (i) to (iii) of (Equation 5) above and the constraint range used for the first purpose may be the same constraint range or they may be different constraint ranges.

[0162] (Modification 2) In the above embodiment, the evaluation value calculation unit 12 calculates an evaluation value. In this modified example, the evaluation value calculation unit 12 calculates not only the evaluation value but also the minimum distance. The minimum distance is the smallest distance between a candidate experimental point and each experimental point already used in the experiment.

[0163] Figure 18 is a diagram illustrating the processing performed by the evaluation value calculation unit 12 in this modified example.

[0164] The evaluation value calculation unit 12 calculates the minimum distance for each candidate experimental point based on the candidate experimental point data 221 and the experimental result data 222. As described above, this minimum distance is the smallest distance between each candidate experimental point and at least one experimental point already used in the experiment, in the experimental space. The evaluation value calculation unit 12 then generates minimum distance data 225 indicating the minimum distance for each candidate experimental point. The evaluation value calculation unit 12 outputs the evaluation value data 224 to the evaluation value output unit 13, as in the above embodiment, and also outputs the minimum distance data 225 to the evaluation value output unit 13. Alternatively, the evaluation value calculation unit 12 may store the minimum distance data 225 in the storage unit 105, and the evaluation value output unit 13 may read the minimum distance data 225 from the storage unit 105 in response to an input operation by the user to the input unit 101a.

[0165] The evaluation value calculation unit 12 may calculate the distance between a candidate experimental point and an experimental point already used in an experiment using, for example, the Lp distance, as shown in (Equation 7) below.

[0166]

number

[0167] In equation (7), D represents the number of control factors, one of x and x' represents a candidate experimental point, and the other represents an experimental point already used in the experiment. When p=2, Lp(x,x') represents the Euclidean distance (i.e., the straight-line distance), and when p=1, Lp(x,x') represents the Manhattan distance (i.e., the distance traveled). Also, when p=0, Lp(x,x') represents the number of different level factors, and when p=∞, Lp(x,x') represents the maximum level difference.

[0168] Figure 19 shows an example of the minimum distance.

[0169] The evaluation value calculation unit 12 calculates the Lp distance (i.e., L2 distance) when p=2, for example as shown in Figure 19. In the example shown in Figure 19, the evaluation value calculation unit 12 calculates the L2 distance between candidate experimental point A and each of the candidate experimental points B, C, D, and E that have already been used in the experiment, in the experimental space. For example, the evaluation value calculation unit 12 calculates L2(A,B)=3 as the L2 distance between candidate experimental point A and experimental point B, and L2(A,C)=1.41412 as the L2 distance between candidate experimental point A and experimental point C. Similarly, the evaluation value calculation unit 12 calculates L2(A,D)=4.47214 as the L2 distance between candidate experimental point A and experimental point D, and L2(A,E)=4.12311 as the L2 distance between candidate experimental point A and experimental point E. The evaluation value calculation unit 12 then determines "1.41412," which is the smallest of these L2 distances, as the minimum L2 distance. In the example shown in Figure 19, the evaluation value calculation unit 12 calculates the Lp distance by setting the minimum difference between two values ​​for each control factor as the reference distance 1, so that the distance does not depend on the scale of each control factor.

[0170] Figure 20 shows an example of minimum distance data 225.

[0171] The minimum distance data 225 indicates the minimum distance corresponding to each experimental point number. Each experimental point number is associated with a candidate experimental point, as shown in Figures 10A and 10B. Therefore, it can be said that the minimum distance data 225 indicates the minimum distance corresponding to each candidate experimental point.

[0172] In a specific example, the minimum distance data 225, as shown in Figure 20, represents the minimum distance "0.00000" corresponding to experimental point number "1", the minimum distance "1.00000" corresponding to experimental point number "2", the minimum distance "2.00000" corresponding to experimental point number "3", and so on. Note that the candidate experimental points corresponding to the minimum distance "0.00000" are experimental points that have already been used in experiments.

[0173] The evaluation value output unit 13 may modify the evaluation value data 224 by including the contents of the minimum distance data 225 in the evaluation value data 224, and then display the modified evaluation value data 224 on the display unit 104.

[0174] Figure 21 shows an example of the modified evaluation value data 224 displayed on the display unit 104.

[0175] The modified evaluation value data 224, as shown in Figure 21 for example, shows, for each rank of the evaluation value, the evaluation value corresponding to that rank, the candidate experimental point corresponding to that evaluation value, and the minimum distance corresponding to that candidate experimental point. Furthermore, each rank of the evaluation value is arranged in ascending order. In other words, each candidate experimental point is arranged in descending order of evaluation value. For example, evaluation value data 224 shows the evaluation value "0.87682" corresponding to rank "1", the candidate experimental point (10,200) corresponding to that evaluation value, and the minimum distance "1.00000" corresponding to that candidate experimental point. Furthermore, evaluation value data 224 shows the evaluation value "0.87682" corresponding to rank "2", the candidate experimental point (20,100) corresponding to that evaluation value, and the minimum distance "1.00000" corresponding to that candidate experimental point.

[0176] The display unit 104 shows this evaluation value data 224, allowing the user to decide whether to continue or end the search for the optimal solution. Furthermore, if the user decides to continue the search for the optimal solution, they select the next experimental point from the one or more candidate experimental points shown in the evaluation value data 224. In this modified example, the user determines the next experimental point based on the evaluation value and the minimum distance. Specifically, the user generally selects a candidate experimental point with a large evaluation value as the next experimental point. However, especially in the early stages when the number of experimental results is small, the evaluation values ​​of candidate experimental points close to past experimental points, such as those near the best-scoring experimental point in past experimental results, tend to appear higher. Even if an experiment is performed at such a candidate experimental point, there is a high probability that a characteristic point with a large improvement will not be obtained. Also, if the user wants to select multiple experimental points and perform batch processing, selecting experimental points that are far apart from each other tends to improve the accuracy of the next calculated predictive distribution and evaluation value. Therefore, the user should consider the minimum distance in addition to the evaluation value when deciding on the next experimental point.

[0177] Furthermore, users may also use the evaluation value and minimum distance as criteria for deciding whether to continue or end the search for the optimal solution. For example, if all candidate experimental points with non-zero evaluation values ​​are located near already executed experimental points, and all candidate experimental points far from the already executed points have evaluation values ​​that are almost zero, then continuing the experiment will not lead to any further improvement. Therefore, in such cases, the user should end the search.

[0178] As described above, in this modified example, the evaluation value calculation unit 12 calculates the minimum distance among the distances between each candidate experimental point and one or more other experimental points. The evaluation value output unit 13 then outputs the minimum distance corresponding to each candidate experimental point. This allows the user of the evaluation device 100 to select the next candidate experimental point not only based on the evaluation value but also on its minimum distance. For example, in the initial stages of searching for an optimal solution, the user can improve the accuracy of the evaluation value and perform appropriate optimization by selecting a candidate experimental point as the next experimental point that corresponds to an evaluation value that is relatively large and has a relatively long minimum distance.

[0179] (Other variations) The above description of an evaluation device 100 according to one aspect of the present disclosure was based on the above embodiment and its various modifications. However, the present disclosure is not limited to the above embodiment and its various modifications. Any modifications that a person skilled in the art could conceive of, without departing from the spirit of the present disclosure, may also be included in the present disclosure.

[0180] For example, in the above embodiment and its various modifications, the evaluation value calculation unit 12 calculates the evaluation value using EHVI, which is based on EI (Expected Improvement), but it may also calculate the evaluation value using PI (Probability of Improvement). That is, in the above embodiment, the evaluation value is calculated by applying EHVI as shown in (Equation 4) and (Equation 5). However, the evaluation value may also be calculated by applying PI. When PI is used, the evaluation value calculation unit 12 calculates the evaluation value using the following (Equation 8) instead of (Equation 5).

[0181]

number

[0182] The evaluation value calculation unit 12 calculates the evaluation value by performing an expected value operation on the volume calculated by (Equation 8), similar to EHVI. The conditions for the case distinctions (i) to (iii) in (Equation 8) are the same as those in (Equation 5). Furthermore, the calculation of the evaluation value using PI and the calculation of the evaluation value using EI may be combined. For example, PI may be used for the first objective characteristic and EI for the second objective characteristic.

[0183] Thus, the evaluation value calculation unit 12 of this disclosure calculates the evaluation value for each candidate experimental point using at least one of the evaluation methods PI and EI. This allows the volume within the constraint range in the characteristic space to be calculated as the improvement amount of the optimization for each candidate experimental point, and the evaluation value to be appropriately calculated from that improvement amount.

[0184] Furthermore, although the above embodiment described the case where the number of dimensions of the objective characteristics is 2, such as the first objective characteristic and the second objective characteristic, the number of dimensions may be 1 or 3 or more. Similarly, although the above embodiment mainly described the case where the total number of control factors is 2, such as the first control factor and the second control factor, the total number may be 1 or 3 or more.

[0185] In the above embodiment, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that implements the evaluation device, etc., of the above embodiment is, for example, a program that causes a computer to execute each step of the flowchart shown in Figure 9.

[0186] The following cases are also included in this disclosure.

[0187] (1) The at least one device described above is specifically a computer system consisting of a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or hard disk unit. The at least one device described above achieves its function by the operation of the microprocessor in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer in order to achieve a predetermined function.

[0188] (2) Some or all of the components constituting at least one of the above-described devices may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system comprising a microprocessor, ROM, RAM, etc. The RAM stores a computer program. The system LSI achieves its function by operating the microprocessor in accordance with the computer program.

[0189] (3) Some or all of the components constituting at least one of the above-described devices may consist of an IC card or a standalone module that is detachable from the device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-described multi-function LSI. The IC card or module achieves its function by the operation of the microprocessor in accordance with a computer program. The IC card or module may be tamper-resistant.

[0190] (4) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of a computer program.

[0191] Furthermore, this disclosure may also refer to a computer program or digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD (Compact Disc)-ROM, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, it may refer to a digital signal recorded on such a recording medium.

[0192] Furthermore, this disclosure may also include the transmission of computer programs or digital signals via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0193] Alternatively, the program or digital signal may be carried out by another independent computer system by recording and transferring it on a recording medium, or by transferring the program or digital signal via a network or the like. [Industrial applicability]

[0194] The evaluation device disclosed herein has the effect of being able to apply Bayesian optimization to optimization problems in which constraints are imposed on the objective characteristics that have the objective of the optimization problem, and can be applied not only to industrial product development or manufacturing process development, but also to devices or systems for optimal control in general development work such as material development. [Explanation of Symbols]

[0195] 10 Receiving control unit 11. Candidate Experiment Site Creation Department 12. Evaluation Value Calculation Unit 13. Evaluation Value Output Unit 100 Evaluation device 101a Input section 101b Communications Department 102 Arithmetic circuit 103 memory 104 Display section 105 Storage section 200 programs 201 Characteristic Point Data 210 Configuration Information 211 Regulatory Factor Data 212 Target Data 213 Constraint Data 221 Candidate experimental site data 222 Experimental Results Data 223 Predictive distribution data 224 Evaluation Value Data 225 Minimum distance data 300 Reception Images 310 Control factor area Input fields 311-314 320 Objective characteristic area 321-328 Input Fields

Claims

1. An evaluation device that evaluates unknown characteristic points corresponding to candidate experimental points by Bayesian optimization based on known characteristic points corresponding to experimental points that have already been tested, a first receiving means for acquiring experimental result data indicating the experimental points and the known characteristic points; the unknown characteristic points indicate values ​​of one or more objective characteristics, at least one of the objective characteristics having an optimization objective, and a second receiving means for acquiring objective data indicating the optimization objective; a third receiving means for acquiring constraint data indicating a constraint condition assigned to the at least one objective characteristic; a calculation means for calculating an evaluation value of the unknown characteristic point based on the experimental result data, the objective data, and the constraint condition data; an output means for outputting the evaluation value, the calculation means assigns a weighting to the evaluation value for the at least one objective characteristic according to the degree of conformance to the constraint condition; Evaluation equipment.

2. the constraint is at least one constraint range; The optimization objectives include a first objective of keeping a target characteristic within one of the at least one constraint ranges, and a second objective of minimizing or maximizing the target characteristic; The calculation means For each of the at least one objective characteristic: (i) the interval of the target characteristic used to calculate the evaluation value is outside each of the at least one constraint range; (ii) the interval is within one of the at least one constraint ranges, and the optimization objective is the first objective; and (iii) the section is within any one of the at least one constraint ranges, and the optimization objective is the second objective; Calculating the evaluation value by performing different weighting processes. The evaluation device according to claim 1 .

3. a candidate experiment point creation means for creating the candidate experiment points by combining values ​​of a plurality of control factors that satisfy respective predetermined conditions; The evaluation device according to claim 2 .

4. The calculation means calculating the evaluation value based on a constraint range having a shape other than a rectangle among the at least one constraint range; The evaluation device according to claim 2 or 3.

5. When the at least one constraint range includes a plurality of constraint ranges, The calculation means The case (ii) is further divided into a plurality of cases, and the evaluation value is calculated by performing a weighting process different from each other for each of the plurality of cases; In each of the plurality of cases, the section is included in different constraint ranges among the plurality of constraint ranges. The evaluation device according to any one of claims 2 to 4.

6. The calculation means assigning a priority to each of the at least one objective characteristic; calculating the evaluation value using the assigned priority; The evaluation device according to any one of claims 1 to 5.

7. The calculation means further calculating a minimum distance between the candidate experimental point and each of one or more of the experimentally tested experimental points; The output means further outputting the minimum distance corresponding to the candidate experimental point. The evaluation device according to any one of claims 1 to 6.

8. The calculation means Calculating the evaluation value using a Monte Carlo method; The evaluation device according to any one of claims 1 to 7.

9. The calculation means Calculating the evaluation value using at least one of PI (Probability of Improvement) and EI (Expected Improvement), which are evaluation methods; The evaluation device according to any one of claims 1 to 8.

10. An evaluation method in which an evaluation device evaluates unknown characteristic points corresponding to candidate experimental points by Bayesian optimization based on known characteristic points corresponding to experimental points that have been tested, a first receiving step of acquiring experimental result data indicating the experimental points and the known characteristic points; a second receiving step of acquiring objective data indicative of the unknown characteristic points, the unknown characteristic points representing values ​​of one or more objective characteristics, at least one of the objective characteristics having an optimization objective; a third receiving step of acquiring constraint data indicating a constraint condition assigned to the at least one target characteristic; a calculation step of calculating an evaluation value of the unknown characteristic point based on the experimental result data, the objective data, and the constraint condition data; an output step of outputting the evaluation value, In the calculation step, a weight according to the degree of conformance of the constraint condition is assigned to the evaluation value for the at least one objective characteristic. Evaluation method.

11. A program for a computer to evaluate unknown characteristic points corresponding to candidate experimental points by Bayesian optimization based on known characteristic points corresponding to experimental points that have already been tested, a first receiving step of acquiring experimental result data indicating the experimental points and the known characteristic points; a second receiving step of acquiring objective data indicative of the unknown characteristic points, the unknown characteristic points representing values ​​of one or more objective characteristics, at least one of the objective characteristics having an optimization objective; a third receiving step of acquiring constraint data indicating a constraint condition assigned to the at least one target characteristic; a calculation step of calculating an evaluation value of the unknown characteristic point based on the experimental result data, the objective data, and the constraint condition data; an output step of outputting the evaluation value; In the calculation step, a weight according to the degree of conformance of the constraint condition is assigned to the evaluation value for the at least one objective characteristic. program.

Images