Parameter adjustment visualization device, parameter optimization device, parameter adjustment visualization program, and parameter adjustment visualization method

Abstract

The present invention comprises: a parameter selection unit (101) that selects a parameter; a partial dependency calculation unit (102) that calculates a partial dependency value of an evaluation index value relating to the parameter selected by the parameter selection unit (101), on the basis of a parameter adjustment data set including a plurality of sets of parameter values and evaluation index values, and a trained machine learning model that can output information indicating a predicted value of the evaluation index value; an uncertainty calculation unit (103) that calculates, on the basis of the trained machine learning model and the parameter adjustment data set, an uncertainty value of the evaluation index value relating to the parameter selected by the parameter selection unit (101); and an uncertainty-attached partial dependency plot output unit (104) that outputs information indicating the partial dependency value calculated by the partial dependency calculation unit (102) and the uncertainty value calculated by the uncertainty calculation unit 103.

Description

Parameter adjustment visualization device, parameter optimization device, parameter adjustment visualization program, and parameter adjustment visualization method

[0001] The present disclosure relates to a parameter adjustment visualization device, a parameter optimization device, a parameter adjustment visualization program, and a parameter adjustment visualization method.

[0002] To improve the efficiency of parameter tuning, the application of parameter tuning methods using machine learning, such as Bayesian optimization, is becoming more widespread. In parameter tuning using Bayesian optimization, the relationship between parameter values ​​and evaluation index values ​​is optimized while learning using a machine learning model called a surrogate model, and parameter tuning is performed. Evaluation index values ​​are evaluation values ​​and constraint values.

[0003] On the other hand, there are cases where the parameter values ​​obtained after adjustment require explanations or visualizations that are easy for humans to understand. For example, the following cases can be considered.

[0004] For example, in order to check whether parameter fluctuations due to the influence of disturbances such as noise cause a sudden deterioration in the evaluation index values ​​or whether constraints are violated, it may be desirable to visualize the fluctuations in the evaluation index values ​​around the obtained parameter values. Also, for example, when an adjustment implementer wants to obtain parameter values ​​that will provide a better solution by further manually fine-tuning the parameter values ​​obtained based on Bayesian optimization, it may be desirable to visualize the fluctuations in the evaluation index values ​​around the obtained parameter values.

[0005] For such tasks, a partial dependency plot (PDP) is disclosed in, for example, Non-Patent Document 1, as a technique for calculating and visualizing the average relationship between parameter values ​​and evaluation index values ​​when the values ​​of one or two parameters of interest vary, using a dataset in which parameter values ​​and evaluation index values ​​are paired and a trained machine learning model.

[0006] This PDP shows the marginal effect that one or two features have on the prediction results of the machine learning model. Partial dependence marginalizes the output of the machine learning model relative to the distribution of features in set (C), thereby showing the relationship between the feature of interest in set (S) and the prediction results. By marginalizing relative to other features, a function that depends only on the features in S can be obtained, including interactions with other features. Set (S) is a set of one or more features of interest (the target of the PDP plot), and set (C) is the feature of the machine learning model excluding set (S).

[0007] Friedman, Jerome H, ``Greedy function approximation: A gradient boosting machine.'' Annals of statistics 2001

[0008] As mentioned above, partial dependence plots only calculate and visualize the relationship between the average evaluation index of a machine learning model and parameter variations. Therefore, if there is variation (uncertainty) in the output of the evaluation index values, it is not possible to calculate and visualize it. In particular, when adjusting multidimensional parameters, the influence of parameters other than the parameter of interest tends to increase the variation.

[0009] For example, when a parameter value is decreased, the partial dependence plot may visualize an increase in the evaluation index value, but in reality the evaluation index value may decrease.

[0010] Therefore, it is necessary to visualize these uncertainties together. However, Non-Patent Document 1 does not disclose a method for expressing a partial dependence plot by combining uncertainties.

[0011] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a parameter adjustment visualization device that can visualize the uncertainty of evaluation index values.

[0012] The parameter adjustment visualization device according to the present disclosure is characterized by comprising: a parameter selection unit that selects parameters; a partial dependency calculation unit that calculates partial dependency values ​​of evaluation index values ​​related to the parameters selected by the parameter selection unit based on a trained machine learning model capable of outputting information indicating predicted values ​​of evaluation index values ​​and a parameter adjustment dataset including a plurality of pairs of parameter values ​​and evaluation index values; an uncertainty calculation unit that calculates uncertainty values ​​of the evaluation index values ​​related to the parameters selected by the parameter selection unit based on the trained machine learning model and the parameter adjustment dataset; and a partial dependency plot output unit with uncertainty that outputs information indicating the partial dependency values ​​calculated by the partial dependency calculation unit and the uncertainty values ​​calculated by the uncertainty calculation unit.

[0013] According to the present disclosure, with the above configuration, it becomes possible to visualize the uncertainty of the evaluation index values.

[0014] FIG. 2A is a diagram illustrating an example of a configuration of a parameter adjustment visualization device according to a first embodiment. FIG. 2A and FIG. 2B are diagrams illustrating an example of a parameter adjustment dataset and a trained machine learning model used in the parameter adjustment visualization device according to the first embodiment, where FIG. 2A is a diagram illustrating an example of the parameter adjustment dataset and FIG. 2B is a diagram illustrating an example of the trained machine learning model. FIG. 3A and FIG. 3B are diagrams illustrating uncertainty handled by the parameter adjustment visualization device according to the first embodiment, where FIG. 3A is a diagram illustrating the uncertainty of the problem and FIG. 3B is a diagram illustrating the uncertainty of learning. FIG. 3B is a flowchart illustrating an example of operation of the parameter adjustment visualization device according to the first embodiment. FIG. 4 is a diagram illustrating an effect of the parameter adjustment visualization device according to the first embodiment. FIG. 4 is a diagram illustrating an example of a configuration of a parameter adjustment visualization device according to a second embodiment. FIG. 4 is a flowchart illustrating an example of operation of the parameter adjustment visualization device according to the second embodiment. FIG. 4 is a diagram illustrating an example of a configuration of a parameter adjustment visualization device according to a third embodiment. FIG. 4 is a flowchart illustrating an example of operation of the parameter adjustment visualization device according to the third embodiment. FIG. 4 is a diagram illustrating an example of operation of the parameter adjustment visualization device according to the third embodiment. FIG. 4 is a diagram illustrating an example of operation of the parameter adjustment visualization device according to the third embodiment. FIG. 21A is a diagram showing an example of the configuration of a parameter adjustment visualization device according to embodiment 5. FIG. 21B is a flowchart showing an example of the operation of the parameter adjustment visualization device according to embodiment 5. FIG. 21C is a diagram showing an example of the configuration of a parameter optimization device according to embodiment 6. FIG. 21D is a flowchart showing an example of the operation of the parameter optimization device according to embodiment 6. FIG. 21D is a diagram showing an example of the configuration of a parameter optimization device according to embodiment 7. FIG. 21E is a flowchart showing an example of the operation of the parameter optimization device according to embodiment 7. FIGS. 21A and 21B are diagrams showing examples of the hardware configuration of a parameter adjustment visualization device according to any one of embodiments 1 to 7.

[0015] Hereinafter, embodiments will be described in detail with reference to the drawings. Embodiment 1. Fig. 1 is a diagram showing an example of the configuration of a parameter adjustment visualization device 1 according to embodiment 1. The parameter adjustment visualization device 1 outputs information indicating partial dependence and uncertainty based on a trained machine learning model and a parameter adjustment dataset.

[0016] The trained machine learning model is a trained machine learning model that can output information indicating the predicted value of the evaluation index value, or a trained machine learning model that can output information indicating the predicted value of the evaluation index value and the uncertainty value of the prediction.

[0017] Examples of trained machine learning models that can output information indicating the predicted value of the evaluation index value and the uncertainty value of the prediction include a Gaussian process regression model, a Bayesian linear regression model, a Bayesian neural network, a neural network using DropOut, a quantile regression model, and a random forest.

[0018] The parameter adjustment dataset is a dataset that includes multiple pairs of parameter values ​​(explanatory variables) that serve as inputs to machine learning and evaluation index values ​​(objective variables) that serve as outputs of machine learning.

[0019] Examples of parameters include setting parameters of a device, apparatus, or system, design parameters of a device, apparatus, or system, operation parameters of a device, apparatus, or system, processing conditions, or operating conditions. Examples of parameter value types include continuous values ​​or categorical values. Examples of evaluation index types include evaluation indexes or constraint indexes. Examples of evaluation index value types include continuous values ​​or numeric values.

[0020] The parameter adjustment data set may be a parameter adjustment data set including a plurality of pairs of parameter values ​​and evaluation index values ​​obtained by Bayesian optimization.

[0021] 2 shows an example of a parameter adjustment dataset and a trained machine learning model used in the parameter adjustment visualization device 1 according to embodiment 1. The parameter adjustment dataset shown in Fig. 2A uses a dataset including multiple pairs of values ​​for parameter A (speed), parameter B (acceleration), and parameter C (gain), and values ​​for an evaluation value (positioning time) and a constraint value (vibration amplitude). Furthermore, the trained machine learning model shown in Fig. 2B uses a machine learning model for evaluation values ​​and a machine learning model for constraint values.

[0022] As shown in FIG. 1, the parameter adjustment visualization device 1 includes a parameter selection unit 101, a partial dependency calculation unit 102, an uncertainty calculation unit 103, and an uncertainty-added partial dependency plot output unit 104.

[0023] The parameter selection unit 101 selects parameters. Furthermore, when there are multiple trained machine learning models, the parameter selection unit 101 also selects a trained machine learning model. Any method can be used as the selection method by the parameter selection unit 101.

[0024] The parameter selection unit 101 may select all parameters indicated by the parameter adjustment dataset, or may select one or more parameters manually specified by the user from among the parameters indicated by the parameter adjustment dataset.

[0025] Furthermore, the parameter selection unit 101 may select all trained machine learning models from the trained machine learning models, or may select one or more trained machine learning models manually specified by the user from the trained machine learning models.

[0026] Furthermore, when three or more parameters are selected, the parameter selection unit 101 may be capable of selecting a combination of parameters to be displayed in the uncertainty-attached partial dependency plot output unit 104 from among the selected three or more parameters. That is, the uncertainty-attached partial dependency plot output unit 104 basically enables visualization in one dimension or two dimensions. Therefore, when three or more parameters are selected, it may be impossible to visualize information about the three or more parameters together on a single display screen. In such a case, multiple combinations are visualized. Furthermore, since some of the multiple combinations may not be necessary for the user to visualize, the parameter selection unit 101 may be capable of selecting a combination of parameters to be visualized. Note that when the parameter selection unit 101 does not select a combination of parameters to be visualized, it is desirable that the uncertainty-attached partial dependency plot output unit 104 visualize all combinations of parameters.

[0027] The partial dependency calculation unit 102 calculates a partial dependency value based on the trained machine learning model, the parameter adjustment dataset, and the selection result by the parameter selection unit 101. The partial dependency value is a value that indicates the average degree of change in the fluctuation of the evaluation index value with respect to the fluctuation of the value of the parameter selected by the parameter selection unit 101. Note that, when a trained machine learning model is selected by the parameter selection unit 101, the partial dependency calculation unit 102 calculates the partial dependency value using the selected trained machine learning model.

[0028] The uncertainty calculation unit 103 calculates an uncertainty value based on the trained machine learning model, the parameter adjustment dataset, and the selection result by the parameter selection unit 101. The uncertainty value is a value that indicates the uncertainty of fluctuations in the evaluation index value with respect to fluctuations in the value of the parameter selected by the parameter selection unit 101. Note that, when a trained machine learning model is selected by the parameter selection unit 101, the uncertainty calculation unit 103 calculates the uncertainty value using the selected trained machine learning model.

[0029] The uncertainty includes, for example, problem uncertainty and learning uncertainty, as shown in FIG. 3 . Problem uncertainty is uncertainty due to data variations (interpolated data) caused by measurement noise or disturbances, as shown in FIG. 3A . Learning uncertainty is uncertainty due to regions (extrapolated data) that do not exist in the learning data, as shown in FIG. 3B . The uncertainty calculation unit 103 then calculates at least one of the problem uncertainty and learning uncertainty.

[0030] The uncertainty calculation unit 103 may calculate the uncertainty using a predicted value of the evaluation index value, or may calculate the uncertainty using a predicted value of the uncertainty value (variance or standard deviation), or may calculate the uncertainty value using both. Details of this uncertainty calculation unit 103 will be described in embodiment 2-4.

[0031] The partial dependency plot with uncertainty output unit 104 outputs information indicating the partial dependency values ​​calculated by the partial dependency calculation unit 102 and the uncertainty values ​​calculated by the uncertainty calculation unit 103 .

[0032] For example, the partial dependency plot output unit 104 with uncertainty may display a broken line with intervals, with the value of the parameter selected by the parameter selection unit 101 on the horizontal axis and the value of the evaluation index selected by the parameter selection unit 101 on the vertical axis. For example, the partial dependency plot output unit 104 with uncertainty may display a contour line with the value of the first parameter selected by the parameter selection unit 101 on the horizontal axis and the value of the second parameter selected by the parameter selection unit 101 on the vertical axis, colored according to the intensity of the evaluation index value selected by the parameter selection unit 101. For example, the partial dependency plot output unit 104 with uncertainty may display a contour line with the value of the first parameter selected by the parameter selection unit 101 on the horizontal axis and the value of the second parameter selected by the parameter selection unit 101 on the vertical axis, colored according to the intensity of the uncertainty value. For example, the uncertainty-attached partial dependence plot output unit 104 may display a three-dimensional plot in which the value of the first parameter selected by the parameter selection unit 101 is on the x-axis, the value of the second parameter selected by the parameter selection unit 101 is on the y-axis, and the evaluation index value selected by the parameter selection unit 101 is on the z-axis.

[0033] Furthermore, the uncertainty-added partial dependence plot output unit 104 may display the above displays together or separately.

[0034] The partial dependence plot output unit 104 with uncertainty may also display any solution obtained by parameter adjustment as a scatter plot. The partial dependence plot output unit 104 with uncertainty may also display ICE (Individual Conditional Expectation) plots. ICE plots are a method of visualizing, with a single line, how the prediction of individual data changes when a certain feature value changes.

[0035] Next, an example of the operation of the parameter adjustment visualization device 1 according to the first embodiment shown in FIG. 1 will be described with reference to FIG.

[0036] In the operation example of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1, first, the parameter selection unit 101 selects a parameter (step ST101), as shown in Fig. 4. Furthermore, when there are multiple trained machine learning models, the parameter selection unit 101 also selects a trained machine learning model.

[0037] Next, the partial dependency calculation unit 102 calculates the partial dependency value based on the trained machine learning model, the parameter adjustment dataset, and the selection result by the parameter selection unit 101 (step ST102). Note that, if a trained machine learning model has been selected by the parameter selection unit 101, the partial dependency calculation unit 102 calculates the partial dependency value using the selected trained machine learning model.

[0038] Furthermore, the uncertainty calculation unit 103 calculates an uncertainty value based on the trained machine learning model, the parameter adjustment dataset, and the selection result by the parameter selection unit 101 (step ST103). Note that, when a trained machine learning model is selected by the parameter selection unit 101, the uncertainty calculation unit 103 calculates the uncertainty value using the selected trained machine learning model.

[0039] Next, the partial dependency plot output unit 104 with uncertainty outputs information indicating the partial dependency value calculated by the partial dependency calculation unit 102 and the uncertainty value calculated by the uncertainty calculation unit 103 (step ST104).

[0040] Conventionally, partial dependence calculations are performed and partial dependence plots are output. In contrast, the parameter adjustment visualization device 1 according to the first embodiment calculates the uncertainty of the evaluation index values ​​in addition to the partial dependence values ​​of the evaluation index values, and outputs this information. As a result, as shown in FIG. 5 , for example, the parameter adjustment visualization device 1 according to the first embodiment makes it possible to confirm the degree of fluctuation of evaluation indexes, such as evaluation values ​​and constraint values ​​for parameters, taking into account the uncertainty values, thereby improving the explainability to the user of the parameters obtained after adjustment. In FIG. 5 , reference numeral 11 denotes the partial dependence values ​​of the evaluation index values, and reference numeral 12 denotes the uncertainty values ​​of the evaluation index values.

[0041] The uncertainty value obtained by the uncertainty calculation unit 103 is a relative value to the partial dependency value obtained by the partial dependency calculation unit 102. Therefore, the partial dependency plot output unit with uncertainty 104 can obtain the upper and lower limit values ​​of the uncertainty value by adding and subtracting the uncertainty value from the partial dependency value.

[0042] As described above, according to the first embodiment, the parameter adjustment visualization device 1 includes: a parameter selection unit 101 that selects parameters; a partial dependency calculation unit 102 that calculates partial dependency values ​​of evaluation index values ​​related to the parameters selected by the parameter selection unit 101 based on a trained machine learning model capable of outputting information indicating predicted values ​​of evaluation index values ​​and a parameter adjustment dataset including a plurality of pairs of parameter values ​​and evaluation index values; an uncertainty calculation unit 103 that calculates uncertainty values ​​of the evaluation index values ​​related to the parameters selected by the parameter selection unit 101 based on the trained machine learning model and the parameter adjustment dataset; and an uncertainty-attached partial dependency plot output unit 104 that outputs information indicating the partial dependency values ​​calculated by the partial dependency calculation unit 102 and the uncertainty values ​​calculated by the uncertainty calculation unit 103. Furthermore, according to the first embodiment, the trained machine learning model is a Gaussian process regression model. Furthermore, according to the first embodiment, the parameter adjustment dataset is a parameter adjustment dataset including a plurality of pairs of parameter values ​​and evaluation index values ​​obtained by Bayesian optimization. As a result, the parameter adjustment visualization device 1 according to the first embodiment can visualize the uncertainty of the evaluation index values.

[0043] Moreover, according to the first embodiment, the parameter adjustment visualization method includes the steps of: a parameter selection unit 101 selecting parameters; a partial dependency calculation unit 102 calculating partial dependency values ​​of evaluation index values ​​related to the parameters selected by the parameter selection unit 101, based on a trained machine learning model capable of outputting information indicating predicted values ​​of evaluation index values ​​and a parameter adjustment dataset including a plurality of pairs of parameter values ​​and evaluation index values; an uncertainty calculation unit 103 calculating uncertainty values ​​of the evaluation index values ​​related to the parameters selected by the parameter selection unit 101, based on the trained machine learning model and the parameter adjustment dataset; and a partial dependency plot output unit 104 outputting information indicating the partial dependency values ​​calculated by the partial dependency calculation unit 102 and the uncertainty values ​​calculated by the uncertainty calculation unit 103. As a result, the parameter adjustment visualization method according to the first embodiment makes it possible to visualize the uncertainty of the evaluation index values.

[0044] Second Embodiment Fig. 6 is a diagram showing an example of the configuration of a parameter adjustment visualization device 1 according to a second embodiment. In the parameter adjustment visualization device 1 according to the second embodiment shown in Fig. 6, the internal configuration of an uncertainty calculation unit 103 is added to the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. The other example of the configuration of the parameter adjustment visualization device 1 according to the second embodiment shown in Fig. 6 is the same as the example of the configuration of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1, and the same reference numerals are used to denote the same components, and description thereof will be omitted.

[0045] The trained machine learning model used in the parameter adjustment visualization device 1 according to embodiment 2 is a trained machine learning model capable of outputting information indicating the predicted value of the evaluation index value and the uncertainty value of the prediction.

[0046] The uncertainty calculation unit 103 in embodiment 2 includes, for example, as shown in FIG. 6, an uncertainty calculation dataset generation unit 1031a, a trained model prediction unit 1032a, and a partial dependent uncertainty calculation unit 1033a.

[0047] The uncertainty calculation data set generation unit 1031 a generates a plurality of uncertainty calculation data sets based on the parameter adjustment data set and the selection result by the parameter selection unit 101 .

[0048] In this case, first, the uncertainty calculation dataset generation unit 1031a generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031a divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0049] The trained model prediction unit 1032a predicts the uncertainty values ​​(variance or standard deviation) by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031a into the trained machine learning model.

[0050] The partial dependence uncertainty calculation unit 1033a calculates the average of the uncertainty values ​​predicted by the trained model prediction unit 1032a. The average of the uncertainties calculated by the partial dependence uncertainty calculation unit 1033a becomes the calculation result by the uncertainty calculation unit 103.

[0051] The uncertainty calculation unit 103 in the second embodiment can handle both problem uncertainty and learning uncertainty.

[0052] Next, an operation example of the parameter adjustment visualization device 1 according to the second embodiment shown in Fig. 6 will be described with reference to Fig. 7. In the operation example of the parameter adjustment visualization device 1 according to the second embodiment shown in Fig. 6, the operation example other than the operation example of the uncertainty calculation unit 103 is the same as the operation example of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. Therefore, only the operation of the uncertainty calculation unit 103 will be described below.

[0053] In an example of operation of the parameter adjustment visualization device 1 according to embodiment 2 shown in FIG. 6, as shown in FIG. 7, the uncertainty calculation dataset generation unit 1031a generates multiple uncertainty calculation datasets based on the parameter adjustment dataset and the selection result by the parameter selection unit 101 (step ST1031a).

[0054] In this case, first, the uncertainty calculation dataset generation unit 1031a generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031a divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0055] Next, the trained model prediction unit 1032a predicts the uncertainty values ​​(variance or standard deviation) by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031a into the trained machine learning model (step ST1032a).

[0056] Next, the partial dependence uncertainty calculation unit 1033a calculates the average of the uncertainty values ​​predicted by the learned model prediction unit 1032a (step ST1033a). The average of the uncertainties calculated by the partial dependence uncertainty calculation unit 1033a becomes the calculation result by the uncertainty calculation unit 103.

[0057] A case will be described in which the uncertainty calculation unit 103 in the second embodiment calculates the uncertainty value using a parameter adjustment data set such as that shown in the upper part of Fig. 8. It is assumed that the parameter selection unit 101 selects x0.

[0058] In this case, first, the uncertainty calculation dataset generation unit 1031a generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031a divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0059] In the example of FIG. 8 , as shown in the middle row, in the first uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.22) of the first set in the parameter adjustment dataset, and the values ​​of the other parameters are unchanged from the values ​​of the parameter adjustment dataset. Similarly, in the second uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.23) of the third set in the parameter adjustment dataset, and the values ​​of the other parameters are unchanged from the values ​​of the parameter adjustment dataset. Similarly, in the third uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.30) of the second set in the parameter adjustment dataset, and the values ​​of the other parameters are unchanged from the values ​​of the parameter adjustment dataset.

[0060] 8 illustrates a case in which the uncertainty calculation dataset generation unit 1031a generates multiple duplicate datasets by duplicating the parameter adjustment dataset by the number of pairs, and then divides the values ​​of each parameter pair selected by the parameter selection unit 101 in the parameter adjustment dataset into duplicate datasets and assigns them as common values ​​for each parameter pair in the duplicate dataset, thereby generating multiple uncertainty calculation datasets. That is, in the example of FIG. 8 , an uncertainty calculation dataset in which the x0 value is 0.22, an uncertainty calculation dataset in which the x0 value is 0.23, and an uncertainty calculation dataset in which the x0 value is 0.30 are generated. On the other hand, for example, even if a pair with x0 = 0.25 does not exist in the parameter adjustment dataset, an additional uncertainty calculation dataset corresponding to this may be generated. This makes it possible to complement uncertainty at values ​​not present in the parameter adjustment dataset, thereby improving accuracy.

[0061] Next, the trained model prediction unit 1032a predicts each uncertainty value (variance or standard deviation) by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031a into the trained machine learning model. The lower part of Figure 8 shows the prediction results of uncertainty values ​​(0.4, 0.3, 0.5, ...) when an uncertainty calculation dataset in which the value of x0 is 0.22 is used, and the prediction results of uncertainty values ​​(0.38, 0.27, 0.55, ...) when an uncertainty calculation dataset in which the value of x0 is 0.23 is used.

[0062] Next, the partial dependent uncertainty calculation unit 1033a calculates the average of the uncertainty values ​​predicted by the trained model prediction unit 1032a. In Figure 8, average_fc(x) is a function that outputs the average of the predicted values ​​of the uncertainty values ​​corresponding to the input x. In the example of Figure 8, when an uncertainty calculation dataset in which the value of x0 is 0.22 is used, the uncertainty prediction results (0.4, 0.3, 0.5, ...) yield an average of 0.33, and when an uncertainty calculation dataset in which the value of x0 is 0.23 is used, the uncertainty prediction results (0.38, 0.27, 0.55, ...) yield an average of 0.28.

[0063] As described above, according to the second embodiment, the trained machine learning model is a trained machine learning model that can output information indicating the predicted values ​​of the evaluation index values ​​and the uncertainty value of the prediction. The uncertainty calculation unit 103 includes an uncertainty calculation dataset generation unit 1031a that generates multiple uncertainty calculation datasets by duplicating the parameter adjustment dataset a number of times equal to or greater than the number of pairs to generate multiple duplicate datasets, and dividing values ​​in the parameter adjustment datasets, including values ​​for each parameter pair selected by the parameter selection unit 101, into the duplicate datasets and assigning them as common values ​​for each parameter pair in the duplicate datasets; a trained model prediction unit 1032a that inputs the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031a into the trained machine learning model and predicts each uncertainty value; and a partial dependent uncertainty calculation unit 1033a that calculates the average of the uncertainty values ​​predicted by the trained model prediction unit 1032a. This makes it possible for the parameter adjustment visualization device 1 according to the second embodiment to obtain uncertainty.

[0064] Embodiment 3. Fig. 9 is a diagram showing an example of the configuration of a parameter adjustment visualization device 1 according to embodiment 3. In the parameter adjustment visualization device 1 according to embodiment 3 shown in Fig. 9, the internal configuration of an uncertainty calculation unit 103 is added to the parameter adjustment visualization device 1 according to embodiment 1 shown in Fig. 1. The other example of the configuration of the parameter adjustment visualization device 1 according to embodiment 3 shown in Fig. 9 is the same as the example of the configuration of the parameter adjustment visualization device 1 according to embodiment 1 shown in Fig. 1, and the same reference numerals are used and their description will be omitted.

[0065] The uncertainty calculation unit 103 in embodiment 3 includes, for example, as shown in FIG. 9, an uncertainty calculation dataset generation unit 1031b, a trained model prediction unit 1032b, and a partial dependent uncertainty calculation unit 1033b.

[0066] The uncertainty calculation data set generation unit 1031 b generates a plurality of uncertainty calculation data sets based on the parameter adjustment data set and the selection result by the parameter selection unit 101 .

[0067] In this case, first, the uncertainty calculation dataset generation unit 1031b generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031b divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0068] The trained model prediction unit 1032b predicts the evaluation index values ​​by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031b into the trained machine learning model.

[0069] The partial dependence uncertainty calculation unit 1033b calculates the variance or standard deviation of the evaluation index values ​​predicted by the trained model prediction unit 1032b as an uncertainty value. The uncertainty value calculated by the partial dependence uncertainty calculation unit 1033b becomes the calculation result by the uncertainty calculation unit 103.

[0070] The uncertainty calculation unit 103 in the third embodiment can handle the uncertainty of the problem.

[0071] Next, an operation example of the parameter adjustment visualization device 1 according to the third embodiment shown in Fig. 9 will be described with reference to Fig. 10. In the operation example of the parameter adjustment visualization device 1 according to the third embodiment shown in Fig. 9, the operation example other than the operation example of the uncertainty calculation unit 103 is the same as the operation example of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. Therefore, only the operation part of the uncertainty calculation unit 103 will be described below.

[0072] In an example of operation of the parameter adjustment visualization device 1 according to embodiment 3 shown in FIG. 9, for example, as shown in FIG. 10, the uncertainty calculation dataset generation unit 1031b generates multiple uncertainty calculation datasets based on the parameter adjustment dataset and the selection result by the parameter selection unit 101 (step ST1031b).

[0073] In this case, first, the uncertainty calculation dataset generation unit 1031b generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031b divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0074] Next, the trained model prediction unit 1032b predicts the evaluation index values ​​by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031b into the trained machine learning model (step ST1032b).

[0075] Next, the partial dependence uncertainty calculation unit 1033b calculates the variance or standard deviation of the evaluation index values ​​predicted by the trained model prediction unit 1032b as uncertainty values ​​(step ST1033b). The uncertainty values ​​calculated by the partial dependence uncertainty calculation unit 1033b become the calculation results of the uncertainty calculation unit 103.

[0076] A case will be described in which the uncertainty calculation unit 103 in the third embodiment calculates the uncertainty value using a parameter adjustment data set such as that shown in the upper part of Fig. 11. It is assumed that the parameter selection unit 101 selects x0.

[0077] In this case, first, the uncertainty calculation dataset generation unit 1031b generates multiple duplicate datasets by duplicating the parameter adjustment dataset a number equal to or greater than the number of sets. Then, the uncertainty calculation dataset generation unit 1031b divides values ​​in the parameter adjustment dataset, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicate datasets, and assigns the values ​​as common values ​​for each set of parameters in the duplicate datasets, thereby generating multiple uncertainty calculation datasets.

[0078] In the example of FIG. 11 , as shown in the middle row, in the first uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.22) of the first set of the parameter adjustment dataset, and the values ​​of the other parameters remain unchanged from the values ​​of the parameter adjustment dataset. Similarly, in the second uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.23) of the third set of the parameter adjustment dataset, and the values ​​of the other parameters remain unchanged from the values ​​of the parameter adjustment dataset. Similarly, in the third uncertainty calculation dataset from the left, all of the x0 values ​​are replaced with the x0 value (0.30) of the second set of the parameter adjustment dataset, and the values ​​of the other parameters remain unchanged from the values ​​of the parameter adjustment dataset.

[0079] 11 illustrates a case in which the uncertainty calculation dataset generation unit 1031b generates multiple duplicate datasets by duplicating the parameter adjustment dataset by the number of pairs, and then divides the values ​​of each parameter pair selected by the parameter selection unit 101 in the parameter adjustment dataset into duplicate datasets and assigns them as common values ​​for each parameter pair in the duplicate dataset, thereby generating multiple uncertainty calculation datasets. That is, in the example of FIG. 11 , an uncertainty calculation dataset in which the x0 value is 0.22, an uncertainty calculation dataset in which the x0 value is 0.23, and an uncertainty calculation dataset in which the x0 value is 0.30 are generated. On the other hand, for example, even if a pair with x0 = 0.25 does not exist in the parameter adjustment dataset, an additional uncertainty calculation dataset corresponding to this may be generated. This makes it possible to complement uncertainty at values ​​not present in the parameter adjustment dataset, thereby improving accuracy.

[0080] Next, the trained model prediction unit 1032b predicts each evaluation index value by inputting the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031b into the trained machine learning model. The lower part of Figure 11 shows the prediction results (0.82, 0.91, 0.75, ...) of the evaluation index values ​​when an uncertainty calculation dataset in which the value of x0 is 0.22 is used, and the prediction results (0.85, 0.96, 0.71, ...) of the evaluation index values ​​when an uncertainty calculation dataset in which the value of x0 is 0.23 is used.

[0081] Next, the partial dependent uncertainty calculation unit 1033b calculates the variance or standard deviation of the evaluation index values ​​predicted by the trained model prediction unit 1032b as the uncertainty value. In FIG. 11 , var / std_f(x) is a function that outputs the variance or standard deviation of the predicted values ​​of the evaluation index values ​​corresponding to the input x. In the example of FIG. 11 , an uncertainty value of 0.11 is obtained from the prediction results (0.82, 0.91, 0.75, ...) of the evaluation index values ​​when an uncertainty calculation dataset in which the value of x0 is 0.22 is used, and an uncertainty value of 0.14 is obtained from the prediction results (0.85, 0.96, 0.71, ...) of the evaluation index values ​​when an uncertainty calculation dataset in which the value of x0 is 0.23 is used.

[0082] As described above, according to the third embodiment, the uncertainty calculation unit 103 includes: an uncertainty calculation dataset generation unit 1031b that generates multiple uncertainty calculation datasets by duplicating the parameter adjustment dataset a number of times equal to or greater than the number of sets, and dividing values ​​in the parameter adjustment datasets, including values ​​for each set of parameters selected by the parameter selection unit 101, into the duplicated datasets and assigning them as common values ​​for each set of parameters in the duplicated datasets; a trained model prediction unit 1032b that inputs the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit 1031b into a trained machine learning model and predicts each evaluation index value; and a partial dependence uncertainty calculation unit 1033b that calculates, as an uncertainty value, the variance or standard deviation of the evaluation index values ​​predicted by the trained model prediction unit 1032b. This makes it possible for the parameter adjustment visualization device 1 according to the third embodiment to obtain uncertainty.

[0083] Fourth Embodiment Fig. 12 is a diagram showing an example of the configuration of a parameter adjustment visualization device 1 according to a fourth embodiment. In the parameter adjustment visualization device 1 according to the fourth embodiment shown in Fig. 12, the internal configuration of an uncertainty calculation unit 103 is added to the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. The other example of the configuration of the parameter adjustment visualization device 1 according to the fourth embodiment shown in Fig. 12 is similar to the example of the configuration of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1, and the same reference numerals are used, and only the different parts will be described.

[0084] The trained machine learning model used in the parameter adjustment visualization device 1 according to embodiment 2 is a trained machine learning model capable of outputting information indicating the predicted value of the evaluation index value and the uncertainty value of the prediction.

[0085] Furthermore, the parameter selection unit 101 in the fourth embodiment specifies values ​​for parameters other than the selected parameters, and the values ​​for the parameters other than the selected parameters may be any values.

[0086] For example, the parameter selection unit 101 may individually specify values ​​for parameters other than the selected parameters. Alternatively, for example, the parameter selection unit 101 may use an optimal value in the parameter adjustment data set and set the value for the parameter other than the selected parameter as the optimal value. Alternatively, for example, the parameter selection unit 101 may use the latest search value in the parameter adjustment data set and set the value for the parameter other than the selected parameter as the search value.

[0087] The uncertainty calculation unit 103 in the fourth embodiment includes, for example, an uncertainty calculation dataset generation unit 1031c and a trained model prediction unit 1032c, as shown in FIG. 12 .

[0088] The uncertainty calculation dataset generation unit 1031 c generates an uncertainty calculation dataset based on the parameter adjustment dataset and the selection result by the parameter selection unit 101 .

[0089] In this case, the uncertainty calculation dataset generation unit 1031c generates an uncertainty calculation dataset by changing the parameters selected by the parameter selection unit 101 in the parameter adjustment dataset to values ​​that change at regular intervals, and assigning values ​​specified by the parameter selection unit 101 as common values ​​for each set to parameters other than the above parameters.

[0090] The trained model prediction unit 1032c predicts an uncertainty value by inputting the uncertainty calculation dataset generated by the uncertainty calculation dataset generation unit 1031c into the trained machine learning model. The uncertainty value predicted by this trained model prediction unit 1032c becomes the calculation result by the uncertainty calculation unit 103.

[0091] The uncertainty calculation unit 103 in the fourth embodiment can handle both the uncertainty of the problem and the uncertainty of learning. Furthermore, the uncertainty calculation unit 103 in the fourth embodiment can obtain more accurate uncertainty, particularly around the value of a specified parameter.

[0092] Next, an operation example of the parameter adjustment visualization device 1 according to the fourth embodiment shown in Fig. 2 will be described with reference to Fig. 13. In the operation example of the parameter adjustment visualization device 1 according to the fourth embodiment shown in Fig. 12, the operation example other than the operation example of the uncertainty calculation unit 103 is the same as the operation example of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. Therefore, only the operation of the uncertainty calculation unit 103 will be described below.

[0093] The parameter selection unit 101 in the fourth embodiment specifies values ​​for parameters other than the selected parameter.

[0094] In the example of operation of the parameter adjustment visualization device 1 according to embodiment 4 shown in FIG. 12, for example, as shown in FIG. 13, the uncertainty calculation dataset generation unit 1031c generates an uncertainty calculation dataset based on the parameter adjustment dataset and the selection result by the parameter selection unit 101 (step ST1031c).

[0095] In this case, the uncertainty calculation dataset generation unit 1031c generates an uncertainty calculation dataset by changing the parameters selected by the parameter selection unit 101 in the parameter adjustment dataset to values ​​that change at regular intervals, and assigning values ​​specified by the parameter selection unit 101 as common values ​​for each set to parameters other than the above parameters.

[0096] Next, the trained model prediction unit 1032c predicts the uncertainty value of the evaluation index value by inputting the uncertainty calculation dataset generated by the uncertainty calculation dataset generation unit 1031c to the trained machine learning model (step ST1032c). The uncertainty value predicted by this trained model prediction unit 1032c becomes the calculation result by the uncertainty calculation unit 103.

[0097] A case will be described in which the uncertainty calculation unit 103 in the fourth embodiment calculates uncertainty using a parameter adjustment data set such as that shown in the upper part of Fig. 14. It is assumed that the parameter selection unit 101 selects x0. It is also assumed that the parameter selection unit 101 specifies 4 as the value of x1, 3 as the value of x2, and 3 as the value of x3.

[0098] In this case, the uncertainty calculation dataset generation unit 1031c first generates an uncertainty calculation dataset by changing the parameters selected by the parameter selection unit 101 in the parameter adjustment dataset to values ​​that change at regular intervals, and assigning values ​​specified by the parameter selection unit 101 as common values ​​for each set to parameters other than the above parameters.

[0099] In the example of Figure 14, as shown in the middle row, each value of x0 in the uncertainty calculation dataset is increased by 0.01, all values ​​of x1 are set to 4, all values ​​of x2 are set to 3, and all values ​​of x3 are set to 3.

[0100] Next, the trained model prediction unit 1032c predicts uncertainty values ​​by inputting the uncertainty calculation dataset generated by the uncertainty calculation dataset generation unit 1031c into the trained machine learning model. The lower part of Figure 8 shows the prediction results (0.4, 0.38, 0.35, ...) of uncertainty values ​​when the uncertainty calculation dataset in the middle part is used.

[0101] As described above, according to the fourth embodiment, the trained machine learning model is a trained machine learning model that can output information indicating predicted values ​​of evaluation index values ​​and uncertainty values ​​of the predictions, and the parameter selection unit 101 specifies values ​​for parameters other than the selected parameters. The uncertainty calculation unit 103 generates an uncertainty calculation dataset by changing the parameters selected by the parameter selection unit 101 in the parameter adjustment dataset to values ​​that change at regular intervals and assigning the values ​​specified by the parameter selection unit 101 as common values ​​for each set to the parameters other than the selected parameters. The trained model prediction unit 1032c inputs the uncertainty calculation dataset generated by the uncertainty calculation dataset generation unit 1031c into the trained machine learning model and predicts the uncertainty value. This makes it possible for the parameter adjustment visualization device 1 according to the fourth embodiment to obtain uncertainty.

[0102] Fifth Embodiment Fig. 15 is a diagram showing an example of the configuration of a parameter adjustment visualization device 1 according to a fifth embodiment. In the parameter adjustment visualization device 1 according to the fifth embodiment shown in Fig. 15, a parameter importance calculation unit 105 is added to the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. The other example of the configuration of the parameter adjustment visualization device 1 according to the fifth embodiment shown in Fig. 15 is similar to the example of the configuration of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1, and the same reference numerals are used, and only the different parts will be described.

[0103] The parameter importance calculation unit 105 calculates the importance of a parameter based on the parameter adjustment data set. The importance of a parameter is a value indicating the degree to which the parameter contributes to improving the evaluation index value.

[0104] Any method can be used as the calculation method by the parameter importance calculation unit 105. For example, the parameter importance calculation unit 105 can calculate the importance of a parameter using Fanova, Permutation Importance, Shape, or Feature Importance using a gradient boosting tree.

[0105] Furthermore, the parameter selection unit 101 in the fifth embodiment selects parameters based on the importance of the parameters calculated by the parameter importance calculation unit 105. That is, the parameter selection unit 101 selects parameters with high importance from among the parameters included in the parameter adjustment data set.

[0106] Next, an example of operation of the parameter adjustment visualization device 1 according to the fifth embodiment shown in Fig. 5 will be described with reference to Fig. 16. In the example of operation of the parameter adjustment visualization device 1 according to the fifth embodiment shown in Fig. 15, the example of operation other than the example of operation of the parameter importance calculation unit 105 and the parameter selection unit 101 is the same as the example of operation of the parameter adjustment visualization device 1 according to the first embodiment shown in Fig. 1. Therefore, only the operation of the parameter importance calculation unit 105 and the parameter selection unit 101 will be described below.

[0107] In the example of operation of the parameter adjustment visualization device 1 according to embodiment 5 shown in FIG. 15, for example, as shown in FIG. 16, the parameter importance calculation unit 105 calculates the importance of the parameter based on the parameter adjustment dataset (step ST501).

[0108] Next, the parameter selection unit 101 selects parameters based on the parameter importance calculated by the parameter importance calculation unit 105. (Step ST502) That is, the parameter selection unit 101 selects parameters with high importance from among the parameters included in the parameter adjustment data set.

[0109] The above describes a case where the parameter importance calculation unit 105 is added to the parameter adjustment visualization device 1 according to embodiment 1. However, the present invention is not limited to this, and the parameter importance calculation unit 105 may be added to the parameter adjustment visualization device 1 according to embodiments 2 to 4, and the same effects as those described above can be obtained.

[0110] As described above, according to the fifth embodiment, the parameter adjustment visualization device 1 includes the parameter importance calculation unit 105 that calculates the importance of a parameter based on the parameter adjustment dataset, and the parameter selection unit 101 selects a parameter based on the importance of the parameter calculated by the parameter importance calculation unit 105. This enables the parameter adjustment visualization device 1 according to the fifth embodiment to select parameters more appropriately.

[0111] Sixth Embodiment In a sixth embodiment, a configuration example of a parameter optimization device including the parameter adjustment visualization device 1 according to any one of the first to fifth embodiments is shown. Although the following describes a parameter optimization device including the parameter adjustment visualization device 1 according to the first embodiment, the same applies to parameter optimization devices including the parameter adjustment visualization device 1 according to any one of the second to fifth embodiments.

[0112] As shown in FIG. 17, the parameter optimization device includes a parameter determination unit 2, a parameter evaluation unit 3, a machine learning unit 4, and a parameter adjustment visualization device 1.

[0113] The parameter determination unit 2 determines the next parameter value to be searched for based on the partial dependency value and the uncertainty value calculated by the parameter adjustment visualization device 1 (the partial dependency calculation unit 102 and the uncertainty calculation unit 103). At this time, for example, the parameter determination unit 2 determines the next parameter value to be searched for as the parameter value with the highest upper limit value based on the upper limit value of the uncertainty value. Note that, when performing the first search, the parameter determination unit 2 sets the parameter value to the initial value.

[0114] The parameter evaluation unit 3 obtains an evaluation index value using the parameter values ​​determined by the parameter determination unit 2. As shown in Fig. 17 , the parameter evaluation unit 3 includes an actual operation unit 301, an evaluation index calculation unit 302, and a parameter adjustment data set storage unit 303.

[0115] The actual operation unit 301 operates the optimization target using the parameter values ​​determined by the parameter determination unit 2. The evaluation index calculation unit 302 calculates evaluation index values ​​based on the operation results of the optimization target by the actual operation unit 301. The parameter adjustment dataset storage unit 303 stores data indicating pairs of parameter values ​​(used by the actual operation unit 301) determined by the parameter determination unit 2 and evaluation index values ​​(calculated by the evaluation index calculation unit 302) obtained from the parameter values.

[0116] 17 shows a case where the parameter adjustment dataset storage unit 303 is provided inside the parameter optimization device, but is not limited to this, and the parameter adjustment dataset storage unit 303 may be provided outside the parameter optimization device.

[0117] The machine learning unit 4 learns the relationship between the parameter values ​​and the evaluation index values ​​based on pairs of the parameter values ​​used in the parameter evaluation unit 3 and the evaluation index values ​​obtained from the parameter values. In this case, the machine learning unit 4 performs learning by extracting data indicating pairs of parameter values ​​and evaluation index values ​​from the parameter adjustment dataset storage unit 303.

[0118] The parameter adjustment visualization device 1 then uses the trained machine learning model that is the result of learning by the machine learning unit 4.

[0119] Next, an example of the operation of the parameter optimization device according to the sixth embodiment shown in Fig. 17 will be described with reference to Fig. 18. In the example of the operation of the parameter optimization device according to the sixth embodiment shown in Fig. 17, as shown in Fig. 18, for example, first, the parameter determination unit 2 sets the parameter values ​​to initial values ​​(step ST601).

[0120] Next, the parameter evaluation unit 3 obtains an evaluation index value using the parameter values ​​determined by the parameter determination unit 2 (step ST602).

[0121] That is, first, the actual operation unit 301 operates the optimization target using the parameter values ​​determined by the parameter determination unit 2. Then, the evaluation index calculation unit 302 calculates an evaluation index value based on the operation result of the optimization target by the actual operation unit 301.

[0122] Next, the parameter adjustment dataset storage unit 303 stores data indicating a pair of the parameter value determined by the parameter determination unit 2 (used in the actual operation unit 301) and the evaluation index value obtained from the parameter value (calculated by the evaluation index calculation unit 302) (step ST603).

[0123] Next, the machine learning unit 4 learns the relationship between the parameter value and the evaluation index value based on the pair of the parameter value used in the parameter evaluation unit 3 and the evaluation index value obtained from the parameter value (step ST604). At this time, the machine learning unit 4 performs learning by extracting data indicating the pair of the parameter value and the evaluation index value from the parameter adjustment dataset storage unit 303.

[0124] Next, the parameter selection unit 101 of the parameter adjustment visualization device 1 selects a parameter (step ST101). Furthermore, if there are multiple trained machine learning models that are the learning results of the machine learning unit 4, the parameter selection unit 101 also selects one of the trained machine learning models.

[0125] Next, the partial dependency calculation unit 102 of the parameter adjustment visualization device 1 calculates the partial dependency value based on the trained machine learning model that is the training result by the machine learning unit 4, the parameter adjustment dataset, and the selection result by the parameter selection unit 101 (step ST102). Note that, when a trained machine learning model is selected by the parameter selection unit 101, the partial dependency calculation unit 102 calculates the partial dependency value using the selected trained machine learning model.

[0126] Furthermore, the uncertainty calculation unit 103 of the parameter adjustment visualization device 1 calculates an uncertainty value based on the trained machine learning model, which is the learning result by the machine learning unit 4, the parameter adjustment dataset, and the selection result by the parameter selection unit 101 (step ST103). Note that, when a trained machine learning model is selected by the parameter selection unit 101, the uncertainty calculation unit 103 calculates the uncertainty value using the selected trained machine learning model.

[0127] Next, the partial dependency plot output unit 104 of the parameter adjustment visualization device 1 outputs information indicating the partial dependency value calculated by the partial dependency calculation unit 102 and the uncertainty value calculated by the uncertainty calculation unit 103 (step ST104).

[0128] Next, the parameter optimization device determines whether to continue the search (step ST605). If the parameter optimization device determines in step ST605 that the search should be ended, the sequence ends.

[0129] On the other hand, if the parameter optimization device determines in step ST605 to continue the search, the parameter determination unit 2 determines the next parameter value to be searched for based on the partial dependency value and the uncertainty value calculated by the parameter adjustment visualization device 1 (the partial dependency calculation unit 102 and the uncertainty calculation unit 103) (steps ST606 and ST607). At this time, for example, the parameter determination unit 2 determines the next parameter value to be searched for as the parameter value with the highest upper limit value based on the upper limit value of the uncertainty value. After that, the sequence returns to step ST602.

[0130] As described above, according to the sixth embodiment, the parameter optimization device includes the parameter adjustment visualization device 1, the parameter determination unit 2 that determines the value of a parameter to be searched next based on the partial dependent value and the uncertainty value calculated by the parameter adjustment visualization device 1, the parameter evaluation unit 3 that has the actual operation unit 301 that operates the optimization target using the parameter values ​​determined by the parameter determination unit 2 and the evaluation index calculation unit 302 that calculates an evaluation index value based on the operation result of the optimization target by the actual operation unit 301, and the machine learning unit 4 that learns the relationship between the parameter values ​​and the evaluation index value based on a pair of the parameter value used by the parameter evaluation unit 3 and the evaluation index value obtained from the parameter value, and the parameter adjustment visualization device 1 uses a trained machine learning model that is the learning result by the machine learning unit 4. This enables the parameter optimization device according to the sixth embodiment to more appropriately optimize parameters by using the uncertainty in the evaluation index value.

[0131] Seventh Embodiment In the seventh embodiment, a configuration example will be described in which the parameter optimization device according to the sixth embodiment is applied to a positioning control device application. In the parameter optimization device according to the seventh embodiment shown in FIG. 19 , the parameter determination unit 2 is changed to a position command parameter determination unit 2b, the actual operation unit 301 and the parameter adjustment data set storage unit 303 in the parameter evaluation unit 3 are changed to a positioning control device operation unit 301b and a position command parameter adjustment data set storage unit 303b, and the parameter selection unit 101 in the parameter adjustment visualization device 1 is changed to a position command parameter selection unit 101b. Furthermore, FIG. 20 shows an operation example of the parameter optimization device according to the seventh embodiment shown in FIG. 19 . That is, the parameter optimization device according to the seventh embodiment uses parameters related to positioning as parameters.

[0132] Finally, with reference to FIG. 21 , an example of a hardware configuration of a parameter adjustment visualization device 1 according to any one of embodiments 1 to 7 will be described. Note that, while an example of a hardware configuration of the parameter adjustment visualization device 1 according to embodiment 1 will be described below, the same applies to example hardware configurations of the parameter adjustment visualization device 1 according to any one of embodiments 2 to 7. The functions of the parameter selection unit 101, partial dependency calculation unit 102, uncertainty calculation unit 103, and uncertainty-added partial dependency plot output unit 104 in the parameter adjustment visualization device 1 are realized by a processing circuit 51. The processing circuit 51 may be dedicated hardware as shown in FIG. 21A , or may be a CPU (also referred to as a central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 that executes a program stored in a memory 53 as shown in FIG. 21B .

[0133] When the processing circuitry 51 is dedicated hardware, the processing circuitry 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each of the parameter selection unit 101, the partial dependency calculation unit 102, the uncertainty calculation unit 103, and the partial dependency plot output unit with uncertainty 104 may be realized individually by the processing circuitry 51, or the functions of each unit may be realized collectively by the processing circuitry 51.

[0134] When the processing circuitry 51 is a CPU 52, the functions of the parameter selection unit 101, the partial dependency calculation unit 102, the uncertainty calculation unit 103, and the partial dependency plot output unit with uncertainty 104 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 53. The processing circuitry 51 realizes the functions of each unit by reading and executing the programs stored in the memory 53. That is, the parameter visualization device includes the memory 53 for storing programs that, when executed by the processing circuitry 51, result in the execution of, for example, each step shown in FIG. 4 . Furthermore, these programs can also be said to cause a computer to execute the procedures and methods of the parameter selection unit 101, the partial dependency calculation unit 102, the uncertainty calculation unit 103, and the partial dependency plot output unit with uncertainty 104. Here, examples of the memory 53 include non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs (Digital Versatile Discs).

[0135] Note that the functions of the parameter selection unit 101, the partial dependency calculation unit 102, the uncertainty calculation unit 103, and the partial dependency plot output unit with uncertainty 104 may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the function of the parameter selection unit 101 may be implemented by the processing circuitry 51 as dedicated hardware, and the functions of the partial dependency calculation unit 102, the uncertainty calculation unit 103, and the partial dependency plot output unit with uncertainty 104 may be implemented by the processing circuitry 51 reading and executing programs stored in the memory 53.

[0136] In this way, the processing circuitry 51 can realize each of the above-described functions by hardware, software, firmware, or a combination of these.

[0137] It should be noted that the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted.

[0138] The parameter adjustment visualization device according to the present disclosure is capable of visualizing the uncertainty of evaluation index values ​​and is suitable for use as a parameter adjustment visualization device, etc.

[0139] 1 Parameter adjustment visualization device, 2 Parameter determination unit, 2b Position command parameter determination unit, 3 Parameter evaluation unit, 4 Machine learning unit, 51 Processing circuit, 52 CPU, 53 Memory, 101 Parameter selection unit, 101b Position command parameter selection unit, 102 Partial dependency calculation unit, 103 Uncertainty calculation unit, 104 Partial dependency plot output unit with uncertainty, 105 Parameter importance calculation unit, 301 Actual operation unit, 301b Positioning control device operation unit, 302 Evaluation index calculation unit, 303 Parameter adjustment dataset storage unit, 303b Position command parameter adjustment dataset storage unit, 1031a, 1031b, 1031c Uncertainty calculation dataset generation unit, 1032a, 1032b, 1032c Trained model prediction unit, 1033a, 1033b Partial dependency uncertainty calculation unit.

Claims

1. A parameter selection section for selecting parameters, A trained machine learning model capable of outputting information indicating predicted values ​​of evaluation index values, and a partial dependency calculation unit that calculates partial dependency values ​​of evaluation index values ​​for parameters selected by the parameter selection unit based on a parameter adjustment dataset containing multiple pairs of values ​​of multiple parameters and evaluation index values. An uncertainty calculation unit calculates uncertainty values ​​for evaluation index values ​​related to the parameters selected by the parameter selection unit based on the trained machine learning model and the parameter tuning dataset. A partial dependency plot output unit with uncertainty outputs information showing the partial dependency value calculated by the partial dependency calculation unit and the uncertainty value calculated by the uncertainty calculation unit. A parameter adjustment visualization device equipped with the following features.

2. The aforementioned trained machine learning model is a trained machine learning model capable of outputting information indicating the predicted value of the evaluation metric and the uncertainty value of the prediction. The uncertainty calculation unit said above, An uncertainty calculation dataset generation unit generates multiple uncertainty calculation datasets by duplicating the parameter adjustment dataset a number of times greater than or equal to the number of sets, and by separating the values ​​including the values ​​for each set of parameters selected by the parameter selection unit from the parameter adjustment dataset and assigning them as common values ​​to each set of parameters in the replicated dataset, A trained model prediction unit takes the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit as input to the trained machine learning model and predicts the uncertainty values ​​accordingly. It includes a partially dependent uncertainty calculation unit that calculates the average of the uncertainty values ​​predicted by the trained model prediction unit. The parameter adjustment visualization device according to claim 1, characterized in that it is a parameter adjustment visualization device.

3. The uncertainty calculation unit said above, An uncertainty calculation dataset generation unit generates multiple uncertainty calculation datasets by duplicating the parameter adjustment dataset a number of times greater than or equal to the number of sets, and by separating the values ​​including the values ​​for each set of parameters selected by the parameter selection unit from the parameter adjustment dataset and assigning them as common values ​​to each set of parameters in the replicated dataset, A trained model prediction unit inputs the uncertainty calculation datasets generated by the uncertainty calculation dataset generation unit to the trained machine learning model and predicts the evaluation index values ​​accordingly. The system includes a partially dependent uncertainty calculation unit that calculates the variance or standard deviation of the evaluation index values ​​predicted by the trained model prediction unit as uncertainty values. The parameter adjustment visualization device according to claim 1, characterized in that it is a parameter adjustment visualization device.

4. The aforementioned trained machine learning model is a trained machine learning model capable of outputting information indicating the predicted value of the evaluation metric and the uncertainty value of the prediction. The parameter selection unit specifies values ​​for parameters other than the selected parameter, The uncertainty calculation unit said above, An uncertainty calculation dataset generation unit generates an uncertainty calculation dataset by changing the parameters selected by the parameter selection unit from the parameter adjustment dataset to values ​​that change at regular intervals, and assigning the values ​​specified by the parameter selection unit as common values ​​for each set to the parameters other than the selected parameters, The trained machine learning model includes a trained model prediction unit that inputs the uncertainty calculation dataset generated by the uncertainty calculation dataset generation unit to predict uncertainty values. The parameter adjustment visualization device according to claim 1, characterized in that it is a parameter adjustment visualization device.

5. The system includes a parameter importance calculation unit that calculates the importance of parameters based on the parameter adjustment dataset. The parameter selection unit selects parameters based on the importance of the parameters calculated by the parameter importance calculation unit. A parameter adjustment visualization device according to any one of claims 1 to 4.

6. The aforementioned trained machine learning model is a Gaussian process regression model. A parameter adjustment visualization device according to any one of claims 1 to 4.

7. The parameter tuning dataset is a parameter tuning dataset that includes multiple pairs of values ​​for multiple parameters obtained by Bayesian optimization and evaluation index values. A parameter adjustment visualization device according to any one of claims 1 to 4.

8. A parameter adjustment visualization device according to any one of claims 1 to 4, A parameter determination unit determines the value of the next parameter to be explored based on the partial dependency value and uncertainty value calculated by the parameter adjustment visualization device, A parameter evaluation unit having an actual operation unit that operates the optimization target using the parameter values ​​determined by the parameter determination unit, and an evaluation index calculation unit that calculates evaluation index values ​​based on the operation results of the optimization target by the actual operation unit, The system includes a machine learning unit that learns the relationship between parameter values ​​and evaluation index values ​​based on the pair of parameter values ​​used in the parameter evaluation unit and evaluation index values ​​obtained from those parameter values, The parameter adjustment visualization device uses a trained machine learning model which is the result of training by the machine learning unit. A parameter optimization device characterized by the following features.

9. A parameter adjustment visualization program for causing a computer to function as a parameter adjustment visualization device according to any one of claims 1 to 4.

10. The parameter selection unit includes the step of selecting a parameter, The partial dependency calculation unit calculates the partial dependency of the evaluation index value for the parameter selected by the parameter selection unit, based on a trained machine learning model capable of outputting information indicating the predicted value of the evaluation index value, and a parameter adjustment dataset containing multiple pairs of values ​​of multiple parameters and evaluation index values. The uncertainty calculation unit calculates uncertainty values ​​for evaluation index values ​​related to the parameters selected by the parameter selection unit, based on the trained machine learning model and the parameter tuning dataset. The uncertainty-included partial dependency plot output unit outputs information indicating the partial dependency value calculated by the partial dependency degree calculation unit and the uncertainty value calculated by the uncertainty calculation unit. A parameter adjustment visualization method having the following characteristics.