Data set creation assistance system, data set creation assistance device, and data set creation assistance program

Abstract

The purpose of the present invention is to facilitate, in the selection of an explanatory variable data group for creating a data set for learning or the like, quantitatively evaluating the property of the explanatory variable data group and selecting an explanatory variable data group for creating the data set.　Therefore, a data set creation assistance system according to the present invention executes: explanatory variable input processing for acquiring the type of an explanatory variable and the value of the explanatory variable; experiment plan candidate generation processing for generating an experiment plan candidate composed of a combination of experimental conditions corresponding to an arbitrary number of experiments from the type of the explanatory variable and the value of the explanatory variable; experiment plan evaluation value calculation processing for calculating the evaluation value of the experiment plan candidate; and experiment plan evaluation value output processing for outputting the relationship between the number of experiments obtained from the plurality of experiment plan candidates and the evaluation value.

Description

Dataset creation support system, dataset creation support device, and dataset creation support program

[0001] The present invention relates to a data set creation support system, a data set creation support device, and a data set creation support program.

[0002] Many solutions are used to solve optimization problems that require finding values ​​of explanatory variables such that the corresponding objective variable is the best value or a value that satisfies a target. Also, a method is used in which a machine learning model or a statistical model is trained based on a data set consisting of a combination of explanatory variables and objective variables to find an optimal solution for the objective variable. Patent Document 1 (JP-A-2005-102666) relates to a method for determining candidate design parameters to be input to an experimental device, and discloses that the input design parameters are finally determined as candidate design parameters by correcting them using a backpropagation algorithm based on the result of adding the distance cost between the input design parameters and the data of the candidate points and the output cost of an inference model for the input design parameters.

[0003] Japanese Patent Application Laid-Open No. 2023-17358

[0004] Building machine learning models and statistical modeling models requires the creation of training datasets. In selecting explanatory variable data sets for such purposes, a means is desired for obtaining desired values ​​of the objective variable with limited experimental costs (time, expense, environmental impact, etc.). For this reason, while experimental design and other methods may be used to efficiently select explanatory variable data sets, the quality of the selected explanatory variable data sets has not been quantitatively evaluated. Patent Document 1 also fails to disclose any recognition of issues related to quantitatively evaluating and selecting input design parameter candidates. An object of the present invention is to quantitatively evaluate the properties of explanatory variable data sets when selecting explanatory variable data sets for creating datasets, such as for training, and to facilitate the selection of explanatory variable data sets for creating datasets.

[0005] In order to solve the above-mentioned problems, one representative dataset creation support system of the present invention executes an explanatory variable input process that acquires the types of explanatory variables and the values ​​of the explanatory variables, an experimental design candidate generation process that generates experimental design candidates consisting of combinations of experimental conditions corresponding to any number of experiments from the types of explanatory variables and the values ​​of the explanatory variables, an experimental design evaluation value calculation process that calculates the evaluation values ​​of the experimental design candidates, and an experimental design evaluation value output process that outputs the relationship between the number of experiments and the evaluation values ​​obtained from multiple experimental design candidates.

[0006] According to the present invention, when selecting a group of explanatory variable data for creating a dataset for learning or the like, the properties of the group of explanatory variable data can be quantitatively evaluated, facilitating the selection of the group of explanatory variable data for creating the dataset. As a specific example, it is possible to quantitatively evaluate the properties of the group of explanatory variable data in an experimental design formulated for a certain number of experiments for one or more types of explanatory variables and their values, and display the evaluation results. Furthermore, in cases where the number of experiments that can be performed is limited due to experimental costs (time, expenses, environmental impact, etc.), it is possible to easily construct a machine learning model or a statistical method model that can estimate the values ​​of explanatory variables that will obtain a desired value of the objective variable. Other issues, configurations, and effects will be made clearer through the description of the following examples.

[0007] FIG. 1 is a functional block diagram of a dataset creation support system according to a first embodiment. FIG. 2 is a flowchart showing the processing flow of the dataset creation support system according to the first embodiment. FIG. 3 shows an example of the types of explanatory variables and the values ​​of the explanatory variables input in step S100. FIG. 4 is an example of an experimental design candidate. FIG. 5 shows a definition of the D-optimality criterion. FIG. 6 is a graph showing an example of an experimental design evaluation value output. FIG. 7 is an example of a dataset acquired in step S102. FIG. 8 is a functional block diagram of a dataset creation support system according to a second embodiment. FIG. 9 is a flowchart showing the processing flow of the dataset creation support system according to the second embodiment.

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention should not be construed as being limited to the description of the embodiments shown below. Those skilled in the art will readily understand that the specific configuration can be modified within the scope of the idea or intent of the present invention. Furthermore, in the configuration of the invention described below, the same or similar configurations or functions are designated by the same reference numerals, and redundant explanations will be omitted.

[0009] First Embodiment First, the first embodiment will be described. FIG. 1 is a functional block diagram of a dataset creation support system according to the first embodiment. The dataset creation support system 1 includes an explanatory variable / objective variable acquisition unit 11, an experimental design calculation unit 12, an experimental design output unit 13, and a storage unit 14. The storage unit 14 also includes a learning data storage unit 141 and an acquired data storage unit 142. In terms of hardware configuration, the explanatory variable / objective variable acquisition unit 11 may be connected to input devices such as a keyboard or mouse as an input interface. The experimental design calculation unit 12 executes program processing using a processor such as a CPU. The experimental design output unit 13 may be connected to a display, printer, or the like as an output interface. The storage unit 14 may be a random access semiconductor memory, a storage device, or a storage medium (either volatile or non-volatile) for storing data or programs (applications). These functional units are connected via wire or wireless connection.

[0010] 2 is a flowchart showing the processing flow of the dataset creation support system according to the first embodiment. By executing step S100, in which information about explanatory variables and target variables is input, step S101, in which an experimental plan for constructing an initial training dataset is formulated, and step S102, in which an initial training dataset is acquired, an initial training dataset is constructed, which is used to generate a prediction model that represents a mathematical model for which a solution is to be searched. Each step will be described in detail below, along with the operation of each functional unit.

[0011] (S100) In step S100, the explanatory variable / objective variable acquisition unit 11 performs an input process (explanatory variable / objective variable input process) to acquire information about the explanatory variables and the objective variables. Examples of the information about the explanatory variables (explanatory variable data) input in step S100 include the type of explanatory variable, the value of the explanatory variable, the range of possible values ​​of the explanatory variable to be searched for a solution, and constraints on the interrelationships between the explanatory variables. Furthermore, constraints on the explanatory variables based on the specifications of the hardware or software device may be input. Furthermore, when searching for a solution using an existing dataset, the type of explanatory variable, the value of the explanatory variable, and the range of possible values ​​of the explanatory variable for the data present in the existing dataset may be input. FIG. 3 shows examples of the types of explanatory variables and values ​​of the explanatory variables input in step S100. Here, the types of explanatory variables correspond to explanatory variables A, B, C, etc. in FIG. 3, and the values ​​of the explanatory variables correspond to their respective levels A1, A2, B1, B2, C1, C2, etc. While specific details depend on the experimental content, explanatory variables include the flow rate of each gas species during plasma etching, the applied power for plasma generation, and the surface temperature of the sample to be etched. For example, if the explanatory variable is temperature, the level can be set to 10°C or 20°C. The number of levels and the value of each level can vary depending on the type of explanatory variable. Constraints on the interrelationships between explanatory variables can vary widely, including theoretical constraints, empirical constraints, and constraints based on equipment specifications. Given conditions such as (A1, A2, ... < B1, B2, A1 + B1 < 100, A × B < C...), an experimental plan must be developed to satisfy these constraints. Hereinafter, the combination of the type of explanatory variable and its value (explanatory variable data) for which a solution is sought is referred to as the experimental condition, and the combination of the type of objective variable and its value (objective variable data) for which a solution is sought is referred to as the experimental result. To create a training dataset, experimental conditions for a demonstration experiment must first be established. Furthermore, in order to formulate experimental conditions, the user must determine the types of explanatory variables and the values ​​of the explanatory variables, which are elements of the experimental conditions.However, the values ​​of the explanatory variables may be determined mechanically by inputting the range of values ​​that the explanatory variables can take.

[0012] In step S100, information about the objective variables can also be input. However, the information about the objective variables input here is used in the steps of the second embodiment described later. Examples of information about the objective variables include the type of objective variable and its target value. While specific details depend on the experiment, examples of the objective variable include the etching amount per unit time, the variation in the etching amount on the surface of the sample to be etched, and an index related to the shape of the sample after etching. The target value can be specified as a numerical range or a specific numerical value. The target value can also be used to estimate the experimental conditions using a prediction model described later. When there are multiple types of objective variables, weights can be input for each type of objective variable to weight the numerical processing based on the importance of each type of objective variable. Furthermore, when a global solution or optimal solution for a solution search target is desired, a solution that exceeds the existing best solution can be searched for by inputting the best solution in an existing data set from the acquired data storage unit 142.

[0013] (S101) In step S101, the experimental design calculation unit 12 and the experimental design output unit 13 perform an experimental design formulation process to formulate an experimental design for constructing an initial training dataset from the information on the explanatory variables input in step S100. The experimental design formulation process includes an experimental design candidate generation process to generate combinations of experimental conditions that become experimental design candidates based on the information on the explanatory variables input in step S100, an experimental design evaluation value calculation process to calculate evaluation values ​​for the experimental design candidates, an experimental design evaluation value output process to output the relationship between the number of experiments and the evaluation values ​​for the experimental design candidates, and finally an experimental design determination output process to determine and output an experimental design for constructing an initial training dataset. Methods for determining an experimental design include a knowledge-based determination method based on academic knowledge or knowledge obtained from experience regarding the mathematical model to be analyzed, a determination method based on experimental design, a random determination method, or a combination of these methods. Examples of experimental design include fractional factorial designs, Plackett-Burman designs, and experimental designs using D-optimality criteria. Each experimental design is used to investigate the relationship between experimental conditions and experimental results with fewer experiments than a full factorial design. Here, in an experimental design using the D-optimality criterion as an evaluation value, when a mathematical model between the experimental conditions and the experimental results is established, the optimality criterion is used, which is a mathematical index for conducting efficient experiments, taking into account the mathematical model. In an experimental design using the optimality criterion, there are no restrictions on the type of explanatory variables, the values ​​of the explanatory variables, or the number of experiments selected, and experimental conditions can be determined taking into account the interactions between the explanatory variables. In addition to the D-optimality criterion, other optimality criteria used in experimental designs include the E-optimality criterion and the A-optimality criterion. This embodiment relates to a dataset creation support system using an experimental design that uses an optimality criterion based on experimental design as an evaluation value. In particular, the following will be described in detail using the D-optimality criterion as an evaluation value.

[0014] <Experimental Design Candidate Generation Process> First, the experimental design calculation unit 12 generates an experimental design candidate from the types and values ​​of explanatory variables input in step S100 (experimental design candidate generation process). FIG. 4 shows an example of an experimental design candidate. Using the types and values ​​of explanatory variables input in step S100, an experimental design candidate is generated as a group of n experimental conditions (explanatory variable data group) consisting of combinations of explanatory variable types and their respective explanatory variable values ​​for the number of experiments (n), such as experimental condition 1 (A2, B1, C2, ...), experimental condition 2 (A4, B3, C2, ...), and experimental condition n (A3, B2, C4, ...). At this time, if there are constraints between the explanatory variables input in step S100, it is possible to select a combination that satisfies those conditions. The maximum value N of the number of experiments (n) ＭＡＸ If constraints are not taken into account, is equal to the product of the number of explanatory variable values ​​(levels) multiplied by each type of explanatory variable. Also, as mentioned above, in an experimental design using optimal criteria, there is no limit to the number of experiments, so it can be calculated from 1 to N. ＭＡＸ It is possible to generate candidate experimental designs for any number of experiments up to

[0015] <Experimental Design Evaluation Value Calculation Process> Next, the experimental design calculation unit 12 calculates an evaluation value for the experimental design candidate (experimental design evaluation value calculation process). FIG. 5 shows the definition of the D-optimal criterion. When an experimental result (objective variable) Y is obtained for an experimental condition (explanatory variable) X, consider the case where the value of Y is maximized by optimizing the experimental conditions. For simplicity, consider the case where the numerical model Y = f(X) is a linear model. The experimental result Y is an n x l matrix where n is the number of experiments and l is the number of types of objective variables, and the experimental condition X is an n x m matrix where n is the number of experiments and m is the number of types of explanatory variables. Using the regression coefficient A of the m x l matrix, Y = XA. Here, each row of the experimental condition X corresponds to each experimental condition 1 to n of the experimental design candidate (see FIG. 4). In this case, the D-optimal criterion is expressed as X Ｔ It is expressed as the value of the determinant of X. Therefore, in an experimental design using the D-optimal criterion (D-optimal design), the experimental condition X that maximizes the D-optimal criterion is found from among the candidate experimental designs, and the combination of each experimental condition there is formulated as the experimental design. In addition, the D-optimal design uses (X Ｔ X)－１ In some cases, an experimental design is formulated to define a determinant of the matrix and minimize that value, but these are synonymous. When a mathematical model between the experimental conditions and the experimental results is established, an experimental design in which the D-optimality criterion takes the best value (also called the best evaluation value) can efficiently train the mathematical model on the relationship between the experimental conditions and the experimental results with fewer experiments. Furthermore, with regard to the evaluation value of the experimental design candidate in formulating the experimental design, the D-optimality criterion is just one example, and known optimization criteria used in experimental design methods, such as the A-optimality criterion or the E-optimality criterion, may also be used.

[0016] <Experimental Design Evaluation Value Output Processing> Next, the experimental design output unit 13 outputs the relationship between the number of experiments and the evaluation value for each experimental design candidate based on the data on the number of experiments and the evaluation value (D-optimal criterion) for each experimental design candidate (experimental design evaluation value output processing). The output format is not particularly limited as long as the relationship between the number of experiments and the evaluation value can be grasped, and may be in the form of a graph, diagram, table, or text. FIG. 6 is a graph showing an example of an experimental design evaluation value output. The horizontal axis is the number of experiments and the vertical axis is the D-optimal criterion. The relationship between the number of experiments and the maximum value of the D-optimal criterion (best evaluation value), which is the experimental design evaluation value at that time, is shown in graphical form. As mentioned above, since it is possible to formulate an experimental design, i.e., to calculate the experimental design evaluation value, for any number of experiments, the relationship between the number of experiments and the experimental design evaluation value can be shown discretely or continuously. Furthermore, the number of experiments can be increased or decreased up to a maximum value N depending on the type and number of values ​​of the explanatory variables input in step S100. ＭＡＸ Furthermore, in the example shown in FIG. 6, X Ｔ Since the value of the determinant of X is used, the number of experiments N ＭＡＸ When , the value of the D optimality criterion is the maximum value S ＭＡＸ (also called the highest best evaluation value). In this case, the best experimental design is equivalent to a full factorial design. Similarly, if the number of types and values ​​of explanatory variables input in step S100 are given, N ＭＡＸ It is possible to calculate the maximum value (best evaluation value) of the D-optimality criterion for any number of experiments that is less than S. ＭＡＸ The value that is 50% of ５０％, 10% of the value is S １０％ , 1% of the value S １％ The maximum value of the D optimum criterion is defined as S ５０％ , S １０％ , S １％ The number of experiments that satisfy the following formula is N. ５０ , N １０ , N １ However, the output format of the D-optimal criterion is not particularly limited, and it may be output as an absolute value.

[0017] <Experimental Design Decision and Output Processing> Next, the experimental design output unit 13 decides and outputs an experimental design for constructing an initial learning dataset (experimental design decision and output processing). The decided experimental design may be presented in a table format as shown in FIG. 4 so that the user can easily check it. An example of the procedure for deciding an experimental design will be described. For example, if the upper limit of the number of experiments is N due to constraints on experiment costs, ５０ , by referring to the graph of FIG. 6, the best value Sx of the D optimality criterion is S ５０％ If the difference is tolerable compared to N, then the initial training data set should be smaller than Nx (<N ５０ ) can be determined to be acceptable for creating a training dataset, thereby increasing the options (degrees of freedom) for the number of experiments and enabling efficient creation of training datasets. In other words, by referring to the output of the relationship between the number of experiments and the experimental design evaluation value, the user can determine the amount of effort and efficiency required to investigate the relationship between the experimental conditions and the experimental results, making it easier to create a good training dataset. In this way, even when the number of experiments that can be performed is limited due to factors such as experimental costs (time, expenses, environmental impact, etc.), it is easy to create a good training dataset that corresponds to the number of experiments selected by the user. Needless to say, the output of the graph in Figure 6 and the determination of the combination of the number of experiments and experimental conditions using it can be automated by establishing predetermined rules.

[0018] When determining the experimental design, the constraints entered in step S100 can also be taken into account. An example of a constraint condition is that when multiple experimental design candidates that provide the best evaluation values ​​for different numbers of experiments are obtained, the experimental design candidate that includes common experimental conditions is prioritized. That is, if an experimental design candidate that provides the best evaluation value is obtained for the number of experiments Ni, Nj, and Nk, and common experimental conditions exist between the experimental design candidates for Ni and Nj but not for the number of experiments Nk, the experimental design candidate for Ni or Nj can be prioritized and determined as the experimental design. In this way, even if the initial number of experiments varies, by switching between Ni and Nj, experimental results or experimental preparations can be reused for the common experimental conditions, thereby saving labor and enabling efficient data set creation.

[0019] Another example of the constraint may be that, when one or more types or values ​​of proven explanatory variables are input from the acquired data storage unit 142 in step S100, experimental conditions related to a predetermined number of proven explanatory variables among the one or more proven explanatory variables are included. In this case, an experimental design including proven experimental conditions can be determined. This makes it possible to refer to proven experimental results and reduce the number of experiments required to create a dataset in step S102 (described below). On the other hand, instead of the constraint, a constraint may be set such that none of the one or more proven explanatory variables include experimental conditions related to the proven explanatory variables. In this case, an experimental design not including proven experimental conditions can be determined, and only experimental results for unknown experimental conditions can be obtained.

[0020] (S102) In step S102, an initial training dataset is stored in the training data storage unit 141. Therefore, an experiment is conducted using hardware or software based on the experimental conditions determined in step S101, and the experimental results are acquired and stored. Alternatively, experimental results corresponding to the experimental conditions determined in step S101 may be extracted from existing datasets in the acquired data storage unit 142 and stored. In this manner, an initial training dataset is acquired. In the present disclosure, an experiment refers to an experiment using hardware or software and the extraction of known experimental results from the acquired data storage unit 142. Note that even if the experimental results conducted based on the experimental conditions determined in step S101 differ from the target value of the objective variable, they are generally stored as an initial training dataset. FIG. 7 shows an example of a dataset acquired in step S102. Each dataset is a combination of corresponding experimental conditions and experimental results, and the experimental conditions to be implemented are equivalent to the experimental plan formulated in step S101. Furthermore, each experimental result is a combination of the corresponding objective variable type and objective variable value. The types of response variables correspond to response variables X, Y, Z, etc. in FIG. 7, and the values ​​of the response variables correspond to the respective measurement or observation results X1, X2, Y1, Y2, Z1, Z2, etc.

[0021] [Second Embodiment] Next, a second embodiment will be described. In the second embodiment, an initial prediction model is generated using the initial training dataset created in the first embodiment and is sequentially updated, and additional training datasets are added and updated during this process. The following will focus on the characteristics of the second embodiment. FIG. 8 is a functional block diagram of a dataset creation support system according to the second embodiment. In addition to the functional units in the first embodiment, the dataset creation support system 2 of this embodiment includes a prediction model generation unit 15, an experiment condition estimation unit 16, and a demonstration experiment result determination unit 17, and these additional functional units execute program processing using a processor such as a CPU.

[0022] The operation of the added functional units will be described together with the flow of the dataset creation support method according to this embodiment. FIG. 9 is a flowchart showing the processing flow of the dataset creation support system according to the second embodiment. In addition to S100 to S102, the following steps are performed: step S103 for generating a prediction model using a training dataset; step S104 for estimating experimental conditions that satisfy the target value of the dependent variable using the prediction model; step S105 for conducting a demonstration experiment under the estimated experimental conditions; step S106 for determining whether the demonstration experiment results satisfy the target value of the dependent variable; and step S107 for adding data obtained by the demonstration experiment to the training dataset and returning to step S103 if the result does not satisfy the target value. The training dataset is updated by executing these steps. Each step will be described in detail below, along with the operation of the functional units.

[0023] (S103) In step S103, the prediction model generation unit 15 generates one or more prediction models indicating the relationship between the experimental conditions and the experimental results using the training data set stored in the training data storage unit 141. Initially, the initial training data set created in the first embodiment is used, and thereafter, updated training data sets are used. When generating the prediction model, a prediction model indicating the relationship between the experimental conditions and the experimental results within the search region is generated by regression analysis such as a machine learning model, such as a support vector machine or a neural network, or statistical analysis such as correlation analysis, principal component analysis, or multiple regression analysis. Furthermore, a machine learning model such as a decision tree or clustering may be used as the prediction model.

[0024] (S104) In step S104, the experimental condition estimation unit 16 performs a solution search using a prediction model, i.e., estimates experimental conditions that satisfy the target, working backward from the target of the dependent variable. In the solution search, when estimating solutions for candidate experimental conditions, solutions are estimated for all experimental conditions or for a limited set of experimental conditions. Furthermore, the reliability of the solution estimation may be evaluated by calculating the variance of the estimation accuracy in addition to the solution estimation using a Bayesian process. Furthermore, if a clustering model is included among the one or more prediction models generated in step S103, solutions may be estimated for each of multiple clusters using the other prediction models generated in step S103. Furthermore, if it is estimated that no experimental conditions satisfy the target of the dependent variable exist, it may estimate experimental conditions that produce results closest to the target value among the estimated experimental results. The experimental conditions proposed by the above method are within the search set range of the experimental conditions input in step S100. If the newly added experimental conditions are outside the range of the search settings, it may be suggested that the device for obtaining the experimental results needs to be changed or that the search setting range of the experimental conditions needs to be expanded.

[0025] (S105) In step S105, a demonstration experiment is carried out based on the experimental conditions estimated in step S104, and experimental results are obtained.

[0026] (S106) In step S106, the demonstration experiment result determination unit 17 checks whether the demonstration experiment results obtained in step S105 satisfy the target values ​​of the objective variables input in step S100. If the demonstration experiment results satisfy the target, it is determined that there is little need to update the prediction model, and the search for a solution to the optimization problem is terminated. However, even in this case, a dataset of new experimental results and experimental conditions may be added to the training dataset. If the demonstration experiment results do not satisfy the target, the process proceeds to step S107.

[0027] (S107) In step S107, the combination of the experimental conditions proposed in step S104 and the demonstration experiment results obtained in step S105 is added to the training data set in the training data storage unit 141, the prediction model is further updated, and steps S103 to S105 are repeatedly performed. Again, in step S106, it is evaluated whether the demonstration experiment results satisfy the goal, and a decision is made to end the solution search for the optimization problem or proceed to step S107. By repeating the above steps, the prediction model is improved and the training data set is expanded.

[0028] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and applications are possible without departing from the spirit of the present invention. For example, the use of creating a dataset is not limited to learning purposes, and it may be applied to various uses such as demonstration experiments and quality control.

[0029] The following are examples of aspects that may be included in the present invention, but the present invention is not limited to these. (Aspect 1) A dataset creation support system that executes an explanatory variable input process to acquire types of explanatory variables and values ​​of the explanatory variables; an experimental design candidate generation process to generate experimental design candidates consisting of combinations of experimental conditions corresponding to an arbitrary number of experiments from the types of explanatory variables and the values ​​of the explanatory variables; an experimental design evaluation value calculation process to calculate evaluation values ​​for the experimental design candidates; and an experimental design evaluation value output process to output a relationship between the number of experiments and the evaluation values ​​obtained from a plurality of experimental design candidates. (Aspect 2) The dataset creation support system according to Aspect 1, wherein the evaluation value is an optimal criterion for experimental design. (Aspect 3) The dataset creation support system according to Aspect 1 or 2, wherein the evaluation value in the relationship between the number of experiments and the evaluation value output by the experimental design evaluation value output process is the best evaluation value (best evaluation value) among the experimental design candidates that satisfy a predetermined number of experiments. (Aspect 4) The dataset creation support system according to Aspect 3, wherein the best evaluation value is expressed as a relative value to the best evaluation value (highest best evaluation value) among all possible number of experiments. (Aspect 5) The dataset creation support system according to Aspect 3 or 4, wherein the explanatory variable input process acquires constraint conditions related to explanatory variables, and when multiple experimental design candidates that give the best evaluation value in different experiment counts are obtained, the constraint conditions prioritize experimental design candidates that include common experimental conditions, and executes an experimental design determination output process to determine an experimental design that satisfies the constraint conditions. (Aspect 6) The dataset creation support system according to any one of Aspects 1 to 5, wherein the explanatory variable input process acquires one or more proven explanatory variables and constraint conditions related to the explanatory variables, and the constraint conditions include experimental conditions related to a predetermined number of proven explanatory variables out of the one or more proven explanatory variables, and executes an experimental design determination output process to determine an experimental design that satisfies the constraint conditions.(Aspect 7) The dataset creation support system according to any one of Aspects 1 to 5, wherein the explanatory variable input process acquires one or more proven explanatory variables and constraint conditions related to the explanatory variables, the constraint conditions not including experimental conditions related to any of the proven explanatory variables among the one or more proven explanatory variables, and executes an experimental design determination output process for determining an experimental design so as to satisfy the constraint conditions. (Aspect 8) A dataset creation support device comprising: an explanatory variable / objective variable acquisition unit that acquires types of explanatory variables and values ​​of the explanatory variables, an experimental design calculation unit that generates experimental design candidates consisting of combinations of experimental conditions corresponding to any number of experiments from the types of explanatory variables and the values ​​of the explanatory variables and calculates evaluation values ​​of the experimental design candidates, and an experimental design output unit that outputs the relationship between the number of experiments obtained from a plurality of experimental design candidates and the evaluation values. (Aspect 9) The dataset creation support device according to Aspect 8, wherein the evaluation value is an optimal criterion for experimental design. (Aspect 10) The dataset creation support device according to Aspect 8 or 9, characterized in that the evaluation value in the relationship between the number of experiments and the evaluation value output by the experimental design output unit is the best evaluation value (best evaluation value) among experimental design candidates that satisfy a predetermined number of experiments. (Aspect 11) The dataset creation support device according to any one of Aspects 8 to 10, wherein the explanatory variable / target variable acquisition unit acquires one or more proven explanatory variables and constraint conditions related to the explanatory variables, the constraint conditions do not include experimental conditions related to any of the proven explanatory variables among the one or more proven explanatory variables, and the experimental design output unit determines an experimental design that satisfies the constraint conditions.(Aspect 12) A dataset creation support program that causes a computer processor to execute: an explanatory variable input process that acquires types of explanatory variables and values ​​of the explanatory variables; an experimental design candidate generation process that generates experimental design candidates consisting of combinations of experimental conditions corresponding to an arbitrary number of experiments, from the types of explanatory variables and the values ​​of the explanatory variables; an experimental design evaluation value calculation process that calculates evaluation values ​​for the experimental design candidates; and an experimental design evaluation value output process that outputs a relationship between the number of experiments and the evaluation value obtained from a plurality of experimental design candidates. (Aspect 13) The dataset creation support program according to Aspect 12, wherein the evaluation value is an optimal criterion for experimental design. (Aspect 14) The dataset creation support program according to Aspect 12 or 13, wherein the evaluation value in the relationship between the number of experiments and the evaluation value outputted in the experimental design evaluation value output process is the best evaluation value (best evaluation value) among experimental design candidates that satisfy a predetermined number of experiments. (Aspect 15) The dataset creation assistance program according to any one of Aspects 12 to 14, wherein in the explanatory variable input process, one or more proven explanatory variables and constraint conditions related to the explanatory variables are acquired, the constraint conditions do not include experimental conditions related to any of the proven explanatory variables among the one or more proven explanatory variables, and an experimental design determination output process is executed to determine an experimental design that satisfies the constraint conditions.

[0030] 1, 2... Data set creation support system, 11... Explanatory variable / target variable acquisition unit, 12... Experimental design calculation unit, 13... Experimental design output unit, 14... Memory unit, 141... Learning data memory unit, 142... Acquired data memory unit, 15... Prediction model generation unit, 16... Experimental condition estimation unit, 17... Demonstration experiment result determination unit

Claims

An explanatory variable input process for acquiring the type of explanatory variable and the value of the explanatory variable; an experimental design candidate generation process for generating experimental design candidates consisting of combinations of experimental conditions corresponding to an arbitrary number of experiments from the types of explanatory variables and the values ​​of the explanatory variables; an experimental design evaluation value calculation process for calculating an evaluation value of the experimental design candidate; Execute an experiment design evaluation value output process that outputs the relationship between the number of experiments and the evaluation value obtained from the plurality of experiment design candidates. A dataset creation support system. 　 The evaluation value is an optimal criterion for experimental design. The data set creation support system according to claim 1 . 　 The evaluation value in the relationship between the number of experiments and the evaluation value output in the experimental design evaluation value output process is the best evaluation value (best evaluation value) among the experimental design candidates that satisfy a predetermined number of experiments.

3. The data set creation support system according to claim 1. 　 The best evaluation value is expressed as a relative value to the best evaluation value (highest best evaluation value) among all possible number of experiments. The data set creation support system according to claim 3. 　 In the explanatory variable input process, constraint conditions related to explanatory variables are acquired; The constraint condition is such that, when a plurality of experimental design candidates that give the best evaluation value in different numbers of experiments are obtained, a candidate experimental design that includes common experimental conditions is prioritized; Execute an experimental design determination output process to determine an experimental design so as to satisfy the constraints. The data set creation support system according to claim 3. 　 In the explanatory variable input process, one or more proven explanatory variables and constraints on the explanatory variables are acquired; The constraint condition includes an experimental condition related to a predetermined number of proven explanatory variables among the one or more proven explanatory variables, Execute an experimental design determination output process to determine an experimental design so as to satisfy the constraints.

3. The data set creation support system according to claim 1. 　 In the explanatory variable input process, one or more proven explanatory variables and constraints on the explanatory variables are acquired; The constraints do not include experimental conditions related to any of the one or more verified explanatory variables, Execute an experimental design determination output process to determine an experimental design so as to satisfy the constraints.

3. The data set creation support system according to claim 1. 　 an explanatory variable / objective variable acquisition unit that acquires the types of explanatory variables and the values ​​of the explanatory variables; an experimental design calculation unit that generates experimental design candidates composed of combinations of experimental conditions corresponding to an arbitrary number of experiments from the types of explanatory variables and the values ​​of the explanatory variables, and calculates an evaluation value of the experimental design candidates; an experiment plan output unit that outputs a relationship between the number of experiments obtained from a plurality of experiment plan candidates and an evaluation value; Dataset creation support device. 　 The evaluation value is an optimal criterion for experimental design. The data set creation support device according to claim 8. 　 The evaluation value in the relationship between the number of experiments and the evaluation value output by the experimental design output unit is the best evaluation value (best evaluation value) among the experimental design candidates that satisfy a predetermined number of experiments.

10. The data set creation support device according to claim 8 or 9. 　 the explanatory variable / objective variable acquisition unit acquires one or more proven explanatory variables and constraints related to the explanatory variables; The constraints do not include experimental conditions related to any of the one or more verified explanatory variables, The experimental design output unit determines an experimental design so as to satisfy the constraints.

10. The data set creation support device according to claim 8 or 9. 　 For computer processors, An explanatory variable input process for acquiring the type of explanatory variable and the value of the explanatory variable; an experimental design candidate generation process for generating experimental design candidates consisting of combinations of experimental conditions corresponding to an arbitrary number of experiments from the types of explanatory variables and the values ​​of the explanatory variables; an experimental design evaluation value calculation process for calculating an evaluation value of the experimental design candidate; Execute an experiment design evaluation value output process that outputs the relationship between the number of experiments and the evaluation value obtained from a plurality of experiment design candidates. A program to assist in creating datasets. 　 The evaluation value is an optimal criterion for experimental design. The data set creation support program according to claim 12. 　 The evaluation value in the relationship between the number of experiments and the evaluation value output in the experimental design evaluation value output process is the best evaluation value (best evaluation value) among the experimental design candidates that satisfy a predetermined number of experiments.

14. The data set creation support program according to claim 12 or 13. 　 In the explanatory variable input process, one or more proven explanatory variables and constraints on the explanatory variables are acquired; The constraints do not include experimental conditions related to any of the one or more verified explanatory variables, Execute an experimental design determination output process to determine an experimental design so as to satisfy the constraints.

14. The data set creation support program according to claim 12 or 13.

Images