Data selection method and data selection apparatus

Abstract

To reduce the number of pieces of data used in relearning of a machine learning model.SOLUTION: A data selection method according to an embodiment of the present disclosure includes obtaining a plurality of first predicted values by inputting a plurality of first parameter sets to a prediction model generated by performing machine learning using a parameter set and a measured value corresponding to the parameter set as teacher data, obtaining a plurality of second predicted values by inputting a plurality of second parameter sets having a combination of values of the parameters different from those of the first parameter set to the prediction model, specifying a first range indicating a preferential adoption range of the second predicted values based on a relationship between the measured value and the first predicted value, and specifying the second predicted value included in the first range from among the plurality of second predicted values obtained in the second predicted value acquisition step.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a method and apparatus for selecting data for use in machine learning. [Background technology]

[0002] Patent Document 1 discloses a data set selection device that makes it possible to apply an experimental design using an active learning method to a plurality of data sets for which data sets are defined in advance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-304782 Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable to reduce the amount of data used in additional machine learning to improve the accuracy of a model generated by machine learning, and to perform re-learning with less time and effort. [Means for solving the problem]

[0005] A data selection method according to an embodiment of the present disclosure includes: a first predicted value acquisition step of inputting a plurality of first parameter sets into a prediction model generated by performing machine learning using, as training data, a parameter set having a combination of values ​​of a plurality of parameters and actual measured values ​​corresponding to the parameter sets, thereby obtaining a plurality of first predicted values; a second predicted value acquisition step of inputting a plurality of second parameter sets, having a combination of values ​​of the parameters different from those of the first parameter set, into the prediction model, thereby obtaining a plurality of second predicted values; a first range identification step of identifying a first range indicating a preferential range for the second predicted values ​​based on a relationship between the actual measured values ​​and the first predicted values; and a second predicted value identification step of identifying the second predicted value included in the first range from the plurality of second predicted values ​​obtained in the second predicted value acquisition step.

[0006] A data selection device according to one embodiment of the present disclosure includes a first acquisition unit that obtains a plurality of first predicted values ​​by inputting a plurality of first parameter sets into a prediction model generated by performing machine learning using, as training data, a parameter set having a combination of values ​​of a plurality of parameters and actual measured values ​​corresponding to the parameter sets; a second acquisition unit that obtains a plurality of second predicted values ​​by inputting a plurality of second parameter sets having a combination of values ​​of the parameters different from those of the first parameter set into the prediction model; a first identification unit that identifies a first range indicating a preferential adoption range of the second predicted values ​​based on a relationship between the actual measured values ​​and the first predicted values; and a second identification unit that identifies the second predicted values ​​included in the first range from the plurality of second predicted values ​​acquired by the second acquisition unit. [Effects of the Invention]

[0007] According to one aspect of the present disclosure, the amount of data used in retraining a machine learning model can be reduced. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a functional block diagram illustrating a configuration of a model generation device according to a first embodiment of the present disclosure. [Figure 2]4 is a flowchart showing an example of a processing flow in the model generating device. [Figure 3] 10 shows an example of the configuration of preliminary test data. [Figure 4] 10 shows an example of the configuration of additional test data. [Figure 5] 10 shows an example of a scatter diagram in which the relationship between the first actual measurement value and the first predicted value is represented by a plurality of plots. [Figure 6] FIG. 10 is a diagram for explaining another method for specifying the first range. [Figure 7] 1 shows an example of a dataset table. [Figure 8] 10 is a graph illustrating a method for identifying a dataset table to be used for re-learning a prediction model. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Embodiment 1] Hereinafter, a first embodiment of the present disclosure will be described. Components having the same functions as those described in the first embodiment will be denoted by the same reference numerals in the following embodiments, and the description thereof will not be repeated. All components and numerical values ​​described in this specification are merely examples, unless there is a contradiction in the content. Therefore, unless there is a contradiction in the content, for example, the positional relationship and connection relationship of each component are not limited to the examples in each figure.

[0010] 1 is a block diagram showing the configuration of a model generation device 10 according to the first embodiment of the present disclosure. As shown in FIG. 1, the model generation device 10 includes a main control unit 1, a storage unit 2, an input unit 3, and a display unit 4.

[0011] The main control unit 1 includes a model generation unit 11, a first acquisition unit 12, a second acquisition unit 13, a first identification unit 14, a second identification unit 15, a dataset group creation unit 16, a priority calculation unit 17, and a third identification unit 18. The data selection device in the present disclosure may include all or some of the first acquisition unit 12, the second acquisition unit 13, the first identification unit 14, the second identification unit 15, the dataset group creation unit 16, the priority calculation unit 17, and the third identification unit 18.

[0012] The storage unit 2 stores (1) the control programs for each unit, (2) the OS program, (3) the application programs, and (4) various data read when these programs are executed by the main control unit 1. The storage unit 2 is configured with a non-volatile storage device such as a hard disk or flash memory.

[0013] In particular, the memory unit 2 stores a predictive model 60, preliminary test data 30 and additional test data 40 used for machine learning of the predictive model 60, as well as a dataset table 50 used to select test data to be used for machine learning from multiple additional test data 40.

[0014] The input unit 3 is an input device for a user to operate the model generating device 10, and is, for example, a keyboard and a mouse.

[0015] The display unit 4 is a display device that displays a screen for inputting information to the model generation device 10, and the results of various processes in the model generation device 10, etc.

[0016] (Example of processing flow in model generation device 10) 2 is a flowchart showing an example of the flow of processing (data selection method) in the model generation device 10. The characteristic value as the objective variable handled in this embodiment can be expressed by a model including multiple parameters (explanatory variables). For example, the characteristic value y can be expressed as shown in the following equation (1) when there are two parameters, x1 and x2, the coefficients of each parameter are a1 and a2, and b is a constant term.

[0017] y=a1x1+a2x2+b (1) Equation (1) is an example of a linear regression model for predicting the characteristic value y. However, the regression model may also be a nonlinear regression model. Therefore, for example, at least one of the parameters may be a squared term.

[0018] Equation (1) is an example in which the number of parameters used to express the characteristic value y is 2. However, the number of parameters may be 3 or more. In the following example, for the sake of simplicity, it is assumed that the characteristic value y is expressed by equation (1).

[0019] First, the model generation unit 11 of the model generation device 10 performs machine learning of the prediction model 60 using the preliminary test data 30 as training data in accordance with a user's instruction (S1).

[0020] 3 shows an example of the configuration of the preliminary test data 30 and 30A. As shown in FIG. 3, the preliminary test data 30A before the first predicted value 33 is added includes a first parameter set 31 having a combination of the values ​​of the plurality of parameters, and a first actual measurement value 32 as a response variable corresponding to the first parameter set 31.

[0021] The number of levels for each parameter is not particularly limited. In the present disclosure, each parameter can have a first level, a second level, and a third level. In the example shown in FIG. 3, the preliminary test data 30A numbered 2 has the first level for parameter x1 and the second level for parameter x2. The actual values ​​of each level can be determined according to individual experiments.

[0022] The model generation unit 11 performs machine learning of the prediction model 60 using the first parameter set 31 as an explanatory variable and the first actual measurement value 32 as a response variable.

[0023] The first acquisition unit 12 inputs a plurality of first parameter sets 31 into the generated trained prediction model 60 to obtain a plurality of first predicted values ​​33 (S2: first predicted value acquisition step). The table denoted by reference numeral 30 in FIG. 3 shows the preliminary test data 30 after the first predicted values ​​33 have been added.

[0024] The parameter set (learning parameter set) used as an explanatory variable when performing machine learning of the prediction model 60 and the first parameter set 31 input to the trained prediction model 60 to obtain the first predicted value 33 may be the same parameter set or different parameter sets. The learning parameter set may be any parameter set that can perform machine learning of the prediction model 60.

[0025] The second acquisition unit 13 obtains multiple second predicted values ​​43 by inputting multiple second parameter sets 41 having a combination of parameter values ​​different from the first parameter set 31 used by the first acquisition unit 12 into the prediction model 60 (S3: second predicted value acquisition process).

[0026] The second predicted value 43 is used in the second predicted value specification step (S5) described later. Therefore, the second predicted value acquisition step (S3) may be performed immediately before the second predicted value specification step (S5). Conversely, the second predicted value acquisition step (S3) may be performed before the first predicted value acquisition step (S2).

[0027] Furthermore, as will be described in a third embodiment below, in an embodiment in which the second predicted value identification step (S5) is omitted, the second predicted value 43 is used in a re-learning step (S9) in which the prediction model 60 is re-learned. Therefore, the second predicted value acquisition step (S3) may be executed immediately before the re-learning step (S9).

[0028] Fig. 4 shows an example of the configuration of additional test data 40A. A combination of a second parameter set 41 and a second predicted value 43 is referred to as additional test data 40A. The additional test data 40A is candidate data to be used for re-training the prediction model 60. In this specification, the upper part of Fig. 4 shows additional test data 40B before the second predicted value 43 is added.

[0029] Fig. 5 shows an example of a scatter diagram in which the relationship between the first actual values ​​32 and the first predicted values ​​33 is represented by a plurality of plots 61. In the graph of Fig. 5, the coordinate axis showing the first actual values ​​32 is the x-axis, the coordinate axis showing the first predicted values ​​33 is the y-axis, and the straight line showing y = x in the xy coordinate system defined by the x-axis and the y-axis is shown as a reference line 62.

[0030] The first identifying unit 14 identifies a first range indicating a preferentially adopted range of the second predicted values ​​43 based on the distribution of the plots 61 in a scatter plot that represents the relationship between the first actual values ​​32 and the first predicted values ​​33 using a plurality of plots 61 (S4: first range identifying step). The first range is the range of the second predicted values ​​43, but the algorithm for identifying the first range identifies the first range based on the distribution of the first predicted values ​​33. Therefore, the second predicted values ​​43 may not have been obtained at the time the first range identifying step is performed. Step S4 may be performed by the user.

[0031] For example, the first identifying unit 14 may calculate the density of the plots 61 and identify, as the first range, a range in which the density of the plots 61 is equal to or less than a first threshold. That is, as shown in Fig. 5, the first identifying unit 14 identifies, as the first range, a range of the first predicted value 33 corresponding to a region 63 in which no plots 61 exist or in which the density of the plots 61 is equal to or less than a predetermined first threshold. The region 63 is a region that includes a part of the reference line 62, and is a region in which the range of the vertical axis coordinate indicating the first predicted value 33 is separated by two line segments parallel to the horizontal axis indicating the first actual measurement value 32.

[0032] As an example, the "existence density of the plots 61" may be defined as the number of plots 61 per reference area on the xy coordinate plane. The reference area may be determined in advance according to the distribution range and number of the plots 61.

[0033] Furthermore, the range on the xy coordinate plane for calculating the density of the plots 61 may be determined based on the distribution range of the plots 61. For example, the first identifying unit 14 may calculate the density of the plots 61 within a rectangular range defined by a line parallel to the y coordinate passing through the minimum x coordinate value in a set of multiple plots 61, a line parallel to the y coordinate passing through the maximum x coordinate value, a line parallel to the x coordinate passing through the minimum y coordinate value, and a line parallel to the x coordinate passing through the maximum y coordinate value.

[0034] The first specifying unit 14 may widen the first range by adding a predetermined margin to the upper limit value and / or the lower limit value of the specified first range.

[0035] 6 is a diagram illustrating another method for specifying the first range. In the graph of FIG. 6, a line indicating y=x in an xy coordinate system represented by an x-axis indicating the first actual values ​​32 and a y-axis indicating the first predicted values ​​33 is shown as a reference line 62. The first specifying unit 14 may specify, as the first range, a range in which the deviation of each plot 61 from the reference line 62 is equal to or greater than a second threshold. That is, the first specifying unit 14 may calculate the deviation of each plot 61 from the reference line 62, and specify, as the first range, a range of the first predicted values ​​33 corresponding to a set of plots 61 in which the deviation is equal to or greater than a predetermined second threshold.

[0036] There are no particular limitations on the calculation of the deviation. For example, as described above, the reference line 62 shown in FIG.

[0037] y = x (2) Here, the deviation of the nth plot 61 is z n It is written as z n may be determined, for example, as shown in the following formula (3).

number

[0038] Furthermore, the first identifying unit 14 may identify the first range based on both the density of the plots 61 and the deviation of each plot 61 from the reference line 62. For example, the first identifying unit 14 may calculate the density of the plots 61 within a range defined to surround a set of plots 61 whose deviation from the reference line 62 is smaller than a second threshold, and identify the range in which the density of the plots 61 is equal to or smaller than the first threshold as the first range.

[0039] Next, the second identification unit 15 identifies the second predicted value 43 that falls within the first range identified in step S4 from among the multiple second predicted values ​​43 obtained in step S3 (S5: second predicted value identification process).Step S5 may be performed by a user.

[0040] A combination of the second predicted value 43 identified in step 5 and the second parameter set 41 corresponding to the second predicted value 43 is referred to as a dataset. A group of datasets including at least one dataset is referred to as a dataset group. However, in the priority calculation step (S7) described below, it is the second parameter set 41 that is used, and not the second predicted value 43. Therefore, the dataset and the dataset group may not include the second predicted value 43.

[0041] The dataset group creation unit 16 creates a plurality of dataset tables 50 each indicating a dataset group including at least one dataset (S6: dataset group creation step). The dataset group does not need to be expressed in a table format, and may be expressed in any data format.

[0042] FIG. 7 shows a first data set table 51 and a second data set table 52 as examples of the data set table 50.

[0043] The priority calculation unit 17 calculates a priority for each of the multiple data set tables 50 (S7: priority calculation step). The priority may be calculated based on an evaluation value of the optimization criterion. As the evaluation value of the optimization criterion, D efficiency of the D optimization criterion, A efficiency of the A optimization criterion, I optimality of the I optimization criterion, E efficiency of the E optimization criterion, or G efficiency of the G optimization criterion may be used. D efficiency or I optimality may be used preferentially.

[0044] When the D efficiency is used as the evaluation value of the optimum standard, the priority calculation unit 17 may calculate the D efficiency shown in the following formula (4) as the priority.

number

number

number

[0045] Next, the third identification unit 18 identifies the dataset tables 50 to be used for re-learning the prediction model 60 based on the relationship between the priority of each dataset table 50 and the number of datasets included in that dataset table 50 (S8: dataset group identification process).

[0046] Fig. 8 is a graph illustrating a method for identifying a dataset table 50 to be used for re-learning the prediction model 60. Fig. 8 shows a graph in which the horizontal axis indicates the number of datasets included in each dataset table 50, and the vertical axis indicates the priority of each dataset table 50. A regression curve derived from multiple plots 71 is indicated by reference numeral 72.

[0047] As the number of datasets included in the dataset table 50 increases, the priority tends to increase, but once the number of datasets increases to a certain extent, the increase in priority tends to slow down. Therefore, the third identification unit 18 may identify the number of datasets at or just before the point at which the increase in priority starts to slow down as the number of datasets to be used for re-training the prediction model 60.

[0048] Specifically, the third identification unit 18 identifies the dataset table 50 to be used for re-learning based on a change in the slope of a tangent line 73 on a graph 72 that indicates the relationship between the priority and the number of datasets included in the dataset table 50. The third identification unit 18 may identify the number of datasets at the point where the slope of the tangent line 73 becomes equal to or less than a predetermined third threshold as the number of datasets to be used for re-learning the prediction model 60.

[0049] Furthermore, the number of data sets used for re-learning the prediction model 60 is not limited to one pattern, and the third specifying unit 18 may specify the number of data sets for multiple patterns. For example, the third specifying unit 18 may specify, as the number of data sets to be used for re-learning the prediction model 60, the number of data sets within a predetermined range based on the number of data sets at the point where the slope of the tangent line 73 becomes equal to or less than a predetermined third threshold.

[0050] As a specific example, if the number of datasets at the point where the gradient of the tangent line 73 is equal to or less than a predetermined third threshold is 10, the dataset groups to be used for re-learning may be 9, 10, or 11. Furthermore, the third identification unit 18 may identify a dataset group with a relatively high priority from among the plurality of dataset groups identified in this way as the dataset group to be used for re-learning.

[0051] When the user executes step S8, it is not necessary to calculate the slope of the tangent line 73, and it is sufficient to identify the number of data sets to be used for re-learning the prediction model 60 from the shape of the graph 72.

[0052] The method for identifying the dataset table 50 used for relearning the prediction model 60 is not limited to the above-described method. The third identification unit 18 may identify the number of datasets based on the upward trend of priority in the graph 72. As an example, the third identification unit 18 may derive an arbitrary index value indicating the upward trend of priority in the graph 72, and identify the number of datasets based on the index value.

[0053] After step S8, the model generating unit 11 acquires, from the storage unit 2, a second measured value 42 as a response variable corresponding to the second parameter set 41. In the example of FIG. 1 , the second measured value 42 is a value acquired by a measurement device 100 external to the model generating device 10. The storage unit 2 may store the second measured value 42 supplied from the measurement device 100 in the additional test data 40.

[0054] Next, the model generation unit 11 retrains the prediction model 60 using the dataset indicated by at least one dataset table 50 having the number of datasets identified in step S8 and the second actual measurement value 42 corresponding to that dataset (S9).

[0055] As described above, the model generation device 10 can select additional test data that is likely to improve the accuracy of the trained model, and can efficiently retrain the trained model even when the number of parameters is large.

[0056] The above-described dataset group creation step (S6), priority calculation step (S7), and dataset group identification step (S8) may be applied to narrowing down the first predicted values ​​33. That is, the dataset group creation unit 16 may create a plurality of first dataset groups including at least one first parameter set 31 from a set of first parameter sets 31 corresponding to the first predicted values ​​33 (first dataset group creation step). The priority calculation unit 17 may calculate a priority for each first dataset group (first priority calculation step). The third identification unit 18 may identify a first dataset group including the first predicted value 33 used in the first range identification step (S4) based on the relationship between the priority and the number of datasets included in the first dataset group (first dataset group identification step).

[0057] This configuration makes it possible to select a preferred first predicted value 33 from the multiple first predicted values ​​33 used in the first range identification step. The first identification unit 14 may identify a first range indicating a preferential adoption range of the second predicted values ​​43 based on the relationship between the first actual values ​​32 and the first predicted values ​​33 included in the first dataset group identified as described above.

[0058] In relation to the above configuration, a dataset group including at least one of the above-described second parameter sets may be referred to as a second dataset group. The process of creating a plurality of second dataset groups including at least one second parameter set 41 may be referred to as a second dataset group creation process. The process of calculating priorities for the second dataset groups may be referred to as a second priority calculation process. The process of identifying a second dataset group to be used for re-training the prediction model 60 may be referred to as a second dataset group identification process.

[0059] [Embodiment 2] In the first embodiment, the case where the prediction model 60 is re-learned through a series of processes from the model generating unit 11 to the third identifying unit 18 has been exemplified. However, it is also possible to omit part of this series of processes.

[0060] For example, among the series of processes described in the first embodiment, the process by the dataset group creation unit 16, the process by the priority calculation unit 17, and the process by the third identification unit 18 may be omitted. That is, steps S6 to S8 in the flowchart of FIG. 2 may be omitted.

[0061] In this case, step S9 is executed following step S5. In this case, after the second identification unit 15 identifies the second predicted value 43 included in the first range in step S5, the model generation unit 11 acquires the second actual measurement value 42 as the dependent variable from the storage unit 2.

[0062] Next, in step S9, the model generation unit 11 re-learns the prediction model 60 using the second parameter set 41 corresponding to the second predicted value 43 identified in step S5 and the second actual measured value 42 corresponding to the second parameter set 41.

[0063] [Embodiment 3] The dataset group creation unit 16 may create a plurality of dataset tables 50 each indicating a dataset group including at least one dataset, for a plurality of datasets indicating all combinations of parameter levels that the model exemplified in formula (1) can take. This configuration makes it possible to efficiently identify which of a plurality of parameters (explanatory variables) in a model indicating characteristic values ​​is important.

[0064] In the case of the above configuration, the processes by the first acquiring unit 12, the first identifying unit 14, and the second identifying unit 15 may be omitted. That is, steps S2, S4, and S5 in the flowchart of Fig. 2 may be omitted. Because the second predicted value 43 is used in the re-learning step (S9) in which the prediction model 60 is re-learned, the second predicted value acquiring step (S3) may be executed immediately before the re-learning step (S9).

[0065] [Software implementation example] The functions of the model generation device 10 (hereinafter referred to as the "device") can be realized by a program for causing a computer to function as the device, and a program (data selection program) for causing a computer to function as each control block of the device (particularly each part included in the main control unit 1).

[0066] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0067] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0068] In addition, some or all of the functions of each of the control blocks can be realized by logic circuits. For example, integrated circuits in which logic circuits that function as each of the control blocks are formed are also included in the scope of the present disclosure. In addition, the functions of each of the control blocks can also be realized by, for example, a quantum computer.

[0069] The invention according to the present disclosure has been described above based on the drawings and examples. However, the invention according to the present disclosure is not limited to the above-described embodiments. In other words, the invention according to the present disclosure can be modified in various ways within the scope of the present disclosure, and embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the invention according to the present disclosure. In other words, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. It should also be noted that these modifications or corrections are included in the scope of the present disclosure.

[0070] 〔summary〕 A data selection method according to a first aspect of the present disclosure includes a first predicted value acquisition step of inputting a plurality of first parameter sets into a prediction model generated by performing machine learning using, as training data, parameter sets having a combination of values ​​of a plurality of parameters and actual measured values ​​corresponding to the parameter sets, thereby obtaining a plurality of first predicted values; a second predicted value acquisition step of inputting a plurality of second parameter sets having a combination of values ​​of the parameters different from those of the first parameter sets into the prediction model, thereby obtaining a plurality of second predicted values; a first range identification step of identifying a first range indicating a preferential adoption range of the second predicted values ​​based on a relationship between the actual measured values ​​and the first predicted values; and a second predicted value identification step of identifying the second predicted value included in the first range from the plurality of second predicted values ​​obtained in the second predicted value acquisition step.

[0071] In a data selection method according to aspect 2 of the present disclosure, in aspect 1, in the first range identification step, the first range may be identified as a range in which the density of the multiple plots in a scatter plot showing the relationship between the actual measured value and the first predicted value is equal to or less than a first threshold value.

[0072] In a data selection method according to aspect 3 of the present disclosure, in aspect 1 or 2, the coordinate axis showing the actual measured value is the x-axis, the coordinate axis showing the first predicted value is the y-axis, and the straight line showing y=x in the xy coordinate system shown by the x-axis and the y-axis is the reference straight line, and in the first range identification step, the first range may be identified as a range in which the deviation of each plot from the reference straight line is equal to or greater than a second threshold.

[0073] A data selection method according to aspect 4 of the present disclosure, in any one of aspects 1 to 3, may include a first dataset group creation step of creating a plurality of first dataset groups, each including at least one of the first parameter sets, from a set of the first parameter sets corresponding to the first predicted value; a first priority calculation step of calculating a priority for each of the first dataset groups; and a first dataset group identification step of identifying the first dataset group including the first predicted value to be used in the first range identification step, based on the relationship between the priority and the number of datasets included in the first dataset group.

[0074] A data selection method according to aspect 5 of the present disclosure, in any one of aspects 1 to 4, further comprises: a second dataset group creation step of creating a plurality of second dataset groups, each including at least one of the second parameter sets, from a set of the second parameter sets corresponding to the second predicted value; and a second priority calculation step of calculating a priority for each of the second dataset groups. The method may further include a second dataset group identification step of identifying the second dataset group to be used for retraining the prediction model based on the relationship between the priority and the number of datasets included in the second dataset group.

[0075] In the data selection method according to a sixth aspect of the present disclosure, in the fourth or fifth aspect, the priority may be calculated based on an evaluation value of an optimality criterion.

[0076] In a data selection method according to aspect 7 of the present disclosure, in aspect 5, the second group of datasets may be identified based on a change in the slope of a tangent to a graph showing the relationship between the priority and the number of datasets.

[0077] A data selection device according to aspect 8 of the present disclosure includes a first acquisition unit that inputs a plurality of first parameter sets into a prediction model generated by performing machine learning using, as training data, a parameter set having a combination of values ​​of a plurality of parameters and actual measurement values ​​corresponding to the parameter sets, thereby obtaining a plurality of first predicted values; a second acquisition unit that inputs a plurality of second parameter sets into the prediction model, each having a combination of values ​​of the parameters that differ from the training data, thereby obtaining a plurality of second predicted values; a first identification unit that identifies a first range indicating a preferential adoption range of the second predicted values ​​based on the relationship between the actual measurement values ​​and the first predicted values; and a second identification unit that identifies the second predicted values ​​included in the first range from the plurality of second predicted values ​​acquired by the second acquisition unit.

[0078] The data selection program of aspect 9 of the present disclosure may be a data selection program for causing a computer to function as the data selection device of aspect 7, or may be a data selection program for causing a computer to function as the first acquisition unit, the second acquisition unit, the first identification unit, and the second identification unit. [Explanation of symbols]

[0079] 10 Model generation device 12 First acquisition part 13 Second acquisition part 14 1st Specific Part 15 Second Specific Part 16 Dataset Group Creation Department 17 Priority calculation unit 18 Third Specific Part 30, 30A preliminary test data 31 First parameter set 32 First measured value 33 First predicted value 40, 40A, 40B Additional test data 41 Second parameter set 42 Second measured value 43 Second predicted value 50, 52 Dataset Table 60 Predictive Models 62 Reference Line 72 graphs 73 Tangent line x1, x2 parameters y characteristic value z n Deviation

Claims

1. a first predicted value acquisition step of inputting a plurality of first parameter sets into a prediction model generated by performing machine learning using a parameter set having a combination of values ​​of a plurality of parameters and actual measurement values ​​corresponding to the parameter set as training data, to obtain a plurality of first predicted values; a second predicted value acquisition step of obtaining a plurality of second predicted values ​​by inputting a plurality of second parameter sets having combinations of values ​​of the parameters different from those of the first parameter set into the prediction model; a first range specifying step of specifying a first range indicating a preferential adoption range of the second predicted value based on the relationship between the actual measurement value and the first predicted value; a second predicted value specifying step of specifying the second predicted value included in the first range from the plurality of second predicted values ​​obtained in the second predicted value obtaining step.

2. 2. The data selection method according to claim 1, wherein in the first range specifying step, a range in which an existence density of a plurality of plots in a scatter plot representing the relationship between the actual measurement value and the first predicted value is equal to or less than a first threshold is specified as the first range.

3. a coordinate axis indicating the actual measurement value is defined as an x-axis, a coordinate axis indicating the first predicted value is defined as a y-axis, and a line indicating y=x in an x-y coordinate system defined by the x-axis and the y-axis is defined as a reference line; 2. The data selection method according to claim 1, wherein the step of specifying the first range specifies, as the first range, a range in which the deviation of each plot from the reference line is equal to or greater than a second threshold value.

4. a first data set group creation step of creating a plurality of first data set groups, each including at least one of the first parameter sets, from the set of first parameter sets; a first priority calculation step of calculating a priority for each of the first data set groups; 2. The data selection method according to claim 1, further comprising: a first dataset group identification step of identifying the first dataset group including the first predicted value to be used in the first range identification step, based on a relationship between the priority and the number of datasets included in the first dataset group.

5. a second dataset group creation step of creating a plurality of second dataset groups, each including at least one of the second parameter sets, from the set of second parameter sets corresponding to the second predicted value; a second priority calculation step of calculating a priority for each of the second data sets; 2. The data selection method according to claim 1, further comprising: a second dataset group identification step of identifying the second dataset group to be used for retraining the prediction model based on a relationship between the priority and the number of datasets included in the second dataset group.

6. 6. The data selection method according to claim 4, wherein the priority is calculated based on an evaluation value of an optimum criterion.

7. The data selection method according to claim 5 , wherein the second group of data sets is identified based on a change in the slope of a tangent line on a graph showing the relationship between the priority and the number of data sets.

8. a first acquisition unit that inputs a plurality of first parameter sets into a prediction model generated by performing machine learning using a parameter set having a combination of values ​​of a plurality of parameters and actual measurement values ​​corresponding to the parameter set as training data, and obtains a plurality of first predicted values; a second acquisition unit that acquires a plurality of second predicted values ​​by inputting a plurality of second parameter sets, each having a combination of values ​​of the parameters different from those of the first parameter set, into the prediction model; a first specifying unit that specifies a first range indicating a preferential adoption range of the second predicted value based on the relationship between the actual measurement value and the first predicted value; a second specifying unit that specifies the second predicted value included in the first range from the plurality of second predicted values ​​acquired by the second acquisition unit.

9. A data selection program for causing a computer to function as the data selection device of claim 8, the data selection program causing a computer to function as the first acquisition unit, the second acquisition unit, the first identification unit, and the second identification unit.

Images