Method and program for determining the reliability of an estimation model, and measurement system.

Abstract

To provide a method for determining reliability of an estimation model so as to improve applicability to a new data group and improve reliability of the estimation model.SOLUTION: A method includes: applying Causal inference in statistics to an explanatory variable and an objective variable to estimate causal direction between the explanatory variable and the objective variable; determining whether the objective variable is a direct and unique cause of the explanatory variable; and determining, based on a result of the determination, whether an estimation model is also applicable to new data other than data used in constructing the estimation model. In the step of estimating the causal direction, when a determination is made that it is unlikely that the objective variable is a cause of the explanatory variable, the explanatory variable may be deleted from the estimation model.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a method and program for determining the reliability of an estimation model that estimates an objective variable according to explanatory variables, and a measurement system.

Background Art

[0002] In multivariate analysis, machine learning, and deep learning, which have been rapidly advancing in recent years, an estimation model that estimates an objective variable from explanatory variables is generated. The estimation model is generated based on a given existing data group (training data) and can also be used for a new data group.

[0003] In multivariate analysis, machine learning, etc., it is important to construct a highly reliable estimation model that can be widely applied to new data groups. The degree to which an estimation model can be applied to a new data group is also referred to as "robustness". The wider the range of applicable new data groups, the higher the robustness of the estimation model can be said to be.

[0004] A number of algorithms for providing a highly reliable estimation model have been proposed. Also, a number of methods for determining the reliability of an estimation model have been proposed. Examples of methods for determining the reliability of an estimation model include cross-validation, test set validation, etc.

[0005] In determining the reliability of an estimation model, existing data is divided into data for constructing an estimation model (training data) and data for reliability testing (test data), the estimation model constructed using the former is applied to the latter, and the error between the measured value and the estimated value of the objective variable is measured. The reliability of the estimation model is determined according to the error. The smaller the error, the higher the reliability of the estimation model is determined to be, and the higher the applicability to a new data group becomes. Also, a method for enhancing the reliability of an estimation model has been devised by combining the selection of training data used for constructing the estimation model and these determination methods.

[0006] However, there is a problem in that it is not easy to determine whether an estimation model generated based on an existing dataset is applicable to a new dataset. In other words, while the applicability of the generated estimation model to a new dataset and the selection of optimal variables are usually performed based on simulations using existing data, such as cross-validation and test set validation, whether the generated estimation model is applicable to a new dataset can only be determined by actually applying the new dataset to the estimation model and verifying the error between the observed and estimated values ​​of the target variable. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2017-51162 [Non-patent literature]

[0008] [Non-Patent Document 1] Campomanes et al. (2014) Origin of the spectral shifts among the early intermediates of the rhodopsin photocycle. Journal of the American chemical society. 136:3842-3851. [Non-Patent Document 2] Trivittayasil et al. (2018)Classification of 1-methylcyclopropene treated apples by fluorescence fingerprint using partial least squares discriminant analysis with stepwise selectivity ratio variable selection method. Chemometrics and Intelligent Laboratory Systems. 175:30-36. [Non-Patent Document 3] Kurihara et al., (2021) Investigation of acceleration of computational kernels for causal search processing of large variables using LiNGAM, Research Report High-Performance Computing, 26:1-8. [Non-Patent Document 4] Oyama et al., (2022) Causal search of specific health checkup data in Osaka Prefecture, Information Processing, 63(2) [Non-Patent Document 5] Esbensen and Geladi (2010) Principles of Proper Validation: use and abuse of re-sampling for validation. Chemometrics and Intelligent Laboratory Systems. 168-187. [Non-Patent Document 6] Andersen and Bro (2010) Variable selection in regression - a tutorial. Chemometrics and Intelligent Laboratory Systems. 728-737 [Non-Patent Document 7] A. Hyvarinen and SM Smith. Pairwise Likelihood Ratios for Estimation of Non-Gaussian Structural Equation Models. J. of Machine Learning Research 14:111-152, 2013. [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] The present invention provides a method, program, and measurement system for determining the reliability of an estimation model, which enable the determination of the reliability of the estimation model without inputting new data sets into the estimation model, and which can improve the reliability of the estimation model. [Means for solving the problem]

[0010] To solve the above problems, the method for determining the reliability of an estimation model according to the present invention is a method for determining the reliability of an estimation model that estimates a dependent variable from explanatory variables, and is characterized by comprising the steps of: applying statistical causal inference to the explanatory variables and the dependent variable to estimate the direction of causality between the explanatory variables and the dependent variable; determining whether the dependent variable is the direct and sole cause of the explanatory variables; and determining, according to the result of the determination, whether the estimation model is applicable to new data other than the data used to construct the estimation model.

[0011] Furthermore, the computer program for determining the reliability of the estimation model according to the present invention may be configured to allow a computer to execute the above method. Also, the measurement system according to the present invention is a measurement system comprising a measurement device that measures a target object and acquires measurement data as explanatory variables, and a computer that applies the explanatory variables to an estimation model to output a target variable and calculates various characteristics of the target object, wherein the computer may be equipped with the above computer program and configured to execute the above method.

[0012] In the step of estimating the direction of causality in the determination method according to the present invention, if it is determined that the dependent variable is unlikely to be the cause of the explanatory variable, a further step of deleting the explanatory variable from the estimation model can be performed. In this case, the step of deleting the explanatory variable can delete all explanatory variables related to the determination that the dependent variable is unlikely to be the cause of the explanatory variable at once. Alternatively, the step of deleting the dependent variable can delete the explanatory variables related to the determination that the dependent variable is unlikely to be the cause of the explanatory variable one by one, calculate the estimation error of the estimation model after the reduction, and stop the deletion step when the estimation error satisfies predetermined conditions.

[0013] Furthermore, this determination method may include the steps of appropriately selecting and combining multiple explanatory variables to generate multiple composite variables, and selecting one composite variable from the multiple composite variables to use as the explanatory variable. The step of generating multiple composite variables may include principal component analysis, factor analysis, t-distributed stochastic neighbor embedding (t-SNE), cluster analysis, non-negative matrix factorization (NMF), multivariate curve resolution (MCR), parallel factor analysis (PARAFAC), partial least squares regression analysis, or ensemble learning. In addition, the step of estimating the direction of causality may use LiNGAM (Linear Non-Gaussian Acyclic Model). [Effects of the Invention]

[0014] According to the present invention, it is possible to provide a method for determining the reliability of an estimation model that can increase its applicability to new data sets and improve the reliability of the estimation model. [Brief explanation of the drawing]

[0015] [Figure 1] This is a schematic diagram illustrating an example of the configuration of the measurement system 1 according to the first embodiment. [Figure 2] Figure 1 is a block diagram illustrating an example of the configuration of computer 10. [Figure 3] This is a flowchart illustrating the procedure for determining the reliability of the estimation model in the measurement system 1 of the first embodiment. [Figure 4] This graph illustrates an example of the determination in step S16 of the flowchart in Figure 3. [Figure 5]It is a graph showing the relationship between the measured values and the estimated values by the estimation model for the target variable y1 for which a direct causal relationship is recognized and the target variable y2 for which no direct causal relationship is recognized. [Figure 6] It is a graph for explaining another example of the determination in step S16 of the flowchart in FIG. 3. [Figure 7] It is a graph showing the relationship between the measured values and the estimated values by the estimation model for the target variable y1 for which a direct causal relationship is recognized and the target variable y3 for which no direct causal relationship is recognized. [Figure 8] It is a configuration diagram for explaining the configuration of the computer 10 according to the second embodiment. [Figure 9] It is a flowchart for explaining the operation of the second embodiment. [Figure 10] It is a graph showing the relationship between the measured values and the estimated values by the estimation model before and after the deletion of the explanatory variables for the target variable y3 for which no direct causal relationship is recognized. [Figure 11] It is a configuration diagram for explaining the configuration of the computer 10 according to the third embodiment. [Figure 12] It is a flowchart for explaining the operation of the third embodiment. [Figure 13] It is a graph showing an example of the measurement results according to the embodiment. [Figure 14] It is a graph showing an example of the measurement results according to the embodiment. [Figure 15] It is a schematic diagram for explaining the causal relationship among the explanatory variables, target variables, and unobserved variables. [Figure 16] It is an example of a heatmap as a result of calculating the likelihood between the composite variable of the explanatory variables and the target variable by the Pairwise LiNGAM algorithm (Non-Patent Document 7). [Figure 17] It is a schematic diagram for explaining a method for determining the direction of causality between the target variable and the composite variable. [Figure 18] It is a schematic diagram for explaining a method for determining the direction of causality between the target variable and the composite variable. [Modes for carrying out the invention]

[0016] This embodiment will be described below with reference to the attached drawings. In the attached drawings, functionally identical elements may be indicated by the same number. The attached drawings show embodiments and implementation examples in accordance with the principles of this disclosure, but they are for the purpose of understanding this disclosure and are not to be used in any way to restrict the interpretation of this disclosure. The descriptions in this specification are merely typical examples and do not limit the claims or applications of this disclosure in any way.

[0017] While this embodiment is described in sufficient detail for those skilled in the art to implement the disclosure, it is important to understand that other implementations and forms are possible, and that the configuration and structure can be modified and various elements replaced without departing from the scope and spirit of the technical idea of ​​this disclosure. Therefore, the following description should not be construed as limiting to this.

[0018] [First Embodiment] Referring to the schematic diagram in Figure 1, an example of the configuration of the measurement system 1 according to the first embodiment will be described. This measurement system 1 is a measurement system that measures various characteristics of agricultural products such as fruits (e.g., sugar content, moisture content, color, saturation, etc.), and is equipped with a computer 10 and a spectroscopic measuring device 20, and is connected to a server 40 and a database 50 via a network NW.

[0019] To measure various characteristics of agricultural products, the computer 10 generates and utilizes estimation models based on multivariate analysis, machine learning, or deep learning. In the following description, a configuration is described in which estimation models are generated using spectral data obtained from the spectroscopic measuring device 20 as input. However, this is just one example, and data from various measuring devices, cameras, sensors, etc., may be input to the computer instead of, or in addition to, the spectroscopic measuring device 20 and used to measure various characteristics.

[0020] Computer 10 is supplied with spectral data as input data (explanatory variables) from the spectroscopic measuring device 20, for example, by measuring agricultural products. It is configured to apply this spectral data to an estimation model and output various characteristics of the agricultural product, such as characteristics y1, y2, y3, etc. (dependent variables). Characteristics y1, y2, and y3 are, for example, the sugar content, moisture content, color, and saturation of the agricultural product (e.g., fruit).

[0021] In generating the estimation model, the computer 10 is supplied with spectral data (xt1, xt2, ...) as training data obtained from the spectroscopic measuring device 20, and generates the estimation model. After the estimation model is generated, if new spectral data (x1, x2, ...) as a new set of data (a set of data not used to generate the estimation model) is obtained from the spectroscopic measuring device 20, this spectral data is input into the estimation model, and various characteristics y1, y2, y3, ... are output.

[0022] Computer 10 uses statistical causal inference techniques to determine the reliability of the generated estimation model. Based on the resulting reliability, the applicability of the estimation model to new data sets x1, x2, ... that were not used in its generation is determined. Furthermore, the estimation model is updated as appropriate based on the reliability evaluation, thereby potentially improving its applicability to the new data sets.

[0023] Computer 10 is connected to server 40 and database 50 via a network NW, and receives various data from server 40 and database 50 via the network NW when generating and updating estimation models. Computer 10 also transmits data related to the generated estimation model and the training data used for its generation to server 40 via the network NW. Furthermore, when determining the reliability of the estimation model, computer 10 receives various data for that determination from server 40 and database 50. In addition, computer 10 can supply data related to the reliability determination results of the estimation model to server 40 via the network NW.

[0024] Referring to the block diagram in Figure 2, an example of the configuration of the computer 10 will be described. As shown in Figure 2, the computer 10 is configured, for example, to include a CPU (Central Processing Unit) 101, ROM 102, RAM 103, flash memory 104, input / output control unit 105, and communication control unit 106.

[0025] The CPU 101 is a central control unit responsible for various calculations, including calculations for generating and updating estimation models, and calculations for output data using the estimation models. In addition to the CPU 101, a GPU (Graphics Processing Unit) may be provided as a control unit for performing image processing and image recognition.

[0026] ROM 102 is a storage device that stores programs for various calculations and various data necessary to execute these programs. RAM 103 is a storage device that temporarily stores the calculation results of the programs. Flash memory 104 is a storage medium that stores programs read from ROM 102 and also stores various update programs provided by server 40.

[0027] The input / output control unit 105 controls data input from the server 40 and other devices, and data output to the server 40 and other devices. The communication control unit 106 controls the transmission and reception of data exchanged with the server 40.

[0028] Computer 10 virtually implements an estimation model generation / update unit 112, a composite variable generation unit 113, a causal inference unit 114, and a new data set applicability determination unit 115 internally using a computer program stored in ROM 102 or the like. This computer program generates / updates the estimation model within the computer and executes a method for determining its reliability.

[0029] The estimation model generation / update unit 112 generates an estimation model according to the training data and updates the estimation model according to the results of calculations performed by the causal inference unit 114. For example, the generation and updating of the estimation model can be performed using linear regression analysis. The composite variable generation unit 113 has the function of generating composite variables Sy1, Sy2, etc. by combining the explanatory variables xt1, xt2, etc. of the training data as input data. As will be described later, if there are multiple explanatory variables xt1, xt2, etc., the composite variable generation unit 113 appropriately selects and combines these multiple explanatory variables to generate multiple composite variables.

[0030] The causal inference unit 114 applies statistical causal inference to the explanatory variables (including composite variables) and the dependent variables y1, y2, y3, ... to estimate the direction of causality between the explanatory variables and the dependent variables y1, y2, y3, ... (which one is the cause and which one is changed). If there are multiple explanatory variables, these multiple explanatory variables are combined to create a composite variable in the composite variable generation unit 113, and this composite variable can be selected as an explanatory variable and used for statistical causal inference in the causal inference unit 114.

[0031] The causal inference unit 114 performs statistical causal inference calculations according to the dependent variables y1, y2, y3, ... and explanatory variables used in the estimation model calculation, and estimates whether the dependent variables y1, y2, y3, ... are the direct and sole causes of the explanatory variables.

[0032] When explanatory variables are acquired by passive sensors, the explanatory variables x1, x2, x3, ... cannot be the cause of the dependent variable or other unobserved variables (z1, z2, z3, ...). Furthermore, individual explanatory variables cannot cause each other. Therefore, the causal relationships between explanatory variables, dependent variables, and unobserved variables are limited to one of the eight patterns shown in Figure 15.

[0033] Here, "direct and sole cause" means that a dependent variable changes an independent variable without the mediation of other variables, and that there are no other variables that change the independent variable. In Figure 15, patterns 1 to 3 fall into this category. If, in a certain estimation model, the dependent variables y1, y2, y3, ... are determined to be the direct and sole cause of the independent variables, then it can be concluded that the estimation model is likely to be applicable to new data sets separate from the training data, as long as the relationship between the dependent variables y1, y2, y3, ... and the independent variables remains unchanged.

[0034] On the other hand, if, in a given estimation model, the dependent variables y1, y2, y3, ... are not the direct and sole cause of the explanatory variables, or if it is determined that there are other unobserved variables that cause either the dependent or explanatory variables, then it can be concluded that the estimation model is unlikely to be applicable to a new set of data separate from the training data.

[0035] Furthermore, if an estimation model determines that the explanatory variables are due to other causes, it can be concluded that the estimation model is unlikely to be applicable to a new dataset separate from the training data. In such cases, the estimation model cannot be considered to accurately reflect the real situation, and even if the estimation model is applied to the new dataset, it can be concluded that the dependent variable will not reflect the actual values.

[0036] The results of the calculations and estimations performed by the causal inference unit 114 are provided to the estimation model generation / update unit 112 and used to update the generated estimation model. The causal inference unit 114 can be constructed based on known models such as LiNGAM (Linear Non-Gaussian Acyclic Model).

[0037] The new data set applicability determination unit 115 determines, according to the results of the calculations performed by the causal inference unit 114, whether the estimation model generated using the training data is applicable to a new data set different from the training data. If the new data set is determined to be applicable, it is input into the estimation model as explanatory variables x1, x2, ..., the target variables y1, y2, y3, ... are calculated and output, and then output as measured values ​​by the measurement system 1.

[0038] Next, referring to the flowchart in Figure 3, the procedure for determining the reliability of the estimation model in the measurement system 1 of the first embodiment will be explained. Once the estimation model is generated based on the training data xt1, xt2, ... the reliability of the estimation model is determined. First, in step S11, it is determined whether or not there are multiple explanatory variables xt1, xt2, ... as training data (step S11).

[0039] If there are no multiple explanatory variables (only one explanatory variable exists), in step S15, statistical causal inference is applied to that explanatory variable and the dependent variables y1, y2, y3... in the causal inference unit 114 to estimate the direction of causality between the explanatory variable and the dependent variables y1, y2... (step S15).

[0040] On the other hand, if there are multiple explanatory variables xt1, xt2, etc., multiple composite variables Sy1, Sy2, etc. are generated by appropriately combining these multiple explanatory variables (step S12). The generation of composite functions can be performed using methods such as principal component analysis, factor analysis, t-distributed stochastic neighbor embedding (t-SNE), cluster analysis, non-negative matrix factorization (NMF), multivariate curve resolution (MCR), parallel factor analysis (PARAFAC), partial least squares regression analysis, and ensemble learning.

[0041] Then, from among these multiple composite variables Sy1, Sy2, ..., the composite variable Syx that can contribute most to the estimation of the dependent variables y1 and y2 is selected (Step S13). The selection of composite variables can be performed using a PLS regression model as an example. The selected composite variable Syx is then used as an explanatory variable, and statistical causal inference is applied to this explanatory variable and the dependent variables y1 and y2 to estimate the direction of causality (Step S14).

[0042] If, as a result of causal inference in step S14 or S15, it is determined that the dependent variables y1, y2, ... are the direct and sole causes of the explanatory variables (Y in step S16), then the estimation model in question is considered to have a high probability of being applicable to new data sets (step S17). On the other hand, if it is determined that the dependent variables y1, y2, ... are not the direct and sole causes of the explanatory variables (N in step S16), then the estimation model in question is considered to have a low probability of being applicable to new data sets (step S18).

[0043] An example of the determination in step S16 will be explained with reference to Figure 4. For example, as shown in Figure 4(a), if data is generated in which the dependent variable y1 is the direct and sole cause of a certain explanatory variable (e.g., absorbance around 700 nm), and further partial least square (PLS) regression analysis is performed to generate a composite variable of the explanatory variables, then calculating the likelihood between the composite variable of the explanatory variables and the dependent variable using the Pairwise LiNGAM algorithm (Non-Patent Literature 7) will yield a heatmap like the one shown in Figure 16. If the value in row i and column j on the heatmap is positive, the direction of causality is determined to be from i to j, and if it is negative, it is determined to be from j to i. Therefore, in the case of Figure 16, the direction of causality is from the dependent variable y1 to the composite variable 1, and it can be determined that y1 is the direct and sole cause of the composite variable 1, and the estimation model for the dependent variable y1 can be determined to have a certain level of reliability. On the other hand, as shown in Figure 4(b), if the unobserved variable z is the direct cause of a certain explanatory variable (e.g., absorbance around 520 nm), and the dependent variable y2 changes due to the unobserved variable z, but not as the cause of the explanatory variable, then Pairwise LiNGAM yields the likelihood heatmap shown in Figure 17, and it is determined that the dependent variable y2 is not the direct and sole cause of the composite variable. Therefore, with respect to the dependent variable y2, it can be concluded that the reliability of the estimation model is low.

[0044] Figure 5 is a graph showing the relationship between measured values ​​and estimated values ​​from the estimation model for the dependent variable y1, for which a direct causal relationship was found, and for the dependent variable y2, for which a direct causal relationship was not found. For the dependent variable y1, for which a direct causal relationship was found, a certain relationship was observed between the measured values ​​and the estimated values, and the root mean square error (RMSEnew) of the dependent variable y1 obtained by inputting a new data set into the estimation model did not differ significantly from the root mean square error (RMSEtrain) of the dependent variable y1 obtained by inputting the training data.

[0045] On the other hand, for the dependent variable y2, for which no direct causal relationship is recognized and it is considered unlikely that the dependent variable is the cause of the aforementioned explanatory variable, the root mean square error (RMSEnew) of the dependent variable y2 obtained by inputting a new data set into the estimation model is significantly larger than the root mean square error (RMSEtrain) of the dependent variable y2 obtained by inputting the training data. This means that the estimation model for the dependent variable y2 cannot be applied to the new data set.

[0046] Another example of the determination in step S16 will be explained with reference to Figure 6. For example, as shown in Figure 6(a), this is the case when it is determined that the dependent variable y1 is the direct and sole cause of a certain explanatory variable (in Figure 6(a), the absorbance around 700 nm), which is similar to Figure 4(a).

[0047] On the other hand, in Figure 6(b), it is determined that the unobserved variable z is the cause of a certain explanatory variable (absorbance around 520 nm in Figure 4(b)), while the dependent variable y3 changes due to the unobserved variable z, and furthermore, the dependent variable y3 causes a change in a certain explanatory variable. In such cases, Pairwise LiNGAM yields the likelihood heatmap shown in Figure 18, and it is determined that the dependent variable y3 is not the direct and sole cause of the explanatory variable. Therefore, the estimation model for the dependent variable y3 is also considered unreliable and unlikely to be applicable to new data sets. Thus, according to this system 1, the reliability of the estimation model can be determined based on the results of inferring the direct causal relationship between the dependent variables y1, y2, y3... and the explanatory variables.

[0048] Figure 7 is a graph showing the relationship between measured values ​​and estimated values ​​from the estimation model for the dependent variable y1, for which a direct causal relationship was found, and for the dependent variable y3, for which a direct causal relationship was not found. For the dependent variable y1, for which a direct causal relationship was found, a certain relationship was observed between the measured values ​​and the estimated values, and the root mean square error (RMSEnew) of the dependent variable y1 obtained by inputting a new data set into the estimation model did not show a significant difference compared to the root mean square error (RMSEtrain) of the dependent variable y1 obtained by inputting the training data.

[0049] On the other hand, for the dependent variable y3, where no direct causal relationship is found and the explanatory variable is considered to be the cause of the dependent variable, the root mean square error (RMSEnew) of the dependent variable y3 obtained by inputting a new data set into the estimation model is significantly larger than the root mean square error (RMSEtrain) of the dependent variable y3 obtained by inputting the training data. This means that the estimation model for the dependent variable y3 cannot be applied to the new data set.

[0050] As explained above, according to the measurement system 1, the method for determining the reliability of the estimation model, and the program according to the first embodiment, the reliability of the estimation model generated using training data (the possibility that the estimation model can be applied to new data sets different from the training data) is determined according to the direction of causality based on statistical causal inference, making it possible to determine the reliability without inputting the new data set into the estimation model for verification.

[0051] [Second Embodiment] Next, the measurement system 1 of the second embodiment will be described with reference to Figures 8 and 9. The overall configuration of the measurement system 1 of this second embodiment may be the same as that of the first embodiment, so redundant explanations will be omitted. Figure 8 is a configuration diagram illustrating the configuration of the computer 10 of the second embodiment. Figure 9 is a flowchart illustrating the operation of the second embodiment. In addition to the configuration of the first embodiment, the computer 10 of the second embodiment is equipped with a variable batch deletion processing unit 116 that batch deletes explanatory variables that are to be deleted. The other configurations are the same as those of the first embodiment (Figure 2), so redundant explanations will be omitted.

[0052] The variable batch deletion processing unit 116 has the function of batch deleting predetermined explanatory variables from the estimation model according to the results of the inference in the causal inference unit 114. That is, as shown in Figure 9, sequential statistical causal inference is performed on each explanatory variable and multiple dependent variables y1, y2, y3... to estimate the direction of causality (step S21), and after obtaining reliability data, explanatory variables that are judged to be caused by the dependent variable are retained, while explanatory variables that are judged not to be caused by the dependent variable are deleted from the estimation model (step S22). The explanatory variables that remain after deletion are adopted as the final explanatory variables for constructing the estimation model (step S23).

[0053] Figures 10(a) and (b) are graphs showing the relationship between the measured value and the estimated value from the estimation model for the dependent variable y3, for which no direct causal relationship was found, before and after the removal of the explanatory variable.

[0054] As shown in Figure 10(a), before removing the explanatory variables, the root mean square error (RMSEnew) of the dependent variable y3 obtained by inputting the new data set into the estimation model is significantly larger than the root mean square error (RMSEtrain) of the dependent variable y3 obtained by inputting the training data. This means that the estimation model for the dependent variable y3 before removing the explanatory functions cannot be applied to the new data set.

[0055] On the other hand, as shown in Figure 10(b), after removing the explanatory variables, the root mean square error (RMSEnew) of the dependent variable y3 obtained by inputting the new data set into the estimation model does not show a significant difference compared to the root mean square error (RMSEtrain) of the dependent variable y3 obtained by inputting the training data. This means that the reliability of the estimation model has improved by removing explanatory variables that are not caused by the dependent variable, and the applicability of the estimation model has increased even for new data sets that were judged to have a low probability of being applicable before removal.

[0056] As described above, this second embodiment can achieve the same effects as the first embodiment, and furthermore, the reliability of the estimation model can be improved by deleting explanatory variables according to the information on the direction of causality.

[0057] [Third Embodiment] Next, the measurement system 1 of the third embodiment will be described with reference to Figures 11 and 12. The overall configuration of the measurement system 1 of this third embodiment may be the same as that of the first embodiment, so redundant explanations will be omitted. Figure 11 is a configuration diagram illustrating the configuration of the computer 10 of the third embodiment. Figure 12 is a flowchart illustrating the operation of the third embodiment. In addition to the configuration of the first embodiment, the computer 10 of the third embodiment is equipped with a variable sequential deletion processing unit 117 that sequentially deletes explanatory variables that are to be deleted. The other configurations are the same as those of the first embodiment (Figure 2), so redundant explanations will be omitted.

[0058] The variable sequential deletion processing unit 117 has the function of sequentially deleting predetermined explanatory variables from the estimated model according to the results of the inference in the causal inference unit 114. That is, as shown in Figure 12, the variable sequential deletion processing unit 117 first determines the number of explanatory variables (k) to be ultimately reduced (step S31), and then, similar to the embodiment described above, generates an estimated model according to the training data and records its cross-validation error and the explanatory variables that remain at this point (step S32).

[0059] Next, similar to the embodiment described above, statistical causal inference is applied to each explanatory variable and the dependent variable to estimate the direction of causality between the explanatory variables and the dependent variable (step S33), and the explanatory variable with the lowest probability of being caused by the dependent variable is removed from the estimation model (step S34). Steps S32 to S34 are repeated until the number of removed explanatory variables reaches k (step S35). Once the number of reduced explanatory variables reaches the target of k, the explanatory variables that remained at the point when the cross-validation error was minimized are adopted as the final explanatory variables for constructing the estimation model. It should be noted that whether or not the number of removed explanatory variables has reached k is just one example of a termination condition, and it goes without saying that other conditions can be adopted.

[0060] As explained above, this third embodiment provides the same effects as the first embodiment, and furthermore, the reliability of the estimation model can be improved by sequentially deleting explanatory variables according to the information on the direction of causality.

[0061] (Example 1) Figure 13 shows an example of measurement results when a tablet is used as the measurement target in the measurement system 1 of Figure 1, its near-infrared spectral data is acquired, and the active ingredient content of the tablet is measured by applying an estimation model. Figure 13(a) shows the distribution of the measured value of the active ingredient content and the estimated value by the estimation model when a method of sequentially deleting explanatory variables is adopted, as in the third embodiment. On the other hand, Figure 13(b) shows the distribution of measured and estimated values ​​according to the conventional technology. According to this embodiment, even with a new data set, the variability between the measured and estimated values ​​is small, and it can be seen that the reliability of the estimation model has improved.

[0062] (Example 2) Figure 14 shows an example of measurement results when this embodiment is applied to measurement data from a gas sensor in a gas chamber. Figure 14(a) shows the distribution of measured and estimated values ​​according to the conventional technology. On the other hand, Figure 14(b) shows the distribution of measured ethylene concentration in the gas and estimated values ​​from the estimation model when a method of sequentially removing explanatory variables is adopted, as in the third embodiment. According to this embodiment, even with a new data set, the variability between measured and estimated values ​​is small, and the reliability of the estimation model is improved.

[0063] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are possible. For example, the embodiments described above are explained in detail to make the present invention easier to understand, and the present invention is not necessarily limited to embodiments having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add configurations from other embodiments to the configuration of one embodiment. Furthermore, it is possible to delete parts of the configuration of each embodiment, or to add or replace other configurations. [Explanation of Symbols]

[0064] 10...Computer, 20...Spectroscopic measuring device, NW...Network, 40...Server, 50...Database, 101...CPU, 102...ROM, 103...RAM, 104...Flash memory, 105...Input / Output control unit, 106...Communication control unit, 112...Estimation model generation / update unit, 113...Composite variable generation unit, 114...Causal inference unit, 115...New data group applicability determination unit, 116...Batch variable deletion processing unit, 117...Sequential variable deletion processing unit.

Claims

1. A method for determining the reliability of an estimation model that estimates a dependent variable from explanatory variables using a computer, wherein the computer: The steps include: applying statistical causal inference to the explanatory variable and the dependent variable to estimate the direction of causality between the explanatory variable and the dependent variable; The steps include determining whether the dependent variable is the direct and sole cause of the independent variable, The step of determining, in accordance with the result of the above determination, whether or not the estimation model is applicable to new data other than the data used to construct the estimation model. Equipped with, In the step of determining whether or not there is a direct and sole cause, the determination of whether or not there is a direct and sole cause is made by using an algorithm that calculates the likelihood of the causal relationship between the dependent variable and the independent variable. A method for determining the reliability of an estimation model, characterized by the features described above.

2. A method for determining the reliability of an estimation model according to claim 1, further comprising the step of removing an explanatory variable from the estimation model if it is determined that the dependent variable is unlikely to be the cause of the explanatory variable in the step of estimating the direction of causality.

3. The method for determining the reliability of an estimation model according to claim 2, wherein the step of deleting the explanatory variables is to delete all explanatory variables that are determined to be unlikely to be the cause of the dependent variable.

4. The method for determining the reliability of an estimation model according to claim 2, wherein the step of deleting the explanatory variables is to sequentially delete the explanatory variables that are judged to be unlikely to be the cause of the dependent variable, and to stop the deletion step when a predetermined condition is met.

5. The steps include: appropriately selecting and combining multiple explanatory variables to generate multiple composite variables; The step of selecting one composite variable from the aforementioned multiple composite variables and using it as the explanatory variable. A method for determining the reliability of an estimation model according to claim 1, comprising:

6. A method for determining the reliability of an estimation model according to claim 5, wherein the step of generating the plurality of composite variables includes principal component analysis, factor analysis, t-distributed stochastic neighbor embedding (t-SNE), cluster analysis, non-negative matrix factorization (NMF), multivariate curve resolution (MCR), parallel factor analysis (PARAFAC), partial least squares regression analysis, or ensemble learning.

7. The method for determining the reliability of the estimation model according to claim 1, wherein the step of estimating the direction of causality uses LiNGAM (Linear Non-Gaussian Acyclic Model).

8. In a computer program for determining the reliability of an estimation model that estimates the dependent variable from the explanatory variables, The steps include: applying statistical causal inference to the explanatory variable and the dependent variable to estimate the direction of causality between the explanatory variable and the dependent variable; The steps include determining whether the dependent variable is the direct and sole cause of the independent variable, The step of determining, in accordance with the result of the above determination, whether or not the estimation model is applicable to new data other than the data used to construct the estimation model. It is configured to have a computer execute it, In the step of determining whether or not there is a direct and sole cause, the determination of whether or not there is a direct and sole cause is made by using an algorithm that calculates the likelihood of the causal relationship between the dependent variable and the independent variable. A computer program for determining the reliability of an estimation model.

9. A measuring device that measures the object to be measured and acquires measurement data as explanatory variables, A computer applies the aforementioned explanatory variables to an estimation model to output the target variable and calculates various characteristics of the object being measured. A measurement system equipped with, The computer includes a computer program for determining the reliability of the estimation model. The aforementioned computer program, The steps include: applying statistical causal inference to the explanatory variable and the dependent variable to estimate the direction of causality between the explanatory variable and the dependent variable; The steps include determining whether the dependent variable is the direct and sole cause of the independent variable, The step of determining, in accordance with the result of the above determination, whether or not the estimation model is applicable to new data other than the data used to construct the estimation model. It is configured to have a computer execute it, In the step of determining whether or not there is a direct and sole cause, the determination of whether or not there is a direct and sole cause is made by using an algorithm that calculates the likelihood of the causal relationship between the dependent variable and the independent variable. Measurement system.

Images