Information processing device, information processing method, and program

Abstract

This information processing device (100) comprises a control unit (130). On the basis of a causal model related to the causal relationship between a plurality of input variables in a prediction model using machine learning and a prediction result, the control unit (130) selects an input variable that influences the prediction result as a first explanatory variable. The control unit (130) outputs information related to the selected first explanatory variable.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, there has been a demand for improved data analysis techniques to extract useful information from data such as big data. Analysis results are used to understand phenomena that have already occurred, predict the future, control, or intervene. For example, there is a known technique for analyzing predictive models and outputting variables that contribute significantly to the predicted results. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] MTRibeiro, S.Singh, C.Guestrin, “Why should I trust you? Explaining the Predictions of Any Classifier”, the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.1135-1144, 2016. [Non-patent document 2] SMLundberg and SILee, “A Unified Approach to Interpreting Model Predictions”, Advances in Neural Information Processing Systems 30”, pp.4765-4774, 2017. Summary of the Invention [Problem to be solved by the invention]

[0004] The above technique performs analysis assuming that the relationship between the prediction result and the variables is linear. Furthermore, the technique adds up the contributions of multiple variables to output multiple variables as a composite reason for the prediction result.

[0005] The above technology has room for improvement in terms of outputting variables suitable as reasons. In particular, when outputting a composite reason, the above technology simply adds up the contributions of multiple variables and does not consider the dependency of the multiple variables on the combination patterns of categories. For example, assume that a variable called "gender" includes categories of "male" and "female," and a variable called "age" includes categories of "teens," "twenties," "thirties," "forties," and "over fifties." The contributions of a combination of variables, such as "female" in "teens" and "male" in "thirties," to a prediction may differ significantly. Thus, it is desirable to output a composite reason that is more suitable for a prediction result by taking into account the contributions of a combination pattern of categories of multiple variables to a prediction.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that can output variables that are more suitable as reasons when multiple reasons for a prediction result are output.

[0007] It should be noted that the above problem or object is merely one of multiple problems or objects that can be solved or achieved by multiple embodiments disclosed in this specification. [Means for solving the problem]

[0008] According to the present disclosure, an information processing device is provided. The information processing device includes a control unit. The control unit selects, as a first explanatory variable, an input variable that influences a prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning. The control unit outputs information regarding the selected first explanatory variable. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram illustrating an example of a logical configuration of an information processing device according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram for explaining an example of a prediction model acquired by a causal model estimation unit according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a diagram for explaining an example of a causal model estimated by a causal model estimation unit according to an embodiment of the present disclosure. [Figure 4] 10 is a flowchart illustrating a flow of a prediction model analysis process executed by an information processing device according to an embodiment of the present disclosure. [Figure 5] 10A and 10B are diagrams for explaining an example of a causal model estimated by an information processing device according to an application example of an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating an example of a UI screen output by an information processing device according to an application example of an embodiment of the present disclosure. [Figure 7] FIG. 10 is a diagram illustrating an example of a UI screen output by an information processing device according to an application example of an embodiment of the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of a UI screen output by an information processing device according to an application example of an embodiment of the present disclosure. [Figure 9] FIG. 10 is a diagram illustrating an example of a UI screen output by an information processing device according to an application example of an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram for explaining the causal relationship of the prediction model of the recidivism assessment system. [Figure 11] 10 is a diagram showing the conditional probability of the level of the nearest node calculated by the information processing device according to the embodiment of the present disclosure. [Figure 12] FIG. 10 is a diagram illustrating an example of a UI screen output by an information processing device according to a modified example of an embodiment of the present disclosure. [Figure 13] FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] One or more embodiments (including examples and modifications) described below can be implemented independently. However, at least a portion of the embodiments described below may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from one another. Therefore, these embodiments may contribute to solving different purposes or problems and may produce different effects.

[0012] The explanation will be given in the following order. 1. Overview of the proposed technology 2. Example of information processing device configuration 3.Analysis Processing 4. Application Examples 4.1. Example of application to a manufacturing data analysis system 4.2. Example of application to a recidivism assessment system 5. Other embodiments 6. Variations 7. Hardware configuration example 8. Summary

[0013] <<1. Overview of the proposed technology>> In recent years, with the improvement of machine learning performance, machine learning-based predictive models (predictors) have come to be used in a variety of situations. However, in many cases, the prediction results output by predictive models are not provided with any basis, which can make them difficult for users to use. As a result, there is a risk that the use of predictive models will remain limited to a limited range.

[0014] In this way, in order to promote the use of prediction models, it is desirable for the predictor to present the basis (reason) for its prediction along with the prediction results.

[0015] In response to this demand, for example, many techniques have been proposed that automatically output input variables that are considered important as reasons for prediction (hereinafter referred to as important variables) among the input variables of a prediction model.

[0016] However, the above technology outputs important variables calculated based on correlation patterns with the output variables, which are the prediction results, and does not take into account a causal perspective. Therefore, the possibility that the important variables output as the reason for the prediction and the output variables may be spuriously correlated cannot be eliminated, and it cannot be said that an appropriate reason is output.

[0017] If the output reasons (important variables) are inappropriate, interventions and measures based on the reasons may not improve the prediction results. Furthermore, because humans tend to interpret correlations as causal, there is a risk that users will interpret the reasons presented based on correlations as causal and learn incorrect knowledge. Therefore, technology that can output more appropriate variables as reasons for prediction results is desirable.

[0018] Therefore, in the technology disclosed herein, an information processing device selects input variables that affect a prediction result as important variables (examples of first explanatory variables) based on a causal model regarding the causal relationships between multiple input variables and predictor variables in a prediction model using machine learning. The information processing device outputs information regarding the selected important variables.

[0019] An information processing device outputs input variables (important variables) that are the reason for a prediction result based on information on whether or not a spurious correlation exists in a prediction model generated using machine learning. For example, an information processing device according to the technology of the present disclosure selects, from the input variables, an input variable that is not in a spurious correlation relationship based on a spurious correlation relationship between the input variables and the output variables of a prediction model generated by machine learning, as an important variable that is the reason for the prediction result of the prediction model. The information processing device outputs information about the selected important variables.

[0020] For the above selection, for example, the information processing device determines whether each input variable is conditionally independent with respect to the output variable, and selects the input variables that are determined not to be conditionally independent as important variables.

[0021] More specifically, the information processing device constructs a causal model in which a variable of interest (e.g., an output variable, or in the neural network described below, a node in the hidden layer immediately before the output variable) is used as the objective variable and input variables are used as explanatory variables.

[0022] The information processing device selects important variables from the input variables included in the constructed causal model, and selects input variables that are determined not to be conditionally independent as important variables.

[0023] The information processing device selects input variables that do not affect the prediction results as second important variables (an example of second explanatory variables) based on the constructed causal model. The information processing device outputs information about the selected second important variables, distinguishing them from the above-mentioned important variables.

[0024] This allows the information processing device to select important variables taking into consideration causality, particularly spurious correlation, and to present to the user important variables that are more appropriate as reasons for the prediction results.

[0025] <<2. Configuration example of information processing device>> 1 is a block diagram showing an example of a logical configuration of an information processing device 100 according to an embodiment of the present disclosure. As shown in FIG. 1, the information processing device 100 according to the present embodiment includes an input unit 110, an output unit 120, and a control unit 130.

[0026] [Input section 110] The input unit 110 has a function of accepting input of various information, and outputs the input information to the control unit 130.

[0027] The input unit 110 receives an input of a prediction model to be analyzed. The prediction model here is a machine learning device generated by machine learning, such as a neural network or a random forest.

[0028] The input unit 110 accepts various inputs for analysis. For example, the input unit 110 accepts inputs such as the number n of explanatory variables included in the combination of explanatory variables described below. The input unit 110 may include any input device such as a touch panel, keyboard, or mouse, and accepts input of this information through user operation of the input device. The input unit 110 may also include an audio input unit such as a microphone.

[0029] [Output section 120] The output unit 120 has a function of outputting various information. The output unit 120 outputs various information based on the control of the control unit 130. The output unit 120 includes a display device and may output, for example, a UI screen showing the analysis results. The output unit 120 includes an audio output device and may output, for example, audio to prompt the user for voice input. The output unit 120 includes a communication device conforming to any communication standard and may output, for example, information showing the analysis results to another information processing device or storage device. The output unit 120 includes a printing device and may, for example, print a report showing the analysis results.

[0030] [Control unit 130] The control unit 130 has a function of controlling the overall operation of the information processing device 100. As shown in Fig. 1, the control unit 130 includes an input / output control unit 131, a causal model estimation unit 132, a selection unit 133, an evaluation unit 134, and an intervention effect calculation unit 135. The control unit 130 may include components other than these components.

[0031] (Input / output control unit 131) The input / output control unit 131 is an interface between the input unit 110 and the output unit 120 and other components included in the control unit 130 (the causal model estimation unit 132, the selection unit 133, the evaluation unit 134, and the intervention effect calculation unit 135). The input / output control unit 131 outputs information input by the input unit 110 to other components corresponding to the information, and causes the output unit 120 to output information output from other components. For example, the input / output control unit 131 outputs a prediction model input by the input unit 110 to the causal model estimation unit 132. The input / output control unit 131 also generates a UI screen showing the analysis results and causes the output unit 120 to output it, and outputs information indicating a user operation on the UI screen input by the input unit 110 to other components corresponding to the user operation.

[0032] (Causal model estimation unit 132) The causal model estimation unit 132 generates a causal model using the variable of interest included in the prediction model acquired via the input / output control unit 131 as a response variable.

[0033] The prediction model may be acquired from, for example, a storage device (not shown) or from a storage unit (not shown) of the information processing device 100. Alternatively, the information processing device 100 may generate the prediction model. The prediction model is, for example, a learning device generated based on machine learning. The prediction model may be any model that performs a predetermined calculation on input variables and outputs output variables that are prediction results. Examples of prediction models include neural networks and random forests, but the following description will be given assuming that the information processing device 100 has acquired a neural network as the prediction model unless otherwise specified.

[0034] In neural networks, information from groups of input variables is combined in complex ways to produce output such as prediction, identification, and classification. In the hidden layer of a neural network, information from groups of input variables is combined in various ways to form patterns, but because these patterns are complex, it is difficult for users to understand how the output variables are determined.

[0035] Therefore, in the technology of the present disclosure, attention is focused on the hidden layer (hereinafter also referred to as the nearest layer) closest to the output layer (output variable), and the information processing device 100 presents to the user information indicating what combination of input variable sets constitutes this node group (hereinafter also referred to as the nearest node group). In other words, the information processing device 100 focuses on the nearest layer immediately before the output layer, and generates information for explaining each nearest node with a set of input variables.

[0036] To this end, first, the causal model estimation unit 132 estimates a causal model for explaining the feature quantity acquired by learning for each of the nearest neighbor nodes. This point will be described with reference to FIGS. 2 and 3.

[0037] FIG. 2 is a diagram illustrating an example of a prediction model acquired by the causal model estimation unit 132 according to an embodiment of the present disclosure.

[0038] 2, the causal model estimation unit 132 acquires, as a prediction model, a neural network consisting of an input layer, two hidden layers HL1 and HL2, and an output layer. Here, the neural network outputs one output variable Y for a set of input variables {X1, X2, X3, X4}, and each of the hidden layers HL1 and HL2 has five hidden nodes. Furthermore, the hidden layer HL2, which is the nearest layer, has hidden nodes L1 to L5, which are nearest nodes.

[0039] The causal model estimation unit 132 estimates a causal model for each variable of interest included in the prediction model (here, the closest nodes L1 to L5) as the objective variable and the set of input variables {X1, X2, X3, X4, ...} as the explanatory variables.

[0040] For example, the causal model estimation unit 132 estimates a causal model using N input variables Xn (n=1 to 4 in FIG. 2) for each of M nearest nodes Lm (m=1 to 5 in FIG. 2). That is, the causal model estimation unit 132 generates M causal models.

[0041] The causal model estimation unit 132 detects the presence or absence and strength of a causal relationship between each variable by calculating causal information, which is an index of the causal relationship between the nearest node Lm and each input variable Xn. The causal information is, for example, a statistical quantity indicating the causal relationship between two variables. The causal model estimation unit 132 detects the presence or absence and strength of a causal relationship between two variables by a method such as independent component analysis, a method of maximizing an estimation result obtained by a penalized maximum likelihood method or a Bayesian method as a score, or a method of estimation by a statistical test of conditional independence between variables. The causal model estimation unit 132 calculates the causal information using, for example, training data used to generate the neural network.

[0042] FIG. 3 is a diagram for explaining an example of a causal model estimated by the causal model estimation unit 132 according to an embodiment of the present disclosure.

[0043] In FIG. 3, the causal model estimated by the causal model estimation unit 132 is shown as causal information indicating the causal relationships between multiple variables, a so-called causal graph. As shown in FIG. 3, a causal graph is estimated for each of the hidden nodes L1 to L5, which are the objective variables. In FIG. 3, the direction of causality of transformation for the hidden nodes L1 and L5 and the explanatory variables Xn is indicated by arrows (cause → effect). In other words, the causal graph shown in FIG. 3 is a directed graph. In addition, the causal information shown in FIG. 3 is, in other words, information on a graphical model accompanied by a probability distribution in which probabilistic and statistical cause and effect variables are connected by arrows.

[0044] The causal graph shown in FIG. 3 is an example of causal information, and the causal information may be information that lists the causal relationships between variables, or may be information that allows the causal relationships between multiple variables to be grasped.

[0045] Of the input variables included in the causal model estimated by the causal model estimation unit 132, the fewer the paths (number of arrows) leading to the hidden node Lm, which is the objective variable, and the closer the input variable is to the hidden node Lm, the stronger the causal relationship with the objective variable. Variables directly connected to the hidden node Lm on the graph are variables that are causes that have a direct influence on Lm rather than an indirect cause, and so may be preferentially extracted as important variables.

[0046] For example, in the causal model of hidden node L1 shown in FIG. 3 , there is no arrow pointing from input variable X10 to hidden node L1. This indicates that input variable X10 has no causal relationship with hidden node L1, but is conditionally independent, and thus has a spurious correlation. Note that the contribution and correlation of input variable X10 calculated using conventional methods can be large, making it difficult to determine a spurious correlation from the magnitude of the contribution or correlation, as with conventional methods. Input variable X3 is connected to hidden node L1 by a single arrow. That is, input variable X3 is directly connected to hidden node L1. On the other hand, input variable X1 is connected to hidden node L1 by two arrows via input variable X4. That is, input variable X1 is not directly connected to hidden node L1. This indicates that input variable X3 has a stronger direct causal relationship with hidden node L1 than input variable X1. Therefore, for example, information processing device 100 can prioritize extracting input variable X3 as an important variable without extracting input variable X1. As will be described later, this makes it possible to eliminate redundant explanatory variables, narrow down the number of combinations of important variables, and improve calculation efficiency, and to reduce the number of similar explanatory variables, making it possible to present the reasoning behind predictions in an easy-to-understand manner to the user.

[0047] The causal model estimated by the causal model estimation unit 132 may include input variables that have a causal relationship with the hidden node Lm. In other words, input variables that are not conditionally independent of the hidden node Lm are included in the causal model, and input variables that are conditionally independent are not included in the causal model. In this way, the causal model estimation unit 132 can generate a causal model for the hidden node Lm using input variables that are not spuriously correlated.

[0048] (Selection unit 133) Returning to FIG. 1, the selection unit 133 selects variables to explain the hidden node Lm from among the input variables based on the causal model estimated by the causal model estimation unit 132. Such variables have a high degree of contribution to the calculation result at the hidden node Lm, and can be said to be variables that contribute to the calculation result. For this reason, such variables are also referred to as important variables.

[0049] The selection unit 133 extracts, from the causal model of each hidden node Lm, a set of factors (input variables) {Xmj} that are directly connected to the hidden node Lm through the network of the causal model. As shown in equation (1), Xmj is a subset of the set of variables DI(Lm) and indicates the input variables Xj that belong to the causal model of the hidden node Lm.

[0050]

number

[0051] The selection unit 133 may select all factors directly connected to the hidden node Lm as important variables, or may select only some of the factors. For example, if the number of factor groups (important variable candidates) {Xmj} directly connected to the hidden node Lm is equal to or less than a threshold TH1, the selection unit 133 selects all of the factor groups as important variables. On the other hand, if the number of factor groups {Xmj} is greater than the threshold TH1, the selection unit 133 selects S input variables as important variables from the important variable candidates according to the strength of the relationship between the important variable candidates and the objective variable.

[0052] Alternatively, the selection unit 133 may select S combinations of P input variables according to the strength of the relationship between candidate combinations of P input variables in the factor set {Xmj} and the objective variable. The selection unit 133 calculates first information indicating the strength of the relationship between the P input variables and the objective variable, and selects each input variable included in the S combinations with the largest first information as an important variable.

[0053] The first information is an index indicating the strength of the relationship between the combination of important variable candidates and the objective variable. The first information is a value indicating the strength of the relationship, calculated based on entropy, mutual information, correlation coefficient, partial correlation coefficient, p-value of a test, other statistical measures of independence or conditional independence, or a combination thereof. The coupling effect is quantified by this value. A large value indicates a large coupling effect, and a small value indicates a small coupling effect.

[0054] Here, multiple input variables may have an effect on a single variable of interest that differs from the effect of each individual input variable. A variable of interest is a variable that is the subject of analysis, and in this case refers to the hidden node Lm. Such an effect is called a joint effect.

[0055] Hereinafter, the first information is also referred to as a first index J of the coupling effect. When the objective variable (variable of interest) is L and the important variable candidates are Xn1 and Xn2 (in other words, P=2), the first index J for Xn1, Xn2, and L can be calculated as shown in formula (3).

[0056] J=H(L|Xn1)-H(L|Xn1,Xn2) =I(L;Xn2|Xn1) …(3)

[0057] Here, H is the information entropy in information theory, and I is the mutual information defined based on information entropy. In the above formula (3), H(L|Xn1) is the conditional information entropy of L conditioned on Xn1, and H(L|Xn1,Xn2) is the conditional information entropy of L conditioned on Xn1 and Xn2. Furthermore, I(L;Xn2|Xn1) is the mutual information between L and Xn2 conditioned on Xn1, and the above formula (3) is identically established (hereinafter, information entropy will be simply referred to as entropy). According to the above formula (3), the first index J of the combined effect is an index that indicates how much information is increased by the newly added explanatory variable Xn2 compared to the correlation value between the two variables, the target variable L and the explanatory variable Xn1. As shown in the above formula (3), in the case of a combination of two important variable candidates, the first index J of the combined effect is the conditional mutual information.

[0058] The first index J may be calculated as shown in equation (4).

[0059] J=I(L;Xn2|Xn1)-I(L;Xn2) …(4)

[0060] According to the above formula (4), the first index J of the coupling effect is the value obtained by subtracting the correlation between the two variables, the variable of interest L and the newly added explanatory variable Xn2, from the value shown in the above formula (3).

[0061] The first information may be calculated as shown in equations (5) to (7) using a second indicator ACMI (Averaged Conditional Mutual Information) instead of the first indicator J.

[0062]

number

[0063] As mentioned above, I(L;Xn2|Xn1) is the mutual information between L and Xn2 conditioned on Xn1. Also, I(L;Xn1|Xn2) is the mutual information between L and Xn1 conditioned on Xn2. The second index ACMI is the average of these conditional mutual information.

[0064] The second index ACMI indicates the average contribution of the important variable candidates Xn1 and Xn2 to the variable of interest L. The contribution of each instance can be expressed by the third index CE shown in equation (8).

[0065]

number

[0066] The second index ACMI of the combination of the three important variable candidates Xn1, Xn2, and Xn3 and the combined effect in the objective variable L can be calculated as shown in formulas (9) to (12).

[0067]

number

[0068] The selection unit 133 selects, as important variables, important variable candidates included in the S combinations in descending order of the calculated first information (for example, the second index ACMI).

[0069] As described above, the selection unit 133 calculates the first information by using the group of factors (input variables) {Xmj} that are directly connected to the hidden node Lm in the network of the causal model as important variable candidates.

[0070] This is based on the property of entropy expressed by formula (13) and the conditional independence property of the causal model (graphical model). That is, it is clear that the first information calculated from the combination of factor groups that are not directly connected to the hidden node Lm in the causal model network is smaller than the first information calculated from the combination of factor group {Xmj}.

[0071]

number

[0072] Therefore, the information processing device 100 can calculate the first information by using the group of factors (input variables) {Xmj} that are directly connected to the hidden node Lm in the causal model network as important variable candidates, thereby reducing the number of variables for calculating the first information.

[0073] (Evaluation section 134) The evaluation unit 134 evaluates the degree of influence that the important variable selected by the selection unit 133 has on the target variable (variable of interest). When the variable of interest is a hidden node Lm, the evaluation unit 134 may calculate the degree of influence that the important variable has on the hidden node Lm. Alternatively, the evaluation unit 134 may calculate the degree of influence that the important variable has on the output variable Y that is directly connected to the hidden node Lm. Such influence is an example of strength information that indicates the strength of the relationship between the variable of interest and the important variable.

[0074] If the set of factors {Xmj} directly connected to the hidden node Lm is treated as a numerical variable, the hidden node Lm can be expressed as a causal regression equation (structural equation) that takes partial correlation into account, as shown in equation (14).

[0075]

number

[0076] Note that αmj is calculated as a regression coefficient, and βm is a constant at hidden node Lm.

[0077] Alternatively, if the factor set {Xmj} is treated as a categorical variable, the hidden node Lm can be expressed as a conditional probability distribution as shown in equation (15), where ωmlm represents the neuron weight.

[0078]

number

[0079] The output variable Y is calculated based on equations (16) and (17). The filter function is included internally. The factor set {Xmj} is a numerical variable.

[0080]

number

[0081] The influence of the important variable Xj can be calculated as a weight for the important variable Xj. The weight for the important variable Xj can be expressed as shown in the following equation (18). As described above, αmj is calculated as a regression coefficient.

[0082]

number

[0083] Here, the evaluation unit 134 may calculate a composite influence. The composite influence corresponds to the above-mentioned combined effect. For example, the evaluation unit 134 calculates the composite influence of the important variables Xn1 and Xn2 based on the following equation (19):

[0084]

number

[0085] Here, CE(xn1, xn2) is the third index CE shown in equation (8), and nm(x) is the value of the hidden node Lm of the neural network, which is determined by all the input variables. Also, n0 is a constant. Equation (19) shows an example of the neural network shown in Figure 2, that is, when m = 1 to 5.

[0086] In addition to the above-described influence degree, the evaluation unit 134 may evaluate the degree of partial correlation between an important variable and a variable of interest as evidence that the important variable and the variable of interest are not spuriously correlated. For example, the evaluation unit 134 may investigate the partial correlation by calculating a p-value of a conditional independence test as the degree of partial correlation between the important variable and the variable of interest. That is, the evaluation unit 134 may calculate the p-value as the above-described strength information.

[0087] Alternatively, the evaluation unit 134 may calculate the conditional probability of the important variables of each of the groups of nearest neighbor nodes for each instance. This may be presented to the user as an analysis result of the neural network itself, rather than as a reason for the output (prediction) result. This point will be described later in the application example.

[0088] The important variables selected by the selection unit 133 and the influences calculated by the evaluation unit 134 are presented to the user by, for example, the input / output control unit 131. For example, the input / output control unit 131 presents the selected important variables and their influences in combination to the user. When there are multiple important variables, the input / output control unit 131 presents them to the user as a list in descending order of influence, for example. Furthermore, when presenting a combination of multiple important variables to the user as the reason for the prediction result, the input / output control unit 131 displays the combination of multiple important variables and the composite influence of the combination in association with each other to the user. Specific presentation examples will be described later in the application examples.

[0089] (Intervention Effect Calculation Section 135) The intervention effect calculation unit 135 calculates the intervention effect that occurs in the dependent variable by intervening in the important variable selected by the selection unit 133. The dependent variable here may be the closest node group or the output variable. The intervention effect calculation unit 135 can calculate the intervention effect of at least one of the closest node group and the output variable.

[0090] The intervention effect calculation unit 135 calculates the causal influence on the objective variable by intentionally changing the value of the important variable (by intervening in the important variable).

[0091] For example, the input / output control unit 131 presents the intervention effect calculated by the intervention effect calculation unit 135 to the user in association with the intervened important variable. The input / output control unit 131 presents the intervention effect to the user together with the evaluation result of the evaluation unit 134. Alternatively, the input / output control unit 131 may present the intervention effect to the user in response to an instruction from the user. In this case, for example, when the user selects an important variable presented by the input / output control unit 131, the intervention effect calculation unit 135 calculates the intervention effect of the selected important variable, and the input / output control unit 131 presents the calculation result.

[0092] Note that, although the intervention effect calculation unit 135 of the information processing device 100 calculates the intervention effect of the important variables here, this is not limiting. For example, the information processing device 100 may select and present important variables, and another device may calculate the intervention effect. In this way, when the information processing device 100 does not calculate the intervention effect, the intervention effect calculation unit 135 shown in FIG. 1 may be omitted.

[0093] <<3. Analysis Processing>> FIG. 4 is a flowchart showing the flow of the analysis process of the prediction model executed by the information processing device 100 according to the embodiment of the present disclosure.

[0094] As shown in FIG. 4, the causal model estimation unit 132 of the information processing device 100 acquires a prediction model to be analyzed (step S101).

[0095] Next, the causal model estimation unit 132 estimates a causal model for each node in the group of closest nodes (step S102).

[0096] The selection unit 133 selects important variables from the input variables that are closest to the variable of interest of each causal model estimated by the causal model estimation unit 132, in other words, that are directly connected in the network (step S103).

[0097] The evaluation unit 134 evaluates the influence of the important variables selected by the selection unit 133 on the variable of interest (step S104). At this time, the evaluation unit 134 can evaluate the combined influence (combined effect) of a combination of multiple important variables on the variable of interest.

[0098] The input / output control unit 131 outputs information about the important variable selected by the selection unit 133 (step S105). At this time, the input / output control unit 131 may output the information about the important variable in association with information about the influence evaluated by the evaluation unit 134 and information about the intervention effect when an intervention is performed on the important variable. The intervention effect is calculated by the intervention effect calculation unit 135.

[0099] <<4. Application Examples>> <4.1. Example of application to manufacturing data analysis system> In a manufacturing plant for a certain product, information is acquired by a large number of measuring instruments and sensors. For example, various physical quantities such as the outside temperature, room temperature, and humidity of the manufacturing plant, as well as voltage, current, power, and vibration waveforms that indicate the state of the equipment when it is used, are measured and stored, for example, in a database, linked to time. In this way, the large number of measuring instruments and sensors acquire information about the operating environment or operating state of the equipment.

[0100] Furthermore, the presence or absence of an abnormality in the manufacturing plant is stored, and a prediction model is constructed, for example, as a neural network, that uses the presence or absence of the abnormality as training data and acquired information as input data to predict the occurrence of an abnormality in the manufacturing plant. A data analysis system predicts the occurrence of an abnormality using the prediction model. Conventional data analysis systems simply display the prediction results of the prediction model, making it difficult for users to understand the reasons for the prediction results.

[0101] Therefore, by applying the information processing device 100 according to the proposed technology to a data analysis system, the data analysis system can present the reason for the prediction to the user for the prediction result.

[0102] First, the information processing device 100 generates a causal model for an already constructed prediction model. Here, it is assumed that a causal model is generated for an output variable indicating whether or not an abnormality has occurred.

[0103] FIG. 5 is a diagram illustrating an example of a causal model estimated by the information processing device 100 according to an application example of the embodiment of the present disclosure.

[0104] For example, the causal model estimation unit 132 of the information processing device 100 estimates that three items, namely, measured quantity A indicating the maximum voltage level in the past day, the elapsed time since device A was reset, and measured quantity B indicating the frequency of vibration occurrence, have a direct causal relationship with the output, i.e., whether or not an abnormality has occurred.

[0105] Here, it is assumed that measured value B is the sole cause of whether or not an abnormality occurs. However, measured value A is not the sole cause, and an abnormality will occur only when the maximum voltage level is at the high level out of three levels (high, middle, low) and the elapsed time since reset is YY hours or more.

[0106] Furthermore, measured quantities A and B change depending on humidity. Note that no measures have been taken within the factory to eliminate the effects of humidity, and the data analysis system also acquires information about measured quantities C and D, which are affected by humidity, but it is assumed that measured quantities C and D do not affect the output (whether or not an abnormality has occurred).

[0107] The data analysis system also acquires information about a measurand F and a measurand E that varies under the influence of the measurand F and the measurand A.

[0108] As shown in Figure 5, when the causal model estimation unit 132 estimates the causal relationships of the prediction model, it estimates that in addition to the measured quantities A and B and the elapsed time since the reset of device A, which are the sole causes of whether or not an abnormality occurs, humidity and measured quantities C, D, and E also have a correlation with the output.

[0109] Therefore, for example, in a system that outputs the reason for a prediction result based on correlation, there is a risk that humidity or measurement quantities C, D, and E, which have little causal relationship to the presence or absence of an abnormality, may be presented as the reason for the output result.

[0110] For example, even if the user intervenes on humidity or measurement quantities C, D, and E based on the presented information, there is a risk that the intervention will not be effective. Alternatively, even if the user does not intervene, there is a risk that the user will accumulate the mistaken belief that the presented information is the cause of the abnormality, which could hinder the user's understanding of the manufacturing process.

[0111] On the other hand, in the proposed technology, the information processing device 100 presents the reason to the user using a cause (input variable) that is estimated to be closest to the true cause for the objective variable (here, output). Therefore, the information processing device 100 can present a more appropriate input variable as the reason. This allows the information processing device 100 to present the user with a reason that is more effective in intervention, and can suppress the accumulation of incorrect knowledge by the user.

[0112] Furthermore, in the proposed technology, the information processing device 100 can combine multiple input variables and present them to the user as reasons (composite effects) for the objective variable. The information processing device 100 does not calculate the effect of multiple input variables by adding the effects of the input variables alone, but calculates the effect (combined effect) according to the combined state of the combination. Therefore, the information processing device 100 can present to the user a combination of input variables that is more appropriate as a reason for the prediction result.

[0113] An example of analysis processing by a data analysis system according to an application example of an embodiment of the present disclosure will be described below with reference to FIGS.

[0114] The reason for the prediction result can be presented to the user simultaneously with the prediction result. Here, the information processing device 100 presents the reason for the prediction to the user, for example, by using a pop-up, at the same time that the data analysis system presents the prediction result.

[0115] Note that predictions using a prediction model may be performed by, for example, a prediction device (not shown) included in the data analysis system, and the prediction results may be displayed on a display device (not shown) by the prediction device. At this time, the information processing device 100 may present information about input variables selected as reasons for the prediction results so that the user can confirm both the prediction results and the reasons.

[0116] As described above, the information processing device 100 may present a plurality of input variables as reasons. In this case, the information processing device 100 may calculate the combined effect by combining a predetermined number of input variables, or may receive from the user the number of input variables to be combined.

[0117] FIG. 6 is a diagram showing an example of a UI screen output by the information processing device 100 according to an application example of the embodiment of the present disclosure.

[0118] The information processing device 100 presents the user with candidates for the number of input variables to be combined, for example, by displaying a UI screen shown in Fig. 6. In Fig. 6, the number of variables that the user can select is one to three, but the number of variables that can be selected as a combination may be four or more.

[0119] For example, the user selects the number of input variables to be combined by clicking on a corresponding location on the UI screen. In this way, the information processing device 100 outputs a UI screen (interface) for determining a combination of input variables, and the user performs an operation corresponding to the UI screen (interface). Based on the operation, the information processing device 100 determines a combination of the number of input variables selected by the user. The information processing device 100 selects to display a reason for the determined combination and calculates the influence of the combination. The information processing device 100 combines the selected combination of input variables with the influence and presents it to the user along with the prediction result.

[0120] First, a case where the user selects one variable will be described.

[0121] For example, in the estimated causal model (see FIG. 5), the information processing device 100 selects three input variables (measurement quantity A indicating the maximum voltage level in the past day, measurement quantity B indicating the elapsed time since the reset of device A, and measurement quantity B indicating the frequency of vibration occurrence) that are directly connected to the output variable, i.e., whether or not an abnormality has occurred, as important variables, and calculates the degree of influence.

[0122] The information processing device 100 associates information about the important variables with the calculated influence degrees and presents them to the user.

[0123] Fig. 7 is a diagram showing an example of a UI screen output by the information processing device 100 according to an application example of the embodiment of the present disclosure. Fig. 7 shows an example of a UI screen output by the information processing device 100 when a user selects one variable. Note that the importance shown in Fig. 7 corresponds to the influence degree described above.

[0124] The information processing device 100 presents information about important variables to the user by sorting them, for example, in descending order of importance. In the example of Fig. 7, the information processing device 100 displays measured quantity A, which has the highest importance of "0.21," as the "maximum voltage level," and measured quantity B, which has the second highest importance of "0.16," as the "frequency of vibration occurrence." In addition, the information processing device 100 displays the elapsed time since reset of device A, which has the third highest importance of "0.14," as the "elapsed time since reset."

[0125] In this way, the information processing device 100 associates important variables with importance and displays them in a list in descending order of importance, thereby making it possible to present the reason for the prediction result to the user in an easy-to-understand manner. At this time, the information processing device 100 can present a more persuasive reason to the user by presenting the strength of the causal relationship with the output variable to the user using a quantitative index such as importance.

[0126] As shown in Fig. 7, the information processing device 100 may display the list by changing the background color of the table according to the magnitude (absolute value) of the importance. For example, the information processing device 100 selects a darker background color for displaying the list as the importance increases. In this way, the information processing device 100 determines the order and color on the display screen corresponding to the importance, allowing the user to more easily recognize the importance of important variables. This also applies to Figs. 8 and 9, which will be described later.

[0127] First, a case where the user selects two variables will be described.

[0128] For example, in the estimated causal model (see FIG. 5), the information processing device 100 selects a combination of two of three input variables (measurement quantity A indicating the maximum voltage level in the past day, and measurement quantity B indicating the elapsed time since the reset of device A and the frequency of vibration occurrence) that are directly connected to the output variable, i.e., whether or not an abnormality has occurred, as a combination of important variables.

[0129] The information processing device 100 calculates the composite influence of the selected combination of important variables. The information processing device 100 associates information about the combination of important variables with the calculated composite influence and presents it to the user.

[0130] Fig. 8 is a diagram showing an example of a UI screen output by the information processing device 100 according to an application example of the embodiment of the present disclosure. Fig. 8 shows an example of a UI screen output by the information processing device 100 when the user selects two variables. Note that the importance shown in Fig. 8 corresponds to the composite influence degree described above.

[0131] The information processing device 100 sorts the information about the important variables, for example, in descending order of importance, and presents it to the user. In the example of Fig. 8, the combination of measured values ​​A and B has the highest importance of "0.73," so the information processing device 100 displays the information about the combination of measured values ​​A and B, "maximum voltage level" and "frequency of vibration occurrence," as reasons for the prediction result, in association with the importance.

[0132] Furthermore, the combination of measurement amount A and the time elapsed since resetting device A has the second highest importance of "0.04." Therefore, the information processing device 100 displays the information "maximum voltage level" and "time elapsed since reset" relating to the combination of measurement amount A and the time elapsed since resetting device A as the reason for the prediction result, in association with the importance.

[0133] Furthermore, the combination of measurement quantity B and the time elapsed since resetting device A has the third highest importance of "0.01." Therefore, the information processing device 100 displays the information "frequency of vibration occurrence" and "time elapsed since resetting" related to the combination of measurement quantity B and the time elapsed since resetting device A as the reason for the prediction result, in association with the importance.

[0134] In this case, the information processing device 100 does not simply add up the importance for one variable to calculate the composite importance, but calculates the composite importance by taking spurious correlation into consideration. Therefore, the information processing device 100 can present a more appropriate importance to the user even when two variables are combined as reasons.

[0135] Here, it is assumed that the number of input variables directly connected to the output variable in the causal graph is three, and the information processing device 100 selects all three input variables as important variables, but this is not limiting. For example, when the number of input variables directly connected to the objective variable in the causal graph is large, the information processing device 100 may limit the number of input variables selected as important variables or the number of input variables selected as a combination of important variables using the above-mentioned first information or the like.

[0136] In the above example, the information processing device 100 presents a combination of a fixed number of variables as reasons to the user, but the present invention is not limited to this. For example, by newly defining a composite impact that allows relative comparison as a unified index, the information processing device 100 can relatively compare the composite impacts when different numbers of variables are combined. This allows the information processing device 100 to present combinations of different numbers of variables as reasons to the user.

[0137] For example, the information processing device 100 calculates the following formulas (20) to (22) as a fourth index CEp instead of the third index CE in formula (8) as unified first information independent of the number of variables to be combined. Note that p represents the number of variables included in the combination, l represents an instance of the objective variable, and xn1 to xn3 represent the category values ​​of the instances of the input variables included in the combination.

[0138]

number

[0139] In this case, the information processing device 100 may calculate the following equations (23) to (25) as a fifth index ANCIp instead of the second index ANCI in equation (7) as unified first information independent of the number of variables to be combined.

[0140]

number

[0141] Here, p represents the number of variables included in the combination, L represents the objective variable, and Xn1 to Xn3 represent the input variables included in the combination.

[0142] Here, an example of the fourth index CEp and the fifth index ANCIp is shown when the number of variables to be combined P is P = 1 to 3, but the number of variables to be combined is not limited to 1 to 3 and may be 4 or more.

[0143] In this way, the information processing device 100 uses a unified index that is independent of the variables to be combined, and thus it becomes possible to present to the user combinations of variables that are equal to or less than a predetermined number as reasons.

[0144] This point will be explained using the example of the data analysis system. Here, a case will be explained in which the information processing device 100 presents combinations of three or less input variables as reasons. Note that the maximum value of the input variables to be combined may be specified by the user or may be determined by the information processing device 100.

[0145] For example, in the estimated causal model (see FIG. 5), the information processing device 100 selects one to three of the three input variables (measurement quantity A indicating the maximum voltage level in the past day, and measurement quantity B indicating the elapsed time since the reset of device A and the frequency of vibration occurrence) that are directly connected to the output variable, i.e., whether or not an abnormality has occurred, and sets the combination of the selected input variables as a combination of important variables.

[0146] The information processing device 100 calculates the composite influence of the combination of the selected important variables using the above formulas (20) to (22). The information processing device 100 associates information about the combination of important variables with the calculated composite influence and presents it to the user.

[0147] Fig. 9 is a diagram showing an example of a UI screen output by the information processing device 100 according to an application example of the embodiment of the present disclosure. Fig. 9 shows an example of a UI screen output by the information processing device 100 when three or less variables are combined. Note that the importance shown in Fig. 9 corresponds to the composite influence degree described above.

[0148] The information processing device 100 presents information about the important variables to the user, sorting them, for example, in descending order of importance. In the example of Fig. 9, the combination of measured values ​​A and B has the highest importance of "0.73", so the information processing device 100 displays information about the combination of measured values ​​A and B, "maximum voltage level" and "frequency of vibration occurrence", as reasons for the prediction result, in association with the importance. In addition, the information processing device 100 presents the number of important variables included in the combination to the user as the number of sets.

[0149] Furthermore, the importance of measurement quantity A is the second highest at "0.21." Therefore, the information processing device 100 displays the information about measurement quantity A, "maximum voltage level," as the reason for the prediction result, in association with the number of important variables (number of sets) included in the combination, "1," and the importance, "0.21."

[0150] 9, the importance of the combination of measurement quantities A and B and the time elapsed since resetting device A is "0.01." Therefore, the information processing device 100 displays the information about the combination of measurement quantities A and B and the time elapsed since resetting device A, "maximum voltage level," "frequency of vibration occurrence," and "time elapsed since reset," as the reason for the prediction result, in association with the number of sets "3" and the importance.

[0151] In this way, the information processing device 100 can present combinations containing different numbers of important variables as reasons to the user by using an index that can be compared uniformly as the first information regardless of the number of important variables included in the combination.

[0152] If the number of important variables included in the combination is variable, the number of combinations for calculating the importance (impact) increases, which may increase the processing load of the information processing device 100. In this case, the information processing device 100 can suppress the increase in the processing load by selecting the number of important variables to be combined based on the above-mentioned formulas (23) to (25).

[0153] As described above, the information processing device 100 according to the embodiment of the present disclosure includes the intervention effect calculation unit 135 (see FIG. 1) and has a function of calculating the intervention effect.

[0154] Therefore, for example, when the user selects one of the presented reasons from a list, the information processing device 100 may calculate the effect of intervening in the input variables included in the selected reason and present it to the user. In this case, for example, the user may be able to instruct a specific intervention pattern for the input variable selected.

[0155] For example, in the above-described data analysis system, suppose that the reason for the prediction result is the amount of time that has elapsed since device A was reset. From this, the user can understand that the prediction predicts that there is a high possibility of an abnormality occurring because a long amount of time has elapsed since device A was reset. Therefore, the user can use the function of the information processing device 100 to calculate the intervention effect that can be obtained by intervening during this elapsed time. In this case, the user can have the information processing device 100 calculate the intervention effect, i.e., the reduction in the probability of an abnormality occurring, by shortening the time interval for resetting device A by 15 minutes, for example.

[0156] In this way, the information processing device 100 presents the reason for the prediction result to the user, allowing the user to consider a specific intervention pattern. Furthermore, the information processing device 100 calculates the effect of the intervention pattern considered by the user, allowing the user to consider the balance between the intervention effect and the cost required to implement the intervention and determine whether to implement the intervention.

[0157] <4.2. Example of application to a recidivism assessment system> For example, a case will be described in which the proposed technology is applied to a recidivism determination system that determines whether a person will commit a recidivism based on the data of that person.

[0158] A recidivism determination system inputs, for example, a person's age and history (for example, criminal history (number of offenses), detention period, etc.) and determines whether the person will commit a recidivism.

[0159] Figure 10 is a diagram for explaining the causal relationships of the prediction model of the recidivism assessment system. Whether or not a person will commit a crime again may have the causal relationships shown in Figure 10.

[0160] As shown in Figure 10, there is a negative correlation between age and whether or not someone will reoffend (hereafter referred to as whether or not they will reoffend). In other words, the older the person, the lower the likelihood of reoffending, and the younger the person, the higher the likelihood of reoffending.

[0161] Furthermore, there is a positive correlation between criminal history (number of crimes) and recidivism. In other words, the more criminal history a person has, the higher the likelihood of recidivism.

[0162] Criminal history and age are thought to be positively correlated, i.e., the older a person is, the more likely they are to have a criminal history.

[0163] Therefore, age plays a role known as confounding, causing bias. Therefore, since there is a possibility that there is a spurious correlation between recidivism and criminal history, it is desirable to estimate the correlation between recidivism and criminal history by removing the influence of age. This influence of age is expressed by the magnitude of partial correlation or conditional mutual information. For example, by using the p-value of a statistical test as an indicator of whether the partial correlation coefficient can be considered zero, the information processing device 100 can generate a causal model that eliminates spurious correlation. In other words, the information processing device 100 generates a causal model by considering input variables that have a causal relationship with the output variable, and that have a partial correlation coefficient that is not zero or that is not conditionally independent.

[0164] Here, it is assumed that the prediction model of the recidivism determination system described above is constructed as a neural network with five nearest node groups (L1 to L5).

[0165] The information processing device 100 estimates a causal model for each of the closest nodes L1 to L5 of the prediction model. The information processing device 100 estimates a causal model in which age, criminal history (number of crimes), detention period, etc. are linked to each of the closest nodes L1 to L5.

[0166] The information processing device 100 presents the reason for the recidivism along with the predicted result of the recidivism based on the estimated causal model.

[0167] Furthermore, when the information processing device 100 estimates the reason using the causal model of the nearest node, it may calculate the probability of the level of the nearest nodes L1 to L5 and present it to the user.

[0168] FIG. 11 is a chart showing the conditional probability of the level of the closest node L1 calculated by the information processing device 100 according to an embodiment of the present disclosure.

[0169] 11, the information processing device 100 calculates the conditional probabilities by dividing the criminal history into two groups using three cases as a threshold and dividing the age into three groups using 27 and 37 as thresholds. The information processing device 100 also calculates the conditional probabilities by dividing the L1 level into three levels (high, middle, low) using 0.37 and 1.578 as thresholds.

[0170] For example, suppose a recidivism assessment system uses a prediction model to predict the likelihood of a person (instance) reoffending. If the person being predicted has five criminal records, the average probability that the closest node L1 will be at a high level when the person has three or more criminal records is 30.4%. This probability varies depending on age, ranging from 3.9% to 39.7%. If the person being predicted is 61 years old, the probability that the closest node L1 will be at a high level is 39.7%. On the other hand, if we focus only on age, the average probability that the closest node L1 will be at a high level at age 61 is 13%. Thus, when focusing only on age, the probability that the closest node L1 will be at a high level is significantly lower than when recidivism history is taken into account.

[0171] In this way, multiple input variables have a composite effect on the objective variable (here, the closest node). The information processing device 100 can present these multiple input variables to the user as reasons for their effect on the objective variable.

[0172] In particular, when there is a nonlinear relationship between the input variables and the objective variable, or when a specific combination of input variables has a significant impact on the objective variable, the information processing device 100 can calculate the composite impact degree, thereby making it possible to present a more appropriate reason to the user.

[0173] As described above, the information processing device 100 calculates the probabilities of the levels of the nearest nodes L1 to L5 and presents them to the user, allowing the user to check the input variables that affect the nearest nodes L1 to L5. This allows the user to more easily analyze the prediction model.

[0174] The input variables may include variables that have a positive correlation with the output variable, i.e., variables that are a positive reason supporting the output of the final prediction result, such as the criminal history mentioned above. Also, the input variables may include variables that have a negative correlation with the output variable, i.e., variables that are a negative reason supporting the output of the final prediction result, such as the age mentioned above.

[0175] Therefore, the information processing device 100 may be configured to separately present important variables that are positive reasons and important variables that are negative reasons for the prediction result to the user. That is, the information processing device 100 may be configured to calculate the composite influence of a combination of input variables that have a positive correlation with an output variable (or a target variable) and the composite influence of a combination of input variables that have a negative correlation with an output variable (or a target variable).

[0176] More specifically, if the predictive model is a neural network that performs binary classification and identification, a positive or negative value is assigned to the weight w of each node (neuron). The neurons that fire and attract each value are determined in advance through learning, so that a positive value selects one of the two values ​​and a negative value selects the other.

[0177] For a specific prediction, neurons are divided into neurons that pull the output in a positive direction and neurons that pull the output in a negative direction. Therefore, the information processing device 100 can calculate weights for reasons separately for positive and negative values. For example, the binary values ​​output by the final output Y are "1" and "0." In the case of the recidivism assessment system described above, for example, "recidivism" may correspond to "1" and "no recidivism," respectively. In this case, the information processing device 100 calculates the weights of input variables that pull toward Y=1, i.e., that make the final output Y 1, using equation (26). Furthermore, the information processing device 100 calculates the weights of input variables that pull toward Y=0, i.e., that make the final output Y 0, using equation (27).

[0178]

number

[0179] Note that Pos in equation (26) represents a set of nearest nodes (final layer neurons) with a positive weight w, and Neg in equation (27) represents a set of nearest nodes with a negative weight w. The information processing device 100 can use equation (26) to evaluate variables or combinations of reasons for attraction in the positive direction. The information processing device 100 can also use equation (27) for the negative direction.

[0180] The information processing device 100 can quantify and evaluate the weight as a reason for an important variable or a combination of important variables by calculating the sum of equations (26) and (27) as the influence EP, as shown in equation (28).

[0181]

number

[0182] The influence EP can be calculated as a positive value or a negative value. Therefore, the information processing device 100 can present to the user the reasons (important variables) that attract the final output Y in the positive direction and the reasons (important variables) that attract the final output Y in the negative direction.

[0183] <<5. Other embodiments>> In the above-described embodiment, the prediction model is a neural network, but this is not limiting. The prediction model for which the information processing device 100 performs reasoning may be a machine learning device other than a neural network.

[0184] For example, when the prediction model is a random forest, the information processing device 100 estimates one causal model that causes the output variable in order to find an input variable that explains the output variable. That is, the information processing device 100 estimates a causal model using the output variable as a response variable and selects important variables. Note that the processing performed by the information processing device 100 is the same as the processing shown in FIG. 4, except that the output variable is used as the response variable.

[0185] In this way, the information processing device 100 can present to the user the reasons for the prediction results of prediction models generated by machine learning, not limited to neural networks.

[0186] In the above-described embodiment, the information processing device 100 presents important variables with high influence to the user, but this is not limiting. For example, the information processing device 100 may present important variables with low influence to the user. The information processing device 100 outputs an input variable that has a spurious correlation with a variable of interest as an input variable that does not affect the variable of interest. Alternatively, the information processing device 100 may output an input variable that is conditionally independent of the variable of interest.

[0187] In this way, the information processing device 100 presents to the user input variables that have a low influence on the variable of interest, for example, input variables that have a spurious correlation with the variable of interest, so that the user can know the input variables that do not affect the prediction result.

[0188] <<6. Modifications>> In the above-described embodiment, the information processing device 100 determines whether each input variable is conditionally independent with respect to the output variable, selects an input variable determined not to be conditionally independent as an important variable, and outputs the selected input variable, but is not limited to this. The information processing device 100 may determine whether each input variable is conditionally independent with respect to the output variable (a spurious correlation relationship), select one or both of an input variable determined not to be conditionally independent and an input variable determined to be conditionally independent, and output the selected input variable.

[0189] Specifically, for example, the selection unit 133 identifies input variables that are conditionally independent with respect to the output variable and input variables that are not conditionally independent based on the causal model estimated by the causal model estimation unit 132, and selects variables to explain the hidden node Lm from the identified input variables. Here, the selection unit 133 may select both input variables that are determined to be not conditionally independent with respect to the output variable and input variables that are determined to be conditionally independent. For example, the selection unit 133 can select input variables by distinguishing whether each input variable is conditionally independent with respect to the output variable or not.

[0190] The evaluation unit 134 evaluates the degree of influence that the selected variables selected by the selection unit 133 have on the target variable (variable of interest). The input variables selected by the selection unit 133 and the influence degrees calculated by the evaluation unit 134 are presented to the user by, for example, the input / output control unit 131. Here, the input / output control unit 131 may present to the user whether the selected variables selected by the selection unit 133 are conditionally independent or not conditionally independent with respect to the output variable.

[0191] The intervention effect calculation unit 135 calculates the intervention effect that occurs on the objective variable by intervening in the input variables that are determined not to be conditionally independent with respect to the output variable, among the input variables selected by the selection unit 133.

[0192] The input variables that are conditionally independent with respect to the output variable and the input variables that are not conditionally independent may be distinguished by a statistical test for correlation or partial correlation, or a calculation of statistics for correlation or partial correlation, by the causal model estimation unit 132. The input variables that are conditionally independent with respect to the output variable and the input variables that are not conditionally independent may be distinguished by a statistical test for independence or conditional independence, or a calculation of statistics for independence or conditional independence, by the causal model estimation unit 132. The input variables that are conditionally independent with respect to the output variable and the input variables that are not conditionally independent may be distinguished by a calculation for a statistical model selection criterion, or a calculation for a penalized likelihood, by the causal model estimation unit 132.

[0193] Fig. 12 is a diagram showing an example of a UI screen output by the information processing device 100 according to a modified example of the embodiment of the present disclosure. Similar to the example of Fig. 7 described above, Fig. 12 shows an example of a UI screen output by the information processing device 100 when the user selects one variable, but a row indicating the measured value C and the measured value E and a column indicating the causal relationship between the output variable and the UI screen of Fig. 7 are added.

[0194] The information processing device 100 may present information about the input variables to the user by sorting them, for example, in descending order of importance. In the example of Fig. 12, the information processing device 100 displays the information in the following order: "Frequency of Vibration Occurrence" with the highest importance of "0.32", "Measured Quantity C" with the second highest importance of "0.21", "Maximum Voltage Level" with the third highest importance of "0.18", "Measured Quantity E" with the fourth highest importance of "0.15", and "Elapsed Time Since Reset" with the fifth highest importance of "0.11".

[0195] In the example of FIG. 12, for "vibration occurrence frequency," "maximum voltage level," and "time elapsed since reset," "causal" is displayed, indicating that there is a causal relationship with the output variable, and "intervention possible" is displayed, indicating that the intervention effect on the target variable can be calculated. Furthermore, for "measurement C" and "measurement E," "non-causal" is displayed, indicating that there is no causal relationship with the output variable. Note that, when the information processing device 100 receives a user selection operation for an input variable that is displayed as "intervention possible," it may display a display inquiring whether to calculate the intervention effect, or may automatically proceed to calculate the intervention effect.

[0196] 12, the information processing device 100 may display the list by changing the background color of the table depending on the magnitude (absolute value) of the importance and the causal relationship with the output variable. For example, the information processing device 100 displays the list by selecting a darker background color as the importance increases. Also, for example, the information processing device 100 displays the list by selecting a darker background color for input variables that have a causal relationship with the output variable than for input variables that do not have a causal relationship with the output variable.

[0197] Even when the user displays two or three variables, a display similar to that shown in FIG. 12 is possible.

[0198] Note that the importance (impact) varies depending on factors other than the strength of the relationship between the variable of interest and the input variables, so the importance of non-causal reasons may be greater than the importance of causal reasons.

[0199] As described in this modification, by presenting non-causal reasons to the user, it becomes possible to present reasons even when, for example, causal reasons cannot be presented as explanations. It also becomes possible to present non-causal reasons of high importance. This makes it possible to present reasons that are easy for the user to understand.

[0200] <<7. Hardware Configuration Example>> Finally, the hardware configuration of an information processing device according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a block diagram showing an example of the hardware configuration of an information processing device 900 according to this embodiment. Note that the information processing device 900 shown in Fig. 13 can realize, for example, the information processing device 100 shown in Fig. 1. Information processing by the information processing device 100 according to this embodiment is realized by cooperation between software and hardware described below.

[0201] 13, the information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, and a host bus 904a. The information processing device 900 also includes a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 911, and a communication device 913. The information processing device 900 may include a processing circuit such as an electric circuit, a DSP, or an ASIC instead of or in addition to the CPU 901.

[0202] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs and calculation parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change as appropriate during the execution. The CPU 901 may form, for example, the control unit 130 shown in FIG. 1.

[0203] The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904a that includes a CPU bus, etc. The host bus 904a is connected to an external bus 904b, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 904. Note that the host bus 904a, bridge 904, and external bus 904b do not necessarily need to be configured separately, and these functions may be implemented on a single bus.

[0204] The input device 906 is realized by a device into which a user inputs information, such as a mouse, keyboard, touch panel, button, microphone, switch, or lever. The input device 906 may also be, for example, a remote control device using infrared or other radio waves, or an externally connected device such as a mobile phone or PDA that operates the information processing device 900. The input device 906 may also include, for example, an input control circuit that generates an input signal based on information input by the user using the above-mentioned input means and outputs the signal to the CPU 901. By operating the input device 906, the user of the information processing device 900 can input various data to the information processing device 900 and instruct processing operations. The input device 906 may form, for example, the input unit 110 shown in FIG. 1 .

[0205] The output device 907 is formed by a device capable of visually or audibly notifying the user of acquired information. Examples of such devices include display devices such as CRT display devices, liquid crystal display devices, plasma display devices, EL display devices, laser projectors, LED projectors, and lamps, audio output devices such as speakers and headphones, and printer devices. The output device 907 outputs, for example, results obtained from various processes performed by the information processing device 900. Specifically, the display device visually displays the results obtained from various processes performed by the information processing device 900 in various formats such as text, images, tables, and graphs. On the other hand, the audio output device converts audio signals consisting of reproduced voice data, acoustic data, etc. into analog signals and outputs them audibly. The output device 907 may form, for example, the output unit 120 shown in FIG. 1.

[0206] The storage device 908 is a data storage device formed as an example of a storage unit of the information processing device 900. The storage device 908 is realized by, for example, a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 908 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, and a deletion device that deletes data recorded on the storage medium. The storage device 908 stores programs executed by the CPU 901, various data, and various data acquired from outside. The storage device 908 may store, for example, reports created by the control unit 130, intermediate results and final results of analysis processing, etc.

[0207] The drive 909 is a reader / writer for a storage medium, and is built into or externally attached to the information processing device 900. The drive 909 reads information recorded on a removable storage medium such as an attached magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 903. The drive 909 can also write information to the removable storage medium.

[0208] The connection port 911 is an interface connected to an external device, and is a connection port for connecting to an external device that can transmit data via, for example, a USB (Universal Serial Bus) or the like.

[0209] The communication device 913 is, for example, a communication interface formed by a communication device or the like for connecting to the network 920. The communication device 913 is, for example, a communication card for a wired or wireless local area network (LAN), a long term evolution (LTE), Bluetooth (registered trademark), or a wireless USB (WUSB). The communication device 913 may also be a router for optical communication, a router for an asymmetric digital subscriber line (ADSL), or a modem for various communications. The communication device 913 can transmit and receive signals, for example, between the Internet and other communication devices in accordance with a predetermined protocol such as TCP / IP. In this embodiment, the communication device 913 functions as the input unit 110 and / or the output unit 120 and can receive information input to the control unit 130 and transmit information output from the control unit 130.

[0210] The network 920 is a wired or wireless transmission path for information transmitted from devices connected to the network 920. For example, the network 920 may include public networks such as the Internet, telephone networks, and satellite communication networks, as well as various LANs (Local Area Networks) including Ethernet (registered trademark), and WANs (Wide Area Networks). The network 920 may also include dedicated network such as an IP-VPN (Internet Protocol-Virtual Private Network).

[0211] The above describes an example of a hardware configuration capable of realizing the functions of the information processing device 900 according to this embodiment. Each of the above components may be realized using general-purpose components, or may be realized by hardware specialized for the function of each component. Therefore, the hardware configuration used can be changed as appropriate depending on the technical level at the time of implementing this embodiment.

[0212] It is possible to create a computer program for implementing each function of the information processing device 900 according to this embodiment as described above and install it on a PC or the like. It is also possible to provide a computer-readable recording medium storing such a computer program. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network without using a recording medium.

[0213] <<8. Summary>> Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0214] Note that each device described in this specification may be realized as a single device, or some or all of them may be realized as separate devices. For example, among the functional configuration example of the information processing device 100 shown in Fig. 1, the control unit 130 may be provided in a device such as a server connected to the input unit 110 and the output unit 120 via a network or the like.

[0215] Furthermore, the processes described herein using flowcharts and sequence diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Furthermore, additional process steps may be employed, and some process steps may be omitted.

[0216] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0217] The following configurations also fall within the technical scope of the present disclosure. (1) selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; An information processing device including a control unit. (2) the control unit selects the first explanatory variable as a reason for the prediction result from among the plurality of input variables based on information on whether or not there is a spurious correlation between the input variables and the prediction result in the prediction model generated using machine learning; outputting information about the selected first explanatory variable; An information processing device according to (1). (3) The information processing device according to (2), wherein the control unit selects the input variable that is not in a spurious correlation relationship with the prediction result as the first explanatory variable. (4) The information processing device according to (2) or (3), wherein the control unit selects, as the first explanatory variable, the input variable that is not conditionally independent with respect to the prediction result. (5) The information processing device according to any one of (2) to (4), wherein the control unit outputs strength information indicating the strength of the relationship between the first explanatory variable selected as the reason and the prediction result. (6) The information processing device according to any one of (2) to (5), wherein the control unit selects a combination of at least two of the input variables as the reason for the prediction result. (7) The information processing device described in (6), wherein the control unit outputs strength information indicating the strength of the relationship between at least two of the input variables included in the combination and the prediction result, in association with information regarding the combination. (8) the control unit determines at least one of an order and a color on a display screen corresponding to the first explanatory variable based on the intensity information; outputting the display screen; An information processing device according to (5) or (7). (9) the control unit outputs an interface for determining a combination of the input variables; determining a combination of the input variables based on an operation corresponding to the interface; The information processing device according to any one of (2) to (8). (10) The information processing device according to any one of (2) to (9), wherein the control unit estimates a causal graph for a plurality of the input variables using an output variable indicating the prediction result as a dependent variable, and selects the first explanatory variable as the reason from the input variables that have a direct causal relationship with the dependent variable. (11) The information processing device according to any one of (2) to (10), wherein the control unit estimates a causal graph relating to the nearest node included in a hidden layer closest to the prediction model for a plurality of the input variables as a target variable, and selects the first explanatory variable to be used as the reason from the input variables that have a direct causal relationship with the target variable. (12) The information processing device according to (11), wherein the control unit selects the first explanatory variable to be the positive reason based on the causal graph related to the nearest node having a positive weight among the nearest nodes, and selects the first explanatory variable to be the negative reason based on the causal graph related to the nearest node having a negative weight among the nearest nodes. (13) The information processing device according to any one of (2) to (12), wherein the control unit calculates an intervention effect when an intervention is performed on the first explanatory variable selected as the reason. (14) The input variables include information acquired by sensors. The information processing device according to any one of (1) to (13). (15) The input variables include information about the operating environment or operating state of the device obtained by a sensor; The information processing device according to any one of (1) to (14). (16) The input variables include information about temperature, humidity, voltage, current, power, or vibration obtained by a sensor; the control unit selects at least one of information regarding temperature, humidity, voltage, current, power, or vibration acquired by the sensor as the first explanatory variable; The information processing device according to any one of (1) to (15). (17) The input variables include information about the age or background of the person. The information processing device according to any one of (1) to (16). (18) the control unit acquires a selection operation for the output first explanatory variable, Calculating an intervention effect on the first explanatory variable selected by the selection operation. The information processing device according to any one of (1) to (17). (19) the control unit selects, based on the causal model, the input variable that does not affect the prediction result as a second explanatory variable; outputting information about the second explanatory variable separately from information about the first explanatory variable; The information processing device according to any one of (1) to (18). (20) the control unit selects the second explanatory variable as a reason for the prediction result from among the plurality of input variables based on information on whether or not there is a spurious correlation between the input variables and the prediction result in the prediction model generated using machine learning; outputting information about the selected second explanatory variable separately from information about the first explanatory variable; (19) An information processing device according to (19). (twenty one) The information processing device according to (20), wherein the control unit selects, as the second explanatory variable, the input variable that is in a spurious correlation with the prediction result or the input variable that is conditionally independent. (twenty two) selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; Information processing methods. (twenty three) Computer, selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; A program to make it work like this. [Explanation of symbols]

[0218] 100 Information processing device 110 Input section 120 Output section 130 Control Unit 131 Input / Output Control Unit 132 Causal Model Estimation Unit 133 Selection Section 134 Evaluation Department 135 Intervention Effect Calculation Section

Claims

1. selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; An information processing device including a control unit.

2. the control unit selects the first explanatory variable as a reason for the prediction result from among the plurality of input variables based on information on whether or not there is a spurious correlation between the input variables and the prediction result in the prediction model generated using machine learning; outputting information about the selected first explanatory variable; The information processing device according to claim 1 .

3. The information processing device according to claim 2 , wherein the control unit selects the input variable that is not in a spurious correlation relationship with the prediction result as the first explanatory variable.

4. The information processing device according to claim 2 , wherein the control unit selects the input variable that is not conditionally independent with respect to the prediction result as the first explanatory variable.

5. The information processing device according to claim 2 , wherein the control unit outputs strength information indicating the strength of a relationship between the first explanatory variable selected as the reason and the prediction result.

6. The information processing device according to claim 2 , wherein the control unit selects a combination of at least two of the input variables as the reason for the prediction result.

7. The information processing device according to claim 6 , wherein the control unit outputs strength information indicating a strength of a relationship between at least two of the input variables included in the combination and the prediction result, in association with information about the combination.

8. the control unit determines an order or a color on a display screen corresponding to the first explanatory variable based on the intensity information; outputting the display screen; The information processing device according to claim 5 .

9. the control unit outputs an interface for determining a combination of the input variables; determining a combination of the input variables based on an operation corresponding to the interface; The information processing device according to claim 6 .

10. 3. The information processing device according to claim 2, wherein the control unit estimates a causal graph for a plurality of the input variables using an output variable indicating the prediction result as a dependent variable, and selects the first explanatory variable as the reason from the input variables that have a direct causal relationship with the dependent variable.

11. 3. The information processing device according to claim 2, wherein the control unit estimates a causal graph relating to a plurality of the input variables, using a closest node included in a hidden layer closest to the prediction model as a response variable, and selects the first explanatory variable as the reason from the input variables that have a direct causal relationship with the response variable.

12. 12. The information processing device according to claim 11, wherein the control unit selects the first explanatory variable to be the positive reason based on the causal graph related to the nearest node having a positive weight among the nearest nodes, and selects the first explanatory variable to be the negative reason based on the causal graph related to the nearest node having a negative weight among the nearest nodes.

13. The information processing device according to claim 2 , wherein the control unit calculates an intervention effect when an intervention is performed on the first explanatory variable selected as the reason.

14. The input variables include information acquired by sensors. The information processing device according to claim 1 .

15. The input variables include information about the operating environment or operating state of the device obtained by a sensor; The information processing device according to claim 14.

16. The input variables include information about temperature, humidity, voltage, current, power, or vibration obtained by a sensor; the control unit selects at least one of information regarding temperature, humidity, voltage, current, power, or vibration acquired by the sensor as the first explanatory variable; The information processing device according to claim 15.

17. The input variables include information about the age or background of the person. The information processing device according to claim 1 .

18. the control unit acquires a selection operation for the output first explanatory variable, Calculating an intervention effect on the first explanatory variable selected by the selection operation. The information processing device according to claim 13.

19. the control unit selects, based on the causal model, the input variable that does not affect the prediction result as a second explanatory variable; outputting information about the second explanatory variable separately from information about the first explanatory variable; The information processing device according to claim 1 .

20. the control unit selects the second explanatory variable as a reason for the prediction result from among the plurality of input variables based on information on whether or not there is a spurious correlation between the input variables and the prediction result in the prediction model generated using machine learning; outputting information about the selected second explanatory variable separately from information about the first explanatory variable; The information processing device according to claim 19.

21. The information processing device according to claim 20 , wherein the control unit selects, as the second explanatory variable, the input variable that is in a spurious correlation with the prediction result or the input variable that is conditionally independent.

22. An information processing device comprising: selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; Information processing methods.

23. Computer, selecting, as a first explanatory variable, the input variable that influences the prediction result based on a causal model regarding a causal relationship between a plurality of input variables and a prediction result in a prediction model using machine learning; outputting information about the selected first explanatory variable; A program to make it work like this.

Images