Information processing device, information processing method, and program

Abstract

An information processing device 100 according to the present invention comprises: an explanation generation unit 121 that generates explanation data which explains a prediction value that a machine learning model outputs with respect to input of training data; and a parameter calculation unit 122 that calculates a parameter of the machine learning model so as to reduce prediction loss, which represents the degree to which the prediction value output by the machine learning model with respect to input of training data differs from a correct value set in advance, and explanation loss, which represents the degree to which the explanation data does not satisfy a criterion set in advance which the explanation data should satisfy.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In the field of machine learning, the explainability of machine learning models is important for humans to determine whether the predictions of machine learning models are reliable. Explanations for machine learning models can be broadly divided into two types: global explanations and local explanations. Global explanations explain the behavior of the machine learning model as a whole. Local explanations explain the basis for predictions made for individual cases.

[0003] Non-Patent Document 1 discloses a technology that, when given an arbitrary machine learning model, generates a simple model that locally approximates the prediction of that model for similar cases that exist in the vicinity of a certain case, and outputs this simple model as a local explanation for the prediction of that case. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] MT Ribeiro, S. Singh, and C. Guestrin, ""why should I trust you?": Explaining the predictions of any classifier," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135-1144. Summary of the Invention [Problem to be solved by the invention]

[0005] The technology disclosed in Non-Patent Document 1 has a problem in that the explanations output for each case do not match what humans expect. This is because machine learning models are trained regardless of what humans expect, and therefore do not necessarily make predictions as humans expect. However, even if a machine learning model outputs a correct prediction, humans cannot use it with confidence if it does not produce the explanations humans expect.

[0006] The above-mentioned problem becomes particularly pronounced when a machine learning model is retrained. When a machine learning model is retrained by adding training examples, humans expect the same explanation to be output for the same prediction on the same example. However, with the technology described in Non-Patent Document 1, there is a risk that different explanations will be output before and after retraining. As a result, humans cannot trust a model whose explanation changes each time it is retrained.

[0007] Therefore, an object of the present disclosure is to provide an information processing device that can solve the above-mentioned problem that the explanation for the predicted value by the machine learning model differs for each case. [Means for solving the problem]

[0008] An information processing device according to an embodiment of the present disclosure includes: an explanation generation unit that generates explanation data that explains the predicted value output by the machine learning model in response to input training data; a parameter calculation unit that calculates parameters of the machine learning model so as to reduce a prediction loss that indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct answer value, and an explanation loss that indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; Equipped with The structure is as follows.

[0009] Furthermore, an information processing method according to an embodiment of the present disclosure includes: Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy. The structure is as follows.

[0010] Furthermore, a program according to an embodiment of the present disclosure includes: Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy. Have the computer perform the process, The structure is as follows. [Effects of the Invention]

[0011] By being configured as described above, the present disclosure can prevent explanations for predicted values ​​by machine learning models from differing for each case, thereby generating highly reliable machine learning models. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining an overview of the present disclosure. [Figure 2] FIG. 1 is a diagram for explaining an overview of a first embodiment of the present disclosure. [Figure 3] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment of the present disclosure. [Figure 4] 4 is a flowchart showing the operation of the information processing device disclosed in FIG. 3. [Figure 5]4 is a flowchart showing the operation of the information processing device disclosed in FIG. 3. [Figure 6] 4 is a flowchart showing the operation of the information processing device disclosed in FIG. 3. [Figure 7] FIG. 4 is a diagram illustrating a process performed by the information processing device disclosed in FIG. 3. [Figure 8] FIG. 4 is a diagram illustrating a process performed by the information processing device disclosed in FIG. 3. [Figure 9] FIG. 4 is a diagram illustrating a process performed by the information processing device disclosed in FIG. 3. [Figure 10] FIG. 10 is a diagram illustrating a process performed by an information processing device according to a second embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram illustrating a process performed by an information processing device according to a second embodiment of the present disclosure. [Figure 12] FIG. 11 is a block diagram showing a hardware configuration of an information processing device according to a third embodiment of the present disclosure. [Figure 13] FIG. 10 is a block diagram showing a configuration of an information processing device according to a third embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] <Embodiment 1> A first embodiment of the present disclosure will be described with reference to Fig. 1 to Fig. 9. Fig. 1 and Fig. 2 are diagrams for explaining an outline of the present disclosure. Fig. 3 is a diagram for explaining the configuration of an information processing device, and Fig. 4 to Fig. 9 are diagrams for explaining the processing operation of the information processing device.

[0014] [overview] First, an overview of the present disclosure will be described with reference to FIG. 1. As shown in FIG. 1, an information processing device according to the present disclosure trains a machine learning model using training examples and updates parameters of the machine learning model. At this time, a prediction and an explanation for the prediction are output from the machine learning model to which the training examples have been input. In such a situation, the information processing device according to the present disclosure learns to update the parameters of the machine learning model so as to reduce a prediction loss representing the difference between the prediction output from the machine learning model and a preset correct label, and an explanation loss representing the difference between the explanation output from the machine learning model and a preset correct explanation. Note that the explanation loss may be expressed as the degree to which the explanation output from the machine learning model does not satisfy a preset standard.

[0015] Next, an overview of the first embodiment will be described with reference to FIG. 2. As shown in FIG. 2, in the first embodiment, perturbation examples are randomly generated for training examples, and predictions of model f for the perturbation examples are assigned. Then, a simplified model g that predicts input and output is trained using the proximity between the training examples and the perturbation examples as weights, and these weights are output as explanations. The difference between the output explanation and a preset explanation is taken as the explanation loss, and the parameters of model f are updated to reduce the prediction loss and explanation loss, as described above. In this case, when the simplified model g is a linear model, the explanation is multiplied as a differentiable function of model f. Therefore, the gradient of the explanation loss with respect to the parameters of model f can be calculated, and the parameters can be updated using this gradient to reduce the explanation loss. Note that the explanation loss may be expressed as the degree to which the explanation output from the machine learning model does not satisfy a preset standard.

[0016] <Details of Embodiment 1> Next, details of the first embodiment will be described. In the first embodiment, for any machine learning model that can update parameters using gradients, the parameters of the machine learning model are updated so that an expected explanation is output when an explanation for each case is output. This embodiment is applicable to any machine learning model that can update parameters using gradients. The explanation in this embodiment is about the weights of a linear model that locally approximates the behavior of the machine learning model. Note that the terms and symbols used in the explanation of this embodiment shall conform to Non-Patent Document 1.

[0017] First, the basic concepts used in the description of this embodiment will be explained. The machine learning model to be trained in the first embodiment is defined as f. f may be any machine learning model that can update parameters using the gradient of the objective function. For example, a neural network or gradient boosting can be used as such a model f. Furthermore, the parameters that determine the behavior of model f are represented by a vector θ. For example, when model f is a neural network, θ is a vector consisting of the weights of the neural network. When model f is gradient boosting, θ is the number of weak learners or the parameters of the weak learners. The output of model f is determined depending on the value of θ.

[0018] In supervised machine learning, a set of training examples and the correct labels associated with each training example in the set are generally input. Then, when each training example is input to model f, the parameters are updated to minimize the difference between the prediction output by model f and the correct labels associated with that training example. The difference between this prediction and the correct label is called the prediction loss.

[0019] However, simply updating the parameters to reduce the prediction loss poses a problem in that the explanation for the prediction does not match what humans expect. Therefore, in this disclosure, in addition to reducing the prediction loss, we also consider the loss related to the explanation. Specifically, in this disclosure, an explanation evaluation criterion, which is a standard for evaluating the appropriateness of the explanation, is received as input. Then, we consider the explanation loss, which is the degree to which the explanation generated for the prediction made by model f for each training example does not satisfy the explanation evaluation criterion. Then, we update the parameter θ of model f to reduce not only the prediction loss but also the explanation loss. It is particularly effective to update the parameters to reduce the weighted sum of the prediction loss and the explanation loss. This makes it possible to balance the prediction loss and the explanation loss.

[0020] A conceivable explanation evaluation criterion is, for example, the degree of agreement with the correct explanation. In this case, parameters are updated to obtain an explanation that matches the correct explanation as closely as possible. For example, an explanation that has already been presented to humans in the past can be used as the correct explanation. Such use is particularly useful when updating the parameters of a model currently in operation. There are cases where a model trained using a certain set of training examples has been used, but several additional training examples have since been obtained, and it is desired to add these to the training example set and retrain the model. In such cases, there is a need to minimize changes in the predictions and explanations for the same examples before and after retraining. This is because if the explanation changes, it is difficult for humans to understand why it differs from the previous explanation. In such cases, the explanation previously presented to humans can be used as the correct explanation. In this case, the present invention can update parameters by considering the balance between prediction loss and explanation loss, so that predictions are not significantly off and explanations do not change significantly.

[0021] Next, a specific configuration and operation of the first embodiment will be described with reference to Fig. 3 to Fig. 9. As shown in Fig. 3, the information processing system in the first embodiment includes an information processing device 10 that performs machine learning. Note that Fig. 3 also includes a correct answer explanation providing unit 20 that is configured by an information processing device that inputs data used in the machine learning. This will be described later, but the correct answer explanation providing unit 20 does not necessarily have to be provided.

[0022] An information processing device 10 that performs machine learning is composed of one or more information processing devices each equipped with a calculation device and a storage device. As shown in Fig. 3, the information processing device 10 includes an input unit 11, a parameter calculation unit, a predicted loss calculation unit 13, an explained loss calculation unit 14, and an explanation generation unit 15. The functions of the input unit 11, the parameter calculation unit 12, the predicted loss calculation unit 13, the explained loss calculation unit 14, and the explanation generation unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device. The operation of each function of each component will be described below.

[0023] Before describing the overall operation of the first embodiment, the operation of the explanation generating unit 15 will be described with reference to the flowchart of FIG.

[0024] (Step S11) The explanation generator 15 receives training examples x (training data) as input. The training examples x are real-valued vectors of length d that represent examples to be input to the model f. x may represent table data, images, or text. Figure 7 shows an example of a training example x.

[0025] (Step S12) The explanation generator 15 generates an interpretable representation x' of a training example x. The interpretable representation x' is a binary vector of length d'. x' represents the training example x in terms of the presence or absence of features that are easy for humans to understand. x' can take various forms, as described in section 3.1 of Non-Patent Document 1. For example, if the training example x is text, x' can be a binary vector representing the presence or absence of a word. Any method can be used to generate an interpretable representation, as long as it can convert the training example x into a binary vector and the result can be interpreted by humans. If the training example x is already a binary vector, x can be used directly as x'.

[0026] As an example, we will explain a possible method (hereafter referred to as the threshold method) for the case where the training example x is a vector of continuous values. Two conditions are generated by dividing each of the d elements that make up x using the median as a threshold. For example, if the median of the first element, x1, of x is 3, two conditions, "x1 ≥ 3" and "x1 < 3," are generated. This process is repeated for the other elements, generating d * 2 conditions. Finally, only the conditions satisfied by x are extracted and used as the features that make up x'. Note that the value of each feature is 1 if the condition is satisfied and 0 otherwise. Figure 8 shows an example of x' generated using the threshold method. As shown in this figure, when created using this method, all elements of x' are necessarily 1 because only the conditions satisfied by x are extracted. The threshold method can also be used to divide x' into four conditions using quartiles instead of the median. The implementation by the authors of Non-Patent Document 1 (https: / / github.com / marcotcr / lime) implements a threshold method using quartiles.

[0027] (Step S13) In step S13, the explanation generation unit 15 generates a set Z of perturbed samples based on x'. The perturbed samples are artificially generated samples and are used as training samples for constructing a second machine learning model that approximates the local prediction of f around x. The method for generating the set Z is based on the algorithm shown in Section 3.3 and Algorithm 1 of Non-Patent Document 1.

[0028] The parameters for generating the set Z are set as follows: The number of perturbation examples to be generated is set to N. The function measuring the proximity to x is set to π. x Let π x (z) is an arbitrary function that returns a larger value the closer the vector z of length d is to x, and a smaller value the farther it is from x. For example, we can use the cosine similarity of vectors.

[0029] Here, the method of generating set Z in step S13 in Fig. 4 is shown in the flowchart of Fig. 5. First, set Z is initialized as an empty set (step S21). The variable i is changed from 1 to N and the following is executed (step S22).

[0030] The i-th perturbation case z' i (Step S23). i is a binary vector of length d', just like x'. The perturbation example can be generated by any method that can obtain a binary vector of length d'. For example, it can be obtained by uniformly randomly generating a binary vector of length d'. Figure 8 shows the generated perturbation example z'. i Here is an example of the perturbation case z', where x' is all 1. i takes on values ​​of 1 and 0.

[0031] Perturbation case z' i z, which is the representation in the original space i (Step S24). iis a vector of the same length d as x. For example, if the task is image classification, obtain the corresponding image from the binary vector. In the case of the threshold method above, for example, the perturbation example z' can be obtained by the following method: i From z i We can obtain the mean and standard deviation of d elements in the training set. Then, we sample from d normal distributions with these means and standard deviations as parameters, and calculate z' i Samples that meet the same conditions as i For example, in the example shown in Figure 8, z'2 satisfies the four conditions "x1 ≥ 3," "x2 ≥ 4," "x3 < 1," and "x4 < 5," so a value that satisfies these conditions is randomly generated and set as z2.

[0032] Next, the model f is used to predict f(z i ) is obtained (step S25). i By inputting into f, the prediction f(z i ) is obtained. Next, the proximity π x (z i ) is obtained (step S26). Then, the set Z is <z’ i ,f(z i ),π x (z i )> is added (step S27).

[0033] The above process is repeated N times (step S28), and finally the set Z is output (step S29). This completes the process of step S13.

[0034] (Step S14) Next, the explanation generation unit 15 receives Z as an input and generates an explanation w (vector w) for x. Specifically, z' i are the training examples, f(z i ) is the correct label, π x (z i ) as weights for the examples, train an interpretable model g, and output the parameters of g obtained by training as w.

[0035] Explanation As a calculation method of w, a case where the interpretable model g is a linear model will be described. When the interpretable model g is a linear model, it can be expressed as in the following equation 1.

number

[0036] In this case, the design matrix D of N×d′ is defined by the following formula 2.

number

[0037] Also, a vector f of length N representing the predictions of model f for N perturbation cases. z is defined by the following formula 3.

number

[0038] Furthermore, the case weight matrix Π is defined as an N×N diagonal matrix expressed by the following equation 4.

number

[0039] In this case, the explanation w is w that minimizes the loss function Lw shown in the following equation 5.

number

[0040] The explanation w that minimizes the above loss can be calculated using the following equation (6).

number

number

number

[0041] As described above, we can obtain the explanation w as a differentiable function of the predictions of model f. Because the explanation is differentiable, we can calculate the gradient of the explanation loss with respect to the parameters θ of model f, and update the parameters θ of model f, as described below.

[0042] Here, differences from Non-Patent Document 1 will be described. The explanation generation method by the explanation generation unit of this embodiment is basically similar to Algorithm 1 in Non-Patent Document 1. However, Non-Patent Document 1 employs a model called k-Lasso, which is difficult to differentiate, as the explanation model g. This is because the ultimate purpose of Non-Patent Document 1 is to generate an explanation, and therefore the parameters of model f are not updated using differentiation.

[0043] On the other hand, in this embodiment, the purpose is to update the parameters of f using the generated explanation. Therefore, by using a linear model as g instead of K-Lasso, the explanation w is expressed in the form of a differentiable function of the prediction of model f. This makes it possible to update the parameters of f using the gradient.

[0044] Although a linear model is used in this embodiment, other models may be used as long as the explanation w can be expressed in the form of a differentiable function of the prediction of the model f. This completes the description of the operation of the explanation generation unit 15.

[0045] Next, the overall operation of the information processing device 10 in this embodiment will be described with reference to FIG.

[0046] First, the training example set, correct answer labels, and explanation evaluation criteria that are input to the input unit 11 will be described. The training example set X is a set of M training examples, as shown in Equation 9. Each example is a vector of length d.

number

[0047] The correct label y is a vector of length M that represents the label to be predicted by the model f, as shown in Equation 10.

number

[0048] Next, the objective function of this embodiment will be described. The objective function L is given in the form of a weighted sum of the prediction loss and the explanation loss, as shown in the following formula 11.

number

[0049] To update the parameters for this objective function, P j The gradient of E j In particular, the update formula for gradient descent is as follows:

number

number

[0050] (Step S31) An input unit 11 receives a set of training examples, a correct answer label, and an explanation evaluation criterion. (Step S32) Repeat the following for j=1,...,M. (Step S33) The predicted loss calculation unit 13 calculates the j-th training example x j Calculate the gradient of the predicted loss with respect to For example, in the case of regression, the predicted loss can be calculated using the squared error shown in the following formula 14.

number

number

number

number

[0051] (Step S34) The explanation generation unit 15 generates the j-th training example x j Explanation regarding j The generation method is as explained above.

[0052] (Step S35) The explanation loss calculation unit 14 calculates the j-th training example x j Here, the explanation loss is defined according to an explanation evaluation criterion. The explanation evaluation criterion is a standard that the explanation of the prediction of the machine learning model for the training examples should satisfy. The explanation evaluation criterion may be any criterion that can evaluate the generated explanation. Typically, the explanation evaluation criterion can be the ground truth explanation, which is the explanation that should be output as the explanation of the prediction for the training examples. In the following, the ground truth explanation associated with the j-th training example is defined as v j Let's say. j is a vector of length d'.

[0053] Explanation Loss E j For example, the squared error shown in the following equation (18) can be used as the loss E j is the explanation w generated for the jth training example. j The correct explanation is v j The greater the deviation, the larger the error.

number

number

number

[0054] Explanation of Equation 20 The gradient of the loss can be calculated as shown in Equation 21 below.

number

number

number

number

number

[0055] (Step S36) The above is repeated for M cases, and then the process moves to the next step. (Step S37) Next, the parameter calculation unit 12 calculates the parameter θ using the gradient. In the case of a neural network, the parameter θ can be updated by the gradient descent method. j and E j Since the gradients of have already been calculated as above, we can use them to update using the gradient descent update formula.

[0056] (Step S38) Then, the updated parameters are output. The procedure from steps S32 to S37 may be repeated multiple times, which will update the parameters multiple times, and is expected to reduce the loss and lead to better parameters.

[0057] Next, further possible variations of the first embodiment will be described. (In the case of gradient boosting) Let us consider the case where model f is gradient boosting. In gradient boosting, instead of using the gradient descent update formula, parameters are updated by adding a base learner whose target value is pseudo-residuals. In gradient boosting, parameters can be thought of as being added with each update.

[0058] In conventional gradient boosting, the target value for the jth training example is the pseudo-residual r j is calculated as shown in the following formula 26.

number

[0059] Then, the base learner is trained and added using Equation 27 as a training example set.

number

[0060] When applied to the present disclosure, N perturbation examples shown in the following equation 28 are added per training example to the training example set when training the base learning device.

number

number

[0061] By adding the base learner calculated as above, the parameters are updated so that not only the prediction loss but also the explanation loss decreases.

[0062] (Example of explanatory evaluation criteria) In the above, we have described a method in which the correct explanation associated with each training example is used as the explanation evaluation criterion. However, a different explanation evaluation criterion may be used. For example, the explanation evaluation criterion may be a set of indices representing the elements of the training examples that should be used for the explanation. An example set is as follows: {2,3,5}

[0063] When the above set is given as the explanation evaluation criterion, explanations involving {x2, x3, x5} are given a negative explanation loss. Then, the parameters are updated so that the explanation loss becomes smaller. This allows explanations involving {x2, x3, x5} to be obtained preferentially. For example, the explanation loss shown in the following equation (30) can be used.

number

[0064] Here, as shown in FIG. 3, a correct answer explanation providing unit 20 may be present before the input unit 11. The correct answer explanation providing unit 20 receives a set of training examples and a correct answer label associated with the training example. The correct answer explanation providing unit 20 associates the correct answer explanation with the training example. The correct answer explanation providing unit 20 provides the correct answer explanation to the input unit 11 together with the set of training examples and the correct answer label as an explanation evaluation criterion.

[0065] For example, the explanation when the initial parameter θ before the update is used can be used as the correct explanation. For each training example, an explanation when calculating the prediction using the initial parameter θ is generated and used as the correct explanation. The degree of deviation from this correct explanation is used as the explanation loss.

[0066] Furthermore, the correct explanation does not need to be associated with all training cases included in the training case set. For example, the correct explanation may be associated only with training cases for which an explanation has already been presented to a human, and not with newly added training cases. In other words, the correct explanation may be associated only with training cases for which the predicted value output by the machine learning model for the input of the training case matches the correct label due to the training. This allows the model to be retrained so that the new explanation matches as closely as possible with the explanation already presented to a human, while also making predictions for recently added training cases accurate. In other words, the information contained in the new case can be incorporated into the model while maintaining consistency with explanations presented in the past.

[0067] <Embodiment 2> Next, a second embodiment of the present disclosure will be described with reference to Fig. 10 to Fig. 11. Fig. 10 to Fig. 11 are diagrams for explaining processing operations in the second embodiment.

[0068] The machine learning model targeted in this embodiment is a model f that makes predictions using multiple rules, such as a decision tree or a decision list. In this case, the rules corresponding to the training data in the decision tree or decision list serve as explanations for the predictions. That is, in this embodiment, when making predictions for input training cases using a decision tree or a decision list, the explanation generation unit 15 generates, as explanation data, the determined rules that lead to the predicted values ​​output by the decision tree or decision list. Then, in this embodiment, the parameter calculation unit 12 calculates and learns the rules (parameters) of the decision tree or decision list, as described above, so as to reduce the prediction loss, which is the difference between the output predicted value and a preset correct value, and the explanation loss, which is the difference between the explanation data, which is the determined rule, and the preset correct rule.

[0069] FIG. 10 shows an example of a decision tree. As shown in this figure, a decision tree is configured by branching from a root node to multiple leaf nodes, and multiple rules exist as paths from the root node to each leaf node. Of these, one leaf node finally reached is a predicted value. For a given example, the path from the root node to each node in order to the leaf node that is the predicted value becomes a decision rule, and this decision rule becomes explanation data. Therefore, when a training example is given to a decision tree such as that shown in FIG. 10, if the path indicated by the white arrow becomes a decision rule, the explanation generation unit 15 generates the decision rule "x0 > 1.5 AND x1 ≦ 3.0 AND x2 > 2.0" enclosed in a dotted line box, which indicates the decision rule, as explanation data.

[0070] 11 shows an example of a decision list. As shown in this figure, the decision list is a list in which multiple rules are arranged in order. When making a prediction, the rules are checked from the top down, and one or more rules that satisfy a given case become decision rules, and these decision rules become explanation data. Therefore, when a training case is given to a decision list such as that shown in FIG. 11, if the three rules enclosed in dotted lines become decision rules, the explanation generation unit 15 generates the decision rule "(x0≦1.0 OR X2≧2.0) AND x1≦2.0 AND x2<3.0" as explanation data.

[0071] Then, similar to the first embodiment, the parameter calculation unit 12 in this embodiment calculates rules, which are parameters of a prediction model, that minimize an objective function L, which is made up of a prediction loss, which is the difference between a predicted value and a correct value, and a weighted sum of an explanation loss, which is the difference between the explanatory data, which is the determined rule, and the correct rule. For example, the objective function L can be expressed by the following equation 31.

number

[0072] The above L acc is the prediction loss that represents the prediction error between the predicted value f(t) and the correct value f(fb(t)). acc can be expressed as an error function such as square error. exp is the explanation loss that represents the error between the rule rule(f,t) used in prediction and the correct rule rule(fb,t). For example, L exp The index shown in Equation 32 can be used for

number

[0073] Here, for example, the objective function L can be expressed as a linear programming problem representation of a loss function, thereby allowing the parameter calculation unit 12 to perform calculations. acc +λL exp " is expressed as a linear programming problem. acc The representation method of L differs depending on the prediction model, and the representation methods of decision trees and decision lists are well known. exp can be expressed as follows:

[0074] First, the two-dimensional matrix S={s ij} i=1,...,m,j=1,...,n Each element of s ij L exp (r j ,rule(f b ,t)) In other words, case x i The rules used by the model fb before training to predict and the rules r output during training i The explanation loss is as follows. Also, the two-dimensional matrix D={d ij} i=1,...,m,j=1,...,n Each element d of ij is case x e The explanation used to predict i When using L, it becomes 1, otherwise it becomes 0. exp can be expressed by Equation 33.

number

[0075] In this embodiment, too, a correct explanation may be associated only with a training example in which the predicted value output by the machine learning model for the input of a training example matches the correct label. Thus, for example, the objective function L may be expressed as the following equation (34). Here, the second term is the prediction loss for the correct training example, and the third term is the explanation loss for the correct training example.

number

[0076] <Embodiment 3> Next, a third embodiment of the present disclosure will be described with reference to Fig. 12 and Fig. 13. Fig. 12 and Fig. 13 are block diagrams showing the configuration of an information processing device in embodiment 3. Note that this embodiment shows an outline of the configuration of the information processing device described in the above embodiments.

[0077] First, the hardware configuration of the information processing device 100 in this embodiment will be described with reference to Fig. 12. The information processing device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example. ·CPU(Central Processing Unit)101(Arithmetic unit) ROM (Read Only Memory) 102 (storage device) RAM (Random Access Memory) 103 (storage device) Programs 104 loaded into RAM 103 A storage device 105 for storing a group of programs 104 A drive device 106 that reads and writes from a storage medium 110 external to the information processing device A communication interface 107 that connects to a communication network 111 outside the information processing device Input / output interface 108 for inputting and outputting data Bus 109 connecting each component

[0078] The information processing device 100 can be equipped with an explanation generation unit 121 and a parameter calculation unit 122 shown in Fig. 13 by having the CPU 101 acquire and execute the group of programs 104. The group of programs 104 is stored in advance in, for example, the storage device 105 or the ROM 102, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The group of programs 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the programs and supply them to the CPU 101. However, the explanation generation unit 121 and the parameter calculation unit 122 described above may be constructed using dedicated electronic circuits for realizing such means.

[0079] 12 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as not including the drive device 106. Furthermore, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof, instead of the above-described CPU.

[0080] The explanation generation unit 121 generates explanation data that explains the predicted value output by the machine learning model in response to input training data. As an example, the machine learning model is a model that can update parameters using the gradient of an objective function, and can generate explanation data based on the importance of the training data to the predicted value. As another example, the machine learning model is a model that predicts a predicted value using multiple rules, and can generate explanation data based on the rule to which the training data corresponds in the machine learning model.

[0081] The parameter calculation unit 122 calculates the parameters of the machine learning model so as to reduce a prediction loss, which represents the degree to which a predicted value output by the machine learning model in response to input training data differs from a preset correct value, and an explanation loss, which represents the degree to which explanation data does not satisfy a preset standard that the explanation data should satisfy. For example, when correct answer data for the explanation data is given, the explanation loss is the difference between the generated explanation data and the correct answer data.

[0082] By configuring the present disclosure as described above, it is possible to prevent explanations for predicted values ​​by a machine learning model from differing for each case by updating the parameters of the machine learning model so as to reduce the prediction loss and explanation loss.

[0083] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0084] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, at least one or more functions of the explanation generation unit and parameter calculation unit described above may be executed by an information processing device installed and connected anywhere on a network, i.e., may be executed by so-called cloud computing.

[0085] <Additional Notes> A part or all of the above-described embodiments can be described as follows: The following provides an overview of the configurations of an information processing device, an information processing method, and a program according to the present disclosure. However, the present disclosure is not limited to the following configurations. (Appendix 1) an explanation generation unit that generates explanation data that explains the predicted value output by the machine learning model in response to input training data; a parameter calculation unit that calculates parameters of the machine learning model so as to reduce a prediction loss that indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct answer value, and an explanation loss that indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; An information processing device comprising: (Appendix 2) 10. The information processing device according to claim 1, the parameter calculation unit calculates parameters of the machine learning model so as to reduce the prediction loss and the explanation loss, which represents a degree of difference between the explanation data and preset correct explanation data; Information processing device. (Appendix 3) 10. The information processing device according to claim 1, the parameter calculation unit calculates parameters of the machine learning model so as to reduce the prediction loss and the explanation loss based on a weighted sum of the explanation data consisting of a plurality of elements. Information processing device. (Appendix 4) 10. The information processing device according to claim 1, the explanation generation unit generates the explanation data based on the importance of each element constituting the training data with respect to the predicted value by the machine learning model. Information processing device. (Appendix 5) 5. The information processing device according to claim 4, the explanation generation unit generates the explanation data using a differentiable function obtained using the machine learning model as the importance; The parameter calculation unit calculates parameters of the machine learning model by calculating a gradient of the explanation loss using a derivative of the function. Information processing device. (Appendix 6) 6. The information processing device according to claim 5, the explanation generation unit generates the explanation data using parameters of a second machine learning model as the function when the second machine learning model is trained based on the machine learning model using second training data generated based on the training data; and generate, Information processing device. (Appendix 7) 10. The information processing device according to claim 1, the machine learning model is a model that predicts a predicted value using a plurality of rules, the explanation generation unit generates, as the explanation data, a rule to which the training data corresponds in the machine learning model; the parameter calculation unit calculates the parameters of the machine learning model so as to reduce the prediction loss and the explanation loss, which represents the degree to which the explanation data differs from a preset correct answer rule; Information processing device. (Appendix 8) 10. The information processing device according to claim 2, a correct answer explanation assigning unit that associates the correct answer explanation data with the training data; the correct answer explanation providing unit acquires the training data, a correct answer label corresponding to the training data, and initial parameters of the machine learning model, and associates the explanation data generated when the training data is input to the machine learning model using the initial parameters as parameters of the machine learning model with the training data as the correct answer explanation data; Information processing device. (Appendix 9) 9. The information processing device according to claim 8, the correct answer explanation providing unit associates the correct answer explanation data with only the training data when the predicted value output by the machine learning model in response to the input of the training data matches the correct answer value; Information processing device. (Appendix 10) Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy. Information processing methods. (Appendix 11) Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy. A computer-readable storage medium that stores a program for causing a computer to execute a process. [Explanation of symbols]

[0086] 10. Information processing equipment 11 Input section 12 Parameter calculation section 13 Predicted loss calculation section 14 Explanation loss calculation part 15 Explanation Generation Unit 20 Correct answer explanation section 100 Information processing device 101 CPU 102 ROM 103 RAM 104 Programs 105 Storage device 106 Drive device 107 Communication Interface 108 Input / Output Interface 109 Bus 110 Storage medium 111 Communication Network 121 Explanation Generation Unit 122 Parameter Calculation Unit

Claims

1. an explanation generation unit that generates explanation data that explains a predicted value output by the machine learning model in response to input training data; a parameter calculation unit that calculates parameters of the machine learning model so as to reduce a prediction loss that indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct answer value, and an explanation loss that indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; Equipped with the explanation generation unit generates the explanation data based on the importance of each element constituting the training data with respect to the predicted value by the machine learning model. Information processing device.

2. 2. The information processing device according to claim 1, the explanation generation unit generates the explanation data using a differentiable function obtained using the machine learning model as the importance; The parameter calculation unit calculates parameters of the machine learning model by calculating a gradient of the explanation loss using a derivative of the function. Information processing device.

3. 3. The information processing device according to claim 2, the explanation generation unit generates the explanation data using parameters of a second machine learning model as the function when a second machine learning model based on the machine learning model is trained using second training data generated based on the training data; Information processing device.

4. an explanation generation unit that generates explanation data that explains a predicted value output by the machine learning model in response to input training data; a parameter calculation unit that calculates parameters of the machine learning model so as to reduce a prediction loss that indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct answer value, and an explanation loss that indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; Equipped with the machine learning model is a model that predicts a predicted value using a plurality of rules, the explanation generation unit generates, as the explanation data, a rule to which the training data corresponds in the machine learning model; the parameter calculation unit calculates the parameters of the machine learning model so as to reduce the prediction loss and the explanation loss, which represents the degree to which the explanation data differs from a preset correct answer rule; Information processing device.

5. an explanation generation unit that generates explanation data that explains a predicted value output by the machine learning model in response to input training data; a parameter calculation unit that calculates parameters of the machine learning model so as to reduce a prediction loss that indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct answer value, and an explanation loss that indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; Equipped with the parameter calculation unit calculates parameters of the machine learning model so as to reduce the prediction loss and the explanation loss, which represents a degree of difference between the explanation data and preset correct explanation data; moreover, a correct answer explanation assigning unit that associates the correct answer explanation data with the training data; the correct answer explanation providing unit acquires the training data, a correct answer label corresponding to the training data, and initial parameters of the machine learning model, and associates the explanation data generated when the training data is input to the machine learning model using the initial parameters as parameters of the machine learning model with the training data as the correct answer explanation data; Information processing device.

6. 6. The information processing device according to claim 5, the correct answer explanation providing unit associates the correct answer explanation data with only the training data when the predicted value output by the machine learning model in response to the input of the training data matches the correct answer value; Information processing device.

7. An information processing device, Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; Furthermore, when generating the explanation data, the explanation data is generated based on the importance of each element constituting the training data with respect to the predicted value by the machine learning model. Information processing methods.

8. An information processing device, Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; moreover, the machine learning model is a model that predicts a predicted value using a plurality of rules, When generating the explanation data, a rule to which the training data corresponds in the machine learning model is generated as the explanation data; When calculating the parameters, the parameters of the machine learning model are calculated so as to reduce the prediction loss and the explanation loss, which represents the degree to which the explanation data differs from a preset correct answer rule. Information processing methods.

9. An information processing device, Generate explanatory data that explains the predicted values ​​output by the machine learning model for the input of training data, Calculating parameters of the machine learning model so as to reduce a prediction loss, which indicates the degree to which a predicted value output by the machine learning model in response to the input of the training data differs from a preset correct value, and an explanation loss, which indicates the degree to which the explanation data does not satisfy a preset standard that the explanation data should satisfy; When calculating the parameters, the parameters of the machine learning model are calculated so that the prediction loss and the explanation loss, which represents the degree to which the explanation data differs from preset correct explanation data, are reduced; moreover, When associating the correct explanation data with the training data, the training data, the correct label corresponding to the training data, and the initial parameters of the machine learning model are acquired, and the explanation data generated when the training data is input into the machine learning model using the initial parameters as parameters of the machine learning model is associated with the training data as the correct explanation data. Information processing methods.

Images