Information processing device and information processing method

Abstract

To provide an information processing device and a method for processing information that can interpret results of processing of a mechanical learning model.SOLUTION: The information processing device includes: an extraction unit for extracting a rule on the basis of a training sample group of a mechanical learning model and a result of processing of the training sample group by the mechanical learning model; a screening unit for screening a rule extracted by the extraction unit on the basis of the distribution of the training sample group and the distribution of rules extracted by the extraction unit; and an interpretation unit for interpreting the result of processing of a single sample by the mechanical learning model by using the rule screened by the screening unit. The information processing technique of the present disclosure allows an interpretation of the results of processing by the mechanical learning model on the basis of the correlation between features of different samples.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] FIELD Embodiments of the present disclosure relate to the technical field of information processing, and in particular to an information processing device and an information processing method for interpreting processing results of a machine learning model. [Background technology]

[0002] With the development of machine learning technology, artificial intelligence has been widely applied in various fields. However, many machine learning models, such as artificial neural network models, are usually black-box models. Although these black-box models have very high accuracy rates, their prediction results are difficult to understand and trust because their rules cannot be specifically interpreted. In particular, when black-box models are applied to fields such as finance, security, transportation, and medicine, interpretability is an important measure of whether the black-box model can be trusted. For example, when applying black-box models to the financial field, it is not enough to simply notify stakeholders of the black-box model's prediction results; it is also necessary to inform business personnel about marketing methods and risk managers about how to identify risks. Black-box artificial intelligence models with excellent predictive performance can lead to misleading decisions, introduce systemic risk, and become unsafe because they may expose system vulnerabilities to attack.

[0003] Therefore, a technology that can interpret the processing results of a machine learning model is needed. Figure 1 is a schematic diagram showing an interpreter that interprets the prediction results of a machine learning model. As shown in Figure 1, the machine learning model, which is a black-box model, processes sample data to obtain prediction results, and the interpreter analyzes the features of the sample data to interpret the prediction results of the machine learning model.

[0004] Interpretability techniques for machine learning models may be divided into global and local interpretability techniques. Global interpretability techniques globally interpret the overall behavior of a sample dataset or model, while local interpretability techniques interpret a single instance (sample). Because local data distributions may not match the global data distribution, local interpretability techniques can provide more accurate interpretable results and more customized services for single instances.

[0005] Currently, local interpretability techniques, such as the LIME method and the SHAP method, are being applied. However, all of the current local interpretability techniques are feature-based, that is, they determine how sample features affect the prediction results of a machine learning model. Therefore, these local interpretability techniques ignore the correlation between features, and therefore do not promote user understanding and trust.

[0006] Therefore, there is a need for a technique that can interpret the results of machine learning models, which are highly accurate black-box models, and can obtain interpretable results based on correlations between features for different samples. Summary of the Invention [Problem to be solved by the invention]

[0007] The following presents a simplified summary of the disclosure in order to provide a basic understanding of aspects of the disclosure. However, this summary is not an exhaustive overview of the disclosure, and it is not intended to identify key or important portions of the disclosure or to limit the scope of the disclosure. Rather, it is intended to merely introduce concepts in a simplified form as a prelude to the more detailed description that is presented later.

[0008] In consideration of the problems with the conventional techniques described above, the present disclosure aims to provide an information processing technique that can interpret the processing results of a machine learning model based on the correlation between sample features for different samples. [Means for solving the problem]

[0009] In order to achieve the object of the present disclosure, one aspect of the present disclosure provides an information processing device that interprets processing results of a machine learning model, the information processing device including: an extraction unit that extracts rules based on a training sample set of the machine learning model and processing results of the training sample set by the machine learning model; a screening unit that screens the rules extracted by the extraction unit based on the distribution of the training sample set and the distribution of the rules extracted by the extraction unit; and an interpretation unit that interprets processing results of a single sample by the machine learning model using the rules screened by the screening unit.

[0010] Another aspect of the present disclosure provides an information processing method for interpreting processing results of a machine learning model, the information processing method including the steps of: extracting rules based on a training sample set of the machine learning model and processing results of the training sample set by the machine learning model; screening the extracted rules based on a distribution of the training sample set and a distribution of the extracted rules; and interpreting processing results of a single sample by the machine learning model using the screened rules.

[0011] In yet another aspect of the present disclosure, there is provided a computer program capable of realizing the above-described information processing method, and a computer program product having at least the form of a computer-readable medium on which computer program code for realizing the above-described information processing method is recorded.

[0012] The information processing techniques disclosed herein can provide highly accurate black-box models with interpretable results based on correlations between sample features for different samples. [Brief explanation of the drawings]

[0013] In order to make the above and other objects, features and advantages of the present disclosure more easily understandable, the following describes embodiments of the present disclosure with reference to the drawings. [Figure 1] FIG. 1 is a schematic diagram illustrating an interpreter that interprets the prediction results of a machine learning model. [Figure 2] FIG. 1 is a block diagram illustrating a configuration of an information processing device according to an embodiment of the present disclosure. [Figure 3] 3 is a schematic diagram showing a process executed by an information processing device corresponding to FIG. 2. FIG. [Figure 4] FIG. 3 is a block diagram showing the configuration of the extraction unit shown in FIG. 2. [Figure 5] FIG. 3 is a block diagram showing the configuration of the interpretation unit shown in FIG. 2. [Figure 6] FIG. 10 is a diagram illustrating an example of a processing result of the information processing device according to an embodiment of the present disclosure. [Figure 7] 1 is a flowchart illustrating a process of an information processing method according to an embodiment of the present disclosure. [Figure 8] 1 is a block diagram showing a configuration of a general-purpose device capable of realizing an information processing method and an information processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014]

[0023] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. When reference numerals are used to refer to elements in the drawings, the same reference numerals are used to refer to the same elements in different drawings. In addition, in the following description of the present disclosure, detailed descriptions of known functions and configurations will be omitted in the present disclosure.

[0015] The terms used herein are merely for the purpose of describing particular embodiments and are not intended to limit the disclosure. As used herein, the singular forms "a," "an," and "the" also include the plural forms unless the context dictates otherwise. It should be noted that the terms "comprise," "contain," and "have" as used herein refer to the presence of stated features, entities, operations, and / or components, but do not exclude the presence or addition of one or more other features, entities, operations, and / or components.

[0016] Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by those skilled in the art. Furthermore, terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant field, and should not be interpreted as having an idealized or overly formal meaning unless expressly defined herein.

[0017] In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. The present disclosure may be practiced without some or all of these specific details. In other instances, only components closely related to the configurations according to the present disclosure are shown, and other details unrelated to the present disclosure are omitted to avoid obscuring the present disclosure with unnecessary detail.

[0018] The problem to be solved by the present disclosure is to interpret the processing results of a machine learning model, which is a highly accurate black-box model, and to provide interpretable results based on correlations between features for different samples.

[0019] The configuration and operation of an information processing device according to an embodiment of the present disclosure will be described below. Fig. 2 is a block diagram showing the configuration of an information processing device 200 according to an embodiment of the present disclosure. Fig. 3 is a schematic diagram showing processing executed by the information processing device 200 corresponding to Fig. 2.

[0020] As shown in FIG. 2, the information processing device 200 according to the embodiment of the present disclosure interprets the processing results of the machine learning model.

[0021] In this specification, an application of the information processing device 200 in the financial field will be described as an example. For example, a machine learning model, which is a black-box model, predicts a customer's income status based on the customer's personal information. For example, the machine learning model includes various artificial intelligence models, such as an artificial neural network model. Assuming that a machine learning model is obtained by training using a training sample set D, a training sample d in the training sample set D is defined as an H-dimensional vector, i.e., has H features, and d=(d1, d2, ..., d H ), where H is a natural number. In financial applications, the training sample d may represent customers whose personal information constitutes an H-dimensional vector. Each feature d in the H-dimensional vector h (1≦h≦H) may be gender (Sex), age (Age), education level (Education-Num), marital status (Married), wife (Wife), husband (Husband), whether or not there are children (Own-child), investment income (Capital Gain), or financial service status (Service). A trained machine learning model is used to predict samples x=(x1, x2, ..., x H ) may obtain a prediction result (target variable) f(x), for example, the income situation of the customer. The purpose of the information processing device 200 according to the embodiment of the present disclosure is to interpret the prediction result f(x).

[0022] In the embodiment of the present disclosure, the information processing device 200 may include an extraction unit 201 , a screening unit 202 , and an interpretation unit 203 .

[0023] In an embodiment of the present disclosure, the extraction unit 201 extracts rules based on a training sample set D of a machine learning model and a processing result of the training sample set D by the machine learning model.

[0024] In interpretable techniques, rule-oriented generation methods typically generate rules through global interpretation techniques. Global interpretation techniques describe the behavior of the entire machine learning model, while rule-based methods typically generate rules through interpretable "white-box" tree structures. However, rules generated based on such tree structures typically only reflect the rule-based target distribution of the entire training data set and cannot interpret a single sample to be predicted.

[0025] Therefore, the information processing technique according to the present disclosure first generates rules using a globally interpretable technique, and then screens a set of rules suitable for local interpretation based on different target distributions.

[0026] In an embodiment of the present disclosure, the extraction unit 201 may extract rules using a decision tree method. Using the decision tree method to learn simple decision rules and build a decision tree has strong interpretability. A decision tree is a tree structure, where each internal node represents a decision on a feature, each branch represents the output of the decision result, and finally, each leaf node represents a classification result. The decision tree method starts from the root node and moves to the next internal node according to the branching decision until it reaches a leaf node, and obtains the final classification result. All branches in the process are connected by a logical "AND" operation.

[0027] Decision tree techniques are known to those skilled in the art, and for the sake of brevity, further details of decision tree techniques will not be discussed here.

[0028] Although the embodiments of the present disclosure use a decision tree approach to extract rules, the present disclosure is not limited thereto, and those skilled in the art may adopt other methods to extract rules based on the teachings of the present disclosure.

[0029] Taking the application of the information processing device 200 in the financial field as an example, the extraction unit 201 may obtain a rule having an IF-THEN format by a decision tree method. For example, if (IF) the customer is married (e.g., the feature Married>0.5) and (and) the education level is high (e.g., the feature Education-Num>7.5), (THEN) the customer's income situation is good (e.g., >50,000 USD).

[0030] However, in the decision tree method, the selection of the root node has a certain degree of randomness, so multiple decision trees can generate more accurate rules compared to a single decision tree. Also, when the training samples have many irrelevant features, the performance of the decision tree is not sufficient. Therefore, since the principle of the decision tree is to extract rules layer by layer by finding the correlation between features, when the correlation between features is strong and the corresponding sample data is small, the decision tree may not be able to extract rules well.

[0031] Therefore, in an embodiment of the present disclosure, the extraction unit 201 may divide the sample set D into unrelated subsets, i.e., clusters, by a clustering method. The clusters obtained by the clustering method usually have very similar feature spaces within the clusters, but do not necessarily have the same output targets. Therefore, by generating multiple decision trees for each cluster obtained by clustering, rules with relatively close correlations between different features can be better extracted.

[0032] FIG. 4 is a block diagram showing the configuration of the extraction unit 201 shown in FIG.

[0033] In the embodiment of the present disclosure, as shown in FIG. 4, the extraction unit 201 may include a clustering subunit 2011 and an extraction subunit 2012.

[0034] In an embodiment of the present disclosure, the clustering subunit 2011 may cluster the training sample set D to divide the training sample set D into a plurality of clusters. In an embodiment of the present disclosure, the clustering subunit 2011 may use a K-means clustering method to cluster the training sample set D to obtain K clusters. In an embodiment of the present disclosure, the value of K may be empirically determined based on a specific application scenario. For example, in the application of the information processing device 200 in the financial field, the value of K may be 4 or 5.

[0035] The K-means clustering method is an iterative clustering method. Its steps include first dividing sample data into K groups, randomly selecting K sample data as cluster centers, calculating the distance between each sample data and the cluster center, and assigning each sample data to the cluster center closest to it. Each cluster center and the sample data assigned to it constitute a cluster. Each time sample data is assigned, the cluster center of the cluster is recalculated based on the current sample data in the cluster. The above process is repeated iteratively until a termination condition is met. The termination condition may be, for example, that the change in cluster center between iterations is less than a threshold.

[0036] The K-means clustering method is known to those skilled in the art, and for the sake of brevity, further details of the K-means clustering method will not be discussed here.

[0037] Although the embodiments of the present disclosure use the K-means clustering method to perform clustering, the present disclosure is not limited thereto, and those skilled in the art may adopt other methods to perform clustering based on the teachings of the present disclosure.

[0038] The extraction subunit 2012 may extract a rule for each cluster of the plurality of clusters. In an embodiment of the present disclosure, the extraction subunit 2012 may extract a rule for each cluster using a decision tree approach.

[0039] In addition, in the embodiment of the present disclosure, the extraction subunit 2012 may merge the extracted rules for each cluster to generate a rule list Rulelist, where the rule list Rulelist is a set of rules R1={r1, r2, ... r N}, where N represents the total number of extracted rules. Each rule r n (1≦n≦N) is formed by connecting multiple condition factors s, and r n =s n1 ^s n2 ^…^s nM where M is the depth of the decision tree.

[0040] Here, each condition factor s may be regarded as a feature filter. Taking the application of the information processing device 200 in the financial field as an example, a sample x to be predicted is expressed as x=(x1, x2, ..., x H ) feature x1 represents age (Age) and feature x5 represents marital status (Married). Furthermore, the depth of the decision tree is 2, and the extracted rule r1=s 11 ^s 12 Assuming that, for example, s 11 is 28 years old or less, that is, x1≦28 and s 12 may represent married, i.e., x5="Married" or x5>0.5 (greater than 0.5 represents married, and less than or equal to 0.5 represents unmarried), then rule r1 may represent x1≦28 and x5=“Married” or >0.5.

[0041] It should be noted that, as will be appreciated by those skilled in the art, the clustering process performed by the clustering subunit 2011 is not essential. In other embodiments of the present disclosure, the clustering process may not be performed on the training sample set D, and rules may be directly extracted based on the entire training sample set D.

[0042] Although the rule R1 is obtained by the extraction unit 201, the quality of the extracted rule R1 cannot be guaranteed because the distribution of the training sample set D may be uneven. Therefore, the rule R1 may be screened based on the overall distribution of the training sample set D.

[0043] In an embodiment of the present disclosure, the screening unit 202 may screen the rule R1 extracted by the extracting unit 201 based on the distribution of the training sample set D and the distribution of the rule R1 extracted by the extracting unit 201.

[0044] In an embodiment of the present disclosure, the screening unit 202 may set an index reflecting the quality of the rule R1 extracted by the extraction unit 201 based on the distribution of the training sample set D and the distribution of the rule R1 extracted by the extraction unit 201, and screen the rule D1 extracted by the extraction unit based on the index.

[0045] Therefore, the index may be set taking into consideration both the data distribution between classes of the rule and the data distribution within the class. Note that here, "class" represents the label of a training sample in the training sample set D, i.e., the processing result (prediction result) of the training sample set D by the machine learning model trained using the training sample set D.

[0046] In an embodiment of the present disclosure, a cover D(r) may be defined to represent the set of training samples in the training sample set D that satisfy rule r, and l may be defined to represent the target variable (label) in the training sample set D, where l∈{1, 2, ..., L}, where L is the number of classes (labels). Furthermore, D1 may be defined to represent the set of training samples in the training sample set D whose target variable is l. Therefore, the proportion θ1 of training samples covered by rule r in class l may be defined according to the following equation (1):

number

[0047] where C represents the complement of the set and || represents the number of elements in the set. Therefore, in equation (1), the numerator represents the number of training samples in the training sample set D that have label l and satisfy rule r, and the denominator represents the number of training samples in the training sample set D that have label l and do not satisfy rule r.

[0048] In addition, in equation (1), if the denominator is zero, that is, if the label in the training sample set D is l and the number of training samples that do not satisfy rule r is zero, it means that rule r covers all of the training samples in the training sample set D that are labeled l, that is, it is a high-quality rule, so rule r is directly selected by screening.

[0049] ratio θ l According to the following formula (2), an index for comparing the quality of covering the training samples by the inter-class rule and the intra-class rule r, i.e., max_ration(r) reflecting the quality of the rule r extracted by the extraction unit 201, may be set according to the following formula:

number

[0050] where ￢l means that the label is not l, and θ ￢lThe calculation of θ given by the above equation (1) l The difference is that the numerator represents the number of training samples in the training sample set D whose label is not l and which satisfy the rule r, and the denominator represents the number of training samples in the training sample set D whose label is not l and which do not satisfy the rule r.

[0051] The index max_ration(r) can reflect the quality of the rule r, and the larger the value of max_ration(r), the higher the quality of the rule r. In the embodiment of the present disclosure, a threshold value for the index max_ration(r) may be set, and the rule R1 extracted by the extraction unit 201 may be screened based on the threshold value, and rules with an index max_ration(r) greater than the threshold value may be retained, and other rules may be filtered out. This results in the rule R2 screened by the screening unit 202.

[0052] In the embodiment of the present disclosure, the interpretation unit 203 uses the rule R2 screened by the screening unit 202 to predict a single sample x=(x1, x2, ..., x H ) processing result f(x) may be interpreted as

[0053] Fig. 5 is a block diagram showing the configuration of the interpretation unit shown in Fig. 2. In the embodiment of the present disclosure, the interpretation unit 203 may include a matching subunit 2031, a training subunit 2032, and a determination subunit 2033.

[0054] In the embodiment of the present disclosure, the matching subunit 2031 selects a rule R2 that matches a single sample x to be predicted from among the rules R2 screened by the screening unit 202. x may be determined, where R x is the rule r of the rules R2 screened by the screening unit 202 that matches the sample x to be predicted. i Specifically, Rx may be expressed by the following equation (3):

number

[0055] where 1≦i≦I, I=|R x |, symbol " (outside 1) "TIFF0007743753000004.tif12170" represents the union.

[0056] As mentioned above, rule r i is r i =s i1 ^s i2 ^…^s iM In general, the sample to be predicted x=(x1, x2, ..., x H ) each feature x h belongs to the corresponding interval, so the condition factor s im may be expressed by the following equation (4):

number

[0057] where x im is the condition factor s of the sample x to be predicted im is a feature corresponding to range im is the condition factor s im The interval (l im ,u im ], where l im is the lower bound of the interval, and u im is the upper limit of the interval. Taking the application of the information processing device 200 in the financial field as an example, a sample to be predicted x=(x1, x2, ..., x H ) feature x1 represents age (Age), and the sample x to be predicted follows the rule r2=s 21 ^s 22 For example, s21 indicates that the age is greater than 28 and less than 35. In this example, the feature x1 of the sample x to be predicted satisfies the condition factor s 21Assuming corresponding to range 21 , that is, the conditional factor s 21 The corresponding interval (l 21 , u 21 ) is (28, 35].

[0058] Also, for example, s 22 may represent that the education level (Education-Num) is higher than high school and lower than master's (for example, 7.5 < Education-Num ≤ 12.5). In this example, assuming that the feature x4 of the sample x to be predicted corresponds to the conditional factor s 22 corresponding to range 22 , that is, the conditional factor s 22 The corresponding interval (l 22 , u 22 ) is (7.5, 12.5].

[0059] Conversely, as described above, the rule r2 = s 21 ^s 22 may be expressed as 28 < x1 ≤ 35 and (and) 7.5 < x4 ≤ 12.5. When the features x1 and x4 of the sample x to be predicted satisfy the above conditions, the sample x is determined to satisfy the rule r2.

[0060] In an embodiment of the present disclosure, z i (x) ∈ {0, 1} is defined to represent whether the sample x to be predicted satisfies the rule r i , and when satisfying the rule r i , z i (x) = 1, and when not satisfying the rule r i , z i (x) = 0, and may be expressed by the following formula (5).

Equation

[0061] Here, " (Outer 2) "TIFF0007743753000007.tif12170" means cumulative multiplication. 1range im is range im The specific definition of this is shown in the following equation (6).

number

[0062] In an embodiment of the present disclosure, the training subunit 2032 uses the rule R determined by the matching subunit 2031. x For each of the above, a local interpretation model may be trained using the training sample set D and the results of the machine learning model f(d).

[0063] In an embodiment of the present disclosure, for each rule that the sample x to be predicted determined by the matching subunit 2031 satisfies, a linear regression model may be trained as a local interpretation model.

[0064] In an embodiment of the present disclosure, the linear regression model may be a generalized additive model (GAM). Specifically, the generalized additive model g i (d) is the set cover D (r i ) training samples d=(d1,d2,…,d H ) feature d h , and the rules it satisfies to perform fitting, so that it has the form shown in equation (7).

number

[0065] where 1≦i≦I, 1≦j≦J, I=J=|R x |. If j=i, β j =0. This is the set cover D (r i ) in the above z iAccording to the definition of (x), z of all samples i This is because (x) is 1, so there is no need to enter it into the above formula.

[0066] In the above equation (7), the first term on the right side, α 0 , is an offset term, the second term is related to the features of training sample d in training sample set D and has coefficient α, and the third term is related to the rule and has coefficient β. As shown in the above equation (7), the training sample set D is used to train the generalized additive model.

[0067] In an embodiment of the present disclosure, the training subunit 2032 may use a ridge regression method to train the generalized additive model of the above equation (7). Specifically, the ridge regression model is used as the loss function of the generalized additive model to train the generalized additive model. The loss function has the form shown in the following equation (8).

number

[0068] where λ is the regularization factor of the ridge regression model. In the embodiment of the present disclosure, in the above formula (8), to train the local interpretation model, the training subunit 2032 uses the training sample set D and the processing result (prediction result) f(d) of the training sample set D by the machine learning model.

[0069] Generalized additive models and ridge regression models are known to those skilled in the art, and for the sake of brevity, further details of generalized additive models and ridge regression models will not be discussed here.

[0070] Although the embodiments of the present disclosure use a generalized additive model and a ridge regression model to obtain a local interpretation model, the present disclosure is not limited thereto, and those skilled in the art may adopt other linear regression techniques to obtain a local interpretation model based on the teachings of the present disclosure.

[0071] In an embodiment of the present disclosure, the determining subunit 2033 may determine the contribution of a rule matched to a single sample x to be predicted based on the local interpretation model trained by the training subunit 2032. In an embodiment of the present disclosure, the determining subunit 2033 may determine the coefficients α and β associated with the rules in the above equation (7) through training, and further determine the contribution of a rule R matched to a single sample x to be predicted. x Generate a local interpretation model for each of the x | local interpretation models may be obtained.

[0072] In an embodiment of the present disclosure, the determining subunit 2033 may obtain the contribution w(x) of each local interpretation model according to the following equation (9):

number

[0073] Here, Norm() means normalization in the interval [-1, 1]. As shown in the above equation (9), the contribution w(x) of each local interpretation model for the sample x to be predicted is determined by the coefficient β associated with the rule.

[0074] In an embodiment of the present disclosure, the decision subunit 2033 determines the rule R that matches the single sample x to be predicted. x The rule R matches a single sample x to be predicted based on the contribution w(x) of x The local interpretation models corresponding to the

[0075] FIG. 6 is a diagram showing an example of a processing result of an information processing device according to an embodiment of the present disclosure. The processing result will be explained using an application of the information processing device 200 in the financial field as an example. As shown in FIG. 6, the sample x to be predicted has at least the following features: age (Age), education level (Education-Num), marital status (Married), wife (Wife), whether or not there are children (Own-child), investment gain (Capital Gain), and financial service status (Service). The left part of FIG. 6 shows each rule R that matches the sample x to be predicted. x , and the right part of Fig. 6 shows the corresponding contribution of each rule to the prediction result. The goal of prediction is to determine whether the sample x to be predicted, i.e., the income status of the customer, for example, is higher than 50K USD.

[0076] As shown in Figure 6, the rules Married>0.5 and Service<=0.5 and Married>0.5 and Education-Num>7.5 have the greatest contribution to the predicted results of income status higher than 50K USD, while the rules Capital Gain<=1.5 and Wife<=0.5 have the greatest contribution to the predicted results of income status lower than 50K USD.

[0077] From the above, the information processing device 200 according to the embodiment of the present disclosure can interpret the processing results of the machine learning model, which is a high-precision black-box model, and can provide interpretable results based on the correlations (i.e., rules) between features for different samples. Here, as will be understood by those skilled in the art, when the depth M of the decision tree used by the extraction unit 201 is set to 1, the obtained local interpretation model can be reduced to provide interpretable results based on a single feature of the sample to be predicted.

[0078] FIG. 7 is a flowchart illustrating the processing of an information processing method 700 according to an embodiment of the present disclosure.

[0079] The information processing method 700 starts at step S701. Next, in a determining step S702, a rule is extracted based on a training sample set of a machine learning model and a processing result of the training sample set by the machine learning model. In an embodiment of the present disclosure, the processing in step S702 may be realized by, for example, the extraction unit 201 described with reference to FIGS. 2 to 6 above, and a detailed description thereof will be omitted here.

[0080] Next, in step S703, the extracted rules are screened based on the distribution of the training sample set and the distribution of the extracted rules. The processing in step S703 may be realized by, for example, the screening unit 202 described above with reference to Figures 2 to 6, and detailed description thereof will be omitted here.

[0081] Next, in step S704, the screened rules are used to interpret the processing result of the single sample by the machine learning model. In the embodiment of the present disclosure, the processing in step S704 may be realized by, for example, the interpretation unit 203 described with reference to Figures 2 to 6 above, and detailed description thereof will be omitted here.

[0082] Finally, the information processing method 700 ends at step S705.

[0083] According to the information processing technology disclosed herein, the training samples in a training sample set are clustered, and multiple decision trees are constructed in each cluster obtained by the clustering process to generate multiple rules, thereby improving the diversity of rules. Furthermore, by screening rules taking into account the data distribution between rule classes and the data distribution within a class, rules that are more important for interpreting prediction results can be selected. Furthermore, by generating multiple local interpretation models that correspond one-to-one to the rules matched to a single sample to be predicted, rule-based interpretations of the prediction results of a machine learning model can be obtained. Therefore, the information processing technology disclosed herein first extracts rules globally, and then extracts rule-based interpretable results that are easy to understand through local interpretation of a single sample. Therefore, the information processing technology disclosed herein can provide interpretable results based on the correlation between sample features for different samples to a high-precision black-box model.

[0084] Although the present specification describes the application of the information processing technology of the present disclosure to the financial field as an example, the present disclosure is not limited thereto. The information processing technology of the present disclosure may be widely applied to fields such as security, transportation, medical care, and industrial control.

[0085] 8 is a block diagram showing the configuration of a general-purpose device 800 capable of realizing an information processing method and an information processing device according to an embodiment of the present disclosure. The general-purpose device 800 may be, for example, a computer system. Note that the general-purpose device 800 is merely an example and does not limit the scope of use or functions of the method and device according to the present disclosure. Furthermore, the general-purpose device 800 does not depend on the components or combinations thereof of the information processing method and information processing device described above.

[0086] 8, a central processing unit (CPU) 801 executes various processes according to programs stored in a read-only memory (ROM) 802 or programs loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 stores data necessary for the CPU 801 to execute various processes as needed. The CPU 801, ROM 802, and RAM 803 are connected to one another via a bus 804. An input / output interface 805 is also connected to the bus 804.

[0087] An input unit 806 (including a keyboard, a mouse, etc.), an output unit 807 (including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage unit 808 (including, for example, a hard disk, etc.), and a communication unit 809 (including a network interface card, such as a LAN card, a modem, etc.) are connected to the input / output interface 805. The communication unit 809 performs communication processing via a network, such as the Internet. If necessary, a driver 810 may be connected to the input / output interface 805. A removable medium 811 is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is set up in the driver 810 as necessary, and a computer program read from the removable medium 811 is installed in the storage unit 808 as necessary.

[0088] When the above processing is performed by software, a program constituting the software is installed via a network, such as the Internet, or a storage medium, such as a removable medium 811 .

[0089] 8, which stores the program and provides the program to the user separately from the device. Removable medium 811 includes, for example, a magnetic disk (including a floppy disk), an optical disk (including an optical disk-read only memory (CD-ROM) and a digital versatile disk (DVD)), a magneto-optical disk (MiniDisc (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium may be ROM 802, a hard disk included in storage unit 808, or the like, which stores the program and is provided to the user together with the device containing the program.

[0090] The present disclosure also provides a program product storing machine-readable instruction codes. When the instruction codes are read and executed by a machine, the information processing method according to the present disclosure can be executed. Therefore, the various storage media described above having this program product recorded thereon are also included within the scope of the present disclosure.

[0091] The above detailed descriptions of block diagrams, flowcharts, and / or embodiments illustrate specific embodiments of the apparatus and / or method of the presently disclosed embodiments. When these block diagrams, flowcharts, and / or embodiments include one or more functions and / or operations, each function and / or operation in these block diagrams, flowcharts, and / or embodiments may be implemented individually and / or collectively in hardware, software, firmware, or any combination thereof. In one embodiment, portions of the subject matter described herein may be implemented in an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or other integrated form. It should be noted that all or some aspects of the embodiments described herein may be equivalently implemented in the form of one or more computer programs executed by one or more computers in an integrated circuit (e.g., one or more computer programs executed by one or more computer systems), one or more programs executed by one or more processors (e.g., one or more programs executed by one or more microprocessors), firmware, or substantially any combination thereof. Furthermore, in accordance with the contents disclosed herein, circuitry for designing the present disclosure and / or code for compiling the software and / or firmware of the present disclosure are all within the capabilities of one skilled in the art.

[0092] It should be noted that the terms "comprise" and "have" refer to the presence of one or more features, elements, steps or components described herein, but do not exclude the presence or addition of one or more other features, elements, steps or components. The ordinal terms "first" and "second" do not denote the order of performance or level of importance of the features, elements, steps or components referred to by these terms, but are merely used to distinguish between these features, elements, steps or components.

[0093] Furthermore, the following supplementary notes are also disclosed regarding the embodiments including the above-described examples, but the present invention is not limited to these supplementary notes. (Appendix 1) An information processing device that interprets a processing result of a machine learning model, an extraction unit that extracts rules based on a training sample set of the machine learning model and a processing result of the training sample set by the machine learning model; a screening unit for screening the rules extracted by the extraction unit based on the distribution of the training sample set and the distribution of the rules extracted by the extraction unit; an interpretation unit that uses the rules screened by the screening unit to interpret a result of processing a single sample by the machine learning model. (Appendix 2) 2. The information processing device according to claim 1, wherein the extraction unit extracts rules using a decision tree method. (Appendix 3) The extraction unit comprises: a clustering subunit for clustering the training sample set into a plurality of clusters; an extraction subunit that extracts a rule for each cluster among the plurality of clusters. (Appendix 4) 4. The information processing device of claim 3, wherein the extraction subunit extracts rules for each cluster using a decision tree method. (Appendix 5) 5. The information processing device of claim 4, wherein the extraction subunit generates multiple decision trees for each cluster. (Appendix 6) The information processing device described in Appendix 1 or 2, wherein the screening unit sets an index reflecting the quality of the rules extracted by the extraction unit based on the distribution of the training sample set and the distribution of the rules extracted by the extraction unit, and screens the rules extracted by the extraction unit based on the index. (Appendix 7) The interpretation unit a matching subunit that determines a rule that matches the single sample among the rules screened by the screening unit; a training subunit for training a local interpretation model using the training sample set and the processing results of the machine learning model for each rule determined by the matching subunit; and a determination subunit that determines a contribution of a rule matched to the single sample based on a local interpretation model trained by the training subunit. (Appendix 8) 8. The information processing device of claim 7, wherein the local interpretation model trained by the training subunit is a generalized additive model. (Appendix 9) 9. The information processing device of claim 8, wherein the training subunit trains the generalized additive model using a ridge regression technique. (Appendix 10) 8. The information processing device of claim 7, wherein the determination subunit fuses local interpretation models corresponding to rules matched to the single sample based on contributions of the rules matched to the single sample. (Appendix 11) An information processing method for interpreting a processing result of a machine learning model, comprising: extracting rules based on a training sample set of the machine learning model and a processing result of the training sample set by the machine learning model; screening the extracted rules based on the distribution of the training sample set and the distribution of the extracted rules; and interpreting the results of processing a single sample by the machine learning model using the screened rules. (Appendix 12) 12. The information processing method of claim 11, wherein the rules are extracted using a decision tree technique. (Appendix 13) The step of extracting rules includes: clustering the training sample set to divide the training sample set into a plurality of clusters; 13. The information processing method according to claim 11, further comprising: extracting a rule for each of the plurality of clusters. (Appendix 14) 14. The information processing method of claim 13, wherein a decision tree technique is used to extract rules for each cluster. (Appendix 15) 15. The information processing method of claim 14, wherein multiple decision trees are generated for each cluster. (Appendix 16) 13. The information processing method according to claim 11, further comprising: setting an index reflecting the quality of the extracted rules based on the distribution of the training sample set and the distribution of the rules extracted by the extraction unit; and screening the extracted rules based on the index. (Appendix 17) The step of interpreting the processing result includes: determining which of the screened rules matched the single sample; For each of the determined rules, training a local interpretation model using the training sample set and the processing results of the machine learning model; and determining the contribution of rules matched to the single sample based on the trained local interpretation model. (Appendix 18) 18. The information processing method of claim 17, wherein the trained local interpretation model is a generalized additive model. (Appendix 19) 19. The information processing method of claim 18, wherein the generalized additive model is trained using a ridge regression technique. (Appendix 20) A computer-readable storage medium storing a program, the program causing the computer to execute an information processing method for interpreting a processing result of a machine learning model when the program is executed by the computer, the information processing method comprising: extracting rules based on a training sample set of the machine learning model and a processing result of the training sample set by the machine learning model; screening the extracted rules based on the distribution of the training sample set and the distribution of the extracted rules; and interpreting a result of processing a single sample by the machine learning model using the screened rules.

[0094] Although the above describes specific embodiments of the present disclosure, those skilled in the art may make various modifications, improvements, or equivalents to the present disclosure within the spirit and scope of the appended claims, and these modifications, improvements, or equivalents belong to the protection scope of the present disclosure.

Claims

1. An information processing device that interprets a processing result of a machine learning model, an extraction unit that extracts rules based on a training sample set of the machine learning model and a processing result of the training sample set by the machine learning model; a screening unit for screening the rules extracted by the extraction unit based on the distribution of the training sample set and the distribution of the rules extracted by the extraction unit; an interpretation unit that uses the rules screened by the screening unit to interpret a processing result of a single sample by the machine learning model; The interpretation unit a matching subunit that determines a rule that matches the single sample among the rules screened by the screening unit; a training subunit for training a local interpretation model using the training sample set and the processing results of the machine learning model for each rule determined by the matching subunit; a determining subunit that determines a contribution of a rule that matched the single sample based on the local interpretation model trained by the training subunit.

2. The information processing device according to claim 1 , wherein the extraction unit extracts the rules using a decision tree method.

3. The extraction unit comprises: a clustering subunit for clustering the training sample set into a plurality of clusters; The information processing apparatus according to claim 1 , further comprising: an extraction subunit that extracts a rule for each of the plurality of clusters.

4. The information processing apparatus according to claim 3 , wherein the extraction subunit extracts rules for each cluster using a decision tree method.

5. 3. The information processing device according to claim 1, wherein the screening unit sets an index reflecting the quality of the rules extracted by the extraction unit based on a distribution of the training sample set and a distribution of the rules extracted by the extraction unit, and screens the rules extracted by the extraction unit based on the index.

6. The information processing device according to claim 1 , wherein the local interpretation model trained by the training subunit is a generalized additive model.

7. The information processing device according to claim 6 , wherein the training subunit trains the generalized additive model using a ridge regression technique.

8. The information processing device according to claim 1 , wherein the decision subunit fuses local interpretation models corresponding to rules matched to the single sample based on contributions of the rules matched to the single sample.

9. An information processing method for interpreting a processing result of a machine learning model, executed by an information processing device, comprising: extracting rules based on a training sample set of the machine learning model and a processing result of the training sample set by the machine learning model; screening the extracted rules based on the distribution of the training sample set and the distribution of the extracted rules; and interpreting the results of processing a single sample by the machine learning model using the screened rules; The step of interpreting the results of processing a single sample by the machine learning model includes: determining which of the screened rules matched the single sample; For each of the determined rules, training a local interpretation model using the training sample set and the processing results of the machine learning model; determining a contribution of a rule matched to the single sample based on the trained local interpretation model.

Images