Information processing program, method and device

Abstract

To explain the basis for the estimation of the machine learning model from multiple standpoints in a way that is both convincing and reliable.SOLUTION: An information processor extracts from the estimation object data the partial data corresponding to each of the multiple high-frequency patterns the index of which according to the frequency of occurrence in the multiple training samples is greater than or equal to the second threshold, and the contribution of which to the estimation of the machine learning model is a pattern of a combination of one or more feature quantities the contribution of which is greater than or equal to the first threshold, calculates the likelihood of the estimated result of each high-frequency pattern when the extracted partial data is inputted into the corresponding partial model, then outputs the partial data selected based on the likelihood as the basis for estimating the machine learning model for the estimation object data.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The disclosed technology relates to an information processing program, an information processing method, and an information processing device. [Background technology]

[0002] In recent years, various systems using AI (Artificial Intelligence) estimation have been widely introduced in various fields such as medicine and finance. AI estimation is sometimes called classification. As such, while technologies using AI estimation become widespread, black-box AI such as deep learning has an aspect that it is difficult to understand how the estimation is made and the basis for the estimation.

[0003] In certain fields, it is important not only to know the results of a prediction, but also to know how the prediction was made. For example, in the medical field, it is important to predict the illness a patient has and present the basis for determining that the illness is present, rather than just presenting the name of the illness. As a more specific example, by clarifying what part of an X-ray image the doctor focused on to make the prediction, doctors and patients can be convinced. In this way, in order to convince others of a prediction that has been made, it is important to share the basis for the prediction, including the process.

[0004] Conventionally, various methods for clarifying the basis for such an estimation made by an AI have been studied. For example, an information processing device has been proposed that outputs information that makes it easy to understand the basis for an estimation made by a machine learning model. This information processing device acquires the contribution of each of the relationships between multiple nodes included in a graph structure indicating the relationships between the nodes to the estimation result of the machine learning model. In addition, the information processing device displays a graph that connects a first structure indicating a first class to which one or more nodes in the graph structure belong, and a second structure indicating a first node that belongs to the first class and has an associated contribution equal to or greater than a threshold.

[0005] For example, a system is proposed for ranking subgraphs as potential explanations of a labeled edge type class. The system obtains a first graph representing labeled digital items, each represented as an entity node connected to a property value node via a labeled edge type. The first graph is combined with a second graph representing structured relationships within the labeled digital items to obtain a combined graph. The system also receives unlabeled digital items and matches them with their respective subgraphs in the combined graph. The system also uses a machine learning model to embed the combined graph to generate a graph vector, and generates an expressiveness score between the matching subgraphs and each labeled edge type based on the generated graph vector. The system then ranks the matched subgraphs based on their expressiveness scores to obtain a set of ranked subgraphs as potential explanations of each labeled edge type class. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2022-111841 A [Patent Document 2] U.S. Pat. No. 1,144,2963 Summary of the Invention [Problem to be solved by the invention]

[0007] However, in a technology that presents features with a contribution rate equal to or greater than a threshold as the basis for estimation, it may only capture and explain a part of the estimation that makes it possible, and may be unconvincing. For example, in this technology, as an explanation of the basis for estimating that a picture of a cat is a cat, only the eyes and ears may be presented from among the features in the picture. In addition, in this technology, in estimating numbers from an image of handwritten characters, as an explanation of the basis for estimating the number 5, only a characteristic part of the number 5 may be presented. In other words, in this technology, when estimation is possible with a specific feature, even if other features can also supplement the explanation, those other features are not presented as an explanation of the basis for estimation.

[0008] In addition, the technique for obtaining a set of subgraphs ranked as potential explanations for a labeled edge type class adds supplementary information to the graph information, refines the explanations, and then ranks them, i.e., the technique combines graphs to supplement the original information explained from a single perspective, rather than adding new parallel inference evidence.

[0009] In one aspect, the disclosed technology aims to explain persuasive and reliable inference grounds for inferences made by machine learning models from multiple perspectives. [Means for solving the problem]

[0010] In one embodiment, the disclosed technology identifies a plurality of patterns, and extracts partial data corresponding to each of the identified plurality of patterns from estimation target data including a plurality of features that are the subject of estimation processing using a machine learning model. The patterns are patterns of combinations of one or more features from a plurality of learning samples including a plurality of features, the contribution degree of which to the estimation of the machine learning model is equal to or greater than a first threshold, and the index according to the occurrence frequency in the plurality of learning samples is equal to or greater than a second threshold. The disclosed technology also calculates the likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern. The disclosed technology then outputs the partial data selected based on the likelihood calculated for each partial model for each pattern as an estimation basis of the machine learning model for the estimation target data. Effect of the Invention

[0011] One aspect is that it has the effect of being able to explain the basis for inferences from machine learning models from multiple perspectives, providing persuasive and reliable inference grounds. [Brief description of the drawings]

[0012] [Figure 1] 1 is a functional block diagram of an information processing device according to an embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram illustrating an example of a learning sample. [Diagram 3] 11 is a diagram for explaining the process of a specification unit; FIG. [Figure 4] FIG. 11 is a diagram for explaining the processing of the second training section. [Diagram 5] FIG. 11 is a diagram for explaining the process of a determination unit. [Figure 6] FIG. 13 is a diagram for explaining the process of an extraction unit. [Figure 7] 11 is a diagram for explaining the processing of an estimation unit and an explanation unit. FIG. [Figure 8]FIG. 1 is a block diagram showing an example of a schematic configuration of a computer that functions as an information processing device. [Figure 9] 1 is a flowchart illustrating an example of a machine learning process. [Figure 10] 11 is a diagram for explaining the process of a specification unit; FIG. [Figure 11] FIG. 11 is a diagram for explaining the processing of the second training section. [Figure 12] FIG. 11 is a diagram for explaining the process of a determination unit. [Figure 13] 13 is a flowchart illustrating an example of an estimated explanation process. [Figure 14] FIG. 11 is a diagram for explaining the effect of the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings.

[0014] 1, the information processing device 10 functionally includes a machine learning unit 20 and an inference explanation unit 40. A partial model set 30 is stored in a predetermined storage area of ​​the information processing device 10. Each of the machine learning unit 20 and the inference explanation unit 40 will be described in detail below.

[0015] First, a description will be given of the machine learning unit 20. The machine learning unit 20 is a functional unit that functions during machine learning. During machine learning, a plurality of learning samples are input to the information processing device 10.

[0016] As shown in Fig. 2, a learning sample 90 includes data 92 including feature amounts for each of a plurality of items, and a label 94 indicating a correct answer for the data. In the example of Fig. 2, the data 92 is graph data in which a plurality of items are represented by hierarchical nodes (nodes represented by circles in Fig. 2) and the feature amounts of each item are represented by nodes (nodes represented by squares in Fig. 2) that connect to the nodes representing the items. In the graph data, nodes corresponding to items in a hierarchical relationship, and nodes corresponding to items and feature amounts, are connected by edges.

[0017] The example in FIG. 2 is an example of a learning sample 90 that assumes that a task such as confirmation of a person's behavior is executed by a machine learning model. In this case, data 92 is configured with parts of the human body as items and information indicating the characteristics of the movements of the parts as features. In the graph data in FIG. 2, "person" is the root node, and nodes indicating "head" and "body", which are major classifications of parts of the human body, are connected to the root node. In addition, nodes indicating "forehead", "eyes", and "mouth", which are subclassifications of parts of the head, are connected to the node indicating "head". In addition, nodes indicating "back", "waist", "hands", and "feet", which are subclassifications of parts of the body, are connected to the node indicating "body".

[0018] To the node of each item, a feature node for that item is connected. The feature is information acquired, for example, by analyzing a video of a person. However, in this embodiment, it is assumed that a learning sample 90 in the form of graph data as shown in FIG. 2 can be acquired, and the method of acquiring the feature is not limited. In addition, in the graph data, a plurality of feature nodes may be connected to a node indicating one item. Also, for nodes to which nodes indicating items are connected below ("head" and "body" in the example of FIG. 2), a feature node does not necessarily need to be connected.

[0019] In the example of FIG. 2, the label 94 is information indicating whether the behavior of the person indicated by the data 92 is a behavior to be estimated by the machine learning model (e.g., shoplifting behavior, etc.). For example, if the behavior of the person indicated by the data 92 is a behavior to be estimated, the label may be set to "1", and if the behavior is not a behavior to be estimated, the label may be set to "0". In addition, if the task of the machine learning model is to classify the behavior of the person into one of multiple behaviors, the label 94 may be an identifier of each behavior.

[0020] As shown in FIG. 1, the machine learning unit 20 includes a first training unit 22, a specification unit 24, a second training unit 26, and a judgment unit .

[0021] The first training unit 22 trains the machine learning model using a plurality of learning samples 90. In order to distinguish the machine learning model trained by the first training unit 22 from the partial models trained by the second training unit 26 described later, the machine learning model is hereinafter referred to as the "global model." The global model may be configured with a deep neural network capable of handling graph data. Specifically, the first training unit 22 optimizes parameters of the global model such that an estimation result obtained by inputting data 92 of each of the learning samples 90 into the global model matches a label 94 of the learning sample 90.

[0022] Furthermore, when the first training unit 22 receives a new learning sample (described in detail later) from the determination unit 28 described later, the first training unit 22 trains a new overall model using the new learning sample. Hereinafter, the overall model trained in the iterative process is denoted as "overall model_i." In other words, the overall model trained with the learning sample 90 input to the information processing device 10 is overall model_1.

[0023] The identification unit 24 identifies pattern candidates indicating a combination of one or more feature quantities whose contribution to the estimation of the overall model is equal to or greater than a threshold value TH1 from each of the multiple learning samples 90. Furthermore, the identification unit 24 identifies, from among the identified pattern candidates, pattern candidates whose index according to the occurrence frequency in the multiple learning samples 90 is equal to or greater than a threshold value TH2 as high-frequency patterns. The threshold value TH1 is an example of a "first threshold value" of the disclosed technology, and the threshold value TH2 is an example of a "second threshold value" of the disclosed technology.

[0024] More specifically, as shown in A of FIG. 3, the specification unit 24 inputs data 92 for each of some or all of the multiple learning samples 90 into the overall model_1, and obtains an estimation result, its estimation score, and the contribution of each feature to the estimation result. The estimation score is a value (likelihood) indicating the likelihood that the estimation result is correct. For example, when the estimation result of the overall model is output as a probability value, the probability value may be used as the estimation score. For the contribution, for example, SHAP (SHapley Additive exPlanations) or the like may be applied. Although not shown in the figure, the contribution of the feature is assigned to an edge connecting a node indicating an item and a node indicating the feature of the item.

[0025] The identification unit 24 identifies features whose contribution rate is equal to or greater than a threshold value TH1 from the data 92 of each learning sample 90, as shown by the dashed line in B of FIG. 3. In FIG. 3, edges to which a contribution rate equal to or greater than the threshold value TH1 is assigned are represented by lines thicker than other edges. The identification unit 24 identifies, as pattern candidates, combinations of items corresponding to the identified features from each learning sample 90. The identification unit 24 calculates a frequency score for each pattern candidate, which increases as the pattern candidate is identified from more learning samples 90. The frequency score is an example of an index according to the frequency of appearance in multiple learning samples 90.

[0026] For example, for a pattern candidate identified for each learning sample 90, the identification unit 24 calculates a value obtained by multiplying the estimated score for that learning sample 90 by the sum of the contributions of the features for the items included in the pattern candidate. The identification unit 24 calculates a frequency score for the candidate pattern by dividing the sum of values ​​similarly calculated for the learning samples 90 in which the same pattern candidate is identified by the number of learning samples. As shown in C of FIG. 3, the identification unit 24 identifies a pattern candidate having a frequency score equal to or greater than a threshold value TH2 as a high-frequency pattern. In C of FIG. 3, the "high-frequency pattern" column for a candidate pattern identified as a high-frequency pattern is represented by "1," and the "high-frequency pattern" column for a candidate pattern not identified as a high-frequency pattern is represented by "0."

[0027] In addition, when the identification unit 24 is instructed by the judgment unit 28 described later to execute identification of a high-frequency pattern based on a new overall model, the identification unit 24 identifies a high-frequency pattern similar to the above using each of the data 92 of some or all of the new learning samples 90.

[0028] The second training unit 26 trains a machine learning model for each high-frequency pattern using a partial learning sample obtained by extracting features corresponding to each of the high-frequency patterns identified by the identification unit 24 from each of the multiple learning samples 90.

[0029] Specifically, the second training unit 26 extracts a path from the node of the feature of each item included in each high-frequency pattern to the root node from the data 92 of each learning sample 90. The second training unit 26 creates a partial learning sample by assigning a label 94 of the original learning sample 90 to a subgraph that aggregates the paths extracted for each item included in the high-frequency pattern. Hereinafter, the partial learning sample for the j-th high-frequency pattern created in the ith repeated process is denoted as "partial learning sample_ij".

[0030] As shown in Fig. 4, the second training unit 26 uses each of the partial learning samples_ij to train a machine learning model in the same manner as the first training unit 22 trains the overall model. Hereinafter, the machine learning model trained using the partial learning samples_ij is referred to as a "partial model_ij". The second training unit 26 adds the trained partial model to the partial model set 30.

[0031] The determination unit 28 creates multiple new learning samples by excluding the partial learning sample_ij portion from each of the multiple learning samples 90, as shown in A of Fig. 5. Hereinafter, the learning sample used in the i-th repeated process will be referred to as "learning sample_i", and the new learning sample by excluding the partial learning sample_ij portion from the learning sample_i will be referred to as "learning sample_i+1". The determination unit 28 passes the created learning sample_i+1 to the first training unit 22, and instructs training of a new overall model_i+1. As a result, the new overall model_i+1 is trained by the first training unit 22, as shown in B of Fig. 5.

[0032] When the first training unit 22 trains a new overall model, the determination unit 28 calculates the accuracy of the new trained overall model and determines whether the accuracy is equal to or greater than a threshold value TH3. The threshold value TH3 is an example of a "third threshold" in the disclosed technology. The accuracy may be, for example, the accuracy rate of the estimation result by the overall model. When the accuracy is equal to or greater than the threshold value TH3, the determination unit 28 instructs the identification unit 24 to identify a high-frequency pattern based on the new overall model. The determination unit 28 repeats the creation of a new learning sample, the instruction to the first training unit 22 to train the overall model, and the instruction to the identification unit 24 to identify a high-frequency pattern based on the new overall model, until the accuracy becomes less than the threshold value TH3. The partial model set 30 at the stage when the accuracy becomes less than the threshold value TH3 becomes the final partial model set 30.

[0033] Next, the inference explanation unit 40 will be described. The inference explanation unit 40 is a functional unit that functions during the inference process. During the inference process, inference target data is input to the information processing device 10. The inference target data is similar to the learning sample 90, except that a label 94 is not assigned to the data.

[0034] As shown in FIG. 1, the estimation explanation unit 40 includes an extraction unit 42, an estimation unit 44, and an explanation unit 46.

[0035] As shown in Fig. 6, the extraction unit 42 extracts from the estimation target data one or more combinations of feature amounts indicating each of the high-frequency patterns corresponding to each partial model included in the partial model set 30. Specifically, the extraction unit 42 extracts from the estimation target data, for each item included in each high-frequency pattern identified by the identification unit 24, a path tracing from the node of the feature amount of that item to the root node. The extraction unit 42 aggregates the paths extracted for each item included in the high-frequency pattern to create a partial graph for each high-frequency pattern. The partial graph is an example of "partial data" in the disclosed technology.

[0036] As shown in A of Fig. 7, the estimation unit 44 inputs each of the partial graphs extracted by the extraction unit 42 into a partial model of a corresponding high-frequency pattern included in the partial model set 30, and obtains an estimated score. The estimation unit 44 calculates the average of the estimated scores of each partial model as an estimation result. Furthermore, the estimation unit 44 may obtain an estimation result that is TRUE when the average of the estimated scores of each partial model is equal to or greater than a predetermined value, and FALSE when the average is less than the predetermined value. Furthermore, the estimation unit 44 is not limited to using the average of the estimated scores of each partial model, and may use a median, etc.

[0037] As shown in B of FIG. 7, the explanation unit 46 specifies a subgraph selected based on the estimation score calculated by the estimation unit 44 as an estimation basis for the estimation result by the partial model set 30 for the estimation target data. Specifically, the explanation unit 46 selects a subgraph input to a partial model having an estimation score equal to or greater than a threshold TH4. The threshold TH4 is an example of a "fourth threshold" in the disclosed technology. Furthermore, as shown in C of FIG. 7, when there are multiple selected subgraphs, the explanation unit 46 aggregates the multiple subgraphs to form a final estimation basis, and outputs it together with the estimation result estimated by the estimation unit 44. Note that the explanation unit 46 may output the multiple selected subgraphs as an estimation result without aggregating them.

[0038] The information processing device 10 may be realized by, for example, a computer 50 shown in Fig. 8. The computer 50 includes a CPU (Central Processing Unit) 51, a GPU (Graphics Processing Unit) 52, a memory 53 as a temporary storage area, and a non-volatile storage device 54. The computer 50 also includes an input / output device 55 such as an input device and a display device, and an R / W (Read / Write) device 56 that controls reading and writing of data from and to a storage medium 59. The computer 50 also includes a communication I / F (Interface) 57 that is connected to a network such as the Internet. The CPU 51, the GPU 52, the memory 53, the storage device 54, the input / output device 55, the R / W device 56, and the communication I / F 57 are connected to one another via a bus 58.

[0039] The storage device 54 is, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage device 54 as a storage medium stores an information processing program 60 for causing the computer 50 to function as the information processing device 10. The information processing program 60 has a first training process control instruction 62, a specific process control instruction 64, a second training process control instruction 66, a judgment process control instruction 68, an extraction process control instruction 72, an estimation process control instruction 74, and an explanation process control instruction 76. The storage device 54 also has an information storage area 80 in which information constituting the partial model set 30 is stored.

[0040] The CPU 51 reads out the information processing program 60 from the storage device 54, expands it in the memory 53, and sequentially executes the control instructions of the information processing program 60. The CPU 51 operates as the first training unit 22 shown in FIG. 1 by executing the first training process control instruction 62. The CPU 51 operates as the identification unit 24 shown in FIG. 1 by executing the identification process control instruction 64. The CPU 51 operates as the second training unit 26 shown in FIG. 1 by executing the second training process control instruction 66. The CPU 51 operates as the determination unit 28 shown in FIG. 1 by executing the determination process control instruction 68. The CPU 51 operates as the extraction unit 42 shown in FIG. 1 by executing the extraction process control instruction 72. The CPU 51 operates as the estimation unit 44 shown in FIG. 1 by executing the estimation process control instruction 74. The CPU 51 operates as the explanation unit 46 shown in FIG. 1 by executing the explanation process control instruction 76. Furthermore, the CPU 51 reads information from the information storage area 80, and loads each partial model included in the partial model set 30 into the memory 53. In this way, the computer 50 that has executed the information processing program 60 functions as the information processing device 10. The CPU 51 that executes the program is hardware. Furthermore, part of the program may be executed by the GPU 52.

[0041] The functions realized by the information processing program 60 may be realized, for example, by a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0042] Next, the operation of the information processing device 10 according to the present embodiment will be described. During machine learning, the information processing device 10 executes the machine learning process shown in Fig. 9, and during estimation processing, the information processing device 10 executes the estimation explanation process shown in Fig. 13. Note that the machine learning process and the estimation explanation process are examples of the information processing method of the disclosed technology.

[0043] First, the machine learning process shown in Fig. 9 will be described. Here, the description will be given according to specific examples shown in Figs. 3 to 5 and Figs. 10 to 12.

[0044] In step S10, the first training unit 22 acquires a plurality of learning samples 90 (learning samples_1) input to the information processing device 10, and trains the overall model_1 using the plurality of learning samples_1. Next, in step S12, the determination unit 28 calculates the accuracy of the trained overall model_1 trained in the above step S10, and determines whether the accuracy is equal to or greater than a threshold value TH3. If the accuracy is equal to or greater than TH3, the process proceeds to step S14.

[0045] In step S14, the identification unit 24 inputs data 92 of some or all of the multiple learning samples_1 into the overall model_1, as shown in A of Figure 3, and obtains the estimation result and its estimation score, as well as the contribution of each feature to the estimation result.

[0046] Next, in step S16, the identification unit 24 identifies features whose contributions are equal to or greater than a threshold value TH1 from the data 92 of each learning sample_1, as shown by the dashed line in B of Fig. 3, and identifies combinations of items corresponding to the identified features as pattern candidates. The identification unit 24 then calculates a frequency score for each pattern candidate based on the estimated score and the contributions, and identifies pattern candidates whose frequency scores are equal to or greater than a threshold value TH2 as high-frequency patterns, as shown in C of Fig. 3.

[0047] Next, in step S18, the second training unit 26 extracts a path from the feature node of each item included in each high-frequency pattern to the root node from the data 92 of each learning sample_1. Then, the second training unit 26 creates a partial learning sample_1-j by assigning the label 94 of the original learning sample 90 to a subgraph that aggregates the paths extracted for each item included in the high-frequency pattern.

[0048] Next, in step S20, the second training unit 26 trains each of the partial models _1-1, _1-2, ... using each of the partial learning samples _1-1, _1-2, ... for each high-frequency pattern, as shown in Fig. 4. Then, the second training unit 26 adds the trained partial models _1-1, _1-2, ... to the partial model set 30.

[0049] Next, in step S22, the judgment unit 28 creates multiple new learning samples_2 by excluding each of the partial learning samples_1-1, partial learning samples_1-2, ... from each of the multiple learning samples_1, as shown in A of Fig. 5. Then, the judgment unit 28 passes the created learning samples_2 to the first training unit 22, instructs training of the overall model_2, and returns to step S10. As a result, in step S10, the first training unit 22 trains a new overall model_2, as shown in B of Fig. 5.

[0050] Next, in step S12, the determination unit 28 calculates the accuracy of the new trained overall model_2 and determines whether the accuracy is equal to or greater than a threshold TH3. For example, as shown in C of FIG. 5, if the calculated accuracy is 90% and the threshold TH3 is 80%, the process proceeds to step S14 again.

[0051] In step S14, the identification unit 24 inputs data 92 of some or all of the multiple new learning samples_2 into the overall model_2, as shown in A of Figure 10, and obtains the estimation result and its estimation score, as well as the contribution of each feature to the estimation result.

[0052] Next, in step S16, the identification unit 24 identifies features whose contributions are equal to or greater than a threshold value TH1 from the data 92 of the new learning sample_2, as shown by the dashed line in B of Fig. 10, identifies pattern candidates in the same manner as above, and calculates a frequency score for each pattern candidate. Then, the identification unit 24 identifies pattern candidates whose frequency scores are equal to or greater than a threshold value TH2 as high-frequency patterns, as shown in C of Fig. 10.

[0053] Next, in step S18, the second training unit 26 creates a partial learning sample_2-1 corresponding to a high-frequency pattern from the data 92 of the new learning sample_2 in the same manner as described above. Next, in step S20, the second training unit 26 trains a partial model_2-1 using the partial learning sample_2-1, and adds the trained partial model_2-1 to the partial model set 30, as shown in FIG.

[0054] Next, in step S22, the determination unit 28 creates multiple new learning samples_3 by excluding the partial learning sample_2-1 from each of the learning samples_2, as shown in A of Fig. 12, passes them to the first training unit 22, and instructs training of the overall model_3. Then, returning to step S10, the first training unit 22 trains the new overall model_3 using the learning sample_3, as shown in B of Fig. 12.

[0055] Next, in step S12, the determination unit 28 calculates the accuracy of the overall model_3 and determines whether the accuracy is equal to or greater than a threshold TH3. For example, as shown in C of Fig. 12, if the calculated accuracy is 60% and the threshold TH3 is 80%, the determination in this step is negative, and the machine learning process ends.

[0056] Next, the estimation explanation process shown in Fig. 13 will be described. Here, the explanation will be given according to the specific example shown in Figs.

[0057] In step S30, the extraction unit 42 acquires the inference target data input to the information processing device 10. Then, the extraction unit 42 extracts, from the inference target data, partial graphs indicating each of the high-frequency patterns corresponding to each partial model included in the partial model set 30, as shown in Fig. 6 .

[0058] Next, in step S32, the estimation unit 44 inputs each of the subgraphs extracted by the extraction unit 42 into a partial model of the corresponding high-frequency pattern, as shown in A of Fig. 7, and obtains an estimated score. Then, the estimation unit 44 calculates an average of the estimated scores of each partial model as an estimation result.

[0059] Next, in step S34, the explanation unit 46 selects subgraphs that have been input to the partial model with an estimation score equal to or greater than the threshold TH4, as shown in B of Fig. 7, and aggregates the selected subgraphs to generate a final estimation basis, as shown in C of Fig. 7. Next, in step S36, the explanation unit 46 outputs the estimation result estimated by the estimation unit 44 and the generated estimation basis, and the estimation explanation process ends.

[0060] As described above, the information processing device in this embodiment identifies pattern candidates that are combinations of one or more feature amounts whose contribution to the estimation of the overall model is equal to or greater than a threshold value TH1 from a plurality of learning samples including a plurality of feature amounts. Then, the information processing device identifies a high-frequency pattern from the pattern candidates based on the occurrence frequency in the plurality of learning samples. Also, the information processing device extracts a subgraph corresponding to each of the high-frequency patterns from the estimation target data. Also, the information processing device calculates an estimation score for a partial model for each high-frequency pattern when the partial graph extracted from the estimation target data is input to a partial model that is trained for each high-frequency pattern and corresponds to the partial graph extracted from the estimation target data. Then, the information processing device presents the subgraph selected based on the estimation score calculated for each partial model for each high-frequency pattern as the estimation grounds of the machine learning model for the estimation target data. This makes it possible to explain the estimation grounds with high persuasiveness and reliability from multiple perspectives.

[0061] Specifically, if the feature amount with a high contribution is simply used as the estimation basis, for example, only the partial graph on the left in the lower part of FIG. 6 is output, and only the estimation basis from one perspective is output. On the other hand, the information processing device according to this embodiment identifies a high-frequency pattern that appears frequently in multiple learning samples by combining the amount of summiting with a high contribution from the learning samples, as shown in FIG. 14. Then, the information processing device according to this embodiment explains the estimation basis using a partial model modeled for each high-frequency pattern. Therefore, when multiple high-frequency patterns exist, multiple partial models are generated, making it possible to explain the estimation basis from various perspectives.

[0062] For example, in the example of Fig. 14 showing a task of estimating shoplifting behavior as a person's behavior estimation, high frequency pattern A is a pattern that represents the characteristics of suspicious behavior, and high frequency pattern B is a pattern that represents the characteristics of tension. In this case, the basis for estimating a person's behavior as shoplifting behavior can be explained from multiple perspectives, such as suspicious behavior, tension, etc.

[0063] In the above embodiment, the information processing program is stored (installed) in advance in the storage device, but this is not limiting. The program according to the disclosed technology may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

[0064] Regarding the above embodiment, the following supplementary notes are further disclosed.

[0065] (Appendix 1) extracting partial data corresponding to each of a plurality of patterns, each of which is a combination of one or more feature quantities whose contribution to an estimation by a machine learning model is equal to or greater than a first threshold value, from a plurality of learning samples including a plurality of feature quantities, the partial data corresponding to each of a plurality of patterns having an index according to an appearance frequency in the plurality of learning samples equal to or greater than a second threshold value, from estimation target data including a plurality of feature quantities that is to be subjected to estimation processing using the machine learning model; Calculating a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; outputting the partial data selected based on the likelihood calculated for each of the partial models for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing program for causing a computer to execute processing including the above.

[0066] (Appendix 2) 2. The information processing program according to claim 1, for causing the computer to execute a process further including: identifying the plurality of patterns from each of the plurality of learning samples; and training a partial model for each of the patterns using partial learning samples obtained by extracting partial data corresponding to each of the identified plurality of patterns.

[0067] (Appendix 3) The information processing program of claim 2, wherein training the partial model includes extracting partial data corresponding to a new pattern from the multiple new learning samples, and training the partial model corresponding to the new pattern, until accuracy of an estimation result of a new machine learning model trained using multiple new learning samples excluding the partial data used in training the partial model from each of the multiple learning samples becomes less than a third threshold.

[0068] (Appendix 4) An information processing program according to any one of Supplementary Note 1 to Supplementary Note 3, wherein selecting the partial data based on the likelihood includes selecting the partial data input to the partial model whose likelihood is equal to or greater than a fourth threshold value.

[0069] (Appendix 5) The information processing program according to any one of Supplementary Note 1 to Supplementary Note 4, wherein outputting the estimation basis includes aggregating and outputting the plurality of partial data selected based on the likelihood.

[0070] (Appendix 6) The information processing program according to any one of Appendices 1 to 5, wherein the learning sample and the estimation target data are graph data representing the plurality of feature quantities using a plurality of nodes and edges connecting the nodes, the partial data are partial graphs of graph data representing the learning sample or the estimation target data, and the machine learning model and the partial model are machine learning models capable of handling graph data.

[0071] (Appendix 7) from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns are identified which are combinations of one or more feature amounts whose contribution to an estimation by a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value; and partial data corresponding to each of the identified plurality of patterns are extracted from estimation target data including a plurality of feature amounts which is to be subjected to estimation processing using the machine learning model; Calculating a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; outputting the partial data selected based on the likelihood calculated for each of the partial models for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing method in which a computer executes a process including the steps of:

[0072] (Appendix 8) 8. The information processing method according to claim 7, further comprising: identifying the plurality of patterns from each of the plurality of learning samples; and training a partial model for each of the patterns using partial learning samples obtained by extracting partial data corresponding to each of the identified plurality of patterns.

[0073] (Appendix 9) The information processing method of claim 8, wherein training the partial model includes extracting partial data corresponding to a new pattern from the multiple new learning samples, and training the partial model corresponding to the new pattern, until accuracy of an estimation result of a new machine learning model trained using multiple new learning samples excluding the partial data used in training the partial model from each of the multiple learning samples becomes less than a third threshold.

[0074] (Appendix 10) An information processing method described in any one of Supplementary Notes 7 to 9, wherein selecting the partial data based on the likelihood includes selecting the partial data input to the partial model whose likelihood is greater than or equal to a fourth threshold.

[0075] (Appendix 11) The information processing method according to any one of Supplementary Note 7 to Supplementary Note 10, wherein outputting the estimation basis includes aggregating and outputting the plurality of partial data selected based on the likelihood.

[0076] (Appendix 12) The information processing method according to any one of Supplementary Note 7 to Supplementary Note 11, wherein the learning sample and the data to be estimated are graph data in which the plurality of feature quantities are represented by a plurality of nodes and edges connecting the nodes, the partial data are partial graphs of graph data representing the learning sample or the data to be estimated, and the machine learning model and the partial model are machine learning models capable of handling graph data.

[0077] (Appendix 13) an extraction unit that identifies, from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns that are combinations of one or more feature amounts whose contribution to an estimation by a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value, and extracts partial data corresponding to each of the identified plurality of patterns from estimation target data including a plurality of feature amounts that is to be subjected to estimation processing using the machine learning model; an estimation unit that calculates a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; an explanation unit that outputs the partial data selected based on the likelihood calculated for each partial model for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing device comprising:

[0078] (Appendix 14) The information processing device according to claim 13, further comprising a training unit that identifies the plurality of patterns from each of the plurality of learning samples, and trains a partial model for each of the patterns using partial learning samples obtained by extracting partial data corresponding to each of the identified plurality of patterns.

[0079] (Appendix 15) The information processing device described in Appendix 14, wherein the training unit extracts partial data corresponding to a new pattern from the multiple new learning samples and trains a partial model corresponding to the new pattern until accuracy of an estimation result of a new machine learning model trained using multiple new learning samples excluding the partial data used to train the partial model from each of the multiple learning samples becomes less than a third threshold.

[0080] (Appendix 16) 16. The information processing device according to any one of claims 13 to 15, wherein the explanation unit selects the partial data input to the partial model for which the likelihood is equal to or greater than a fourth threshold.

[0081] (Appendix 17) 17. The information processing device according to any one of claims 13 to 16, wherein the explanation unit aggregates and outputs the plurality of partial data selected based on the likelihood.

[0082] (Appendix 18) The information processing device according to any one of Supplementary Note 13 to Supplementary Note 17, wherein the learning sample and the estimation target data are graph data representing the plurality of feature amounts using a plurality of nodes and edges connecting the nodes, the partial data are partial graphs of graph data representing the learning sample or the estimation target data, and the machine learning model and the partial model are machine learning models capable of handling graph data.

[0083] (Appendix 19) from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns are identified which are combinations of one or more feature amounts whose contribution to an estimation by a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value; and partial data corresponding to each of the identified plurality of patterns are extracted from estimation target data including a plurality of feature amounts which is to be subjected to estimation processing using the machine learning model; Calculating a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; outputting the partial data selected based on the likelihood calculated for each of the partial models for each pattern as an estimation basis of the machine learning model for the estimation target data; A storage medium storing an information processing program for causing a computer to execute a process including the above. [Explanation of symbols]

[0084] 10. Information processing device 20 Machine Learning Department 22 1st Training Department 24 Specific part 26 2nd Training Department 28 Judgment section 30 Submodel Set 40 Estimated Explanation Section 42 Extraction part 44 Estimation part 46 Explanation Section 50 Computer 51 CPU 52 GPU 53 Memory 54 Storage device 55 Input / Output Devices 56 R / W device 57 Communication I / F 58 Bus 59 Storage medium 60 Information Processing Program 62 First Training Process Control Instructions 64 Specific Process Control Instructions 66 Second Training Process Control Instructions 68 Decision Process Control Instructions 72 Extraction process control instructions 74 Estimation Process Control Instructions 76 Explanation Process Control Instructions 80 Information storage area 90 training samples

Claims

1. from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns are identified which are combinations of one or more feature amounts whose contribution to an estimation of a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value; and partial data corresponding to each of the identified plurality of patterns are extracted from estimation target data including a plurality of feature amounts which is to be subjected to estimation processing using the machine learning model; Calculating a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; outputting the partial data selected based on the likelihood calculated for each of the partial models for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing program for causing a computer to execute processing including the above.

2. 2. The information processing program according to claim 1, for causing the computer to execute a process further including training a partial model for each of the plurality of patterns using a partial learning sample obtained by extracting partial data corresponding to each of the plurality of patterns from each of the plurality of learning samples.

3. 3. The information processing program according to claim 2, wherein training the partial model includes extracting partial data corresponding to a new pattern from the multiple new learning samples, and training the partial model corresponding to the new pattern, until the accuracy of an estimation result of a new machine learning model trained using multiple new learning samples excluding the partial data used to train the partial model from each of the multiple learning samples becomes less than a third threshold.

4. An information processing program according to any one of claims 1 to 3, wherein selecting the partial data based on the likelihood includes selecting the partial data input to the partial model whose likelihood is equal to or greater than a fourth threshold value.

5. 4. The information processing program according to claim 1, wherein outputting the estimation basis includes aggregating and outputting the plurality of partial data selected based on the likelihood.

6. The information processing program according to any one of claims 1 to 3, wherein the learning sample and the data to be estimated are graph data representing the plurality of feature quantities using a plurality of nodes and edges connecting the nodes, the partial data are partial graphs of graph data representing the learning sample or the data to be estimated, and the machine learning model and the partial model are machine learning models capable of handling graph data.

7. from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns are identified which are combinations of one or more feature amounts whose contribution to an estimation of a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value; and partial data corresponding to each of the identified plurality of patterns are extracted from estimation target data including a plurality of feature amounts which is to be subjected to estimation processing using the machine learning model; Calculating a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; outputting the partial data selected based on the likelihood calculated for each of the partial models for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing method in which a computer executes a process including the steps of:

8. an extraction unit that identifies, from a plurality of learning samples including a plurality of feature amounts, a plurality of patterns that are combinations of one or more feature amounts whose contribution to an estimation of a machine learning model is equal to or greater than a first threshold value, and whose index according to an appearance frequency in the plurality of learning samples is equal to or greater than a second threshold value, and extracts partial data corresponding to each of the identified plurality of patterns from estimation target data including a plurality of feature amounts that is to be subjected to estimation processing using the machine learning model; an estimation unit that calculates a likelihood of an estimation result of a partial model for each pattern when the partial data extracted from the estimation target data is input to a corresponding partial model among partial models trained for each pattern; an explanation unit that outputs the partial data selected based on the likelihood calculated for each partial model for each pattern as an estimation basis of the machine learning model for the estimation target data; An information processing device comprising:

Images