Information processing method, program, and information processing device

Abstract

Provided is an information processing method or the like for assisting in understanding of explanatory data created using an XAI technology.　In this information processing method, a computer executes a process for: acquiring explanatory data relating to a learning model (22) that receives input of an explanation data vector xn and outputs a target data vector yn, the explanatory data being created using an XAI technology that describes the behavior of the learning model (22); inputting a prompt including the explanatory data to a language model (24); and outputting an explanation from the language model (24).

Description

Information processing method, program, and information processing device

[0001] The present invention relates to an information processing method, a program, and an information processing device.

[0002] Various AI (Artificial Intelligence) technologies have been developed, including XAI (Explainable Artificial Intelligence) technology, which enables an explanation of why an output generated by an AI was generated (Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2023-83555

[0004] It is said that in order to widely utilize AI in the real world, XAI technology that can explain the AI ​​is also necessary. If users can understand the reasons for the output of the AI, they can make appropriate decisions on whether to adopt the output of the AI, whether to review the input data to the AI, etc.

[0005] However, to understand the explanation data created using XAI technology, specialized knowledge about AI and XAI technology is required, so it is difficult for general users who are not experts to understand the explanation data created using XAI technology, and AI remains a black box.

[0006] In one aspect, an object of the present invention is to provide an information processing method, etc. that supports the understanding of explanatory data created using XAI technology.

[0007] The information processing method involves a computer acquiring explanatory data about a learning model created using XAI technology that explains the behavior of the learning model, which accepts input of an explanatory data vector and outputs a target data vector, inputting a prompt including the explanatory data into a language model, and outputting an explanation from the language model.

[0008] In one aspect, an information processing method or the like can be provided that supports the understanding of explanatory data created using XAI technology.

[0009] FIG. 1 is an explanatory diagram outlining the procedure for creating an explanation using XAI technology. FIG. 1 is an explanatory diagram outlining the configuration of an information processing device. FIG. 2 is a flowchart outlining the processing flow of a program. FIG. 2 is an explanatory diagram outlining AIME. FIG. 3 is an explanatory diagram outlining a method for calculating an interpretation matrix A†. FIG. 4 is a flowchart outlining the processing flow of a program for calculating an interpretation matrix A†. FIG. 3 is an explanatory diagram outlining the configuration of a learning model. FIG. 4 is an explanatory diagram outlining the record layout of an explanation-purpose DB. FIG. 5 is an explanatory diagram outlining the processing after calculating an interpretation matrix A†. FIG. 6 is an example of a global feature importance graph. FIG. 7 is an example of a local feature importance graph.

[0010] [First Embodiment] A machine learning model that receives input of explanatory data and outputs target data is generated using various machine learning algorithms. The generated machine learning model is usually a black box, and it is difficult for humans to interpret the decision-making process from input of explanatory data to output of target data.

[0011] However, when utilizing machine learning models for real-world decision-making, it is important that users can interpret the decision-making process of the machine learning model. For example, if the output target data appears to deviate significantly from common sense, users can appropriately interpret the decision-making process of the machine learning model to determine how to handle the target data and the machine learning model.

[0012] Techniques for explaining the behavior of machine learning models are called XAI techniques. Various XAI techniques have been proposed, including LIME (Local Interpretable Model Agnostic Explanations), which is suitable for explaining the local behavior of machine learning models, SHAP (Shapley Additive Explanation), which is suitable for explaining the global behavior of machine learning models, and Grad-CAM, which uses a heat map display to show which parts of an image are the basis for the recognition results of an image recognition model.

[0013] 1 is an explanatory diagram outlining the procedure for creating an explanation using XAI technology. The following describes the procedure for creating an explanation regarding the behavior of the learning model 22. The learning model 22 accepts an input of an explanation data vector xn and outputs a target data vector yn.

[0014] The learning model 22 is a machine learning model generated using any supervised machine learning algorithm, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a transformer, a random forest, or a light gradient boosting machine (LightGBM). The learning model 22 may also be a machine learning model generated using any unsupervised machine learning algorithm, such as reinforcement learning or a generative adversarial network (GAN).

[0015] The explanation data vector xn is a one-dimensional sequence of explanation data. The target data vector yn is a one-dimensional sequence of target data. Even complex data such as image data and text data can be expressed as a one-dimensional sequence by appropriately defining conversion rules.

[0016] XAI techniques are used to generate explanatory data about the behavior of the learning model 22, such as global feature importance and local feature importance. Global feature importance describes the overall behavior of the learning model 22.

[0017] Local feature importance indicates the degree of contribution of each element of the target explanation vector x when a target explanation vector x, which is an explanatory data vector x relating to a specific target, is input to the learning model 22 and a target objective vector y is output from the learning model 22, i.e., the reason why the target objective vector y was output based on the target explanation vector x.

[0018] The explanatory data created by the XAI technology can be represented by a diagram such as a global feature importance graph 41, a local feature importance graph 42, a similarity distribution map 43, or a heat map. Various types of diagrams have been proposed depending on the algorithm of the XAI technology. Details of the global feature importance graph 41 and the local feature importance graph 42 will be described later. Explanation of other types of diagrams will be omitted.

[0019] Note that explanatory data created using XAI technology also includes numerical table data in formats such as CSV (Comma Separated Values), which can be used to create these charts. Users can visualize the numerical table data using, for example, spreadsheet software or graph creation software. Experienced users may be able to directly read the necessary information from the numerical table data.

[0020] A prompt containing explanatory data created using XAI technology and information about the learning model 22 is input to the language model 24. The language model 24 is, for example, a large language model (LLM) such as GPT-4, Claude, Copilot, or Gemini. Specific examples of the prompt are described below. A natural language explanation is output from the language model 24.

[0021] Even users who do not have sufficient specialized knowledge of AI and XAI technology can understand explanations created using XAI technology by referring to the natural language explanations generated by the language model 24. Therefore, general users can make appropriate use of AI without being overly cautious or overly trusting it.

[0022] 2 is an explanatory diagram illustrating the configuration of the information processing device 10. The information processing device 10 includes a control unit 11, a main memory device 12, an auxiliary memory device 13, a communication unit 14, a display unit 15, an input unit 16, a reading unit 19, and a bus.

[0023] The control unit 11 is an arithmetic and control device that executes the program of this embodiment. The control unit 11 uses one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), multi-core CPUs, etc. The control unit 11 is connected to each hardware unit that constitutes the information processing device 10 via a bus.

[0024] The main memory device 12 is a storage device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc. The main memory device 12 temporarily stores information required during processing performed by the control unit 11 and programs currently being executed by the control unit 11.

[0025] The auxiliary storage device 13 is a storage device such as an SRAM, a flash memory, a hard disk, or a magnetic tape. The auxiliary storage device 13 stores the learning model 22, the language model 24, a program 97 to be executed by the control unit 11, and various data required for executing the program 97. The learning model 22 and the language model 24 may be stored in an external storage device connected via a network.

[0026] The communication unit 14 is an interface for communication between the information processing device 10 and a network. The display unit 15 is, for example, a liquid crystal display device or an organic EL (Electro Luminescence) display device. The input unit 16 is, for example, an input device such as a keyboard, a mouse, a trackball, or a microphone.

[0027] The portable recording medium 96 is, for example, a USB (Universal Serial Bus) memory, a CD-ROM (Compact Disc Read Only Memory), a magneto-optical disk medium, other optical disk medium, an SD memory card, etc. A program 97 is stored in the portable recording medium 96.

[0028] The reading unit 19 is an interface that can connect to a portable recording medium 96, such as a USB connector, a CD-ROM drive, or an SD memory reader. The semiconductor memory 98 stores a program 97 and is a memory that can be attached inside the information processing device 10.

[0029] The information processing device 10 is an information device used by a user, such as a general-purpose personal computer, a tablet, a smartphone, etc. The information processing device 10 may be configured as a combination of a mainframe computer, a virtual machine running on a mainframe computer, a plurality of personal computers operating in cooperation with each other, or a cloud computing system, and an information device used by the user.

[0030] The control unit 11 reads the program 97 via the reading unit 19 and stores it in the auxiliary storage device 13. The control unit 11 may also read out the program 97 stored in the semiconductor memory 98. Furthermore, the control unit 11 may download the program 97 from another server computer (not shown) connected via the communication unit 14 and a network (not shown) and store it in the auxiliary storage device 13.

[0031] The program 97 is installed as a control program for the information processing device 10, and is loaded into and executed by the main storage device 12. The program 97 of this embodiment is an example of a program product.

[0032] 3 is a flowchart illustrating the processing flow of the program 97. The control unit 11 acquires information related to the XAI technology to be used (step S601). The information related to the XAI technology is specifically information related to the specifications of the description data to be created using the XAI technology.

[0033] The explanatory data created using XAI techniques may be, for example, global feature importance for the learning model 22. The explanatory data created using XAI techniques may also be local importance for why a target objective vector y is output when a target explanation vector x is input to the learning model 22.

[0034] For example, if the explanatory data is output as a graph, the information includes the meaning of each of the vertical and horizontal axes, the legend of the plotted graph, etc. If the explanatory data is related to a specific explanatory data vector xn or target data vector yn, the information on the XAI technology includes information about the explanatory data vector xn or target data vector yn, more specifically, the meaning of each element that makes up these vectors, the value of each element, etc.

[0035] The control unit 11 acquires explanatory data related to the learning model 22 created using XAI technology (step S602). The explanatory data related to the learning model 22 is created, for example, by other hardware connected via a network. The explanatory data related to the learning model 22 may be created by the control unit 11.

[0036] The control unit 11 generates a prompt (step S603) by combining the information acquired in step S601 and the explanation data acquired in step S602 with a template stored in the auxiliary storage device 13, for example. Specific examples of templates will be described later.

[0037] The control unit 11 inputs the prompt to the language model 24 and acquires an explanation output from the language model 24 (step S604). The control unit 11 outputs the acquired explanation to the display unit 15 (step S605). It is desirable that the control unit 11 output the explanation data acquired in step S602 together with the explanation acquired in step S604. The control unit 11 may output the explanation data and the explanation to separate hardware. Thereafter, the control unit 11 ends the processing.

[0038] According to this embodiment, the user can understand the behavior of the learning model 22 by viewing the explanation data created using XAI technology while referring to the explanation generated by the language model 24. Therefore, the user can appropriately determine whether or not to trust the target data vector yn output from the learning model 22. In other words, according to this embodiment, the user can be assisted so that they can appropriately understand the explanation data created using XAI technology.

[0039] The template used in step S603 may be prepared for each user's knowledge level, for example. By specifying the desired level, the user can obtain an explanation that is easy for the user to understand.

[0040] [Embodiment 2] In this embodiment, AIME (Approximate Inverse Model Explanations) is used for the XAI technology. AIME is an information processing method that supports user understanding by explaining the behavior of a machine learning model from various perspectives. Explanations of parts common to embodiment 1 will be omitted.

[0041] Fig. 4 is an explanatory diagram illustrating an overview of AIME. Using Fig. 4, a procedure for creating explanatory data for a learning model 22, which is a generated machine learning model, using AIME will be described. As described above, the learning model 22 is a machine learning model that accepts an input of an explanatory data vector xn and outputs a target data vector ynn.

[0042] The procedure of inputting the explanation data vector xn into the learning model 22 and obtaining the target data vector yn is repeated multiple times. If the learning model 22 is generated by supervised machine learning, some or all of the training data used in the machine learning may be used as the explanation data vector xn. The explanation data vector xn may be generated randomly or based on a predetermined rule.

[0043] A two-dimensional matrix, explanation matrix X, is generated by arranging multiple explanation data vectors xn in the row direction. Similarly, a two-dimensional matrix, objective matrix Y, is generated by arranging multiple objective data vectors yn in the row direction. Here, the arrangement order of the explanation data vectors xn and the arrangement order of the corresponding objective data vectors yn are the same.

[0044] In order to execute the subsequent processes, the target data vectors yn must be linearly independent, i.e., the vector product of the target matrix Y and its transpose must be a regular matrix.

[0045] Based on the explanation matrix X and the objective matrix Y, an interpretation matrix A† is calculated, which is a matrix whose vector product with the objective matrix Y is equal to the explanation matrix X, as shown in equation (1). The interpretation matrix A† is an example of explanation data related to the learning model 22. Details of the method for calculating the interpretation matrix A† will be described later. X = A†Y (1)

[0046] By using the interpretation matrix A†, it is possible to create explanatory data regarding the behavior of the learning model 22, such as a global feature importance graph 41 that shows the overall behavior of the learning model 22, a local feature importance graph 42 that explains why each target data vector yn is output, and a similarity distribution diagram 43 that shows the distribution of similarities between explanatory data vectors xn. Specific examples of how to use the interpretation matrix A† will be described later.

[0047] 5 is an explanatory diagram illustrating a method for calculating the interpretation matrix A†. In the following description, N is a natural number indicating the number of times the process of inputting the explanation data vector xn into the learning model 22 and obtaining the target data vector yn is repeated. n is a natural number indicating the number of times the vector is input to or output from the learning model 22.

[0048] The explanation data vector xn has L elements from Ex1n to ExLn. The target data vector yn has M elements from Ob1n to ObMn, where L and M are natural numbers. In Figure 5, the explanation data vector xn and the target data vector yn, where n=2, are surrounded by dashed lines.

[0049] As described above, the explanation matrix X, which is a two-dimensional matrix, is created by arranging the N explanation data vectors xn obtained through N processing steps in the row direction. The explanation matrix X is a two-dimensional matrix with L rows and N columns. Similarly, the target matrix Y, which is a two-dimensional matrix, is generated by arranging the N target data vectors yn in the row direction in an order corresponding to the explanation data vectors xn. The target matrix Y† is a two-dimensional matrix with M rows and N columns.

[0050] For the target matrix Y, Y†, which is the Moore-Penrose generalized inverse matrix, is calculated. In the following explanation, Y† may be referred to as the target inverse matrix Y†. The target inverse matrix Y† is calculated using equation (2). The target inverse matrix Y† is a matrix with N rows and M columns. Y† = Y T (YY T ) -1 ... (2)

[0051] The interpretation matrix A† is the vector product of the explanation matrix X and the inverse objective matrix Y†. The formula for calculating the interpretation matrix A† is shown in formula (3). A† = XY† (3)

[0052] The interpretation matrix A† is a two-dimensional matrix with L rows and M columns, i.e., the number of rows is the same as the number of elements in the explanation data vector xn, and the number of columns is the same as the number of elements in the target data vector yn. The element in the a-th row and b-th column of the interpretation matrix A† indicates the influence of the a-th element of the explanation data vector xn on the b-th element of the target data vector yn.

[0053] As described above, even if there is no information such as the algorithm and training data that generated the learning model 22, the interpretation matrix A† can be generated as long as there is an environment in which the learning model 22 can be used. In other words, even if the learning model 22 is in a black box state generated by a third party, the interpretation matrix A† can be generated.

[0054] For reference, the outline of the transformation to derive equation (3) from equations (1) and (2) is shown below. First, both sides of equation (1) are multiplied by the transpose of the target matrix Y from the right to obtain equation (4). XY T = A†YY T ... (4)

[0055] As mentioned above, the vector product of the objective matrix Y and the transposed matrix of the objective matrix Y is a regular matrix, so the inverse matrix can be calculated. This inverse matrix is ​​multiplied from the right of both sides of equation (4) to obtain equation (5). XY T (YY T ) -1 = A†(YY T ) (YY T ) -1 = A† ‥‥‥ (5)

[0056] After swapping the left and right sides of equation (5), we can substitute equation (2) into the right side to obtain equation (6). Equation (3) is derived from both ends of equation (6). A† = XY T (YY T ) -1 = XY† ‥‥‥ (6)

[0057] 6 is a flowchart illustrating the processing flow of a program for calculating the interpretation matrix A†. In the following explanation, an example will be described in which the control unit 11 calculates the interpretation matrix A†. However, the interpretation matrix A† may also be calculated, for example, by a mainframe computer, a virtual machine running on a mainframe computer, or a cloud computing system, and transferred to the information processing device 10 via a network.

[0058] The control unit 11 determines the explanatory data vector x to be input to the learning model 22 for the nth time (step S501). If the training data used for the machine learning of the learning model 22 is available, the learning model 22 may extract the explanatory data vector x from the training data.

[0059] The control unit 11 may generate the explanatory data vector xn randomly or based on a predetermined rule. When the control unit 11 generates the explanatory data vector xn, it is desirable to generate the explanatory data vector xn within the range in which the learning model 22 is expected to be used. For example, if the control unit 11 generates an explanatory data vector xn that includes unexpected elements such as "the age of a human being is 200 years," it will be impossible to correctly explain the behavior of the learning model 22.

[0060] Similarly, it is desirable that the distribution of the multiple explanatory data vectors x matches the expected range of use of the learning model 22. For example, in the case of the learning model 22 predicting the behavior of a typical adult, it is desirable that the element of "human age" be close to the distribution of data such as demographics.

[0061] The control unit 11 inputs the explanatory data vector x acquired in step S501 into the learning model 22 to acquire the target data vector yn (step S502). The control unit 11 associates the explanatory data vector xn with the target data vector yn and records them in the main storage device 12 or the auxiliary storage device 13 (step S503).

[0062] The control unit 11 determines whether to end the generation of pairs of the explanation data vector x and the target data vector yn (step S504). For example, the control unit 11 determines to end the process in step S504 when the control unit 11 has finished processing the explanation data vector x recorded in the training data. The control unit 11 may also determine to end the process when a predetermined number of pairs have been generated.

[0063] If it is determined not to end (NO in step S504), the control unit 11 returns to step S501. If it is determined to end (YES in step S504), the control unit 11 generates an explanation matrix X based on the data recorded in step S503 (step S505).

[0064] The control unit 11 generates a target matrix Y based on the data recorded in step S503 (step S506). The control unit 11 calculates a target inverse matrix Y†, which is a Moore-Penrose generalized inverse matrix, based on the target matrix Y (step S507). The control unit 11 calculates an interpretation matrix A†, which is the vector product of the explanation matrix X and the target inverse matrix Y† (step S508). The control unit 11 then ends the process.

[0065] The following provides an overview of a method for explaining the behavior of the learning model 22 using the generated interpretation matrix A†.

[0066] [Global Feature Importance] Global feature importance refers to the degree to which each element constituting the explanatory data vector x contributes to a specific prediction result output from the learning model 22. Global feature importance explains the overall behavior of the learning model 22.

[0067] As mentioned above, the element in row a, column b of the interpretation matrix A† indicates the influence of the a-th element of the explanation data vector xn on the b-th element of the target data vector yn. For example, the degree of influence that each element of the explanation data vector xn has on a specific item in the target data vector yn can be visualized using a bar graph or line graph in which the item names corresponding to each element of the explanation data vector xn are used on the first axis and the values ​​of any column of the interpretation matrix A† are used on the second axis. Specific examples of charts that visualize global feature importance will be described later.

[0068] [Local Feature Importance] The local feature importance vector L represents the degree to which each element constituting a specific explanatory data vector x contributes to the target data vector yn output from the learning model 22. The local feature importance vector L is a vector having the same number of elements as the explanatory data vector xn.

[0069] In the following description, a specific explanation data vector xn will be referred to as a target explanation vector xo. The target explanation vector xo may be data included in the training data or data created separately from the training data. By inputting the target explanation vector xo into the learning model 22, a target data vector yn corresponding to the target explanation vector xo is output from the learning model 22.

[0070] Here, to calculate the degree to which each element of the target explanation vector x contributes to each element of the target data vector yn, calculations are performed assuming that the target data vector yn is a unit target vector yuk. The definition of the i-th element yuk(i) of the unit target vector yuk is shown in equation (7).

[0071]

[0072] When the target data vector yn is the unit target vector yuk, the local feature importance vector L is calculated by equation (8).

[0073]

[0074] For example, the local feature importance vector L can be visualized using a bar graph or line graph in which the first axis represents the item names corresponding to the elements constituting the local feature importance vector L, i.e., the item names corresponding to the elements constituting the explanation data vector x, and the second axis represents the values ​​of the elements constituting the local feature importance vector L. Specific examples of diagrams for visualizing the local feature importance vector L will be described later.

[0075] The local feature importance vector L explains why the target data vector yn was output from the learning model 22 after receiving the explanatory data vector xn as input. The local feature importance vector L also explains the elements of the explanatory data vector xn that affect the target data vector yn.

[0076] The data on global feature importance and local feature importance described above are examples of explanatory data on the learning model 22 created using AIME. A user with sufficient knowledge of the learning model 22 and AIME can quickly understand the behavior of the learning model 22 by observing graphs or the like that illustrate the global feature importance and local feature importance.

[0077] However, as AI becomes more widely used, there is a possibility that users may not have sufficient knowledge. In the following explanation, a concrete example will be given of explanation data created using AIME and a method of utilizing the language model 24 to enable users to understand the explanation data.

[0078] [Specific example using the Census Income Dataset (hereinafter referred to as the "income dataset")] The income dataset is a dataset extracted from the 1994 U.S. Census Bureau database and is one of the datasets frequently used in machine learning research, etc. The income dataset contains data on the characteristics and annual income of approximately 300,000 people. Annual income is divided into two classes: under $50,000 and over $50,000. Table 1 shows an excerpt of some of the features included in the income dataset. Note that the feature names and feature values ​​are shown as they are in the original data, and explanations of each item are provided in Japanese.

[0079]

[0080] The following describes an example of a learning model 22 that is generated by a machine learning algorithm using a census database as training data. FIG. 7 is an explanatory diagram illustrating the configuration of the learning model 22.

[0081] The explanatory data vector xn has elements corresponding to each feature shown in Table 1. For features whose values ​​are numeric, such as "Age," the name of the feature is used as the name of the element in the explanatory data vector xn, and the corresponding numeric value is used as the value of the element. For features whose values ​​can be selected alternatively, such as "Education," the name of the feature and its value are concatenated with an underscore, such as "Education_HS-grad," and the name of the elements that make up the explanatory data vector xn is used, with "1" used for elements that correspond to a value and "0" used for elements that do not correspond to a value.

[0082] That is, one of the values ​​corresponding to each of the multiple features having names beginning with "Education_" is "1" and the others are "0". As described above, one explanatory data vector xn is created from the data of one person included in the income dataset. By defining names for each element that makes up the explanatory data vector xn in this way, the meaning of the element is expressed by the element name.

[0083] The target data vector yn is a 2-row, 1-column matrix having two elements: the probability that the annual income is in the class of less than $50,000, and the probability that the annual income is in the class of more than $50,000. The target data vector yn is output from the learning model 22 when the explanation data vector xn is input to the learning model 22.

[0084] 8 is an explanatory diagram illustrating the record layout of the explanation-purpose DB 31. The explanation-purpose DB 31 has a No. field, an explanation data vector field, and a purpose data field. The explanation data vector field has fields corresponding to each element of the explanation data vector xn explained using Table 1, such as an Age field, an Education field, and a Marital Status field, as well as subfields corresponding to lower-level items. The purpose data field has a 50k or less field and a 50k or more field.

[0085] The No. field records consecutive numbers from 1 to N. Each subfield of the explanatory data field records an element of the explanatory data vector xn. The 50k or less field records the probability that the annual income of a person having the characteristics of the explanatory data vector xn is 50k dollars or less. The 50k or more field records the probability that the annual income of a person having the characteristics of the explanatory data vector xn is more than 50k dollars. For each record, the sum of the values ​​recorded in the 50k or less field and the 50k or more field is 1.

[0086] In step S503 of the program described using Fig. 6, the control unit 11 adds one record to the explanation-objective DB 31. The part surrounded by the dashed line in Fig. 8 is the transposed matrix of the explanation matrix X shown in Fig. 5. The part surrounded by the solid line in Fig. 8 is the transposed matrix of the objective matrix Y shown in Fig. 5.

[0087] Fig. 9 is an explanatory diagram illustrating the processing after the interpretation matrix A† is calculated. The upper part of Fig. 9 shows a schematic diagram of the interpretation matrix A† calculated in step S508 of the program explained using Fig. 6. Since the target data vector yn has two elements, the interpretation matrix A† is a matrix with two columns.

[0088] As mentioned above, the element in row a, column b of the interpretation matrix A† indicates the influence of the a-th element of the explanatory data vector xn on the b-th element of the objective data vector yn. Thus, AD11 indicates the influence of age on annual income of $50,000 or less, and AD12 indicates the influence of age on annual income of $50,000 or more.

[0089] The interpretation matrix A†, in which the rows are rearranged using the element values ​​as keys, is shown in the lower part of Fig. 9. For example, by rearranging the rows in descending order using the sum, average, or square mean of the values ​​in the first and second columns of each row as keys, the features that have a large influence on annual income are arranged at the top.

[0090] [Creating Commentary on Global Feature Importance Graph 41] Figure 10 is an example of the global feature importance graph 41. Figure 10 illustrates the top 20 features. The vertical axis indicates the names of the elements that make up the explanation data vector xn. The horizontal axis indicates the values ​​of each element of the interpretation matrix A†. The hatching that slopes downward to the right indicates the influence of each element on the values ​​in the left column of the interpretation matrix A†, i.e., on an annual income of 50,000 dollars or less. The hatching that slopes downward to the left indicates the influence of each element on the values ​​in the right column of the interpretation matrix A†, i.e., on an annual income of more than 50,000 dollars.

[0091] FIG. 10 shows an example of explanatory data for the learning model 22 created using AIME, an example of XAI technology. In FIG. 10, the vertical axis is an example of the first axis, and the horizontal axis is an example of the second axis. The vertical and horizontal axes may be displayed in reverse. Note that data for the global feature importance graph 41 can also be expressed in CSV format. Mutual conversion between graphs and CSV format is well known, so a description thereof will be omitted. Table 2 shows an overview of the CSV format file corresponding to FIG. 10.

[0092]

[0093] As shown in Table 2, the CSV file records the name of each element that makes up the explanatory data vector xn, in association with the value of each element. As mentioned above, the meaning of an element is expressed by its name, so the CSV file contains information about the meaning of each element that makes up the explanatory data vector xn. Table 2 is also an example of explanatory data for the learning model 22 created using AIME.

[0094] Table 3 shows a prompt template for causing ChatGPT, an example of the language model 24, to create an explanation about the global feature importance graph 41. The right side of Table 3 shows a reference translation in Japanese.

[0095]

[0096] In the above prompt, the part in {} indicates the part that changes depending on the global feature importance graph 41 that is the subject of the explanation. A specific example will be described using the global feature importance graph 41 in FIG. 10 as an example.

[0097] k indicates the number of features to be displayed on the vertical axis of the global feature importance graph 41. When explaining FIG. 10, k is "20." class1_description and class2_description indicate descriptions of the first and second elements constituting the target data vector yn. When explaining FIG. 10, class1_description is "annual income is less than $50,000," and class2_description is "annual income is more than $50,000." If the target data vector yn has three or more components, descriptions according to the number of components are added.

[0098] class1_name and class2_name indicate the names of the first and second elements that make up the target data vector yn. When explaining Figure 10, as described in the legend, class1_name is "income_<=50k" and class2_name is "income_>50k". In other words, the template is created to include information about the meaning of each element that makes up the target data vector yn.

[0099] The "+CSV" at the end of the prompt indicates that a CSV file is attached. When explaining FIG. 10, the CSV file shown in Table 2 will be used. Note that in the multimodal language model 24, the graph shown in FIG. 10 itself may be input instead of the CSV file.

[0100] The part just before that, "do not give the existence of a table in your answer," is a statement to prevent the language model 24 from outputting an explanation about the structure in CSV format. Note that the template shown in Table 3 is an example. The template is adjusted as appropriate depending on the characteristics of the language model 24 to be used.

[0101] Table 4 shows an example of the opening part of the explanation output from the language model 24 that has received the prompt input. The right side of Table 4 shows the Japanese reference translation.

[0102]

[0103] By looking at the global feature importance graph 41 while referring to the explanation output from the language model 24, the user can understand the properties of the learning model 22 explained by the global feature importance graph 41. A user who is not good at reading graphs can understand the properties of the learning model 22 simply by reading the explanation output from the language model 24.

[0104] The user may modify the template shown in Table 3 to instruct the user to output the explanation in a language other than English, such as Japanese or French. The user may also limit the length of the explanation to, for example, "up to 200 words."

[0105] The user may cause the language model 24 to perform the operations of rearranging the elements of the interpretation matrix A† and extracting the top rows, which have been described using Fig. 9. The user may cause the language model 24 to perform the operation of calculating the interpretation matrix A†, which has been described using Fig. 5.

[0106] [Creating an explanation for the local feature importance graph 42] Fig. 11 is an example of a local feature importance graph 42. Fig. 11 is a local feature importance graph 42 that shows which elements of the target explanation vector xo, which indicates the characteristics of a specific person, affect the estimation result when the learning model 22 estimates that the specific person has an "annual income of 50,000 or less." In the following explanation, the specific person will be referred to as the target person.

[0107] The local feature importance graph 42 is calculated using the local feature importance vector L described in equation (8). Here, the target explanation vector x in equation (8) is the explanation data vector x related to the data of the target person. The unit objective vector y is a transposed matrix of (1, 0), where k=1 indicates the class of "annual income 50k or less."

[0108] In Figure 11, the vectors calculated using equation (8) are sorted in descending order, and the top 20 features are shown. The vertical axis indicates the names of the elements that make up the explanatory data vector xn. The horizontal axis indicates the values ​​of each element of the local feature importance vector L. In Figure 11, the vertical axis is an example of the first axis, and the horizontal axis is an example of the second axis. The vertical and horizontal axes may be displayed in reverse. Figure 11 is an example of explanatory data for a learning model 22 created using AIME, an example of XAI technology.

[0109] Table 5 shows an outline of the CSV format file corresponding to FIG.

[0110]

[0111] Table 6 shows a prompt template for causing ChatGPT, an example of language model 24, to create an explanation about local feature importance graph 42. The right side of Table 6 shows a reference translation in Japanese.

[0112]

[0113] In the above prompt, the part in {} indicates the part that changes depending on the local feature importance graph 42 that is the subject of explanation. A specific example will be described using the local feature importance graph 42 in FIG. 11 as an example.

[0114] k indicates the number of features to be displayed on the vertical axis of the local feature importance graph 42. In the case of explaining FIG. 11, k is "20". i in classi_name is "income_<=50k", which is the name of the first element constituting the target data vector yn. classi_description is "annual income less than $50k", which is the description of the first element constituting the target data vector yn.

[0115] Table 7 shows an example of the opening part of the explanation output from the language model 24 that has received the prompt input. The right side of Table 7 shows the Japanese reference translation.

[0116]

[0117] By looking at the local feature importance graph 42 while referring to the explanation output from the language model 24, the user can understand why the learning model 22 estimated that the target person's annual income is less than $50,000.

[0118] If the reason is appropriate, the user can trust the output of the learning model 22. If the reason is inappropriate, instead of trusting the output of the learning model 22, the user can review the target explanation vector xo for the target person, consider an estimation method other than the learning model 22, or the like.

[0119] Similarly, a local feature importance graph 42 explaining why the target person's annual income exceeds 50,000 dollars can be created, and the reason can be explained by the language model 24. Specifically, by changing the unit objective vector yuk in equation (8) to k=2, which indicates the class of "annual income over 50,000 dollars," that is, to the transposed matrix of (0, 1), a local feature importance graph 42 explaining why the target person's annual income exceeds 50,000 dollars can be created.

[0120] The i in the prompt's classi_name is set to "income_>50k," which is the name of the second element constituting the target data vector yn, and the classi_description is set to "annual income over $50k," which is the description of the second element constituting the target data vector yn. As a result, an explanation of the local feature importance graph 42 regarding the same target person's annual income exceeding $50k is output from the language model 24.

[0121] According to this embodiment, AIME, a type of XAI technology, can be used to generate explanatory data that explains the behavior of the learning model 22 generated using the income dataset as training data. The explanatory data can be represented by a diagram, such as a global feature importance graph 41 or a local feature importance graph 42. A natural language explanation can then be generated for the explanatory data generated by AIME using the language model 24.

[0122] According to this embodiment, even a user who does not have sufficient specialized knowledge and is unfamiliar with graphs and other diagrams can be assisted in understanding explanatory data created using XAI technology. The assisted user can then appropriately determine how to handle the output of the learning model 22.

[0123] A program is an example of a program product. A computer program can be deployed to be executed on a single computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0124] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should be considered not to be limiting. The scope of the present invention is defined by the claims, not by the meaning described above, and is intended to include all modifications within the meaning and scope of the claims.

[0125] Independent and dependent claims may be combined with each other in any combination, regardless of the reference format. Furthermore, while the claims use a format in which a claim references two or more other claims (multiple claim format), this is not a limitation. Multiple claims that reference at least one other claim (multi-multi claim format) may also be used.

[0126] REFERENCE SIGNS LIST 10 Information processing device 11 Control unit 12 Main memory device 13 Auxiliary memory device 14 Communication unit 15 Display unit 16 Input unit 19 Reading unit 22 Learning model 24 Language model 31 Explanation-purpose DB 41 Global feature importance graph 42 Local feature importance graph 43 Similarity distribution map 96 Portable recording medium 97 Program 98 Semiconductor memory

Claims

1. An information processing method in which a computer executes the following processes: obtain explanatory data for a learning model, the explanatory data being created using XAI (Explainable AI) technology that explains the behavior of a learning model that accepts input of an explanatory data vector and outputs a target data vector; input a prompt including the explanatory data into a language model; and output an explanation from the language model.

2. The information processing method according to claim 1, wherein the prompt includes information regarding the meaning of each element constituting the target data vector.

3. The information processing method according to claim 1, wherein the prompt includes information regarding the meaning of each element constituting the explanatory data vector.

4. The information processing method according to claim 1, wherein the explanatory data is data relating to diagrams that explain the behavior of the learning model.

5. The information processing method of claim 1, further comprising: inputting a target explanation vector into the learning model to obtain a target objective vector output from the learning model; obtaining explanatory data regarding the reason for the output of the target objective vector, created using the XAI technology; inputting a prompt including the target explanation vector, the target objective vector, and the obtained explanatory data into the language model; and outputting an explanation from the language model.

6. The information processing method according to claim 1, wherein the XAI technology uses an interpretation matrix which is the vector product of an explanation matrix in which multiple sets of the explanation data vectors are arranged, and a generalized inverse matrix of a target matrix in which the target data vectors are arranged in an order corresponding to the explanation data vectors.

7. The information processing method described in claim 6, wherein the explanatory data is data regarding a global feature importance graph using components of the explanatory data vector on a first axis and values ​​of each element constituting the interpretation matrix on a second axis, and the prompt includes data regarding the global feature importance graph and information regarding the meaning of each element constituting the explanatory data vector.

8. The information processing method according to claim 6, wherein the explanatory data is the interpretation matrix, and the prompt includes information regarding the interpretation matrix and the meaning of each element constituting the explanatory data vector.

9. The information processing method of claim 6, wherein the explanatory data is data relating to a local feature importance graph in which the components of the explanatory data vector are used as a first axis and the values ​​of each element constituting the local feature importance vector calculated by equation (1) are used as a second axis, and the prompt includes data relating to the local feature importance graph, information relating to the meaning of each element constituting the explanatory data vector, and information relating to the target explanation vector and unit objective vector used in calculating the local feature importance vector.

10. An information processing method according to any one of claims 1 to 9, wherein the prompt includes information regarding the number of elements of the explanatory data vector for which the commentary is to be output.

11. A program that causes a computer to execute the process of acquiring explanatory data for a learning model, the explanatory data being created using XAI technology that explains the behavior of the learning model that accepts input of an explanatory data vector and outputs a target data vector, inputting a prompt including the explanatory data into a language model, and outputting an explanation from the language model.

12. An information processing device having a control unit, wherein the control unit acquires explanatory data relating to the learning model, the explanatory data being created using XAI technology that explains the behavior of the learning model that accepts input of an explanatory data vector and outputs a target data vector, inputs a prompt including the explanatory data into a language model, and outputs an explanation from the language model.

Images