Analysis support device, analysis support method, and analysis support program

Abstract

To provide an analysis support device, an analysis support method, and an analysis support program which estimate a detailed logical relation between an explanatory variable and an objective variable.SOLUTION: In an analysis support system 100, an analysis support device 20 comprises a storage device and a processor. The storage device stores data indicating a correlation between a value of an explanatory variable and a value of an objective variable output from a learned model which outputs the objective variable being output data from the explanatory variable being input data. The processor performs: processing for generating a phrase representing a foundation for processing of the learned model in which the value of the objective variable is calculated from the value of the explanatory variable, on the basis of a sentence including a word representing the explanatory variable and a sentence including a word representing the objective variable and a word representing the correlation; processing for specifying a text representing an idea of the foundation for the processing of the learned model on the basis of the generated phrase; and processing for outputting an estimated text to an output device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an analysis support device, an analysis support method, and an analysis support program. [Background technology]

[0002] In recent years, there has been a growing interest in using machine learning-based AI (Artificial Intelligence) to make decisions in government policy reviews. For example, AI can be used to derive objective information from explanatory variables. Make decisions by predicting environmental variables.

[0003] On the other hand, when making decisions, governments are also required to fulfill their responsibility to explain things to society, which is called accountability. In other words, they need to explain the basis for their decisions and ensure that the decisions they make are plausible even to humans, so that citizens can accept their decisions. When using AI, the prediction results need to be interpretable (Model Interpretability) and socially acceptable (Public Acceptance). Acceptance is required.

[0004] However, with AI using white-box machine learning techniques such as decision trees, which can explain the target variable based on clear conditions for explanatory variables, predictions tend to be made from a limited number of explanatory variables selected subjectively by the analyst. As a result, there is a possibility that explanatory variables may be overlooked when making predictions, which could cause problems from the perspective of accountability.

[0005] As a technology for preventing such overlooking of explanatory variables, Patent Document 1 describes an information processing device that is equipped with a deep learning prediction unit that performs prediction processing using a deep learning model based on data stored in a database so that the main explanatory variables in the deep learning model can be extracted, and a variable extraction unit that performs multiple regression analysis using the prediction results from the deep learning prediction unit as the objective variable and the data as the explanatory variables, and determines variables to explain the prediction results of the deep learning model based on the results of the multiple regression analysis. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. WO2018 / 142753 Summary of the Invention [Problem to be solved by the invention]

[0007] From the prediction results of AI, etc., users can infer that there is a correlation between the objective variable and the explanatory variables. However, when users need to consider the above-mentioned interpretability and social acceptability, there are times when they need to specify and explain more detailed relationships, such as causal relationships between the explanatory variables and the objective variable, rather than just correlations.

[0008] On the other hand, users may mistakenly select explanatory variables that are difficult to interpret and have little relationship to the target variable as explanatory variables to incorporate into the AI.

[0009] In this way, in order to identify the causal relationship between explanatory variables and target variables, it is necessary to identify appropriate explanatory variables from the prediction results, but this requires manual analysis and interpretation using specialized knowledge.

[0010] For example, Patent Document 1 states that "according to the present invention, the main The only statements made are: "It becomes possible to extract important explanatory variables (variables that explain the prediction results well)," "'Well explaining the dependent variable' corresponds to a high degree of influence on the dependent variable (large standard partial regression coefficient)," and "The use of deep learning models allows analysis without narrowing down the explanatory variables, thereby resolving the aforementioned problem of explanatory variables being overlooked when narrowing down the explanatory variables."

[0011] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an analysis support device, an analysis support method, and an analysis support program that are capable of estimating detailed logical relationships between explanatory variables and response variables. [Means for solving the problem]

[0012] One aspect of the present invention for solving the above problem is an analysis support device that includes a storage device that stores data indicating the correlation between the values ​​of explanatory variables output by a trained model that outputs a dependent variable as output data from explanatory variables as input data and the values ​​of the dependent variable, and a processing device that executes the following processes: a process of generating phrases that represent the processing basis of the trained model that calculated the value of the dependent variable from the values ​​of the explanatory variables, based on a sentence including a word that represents the explanatory variable and a sentence including a word that represents the dependent variable and a word that represents the correlation; a process of identifying sentences that indicate the intention of the processing basis of the trained model, based on the generated phrases; and a process of outputting the identified sentences to an output device. [Effects of the Invention]

[0013] According to the present invention, it is possible to estimate detailed logical relationships between explanatory variables and response variables. Configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0014] [Figure 1]1 is a block diagram illustrating an example of a configuration of an analysis support system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating an example of functions provided in the analysis support device according to the present embodiment. [Figure 3] FIG. 2 is a flow diagram illustrating an example of processing performed by the analysis support system. [Figure 4] FIG. 10 is a diagram illustrating an example of an input screen for input data. [Figure 5] FIG. 10 is a diagram illustrating an example of input data. [Figure 6] FIG. 10 is a flowchart illustrating details of a classification criterion matrix extraction process. [Figure 7] FIG. 10 is a diagram illustrating an example of a classification criterion matrix generated by the classification criterion extraction process. [Figure 8] FIG. 10 is a flowchart illustrating details of a mask portion prediction process. [Figure 9] FIG. 10 is a diagram illustrating an example of a classification criterion matrix generated in the mask portion prediction process. [Figure 10] FIG. 10 is a flowchart illustrating details of a relationship prediction process. [Figure 11] FIG. 10 is a diagram illustrating an example of a classification criterion matrix generated in the relationship prediction process. [Figure 12] FIG. 10 is a flowchart illustrating details of related word prediction processing. [Figure 13] FIG. 10 is a diagram illustrating an example of a classification reference matrix generated in the related word prediction process. [Figure 14] FIG. 10 is a flowchart illustrating details of a supplemental explanation generation process. [Figure 15] FIG. 10 is a diagram illustrating an example of a reason intention list. [Figure 16] FIG. 10 is a diagram illustrating an example of a supplementary explanation matrix. [Figure 17] FIG. 10 is a diagram showing an example of a screen displayed by a screen display program. [Figure 18] FIG. 10 is a diagram illustrating an example of input data corresponding to output data of a trained model based on machine learning using a decision tree. DETAILED DESCRIPTION OF THE INVENTION

[0015] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the accompanying drawings. Hereinafter, the embodiments will be described with reference to the drawings. In each drawing, the same or equivalent components are used. and parts are given the same reference numerals.

[0016] 1 is a block diagram showing an example of the configuration of an analysis support system 100 according to this embodiment. The analysis support system 100 includes an analysis support device 20, an input device 31, an output device 32, and an external device .

[0017] The analysis support system 100 uses a trained model (AI: Artificial Intelligence). Specifically, the analysis support system 100 estimates the basis (intention) on which the trained model (hereinafter referred to as the analytical model) has output output data (objective variables) from input data (explanatory variables), and presents the estimation results to the user.

[0018] The analysis support device 20 includes a processing unit 21, a device interface 22, a communication interface 23, a main memory device 24, and an auxiliary memory device 25, which are connected via a bus 26.

[0019] The processing device 21 is an electronic component that performs arithmetic processing based on data and programs input from each device, etc. that constitutes the analysis support device 20, and outputs the arithmetic results and control signals to each device, etc. The processing device 21 is, for example, a CPU (Central Processing Unit) or a computer The processing device 21 is an electronic circuit (GPU: Graphics Processing Unit) including a control device and a calculation function. The processing device 21 controls each component constituting the analysis support device 20 by executing the OS (Operating System) or applications of the analysis support device 20.

[0020] The device interface 22 is an interface that connects to the input device 31 and the output device 32 wirelessly or via a wired connection. The input device 31 is composed of devices such as a keyboard, mouse, buttons, or touch panel for inputting various operation instructions to the analysis support device 20. The output device 32 is composed of devices such as a display or touch panel for displaying a GUI (Graphical User Interface) to the user. The device interface 22 may be configured to allow connection to an external storage device such as a USB memory.

[0021] The communication interface 23 is an interface for connecting to an external device 34 (described later) via a communication network 33. The communication interface 23 may be one that conforms to existing communication standards. The communication network 33 may be, for example, a wired or wireless communication network such as the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or a dedicated line. It is a communication network.

[0022] The main memory device 24 is any electronic component that can store electronic information such as programs executed by the processing device 21 and various data. The main memory device 24 is configured, for example, by a ROM (Read Only Memory) or a RAM (Random Access Memory).

[0023] The auxiliary storage device 25 is a storage device other than the main storage device 24, and is any electronic component capable of storing electronic information. In this embodiment, the auxiliary storage device 25 is configured by, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0024] The external device 34 is one or more information processing devices (for example, a cloud) and includes one or more storage devices.

[0025] In this embodiment, the analysis support device 20 includes one of each of the above hardware components, but may include multiple of the same components.

[0026] 2 is a block diagram illustrating an example of functions of the analysis support device 20 according to this embodiment. The analysis support device 20 stores a prediction result selection program 1, a word management program 7, a classification criteria extraction program 2, a relationship prediction program 3, a related word prediction program 4, a supplemental explanation generation program 5, and a screen display program 6.

[0027] The analysis support device 20 also stores data on a word dictionary 11, a text collection 12, a mask part prediction model 13, a relationship prediction model 14, a related word prediction model 15, and a reason intention list 16.

[0028] Some or all of these programs and data may be stored in advance in the auxiliary storage device 25, or may be acquired from another storage device or storage medium communicatively connected to the analysis support device 20.

[0029] The prediction result selection program 1 accepts from the user a selection of data (hereinafter referred to as correlation data) showing the correlation between explanatory variables as input data and dependent variables output by an analytical model that outputs dependent variables as output data from explanatory variables as input data.

[0030] In this embodiment, the analytical model is a so-called white-box type "explainable AI" (XAI) in which the output data contains the underlying data. It may also be a so-called black-box AI that outputs a target variable from explanatory variables. In the case of black-box AI, the correlation data only needs to be accompanied by evidence data for the output data.

[0031] The analytical model is constructed based on the algorithm of, for example, a neural network, a decision tree, a random forest, or a support vector machine (SVM). The neural network is a neural network having an input layer that receives input data, one or more intermediate layers (hidden layers) that extract and output features from the input data, and an output layer that outputs output data, such as a CNN (Convolution Neural Network). The analytical model of this embodiment is a trained model based on a decision tree, but this is not intended to limit the algorithm of the analytical model.

[0032] The word management program 7 has a word division function that, when a character string is input, divides the character string into words and outputs them. In this embodiment, when there are two or more word division methods for a character string, the word management program 7 performs word division by processing the longest word first, and for words of the same length, processing them in order from the left of the character string.

[0033] Furthermore, the word management program 7 has a function of outputting one or more parts of speech for a word when the word is input, while referring to a word dictionary 11, which will be described later. Note that the word management program 7 may output a statistically likely word segmentation result from an existing morphological analysis model and the corresponding parts of speech.

[0034] The classification criterion extraction program 2 extracts combinations of patterns of values ​​of the objective variable (hereinafter also referred to as classification criteria) and patterns of values ​​of each explanatory variable in the correlation data.

[0035] The relationship prediction program 3 generates a phrase (hereinafter referred to as a relationship label) that represents the processing basis of an analytical model that has performed a process of calculating the value of a dependent variable from the value of an explanatory variable, based on a sentence that includes words that represent explanatory variables (the names and elements of the explanatory variables (note that, in this specification, "elements" refers to the values ​​or contents of parameters)) and a sentence that includes words that represent dependent variables (the names and elements of the dependent variable) and words that represent correlations (such as "correlated" or "no correlation"). The relationship label is, for example, a phrase such as "necessity" or "feasibility."

[0036] The related word prediction program 4 estimates related words related to the explanatory variables and the response variable based on the words representing the explanatory variables and the response variable.

[0037] The supplementary explanation generation program 5 identifies a sentence indicating the intention of the processing basis of the trained model (a supplementary explanation sentence for the relationship label; hereinafter referred to as the intention sentence) based on the relationship label identified by the relationship prediction program 3.

[0038] The screen display program 6 displays various types of information on the output device 32. For example, the screen display program 6 outputs the intended sentence identified by the supplemental explanation generating program 5 to the output device 32.

[0039] Next, the word dictionary 11 is a database that stores words and their corresponding parts of speech.

[0040] The text set 12 is a database that stores a plurality of text data (data of sentences). The text set 12 is, for example, a collection of text data that is publicly available on the Internet (Web). The content of the text data stored in the text set 12 is not particularly limited.

[0041] The mask part prediction model 13 is a word array W={w1, w2, ..., [MASK], ..., w N} as an input value, and outputs a word (hereinafter referred to as a conjunction) that should be inserted at the position of the predicted mask token contextually or logically as indicated by the word sequence W. That is, a conjunction is a word that connects the preceding and following sentences or phrases connected by the predicted mask token. The conjunction is, for example, a conjunction of a forward connection, a conjunction of an adverse connection, or other phrase. The mask part prediction model 13 outputs one or more conjunctions together with their rank (accuracy).

[0042] In this embodiment, the analysis support device 20 is assumed to have created the masked part prediction model 13 in advance based on the text set 12, but may also acquire the created masked part prediction model 13 from another device. Also, in this embodiment, the masked part prediction model 13 is assumed to be a masked language model (MLM) of the Bidirectional Encoder Representation from Transformers (BERT) model based on Transformers. The masked part prediction model 13 may also be created based on the permutation language model (PLM) mechanism of the XLNet model.

[0043] The relationship prediction model 14 is a trained model that takes some or all of the output values ​​of the mask part prediction model 13 as input values ​​and outputs information (i.e., relationship labels) indicating the logical relationship between the group of words before the predicted mask token and the group of words after the predicted mask token.

[0044] In this embodiment, the relationship prediction model 14 takes as input values ​​the word group with the top T most accurate words (here, T=100) from the word group that is the output value of the mask part prediction model 13. Furthermore, the output value of the relationship prediction model 14 is assumed to be "necessity," "feasibility," or "other correlation" as the relationship between the word group before the predicted mask token and the word group after the predicted mask token. Here, "necessity" refers to the case where the predicted mask token has many consecutive conjunctions. "Feasibility" refers to the case where the predicted mask token has many adverse conjunctions. "Other correlation" refers to the case where the relationship is neither "necessity" nor "feasibility."

[0045] The relationship prediction model 14 described here is an example. There are no particular limitations on the type of trained model as long as the output value of the mask part prediction model 13 corresponds to the relationship label.

[0046] The related word prediction model 15 is a trained model that predicts words related to two words (hereinafter referred to as related words).

[0047] In this embodiment, the related word prediction model 15 is configured with "word2vec," a word distributed representation model, and a similar vector search unit using cosine similarity. "word2vec" can convert each word into a word vector (data encoded into an r-dimensional numerical vector including the meaning of the corresponding word; in this embodiment, r=200). (Since word vectors are numerical vectors, they can be added to each other.) Meanwhile, the similar vector search unit calculates the cosine similarity between two word vectors (a parameter that indicates that the larger the cosine value of the angle between the two word vectors, the greater the similarity) (e.g., "Japan" + "capital" = "Tokyo"). This related word prediction model 15 can be applied not only to English but also to various languages ​​such as Japanese.

[0048] The related word prediction model 15 may be a trained model that outputs, as related words, independent words that are likely to co-occur with each of the words included in the target variable name, the explanatory variable name, and the relationship label from a co-occurrence word dictionary. In this case, the analysis support device 20 may store an existing dictionary as the co-occurrence word dictionary in the related word prediction model 15, or may create a co-occurrence word dictionary by statistically determining the probability of word co-occurrence from the data of the text set 12 and store this in the related word prediction model 15.

[0049] Next, the reason intention list 16 is information of a list of intention sentences associated with combinations of objective variables and relationship labels.

[0050] The programs of the analysis support device 20 described above are executed by the processing device 21 by reading them from the main storage device 24 or the auxiliary storage device 25. Each program can be recorded on, for example, a portable or fixed recording medium and distributed. Note that all or part of the analysis support device 20 (the programs thereof) may be realized using virtual information processing resources provided using virtualization technology, process space separation technology, or the like, such as a virtual server provided by a cloud system. All or part of the programs of the analysis support device 20 may also be realized by a service provided by a cloud system via an API (Application Programming Interface), for example. Next, the processing performed by analysis support system 100 will be described.

[0051] FIG. 3 is a flow diagram illustrating an example of processing performed by the analysis support system 100. First, the prediction result selection program 1 acquires input data D1, which is correlation data (step S101).

[0052] (input data) 4 is a diagram showing an example of an input screen 150 for input data D1 that the prediction result selection program 1 displays on the output device 32. The prediction result selection program 1 displays the input screen 150 and accepts designation of the location of the input data D1 from the user via the input device 31. The input data D1 may be stored in the auxiliary storage device 25, or may be stored in an external storage device such as a USB memory connectable to the device interface 22, or may be stored in a storage device connected via the communication network 33.

[0053] (input data) 5 is a diagram showing an example of input data D1. The input data D1 is a response variable The input data D1 has the following data items: place name 501 in which a place name, which is a data name (data related to the execution unit of the trained model), is set; objective variable 502 in which a disaster prevention issue, which is an objective variable, is set; explanatory variable 503 in which data of one or more explanatory variables is set; and correlation 504 in which a correlation between the objective variable and each explanatory variable is set.

[0054] In this embodiment, the explanatory variables 503 have data sub-items such as agricultural worker ratio 505 (e.g., year-on-year change in the number of agricultural workers), academic researcher number ratio 506 (e.g., year-on-year change in the number of academic researchers), and child number to total population ratio 507 (e.g., year-on-year change in the number of children to the total population). Note that these explanatory variable items are merely examples.

[0055] In this embodiment, correlation 504 has the following data sub-items: explanatory variable X1 influence 508, in which a parameter (e.g., correlation coefficient) representing the strength of the correlation between the academic researcher ratio and the objective variable is set; explanatory variable X2 influence 509, in which a parameter (e.g., correlation coefficient) representing the strength of the correlation between the agricultural worker ratio and the objective variable is set; and explanatory variable X3 influence 510, in which a parameter (e.g., correlation coefficient) representing the strength of the correlation between the number of children to total population ratio and the objective variable is set.

[0056] Next, as shown in FIG. 3, the classification criterion extraction program 2 executes a classification criterion matrix extraction process S102 for generating combinations of data between the objective variable and each explanatory variable.

[0057] <Classification criteria matrix extraction process> FIG. 6 is a flow diagram illustrating the classification criterion matrix extraction process S102 in detail. The classification criterion extraction program 2 checks whether or not there is a (previously unprocessed) objective variable in the input data D1 (step S201).

[0058] If there are unprocessed objective variables (step S201: YES), the classification criterion extraction program 2 selects one of the objective variables and executes the processing of step S202. If there are no unprocessed objective variables (step S201: NO), the generated classification criterion matrix D2 is stored in the main storage device 24 (step S210), and the classification criterion extraction processing ends.

[0059] In step S202, the classification criterion extraction program 2 generates a classification criterion matrix D2, which is a matrix containing the elements (contents) of the previously selected objective variable (hereinafter referred to as the selected objective variable) as row data and the relationship 702 of each explanatory variable with the selected objective variable as column data (if this process is performed for the second or subsequent time, this matrix data is added to the existing classification criterion matrix D2). Details of the classification criterion matrix D2 will be described later.

[0060] Then, the classification criterion extraction program 2 calls the word management program 7, divides the character string of the name of the selected objective variable into multiple words, and sets each word of the divided objective variable as the name of the column of the objective variable in the classification criterion matrix D2 (step S203). For example, the classification criterion extraction program 2 divides the character string of the name of the objective variable into each word in the two word arrays (Wa={wa1, wa2, ..., wa k}, Wc={wc1,wc2,…,wc m}), and each divided word is set as the name of a column of the objective variable 701 of the classification criterion matrix D2.

[0061] In addition, the classification criterion extraction program 2 divides the character string of the name of the content (element) of the selected objective variable into a word array Wb={wb1, wb2, ..., wbl}, and sets the divided words as data in the column of the objective variable 701 of the classification criterion matrix D2 (step S204).

[0062] The classification criteria extraction program 2 selects the following explanatory variables that correspond to the selected objective variables: It is determined whether there are any explanatory variables for which the processes in steps S206 to S209 described below have not been performed (step S205).

[0063] If there are explanatory variables for which the processes of steps S206 to S209 have not been performed (step S205: YES), the classification criteria extraction program 2 executes the process of step S206, and if there are no explanatory variables for which the processes of steps S206 to S209 have not been performed (step S205: YES), the classification criteria extraction program 2 repeats the process of step S201 to select a new objective variable.

[0064] In step S206, the classification criterion extraction program 2 selects one of the explanatory variables associated with the selected objective variable (hereinafter referred to as the selected explanatory variable).

[0065] The classification criteria extraction program 2 converts the string of the names of the selected explanatory variables into a word array Wc={wc1, wc2, ..., wc m}, and each divided word is set as the column name (column name in the relationship 702) related to the selected explanatory variable in the classification criterion matrix D2 (step S207).

[0066] The classification criterion extraction program 2 identifies the strength of correlation between the selected objective variable and the selected explanatory variables (step S208).

[0067] Specifically, the classification criterion extraction program 2 refers to the input data D1 and determines that if the strength of correlation is equal to or greater than a predetermined threshold (0.5 in this embodiment), {correlation, strong} should be set in the column related to the selected explanatory variable in the row related to the selected objective variable in the classification criterion matrix D2 (the column in the relationship 702), or if the strength of correlation is less than the predetermined threshold, {correlation, weak} should be set.

[0068] Then, the classification criteria extraction program 2 extracts a word group (word array Wd={wd1, wd2, ..., wd n}) ({correlation, strong}, {correlation, weak}, etc.) is set in the column (column in the relationship 702) related to the selected explanatory variable in the row related to the selected objective variable in the classification criterion matrix D2 (step S209). Thereafter, the classification criterion extraction program 2 executes the process of step S205 to select a new explanatory variable.

[0069] The input data D1 may also correspond to the output data of a trained model based on machine learning using a decision tree. In this case, the input data, like the input data D8 shown in FIG. 18, includes not only the presence or absence of a correlation between the objective variable and the explanatory variables but also the relationship between the explanatory variables, i.e., whether the explanatory variables are greater than or less than a numerical reference. Therefore, in addition to the aforementioned "strong correlation" and "weak correlation," data indicating other relationships, such as "large" if the correlation is greater than the reference, "small" if the correlation is smaller than the reference, or "weak correlation" if there is no correlation, may be set within each relationship 702 of the classification reference matrix.

[0070] In step S201, there may be two or more correlated explanatory variables for one element of a target variable (for example, the presence of a problem). In this case, the classification criterion extraction program 2 may set different classification criteria for each of the multiple explanatory variables, or may set one classification criterion by combining the relationships with the multiple explanatory variables.

[0071] (classification standard matrix) 7 is a diagram showing an example of the classification criterion matrix D2 generated in the classification criterion extraction process S102. The classification criterion matrix D2 is composed of a plurality of rows of classification criteria of dependent variables, dependent variables 701 in which the classification criteria of the dependent variables are set, and respective columns of relationships 702 in which information indicating the strength of the correlation between the classification criterion and each explanatory variable is set.

[0072] For the objective variable 701, the constituent words of the item names of the objective variable (such as "disaster prevention", "issues") are used as the column names. The data in the column of the objective variable 701 is set to the content of the classification criteria.

[0073] Relationship 702 further comprises columns of multiple sub-items. Each column in relationship 702 is assigned a name that includes words from the explanatory variable. For example, column 703 for the ratio of academic researchers is assigned "Academic" and "Ratio of researchers." Column 704 for the ratio of agricultural workers is assigned "Agriculture," "Workers," and "Number Ratio." Column 705 for the number of children to total population is assigned "Children," "Number," "Total population," and "Ratio." Furthermore, each column of data in relationship 702 is assigned a word that indicates the correlation between the objective variable and the explanatory variable. For example, each column of data in relationship 702 is assigned "Correlation" or "Strong" if the correlation is strong, and "Correlation" or "Weak" if the correlation is weak.

[0074] Next, as shown in FIG. 3, the classification criterion extraction program 2 checks whether there are any classification criterion that have not been subjected to the processes of steps S104 to S108 (described later) (step S103).

[0075] If there are classification criteria for which the processes of steps S104 to S108 have not been performed (step S103: YES), the classification criteria extraction program 2 selects one of the classification criteria (selected classification criteria) (step S104), and then performs the process of step S105. If there are no classification criteria for which the processes of steps S104 to S108 have not been performed (step S103: NO), the screen display program 6 performs the process of step S109.

[0076] In step S105, the relationship prediction program 3 executes the masked part prediction process S105.

[0077] <Mask part prediction processing> FIG. 8 is a flowchart illustrating the mask portion prediction process S105 in detail.

[0078] The relationship prediction program 3 acquires data of rows related to the selected classification criterion selected in step S104 (selected classification criterion row data) from the classification criterion matrix D2 generated in the classification criterion matrix extraction process S103 (step S301).

[0079] The relationship prediction program 3 determines whether or not there is any explanatory variable for which the processes of steps S303 to S310 described below have not been executed among the explanatory variables associated with the selected classification criterion in the selected classification criterion row data (step S302).

[0080] If there are explanatory variables for which the processing of steps S303 to S310 has not been performed (step S302: YES), the relationship prediction program 3 performs the processing of step S303, and if there are no explanatory variables for which the processing of steps S303 to S310 has not been performed (step S302: NO), the relationship prediction program 3 saves the classification criterion matrix D2 generated by the processing up to this point as a classification criterion matrix D3 (described later), and the mask part prediction processing S105 ends.

[0081] In step S303, the relationship prediction program 3 selects explanatory variables (selected explanatory variables) for which the processes of steps S304 to S310 have not been executed.

[0082] Then, the relationship prediction program 3 extracts from the classification criterion matrix D2 a word group of the name of the objective variable (word array Wa={wa1,wa2,...,wak}), a word group of the elements (contents) of the objective variable (word array Wb={wb1,wb2,...,wbl}), a word group of the name of the relationship between the objective variable and the selected explanatory variables (word array Wc={wc1,wc2,...,wcm}), and a word group of the elements (contents) of the relationship between the objective variable and the selected explanatory variables (word array Wd={wd1,wd 2, ..., wdn}) (step S304).

[0083] Then, the relationship prediction program 3 determines whether the word sequence Wd (the relationship between the objective variable and the selected explanatory variables) indicates a correlation stronger than a certain level (step S305). For example, the relationship prediction program 3 determines whether the word sequence Wd is {correlation, strong} (when the columns of the selected explanatory variables are {children, number, total population, ratio} in the classification criterion matrix data D2 shown in FIG. 7).

[0084] If the word sequence Wd indicates that the correlation is stronger than a certain level (step S305: YES), the relationship prediction program 3 executes the processing of step S307, and if the word sequence Wd does not indicate that the correlation is stronger than a certain level (step S305: NO), the relationship prediction program 3 executes the processing of step S306.

[0085] In step S306, the relationship prediction program 3 ends the processing for the selected explanatory variable, and repeats the processing of step S302 to perform processing for other explanatory variables.

[0086] In step S307, the relationship prediction program 3 connects the word array Wa of the name of the objective variable and the word array Wb of the element (content) of the objective variable with the particle "ga" to form a sentence, namely, the word array Wab={wa1, wa2, ..., wa k , “ga”, wb1,wb2,…,w b l}, and the word array Wcd = {wc1, wc2, ..., wc m , “ga”,wd1,wd2,…,wd n}.

[0087] Here, the relationship prediction program 3 generates sentences by inserting the particle "ga," but other particles may be used. Furthermore, the sentences generated by the relationship prediction program 3 do not necessarily have to be complete sentences, as long as they are a collection of words or phrases that allow the trained model to determine their context and logic. That is, the term "sentence" in step S307 and the like is used in this sense.

[0088] In step S302, if the selected explanatory variable does not have a correlation strength with the objective variable that is equal to or greater than a predetermined threshold, the relationship prediction program 3 may execute the process of step S306, or may execute the process of step S307 while outputting information indicating a weak correlation. However, if the input data D1 has a large number of explanatory variables (for example, if the input data D1 is generated by machine learning using deep learning), outputting the information would be inconvenient for the user, so the relationship prediction program 3 may execute the process of step S306.

[0089] Next, the relationship prediction program 3 combines the word sequence Wab and the word sequence Wcd with a prediction mask token [MASK] to generate a word sequence Wabcd={wc1, wc2, ..., wd n , [ MASK], wa1,wa2,…,wb l} is generated (step S308).

[0090] The relationship prediction program 3 inputs the word sequence Wabcd into the mask part prediction model 13 and obtains the output word group (list of conjunctions) (step S309).

[0091] The relationship prediction program 3 adds a new column (mask part prediction result 706) with the selected explanatory variable as its name to the classification criterion matrix D2, and stores the word group (list of conjunctions) acquired in step S309 in the row of that column related to the selected classification criterion selected in step S104 (step S310). Thereafter, the process of step S302 is performed to select a new explanatory variable.

[0092] (classification standard matrix) 9 is a diagram showing an example of the classification criterion matrix D3 generated in the masked portion prediction process S105. The classification criterion matrix D3 has a column of the masked portion prediction result 706 added to the classification criterion matrix D2.

[0093] The masked part prediction result 706 consists of multiple subitems related to each explanatory variable. Each column in the masked part prediction result 706 is set with the name of the explanatory variable. For example, in the column 707 for the ratio of the number of academic researchers, "Academic" and "Ratio of the number of researchers" are set, and in the column 708 for the ratio of agricultural workers, "Agriculture," "Worker," and "Ratio of number" are set, respectively. In the data of each column in the masked part prediction result 706, each conjunction that connects the word group of the objective variable and the word group of each explanatory variable is set. For example, in the data of each column of each masked part prediction result 706, a conjunction of a direct connection (such as "so"), a conjunction of a contradiction (such as "mono"), or other word (such as "nature") is set.

[0094] Next, as shown in FIG. 3, the relationship prediction program 3 executes a relationship prediction process S106. <Relationship prediction processing> FIG. 10 is a flow diagram illustrating the relationship prediction process S106 in detail.

[0095] The relationship prediction program 3 acquires data of rows related to the selected classification criterion selected in step S104 (selected classification criterion row data) from the classification criterion matrix D3 generated in the masked portion prediction process S105 (step S401).

[0096] The relationship prediction program 3 determines whether or not there is any explanatory variable for which the processes of steps S403 to S406 described below have not been executed among the explanatory variables associated with the selected classification criterion in the selected classification criterion row data (step S402).

[0097] If there are explanatory variables for which the processing of steps S403 to S406 has not been executed (step S402: YES), the relationship prediction program 3 executes the processing of step S403, and if there are no explanatory variables for which the processing of steps S403 to S406 has not been executed (step S402: NO), the relationship prediction program 3 stores the classification criterion matrix D3 generated up to this point in the main memory device 24 as a classification criterion matrix D4 (described below) (step S407), and the relationship prediction processing S105 ends.

[0098] In step S403, the relationship prediction program 3 selects explanatory variables (selected explanatory variables) for which the processes in steps S404 to S406 have not been executed.

[0099] The relationship prediction program 3 acquires data of the masked part prediction result 706 related to the selected explanatory variables from the classification criterion matrix D3 (step S404).

[0100] The relationship prediction program 3 acquires one or more relationship labels generated by inputting the data acquired in step S404 into the relationship prediction model 14 (step S405). In this embodiment, one of the three relationship labels, "necessity," "feasibility," and "other correlation," is output from the relationship prediction model 14.

[0101] The relationship prediction program 3 calls the word management program 7 and divides the character string of the relationship label acquired in step S405 into word groups (word array We={we1, we2, ..., wep}). Then, the relationship prediction program 3 adds a new column (relationship label 709) with the selected explanatory variable as its name to the classification criterion matrix D3, and performs step S1 The word array We is stored in the row related to the selected classification criterion selected in 04 (step S406). After that, the relationship prediction program 3 performs step S402 to select a new explanatory variable.

[0102] (classification standard matrix) 11 is a diagram showing an example of the classification criterion matrix D4 generated in the relationship prediction process S106. The classification criterion matrix D4 has a column of relationship labels 709 added to the classification criterion matrix D3.

[0103] The relationship label 709 is made up of multiple subitems related to each explanatory variable. A word constituting the name of the explanatory variable is set as the name of each column in the relationship label 709. For example, in the case of column 710 for the ratio of the number of academic researchers, "Academic" and "Ratio of the number of researchers" are set, and in the case of column 711 for the ratio of agricultural workers, "Agriculture", "Workers", and "Ratio of the number" are set, respectively. Furthermore, a word representing the relationship label is set for the data in each column in the relationship label 709. For example, in the case of data in each column in the relationship label 709, the content of the relationship is set as "Necessity" if the objective variable (content) is necessary for the explanatory variable (content); "Feasibility" if the objective variable (content) is not necessarily consistent with the explanatory variable (content) but is deliberately set as the objective variable (content); and "Other, Correlation" in other cases.

[0104] Next, as shown in FIG. 3, the relationship prediction program 3 executes a related word prediction process S107. <Related word prediction processing> FIG. 12 is a flow diagram illustrating the related word prediction process S107 in detail.

[0105] The related word prediction program 4 acquires data of rows related to the selected classification criterion selected in step S104 (selected classification criterion row data) from the classification criterion matrix D4 generated in the relationship prediction process S106 (step S501).

[0106] The related word prediction program 4 determines whether or not there is any explanatory variable for which the processes of steps S503 to S508 described below have not been executed among the explanatory variables associated with the selected classification criterion in the selected classification criterion row data (step S502).

[0107] If there are explanatory variables for which the processes of steps S503 to S508 have not been executed (step S502: YES), the related word prediction program 4 executes the process of step S503, and if there are no explanatory variables for which the processes of the related word prediction program 4 have not been executed (step S502: NO), the related word prediction program 4 stores the classification criterion matrix D4 generated up to this point in the main memory device 24 as a classification criterion matrix D5 (described later) (step S509), and the relationship prediction process S105 ends.

[0108] In step S503, the relationship prediction program 3 selects explanatory variables (selected explanatory variables) for which the processes in steps S504 to S508 have not been executed.

[0109] The related word prediction program 4 extracts the word group (word array Wa={wa1, wa2, ..., wa k}) and the word group of the element (content) of the objective variable (word array Wb = {wb1, wb2, ..., wb l}) and the word group of the relationship label name between the target variable and the selected explanatory variables (word array Wc={wc1,wc2,…,wc m}) and the word group of the elements (contents) of the relationship labels (word array Wd={wd1,wd2,...,wd n}) and the word group of relation labels (word array We={we1,we2,…,we p}) (step S504). The combination of word groups described here is an example, and the related word prediction program 4 may extract all or part of any items in the classification criterion matrix D4. A group of words may be extracted.

[0110] The related word prediction program 4 identifies the part of speech of each word in the word sequences Wa, Wb, Wc, Wd, and We using the word dictionary 11. Based on the information on each identified part of speech, the related word prediction program 4 generates a new word sequence Wf={wf1, wf2, ..., wf q} (step S505). For example, the related word prediction program 4 generates a word array Wf by removing auxiliary words such as particles or auxiliary verbs from each word in the word arrays Wa, Wb, Wc, Wd, and We.

[0111] Then, the related word prediction program 4 inputs the generated word array Wf into the word distributed representation model of the related word prediction model 15, converts each word output from the word distributed representation model into an r-dimensional word vector, and integrates each converted word vector into a word vector set Vf={Vf1, Vf2, ..., Vf q}(Vf1={vf 1_1 ,vf 1_2 ,…,vf 1_r}) is obtained (step S506).

[0112] The related word prediction program 4 uses the word vector set Vf={Vf1, Vf2, ..., Vf q} is the sum vector Vf t ={vf t_1 ,vf t_2 ,…,vf t_r} is obtained (step S507).

[0113] The related word prediction program 4 uses the similar vector search unit of the related word prediction model 15 to search for the total vector Vf t The program calculates the cosine similarity between each word in the word vector group stored in the word distributed representation model and identifies all word vectors whose calculated cosine similarity is equal to or greater than a threshold. The related word prediction program 4 acquires words (synonyms, i.e., related words) corresponding to each identified word vector. In this embodiment, the threshold is set to 0.5.

[0114] The related word prediction program 4 adds a new column (related word list 712) with the selected explanatory variable as its name to the classification criterion matrix D4, and stores the acquired related words (list of related words) in the row of the column related to the selected classification criterion selected in step S104 (step S508). Thereafter, the process of step S502 is performed to select a new explanatory variable.

[0115] (classification standard matrix) 13 is a diagram showing an example of the classification criterion matrix D5 generated in the related word prediction process S107. The classification criterion matrix D5 has a column of the related word list 712 added to the classification criterion matrix D4.

[0116] The related word list 712 is made up of multiple sub-items related to each explanatory variable. In each column in the related word list 712, a word in the name of the explanatory variable is set as the column name. For example, in the column 713 for the ratio of the number of academic researchers, "academic" and "ratio of number of researchers" are set, and in the column 714 for the ratio of agricultural workers, "agriculture," "worker," and "ratio of number" are set, respectively. A list of conjunctions is set in the data of each column in the related word list 712. For example, in the data of each column in the related word list 712, related words that are related to the name of the objective variable and the name of the explanatory variable, etc. (e.g., "medical care," "public health," "knowledge," "environmental issues," etc.) are set.

[0117] Next, as shown in FIG. 3, the relationship prediction program 3 executes a supplemental explanation generation process S108. <Supplementary explanation generation process> FIG. 14 is a flowchart illustrating the details of the supplemental explanation generation process S108.

[0118] The supplementary explanation generation program 5 uses the classification criterion matrix D5 generated in the related word prediction process S107. Then, data of the row related to the selected classification criterion selected in step S104 (selected classification criterion row data) is acquired (step S601).

[0119] The supplemental explanation generating program 5 determines whether or not there is any explanatory variable for which the processes of steps S603 to S606 described below have not been executed among the explanatory variables associated with the selected classification criterion in the selected classification criterion row data (step S602).

[0120] If there are explanatory variables for which the processing of steps S603 to S606 has not been executed (step S602: YES), the supplementary explanation generation program 5 executes the processing of step S603, and if there are no explanatory variables for which the processing of the related word prediction program 4 has not been executed (step S602: NO), the supplementary explanation generation program 5 stores the classification criterion matrix D5 generated so far in the main memory device 24 as a classification criterion matrix D7 (described later) (step S607), and the supplementary explanation generation processing S108 ends.

[0121] In step S603, the supplemental explanation generating program 5 selects explanatory variables (selected explanatory variables) for which the processes in steps S604 to S606 have not been executed.

[0122] The supplemental explanation generating program 5 acquires the contents (elements) of the selected objective variable, related words related to the selected explanatory variable, and relationship labels related to the selected explanatory variable from the classification criterion matrix D5 (step S604).

[0123] The supplemental explanation generation program 5 acquires an intention sentence associated with the content (element) of the selected objective variable acquired in step S604 and the relationship label related to the selected explanatory variable from the rationale intention list 16 (step S605). For example, when the relationship label is "necessity" and the content (element) of the objective variable is "yes", the supplemental explanation generation program 5 acquires a standard sentence that is the rationale intention, such as "urgent action is required".

[0124] The annotation generation program 5 generates an annotation matrix D7 that associates the contents (elements) of the selected objective variable acquired in step S604 with the intention sentence acquired in step S605. Thereafter, the annotation generation program 5 executes the process of step S602 to select a new explanatory variable.

[0125] (List of rationale intentions) 15 is a diagram showing an example of the reason intention list 16. The reason intention list 16 has data items of an objective variable 161 in which information on an objective variable is set, a relationship label 162 in which information on a relationship label associated with the objective variable is set, and a fixed phrase 163 in which an intention sentence, which is a sentence corresponding to a combination of the objective variable 161 and the relationship label 162, is set. Note that a sentence such as "urgent action required" is set in the fixed phrase 163 by the user in advance.

[0126] (Supplementary explanation matrix) 16 is a diagram showing an example of a supplementary explanation matrix D7. The supplementary explanation matrix D7 has rows of a plurality of classification criteria, columns of objective variables 171 in which elements (contents) of the classification criteria are set, and columns of supplementary explanations 172 in which supplementary explanations of explanatory variables are set.

[0127] The supplementary explanation 172 is made up of a plurality of items, and each supplementary explanation 172 has an intended sentence 174 and related words 175 related to each explanatory variable set therein.

[0128] Next, as shown in FIG. 3, the screen display program 6 displays the contents of the input data D1 and the supplemental explanation matrix D7 on the output device 32.

[0129] 17 is a diagram showing an example of a screen displayed by the screen display program 6. This screen 200 has a display field 201 for input data D1 and a display field 202 for a supplemental explanation matrix D7. The screen display program 6 also accepts input from the user in an end field 203 via the input device 31, and if there is input, the supplemental explanation generation process S108 ends.

[0130] This screen 200 allows the user to check the basis of the input data D1 in detail, improving the interpretability and social acceptability of the contents of the input data D1.

[0131] As described above, the analysis support device 20 of this embodiment generates relationship labels, which are phrases that represent the processing basis of the analysis model that calculated the value of the objective variable, based on sentences (word arrays Wab) containing words that represent explanatory variables in the input data D1 and sentences (word arrays Wab) containing words that represent objective variables and words that represent correlations, and identifies sentences that indicate the intention of the basis based on the generated relationship labels, and outputs the identified sentences.

[0132] That is, the analysis support device 20 of this embodiment uses sentences related to the objective variable and sentences related to the explanatory variables to predict words (relationship labels) that represent the basis for calculation from the explanatory variables to the objective variable, and thereby outputs sentences that represent the intention of the calculation basis of the analytical model.

[0133] This makes it possible to logically identify the basis for the analysis results obtained by the analytical model. In this way, the analysis support device 20 of this embodiment can estimate detailed logical relationships between explanatory variables and target variables.

[0134] In addition, the analysis support device 20 of this embodiment infers conjunctions that connect sentences containing words representing explanatory variables with sentences containing words representing target variables and words representing correlations, and identifies relationship labels corresponding to the inferred conjunctions.

[0135] This allows the logical relationship between the explanatory variables and the response variable to be clearly identified.

[0136] In particular, the analysis support device 20 of this embodiment infers a conjunction of order or adversative nature as the conjunction.

[0137] As a result, for example, when a sequential conjunction is inferred, the relationship between the explanatory variable and the objective variable can be identified as "necessity," and when a contradictory conjunction is inferred, the relationship between the explanatory variable and the objective variable can be identified as "feasibility."

[0138] Furthermore, the analysis support device 20 of this embodiment infers related words related to the explanatory variables and the target variables based on the words representing the explanatory variables and the words representing the target variables, and identifies sentences that indicate the intent of the evidence based on the inferred related words and relationship labels.

[0139] In this way, by using related words related to the objective variable and the explanatory variables to identify sentences that indicate the intention of the evidence, sentences that persuasively indicate the intention of the evidence can be identified.

[0140] Furthermore, the analysis support device 20 of this embodiment outputs the identified related words.

[0141] This allows the user to obtain more detailed information about the relationship between the dependent variable and the explanatory variables, enabling more accurate analysis of the variables.

[0142] Furthermore, the analysis support device 20 of this embodiment identifies a relationship label based on a sentence including the name and value of an explanatory variable, a sentence including the name and value of a target variable, and a sentence including a word that indicates a correlation. do.

[0143] In this way, by specifying the relationship label using the names and values ​​of the explanatory variables and the response variable, it is possible to specify a more accurate logical relationship between the explanatory variables and the response variable using the relationship label.

[0144] The present invention is not limited to the above-described embodiments, and can be implemented using any components within the scope of the present invention. The above-described embodiments and modifications are merely examples, and the present invention is not limited to these contents as long as the features of the invention are not impaired. Furthermore, although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of ​​the present invention are also included within the scope of the present invention.

[0145] For example, some of the hardware included in each device of this embodiment may be provided in another device. For example, the classification criteria extraction program 2, word management program 7, relationship prediction program 3, related word prediction program 4, supplementary explanation generation program 5, word dictionary 11, text set 12, mask part prediction model 13, relationship prediction model 14, related word prediction model 15, or reason intention list 16 of the analysis support device 20 can be realized as separate hardware, and may be installed outside a device external to the analysis support device 20, or may be installed so as to be accessible via the communication network 33.

[0146] Furthermore, each program of analysis support device 20 may be provided in another device, a program may consist of multiple programs, or multiple programs may be integrated into one program. [Explanation of symbols]

[0147] 1. Prediction result selection program 2 Classification criteria extraction program 3. Relationship prediction program 4 Related Word Prediction Program 5 Supplementary explanation generator 6 Screen Display Program 20 Analysis support equipment

Claims

1. a storage device that stores data indicating a correlation between values ​​of explanatory variables output by a trained model that outputs a response variable as output data from explanatory variables as input data and values ​​of the response variable; and a process of estimating words that contextually connect a sentence including a word representing the explanatory variable with a sentence including a word representing the objective variable and a word representing the correlation, and generating phrases that correspond to the estimated words and represent processing grounds of the trained model that calculates the value of the objective variable from the value of the explanatory variable; A process of identifying sentences that indicate the intention of the processing basis of the trained model based on the generated phrases; and outputting the identified sentence to an output device. Analysis support equipment.

2. The analysis support device according to claim 1 , wherein the processing device infers that the connecting word is a conjunction of a direct or adversative nature.

3. 2. The analysis support device according to claim 1, wherein the processing device estimates related words associated with the explanatory variables and the dependent variable based on the words representing the explanatory variables and the words representing the dependent variable, and identifies sentences indicating an intention of a processing basis of the trained model based on the estimated related words and the generated phrases.

4. The analysis support device according to claim 3 , wherein the processing device outputs the estimated related words to an output device.

5. 2. The analysis support device according to claim 1, wherein the processing device generates a phrase expressing a processing basis of the trained model that calculated the value of the dependent variable, based on a sentence including a name and a value of the explanatory variable, a sentence including a name and a value of the dependent variable, and a sentence including a word expressing the correlation.

6. The information processing device storing data indicating a correlation between values ​​of explanatory variables output by a trained model that outputs dependent variables as output data from explanatory variables as input data and values ​​of the dependent variables; a process of estimating words that contextually connect a sentence including a word representing the explanatory variable with a sentence including a word representing the objective variable and a word representing the correlation, and generating phrases that correspond to the estimated words and represent processing grounds of the trained model that calculates the value of the objective variable from the value of the explanatory variable; A process of identifying sentences that indicate the intention of the processing basis of the trained model based on the generated phrases; and outputting the identified sentence to an output device. Analysis support method.

7. The information processing device estimates a conjunction indicating a sequence or a contrast as the word to be connected. The analysis support method according to claim 6.

8. 7. The analysis support method according to claim 6, wherein the information processing device estimates related words associated with the explanatory variables and the target variable based on the words representing the explanatory variables and the words representing the target variable, and identifies sentences indicating the intention of a processing basis of the trained model based on the estimated related words and the generated phrases.

9. The analysis support method according to claim 8, wherein the information processing device outputs the estimated related words to an output device.

10. 7. The analysis support method according to claim 6, wherein the information processing device generates a phrase expressing a processing basis of the trained model that calculated the value of the dependent variable, based on a sentence including a name and a value of the explanatory variable, a sentence including a name and a value of the dependent variable, and a sentence including a word expressing the correlation.

11. In the information processing device, a process of storing data indicating a correlation between the values ​​of the explanatory variables output by a trained model that outputs a response variable as output data from an explanatory variable as input data and the response variable; a process of estimating words that contextually connect a sentence including a word representing the explanatory variable with a sentence including a word representing the objective variable and a word representing the correlation, and generating phrases that correspond to the estimated words and represent processing grounds of the trained model that calculates the value of the objective variable from the value of the explanatory variable; A process of identifying sentences that indicate the intention of the processing basis of the trained model based on the generated phrases; The process of outputting the identified text to the output device is executed. Analysis support program.

Images