Verification device, verification method, and verification program

Abstract

In order to automate the verification of a hypothesis sentence, a verification device (1) comprises a hypothesis sentence acquisition unit (11) that acquires a hypothesis sentence to be verified and a verification unit (12) that determines the truth or falsehood of the hypothesis sentence on the basis of a degree to which a premise sentence generated from knowledge derived from verification data used for hypothesis verification entails the hypothesis sentence.

Description

[Technical Field]

[0001] This relates to a verification device that verifies hypotheses. [Background technology]

[0002] Hypothesis verification based on data is important. For example, suppose a store employee feels that "products of category A sell well on rainy days." However, this is merely the employee's feeling, and it is not known whether "products of category A sell well on rainy days." In this example, the hypothesis that "products of category A sell well on rainy days" has not been verified based on data, so there is no rational reason to take action based on this hypothesis (for example, increasing the purchase of products of category A on rainy days).

[0003] Generally, hypothesis verification is performed by designing a data analysis task to verify whether the hypothesis is correct and then executing that data analysis task. For example, in the above example, if product sales data for each weather condition is collected, it can be verified based on the collected data whether the hypothesis "Product category A sells well on rainy days" is correct or not. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2019-117556 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional verification methods like those described above have the drawback of requiring a large amount of manpower and time. For this reason, there has been a demand for technology that can automatically verify hypotheses, especially hypotheses expressed in text like the examples above, but such technology has not yet existed.

[0006] An example of prior art related to hypothesis generation is the aforementioned Patent Document 1. Patent Document 1 discloses a technique for verifying whether a hypothesis that an error is contained in the recognition result of at least one of the recognition means in a recognition process using a local recognizer and a global recognizer is valid. However, this technique can only verify the hypothesis that an error is contained in the recognition result of at least one of the recognition means, and cannot verify a hypothesis sentence expressed in text.

[0007] One aspect of the present invention has been made in view of the above problems, and one object of the present invention is to provide a verification device or the like that can automatically verify a hypothesis sentence. [Means for solving the problem]

[0008] A verification device according to one aspect of the present invention comprises a hypothesis sentence acquisition means for acquiring a hypothesis sentence to be verified, and a verification means for determining the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0009] A verification method according to one aspect of the present invention includes at least one processor acquiring a hypothesis sentence to be verified, and determining the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0010] A verification program according to one aspect of the present invention causes a computer to function as a hypothesis sentence acquisition means for acquiring a hypothesis sentence to be verified, and a verification means for determining the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to automatically verify a hypothesis sentence. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of a verification device according to a first exemplary embodiment of the present invention. [Figure 2] 1 is a flowchart showing the flow of a verification method according to the first exemplary embodiment of the present invention. [Figure 3] FIG. 10 is a diagram illustrating an overview of a verification method according to a second exemplary embodiment of the present invention. [Figure 4] FIG. 10 is a block diagram showing the configuration of a verification device according to a second exemplary embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing an example from derivation of an insight to generation of a premise sentence in exemplary embodiment 2 of the present invention. [Figure 6] FIG. 10 is a diagram showing another example from the derivation of an insight to the generation of a premise sentence in exemplary embodiment 2 of the present invention. [Figure 7] FIG. 10 is a flowchart showing the flow of processing executed by a verification device according to the second exemplary embodiment of the present invention. [Figure 8] FIG. 1 is a diagram illustrating an example of a computer that executes instructions of a program, which is software that realizes the functions of each device according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0014] (Configuration of verification device) The configuration of a verification device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the verification device 1. As shown in Fig. 1, the verification device 1 includes a hypothesis sentence acquisition unit 11 (hypothesis sentence acquisition means) and a verification unit 12 (verification means).

[0015] The hypothesis sentence acquisition unit 11 acquires a hypothesis sentence to be verified. The verification unit 12 determines whether the hypothesis sentence is true or false based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0016] As described above, the verification device 1 according to this exemplary embodiment is configured to include a hypothesis sentence acquisition unit 11 that acquires a hypothesis sentence to be verified, and a verification unit 12 that determines the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence. With this configuration, the verification of the hypothesis sentence can be performed automatically.

[0017] (Verification Program) The functions of the verification device 1 described above can also be realized by a program. The verification program according to this exemplary embodiment causes a computer to function as a hypothesis sentence acquisition unit that acquires a hypothesis sentence to be verified, and a verification unit that determines the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence. This verification program enables automatic verification of the hypothesis sentence.

[0018] (Verification method flow) The flow of the verification method according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the verification method. Note that the execution entity of each step in this verification method may be a processor provided in the verification device 1, or a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0019] In S11, at least one processor obtains a hypothesis sentence to be verified.

[0020] In S12, at least one processor determines whether the hypothesis sentence is true or false based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0021] As described above, the verification method according to this exemplary embodiment includes at least one processor acquiring a hypothesis sentence to be verified (S11), and determining whether the hypothesis sentence is true or false based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence (S12). This verification method enables the verification of the hypothesis sentence to be performed automatically.

[0022] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will now be described in detail with reference to the drawings.

[0023] (Outline of verification method) FIG. 3 is a diagram showing an overview of a verification method (hereinafter referred to as the present method) according to this exemplary embodiment. The present method is a method for verifying an input hypothesis sentence (hereinafter also referred to as the input hypothesis). In the example of FIG. 3, the input hypothesis is a text "A man in his 40s is watching xxx," and it is determined whether this input hypothesis is correct or not based on verification data. The input hypothesis may be anything that can be determined to be correct or not. Furthermore, the input hypothesis may be anything that can be determined to be correct, incorrect, or neither.

[0024] The verification data may be any data from which insights, described below, can be derived, and its format (modality) is not particularly limited. For example, the verification data may be structured data such as a data table, semi-structured data in a format such as JSON (JavaScript Object Notation) or XML (eXtensible Markup Language), or unstructured data such as text data, image data, or audio data. The verification data may be obtained, for example, from a data lake that stores various data in various formats.

[0025] In this method, rather than using the validation data as is, insights are first derived from the validation data. Insights are knowledge that is meaningful to people. Therefore, "insights" in the following explanation can be read as "knowledge." Alternatively, insights can be said to be data related to insights that become clear from data on consumers or statistical subjects, such as consumer surveys or statistical research. For example, visualizations of validation data, such as various graphs (bar graphs, line graphs) as shown in Figure 3, are also insights.

[0026] Causal graphs and predictive models generated using validation data are also insights. A causal graph is data structured with multiple nodes and links connecting the nodes. In a causal graph, the causal relationships between nodes are represented by links.

[0027] In this method, it is possible to use, as insights, results obtained by analyzing the verification data using various analytical techniques. For example, if the verification data indicates product purchase history, information indicating combinations of products that are purchased together, identified by basket analysis of the verification data, may be used as insights. Details of the method for deriving insights will be described later.

[0028] Next, the method verbalizes the insight derived as described above. In other words, the method generates text related to the insight from the insight. The method uses the text generated in this way as a premise sentence to be used for verifying the input hypothesis. For example, in the example of Figure 3, premise sentences such as "Men in their 40s watch xxx the most," "People who like alcohol have a large amount of cryptocurrency," and "When yyy, zzz occurs" are generated. The method for generating premise sentences will be described in detail later.

[0029] The process from deriving insights to generating premise sentences does not necessarily have to be performed at the time of hypothesis verification, but can be performed beforehand. Deriving various insights from various verification data and generating premise sentences from those insights can improve the accuracy of hypothesis verification.

[0030] In this method, the truth or falsity of an input hypothesis is determined based on the degree to which the premise sentence generated as described above implies the input hypothesis. When a premise sentence entails a hypothesis sentence, it means that the premise sentence and the hypothesis sentence contain the same content. If the premise sentence generated based on the verification data contains the same content as the input hypothesis, it can be said that the verification data supports the fact that the input hypothesis is correct. Therefore, the truth or falsity of an input hypothesis can be determined based on the degree to which the premise sentence generated based on the verification data implies the input hypothesis.

[0031] In the example of Figure 3, the premise sentence "The people who watch xxx the most are men in their 40s" is determined to imply the input hypothesis, and based on this determination result, a verification result that the input hypothesis is correct is output. In this way, according to this method, the verification of the input hypothesis can be performed automatically.

[0032] This method can be used to verify, for example, whether the tendency felt by on-site personnel (e.g., the hypothesis exemplified in the "Background Technology") is correct, based on verification data. This method can also be used to verify the results of AI (Artificial Intelligence) estimation. In this case, the AI ​​estimation results can be expressed in text form as a hypothesis statement. This allows the validity of the AI ​​estimation results to be verified using verification data that is unrelated to the training data used for AI learning.

[0033] (Configuration of verification device) The configuration of the verification device 2 according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the verification device 2. The verification device 2 is a device that verifies hypothesis sentences, and as shown in the figure, includes a control unit 20 that controls each unit of the verification device 2 in an integrated manner, and a storage unit 21 that stores various data used by the verification device 2. The verification device 2 also includes an input unit 22 that accepts user input operations to the verification device 2, and an output unit 23 that outputs data from the verification device 2. The verification device 2 may be a device dedicated to verifying hypothesis sentences, or may be a general-purpose device that can be used for other purposes.

[0034] The control unit 20 also includes a data acquisition unit 201 (hypothesis sentence acquisition means, verification data acquisition means), an insight derivation unit (knowledge derivation means) 202, a premise sentence generation unit (premises sentence generation means) 203, a verification unit (verification means) 204, and a verification result display unit 205. The storage unit 21 stores a hypothesis sentence 211, verification data 212, a premise sentence 213, a language understanding model 214, and a verification result 215.

[0035] The data acquisition unit 201 acquires a hypothesis sentence to be verified, and stores the acquired hypothesis sentence in the storage unit 21 as a hypothesis sentence 211. The data acquisition unit 201 also acquires verification data to be used for verifying the hypothesis, and stores the acquired verification data in the storage unit 21 as verification data 212. Of course, the configuration may be such that the acquisition of the hypothesis sentence and the acquisition of the verification data are executed by separate processing blocks.

[0036] The insight derivation unit 202 derives an insight from the verification data 212 acquired by the data acquisition unit 201. Then, the premise sentence generation unit 203 generates a premise sentence related to the insight derived by the insight derivation unit 202, and stores it in the storage unit 21 as a premise sentence 213. Note that the method of deriving an insight and the method of generating a premise sentence will be described later.

[0037] The verification unit 204 determines whether the hypothesis sentence to be verified acquired by the data acquisition unit 201 is true or false based on the degree to which a premise sentence 213 generated from an insight derived from the verification data 212 implies the hypothesis sentence to be verified, which is acquired by the data acquisition unit 201. Then, the verification unit 204 stores the determination result of whether the hypothesis sentence is true or false in the storage unit 21 as a verification result 215. The method for determining the degree of implication will be described later.

[0038] The verification result display unit 205 displays the verification result 215. There is no particular limitation on the type of device on which the verification result 215 is displayed. For example, if the output unit 23 is a display device, the verification result display unit 205 may display and output the verification result 215 to the output unit 23. Furthermore, for example, the verification result display unit 205 may display the verification result 215 on a display device connected to the verification device 2 by wire or wirelessly. Of course, the output mode of the verification result 215 is arbitrary and is not limited to display and output. Furthermore, the verification result 215 may not be output, but may be stored in the storage unit 21 or a storage device connected to the verification device 2 by wire or wirelessly.

[0039] As described above, the verification device 2 according to this exemplary embodiment is configured to include a data acquisition unit 201 that acquires a hypothesis sentence to be verified, and a verification unit 204 that determines the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from an insight derived from verification data 212 used to verify the hypothesis implies the hypothesis sentence to be verified. This configuration has the effect of enabling the verification of the hypothesis sentence to be performed automatically.

[0040] As described above, the verification device 2 according to this exemplary embodiment may include a data acquisition unit 201 that acquires verification data 212, an insight derivation unit 202 that derives an insight from the verification data 212 acquired by the data acquisition unit 201, and a premise sentence generation unit 203 that generates a premise sentence related to the insight from the insight derived by the insight derivation unit 202. This configuration provides the effect of being able to automatically generate a premise sentence from the verification data 212, in addition to the effects achieved by the verification device 1 according to the first exemplary embodiment.

[0041] (How to derive insights) A description will now be given of a method for deriving insights by the insight deriving unit 202. As described with reference to Fig. 3, this method can apply various types of insights, and therefore various methods for deriving insights can also be applied.

[0042] For example, a technology called QuickInsights described in the following document can be used to automatically generate a graph from a table. Therefore, the insight derivation unit 202 may derive a graph, i.e., an insight, from the verification data 212 in table format using QuickInsights. Rui Ding, Shi Han, Yong Xu, Haidong Zhang, Dongmei Zhang “QuickInsights: Quick and Automatic Discovery of Insights from Multi-Dimensional Data” The insight derivation unit 202 may also generate a machine learning model from the verification data 212 using an AutoML (Automated Machine Learning) technique. In this case, the machine learning model is the insight. When applying AutoML, if it is necessary to specify a target variable and explanatory variables of the machine learning model from elements included in the verification data 212, the insight derivation unit 202 may specify them automatically, or may allow the user of the verification device 2 to specify them. In addition, the insight derivation unit 202 can also derive insights from the verification data 212 using techniques such as AutoBI or AutoCI.

[0043] AutoBI (Automated Business Intelligence) is a technology that streamlines sales activities by automatically aggregating and visualizing accumulated data such as table data. When applying AutoBI, if it is necessary to specify an analysis perspective (for example, coordinate axes for deriving a visualized graph) from among the elements included in the verification data 212, these may be specified automatically or by the user.

[0044] AutoCI (Automated Customer Intelligence) is a technology that automates data analysis related to business activities such as marketing, sales, and service, and aims to improve the efficiency of customer understanding. When applying AutoCI, if it is necessary to specify parameters related to customer understanding (for example, customer rank, determined by purchase amount, as to whether a customer is a good customer or not) from among the elements included in the verification data 212, these may be specified automatically or by the user.

[0045] Furthermore, the insight derivation unit 202 can also derive, as an insight, a causal graph that represents the relationships using nodes and links from the verification data 212 that indicates correspondences and correlations.

[0046] In addition, the insight derivation unit 202 may use, as an insight, results obtained by analyzing the verification data 212 using various analytical methods. For example, the insight derivation unit 202 may use, as an insight, a prediction formula obtained by regression analysis or multiple regression analysis from the tabular verification data 212. Furthermore, when the verification data 212 indicates a product purchase history, the insight derivation unit 202 may use, as an insight, information indicating a combination of products that are purchased together, which is identified by performing a basket analysis of the verification data 212.

[0047] Furthermore, when the verification data 212 is text, the insight derivation unit 202 may generate a summary of the text as an insight. When the verification data 212 is image data, the insight derivation unit 202 may determine the subject of the image data and use information indicating the subject as an insight. When the verification data 212 is audio data, the insight derivation unit 202 may convert the audio data into text by speech recognition and use the converted text as an insight. Note that known methods may be used for generating a summary from text, determining the subject of image data, and converting audio data into text.

[0048] In this way, various insights can be derived from various formats of verification data 212. Therefore, the data acquisition unit 201 may acquire verification data 212 in a plurality of data formats. In this case, the insight derivation unit 202 may apply derivation rules prepared for each data format to derive insights from the verification data 212 in each data format.

[0049] This configuration provides the effect of being able to automatically derive insights from verification data 212 in multiple data formats, in addition to the effect of the verification device 1 according to exemplary embodiment 1. Note that the derivation rules may be, for example, a rule base that indicates how insights should be generated from what kind of verification data 212, or may be a machine learning model that learns by machine learning the relationship between the verification data 212 and the corresponding insights.

[0050] (How to generate a premise sentence) The method for generating premise sentences by the premise sentence generation unit 203 will now be described. As described above, the insights derived by the insight derivation unit 202 may be in a variety of formats. For this reason, it is sufficient to prepare generation rules for premise sentences according to the data format of the insight. This allows the premise sentence generation unit 203 to generate premise sentences by applying the generation rules according to the data format of the insight.

[0051] For example, by using the Chart-to-Text technology described in the following document, it is possible to generate natural language summaries from graphs such as bar graphs and line graphs. Therefore, Chart-to-Text can also be applied as a generation rule for generating premise sentences from graphs such as bar graphs and line graphs. Jason Obeid, Enamul Hoque “Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model” arXiv:2010.09142v2 [cs.CL] 29 Nov. 2020 Furthermore, if a generation rule for premise sentences for a causal graph is prepared, when the insight derived by the insight derivation unit 202 is a causal graph, it is possible to generate premise sentences from the causal graph. For example, a template such as "{node} is in a {link} relationship with {node linking to this node}" may be used as a generation rule for a causal graph. The premise sentence generation unit 203 can generate premise sentences related to the causal graph by inputting various information related to the nodes and links shown in the causal graph into this template.

[0052] In this way, when the insight derivation unit 202 derives insights of multiple data formats, the premise sentence generation unit 203 may apply generation rules prepared for each data format of the insight to generate premise sentences from the insights of each data format.

[0053] This configuration achieves the effect of being able to automatically generate premise sentences from insights in multiple data formats, in addition to the effect achieved by the verification device 1 according to the first exemplary embodiment. The generation rules may be, for example, a rule base that indicates how premise sentences are generated from what types of insights, or a machine learning model that learns the relationship between insights and their corresponding premise sentences through machine learning. The above-described template is an example of a configuration that applies a rule base, and the above-described Chart-to-Text is an example of a configuration that applies a machine learning model.

[0054] (Example 1: From deriving an insight to generating a premise sentence) 5 is a diagram showing an example of the process from deriving an insight to generating a premise sentence. Verification data 212 shown in Fig. 5 is data in a table format showing the relationship between product names, unit prices, and sales.

[0055] In this case, the insight derivation unit 202 applies the insight derivation rules for the table-formatted verification data 212 to derive insights from the verification data 212. Specifically, in the example of Fig. 5, the insight derivation unit 202 applies the derivation rules that set the elements in the leftmost column as the horizontal axis series and the elements in the other columns as the vertical axis series, thereby deriving a bar graph showing sales for each product name.

[0056] Although not shown, the insight derivation unit 202 can similarly derive a bar graph showing the unit price for each product name. In this way, multiple insights may be derived from one piece of verification data 212. When deriving multiple insights from one piece of verification data 212, different derivation rules may be applied to each of the multiple insights.

[0057] When generating a premise sentence for the above-described graph-format insight, the premise sentence generation unit 203 may apply a generation rule for premise sentences for graph-format insights. For example, the premise sentence generation unit 203 may use a template such as "The series name on the vertical axis is highest {the element on the horizontal axis with the highest vertical axis value}" as a generation rule. This allows the premise sentence generation unit 203 to generate the premise sentence "B has the highest sales," as shown in FIG. 5. Of course, the premise sentence generation unit 203 may generate a premise sentence using a machine learning model such as the Chart-to-Text model described above.

[0058] (Example 2: From insight derivation to premise generation) Fig. 6 is a diagram showing another example of the process from deriving an insight to generating a premise sentence. The verification data 212 shown in Fig. 6 is data in a table format showing the relationship between the user name, the purchase amount for each product category (specifically, the purchase amount for alcohol, pet supplies, etc.), and the amount of cryptocurrency held.

[0059] 6 shows an example in which the insight derivation unit 202 derives, as an insight, a prediction formula that predicts a dependent variable from explanatory variables. In this case, the insight derivation unit 202 selects elements to be used as explanatory variables and dependent variables from among elements included in the verification data 212. If selection criteria are set in advance, the insight derivation unit 202 can perform this selection automatically. Alternatively, the insight derivation unit 202 may present each element included in the verification data 212 to the user, allowing the user to select elements to be used as explanatory variables and dependent variables.

[0060] In the example of Figure 6, the amount of cryptocurrency held is selected as the objective variable Y, and the purchase amount for each product category is selected as the explanatory variable, and a prediction formula (specifically, a regression model) is derived: Y (amount of cryptocurrency held) = 0.4 * (amount of alcohol purchases) + 0.1 * (amount of pet supplies purchases) + ...

[0061] When generating a premise sentence from the regression model described above, the premise sentence generation unit 203 may apply a generation rule for premise sentences for regression models. For example, the premise sentence generation unit 203 may use a template such as "People with a large number of {explanatory variables} with the highest correlation coefficient have a large number of {target variables}" as a generation rule. This allows the premise sentence generation unit 203 to generate a premise sentence such as "People who spend a lot on alcoholic beverages have a large amount of cryptocurrency holdings," as shown in FIG. 6.

[0062] (Method for determining the degree of implication) The following describes a method for determining the degree of entailment by the verification unit 204. The determination method may be any method capable of deriving information indicating the degree of entailment between two texts (i.e., a hypothesis sentence and a premise sentence) from the two texts. The verification unit 204 of this exemplary embodiment determines the degree of entailment using the language understanding model 214.

[0063] The language understanding model 214 is a model constructed so that, when a pair of a premise sentence and a premise sentence is input, it outputs an entailment score, which is an index value indicating the degree to which the premise sentence implies the premise sentence. Such a language understanding model 214 can be constructed by learning whether or not the premise sentence entails the premise sentence, using pairs of premise sentences and premise sentences whose entailment relationships are known as training data.

[0064] The language understanding model 214 may be a combination of a pre-trained language model that converts a document into a vector based on its context and a language task model that classifies the document. In this case, the pre-trained language model converts each of the premise sentence and the hypothesis sentence into vectors, and the language task model calculates an entailment score that indicates the degree to which the premise sentence implies the hypothesis sentence from these vectors.

[0065] The implication score calculated using the language understanding model 214 indicates the degree to which a premise sentence input to the language understanding model 214 implies a hypothesis sentence also input to the language understanding model 214. A high implication score indicates that the hypothesis sentence is likely to be correct, and this is supported by the insight or verification data 212 that is the basis of the premise sentence.

[0066] The method for determining the degree of implication is not limited to the above-described method that uses the language understanding model 214 constructed using training data. For example, the verification unit 204 may calculate the similarity between the premise sentence and the hypothesis sentence that have been vectorized by the pre-trained language model, and use the calculated similarity as an index value indicating the degree of implication.

[0067] Furthermore, any method can be applied to determine the degree of entailment as long as it can define the relationship between the hypothesis sentence and the premise sentence. For example, existing methods such as keyword matching and TF-IDF (Inverse Document Frequency) can be used to determine the degree of entailment.

[0068] Furthermore, auxiliary information of a hypothesis sentence may be used in determining the degree of implication. The auxiliary information of a hypothesis sentence is information related to the hypothesis sentence but not expressed as text in the hypothesis sentence. An example of auxiliary information is time information when the hypothesis sentence is generated or input. The auxiliary information may also be information about the entity that generates the hypothesis sentence. For example, when planning an advertising strategy in the retail industry, if the hypothesis sentence "men in their 40s watch xxx the most" is verified, the entity that generates the hypothesis sentence is information about the company that plans the advertising strategy. Examples of information about a company include information indicating the company's attributes, such as the company's products, services it offers, sales format, industry, size, etc.

[0069] In the method for determining the degree of implication, the hypothesis sentence may be expanded using this auxiliary information. For example, for a hypothesis sentence "The people who watch xxx the most are men in their 40s," if the auxiliary information is information indicating a corporate attribute, "movie industry," the verification unit 204 may generate a hypothesis sentence, "At movie theaters, the people who watch xxx the most are men in their 40s." Alternatively, the verification unit 204 may add vector data for the word "movie industry" to the vector data of the generated hypothesis sentence. In this case, as in the case of generating the hypothesis sentence described above, verification that takes auxiliary information into account can be realized.

[0070] The verification unit 204 may store the calculated implication scores as they are in the storage unit 21 as the verification result 215. Furthermore, the verification unit 204 may determine that the hypothesis sentence is correct if a predetermined number or more of the implication scores calculated for each of the multiple premise sentences exceed a predetermined threshold, and may determine that the hypothesis sentence is incorrect if the number of implication scores exceeding the threshold is less than a predetermined number. In this case, the verification unit 204 may store whether the hypothesis sentence is correct or incorrect as the verification result 215. Furthermore, the verification unit 204 may, for example, use a predetermined number of premise sentences with the highest implication scores among the multiple premise sentences used to calculate the implication scores as a result of determining whether the hypothesis sentence is true or false. In this case, the verification unit 204 may store the predetermined number of premise sentences with the highest implication scores as the verification result 215.

[0071] As described above, the verification unit 204 may determine the truth or falsity of a hypothesis sentence based on an implication score that indicates the degree to which a premise sentence generated from an insight implies the hypothesis sentence to be verified, calculated using the language understanding model 214 constructed by learning whether a premise sentence implies the hypothesis sentence. This configuration provides the effect of obtaining a highly accurate truth or falsity determination result in addition to the effects achieved by the verification device 1 according to the first exemplary embodiment.

[0072] (Processing flow) The flow of the process (verification method) executed by the verification device 2 will be described with reference to Fig. 7. Fig. 7 is a flow chart showing the flow of the process executed by the verification device 2.

[0073] In S21, the data acquisition unit 201 acquires verification data and stores it in the storage unit 21 as verification data 212. The data acquisition unit 201 may acquire verification data input via the input unit 22, or may acquire verification data from a storage location specified by the user of the verification device 2 (which may be within the storage unit 21 of the verification device 2 or may be a storage device outside the verification device 2).

[0074] In S22, the insight derivation unit 202 derives an insight from the verification data 212 acquired and stored in S11. The insight derivation unit 202 may store the derived insight in the storage unit 21. Subsequently, in S23, the premise sentence generation unit 203 verbalizes the insight derived in S12 to generate a premise sentence, and stores the generated premise sentence in the storage unit 21 as premise sentence 213.

[0075] In S24, the data acquisition unit 201 acquires the hypothesis sentence to be verified and stores it in the storage unit 21 as the hypothesis sentence 211. The data acquisition unit 201 may acquire the hypothesis sentence input via the input unit 22. Note that the data acquisition unit 201 may also acquire the hypothesis sentence when acquiring the verification data in S21.

[0076] In S25, the verification unit 204 inputs the premise sentence 213 and the hypothesis sentence 211 into the language understanding model 214, and calculates an implication score indicating the degree to which the premise sentence 213 implies the hypothesis sentence 211. Note that the implication score is calculated for each of the plurality of premise sentences 213 stored in the storage unit 21.

[0077] In S26, the verification unit 204 determines whether the hypothesis sentence 211 is true or false using the implication scores calculated in S25. For example, the verification unit 204 may use the implication scores calculated in S25 as the true or false determination result, or may use a predetermined top number of the implication scores calculated in S25 as the true or false determination result. Furthermore, for example, the verification unit 204 may determine whether the hypothesis sentence 211 is true or false based on whether the implication scores calculated in S25 include any that exceed a predetermined threshold. Then, the verification unit 204 stores such a determination result in the storage unit 21 as the verification result 215.

[0078] In S27, the verification result display unit 205 displays the determination result of S26. At this time, the verification result display unit 205 may display the premise sentence that served as the basis for determining whether the hypothesis is true or false, as well as the insight and verification data 212 that were the basis for the premise sentence. For example, the verification result display unit 205 may display the premise sentence for which an implication score exceeding a threshold was calculated, as well as the insight and verification data 212 that were the basis for the premise sentence. Furthermore, for example, the verification result display unit 205 may display a predetermined number of premise sentences with the highest implication scores in descending order of implication score, and may also display the insight and verification data 212 that were the basis for those premise sentences.

[0079] [Modification] The execution entity of each process described in the above embodiment is arbitrary and is not limited to the above example. In other words, a verification system having the same functions as the verification device 2 can be constructed using multiple devices that can communicate with each other. For example, a verification system having the same functions as the verification device 2 can be constructed by distributing each block shown in FIG. 4 among multiple devices. For example, the derivation of insights, the generation of premise sentences, and the verification of hypotheses may each be executed by separate devices.

[0080] [Software implementation example] Some or all of the functions of the verification device 2 may be realized by hardware such as an integrated circuit (IC chip), or by software. In the latter case, the verification device 2 is realized by, for example, a computer that executes instructions in a program (verification program), which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 8. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the verification device 2. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the verification device 2.

[0081] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0082] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0083] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0084] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0085] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0086] (Appendix 1) A verification device comprising: a hypothesis sentence acquisition means for acquiring a hypothesis sentence to be verified; and a verification means for determining the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0087] (Appendix 2) The verification device described in Appendix 1, wherein the verification means determines the truth or falsity of the hypothesis sentence based on an implication score indicating the degree to which the premise sentence generated from the knowledge implies the hypothesis sentence acquired by the hypothesis sentence acquisition means, the implication score being calculated using a language understanding model constructed by learning whether the premise sentence implies the hypothesis sentence.

[0088] (Appendix 3) The verification device described in Appendix 1 or 2 comprises a verification data acquisition means for acquiring the verification data, a knowledge derivation means for deriving the knowledge from the verification data acquired by the verification data acquisition means, and a premise sentence generation means for generating the premise sentence related to the knowledge derived by the knowledge derivation means.

[0089] (Appendix 4) The verification device described in Appendix 3, wherein the verification data acquisition means acquires the verification data in a plurality of data formats, and the knowledge derivation means applies derivation rules prepared for each data format to derive the knowledge from the verification data in each data format.

[0090] (Appendix 5) The verification device described in Appendix 3 or 4, wherein the knowledge derivation means derives the knowledge in a plurality of data formats, and the premise sentence generation means applies generation rules prepared for each data format of the knowledge to generate the premise sentence from the knowledge in each data format.

[0091] (Appendix 6) A verification method comprising: at least one processor acquiring a hypothesis sentence to be verified; and determining whether the hypothesis sentence is true or false based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0092] (Appendix 7) Computer, A verification program that functions as a hypothesis sentence acquisition means that acquires a hypothesis sentence to be verified, and a verification means that determines the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0093] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows: A verification device comprising at least one processor that executes a hypothesis sentence acquisition process for acquiring a hypothesis sentence to be verified, and a verification process for determining the truth or falsity of the hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis implies the hypothesis sentence.

[0094] The verification device may further include a memory that stores a program for causing the processor to execute the hypothesis sentence acquisition process and the verification process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0095] 1, 2 Verification device 11 Hypothesis sentence acquisition section 12, 204 Verification Department 20 Control Unit 21 Memory section 22 Input section 23 Output section 201 Data Acquisition Department 202 Insight Derivation Department 203 Premise sentence generation part 205 Verification result display section 211 Hypothetical Sentences 212 Verification Data 213 Premise sentence 214 Language Understanding Model 215 Verification Results C1 processor C2 Memory

Claims

1. a hypothesis sentence acquisition means for acquiring a hypothesis sentence to be verified; and a verification means for determining the truth or falsity of a hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify a hypothesis by applying a generation rule corresponding to the data format of the knowledge implies the hypothesis sentence.

2. the verification means determines the truth or falsity of the hypothesis sentence based on an implication score indicating the degree to which the premise sentence generated from the knowledge implies the hypothesis sentence acquired by the hypothesis sentence acquisition means, the implication score being calculated using a language understanding model constructed by learning whether the premise sentence implies the hypothesis sentence; The verification device according to claim 1 .

3. verification data acquisition means for acquiring the verification data; knowledge derivation means for deriving the knowledge from the verification data acquired by the verification data acquisition means; 3. The verification device according to claim 1, further comprising: a premise sentence generation means for generating the premise sentence related to the knowledge derived by the knowledge derivation means by applying a generation rule according to a data format of the knowledge.

4. the verification data acquisition means acquires the verification data in a plurality of data formats; 4. The verification device according to claim 3, wherein the knowledge derivation means derives the knowledge from the verification data of each data format by applying a derivation rule prepared for each data format.

5. the knowledge derivation means derives the knowledge in a plurality of data formats; 5. The verification device according to claim 3, wherein the premise sentence generation means applies a generation rule prepared for each data format of the knowledge, and generates the premise sentence from the knowledge in each data format by applying a generation rule corresponding to the data format of the knowledge.

6. At least one processor Obtaining a hypothesis sentence to be verified; and determining whether the hypothesis sentence is true or false based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify the hypothesis by applying a generation rule corresponding to the data format of the knowledge implies the hypothesis sentence.

7. Computer, a hypothesis sentence acquisition means for acquiring a hypothesis sentence to be verified; a verification program that functions as a verification means that determines the truth or falsity of a hypothesis sentence based on the degree to which a premise sentence generated from knowledge derived from verification data used to verify a hypothesis by applying a generation rule corresponding to the data format of the knowledge implies the hypothesis sentence.

Images