Unsupervised common sense automatic question answering method based on different knowledge carriers

Abstract

The invention belongs to the technical field of natural language processing, and particularly relates to an unsupervised common sense automatic question answering method based on different knowledge carriers. In order to accurately select correct options of common sense questions and answers, the method comprises the following steps: crawling common sense question and answer selection questions from an examination website of the Internet, preprocessing the common sense question and answer selection questions, and constructing a common sense question and answer data set; classifying common knowledge problems; generating common knowledge; and selecting two typical samples from the common sense question and answer data set, sending the samples into the question and answer model and obtaining answers.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to an unsupervised common sense automatic question answering method based on different knowledge carriers. Background Technology

[0002] In recent years, equipping computers with common sense has become a major challenge in the field of artificial intelligence. To choose the correct reason for the fact that "my body casts a shadow on the grass," a model should know that "light is blocked, creating a shadow" and "the sun shines." As humans, we can make this association and judgment immediately because this knowledge is deeply ingrained in our minds through daily observations and experiences, perhaps through long-term education—this is what we call common sense. However, equipping models with this ability is far more difficult. Therefore, common sense question answering tasks are a key focus and a major challenge in the field of natural language processing.

[0003] Commonsense question answering (CQA) is a more challenging natural language understanding task than traditional question answering tasks because it requires additional commonsense knowledge that cannot be directly obtained from the given context to arrive at appropriate answers. Different expressions of commonsense play a crucial role in CQA. Unsupervised CQA models, such as Self-talk and ArT, struggle to design universal question prefixes, and even when they can, they neglect the role of different knowledge carriers in answering commonsense questions. Everyday commonsense is often more implicit and better expressed through discourse; while scientific commonsense is usually organized in rule-based forms, with questions and options often containing scientific terminology, making it more suitable for statements. Therefore, traditional unsupervised models often struggle to accurately select the correct answer in commonsense questions. Summary of the Invention

[0004] This invention addresses the aforementioned problems by providing an unsupervised common sense automatic question answering method based on different knowledge carriers, which can fully utilize the advantages of different knowledge carriers to help the system answer questions more accurately.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] An unsupervised commonsense automatic question answering method based on different knowledge carriers includes the following steps:

[0007] S1, Construct a commonsense question-and-answer dataset;

[0008] S2, categorize common sense questions;

[0009] S3, Common Sense Knowledge Generation;

[0010] S4, Reasoning for the answer to the question.

[0011] Furthermore, S1 specifically refers to:

[0012] S101 uses web scraping technology to crawl common sense multiple-choice questions from relevant websites on the Internet;

[0013] S102 preprocesses the common sense question-and-answer multiple-choice questions crawled in S101, i.e., constructs a dictionary consisting of context, question, all candidate options, and correct answer label.

[0014] Furthermore, S2 specifically refers to:

[0015] S201, combine the context, question, and options to form a statement STi O: with O i Taking C and Q as input, the original interrogative sentence Q is rewritten as a declarative statement, and then... <C,Q,O i >Concatenate into a declarative statement STi O;

[0016] S202, Design a hint example and hint template, fill in the STi O corresponding to the correct answer into the hint template to form a complete hint template;

[0017] S203, determine whether STi O is a daily event or common scientific knowledge. Input the prompt example and prompt template into the pre-trained large language model, and let the large language model determine whether the STi O is a daily event or common scientific knowledge.

[0018] Furthermore, S3 specifically refers to:

[0019] S301, Keyword Extraction: For the declarative statement STi O mentioned in S201, an unsupervised key phrase extractor is used to extract key phrases. Three types of key phrases are extracted: noun phrases, verb phrases, and personal name phrases. Context-free grammar rules are designed for each type of key phrase.

[0020] NP→(nn|adj) ∗+nn;

[0021] VP→vb+(pr){0,1}+NP;

[0022] PNP→(pn){1,2};

[0023] Where NP is a noun phrase, VP is a verb phrase, PNP is a personal name phrase and is a non-terminator; vb is a verb, nn is a noun, adj is an adjective, pn is a personal name, pr is a preposition and is a determiner; + indicates concatenation; {a,b} indicates the repetition frequency range from a to b; ∗ is equivalent to {0,∞};

[0024] Adding context-free grammar rules to NLTK's RegexpPaser tool extracts the top 5 most important key phrases without requiring labels or model fine-tuning, but the word embeddings are obtained from a pre-trained language model;

[0025] S302, Discourse or Sentence Generation: After obtaining key phrases, retrieve discourse or sentence template sets according to common sense question types, and generate discourse prefixes or sentence prefixes;

[0026] If the common-sense question pertains to everyday events, when generating discourse prefixes, consider the following:

[0027] (1) For a given object, consider "a past experience with the object";

[0028] (2) For a behavior, consider “the specific experiences after the behavior occurred”, “the specific experiences before the behavior occurred”, or “the specific experiences related to the behavior”;

[0029] (3) For a person, think about “the impression of him” or “the specific experiences related to his motivation / feeling / reaction to do something”;

[0030] If the common sense question is scientific common sense, generate a statement prefix, considering the following cases:

[0031] (1) For an object, think about "what it is" and "what its purpose is";

[0032] (2) For a behavior, think about "what it means" and "what happened before and after it occurred";

[0033] (3) For a person, think about "who he / she is" and "what his / her feelings / motivations / reactions are";

[0034] A lookup table is constructed using key phrase types as keys and a list of discourse or sentence templates as values. The lookup table is used to quickly retrieve the templates corresponding to the key phrases and replace the tags with the key phrases to form discourse or sentence prefixes.

[0035] For each discourse or statement prefix, it is concatenated with the context and input into a generative pre-trained language model to generate a complete discourse or statement. A kernel sampling of p=0.8 is used as the decoding strategy to increase the diversity of the generated text. Simultaneously, to ensure the quality and quantity of the generated content, all generated discourses or statements are sorted according to the perplexity estimated by the pre-trained language model, and the top K discourses or statements are retained to construct a discourse or statement set: {discourse k}k K=1 / {sentence k}kK=1.

[0036] Furthermore, S4 specifically includes:

[0037] S401, construct the score calculation formula for the discourse or statement, and insert STi O as additional knowledge into each discourse or statement to assist in the scoring of subsequent options;

[0038] O i Regarding Discourse k or sentence k The formula for calculating a fraction is:

[0039] (1);

[0040] (2);

[0041] (3);

[0042] Where "+" indicates connection and "-" indicates deletion, P LM Indicates the likelihood of a sentence.

[0043] S402, constructing a calculation formula for answer reasoning. For causal reasoning questions, introduce reverse thinking to perform bidirectional inference between cause and effect. <O i Q R C> are connected in order, denoted as STiR, where Q R This is the antonym of Q;

[0044] Set the reverse reasoning scoring function Scorei R:O i +Q R →C is:

[0045] (4);

[0046] To design a hybrid scoring function, calculate the average of the two scoring functions above:

[0047] (5);

[0048] For non-causal reasoning questions, the final predicted answer is... Select as:

[0049] (6);

[0050] Here, arg max represents the parameter that yields the maximum value, which in this case is the option corresponding to the maximum score.

[0051] For causal reasoning questions, the answer prediction can be achieved simply by replacing Scorei O with Scorei X in equation (6).

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] (1) This invention proposes a discourse template to generate discourse based on discourse prefixes.

[0054] (2) This invention classifies common sense questions into daily life questions and scientific common sense questions. Daily life questions are answered with discourse assistance, and scientific common sense questions are answered with statements assistance, which improves the accuracy of answer selection to a certain extent.

[0055] (3) The method proposed in this invention can provide high-tech educational products and services for primary and secondary school students, teachers and parents in my country, generating direct practical value and economic benefits. Attached Figure Description

[0056] Figure 1 It is a flowchart of unsupervised common sense automatic question answering based on different knowledge carriers;

[0057] Figure 2 This is a flowchart of the process of constructing a commonsense question-answering dataset using S1;

[0058] Figure 3 This is a flowchart of the S2 common sense question classification process;

[0059] Figure 4 This is a flowchart of the S3 common knowledge generation process;

[0060] Figure 5 This is a flowchart of the reasoning process for answering question S4. Detailed Implementation

[0061] To further illustrate the technical solution of the present invention, the present invention will be further described below through embodiments.

[0062] like Figure 1 As shown in this embodiment, an unsupervised commonsense automatic question answering method based on different knowledge carriers includes the following steps:

[0063] S1, Construct a commonsense question-and-answer dataset;

[0064] S2, categorize common sense questions;

[0065] S3, Common Sense Knowledge Generation;

[0066] S4, Question-Answer Reasoning: Construct the calculation formula for answer reasoning, take two typical samples from the constructed common sense question-answering dataset, feed them into the question-answering model, and obtain the answer.

[0067] like Figure 2 As shown, the specific steps for constructing the commonsense question-answering dataset in S1 are as follows:

[0068] S101. A large number of general knowledge multiple-choice questions are available online. Using the Scrapy web crawler, general knowledge multiple-choice questions were scraped from relevant websites to construct a general knowledge dataset. The contextual information, question options, and correct answers for multiple-choice general knowledge questions are shown in Table 1.

[0069] Table 1

[0070]

[0071] S102 involves preprocessing the common sense multiple-choice questions crawled in S101, mainly including constructing a dictionary data structure consisting of key-value pairs such as context, question, all candidate options, and correct answer label, as shown in Table 2:

[0072] Table 2

[0073]

[0074] like Figure 3 As shown, the specific steps for classifying common sense questions in S2 are as follows:

[0075] S201, combine the context, question, and options to form a statement STi O: with O i (Short for Option) and its corresponding C (short for Context) and Q (short for Question) are used as input. First, the original question Q is rewritten as a declarative statement, and then... <C,Q,O i >Concatenate them into a declarative statement STi O, as shown in Table 3:

[0076] Table 3

[0077]

[0078] S202, Design prompt examples and prompt templates. In this embodiment, six prompt examples and one prompt template were manually designed. Fill the STi O corresponding to the correct answer into the prompt template to form a complete prompt template. The prompt template is shown in Table 4.

[0079] Table 4

[0080]

[0081] S203. Based on the results of S201 and S202, determine whether STi O is a daily event or scientific commonsense. Input six prompt examples and one prompt template into the pre-trained large language model, and let the large language model determine whether the STi O is a daily event or scientific commonsense. Example 1 is scientific commonsense, and example 2 is a daily event.

[0082] like Figure 4 As shown, the specific steps for S3 to generate common sense knowledge are as follows:

[0083] S301, Keyword Extraction: For the declarative statements STi O mentioned in S201, this stage uses an unsupervised key phrase extractor to extract key phrases. This stage extracts three types of key phrases: noun phrases (NP), verb phrases (VP), and personal name phrases (PNP), and designs context-free grammar rules for each type of key phrase:

[0084] NP→(nn|adj) ∗+nn;

[0085] VP→vb+(pr){0,1}+NP;

[0086] PNP→(pn){1,2};

[0087] Where NP is a noun phrase, VP is a verb phrase, PNP is a personal name phrase and is a non-terminator; vb is a verb, nn is a noun, adj is an adjective, pn is a personal name, pr is a preposition and is a determiner; + indicates concatenation; {a,b} indicates the repetition frequency range from a to b; ∗ is equivalent to {0,∞};

[0088] Adding context-free grammar rules to NLTK's RegexpPaser tool extracts the top 5 most important key phrases. For example, Example 1 extracts 2 noun phrases: grape juice, juice, and 3 verb phrases: fermented, turned to wine, evaporated; Example 2 extracts 2 noun phrases: man, groceries, and 3 verb phrases: got a discount, greeted the cashier, used a coupon. No labels or model fine-tuning are required, but the word embeddings are obtained from a pre-trained language model (PLM), such as ELMo.

[0089] S302, Discourse or Sentence Generation: After obtaining key phrases, retrieve discourse or sentence template sets according to common sense question types, and generate discourse prefixes or sentence prefixes;

[0090] If the common-sense question pertains to everyday events, when generating discourse prefixes, consider the following:

[0091] (1) For an object (NP), think about "a past experience with the object";

[0092] (2) For a behavior (VP), consider “the specific experience after the behavior occurred”, “the specific experience before the behavior occurred”, or “the specific experience related to the behavior”;

[0093] (3) For a person (PNP), think about “the impression of him” or “the specific experiences related to his motivation / feeling / reaction to do something”;

[0094] The discourse template set proposed in this embodiment is shown in Table 5:

[0095] Table 5

[0096]

[0097] If the common sense question is scientific common sense, generate a statement prefix, considering the following cases:

[0098] (1) For an object, think about "what it is" and "what its purpose is";

[0099] (2) For a behavior, think about "what it means" and "what happened before and after it occurred";

[0100] (3) For a person, think about "who he / she is" and "what his / her feelings / motivations / reactions are";

[0101] The set of statement templates proposed in this embodiment is shown in Table 6:

[0102] Table 6

[0103]

[0104] A lookup table is constructed using key phrase types as keys and a list of discourse or sentence templates as values. This table allows for the rapid retrieval of templates corresponding to key phrases, and the replacement of labels ([NP], [VP], and [PNP]) with the key phrases to form discourse or sentence prefixes. Despite its simple structure, the lookup table demonstrates effectiveness and versatility across various benchmark tests.

[0105] For each discourse or statement prefix, it is concatenated with the context and input into a generative pre-trained language model (PLM) to generate a complete discourse or statement. Specifically, a kernel sampling of p=0.8 is used as the decoding strategy to increase the diversity of the generated text. Simultaneously, to ensure the quality and quantity of the generated content, all generated discourses or statements are sorted according to the perplexity estimated by the PLM, and the top K discourses or statements are retained to construct a discourse or statement set: {discourse k}k K=1 / {sentence k}k K=1.

[0106] This embodiment contains the first 7 discourses and the first 32 sentences. Due to space limitations, only 1 discourse and 4 sentences are shown. The generated discourses or sentences are shown in Table 7:

[0107] Table 7

[0108]

[0109] like Figure 5 As shown, the specific steps for reasoning the answer to question S4 are as follows:

[0110] S401, Constructing a score calculation formula for a discourse or statement: Each discourse or statement will be inserted with STi O as additional knowledge to assist in scoring subsequent options.

[0111] O i Regarding Discourse k or sentence k The formula for calculating a fraction is:

[0112] (1);

[0113] (2);

[0114] (3);

[0115] Where "+" indicates connection and "-" indicates deletion, P LM Indicates the likelihood of a sentence.

[0116] Formulas (1) and (2) represent the combination of discourse k / sentence k And STiO, calculate the removal of its own O i Even after that, it can still generate O. i The probability of O. Formula (3) represents the probability of a discourse or statement on option O. i The overall score.

[0117] S402, constructing the calculation formula for answer reasoning. For causal reasoning questions, this step introduces reverse thinking (RT) to perform bidirectional inference between causes and effects. In addition to the ordered rewriting (STi O) in S201, reverse rewriting is also applied. <O i Q R C> are connected in order, denoted as STiR, where Q R This is the opposite form of Q. Specifically, in causal reasoning tasks, "Because" and "Therefore" are two opposite questions. For bidirectional reasoning, besides the ordered reasoning C+Q→O... i In addition to the Scorei O, a reverse reasoning scoring function Scorei R:O is also set.i +Q R →C is:

[0118] (4);

[0119] To leverage the advantages of bidirectional inference, a hybrid scoring function is designed by averaging the two scoring functions above:

[0120] (5);

[0121] For non-causal reasoning questions, the final predicted answer is... Select as:

[0122] (6);

[0123] Here, arg max represents the parameter that yields the maximum value, which in this case is the option corresponding to the maximum score.

[0124] For causal reasoning questions, the answer prediction can be achieved simply by replacing Scorei O with Scorei X in equation (6).

[0125] The forward inference scores, backward inference scores, and bidirectional inference scores in Examples 1 and 2 are shown in Tables 8 and 9, respectively.

[0126] Table 8

[0127] Options Positive inference score Reverse inference of scores Two-way inference score A 0.7897 0.8365 0.8131 B 0.2103 0.1635 0.1869

[0128] Table 9

[0129] Options Positive inference score Reverse inference of scores Two-way inference score A 0.3572 0.4594 0.4083 B 0.6428 0.5406 0.5917

[0130] The option with the highest bidirectional inference score is selected from all options and pushed to the user as the answer. Specifically, the answer is selected based on the bidirectional inference score. This involves processing multiple question-and-answer inputs (qa_inputs) and calculating the loss value for each option. Finally, a softmax function is used to calculate the option score, and the option with the higher bidirectional inference score is selected as the final answer. For example, in Example 1, option A is selected as the final answer, and in Example 2, option B is selected as the final answer.

[0131] The foregoing has shown and described the main features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0132] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An unsupervised common-sense automatic question-answering method based on different knowledge carriers, characterized in that, Includes the following steps: S1, Construct a commonsense question-and-answer dataset; S2, categorize common sense questions; S3, Common Sense Knowledge Generation; S4, Reasoning for the answer to the question.

2. The unsupervised common-sense automatic question answering method based on different knowledge carriers according to claim 1, characterized in that, Specifically, S1 is: S101 uses web scraping technology to crawl common sense multiple-choice questions from relevant websites on the Internet; S102 preprocesses the common sense question-and-answer multiple-choice questions crawled in S101, i.e., constructs a dictionary consisting of context, question, all candidate options, and correct answer label.

3. The unsupervised common-sense automatic question answering method based on different knowledge carriers according to claim 2, characterized in that, Specifically, S2 is: S201, combine the context, question, and options to form a statement STi O: with O i Taking its corresponding C and Q as input, the original interrogative sentence Q is rewritten as a declarative statement, and then... <C,Q,O i >Concatenate into a declarative statement STi O; S202, Design a hint example and hint template, fill in the STi O corresponding to the correct answer into the hint template to form a complete hint template; S203, determine whether STi O is a daily event or common scientific knowledge. Input the prompt example and prompt template into the pre-trained large language model, and let the large language model determine whether the STi O is a daily event or common scientific knowledge.

4. The unsupervised common-sense automatic question answering method based on different knowledge carriers according to claim 3, characterized in that, Specifically, S3 is: S301, Keyword Extraction: For the declarative statement STi O mentioned in S201, an unsupervised key phrase extractor is used to extract key phrases. Three types of key phrases are extracted: noun phrases, verb phrases, and personal name phrases. Context-free grammar rules are designed for each type of key phrase. NP→(nn|adj) ∗+nn; VP→vb+(pr){0,1}+NP; PNP→(pn){1,2}; Where NP is a noun phrase, VP is a verb phrase, PNP is a personal name phrase and is a non-terminator; vb is a verb, nn is a noun, adj is an adjective, pn is a personal name, pr is a preposition and is a determiner; + indicates concatenation; {a,b} indicates the repetition frequency range from a to b; ∗ is equivalent to {0,∞}; Adding context-free grammar rules to NLTK's RegexpPaser tool extracts the top 5 most important key phrases without requiring labels or model fine-tuning, but the word embeddings are obtained from a pre-trained language model; S302, Discourse or Sentence Generation: After obtaining key phrases, retrieve discourse or sentence template sets according to common sense question types, and generate discourse prefixes or sentence prefixes; If the common-sense question pertains to everyday events, when generating discourse prefixes, consider the following: (1) For a given object, consider "a past experience with the object"; (2) For a behavior, consider "the specific experiences after the behavior occurred", "the specific experiences before the behavior occurred" or "the specific experiences related to the behavior"; (3) For a person, think about "the impression of him" or "the specific experiences related to his motivation / feeling / reaction to do something"; If the common sense question is scientific common sense, generate a statement prefix, considering the following cases: (1) For an object, think about "what it is" and "what its purpose is"; (2) For a behavior, think about "what it means" and "what happened before and after it occurred"; (3) For a person, think about "who he / she is" and "what his / her feelings / motivations / reactions are"; A lookup table is constructed using key phrase types as keys and a list of discourse or sentence templates as values. The lookup table is used to quickly retrieve the templates corresponding to the key phrases and replace the tags with the key phrases to form discourse or sentence prefixes. For each discourse or statement prefix, it is concatenated with the context and input into a generative pre-trained language model to generate a complete discourse or statement. A kernel sampling of p=0.8 is used as the decoding strategy to increase the diversity of the generated text. Simultaneously, to ensure the quality and quantity of the generated content, all generated discourses or statements are sorted according to the perplexity estimated by the pre-trained language model, and the top K discourses or statements are retained to construct a discourse or statement set: {discourse k }k K=1 / {sentence k }k K=1.

5. The unsupervised common-sense automatic question answering method based on different knowledge carriers according to claim 4, characterized in that, Specifically, S4 is: S401, constructs a score calculation formula for a discourse or statement, and inserts STiO as additional knowledge into each discourse or statement to assist in scoring subsequent options; O i Regarding Discourse k or sentence k The formula for calculating a fraction is: （1）； （2）； （3）； Where "+" represents connection, "-" represents deletion, and P LM Indicates the likelihood of a sentence; S402, constructing a calculation formula for answer reasoning. For causal reasoning questions, introduce reverse thinking to perform bidirectional inference between cause and effect. <O i Q R C> are connected in order, denoted as STiR, where Q R This is the antonym of Q; Set the reverse reasoning scoring function Scorei R:O i +Q R →C is: （4）； To design a hybrid scoring function, calculate the average of the two scoring functions above: （5）； For non-causal reasoning questions, the final predicted answer is... Select as: （6）； Where arg max represents the parameter that yields the maximum value, which in this case is the option corresponding to the maximum score; For causal reasoning questions, the answer prediction can be achieved simply by replacing Scori O with Scori X in equation (6).

Images