Question answering method for complex knowledge base based on prompt learning

Abstract

The present invention discloses a complex knowledge base question-answering method based on prompt learning, comprising: constructing a prompt model for the question-answering task, obtaining input from a pre-trained language model based on the prompt model; using a semantic selector to dynamically select contextual information in the knowledge base that matches the question based on the input from the pre-trained language model to obtain answer features; and classifying the answer features to obtain a final predicted answer. By adding a semantic selector to the prompt template, the present invention effectively addresses the interference of redundant and ambiguous knowledge and improves the accuracy of the PLM's semantic understanding. By optimizing the prompts through the introduction of preset semantic tags, the present invention effectively addresses the time-consuming and labor-intensive nature of the prompt template and its lack of robustness, thereby improving the performance of knowledge base question-answering.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge base question answering, and in particular relates to a complex knowledge base question answering method based on prompt learning. Background Art

[0002] In recent years, pre-trained language models (PLMs) have been widely used in knowledge graph question-answering tasks. They unify the input of natural language processing tasks and improve model reasoning performance. Pre-trained models can better initialize the model and enhance its generalization capabilities. Depending on the method used, PLMs can be divided into the representation learning phase and the prompt learning phase.

[0003] In the representation learning phase of PLM, in order to enable PLM to better understand semantic information, ERNIE embeds the subject entities in the question that are linked to the knowledge base into the pre-trained model to enhance the reasoning ability of the pre-trained model, but it does not take into account the connection between multiple subject entities in complex knowledge base question and answer. Wei et al. and Machel et al. improved the ability of pre-trained language models to perform complex reasoning by designing thought chain intermediates. KnowPrompt is used to enhance the semantic understanding ability of the model by injecting all the knowledge in the knowledge base into the prompt template. Zhang et al. solved the problem of missing external knowledge in the pre-trained model by proposing automatic tuning of ontology enhancement. However, not all knowledge in the knowledge base is relevant to the question context, and knowledge interfered with by redundant and ambiguous information may interfere with semantic understanding, thereby reducing the reasoning ability of the model.

[0004] In the PLM prompt learning stage, existing methods based on manually designed prompt templates are time-consuming, labor-intensive, and lack robustness. Recent studies have shown that the manual prompt format is not optimal. Previous methods have attempted to automatically search for discrete tags, but prompt templates composed of such discrete tags lack robustness. Zhang et al. optimized the prompt templates through backpropagation and achieved certain results in few-shot learning. TemplateNER introduced prompt learning in the sequence labeling task, but this approach ignored the interdependence between different topic entities in a sequence. Li et al. designed prefix tuning to replace prompt learning, thereby improving the performance of the model. AdaPrompt adds prompt tags on both sides of the entity and uses discrete templates to convert the question-answering task into a masked language model task. However, such templates require human experience to design, have no fixed form, and lack robustness.

[0005] In summary, previous work has introduced pre-trained language models into knowledge-based question answering, and significantly improved the performance of a large number of tasks with a unified model architecture. However, existing work still has the following two problems: (1) During the PLM representation learning phase, not all knowledge in the knowledge base is relevant to the question context, and there is a large amount of redundant and ambiguous knowledge that interferes with semantic understanding; (2) During the prompt learning phase of the pre-trained model, existing methods based on manually designed prompt templates are time-consuming, labor-intensive, and lack robustness. Summary of the Invention

[0006] The purpose of the present invention is to propose a complex knowledge base question answering method based on prompt learning. By adding special semantic tags as prompt templates, backpropagation is used to optimize the continuous vectors in the word embedding space, and the best prompt learning is obtained by searching in such a (continuous) embedding space. By designing a semantic selector, contextual information that matches the question in the knowledge base is dynamically selected, and irrelevant knowledge base information is filtered out. By learning the importance of relationships, the representation of each entity in the question context is enhanced, thereby improving the reasoning ability of the model.

[0007] To achieve the above objectives, the present invention provides a complex knowledge base question answering method based on prompt learning, comprising:

[0008] Building a prompt model for the question-answering task, and obtaining input for the pre-trained language model based on the prompt model;

[0009] Based on the input of the pre-trained language model, a semantic selector is used to dynamically select contextual information matching the question in the knowledge base to obtain answer features;

[0010] The answer features are classified to obtain the final predicted answer.

[0011] Optionally, the method for constructing a prompt model for a question-answering task is: constructing a prompt model by adding a preset semantic tag as a prompt in a question; wherein the preset semantic tag is a word that has not been used in the vocabulary of the pre-trained language model.

[0012] Optionally, obtaining an input of a pre-trained language model based on the prompt model includes:

[0013] Optimizing the prompts through back propagation to obtain the best prompts;

[0014] The best prompt, question, and reasoning paths in the subgraph are concatenated with SEP characters as input to the pre-trained language model.

[0015] Optionally, the method for obtaining the reasoning path in the subgraph includes:

[0016] Using named entity tools to identify the subject entities in the question;

[0017] A subgraph specific to the problem is generated according to the subject entity, wherein the plurality of fact triples in the subgraph are reasoning paths in the subgraph.

[0018] Optionally, the prompt model is:

[0019] X template =[X][θ]1[θ]2...[θ] i [MASK]

[0020] Where [X] represents the head entity e0, θ i Indicates the relationship prompt, X template Represents the prompt model, and [MASK] represents the mask form of cloze test.

[0021] Optionally, the method for optimizing the prompt by backpropagation includes:

[0022]

[0023] Among them, ξ i Represents a fact triple in the knowledge base, e0 represents the head entity of a fact triple, e t represents the target entity, sim(q, r) represents the cosine similarity between the question and the current fact triple, q represents the natural language question, represents the optimization function, and P represents the probability function.

[0024] Optionally, the input of the pre-trained language model is:

[0025] X input =[cls]X template .[SEP] q .[SEP]ξ i [SEP]

[0026] Among them, X input represents the input of the pre-trained language model, and q represents the question.

[0027] Optionally, the semantic selector is:

[0028]

[0029] in, Represents a semantic selector, l represents the length of the question, h i represents the embedding representation of the i-th word in the question, k represents the K-th step, i represents the number of times, Indicates the information of the i-th selection.

[0030] Optionally, using a semantic selector to dynamically select contextual information matching the question in the knowledge base includes:

[0031] Obtain the semantic matching vector between the original knowledge base and the semantic selector through the sigmoid function;

[0032] Update the embedding representations of the head entity and the tail entity in the knowledge base according to the semantic matching vector,

[0033] The knowledge context representation is updated using the new embedding representation.

[0034] Optionally, the answer features are:

[0035]

[0036] Among them, f Answer Indicates the answer characteristics, Indicates the addition of the eigenvalues ​​at the corresponding positions, h cls and h Mask Represent the eigenvalues ​​of [cls] and [Mask] respectively;

[0037] The final predicted answer is:

[0038] p=softmar(f Answer )

[0039] Among them, p represents the final predicted answer and softmax represents the classifier.

[0040] The present invention has the following beneficial effects:

[0041] This method constructs a prompt model for question-answering tasks by adding preset semantic tags as prompts to questions. Based on this prompt model, it obtains input from a pretrained language model. Based on this input from the pretrained language model, it uses a semantic selector to dynamically select contextual information in the knowledge base that matches the question to obtain answer features. By adding a semantic selector to the prompt template, it effectively addresses the interference of redundant and ambiguous knowledge and improves the accuracy of the PLM's semantic understanding. By optimizing prompts with preset semantic tags, it effectively addresses the time-consuming and labor-intensive nature of prompt templates and their lack of robustness, thereby improving the performance of knowledge-based question-answering. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0043] Figure 1 Schematic diagram of the overall framework of the KBPL model according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the F1 values ​​of the datasets CWQ and WebQSP at different dims according to an embodiment of the present invention;

[0045] Figure 3 Schematic diagram of the F1 value of the datasets CWQ and WebQSP under different α in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] This embodiment proposes a complex knowledge base question answering method based on prompt learning. In this embodiment, the question is defined as:

[0049] Definition 1: Knowledge base (KB): represents the <e0,r,e t >The fact triples composed of e0 and e t They represent the subject entity and the target entity respectively, r is used to describe the relationship between the two entities, and R represents the relationship set.

[0050] Definition 2: Knowledge-Based Question Answering (KBQA): Given a question and a knowledge base (KB), KBQA aims to link the subject entity in the natural language question with the target entity in the KB and then predict the answer entity using an optimal reasoning process. In this example, we focus on solving complex natural language questions that may contain multiple subject entities.

[0051] The specific steps of the complex knowledge base question answering method based on prompt learning proposed in this embodiment are as follows:

[0052] Phase 1: Build a prompt model for the question-answering task and obtain input for the pre-trained language model based on the prompt model;

[0053] Specifically, in this embodiment, a prompt template is created for the question-answering task, by adding special semantic tags and expressing them as cloze prompts to retrieve answers from the pre-trained model and generate the correct reasoning path.

[0054] Furthermore, a prompt model is constructed by adding preset semantic tags as prompts in the question; wherein the preset semantic tags are words that have not been used in the vocabulary of the pre-trained language model.

[0055] Furthermore, obtaining the input of the pre-trained language model based on the prompt model includes:

[0056] Optimize the prompts through backpropagation to obtain the best prompts;

[0057] The best prompt, question, and reasoning paths in the subgraph are concatenated with SEP characters and used as input to the pre-trained language model.

[0058] In this embodiment, during the PLM prompt learning phase, the prompt templates designed in previous work are time-consuming and labor-intensive and lack robustness. To address the above issues, this embodiment introduces special semantic tags to construct prompt templates (by using words that are not used in the PLM vocabulary v) and optimizes them using backpropagation. Figure 1 As shown in the first stage, this embodiment adds semantic tags θ to the question i As a prompt, a prompt model was developed.

[0059] In order to solve the problem of time-consuming and labor-intensive manual prompt templates, this embodiment uses semantic tags (vocabulary not used in the PLM vocabulary v) to automatically construct prompt templates, and then optimizes them through back propagation to find the best prompts. Specifically, traditional discrete prompt templates are composed of templates written by experts based on triples (subject entity, relationship, target entity) and the relationships they express. They are mainly manually designed based on expert knowledge. For example, a fact is defined as a triple (e0, r, e t ), where e0 is the head entity of a relation triple (e.g., James), r is a relation from a fixed set of relations R (e.g., teammates), and e t is the target entity (e.g., Wade). Previous work designs prompts based on the set of relations embedded in the pre-trained model. Each relation is associated with a human-written prompt containing a single [MASK] token. Previous work has shown that pre-trained models (e.g., BERT) can predict the target entity e under cloze-style prompts. t For example: "[X] teammate is [MASK]". In Table 1, this embodiment shows through examples the difference between traditional discrete template design and the template designed in this embodiment.

[0060] Table 1 Example of generating prompt template for triple (James, hometown,?)

[0061]

[0062] However, previous work manually designed prompts based on facts stored in the pre-trained model, which can lead to the failure to elicit the best prompts, and is time-consuming and labor-intensive and lacks robustness. Different from traditional discrete prompt templates, this embodiment uses several unused semantic tags θ i ∈v is used as a relation prompt. By searching in the space of semantically labeled entities, the best prompt is automatically obtained, which solves the problem of time-consuming and labor-intensive manual prompt design. The robustness of the prompt is enhanced through backpropagation. Given the subject entity e0 and the relation prompt θ i and pre-trained language model (BERT), the target entity e can be obtained through PLM reasoning t .X template Indicates that it contains the head entity e0 and the relationship prompt θ i Tip Template X template Can be defined as:

[0063] X template =[X][0]1[θ]2..[0] i [MASK] (1)

[0064] Among them, [X] represents the head entity e0, and the relationship prompt θ i is a learnable hint parameter, and v is a set of words selected from the vocabulary of PLM. This embodiment uses the relationship between fact triples to initialize the semantic tag θ i For example, for the fact triple (James, teammate, Wade), this embodiment initializes the prompt θ through the relationship teammate i In order to search for the best hint, this embodiment optimizes the hint through back propagation, which is defined as follows:

[0065]

[0066] Among them, ξ i Represents the fact triple in the knowledge base, e0 is the head entity of a fact triple, e t is the target entity, and sim(q,r) represents the cosine similarity between the question and the current fact triple.

[0067] Given a complex question, we first use existing named entity tools to identify the subject entity e0 in the question and generate a question-specific subgraph based on the subject entity e0. We then connect the question, prompt template, and the reasoning path in the subgraph (consisting of multiple fact triples) with the [SEP] character and use it as the input to the pre-trained language model. The input of the pre-trained model is defined as:

[0068] X input =[cls]X template.[SEP]q.[SEP]ξ i [SEP] (3)

[0069] An interesting observation from the prompts generated by the KBFPL model is that the most prominent generated sentences are "The object is [MASK]" and "The relationship is [MASK]." For example, for the question "Which team have Yao Ming and McGrady played for?", the best prompts automatically generated based on the knowledge base embedded in the pre-trained model are "The relationship between Yao Ming and McGrady is [MASK]" and "The team is [MASK]."

[0070] Phase 2: Based on the input of the pre-trained language model, a semantic selector is used to dynamically select contextual information that matches the question in the knowledge base to obtain answer features.

[0071] Furthermore, the semantic selector is used to dynamically select contextual information that matches the question in the knowledge base, including:

[0072] Obtain the semantic matching vector between the original triple information and the semantic selector through the sigmoid function;

[0073] Update the embedding representations of the head entity and tail entity in the fact triple according to the semantic matching vector,

[0074] The new embedding representation is utilized to update the knowledge context representation.

[0075] In this example, the ideal process for learning the PLM representation is to first identify the subject entities mentioned in the question, then dynamically select a subgraph ("knowledge context") centered around these mentioned entities from the knowledge group (KG), and finally embed the selected knowledge context into the PLM. However, in real life, not all knowledge in the knowledge base is relevant to the question context. Knowledge that is redundant or ambiguous can interfere with semantic understanding, thereby reducing the model's reasoning ability.

[0076] To address the above challenges, this embodiment proposes dynamically embedding appropriate knowledge base information based on semantic selectors to improve the reasoning ability of PLM.

[0077] First, identify the natural language question q = {x1, x2, ..., x l} in the subject entity (e i ∈E), this embodiment defines the original knowledge context (Raw Konwledge Context) as the subject entity ei The subgraph centered on the subject entity e i The reasoning path is at most k hops away. The knowledge context contains the triple ξ i and subject entity description (d i ∈d).

[0078] This embodiment uses question text to construct a semantic selector in Represents the kth semantic selector used to guide reasoning on the knowledge graph. When learning semantic selectors at different time steps, it focuses on specific parts of the problem. In this process, this embodiment also dynamically updates the problem representation so that it can include information from previous semantic selectors. Repeating the above steps, this embodiment can finally obtain the semantic selector Through such a process, the matching degree of the relation path (from the subject entity to the tail entity) and the question can be encoded into the node embedding and used to update the embedding representation of the fact triple, thereby dynamically embedding appropriate knowledge base information and avoiding knowledge interference from redundant and ambiguous information.

[0079] Specifically, this embodiment uses the attention mechanism to focus on different parts of the question, and then uses the semantic selector to select specific relations based on the different parts of the question in each reasoning step, thereby filtering out irrelevant triples and updating the representation of entities embedded in the pre-trained model. The definition is as follows:

[0080]

[0081]

[0082] q k =α[q;M i (k-1) ]+η (6)

[0083] Among them, q represents the natural language question, l represents the length of the question, and h i represents the embedding representation of the i-th word in the question, f α , α and η are learnable parameters, q k Indicates dynamic matching problem context information, Represents the reasoning information obtained by the semantic selector in the previous step.

[0084] Then, this embodiment obtains the semantic matching vector between the original triple information and the semantic selector through the sigmoid function, which is defined as follows:

[0085]

[0086] in, Represents the semantic selector, and then updates the embedding representation of the head entity and tail entity in the fact triple according to the semantic matching vector, which is defined as follows:

[0087] e t ′=N K e t (8)

[0088] e0′=N K e0 (9)

[0089] Finally, the new entity embedding representation is used to update the knowledge context representation, which is defined as follows:

[0090]

[0091] d′ i ={e′0, description} (11)

[0092] This embodiment uses a semantic selector to filter out irrelevant knowledge base information and aggregate important information, thereby limiting its impact on the prompt template. The semantic selector is used to adjust the weight of question feature aggregation, ultimately selecting knowledge base information relevant to the question context and embedding it into the PLM.

[0093] The third stage: classify the answer features and obtain the final predicted answer.

[0094] This embodiment extracts [cls] and [Mask] from the sentence as the features of the answer. This embodiment obtains the features of the answer by summing different feature positions. The definitions are as follows:

[0095]

[0096] Among them, f Answer Indicates the answer characteristics, Indicates the addition of the eigenvalues ​​at the corresponding positions, h cls and h Mask Represent the eigenvalues ​​of [cls] and [Mask] respectively.

[0097] Use the softmax classifier to classify f Answer Perform classification and finally obtain the predicted answer, which is defined as follows:

[0098] p=softmar(f Answer ) (13)

[0099] Compared to the pre-training and fine-tuning paradigm of training additional task-specific neural models, the KBPL model of this embodiment only needs to adjust the parameters of the PLM. To ensure the best performance of the model, this embodiment uses the cross-entropy loss function to train the model of this embodiment, using the predicted and true answers to calculate the loss. The loss function is defined as follows:

[0100]

[0101] Among them, y i and p i are the probabilities of the answer label and the predicted label of the i-th training instance, respectively. In order to quickly learn high-quality knowledge base information embedding during the training process, this embodiment adopts the Adam optimizer training model. The optimizer can converge quickly, requires less memory, and is easy to adjust parameters.

[0102] This embodiment also uses two English data sets to verify the beneficial effects of this embodiment, as follows:

[0103] To fully evaluate the model's effectiveness, this example was run on two English datasets: ComplexWebQuestion (CWQ) and WebQuestionSP (WebQSP), as shown in Table 2. Both CWQ and WebQSP, English knowledge-based question-answering datasets, use Freebase as their knowledge base. The WebQSP dataset contains 4,373 natural language questions, all of which require relational reasoning within two hops. The CWQ dataset contains a higher proportion of complex questions, divided into four types, requiring up to four hops of relational reasoning.

[0104] Table 2 Experimental dataset statistics

[0105]

[0106]

[0107] To conduct comparative experiments, this example divides each dataset into three parts: a training set, a validation set, and a test set. Detailed statistics of the experimental datasets are shown in Table 2. As can be seen from the table, all experimental datasets contain a high proportion of complex problems, thus effectively reflecting the model's ability to solve complex problems.

[0108] Evaluation indicators

[0109] This example uses the average (balanced) precision (F1 (average precision and recall) and Hits@1) metrics to measure the effect. To ensure fairness in experimental comparison, this example uses the same evaluation metric, Hits@1, as the baseline model. On the CWQ and WebQSP datasets, this example uses both Hits@1 and F1 to evaluate the effect.

[0110] Parameter settings

[0111] This example uses an NVIDIA GTX A5000 GPU to implement the model. The Adam optimizer is used to learn parameters in the experiments. For both datasets, a grid search strategy is used to adjust the parameters on the validation set. The embedding dimensions dim = {100, 150, 200, 250} are adjusted, with the optimal dim settings being 150 and 200 on the CWQ and WebQSP datasets, respectively. The training batch size is batch = {256, 512, 1024}, with the optimal batch setting being 1,024 on the CWQ and WebQSP datasets. The attention weight α is searched in the range [0, 1], with the optimal α settings being 0.6 and 0.8 on the CWQ and WebQSP datasets, respectively.

[0112] Baseline system

[0113] To verify the effectiveness of the method proposed in this example, we selected the following 14 representative recent works as baseline systems for comparison. For the English datasets CWQ and WebQSP, the KBPL model proposed in this example was compared with two baselines: a traditional complex knowledge-based question answering method and a complex knowledge-based question answering method based on a pretrained language model.

[0114] Among them, the traditional complex knowledge base question answering methods mainly include TextRay, ReifKB, UHop, NSM, SR+NSM, TUL, ARL, ReaRev and PullNet.

[0115] Complex knowledge base question answering based on pre-trained models applies pre-trained models to complex knowledge base question answering, enhancing the powerful representation ability learned from text corpus and improving the semantic parsing and retrieval matching performance of traditional methods. Its representative models include: UniKGQA, KD-CoT, KB-BINDER-Codex and BeamQA.

[0116] Experimental results analysis

[0117] Table 3 shows the experimental results on the CWQ and WebQSP datasets. The baselines in the table are divided into two categories: the first category is traditional complex knowledge-based question answering methods; the second category is methods based on pre-trained language models. The results of these baseline methods are all based on the results reported in the original papers. "-" indicates that the original papers did not report the relevant results. Analysis of the results in Table 3 reveals the following two important findings:

[0118] Table 3 Experimental results on datasets WebQSP and CWQ

[0119]

[0120] Compared to pre-trained knowledge-based question answering methods, the KBPL approach proposed in this example achieves competitive results on both the CWQ and WebQSP datasets. Compared to the current state-of-the-art model, UniKGQA, the KBPL model achieves improvements of 9.9% and 7.7% on the CWQ and WebQSP datasets, respectively, compared to the Hits@1 evaluation metric. KBPL also achieves comparable performance to UniKGQA on the F1 evaluation metric. These experimental results demonstrate that this example, by constructing hint learning and introducing semantic selectors, can further enhance the reasoning capabilities of the PLM.

[0121] Compared with traditional complex knowledge base question answering methods (such as TUL and ARL), pre-training-based methods have achieved better results overall, such as UniKGQA. This is mainly due to the fact that the pre-training model can learn more powerful representation learning capabilities from a large-scale corpus, and can encode the input question and the information in the knowledge base into a unified vector space, thereby improving the performance of traditional methods in semantic parsing and retrieval matching. At the same time, this embodiment also found that the KBPL proposed in this embodiment can significantly improve the performance of traditional methods. For the evaluation index F1, compared with the ARL model, the KBPL model improved by 5.8% and 7.0% on the CWQ and WebQSP datasets, respectively. The above results show that by introducing pre-training models and semantic selectors and constructing prompt learning, the ability of the question answering system to handle complex problems can be effectively improved. The F1 values ​​of the datasets CWQ and WebQSP at different dims are as follows: Figure 2 As shown, the F1 values ​​of the datasets CWQ and WebQSP under different α are as follows Figure 3 shown.

[0122] Ablation experiments

[0123] To study the impact of various parts of the model on the experiment, this example conducted an ablation experiment. As shown in Table 4, to further clarify the impact of the mechanism proposed in this example on the model, this article conducted an ablation experiment to explore the impact of prompt learning and semantic selectors on the results. In the same experimental environment, F1 and Hits@1 were selected as evaluation indicators, and ablation experiments were conducted on the CWQ and WebQSP datasets. As shown in Table 4, the second row shows the experimental results using only the conventional pre-trained model (KBPL w / o prompt). For the indicator Hits@1, the method proposed in this article has an improvement of 4.50% and 2.70% on the CWQ and WebQSP datasets, respectively. As for the impact of knowledge base information embedding, as shown in the third row of Table 4, the experiment was conducted without introducing the semantic selector (KBPL w / o Selector). For the indicator Hits@1, the method proposed in this example has an improvement of 2.86% and 2.85% on the CWQ and WebQSP datasets, respectively. As can be seen from Table 4, adding prompt learning to the pre-trained model has significantly improved the model performance, and dynamically embedding knowledge base information that matches the question context through the semantic selector has improved the performance of the question-answering model.

[0124] Table 4 Experimental results of ablation study on WebQSP and CWQ datasets

[0125]

[0126]

[0127] The impact of hyperparameters on experiments

[0128] In order to obtain the optimal values ​​of the model parameters, this example sets the number of iterations to 3,000 and determines the optimal values ​​of the parameters based on the performance on the validation set. The following two parameters affect the performance of the model: 1) attention weight α; 2) embedding dimension dim;. On the validation set, the attention weight and embedding dimension are searched in the ranges of [0, 1] and {100, 150, 200, 250, 300} respectively to set the optimal values ​​to achieve the best model performance. Figure 2 and Figure 3 As shown in Figure 3, the model achieves the best results for the datasets CWQ and WebQSP when dim=150, α=0.8 and dim=200, α=0.8 respectively.

[0129] This embodiment proposes a complex knowledge base question answering model based on prompt learning. By designing a semantic selector, this model effectively addresses the interference of redundant and ambiguous knowledge, improving the accuracy of PLM's semantic understanding. By introducing semantic tags into the prompt template, it effectively addresses the time-consuming and labor-intensive nature of the prompt template and its lack of robustness, thereby improving the model's question answering performance. This embodiment conducted a large number of experiments on two public datasets. The experimental results show that the method proposed in this embodiment outperforms previous question answering models. Specifically, compared with the existing optimal model UniKGQA, the model of this embodiment improves the evaluation metric Hits@1 by 7.7% and 9.9% on the WebQSP and CWQ datasets, respectively.

[0130] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A complex knowledge base question answering method based on prompt learning, characterized by: include: Building a prompt model for the question-answering task, and obtaining input for the pre-trained language model based on the prompt model; Based on the input of the pre-trained language model, a semantic selector is used to dynamically select contextual information matching the question in the knowledge base to obtain answer features; Classify the answer features to obtain the final predicted answer; The method for constructing a prompt model for a question-answering task is as follows: constructing the prompt model by adding a preset semantic tag as a prompt in a question sentence; wherein the preset semantic tag is a word that has not been used in the vocabulary of the pre-trained language model; Obtaining input to a pre-trained language model based on the prompt model includes: Optimizing the prompts through back propagation to obtain the best prompts; Concatenate the best prompt, question, and reasoning path in the subgraph with the SEP character as input to the pre-trained language model; The method for optimizing the prompt by back propagation includes: Among them, ξ i Represents a fact triple in the knowledge base, e0 represents the head entity of a fact triple, e t represents the target entity, sim(q,r) represents the cosine similarity between the question and the current fact triple, q represents the natural language question, represents the optimization function, P represents the probability function, X template Indicates the prompt model, [MASK] indicates the mask form of cloze test; The input of the pre-trained language model is: X input =[cls]X template .[SEP]q.[SEP]ξ i [SEP] Among them, X input represents the input of the pre-trained language model, and q represents the question.

2. The complex knowledge base question answering method based on prompt learning according to claim 1 is characterized in that The method for obtaining the reasoning path in the subgraph includes: Using named entity tools to identify the subject entities in the question; A subgraph specific to the problem is generated according to the subject entity, wherein the plurality of fact triples in the subgraph are reasoning paths in the subgraph.

3. The complex knowledge base question answering method based on prompt learning according to claim 1, characterized in that The prompt model is: X template =[X][θ]1[θ]2...[θ] i [MASK] Where [X] represents the head entity e0, θ i Indicates the relationship prompt, X template Represents the prompt model, and [MASK] represents the mask form of cloze test.

4. The complex knowledge base question answering method based on prompt learning according to claim 1, characterized in that The semantic selector is: in, Represents a semantic selector, l represents the length of the question, h i represents the embedding representation of the i-th word in the question, k represents the K-th step, S k Represents the information selected in the Kth step.

5. The complex knowledge base question answering method based on prompt learning according to claim 1, characterized in that Using semantic selectors to dynamically select contextual information that matches the question in the knowledge base includes: Obtain the semantic matching vector between the original knowledge base and the semantic selector through the sigmoid function; Update the embedding representations of the head entity and the tail entity in the knowledge base according to the semantic matching vector, The knowledge context representation is updated using the new embedding representation.

6. The complex knowledge base question answering method based on prompt learning according to claim 1, characterized in that The answer characteristics are: Among them, f Answer Indicates the answer characteristics, Indicates the addition of the eigenvalues ​​at the corresponding positions, h cls and h Mask Represent the eigenvalues ​​of [cls] and [Mask] respectively; The final predicted answer is: p=softmax(f Answer ) Among them, p represents the final predicted answer and softmax represents the classifier.

Images