A method for implementing conversational knowledge base question answering

By adopting technologies such as fuzzy reasoning grammar system and multi-head attention mechanism in multi-round dialogue knowledge base Q&A, the shortcomings of dialogue understanding and answer accuracy in the existing technology are solved, and effective treatment of fuzzy reasoning and entity disambiguation is achieved. , improves the accuracy and user experience of dialogue responses.

CN114443818BActive Publication Date: 2024-11-22TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210114303.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-30
Publication Date
2024-11-22
Estimated Expiration
2042-01-30

AI Technical Summary

Technical Problem

The prior art is difficult to deal with fuzzy reasoning and entity disambiguation in multi-round dialogue knowledge base Q&A, resulting in insufficient accuracy of dialogue comprehension and answers, especially in large-scale knowledge bases that entity type and relationship matching information cannot be fully utilized.

Method used

Using a pre-defined fuzzy inference syntax system, combining the GloVe model, multi-head attention mechanism encoder, LSTM named entity recognition model and XLNet entity disambiguation model, logical forms are generated through multi-head attention decoder, and the entity types and entities in the knowledge base are used. Relational information performs entity disambiguation and relationship prediction, and builds executable query statements.

Benefits of technology

It improves the accuracy of intention recognition and entity recognition of user problems in conversation, effectively solves fuzzy reasoning problems, improves the accuracy of dialogue response, and is suitable for open domain knowledge graph questions and answers, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114443818B_ABST
    Figure CN114443818B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for implementing conversational knowledge base question and answer, step 1: pre-define a fuzzy reasoning grammar system; step 2: obtain the current question to be answered and the conversation history information, and obtain the vector representation through the GloVe model; step 3: obtain the hidden state vector of the encoder; step 4: identify the named entity and its type from the feature representation; step 5: obtain the entity ID in the knowledge base; step 6: obtain the logical form of the question under the current grammatical framework; step 7: predict the relationship ID and entity type ID in the question; step 8: obtain the query statement that can be executed on the knowledge base; step 9: execute the query statement in the knowledge base to obtain the answer corresponding to the current question. Compared with the prior art, the present invention automatically identifies the intention of the user's question in the conversation, improves the accuracy of the answer in the conversation, is suitable for open domain knowledge graph question and answer, and can improve user experience satisfaction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, and in particular relates to a method for implementing conversational knowledge base question and answer. Background Art

[0002] In recent years, with the development of knowledge bases such as DBPedia, Wikidata, and YAGO, their scale has become increasingly large. This makes it increasingly difficult for ordinary people without professional backgrounds to obtain the knowledge they need from them. Knowledge Base Question Answering (KBQA) provides an effective and convenient way to use the knowledge in the knowledge base to answer questions raised by users in natural language. Despite a large amount of research, KBQA is still a challenging task, especially in multi-round dialogue KBQA, where existing methods are difficult to understand the questions in the dialogue as accurately as humans.

[0003] With the development of deep learning technology, the mainstream methods of using neural networks for KBQA include semantic analysis-based and information retrieval-based methods. The semantic analysis-based method first designs a semantic parser to map the question to a logical form, such as λ-DCS, SPARQL, or SQL, and then uses an execution module to execute the query in the knowledge base. The information retrieval-based method first identifies the subject entity, then retrieves the subgraph that is one or more hops away from the subject entity from the knowledge graph as the candidate answer, then designs various features to map the candidate answers to a low-dimensional vector space, and then compares and sorts them.

[0004] For conversational question answering for large-scale knowledge bases, the current methods with better results are mostly based on semantic parsing, because predefined logical forms can complete complex reasoning. As long as the golden logical form corresponding to the question can be found, the logical form can be executed to get the correct answer. There are two key subtasks in conversational KBQA based on semantic parsing, namely the design of logical form grammar and the generation of logical form. The purpose of the design of logical form grammar is to design a grammatical system that can meet the requirements of complex reasoning, so as to ensure the effect of the model. The generation of logical form is to understand the semantics of the question and convert it into a logical form under a predefined grammatical framework. At present, most of the sequence-to-sequence methods are used to complete this subtask. The KBQA method based on semantic parsing is very dependent on the predefined logical form grammatical framework and can only complete a limited number of operations defined in the grammatical system. However, most of the current grammatical systems only have deterministic operations, which makes the current methods unable to cope with fuzzy reasoning problems in reality. For example, when the question is "Which nutrients can interact with approximately 89 chemical substances and drugs?", the current grammatical system is difficult to express the fuzzy semantics of "approximately 89".

[0005] In addition, the problem of loose integration between knowledge base and semantic parsing has always existed in existing methods. Most current methods divide semantic parsing into two separate steps: generating logical forms and executing queries in the knowledge base. When generating logical forms, the information provided by the knowledge base is often not used or rarely used. This will make it difficult to use the matching information of entity types and relationships already in the knowledge base when predicting the entity types or relationships that appear in the problem. Due to the large size of the knowledge base, there are many entities with the same entity name and type. It is difficult to disambiguate by relying solely on the ID of the entity in the knowledge base, which leads to errors. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a method for improving the semantic parsing capability of a conversational knowledge base question-answering model, which utilizes a knowledge base to automatically identify user questions in a conversation and can improve the semantic parsing capability of a conversational knowledge base question-answering model.

[0007] The present invention is achieved by adopting the following technical solutions:

[0008] A method for implementing a conversational knowledge base question and answer, the method comprising the following steps:

[0009] Step 1: pre-define the fuzzy reasoning grammar system;

[0010] Step 2: Get the current question to be answered and the conversation history information, and get the vector representation through the GloVe model;

[0011] Step 3: Through the encoder based on the multi-head attention mechanism, the hidden state vector of the encoder is obtained, which is expressed as follows:

[0012] h enc =encoder(x;θ enc ) where encoder() is the encoder function, θ dec are the trainable parameters of the encoder function;

[0013] Step 4: Identify named entities and their types from feature representations through an LSTM-based named entity recognition model;

[0014] Step 5: Link the identified entity to the entity in the knowledge base through the XLNet-based entity disambiguation model to obtain the entity ID in the knowledge base;

[0015] Step 6: According to the hidden state vector of the encoder, the logical form of the question in the current grammatical framework is obtained through the decoder based on the multi-head attention mechanism; it is expressed as follows:

[0016] z dec = decoder(h enc θ dec )

[0017]

[0018] Where decoder( ) is the decoder function, θ dec is the trainable parameter of the encoder function, z dec is the hidden state vector of the decoder function, P(y dec |x) represents y dec Relative to y dec The probability of y dec is the input sequence, y dec is the true label of the decoding task, W dec is a linear layer that transforms the dimension of the decoder hidden state vector;

[0019] Step 7: Predict the relation ID and entity type ID in the question based on the last bit of the encoder's hidden state vector, the decoder's hidden state vector, and the knowledge base knowledge;

[0020] Step 8: Integrate the entity ID, relationship ID, entity type ID and the decoded logical form to obtain a query statement that can be executed on the knowledge base;

[0021] Step 9: Execute the query statement in the knowledge base to obtain the answer corresponding to the current question.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] 1) It can accurately and automatically identify the intention of user questions in the conversation, construct query statements in the knowledge base based on the intention, and improve the accuracy of intention recognition and entity recognition;

[0024] 2) It can effectively solve fuzzy reasoning problems in real life and improve the accuracy of responses in conversations;

[0025] 3) The present invention is applicable to open domain knowledge graph question and answer, and can improve user experience satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an overall flow chart of a conversational knowledge base question-answering method of the present invention;

[0027] Figure 2 is a flowchart of a method for obtaining current questions to be answered and conversation history information;

[0028] Figure 3 is a flowchart of the named entity recognition method;

[0029] Figure 4 is a flow chart of the entity disambiguation method;

[0030] Figure 5 It is a flowchart of the relationship ID and entity type ID in the prediction problem. DETAILED DESCRIPTION

[0031] Before semantic parsing, the various models to be used need to be trained and tested. Including: obtaining training sets and test sets of questions, as well as training sets and test sets of triples (in the knowledge base), and training and testing the models used in entity linking. Specifically, on the one hand, multi-task training and testing are performed on the encoder-decoder model, LSTM (long short-term memory network), GAT (graph attention network) model and parameters in the feedforward neural network used in extracting question type features. On the other hand, the XLNet model and linear layer network used for entity disambiguation are trained and tested, and its input is a text consisting of the current question, ambiguous entity name, and related triple information, and the output is the probability that the text is a positive example and a negative example.

[0032] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0033] like Figure 1As shown, it is an overall flow chart of a conversational knowledge base question-answering method of the present invention. The process specifically includes the following steps:

[0034] Step S10: pre-define a fuzzy reasoning grammar system, and the query statement under the system is executed on the knowledge base to obtain an answer. The fuzzy reasoning grammar system specifically defined in step S10 includes:

[0035] Query operations include:

[0036] Find_subject(e,r): retrieves the triples in the knowledge base with entity e as the tail entity and r as the relation, and returns the set of head entities in all such triples;

[0037] Find_object(e,r): retrieves the triples in the knowledge base with entity e as the head entity and r as the relation, and returns the set of tail entities in all such triples;

[0038] Find_in(e,set): queries whether entity e is an element in set set and returns a Boolean value;

[0039] Screening operations include:

[0040] Filter_types(set e ,set t ); According to the entity type set e Type in the entity set e Filter the entities in and return the filtered results;

[0041] Counting operations:

[0042] Count(set): returns the number of elements in the set;

[0043] Count_subject_triple(r,τ1,τ2): retrieves the triples in the knowledge base that satisfy the relation r, the head entity type is τ1, and the tail entity type is τ2, and returns a dictionary where the key is the head entity and the value is the number of triples containing the head entity;

[0044] Count_object_triple(r,τ1,τ2): retrieves the triples in the knowledge base that satisfy the relation r, the head entity type is τ1, and the tail entity type is τ2, and returns a dictionary where the key is the tail entity and the value is the number of triples containing the tail entity;

[0045] Strict comparison operations:

[0046] Greater(dict,num): Returns the set of keys in the dictionary dict whose values ​​are greater than num;

[0047] Lesser(dict,num): Returns the set of keys in the dictionary dict whose values ​​are less than num;

[0048] Equal(dict,num): Returns the set of keys in the dictionary dict whose values ​​are equal to num;

[0049] Atleast(dict,num): Returns the set of keys in the dictionary dict whose values ​​are not less than num;

[0050] Atmost(dict,num): Returns the set of keys in the dictionary dict whose values ​​are not greater than num;

[0051] Argmax(dict): Returns the set of keys with the largest value in the dictionary dict;

[0052] Argmin(dict): Returns the set of keys with the smallest value in the dictionary dict;

[0053] Fuzzy comparison operations:

[0054] Fuzzy_greater(dict, num): Returns the values ​​in the dictionary dict that belong to the fuzzy set The set of keys;

[0055] Fuzzy_lesser(dict, num): Returns the values ​​in the dictionary dict that belong to the fuzzy set The set of keys;

[0056] Fuzzy_equal(dict, num): Returns the values ​​in the dictionary dict that belong to the fuzzy set The set of keys;

[0057] Collection operations:

[0058] Intersection(set1,set2): returns the intersection of set1 and set2;

[0059] Union(set1,set2): returns the union of set1 and set2;

[0060] Difference(set1,set2): Returns elements that exist only in set1.

[0061] Furthermore, given the above fuzzy comparison operations, the fuzzy set For a constant a, according to fuzzy set theory (Zadeh, 1965), fuzzy sets are defined as where μ(x) is the membership function of the number x to a, which is defined using a bell-shaped membership function as follows:

[0062]

[0063] Where b is a natural number and c is a constant. Define a threshold λ∈(0,1], then the fuzzy set The specific definition is as follows:

[0064]

[0065]

[0066]

[0067] when When x is approximately equal to a; when When x is approximately equal to a or greater than a; when When , x can be considered to be approximately equal to a or less than a. x is the input value, a is the constant to be compared with it, and the fuzzy set is used to measure whether x and a are approximately equal, approximately greater than, or approximately less than.

[0068] In this embodiment, b=1, λ=0.85.

[0069] Step S20: Obtain the current question to be answered and the conversation history information, such as Figure 2 , which is a flow chart of a method for obtaining the current question to be answered and the conversation history information. The process includes:

[0070] Step S201: obtaining the current question and the conversation history information, and preprocessing the question, the preprocessing includes: removing punctuation marks in the question (such as the question mark and period at the end of the question), and converting the words in the question into lowercase, etc.;

[0071] Step S202: concatenate the current question and the dialogue history information to form a new question. The dialogue history information refers to the question and answer of the previous round. The concatenation is to concatenate the question sentence and the dialogue history using the "[SEP]" tag, and add the "[CLS]" tag at the end to obtain the semantic representation of the whole input;

[0072] Step S203: Obtain the input vector representation, and use the GloVe model to obtain the vector representation of each word;

[0073] Step S30: According to the input vector representation, the hidden state vector is obtained through an encoder based on a multi-head attention mechanism, which is expressed as follows:

[0074] h enc =encoder(x;θ enc ) (2)

[0075] Where encoder() is the encoder function, θ dec are the trainable parameters of the encoder function;

[0076] Step S40: According to the hidden state vector h of the encoder enc , through the LSTM-based named entity recognition model, the named entities and their types are identified from the feature representation. Figure 3 As shown, a flowchart of the named entity recognition method. The process includes:

[0077] Step S401: The named entity and its type are identified from the feature representation through the LSTM-based named entity recognition model, as shown below:

[0078] h LSTM =LSTM(h enc θ LSTM ) (3)

[0079] Among them, LSTM() is a long short-term memory network, θ LSTM is the trainable parameter of LSTM, h LSTM is the hidden state vector obtained by LSTM;

[0080] Step S402: Filter the identified entities, and filter the entities in the question through a feedforward neural network, as shown below:

[0081] h FFN =LeakyReLU(W FFN1 [h enc ;h LSTM ]) (4)

[0082]

[0083] Among them, LeakyReLU() is a leaky rectified linear unit, W FFN1 and W FFN2 is the trainable parameter of linear layers 1 and 2, used to transform the matrix dimension, h FFN is the hidden state vector of the feedforward neural network, P(y ER |x) represents y ER The probability of x relative to x, where x is the input sequence, y ER is the true label of the named entity recognition task, is the hidden state vector of the feedforward neural network at time step t;

[0084] Step S50: Based on the named entity recognition result, the recognized entity is linked to the entity in the knowledge base through the entity disambiguation model based on XLNet to obtain the entity ID in the knowledge base. Figure 4 As shown, it is a flowchart of the entity disambiguation method, including:

[0085] Step S501: According to the result of named entity recognition (entity type, entity name), all entities that meet the result in the knowledge base are retrieved.

[0086] Step S502: determine whether the number of entities is 1. If so, directly return the entity ID and step S50 ends; otherwise, all entities constitute a candidate entity set;

[0087] Step S503: for each entity in the candidate entity set, retrieve the triples related to it in the knowledge base. In one embodiment, retrieve 3 related triples. Specifically, if the number of related triples is less than 3, use the entity name as the triple to fill;

[0088] Step S504: concatenate the current question, the candidate entity name, and the n entity-related triples using “[SEP]” to obtain the input of the disambiguation model;

[0089] Step S505: Obtain feature representation through the XLNet model;

[0090] Step S506: Perform binary classification through the linear layer to calculate the probability of the input being true or false;

[0091] Step S507: sort the entities in the candidate entity set in descending order according to the probability that the input is true, and return the ID of the first entity;

[0092] Step S60: According to the hidden state vector of the encoder, the logical form of the question in the current grammatical framework is obtained through the decoder based on the multi-head attention mechanism, which is expressed as follows:

[0093] z dec = decoder(h enc θ dec ) (6)

[0094]

[0095] Among them, decoder() is the decoder function, θ dec is the trainable parameter of the encoder function, z dec is the hidden state vector of the decoder function, P(y dec |x) represents y dec Relative to ydec The probability of y dec is the input sequence, y dec is the true label of the decoding task, W dec is a linear layer that transforms the dimension of the decoder hidden state vector;

[0096] Step S70: predict the relationship ID and entity type ID in the question based on the last bit of the encoder's hidden state vector, the decoder's hidden state vector and the knowledge base. Figure 5 As shown, it is a flowchart of the relationship ID and entity type ID in the prediction problem, including:

[0097] Step S701: Use the knowledge base information to obtain the embedded representation of entity types and relationships. Specifically, retrieve n related triples of entity types and relationships from the knowledge base, concatenate the entity type name (or relationship name) and the three triples using "[SEP]" to obtain the actual semantic representation, and obtain the embedded representation e through the BERT model. τr ;

[0098] Step S702: Calculate the probability of the input on different labels through the multi-label classification model. Specifically, calculate the last bit of the encoder hidden state vector The product of the embedding representation of different labels (all entity types and relations in the knowledge base) is obtained through the Sigmoid function to obtain the probability of the current input on different labels, which is expressed as:

[0099]

[0100] Among them, P(y MLC |x) is y MLC The probability of y relative to x MLC is the true label of the multi-label classification task, Sigmoid() is the S-shaped growth curve, is the last bit of the encoder’s hidden state vector, W MLC () is the linear layer parameter in the multi-label classification model, is the embedding representation of the transposed entity types and relations;

[0101] Step S703: Abstract the type-relationship graph from the knowledge base. Specifically, abstract the entity types and relationships in the knowledge base into nodes. If there is a triple (e h ,r,e t ), satisfying e h The type is τ1, e t If the type is τ2, then construct the edge <τ1, r> and<r,τ2> ;

[0102] Step S704: Obtain the feature representation h of the type-relationship graph through the GAT model GAT ;

[0103] Step S705: Perform personalized processing on the feature representation. Specifically, the probability distribution and the graph feature are calculated to obtain a personalized graph feature representation, that is:

[0104] h MLC =W TRP (h GAT ⊙P(y MLC |x)) (9)

[0105] Among them, h MLC is the hidden state vector output by the multi-label classification model, h GAT is the type-relation graph feature representation obtained by the GAT model, W TRP ( ) are the linear layer parameters;

[0106] Step S706: predicting relationships and entity types, including: predicting entity types and relationships contained in the input according to the last bit of the encoder feature representation, the decoder's hidden state vector and the personalized graph feature representation through a feedforward neural network, as shown below:

[0107]

[0108] Among them, P(y TRP |x) is y TRP The probability of y relative to x TRP is the real label of entity type and relationship prediction task, FFN( ) is a feedforward neural network, is the last bit of the encoder’s hidden state vector, is the decoder hidden state vector at time step t, T represents transpose;

[0109] Step S80: Integrate the entity ID, relationship ID, entity type ID and the decoded logical form to obtain a query statement that can be executed on the knowledge base;

[0110] Step S90: Execute the query statement in the knowledge base to obtain the answer corresponding to the current question.

[0111] In the above method, step 2 includes: constructing new input based on the current question and dialogue history, and using the GloVe model to obtain the embedding representation of the word.

[0112] In the above method, step 5 includes: constructing a candidate entity set and performing entity disambiguation through an XLNet-based model.

[0113] In the above method, constructing a candidate entity set includes: according to the result of named entity recognition (entity type, entity name), retrieving all entities and their related n triples in the knowledge base that meet the result. Among them, the entity-related triples refer to the triples when the entity is the head entity or the tail entity.

[0114] In the above method, entity disambiguation through the XLNet model includes: constructing the input of the disambiguation model for each entity, obtaining feature representation through the XLNet model, binary classifying the features through a linear layer, and selecting the entity with the highest probability as the entity to be finally linked.

[0115] In the above method, the input for constructing a disambiguation model for each entity includes: concatenating the current question, the candidate entity name, and n entity-related triples using "[SEP]" to obtain the input of the disambiguation model.

[0116] In the above method, step 7 includes: using knowledge base information to obtain embedded representations of entity types and relationships, calculating the probability of input on different labels through a multi-label classification model, abstracting a type-relationship graph from the knowledge base, obtaining a feature representation of the type-relationship graph through a GAT model, personalizing the feature representation, and predicting relationships and entity types.

[0117] In the above method, using knowledge base information to obtain embedded representations of entity types and relationships includes: constructing actual semantic representations of entity types and relationships based on relevant triple information, and obtaining embedded representations of entity types and relationships through a BERT model.

[0118] In the above method, constructing the actual semantic representation of entity types and relationships based on relevant triple information includes: retrieving n relevant triples of entity types and relationships from the knowledge base, concatenating the entity type name (or relationship name) and the n triple information using "[SEP]" to obtain the actual semantic representation.

[0119] In the above method, calculating the probability of input on different labels through the multi-label classification model includes: calculating the product of the last bit of the encoder hidden state vector and the embedded representation of different labels (all entity types and relations in the knowledge base), and obtaining the probability of the current input on different labels through the Sigmoid function.

[0120] In the above method, abstracting the type-relationship graph from the knowledge base includes: abstracting the entity types and relationships in the knowledge base into nodes, if for entity types τ1 and τ2 and relationship r, there exists a triple (e h ,r,e t ), satisfying e h The type is τ1, e tIf the type is τ2, then construct the edge <τ1, r> and<r,τ2> .

[0121] In the above method, personalizing the feature representation includes: calculating the Hadamard product between the probability distribution and the graph feature to obtain a personalized graph feature representation.

[0122] In the above method, predicting the relationship and entity type includes: predicting the entity type and relationship contained in the input based on the last bit of the encoder feature representation, the latent state vector of the decoder and the personalized graph feature representation through a feedforward neural network.

[0123] The effectiveness of the conversational knowledge base question answering method of the present invention is verified by experiments: the CSQA (Saha et al., 2018) dataset is used as the training dataset and the test dataset. The dataset consists of 200K dialogues of 1.6 million rounds of dialogue and more than 12.8 million entities from Wikidata, of which 153K, 16K, and 28K rounds of dialogue are used for training, verification, and testing, respectively. The questions in the dataset cover a variety of language phenomena, such as coreference, ellipsis, and inference.

[0124] For question types whose answers are entity sets, the F1 score is used to evaluate the effect; for question types whose answers are Boolean values ​​or numbers, the accuracy is used to evaluate the effect. The results show that compared with traditional semantic parsing methods, the accuracy of the present invention has greatly improved the accuracy of question answering. This shows that the method of the present invention can better answer complex questions under a large-scale knowledge base. As shown in Table 1, the experimental results of the present invention on the CSQA dataset are shown.

[0125] Table 1

[0126]

[0127] In the embodiment of the present invention, a function is used to define the grammar system, while in other embodiments, query statements such as SPARQL may also be used to complete the corresponding functions.

[0128] In the embodiment of the present invention, a generalized bell-shaped membership function is used to define the membership function in the fuzzy set, while in other embodiments, a Gaussian membership function or the like may also be used to define the membership function.

[0129] In an embodiment of the present invention, the GloVe model is used to obtain the encoding of the words in the question, while in other embodiments, models such as BERT may also be used to obtain the encoding of the words in the question.

[0130] In an embodiment of the present invention, an encoder based on a multi-head attention mechanism is used to obtain a hidden state vector, while in other embodiments, a model such as LSTM may also be used to obtain a hidden state vector.

[0131] In an embodiment of the present invention, a LSTM-based model is used for named entity recognition, while in other embodiments, a model such as BiLSTM (bidirectional long short-term memory network) may be used or a CRF (conditional random field) layer may be added for named entity recognition.

[0132] In an embodiment of the present invention, entity disambiguation is performed by fine-tuning a model based on XLNet, while in other embodiments, other pre-trained models (such as BERT, etc.) may be used to fine-tune or train neural network models such as GRU to perform entity disambiguation.

[0133] In an embodiment of the present invention, a GAT model is used to obtain features of a type-relationship graph, while in other embodiments, models such as GCN (graph convolutional neural network) may also be used to obtain features.

[0134] It should be noted that some exemplary methods are depicted as flow charts. Although the flow charts describe the operations as being performed sequentially, it is understood that many operations may be performed in parallel, simultaneously, or synchronously. In addition, the order of the operations may be rearranged. The process may terminate when the operations are completed, but may also have additional steps not included in the figures or in the embodiments.

Claims

1. A method for implementing conversational knowledge base question answering, It is characterized in that The method comprises the following steps: Step 1: pre-define the fuzzy reasoning grammar system; Step 2: Get the current question to be answered and the conversation history information, and get the vector representation through the GloVe model; Step 3: Through the encoder based on the multi-head attention mechanism, the hidden state vector of the encoder is obtained, which is expressed as follows: h enc =encoder(x;θ enc ) Where encoder() is the encoder function, θ dec are the trainable parameters of the encoder function; Step 4: Identify named entities and their types from feature representations using the LSTM-based named entity recognition model, which includes the following operations: Step S401: The named entity and its type are identified from the feature representation through the LSTM-based named entity recognition model, as shown below: h LSTM =LSTM(h enc ;θ LSTM ) Among them, LSTM () is a long short-term memory network, θ LSTM is the trainable parameter of LSTM, h LSTM is the hidden state vector obtained by LSTM; Step S402: Filter the identified entities, and filter the entities in the question through a feedforward neural network, as shown below: h FFN =LeakyReLU(W FFN1 [h enc ;h LSTM ]) Among them, LeakyReLU () is a leaky rectified linear unit, W FFN1 and W FFN2 is the trainable parameter of linear layers 1 and 2, used to transform the matrix dimension, h FFN is the hidden state vector of the feedforward neural network, P(y| ER |x) represents y ER The probability of x relative to x, where x is the input sequence, y ER is the true label of the named entity recognition task, is the hidden state vector of the feedforward neural network at time step t; Step 5: Link the identified entity to the entity in the knowledge base through the XLNet-based entity disambiguation model to obtain the entity ID in the knowledge base; Step 6: According to the hidden state vector of the encoder, the logical form of the question in the current grammatical framework is obtained through the decoder based on the multi-head attention mechanism; it is expressed as follows: z dec decoder(h enc ;θ dec ) Where decoder( ) is the decoder function, θ dec is the trainable parameter of the encoder function, z dec is the hidden state vector of the decoder function, P(y dec |x) represents y dec Relative to y dec The probability of y dec is the input sequence, y dec is the true label of the decoding task, W dec is a linear layer that transforms the dimension of the decoder hidden state vector; Step 7: Predict the relation ID and entity type ID in the question based on the last bit of the encoder's hidden state vector, the decoder's hidden state vector, and the knowledge base knowledge; Step 8: Integrate the entity ID, relationship ID, entity type ID and the decoded logical form to obtain a query statement that can be executed on the knowledge base; Step 9: Execute the query statement in the knowledge base to obtain the answer corresponding to the current question.

2. A method for implementing a conversational knowledge base question and answer system as claimed in claim 1, It is characterized in that The step 1 includes defining query operations, screening operations, counting operations, strict comparison operations, fuzzy comparison operations and set operations.

3. A method for implementing a conversational knowledge base question and answer system as claimed in claim 1, It is characterized in that The specific operations of step 2 are as follows: Step S201: obtaining the current question and the conversation history information, and preprocessing the question, the preprocessing includes: removing punctuation marks in the question sentence, and converting the words in the question sentence into lowercase; Step S202: combining the current question and the conversation history information to form a new question; Step S203: Obtain the input vector representation and use the GloVe model to obtain the vector representation of each word.

4. A method for implementing a conversational knowledge base question and answer system as claimed in claim 1, It is characterized in that The step 5 specifically includes the following operations: Step S501: According to the result of named entity recognition, all entities satisfying the result in the knowledge base are retrieved; Step S502: determine whether the number of entities is 1. If so, directly return the entity ID and step S50 ends; otherwise, all entities constitute a candidate entity set; Step S503: for each entity in the candidate entity set, retrieve the triples related to it in the knowledge base. In one embodiment, retrieve 3 related triples. Specifically, if the number of related triples is less than 3, use the entity name as the triple to fill. Step S504: concatenate the current question, the candidate entity name, and the n entity-related triples using "[SEP]" to obtain the input of the disambiguation model; Step S505: Obtain feature representation through the XLNet model; Step S506: Perform binary classification through the linear layer to calculate the probability of the input being true or false; Step S507: Sort the entities in the candidate entity set in descending order according to the probability that the input is true, and return the ID of the first entity.

5. A method for implementing a conversational knowledge base question and answer system as claimed in claim 1, It is characterized in that The step 7 specifically includes the following operations: Step S701: Use the knowledge base information to obtain the embedded representation of entity types and relations; specifically, retrieve n related triples of entity types and relations from the knowledge base to obtain the actual semantic representation, and obtain the embedded representation e through the BERT model. τr ; Step S702: Calculate the probability of the input on different labels through the multi-label classification model; specifically, calculate the last bit of the encoder hidden state vector The product of the embedding representation of different labels is obtained through the Sigmoid function to obtain the probability of the current input on different labels, which is expressed as: Among them, P(y MLX |x) is y MLC The probability of y relative to x MLC is the true label of the multi-label classification task, Sigmoid ( ) is the S-shaped growth curve, is the last bit of the encoder’s hidden state vector, W MLC ( ) is the linear layer parameter in the multi-label classification model, (e τr ) T is the embedding representation of the transposed entity types and relations; Step S703: Abstract the type-relationship graph from the knowledge base, abstract the entity types and relationships in the knowledge base into nodes, if the entity type τ 1 and τ 2 With relation r, there exists a triple (e h ,r,e t ), satisfying e h The type is τ 1 , e t The type is τ 2 , then construct an edge <τ in the type-relation graph 1 ,> and <r,τ 2 > Step S704: Obtain the feature representation h of the type-relationship graph through the GAT model GAT ; Step S705: Perform personalized processing on the feature representation, calculate the Hadamard product between the probability distribution and the graph feature, and obtain a personalized graph feature representation, that is: h MLC =w TRP (h GAT ☉P( MIC |x)) Among them, h MLC is the hidden state vector output by the multi-label classification model, h GAT is the type-relation graph feature representation obtained by the GAT model, W TRP ( ) are the linear layer parameters; Step S706: predicting relationships and entity types, including: predicting entity types and relationships contained in the input according to the last bit of the encoder feature representation, the decoder's hidden state vector and the personalized graph feature representation through a feedforward neural network, as shown below: Among them, P(y TRP |x) is y TRP The probability of y relative to x TRP is the real label of entity type and relationship prediction task, FFN( ) is a feedforward neural network, is the last bit of the encoder’s hidden state vector, is the decoder hidden state vector at time step t, and T represents transpose.