Multi-channel characterization interaction enhanced incomplete knowledge base multi-hop question and answer method

Abstract

The multi-channel representation interaction enhanced incomplete knowledge base multi-hop question and answer method comprises the steps that a fine-grained multi-round representation fusion module and a multi-channel double-tower matching module are constructed, and a representation similarity evaluation function is optimized according to the embedding characteristics of a knowledge base; the method aims to align query characterization and knowledge base characterization through a fine-grained multi-channel characterization interaction strategy. And the fine-grained multi-round representation fusion module realizes early fusion of knowledge base and query question semantics through representation interaction nodes, and performs multi-round information interaction in a representation updating process. The multi-channel double-tower matching module utilizes different semantic extraction matrixes to process the representation of a query question and a knowledge base, so that the information interaction capability is expanded.

Description

Technical Field

[0001] The present application relates to natural language processing technology, and in particular, to the field of knowledge base multi-hop question answering in natural language processing. Background Art

[0002] The knowledge base intelligent question answering system is committed to meeting the complex information needs of users. It allows users to ask questions in the form of natural language queries and can directly provide users with accurate answers. Different from the traditional knowledge base question answering task that usually only focuses on a single triple in the knowledge base, the multi-hop question answering task of an incomplete knowledge base requires performing multi-hop reasoning on a knowledge base with a large number of missing tuples. This reasoning process not only involves more triples, but also lacks guidance from the intermediate steps.

[0003] like Figure 1 As shown in the figure, the current mainstream knowledge base multi-hop question answering methods usually map natural language questions and knowledge base entities to a low-dimensional space of the same dimension through a pre-trained language model and a knowledge base embedding model, and then carry out the reasoning process based on a dual-tower matching model. Although this method of using a pre-trained model can obtain the semantics of questions and knowledge bases, their vector representations are usually generated by two non-interacting encoders and decoders. During the model training process, these two vector representations need to further interact to fuse semantic information. However, since the model needs to use the similarity between the two vector representations to guide the reasoning process during training, this late fusion strategy is difficult to generate correct guidance information when the model has not converged, making it difficult for the model to learn effective information in the initial stage.

[0004] In short, the existing dual-tower matching model faces significant challenges in dealing with the alignment of queries and knowledge base entities. Although it has advantages in terms of computational complexity, it only performs shallow and one-way interactions between natural language queries and knowledge base entities, and has limited ability to fuse information from the two modalities. In particular, when the fine-tuned knowledge base and the predicted knowledge base are inconsistent, the model performs poorly. In addition, in the information transmission process of the dual-tower model, feature information is gradually compressed between layers, and only realizing information interaction at the final layer may cause the loss of original detailed information. In addition, due to different representation distributions and training methods, it is difficult to fuse the representations on both sides.

[0005] In view of the above problems, the present invention proposes a multi-channel representation interaction model (MCRIM) for improving the accuracy of information integration and the overall performance of the question-answering system, which aims to align the query representation and the knowledge base representation through a fine-grained multi-channel interaction strategy, and mainly includes two modules: a fine-grained multi-round representation fusion module and a multi-channel dual-tower matching module. The fine-grained multi-round representation fusion module uses the representation interaction node as a bridge for information transmission in the process of updating the representation, and adds multiple rounds of representation interaction in the dual-tower structure to fuse the knowledge base and question semantics, thereby achieving alignment of the semantic spaces of the two; the multi-channel dual-tower matching module is a multi-channel dual-tower matching architecture for the current complex vector space feature matching task. In the representation interaction process, different semantic extraction matrices are designed for the representations on the query side and the knowledge base side, respectively, and the information interaction capability of the dual-tower architecture is expanded by connecting different semantic features in parallel, thereby achieving richer semantic extraction and matching. Summary of the invention

[0006] The present invention aims to enhance the information interaction between the dual-tower models and promote the effective fusion of query representation and knowledge base representation to improve the performance of the model when processing complex queries. The present invention introduces a fine-grained multi-round representation fusion module to guide the fusion of query representation and knowledge base representation, and proposes an optimized representation similarity evaluation function to improve the consistency of training objectives and prediction objectives. At the same time, in order to fully capture the similarity between representations in different feature spaces, the present invention introduces a multi-channel matching method, and uses different loss functions on different channels to guide the model to learn representation similarities at different levels, further improving the accuracy of knowledge base question-answering tasks.

[0007] The main innovative features of the present invention are as follows:

[0008] 1. Enhanced dual-tower representation interaction: In order to align the semantics and representation information of the pre-trained model, the dual-tower interaction module after the embedding layer of the knowledge base question-answering model was reconstructed. By introducing representation interaction nodes, the weights between different representations are dynamically adjusted using the attention mechanism to ensure that key information receives more attention and is transmitted.

[0009] 2. Optimize the representation similarity evaluation function: In order to enhance the ability of the feature matching module to capture entities and relations related to the query, the original representation similarity evaluation method is optimized to more accurately measure the semantic similarity between the query and the knowledge base. In addition, the contrast loss function is introduced to make full use of the negative examples naturally existing in the knowledge base.

[0010] 3. Multi-channel representation interaction enhancement method: In order to enhance the model's ability to capture features in complex vector space, multiple feature extraction channels are designed to enhance the model's ability to represent and extract vectors in different feature spaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a structural diagram of the fine-grained multi-round characterization interaction module of the present invention.

[0012] Figure 2 This is the basic structural diagram of the dual-tower model in the knowledge-based question-answering system.

[0013] Figure 3 This is a structural diagram of the multi-channel dual-tower matching module of the present invention. DETAILED DESCRIPTION

[0014] The present invention is used to realize early fusion of features in the dual-tower model and representation matching of multiple feature channels, thereby optimizing the accuracy of the dual-tower matching module and the inference model, including constructing a fine-grained multi-round representation fusion module and a multi-channel dual-tower matching module, and optimizing the representation similarity evaluation function according to the characteristics of knowledge base embedding.

[0015] The current knowledge base question answering systems all use a dual-tower architecture, mapping queries and knowledge bases into vector spaces of the same dimension using different embedding models. A similarity calculation module is introduced at the top of the dual towers to measure the correlation between query representations and knowledge base entity or relationship representations, thereby selecting the most semantically matching entity or relationship. This method improves the efficiency of the embedding module, but due to the inconsistency of the pre-trained corpora of the two pre-trained models and the different training objectives, the semantic gap between their vector representations is large. Figure 2 As shown in the figure, the basic structure of a commonly used dual-tower model in knowledge base question answering consists of two independent neural networks, one for query encoding and the other for encoding knowledge base entities and relationships. Each tower works independently to represent and construct entities on the query side and in the knowledge base respectively, and finally matches them by calculating the similarity between the two to find the answer that best matches the query. The left tower embeds each word of the query into the word vector space, and generates a contextual representation of the question through the neural network of the left tower. These representations are then further processed and finally a vector Q marked with a special marker symbol [CLs] is generated. [CLS] Represents the features of the query. This representation captures the global semantic information of the question and is used to match it with entities in the knowledge base. The right tower is used to convert the structured information (entities, relations, etc.) in the knowledge base into a vector representation. Specifically, it converts the entity E in the knowledge graph into i , relation R i , and the triple T i Embedded into a vector space. With these embeddings, the system can represent the structured data in the knowledge base and generate a global representation for each entity and relationship.

[0016] The core goal of the present invention is to enhance the information interaction between the dual-tower models and promote the effective fusion of query representation and knowledge base representation to improve the performance of the model when processing complex queries. The present invention includes constructing a fine-grained multi-round representation fusion module and a multi-channel dual-tower matching module, and optimizing the representation similarity evaluation function according to the characteristics of knowledge base embedding. The fine-grained multi-round representation fusion module realizes the early fusion of knowledge base and query semantics by introducing interactive nodes, and performs multiple rounds of interaction during the representation update process. The multi-channel dual-tower matching module uses different semantic extraction matrices to process the representation of questions and knowledge bases to expand the information interaction capabilities. In addition, in order to align the semantics of questions and knowledge bases, the present invention also optimizes the representation similarity evaluation function and proposes an evaluation method suitable for query question semantics and relationships. The improvements of the present invention based on the dual-tower architecture include:

[0017] 1. Construct a fine-grained multi-round representation fusion module

[0018] The goal of the fine-grained multi-round representation fusion module is to interact with the opposite side’s representation multiple times as the dual-tower level rises, thereby retaining the combined features, improving the degree of information fusion and sensitivity to the opposite side’s information.

[0019] The module architecture is as follows Figure 1 This module first adds two additional interaction characters, which are used for query side and knowledge base side respectively, denoted as q int and e int , and a shallow representation fusion network composed of multi-layer perceptrons, denoted as fusion layer FuNet(q, e). When performing query representation, the present invention uses the pre-trained language model BERT (https: / / paperswithcode.com / method / bert) to map the question into a low-dimensional vector space, and uses the [cls] character as the representation fusion character on the query side to obtain the initial representation of the query

[0020] Enc(Q) = {w 1 , ..., w n , [cls], q int}

[0021] Where [cls] represents the query-side representation fusion character, Q = {w 1 , w 2 , ..., w n} represents a query sentence, w i Indicates the constituent words of the query. Then, the Transformer model (https: / / paperswithcode.com / method / transformer) is used in each layer to update the query representation. In this invention, the representation of the query side intermediate layer is recorded as Where j represents the position of the word in the sentence, and l represents the current layer. In particular, the state of the middle layer representing the fused character is recorded as [cls]l, and the representation update of each layer can be expressed as follows:

[0022]

[0023] When representing the knowledge base, the present invention first needs to construct a subgraph related to the query central entity according to the extraction method proposed by the PullNet model (https: / / paperswithcode.com / paper / pullnet-open-domain-question-answering-with), which contains neighbor entities of up to 3 hops around the central entity of the question. The core concept of the PullNet model is to construct a query subgraph in an iterative manner, and the initial subgraph contains the query and related entities. In each iteration, the graph convolutional network GCN (https: / / paperswithcode.com / paper / semi-supervised-classification-with-graph) is used to calculate the probability of the entity node in the query subgraph participating in the next round of iteration, and select the entity whose probability exceeds the threshold. Then, for each selected entity, the related sentences and triples in the knowledge graph are retrieved, and the new entity and triple are used as new nodes, and the connection between the new nodes is used as a new edge, so as to update the question subgraph.

[0024] G sub ={e 1 , e 2 , ..., e n , e int}

[0025] Among them, G sub is the query subgraph, which represents a subset of the knowledge graph related to the query and the central entity of the query. j Represents the entity nodes that constitute the subgraph. Note that j only represents the entity number and does not contain location information. As mentioned above, query subgraph G sub An additional interaction node e is introduced in int , the node will extend a link without semantic information to all nodes in the subgraph. The method proposed in this module uses the graph attention mechanism to update the representation of the knowledge base. In each layer of GCN, the representation of the current node embedding is input into the graph attention layer to propagate information between nodes and generate a pre-fused node for each entity. The embedding is shown in the following formula:

[0026]

[0027] where em and e n denote the representations of node m and node n respectively, MLP(·) denotes a multi-layer perceptron, and W node and W edge represents the weight matrix, d represents the dimension of representation, represents the set of nodes adjacent to node m, Attention(·) represents the attention mechanism, r nm Represents the relationship between node m and node n. The symbol [,] in the formula represents the concatenation operation of the vector, and the symbol [·] represents the matrix multiplication. Through the attention mechanism, the attention weight of node m and its adjacent node n is first calculated, which takes into account both the node representation and the edge representation. Then, the neighbor aggregation representation of node m is obtained by summing the weighted adjacent node representations. Finally, the multi-layer perceptron and weight matrix are used to perform linear transformation and activation function operations to obtain the updated node representation. The introduction of the attention mechanism can dynamically adjust the weight of features, strengthen the features that need attention, and ignore irrelevant information. After using the Transformer layer and the GCN layer to update the question embedding and the knowledge base embedding respectively, the fusion module will perform semantic fusion on the interaction nodes on the query side and the knowledge base side. Know After splicing, it is sent to the MLP layer, and the two are mixed and then divided into Know

[0028]

[0029] in, It is a semantic interaction node in the middle layer of the query side. is the semantic interaction node after semantic fusion in the current layer, and MLP[·] represents multi-layer perceptron.

[0030] In the fusion layer, except for the two representation interaction nodes to be fused, other nodes do not participate in the fusion layer, which can appropriately reduce the complexity of the dual-tower interaction and improve the efficiency of the model. By stacking several identical fusion layers, the representations on the user side and the knowledge base side can have sufficient interaction, which improves the correlation between the representations of the two, and thus improves the accuracy of subsequent entity selection.

[0031] 2. Optimize the representation similarity evaluation function

[0032] In the current more advanced knowledge base question answering algorithms, query embedding usually uses the pre-trained language model BERT (https: / / paperswithcode.com / method / bert) series to obtain query representations, while knowledge base embedding uses the CompIEx model (https: / / paperswithcode.com / paper / complex-embeddings-for-simple-link-prediction) or RotatE (https: / / paperswithcode.com / method / rotate) model. The vector representation output by the BERT model is a ball in a high-dimensional Euclidean space, in which most vectors are concentrated in a spherical cone in a specific area. This spherical cone structure can be regarded as a vector aggregation area representing different semantic categories and features, allowing the BERT model to effectively distinguish and encode different language information in the entire space. In contrast, the representations output by the CompIEx model and the RotatE model are located in a high-dimensional complex space. These models use the particularity of complex numbers to capture richer and more complex semantic relationships. The characteristics of the complex space allow these models to model the real and imaginary parts separately, thereby better representing and capturing the associated information and structured information in the knowledge base.

[0033] In the dual-tower architecture, the representation similarity evaluation function usually uses a bilinear function to measure the degree of match between vectors. The bilinear function performs a dot product operation on two feature vectors and then maps them to a scalar value through a linear mapping, which is directly used in subsequent entity selection. This method is simple and direct, but when the semantic space of the question and knowledge base is inconsistent with the representation modeling space, directly calculating the dot product or cosine similarity cannot accurately measure the semantic match between the two.

[0034] The present invention improves the CompIEx algorithm used for knowledge base embedding representation in the dual-tower model architecture, that is, introduces query-related loss terms and negative samples, thereby optimizing the performance of the representation similarity evaluation function. Among them, CompIEx is a knowledge graph embedding algorithm used to embed entities and relationships in a knowledge graph into a complex space to perform reasoning and link prediction tasks on the knowledge graph. By modeling the representation of entities and relationships in complex space, the algorithm can capture more complex interactions between entities and relationships and support the modeling of asymmetric relationships. The dot product in complex space is usually the Hermitian Product. For the complex vector u=x u +iy u , v = x v +iy v , the dot product of the two is defined as:

[0035]

[0036] in Represents a complex vector v The conjugate vector of . The representation similarity evaluation function of the knowledge base is defined as:

[0037]

[0038] Among them, e h 、e r 、e t They represent the vector representation of the head entity node, the vector representation of the knowledge base relationship, and the vector representation of the tail entity node, respectively. Indicates e t Taking the conjugate, Re(·) represents the real part of the complex vector, The corresponding value is e h 、e r , The complex number representation in the kth dimension, d represents the dimension of the embedding vector. For all existing triple relations (h, r, t), f(h, r, t)>0; if f(h, r, t)<0, it means that there is no such knowledge in the knowledge base.

[0039] When evaluating the semantic similarity between a query and a relation, the representation similarity evaluation function becomes:

[0040]

[0041] Among them, e h 、e q 、e a They represent the vector representation of the head entity node, the vector representation of the knowledge base query, and the vector representation of the answer entity node, respectively. Indicates e a Taking the conjugate, Re(·) represents the real part of the complex vector, The corresponding value is e h 、e q , In the complex number representation of the kth dimension, d represents the dimension of the embedding vector. For all knowledge base queries (h, q, a) with correct answers, f(h, q, a)>0; if f(h, q, a)<0, it means that the answer to the knowledge base query is wrong. The query sentence is regarded as a special "relationship" between the head entity and the answer entity, and the goal of the representation similarity evaluation function is to prefer the relationship that is closest to the semantics of the query sentence. In order to optimize the CompIEx algorithm's ability to capture query representations, the present invention introduces an evaluation function that measures the correlation between query representations and knowledge base relationships on the basis of ensuring that the goal of the original representation similarity evaluation function remains unchanged:

[0042]

[0043] Among them, h, q, and r represent the head entity, query sentence, and relation respectively, and e h 、e q 、e r They represent the vector representation of the head entity node, the vector representation of the knowledge base query, and the vector representation of the knowledge base relationship. Indicates e r Taking the conjugate, Re(·) represents the real part of the complex vector, The corresponding value is e h 、e q , The complex number representation in the kth dimension, d represents the dimension of the embedding vector. The value of f(h, q, r) is used to evaluate the degree of match between the triples (h, q, r). The higher the score, the more likely the triple exists. By replacing the answer entity a with the relation r, the training task and the prediction task are aligned.

[0044] In the knowledge base question answering method proposed in the present invention, it is necessary to compare the user's query and its hidden state with multiple candidate answers in the knowledge base and find the most relevant answer. The contrast loss function is a loss function used to learn similarity metrics. It is based on the following idea: for correct query-answer pairs, their similarity scores should be higher; and for incorrect query-answer pairs, their similarity scores should be lower. The present invention selects strong negative examples and randomly selects weak negative examples to implement the contrast loss function:

[0045]

[0046] Loss = L pos +ω 1 ·L neg-hard +ω 2 ·L neg-soft

[0047] Among them, R pos is a set of positive examples, which is selected from the relationship between the central entity and the answer in the single-hop question answering; R nh is a set of strong negative examples, which selects the relationships around the central entity that do not point to the answer entity; R ns is a weak negative example set, which selects the uninvolved relations around the central entity. γ represents a fixed boundary parameter used to control the score difference between positive and negative samples. σ 1 , σ 2 , σ 3 represents the activation function, L pos Represents the loss function of the positive sample, L neg-hard represents the loss function of strong negative examples, L neg-soft represents the loss function of weak negative examples, ω 1 and ω2 is a hyperparameter that controls the loss weights of strong negative examples and weak negative examples. Loss is the total loss function, which combines the losses of positive samples, strong negative examples, and weak negative examples. 1 and ω 2 To adjust the balance of positive and negative samples. The present invention uses the Loss function to guide the MCRIM model to optimize the representation similarity evaluation function f(h, q, a), and uses ω 1 and ω 2 Guide the MCRIM model to learn to distinguish different types of negative samples (strong negative examples and weak negative examples), so that it can find the correct answer more accurately in real applications.

[0048] 3. Build a multi-channel dual-tower matching module

[0049] In the knowledge base question answering task, the dual-tower architecture has the problem of single-channel information congestion and mixing. Even if the query does not contain other features such as user preferences, information such as query semantics and central entity semantics still interfere with each other in the tower. Especially in the current mainstream modeling method, the real and imaginary parts of the representation model semantic information at different levels respectively. Previous methods usually simply linearly transform the representation and map it to the same dimension, which makes it impossible to distinguish the rich semantic information contained in the representation.

[0050] In order to solve this problem, the present invention constructs a multi-channel dual-tower matching module. This scheme allows information at different levels to flow upward through channels suitable for themselves, thereby reducing the mixing and interference of information. The present invention uses the real and imaginary parts of the original information as two independent channels to construct a multi-channel dual-tower model, which can capture different features and information and provide a more comprehensive and diverse representation. This multi-channel architecture allows information to flow upward and compress in their respective channels, avoiding the problems of information congestion and mutual interference. In this way, the multi-channel architecture can improve the performance of the dual-tower architecture and improve the signal-to-noise ratio.

[0051] In a multi-channel architecture, each channel c attempts to extract useful features s from the input data. c And suppress noise n c The signal-to-noise ratio (SNR) of each characteristic channel is defined as

[0052]

[0053] Where P s，c is the signal power of channel c, P n，c is the noise power of channel c.

[0054] In the single-channel case, the signal-to-noise ratio is fixed and is determined by the design of the channel and the nature of the input data. However, in a multi-channel architecture, the overall signal-to-noise ratio can be improved by properly weighting and integrating the output of each channel. Let the total number of channels be C and the output of each channel be v c =s c +n c The goal of the model is to find a way to integrate the outputs of these channels to maximize the overall signal-to-noise ratio. By weighted averaging, the integrated output can be obtained:

[0055]

[0056] Among them, w c is the weight of the cth channel, which satisfies the non-negative property and Since the noise in the information is additive white noise, when integrating the information of multiple channels, it is hoped that the power of the signal will be enhanced, while the power of the noise will be only slightly affected or even suppressed. In order to approximately calculate the noise power P after integration noise,output , considering the noise power of each channel and assuming that they are independent and uniformly distributed, the integrated noise power is:

[0057]

[0058] where w c is the weight of the cth channel, which is regarded as the frequency of the information in this channel, which determines the contribution of this channel in the integration process, P noise,c represents the noise power in channel c. The integrated signal power can be approximated as:

[0059]

[0060] Among them, Cov(s i ,s j ) is the covariance of signals from different channels, s k represents the kth channel signal, P signal,c represents the signal power in channel c, w q Represents the weight of the qth channel. Combined with the SNR formula, if the covariance between different channels is positive, the integrated signal power will increase, which means that the noise is suppressed between channels.

[0061] Each matching channel of the multi-channel dual-tower matching module can focus on features at different levels in the query and knowledge, use different functions at different feature levels to evaluate the similarity or correlation between features, and promote feature interactions at different levels. In this way, on the one hand, it can capture the correlation information of multiple semantic aspects between the query and knowledge, improve the matching performance and semantic understanding ability, and on the other hand, multiple channels can use different functions to calculate the correlation and optimize the feature capture ability. Aggregate the representations of parallel matching channels at the interaction layer to comprehensively consider the real and imaginary information. Such a multi-channel dual-tower model can better capture the characteristics and correlations of the original information and improve the expression ability and performance of the model. The architecture of the multi-channel dual-tower matching algorithm is shown in the figure. Figure 3 As shown. It allows feature interactions at different levels by introducing multiple matching channels, and comprehensively considers information from different channels during the aggregation process. The method proposed by the multi-channel dual-tower matching module of the present invention includes three parallel channels, namely the original characterization channel, the real characterization information, and the imaginary characterization information. The original characterization channel is responsible for the lossless delivery of the original information to the dual-tower interaction layer, while the real and imaginary characterization channels respectively handle the information interaction with the real and imaginary parts of the opposite side. The design of each channel is introduced in detail below.

[0062] The original representation channel contains all the information of the query. During the process of the two towers rising, with each round of interaction with the information on the other side, the original information of the query semantics will gradually become more blurred. In order to preserve the original query semantics, an additional bridge from the initial representation to before the interaction is designed, that is, the final representation for similarity evaluation:

[0063]

[0064] Where L is the number of interaction layers stacked, τ θ is a hyperparameter that controls the retention rate of original information. Represents the representation vector e x The initial embedding vector of . Before introducing the real channel and the imaginary channel, we first explain their semantic meaning. The representation similarity evaluation function in the form of complex Hermitian product is defined as:

[0065]

[0066] Where Im(·) represents the imaginary part of the complex vector. q When e is a pure real vector, f(h, q, a) = f(a, q, h), that is, the relationship satisfies symmetry; when e qWhen is a pure imaginary vector, f(h, q, a) + f(a, q, h) = 0, that is, the relationship satisfies antisymmetry. Under this setting, the real channel specifically processes the real part of the complex vector representation, while the imaginary channel processes the imaginary part. This separation ensures that the real and imaginary information can be processed independently in the subsequent feature representation interaction process. From the overall architecture, although the width of the real and imaginary channels is half the width of the original complex representation channel, when they are merged, the original complex representation can be reproduced. It is worth noting that during the feature representation update process, due to the effect of the fully connected layer, the original complex representation channel may contain cross-features of the real and imaginary parts. In contrast, the real and imaginary channels only retain the feature information within their respective channels, avoiding the intersection of features, thereby ensuring the purity and accuracy of information processing. In the subsequent feature interaction layer, the original representation channel is calculated as a complex vector to calculate the dot product, while the vectors of the real channel and the imaginary channel are calculated as real vectors to calculate the dot product, and the calculated representation similarity evaluation scalar is integrated by linear regression:

[0067] F(h, q, a)=f(h, q, a)+θ 1 ·f Re (h,q,a)+θ 2 ·f Im (h, q, a)

[0068] Among them, h, q, a represent the head entity, query question, and answer entity respectively, and f Re (h, q, a) represents the similarity evaluation function based on the real part of the complex vector, f Im (h, q, a) represents the similarity evaluation function based on the imaginary part of the complex vector, θ 1 ,θ 2 represents the weighting coefficient, and F(h, q, a) represents the representation similarity evaluation function based on the overall information. In the reasoning stage, the final selected answer is based on the result of the representation similarity evaluation function F(h, q, a), and the answer with the highest score is considered to be the most relevant answer.

[0069] The present invention evaluates the effectiveness of the proposed model MCRIM on multiple multi-hop knowledge graph question answering benchmark datasets, including MetaQA (https: / / paperswithcode.com / dataset / metaqa), WebQSP (https: / / paperswithcode.com / dataset / webquestionssp), and ComplexWebQSP (https: / / paperswithcode.com / dataset / complexwebquestions) for comparative experiments. In addition, in order to verify the effect of the representation similarity evaluation function proposed in the present invention, the present invention uses the BERT model as the query encoder and the CompIEx model as the knowledge base encoder, respectively, and conducts experiments on the MetaCA single-hop dataset to train and test the feature matching module. The baseline models used are GraftNet model (https: / / paperswithcode.com / paper / open-domain-question-answering-using-early), PullNet model (https: / / paperswithcode.com / paper / pullnet-open-domain-question-answering-with), KV-Mem model (https: / / paperswithcode.com / paper / key-value-memory-networks-for-directly), EmbedKGCA model (https: / / paperswithcode.com / paper / improving-multi-hop-question-answering-over), LEGO model (https: / / par.nsf.gov / biblio / 10300277-lego-latent-execution-guided-reasoning-multi-hop-question-answering-knowledge-graphs) and GenBoost model (https: / / ieeexplore.ieee.org / document / 10476130).

[0070] The experimental results are shown in Table 1. The evaluation index used in the present invention is the Hit@1 index. The experimental conclusions are as follows:

[0071] (1) MetaCA dataset: On the MetaCA dataset, the model made some progress on the 1-hop task, slightly lower than the EmbedKGCA (https: / / paperswithcode.com / paper / improving-multi-hop-question-answering-over) method. On the 2-hop task, the GenBoost+MCRIM model achieved the best performance (63.2%), which was 3% higher than the original GenBoost model. This shows that by introducing MCRIM, the GenBoost model's ability to handle more complex problems has been significantly improved. On the 3-hop problem, GenBoost+MCRIM also showed the best performance (70.3%), which was 2.2% higher than the original GenBoost model. This further confirms the effectiveness of MCRIM in dealing with complex problems. On the single-hop problem, GenBoost+MCRIM is slightly inferior to the performance of EmbedKGCA.

[0072] Table 1 Comparison of MCRIM performance on the remaining 50% tuple dataset (%)

[0073]

[0074] (2) WebQSP dataset: The GenBoost model also performs quite well on the WebQSP task. After introducing MCRIM as an improvement, the accuracy of GenBoost+MCRIM is further improved to 58.5%, which is 1.9% higher than the original GenBoost model. This shows that the introduction of MCRIM is effective in improving the performance of the model on the WebQSP task. By comparing the performance of GenBoost and GenBoost+MCRIM, it can be observed that MCRIM has a positive effect on improving model performance. In the WebQSP task, the addition of MCRIM increases the accuracy of the model, which may be because MCRIM can better capture and fuse information from different channels, thereby improving the accuracy of the model's understanding and answering of questions.

[0075] (3) ComplexWebCSP dataset: Due to the limited number of submissions in the CWC test set, the present invention compares the results on the validation set. On the CWC dataset with 50% missing tuples, the overall performance of EmbedKGCA and GenBoost+MCRIM on the CWC dataset is relatively outstanding, especially EmbedKGQA achieves high accuracy on 1-hop questions and GenBoost+MCRIM achieves high accuracy on 2-hop and 3-hop questions. These results show that combining effective model improvement and feature fusion strategies can significantly improve the performance on complex question-answering tasks.

[0076] (4) EmbedKGCA enhanced experiment: The present invention enhances the EmbedKGCA model and conducts corresponding experiments. On the 1-hop problem, the accuracy of EmbedKGQA+MCRIM slightly decreases to 67.8%, which is 2.8% lower than the original EmbedKGCA model. This may indicate that when dealing with direct, single-hop problems, the introduction of MCRIM does not bring about performance improvements, and may even lead to performance degradation due to the increase in model complexity. EmbedKGQA+MCRIM shows good generalization ability on different multi-hop problems. Whether on the MetaCA, WebQSP or CWC datasets, the model can adapt to different types of questions and knowledge graph structures and provide accurate answers. It shows that the fusion of the semantic features and knowledge base structure information of the two is significantly beneficial to the knowledge base multi-hop reasoning task.

Claims

1. A method for multi-hop question answering based on an incomplete knowledge base with enhanced characterization interaction, characterized in that: The following steps are involved: Step 1: Get the query sentence input by the user; Step 2: On the query side of the dual-tower architecture, use the pre-trained language model BERT to map the input query sentence to a low-dimensional vector space, add a representation interaction node, and extract the initial representation of the query: Enc(Q)={w0,w1,...,w n ,[cls],q int } Where [cls] represents the query-side representation fusion character, w i represents the query vector representation, q int represents the query-side representation interaction node, Q = {w1, w2, ..., w n } represents a query sentence, w i Represents the constituent words of the query sentence; Step 3: On the knowledge base side of the dual-tower architecture, use the PullNet model to build a query subgraph G related to the query and the query central entity sub , query subgraph G sub It consists of at most three neighbor entities related to the query central entity: G sub {e1,e2,…,e n ,the int } Among them, G sub represents the query subgraph, e j Represents the entity nodes that constitute the query subgraph, e int Represents the representation interaction node on the knowledge base side; Step 4: Complete the initial construction of query representation and knowledge base entity representation in the dual-tower architecture and input them into the fine-grained multi-round representation fusion module for further processing; Step 5 includes the following steps: Step 5.1: Use the shallow representation fusion network FuNet(q, e) to semantically fuse the interaction nodes on the query side and the knowledge base side. Know After splicing, it is sent to the fusion layer, and the two are mixed and then divided into Know in, It is a representation interaction node in the middle layer of the query side. is the representation interaction node after semantic fusion at the current layer, MLP[·] represents the multi-layer perceptron, It is a representation interaction node in the middle layer of the knowledge base side. It is the representation interaction node after semantic fusion at the current layer; Step 5.2: On the query side of the dual-tower architecture, the Transformer model is used to update the query representation layer by layer. The representation update of each layer is completed through the self-attention mechanism, which follows the formula: in, represents the representation of the middle layer on the query side, j represents the position of the word in the sentence, l represents the current layer number, [cls]l represents the state of the middle layer representing the fused character, It is the semantic interaction node after semantic fusion at the current layer; Step 5.3: On the knowledge base side of the dual-tower architecture, the attention weights of node m and its adjacent node n are first calculated through the attention mechanism, and the representations of adjacent nodes are weighted summed to obtain the neighbor aggregation representation of node m. Then, a linear transformation is performed using a multi-layer perceptron and a weight matrix, and an updated node representation is generated through the activation function Softmax(·). The graph attention mechanism is used to update the representation of the knowledge base. In each layer of the graph convolutional network GCN, the representation of the current node embedding is input into the graph attention layer, and a pre-fused node for each entity is generated. The node embedding is: Among them, e m and e n denote the representations of node m and node n respectively, MLP(·) denotes a multi-layer perceptron, and W node , W edge and W out represents the weight matrix, d represents the dimension of representation, represents the set of nodes adjacent to node m. The symbol [,] in the formula represents the concatenation operation of the vector, the symbol [·] represents the matrix multiplication, Attention(·) represents the attention mechanism, and r nm Represents the relationship between node m and node n; after updating the knowledge base embedding at the GCN layer, the final subgraph is obtained: Step 5.4: Complete the construction of query representation and knowledge base entity representation in the dual-tower architecture, and input them into the multi-channel dual-tower matching module to evaluate the representation similarity; Step 6 includes the following steps: Step 6.1: Transform the representations for representation similarity evaluation: Where L is the number of interaction layers stacked, τ θ is a hyperparameter that controls the retention rate of original information. Represents the representation vector e x The initial embedding vector of Step 6.2: Using the CompIEx algorithm, define the knowledge base representation similarity evaluation function as: Among them, e h 、e r 、e t They represent the vector representation of the head entity node, the vector representation of the knowledge base relationship, and the vector representation of the tail entity node, respectively. Indicates e t Taking the conjugate, Re(·) represents the real part of the complex vector, The corresponding value is e h 、e r , In the complex representation of the kth dimension, d represents the dimension of the embedding vector; for all existing triple relations (h, r, t), f(h, r, t)>0, if f(h, r, t)<0, it means that there is no such knowledge in the knowledge base; Step 6.3: Construct three parallel representation processing channels, namely the original representation channel, the real part representation information, and the imaginary part representation information; the original representation channel contains all the information of the query, the real part representation channel processes the real part of the complex vector representation, and the imaginary part representation channel processes the imaginary part of the complex vector representation; when evaluating the semantic similarity between the query and the relationship, the similarity evaluation function is: Among them, e h 、e q 、e a They represent the vector representation of the head entity node, the vector representation of the knowledge base query, and the vector representation of the answer entity node, respectively. Indicates e a Take the conjugate, Re(·) represents the real part of the complex vector, Im(·) represents the imaginary part of the complex vector; if f(h, q, a)>0, it means that the answer to the knowledge base query is correct, if f(h, q, a)<0, it means that the answer to the knowledge base query is wrong; Step 6.4: Perform feature interaction at the interaction layer. The original representation channel is used as a complex vector to calculate the dot product, while the vectors of the real channel and the imaginary channel are used as real vectors to calculate the dot product. The representation similarity evaluation scalars calculated by each are integrated by linear regression: F(h,q,a)=f(h,q,a)+θ1·f Re (h,q,a)+θ2·f Im (h,q,a) Among them, h, q, a represent the head entity, query question, and answer entity respectively, f(h, q, a) represents the representation similarity evaluation function of the original representation channel based on the real and imaginary parts of the complex vector, and f Re (h, q, a) represents the similarity evaluation function of the real part channel based on the real part of the complex vector, f Im (h, q, a) represents the representation similarity evaluation function of the imaginary channel based on the imaginary part of the complex vector, θ1 and θ2 are weighting coefficients, and F(h, q, a) represents the representation similarity evaluation function based on the overall information; Step 7: In the answer reasoning stage, for the query sentence input by the user, the relevant answer selection is based on the result of the representation similarity evaluation function F(h, q, a); the value of F(h, q, a) is used as the answer score, and the answer nodes with higher scores are screened out as candidate answers. The higher the F(h, q, a) value, the higher the correlation between the candidate answer node and the query sentence.

2. According to the representation interaction enhanced incomplete knowledge base multi-hop question answering method of claim 1, it also includes optimizing the representation similarity evaluation function in step 6.2, characterized in that: The following steps are involved: Step A1: On the basis of the unchanged objective of the representation similarity evaluation function f(h, q, a) described in step 6.2, an evaluation function for measuring the correlation between the query representation and the knowledge base relationship is added: Among them, h, q, and r represent the head entity, query sentence, and relation respectively. h 、e q 、e r They represent the vector representation of the head entity node, the vector representation of the knowledge base query, and the vector representation of the knowledge base relationship. Indicates e r Taking the conjugate, Re(·) represents the real part of the complex vector, The corresponding value is e h 、e q , The complex number representation in the kth dimension, d represents the dimension of the embedding vector; the value of f(h, q, r) is used to evaluate the matching degree between the triples (h, q, r), the higher the value, the more likely the triples are to exist; Step A2: Add query-related loss items and negative samples, and implement the contrast loss function by selecting strong negative examples and randomly selecting weak negative examples: Loss=L pos +ω1·L neg-hard +ω2·L neg-soft Among them, R pos is a set of positive examples, which is selected from the relationship between the central entity and the answer in the single-hop question answering. nh is a set of strong negative examples, which selects the relationships around the central entity that do not point to the answer entity. ns is a weak negative example set, which selects the uninvolved relations around the central entity. γ represents a fixed boundary parameter used to control the score difference between positive and negative samples. σ1, σ2, and σ3 are activation functions. L pos Represents the loss function of the positive sample, L neg-hard represents the loss function of strong negative examples, L neg-soft represents the loss function of weak negative examples, ω1 and ω2 are hyperparameters that control the loss weights of strong negative examples and weak negative examples, and Loss is the total loss function, which combines the losses of positive samples, strong negative examples, and weak negative examples; Step A3: Continuously optimize the representation similarity evaluation function f(h, q, a) through the Loss function training described in step A2.

3. A computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to claim 1 or 2 when executing the program.

Images