A multi-channel representation interaction enhanced multi-hop question answering method for incomplete knowledge base

Abstract

A multi-channel representation interaction enhanced multi-hop question answering method for incomplete knowledge base is provided, which comprises a fine-grained multi-round representation fusion module and a multi-channel double-tower matching module, and a representation similarity evaluation function is optimized according to the characteristics of knowledge base embedding, aiming at aligning query representation and knowledge base representation through a fine-grained multi-channel representation interaction strategy. The fine-grained multi-round representation fusion module realizes early fusion of knowledge base and query sentence semantics through a representation interaction node, and performs multi-round information interaction in the representation updating process. The multi-channel double-tower matching module uses different semantic extraction matrices to process the representations of the query question and the knowledge base, thereby expanding the information interaction capability.

Description

TECHNICAL FIELD

[0001] The present application relates to natural language processing technology, in particular, to the field of knowledge base multi-hop question answering in natural language processing. BACKGROUND

[0002] The knowledge base intelligent question answering system is committed to meeting the complex information needs of users. It allows users to ask questions in the form of natural language queries and can directly provide accurate answers to users. Unlike traditional knowledge base question answering tasks that only focus on a single triple in the knowledge base, multi-hop question answering tasks in incomplete knowledge bases require multi-hop reasoning on knowledge bases with a large number of missing triples. This reasoning process not only involves more triples, but also lacks guidance for intermediate steps.

[0003] As shown in Figure 1 The current mainstream knowledge base multi-hop question answering method usually maps natural language questions and knowledge base entities to the same low-dimensional space through pre-trained language models and knowledge base embedding models, and then carries out the reasoning process based on a double-tower matching model. Although this method using pre-trained models can obtain the semantics of the question and the knowledge base, the vector representation is usually generated by two non-interactive encoders and decoders. During model training, the two vector representations need to be further interacted to fuse semantic information. However, since the model needs to use the similarity between the two vector representations to guide the reasoning process during training, this late fusion strategy is difficult to produce correct guidance information when the model has not converged, making it difficult for the model to learn effective information in the initial stage.

[0004] In short, the existing double-tower matching model faces significant challenges in aligning queries and knowledge base entities. Although it has an advantage in terms of computational complexity, it has limited information fusion capabilities for the two modalities due to only shallow and one-way interaction between natural language queries and knowledge base entities, especially when fine-tuning the knowledge base and predicting inconsistencies in the knowledge base. In addition, in the information transmission process of the double-tower model, feature information is gradually compressed between layers, and information interaction is only achieved in the final layer, which may result in the loss of original detailed information. Furthermore, due to different representation distributions and training methods, there are certain difficulties in fusing the representations on both sides.

[0005] The present application aims at the above-mentioned problems, and proposes a multi-channel representation interaction model (MCRIM) for improving information integration accuracy and overall performance of the question and answer system, which aims to align the query representation and the knowledge base representation through a fine-grained multi-channel interaction strategy, mainly including a fine-grained multi-round representation fusion module and a multi-channel double tower matching module. In the process of updating the representation, the fine-grained multi-round representation fusion module uses the representation interaction node as the bridge for information transmission, and fuses the knowledge base and the query sentence semantics by adding multi-round representation interaction in the double tower structure, so as to realize the alignment of the semantic spaces of the two; the multi-channel double tower matching module is matched with the current complex vector space feature matching task, and different semantic extraction matrices are designed for the representations on the query side and the knowledge base side in the representation interaction process, so as to expand the information interaction ability of the double tower structure by parallel connection of different semantic features, thereby realizing more abundant semantic extraction and matching. SUMMARY

[0006] The present application aims to enhance the information interaction between the double tower models and promote the effective fusion of the query representation and the knowledge base representation, so as to improve the performance of the model in processing complex queries. The present application introduces a fine-grained multi-round representation fusion module to guide the fusion of the query representation and the knowledge base representation, proposes an optimized representation similarity evaluation function to improve the consistency of the training target and the prediction target, and at the same time, in order to fully capture the similarity between the representations in different feature spaces, the present application introduces a multi-channel matching method, which uses different loss functions on different channels to guide the model to learn the representation similarity at different levels, thereby further improving the accuracy of the knowledge base question and answer task.

[0007] The main innovations of the present application are as follows:

[0008] 1. Enhanced double tower representation interaction: In order to align the semantics and representation information of the pre-trained model, the double tower interaction module after the embedding layer of the knowledge base question and answer model is reconstructed. By introducing the representation interaction node, the attention mechanism is used to dynamically adjust the weight between different representations, so as to ensure that the key information obtains more attention and transmission.

[0009] 2. Optimized representation similarity evaluation function: In order to enhance the ability of the feature matching module to capture the query related entities and relationships, the original representation similarity evaluation method is optimized, which can more accurately measure the semantic similarity between the query and the knowledge base. In addition, a contrastive loss function is introduced to fully utilize the negative examples naturally existing in the knowledge base.

[0010] 3. Multi-channel representation interaction enhancement method: In order to enhance the model's ability to capture features in the complex vector space, multiple feature extraction channels are designed to enhance the model's representation extraction ability for vectors in different feature spaces. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A fine-grained multi-round representation interaction module structure of the present application.

[0012] Figure 2 A basic structure diagram of the double tower model in the knowledge base question answering system.

[0013] Figure 3 A multi-channel double tower matching module structure of the present application. DETAILED DESCRIPTION

[0014] The present application is used to realize early fusion of features in the double tower model and representation matching of multiple feature channels, thereby optimizing the accuracy of the double tower matching module and the reasoning model, including constructing a fine-grained multi-round representation fusion module and a multi-channel double tower matching module, and optimizing the representation similarity evaluation function according to the characteristics of knowledge base embedding.

[0015] The current knowledge base question answering system all adopts a double tower architecture, maps the query and the knowledge base to the same dimensional vector space using different embedding models, and introduces a similarity calculation module at the top of the double tower to measure the relevance between the query representation and the knowledge base entity or relationship representation, so as to select the most matched entity or relationship at the semantic level. This method improves the efficiency of the embedding module, but because the pre-training corpus of the two pre-training models is inconsistent and the training target is different, the semantic gap between the vector representations is large. As shown in the following formula, the semantic gap between the query representation and the knowledge base representation is large, which affects the accuracy of the knowledge base question answering system. Figure 2 As shown, the basic structure of a common double tower model in knowledge base question answering is composed of two independent neural networks, one for query encoding and the other for knowledge base entity and relationship encoding. Each tower works independently to construct representations of entities in the query side and the knowledge base, respectively, and finally matches them by calculating their similarity to find the answer that best matches the query. The left tower embeds each word in the query question into a word vector space and generates a context representation of the question through the neural network of the left tower. These representations are further processed, and finally the vector Q [CLS] representing the features of the query question. This representation captures the global semantic information of the question and is used for matching with entities in the knowledge base. The right tower is used to convert structured information (entities, relationships, etc.) in the knowledge base into vector representations. Specifically, the entities E i , relationships R i , and triples T i in the knowledge graph are embedded into the vector space. Through these embeddings, the system can represent structured data in the knowledge base and generate a global representation for each entity and relationship.

[0016] The core objective of the present application is to enhance the information interaction between the double tower model and promote the effective fusion of query representation and knowledge base representation to improve the performance of the model in processing complex queries. The present application includes constructing a fine-grained multi-round representation fusion module and a multi-channel double tower matching module, and optimizing the representation similarity evaluation function according to the characteristics of knowledge base embedding. The fine-grained multi-round representation fusion module realizes the early fusion of knowledge base and query semantics by introducing interaction nodes, and performs multi-round interaction in the representation updating process. The multi-channel double tower matching module uses different semantic extraction matrices to process and expand the information interaction capability of the representations of the question and the knowledge base. In addition, in order to align the semantics of the question and the knowledge base, the present application also optimizes the representation similarity evaluation function and proposes an evaluation method suitable for the semantics and relationships of the query question. The improvements of the present application based on the double tower architecture include:

[0017] 1. Constructing a fine-grained multi-round representation fusion module

[0018] The objective of the fine-grained multi-round representation fusion module is to interact with the opposite side multiple times as the double tower level rises, thereby preserving the combined features, improving the degree of information fusion, and increasing the sensitivity to the information on the opposite side.

[0019] The module architecture is shown in Figure 1 . The module first adds two additional representation interaction characters, one for the query side and one for the knowledge base side, denoted as q int and e int , and a shallow representation fusion network composed of multiple layers of perception, denoted as the fusion layer FuNet(q, e). When performing query representation, the present application maps the question to a low-dimensional vector space using the pre-trained language model BERT, and uses the [cls] character as the representation fusion character on the query side to obtain the initial representation of the query

[0020] Enc(Q)={w1,…,w n ,[cls],q int}

[0021] where [cls] represents the representation fusion character on the query side, Q={w1,w2,…,w n} represents the query question, and w i represents the constituent words of the query question. Then, the query representation is updated in each layer using the Transformer model, and in the present application, the representation of the intermediate layer on the query side is denoted as where j represents the position of the word in the sentence, and l represents the current layer number. In particular, the intermediate layer state of the representation fusion character is denoted as [cls] l , and the representation update of each layer can be represented by the following formula:

[0022]

[0023] In the process of knowledge base representation, the present application first needs to construct a subgraph related to the center entity of the query question according to the extraction method proposed by the PullNet model, which contains at most 3-hop neighbor entities around the center entity of the query question. The core idea of the PullNet model is to construct a query subgraph through iterative method, and the initial subgraph contains the query question and related entities. In each iteration, the graph convolution network GCN is used to calculate the probability of entity nodes in the query subgraph participating in the next round of iteration, and the entities with probability exceeding the threshold are selected. Then, for each selected entity, relevant sentences and triples in the knowledge graph are retrieved, and the new entities and triples are taken as new nodes, and the connections between the new nodes are taken as new edges, so as to update the problem subgraph.

[0024] G sub = {e1, e2, …, e n , e int}

[0025] Where G sub is a query subgraph, representing a knowledge graph subset related to the query question and the center entity of the query question, e j represents the entity nodes constituting the subgraph, and note that j only represents the number of entities and does not contain position information. As described earlier, the query subgraph G sub additional introduces an interaction node e int , which will extend a link that does not contain semantic information with all nodes in the subgraph. The method proposed by this module uses graph attention mechanism to update the representation of the knowledge base. In each layer of GCN, the representation of the current node is input into the graph attention layer, the information between nodes is propagated, and the pre-fusion node of each entity is generated. The embedding is as follows:

[0026]

[0027] Where e m and e n represent the representations of nodes m and n respectively, MLP(·) represents a multi-layer perceptron, W node and W edge represent weight matrices, d represents the dimension of the representation, represents the set of nodes adjacent to node m, Attention(·) represents the attention mechanism, and r nmdenotes the relationship between node m and node n, the symbol [ ] in the formula denotes the splicing operation of the vector, and the symbol [·] denotes matrix multiplication. Through the attention mechanism, first, the attention weight of node m and its adjacent node n is calculated, in which the node representation and the edge representation are considered at the same time, then the adjacent node representation weighted by attention is summed to obtain the neighbor aggregation representation of node m, and finally the updated node representation is obtained by using the multilayer perception and the weight matrix for linear transformation and activation function operation The introduction of the attention mechanism can dynamically adjust the weight of the features, strengthen the features that need to be paid attention to, and ignore irrelevant information. After updating the query sentence embedding and the knowledge base embedding using the Transformer layer and the GCN layer respectively, the fusion module will perform semantic fusion on the interactive nodes on the query side and the knowledge base side. The module will and After splicing, it is sent to the MLP layer, and after mixing, it is divided into and

[0028]

[0029] wherein, is a semantic interactive node in the intermediate layer on the query side, is a semantic interactive node after semantic fusion in the current layer, and MLP[·] represents a multilayer perception.

[0030] In the fusion layer, in addition to the two interactive nodes to be fused, other nodes do not participate in the fusion layer, which can appropriately reduce the complexity of the double-tower interaction and improve the efficiency of the model. By stacking several identical fusion layers, the representations on the user side and the knowledge base side can have sufficient interaction, improving the correlation between the two representations and thus improving the accuracy of subsequent entity selection.

[0031] 2、Optimize the representation similarity evaluation function

[0032] In the current advanced knowledge base question answering algorithm, the query embedding usually adopts the pre-trained language model BERT series to obtain the query sentence representation, and the knowledge base embedding adopts the ComplEx model or the RotatE model. The vector representation output by the BERT model is a ball in a high-dimensional Euclidean space, and most vectors are concentrated in a ball cone in a specific area. This ball cone structure can be regarded as representing the vector aggregation area of different semantic categories and features, so that the BERT model can effectively distinguish and encode different language information in the whole space. In contrast, the representation output by the ComplEx model and the RotatE model is in a high-dimensional complex space. These models use the special nature of complex numbers to capture more rich and complex semantic relationships, and the characteristics of complex space allow these models to model the real part and the imaginary part respectively, so as to better represent and capture the associated information and structured information in the knowledge base.

[0033] In the double tower architecture, the representation similarity evaluation function usually uses a bilinear function to measure the matching degree between vectors. The bilinear function performs a dot product operation on two feature vectors, and then maps it to a scalar value through a linear mapping, which is directly used in subsequent entity selection. This way is simple and direct, but when the semantic space of the question and the knowledge base and the representation modeling space are inconsistent, directly calculating the dot product or cosine similarity cannot accurately measure the semantic matching degree of the two.

[0034] The ComplEx algorithm for knowledge base embedding representation in the double tower architecture is improved, that is, a query question related loss term is introduced and negative samples are introduced, so as to optimize the performance of the representation similarity evaluation function. Among them, ComplEx is a knowledge graph embedding algorithm, which is used to embed entities and relationships in a knowledge graph into a complex space for knowledge graph reasoning and link prediction tasks. This algorithm can capture more complex interactions between entities and relationships by modeling their representations in a complex space, and supports the modeling of asymmetric relationships. The dot product in the complex space is usually a Hermitian product, and the dot product of two complex vectors u=x u +iy u ,v=x v +iy v is defined as:

[0035]

[0036] where represents the conjugate vector of the complex vector v. The representation similarity evaluation function of the knowledge base is defined as:

[0037]

[0038] where, e h 、er t respectively represent the vector representation of the head entity node, the vector representation of the knowledge base relation and the vector representation of the tail entity node, and the symbol represents taking the conjugate of e t , and Re(·) represents the real part of the complex vector, corresponds to e h , e r , the complex representation in the kth dimension, and d represents the dimension of the embedding vector. For all existing triple relations (h, r, t), f(h, r, t) > 0; if f(h, r, t) < 0, it indicates that there is no such knowledge in the knowledge base.

[0039] When evaluating the semantic similarity between the query and the relation, the representation similarity evaluation function becomes:

[0040]

[0041] where e h , e q , e a respectively represent the vector representation of the head entity node, the vector representation of the knowledge base query and the vector representation of the answer entity node, and the symbol represents taking the conjugate of e a , and Re(·) represents the real part of the complex vector, corresponds to e h , e q , the complex representation in the kth dimension, and d represents the dimension of the embedding vector. For all correct knowledge base queries (h, q, a), f(h, q, a) > 0; if f(h, q, a) < 0, it indicates that the answer of the knowledge base query is wrong. The query question is regarded as a special "relation" between the head entity and the answer entity, and the goal of the representation similarity evaluation function is to optimize the semantic similarity between the query question and the relation. In order to optimize the capture ability of the ComplEx algorithm to the query representation, on the basis of ensuring that the original representation similarity evaluation function target is unchanged, the evaluation function for measuring the relevance of the query representation and the knowledge base relation is introduced:

[0042]

[0043] where h, q, r respectively represent the head entity, the query question and the relation, e h , e q , e r respectively represent the vector representation of the head entity node, the vector representation of the knowledge base query and the vector representation of the knowledge base relation, and the symbol represents taking the conjugate of e r ​Take the conjugate, Re(·) represents the real part of the complex vector, Corresponding to e h , e q , The complex representation in the kth dimension, d represents the dimension of the embedding vector. The value of f(h, q, r) is used to evaluate the matching degree between the triple (h, q, r), the higher the score, the more likely the triple exists. By replacing the answer entity a with the relationship r, the training task and the prediction task are aligned.

[0044] In the knowledge base question answering method proposed in the application, the query and its hidden state of the user need to be compared with multiple candidate answers in the knowledge base, and the most relevant answer is found. The contrast loss function is a loss function for learning similarity measurement. It is based on the following idea: for the correct query-answer pair, their similarity score should be higher; while for the wrong query-answer pair, their similarity score should be lower. The application selects strong negative examples and randomly selects weak negative examples to realize the contrast loss function:

[0045]

[0046] Loss=L pos +ω1·L neg-hard +ω2·L neg-soft

[0047] Where, P pos is the positive example set, which is selected from the relationship between the center entity and the answer in the single-hop question answering; R nh is the strong negative example set, which is selected from the relationship between the center entity and the non-pointing answer entity around the center entity; R ns is the weak negative example set, which is selected from the relationship around the center entity that is not involved, γ represents a fixed boundary parameter for controlling the score difference between positive samples and negative samples, σ1, σ2, σ3 represent the activation function, L pos represents the loss function of the positive sample, L neg-hard represents the loss function of the strong negative example, L neg-soft represents the loss function of the weak negative example, ω1 and ω2 are hyperparameters for controlling the loss weight of strong negative examples and weak negative examples, Loss is the total loss function, which integrates the loss of positive samples, strong negative examples and weak negative examples, and adjusts the balance of positive and negative samples through ω1 and ω2. The application guides the MCRIM model to optimize the representation similarity evaluation function f(h, q, a) through the Loss loss function, and guides the MCRIM model to learn to distinguish different types of negative samples (strong negative examples and weak negative examples) through ω1 and ω2, so that the correct answer can be found more accurately in real applications.

[0048] 3. Constructing a multi-channel double-tower matching module

[0049] In the knowledge base question answering task, the dual tower architecture has the problem of single-channel information congestion and confusion. Even if there is no user preference or other features in the query sentence, the query semantics and the center entity semantics still interfere with each other in the tower. In the current mainstream modeling method, the real part and the imaginary part of the representation respectively model the semantic information at different levels. The previous method usually simply performs linear transformation on the representation to map it to the same dimension, which cannot distinguish the rich semantic information contained in the representation.

[0050] To solve this problem, the present application constructs a multi-channel dual tower matching module. This scheme allows information at different levels to flow upwards through the appropriate channel, thereby reducing information confusion and interference. The present application constructs a multi-channel dual tower model by taking the real part and the imaginary part of the original information as two independent channels, which can capture different features and information and provide more comprehensive and diverse representation. This multi-channel architecture allows information to flow and compress in its own channel, avoiding information congestion and mutual interference. In this way, the multi-channel architecture can improve the performance of the dual tower architecture and improve the signal-to-noise ratio.

[0051] In the multi-channel architecture, each channel c tries to extract useful features s c and suppress noise n c from the input data. Define the signal-to-noise ratio SNR of each feature channel as

[0052]

[0053] where P s,c is the signal power of channel c, and P n,c is the noise power of channel c.

[0054] In the single-channel case, the signal-to-noise ratio is fixed and determined by the design of the channel and the nature of the input data. However, in the multi-channel architecture, the overall signal-to-noise ratio can be improved by properly weighting and integrating the output of each channel. Let the total number of channels be C, and the output of each channel be v c = s c +n c The goal of the model is to find a way to integrate these channel outputs to maximize the overall signal-to-noise ratio. Through weighted averaging, the integrated output can be obtained as follows:

[0055]

[0056] where w c is the weight of the cth channel, satisfying non-negativity and Since the noise in the information is additive white noise, when integrating the information of multiple channels, the power of the desired signal is enhanced, while the power of the noise is only slightly affected or even suppressed. In order to approximately calculate the integrated noise power P noise,output , the noise power of each channel is considered, and it is assumed that they are independent and uniformly distributed, then the integrated noise power is:

[0057]

[0058] where w c is the weight of the cth channel, which is regarded as the frequency of information in the channel, which determines the contribution of the channel in the integration process, P noise,c represents the noise power in channel c. The integrated signal power can be approximated as:

[0059]

[0060] where Cov(s i ,s j ) is the covariance of different channel signals, s k represents the kth channel signal, P signal,c represents the signal power in channel c, and w q represents the weight of the qth channel. According to the SNR formula, if the covariance between different channels is positive, the integrated signal power will increase, which means that the noise is suppressed between channels.

[0061] Each matching channel of the multi-channel double-tower matching module can focus on different levels of features in the query and knowledge, and use different functions to evaluate the similarity or correlation between features at different feature levels, promoting the interaction of features at different levels. In this way, on the one hand, multiple semantic aspects of the association information between the query and the knowledge can be captured, and the matching performance and semantic understanding ability can be improved, and on the other hand, multiple channels can use different functions to calculate the correlation, and the feature capturing ability can be optimized. The representations of the parallel matching channels are aggregated at the interaction layer to comprehensively consider the information of the real part and the imaginary part. Such a multi-channel double-tower model can better capture the features and associations of the original information, and improve the expression ability and performance of the model. The architecture of the multi-channel double-tower matching algorithm is shown in Figure 3 It allows different levels of feature interaction by introducing multiple matching channels, and comprehensively considers the information of different channels in the aggregation process. The method proposed by the multi-channel double-tower matching module of the present application includes three parallel channels, which are the original representation channel, the real part representation information, and the imaginary part representation information. The original representation channel is responsible for the original information reaching the double-tower interaction layer without loss, and the real part and imaginary part representation channels process the information interaction with the opposite real part and imaginary part, respectively. The designs of the channels are described in detail below.

[0062] The original representation channel contains all the information of the query question. In the process of rising in the double tower, with each round of interaction with the opposite side information, the original information of the query question semantics will gradually become more blurred. In order to preserve the original query question semantics, a bridge from the initial representation to the interaction before is additionally designed, that is, the final representation for similarity evaluation:

[0063]

[0064] Where L is the number of layers of the interaction layer stack, τ θ is a hyperparameter that controls the retention rate of the original information, represents the initial embedding vector of the representation vector e x Before introducing the real part channel and the imaginary part channel, first explain the semantic meaning. The representation similarity evaluation function in the form of complex Hermite product is defined as:

[0065]

[0066] Where Im(·) represents the imaginary part of the complex vector. When e q is a pure real vector, f(h,q,a) = f(a,q,h), that is, the relationship satisfies symmetry; when e q is a pure imaginary vector, f(h,q,a) + f(a,q,h) = 0, that is, the relationship satisfies anti-symmetry. Under such settings, the real part channel is specifically designed to handle the real part of the complex vector representation, while the imaginary part channel handles the imaginary part. This separation ensures that the real part and the imaginary part information are independently processed in the subsequent feature representation interaction process. From the overall architecture, although the width of the real part and the imaginary part channel is half of the original complex representation channel width, when they are combined, the original complex representation can be reproduced. It is worth noting that in the process of feature representation update, due to the effect of the fully connected layer, the original complex representation channel may contain cross features of the real part and the imaginary part. In contrast, the real part and the imaginary part channel only retains the feature information within the respective channel, avoiding the cross of the features, thereby ensuring the purity and accuracy of the information processing. In the subsequent feature interaction layer, the original representation channel is calculated as a dot product of complex vectors, while the real part channel and the imaginary part channel are calculated as dot products of real vectors. The calculated representation similarity evaluation scalar is integrated through linear regression:

[0067] F(h,q,a) = f(h,q,a) + θ1·f Re (h,q,a) + θ2·f Im (h,q,a)

[0068] Where h, q, a represent the head entity, query question, and answer entity, respectively, f Re (h,q,a) represents the representation similarity evaluation function based on the real part of the complex vector, and fIm (h, q, a) represents a representation similarity evaluation function based on the imaginary part of the complex vector, and represents a weighting coefficient, and F(h, q, a) represents a representation similarity evaluation function based on the overall information. In the reasoning stage, the final selected answer is based on the result of the representation similarity evaluation function F(h, q, a), and the answer with the highest score is considered to be the most relevant answer.

[0069] The present application evaluates the effectiveness of the proposed model MCRIM on multiple multi-hop knowledge graph question answering benchmark datasets, including MetaQA, WebQSP, and ComplexWebQSP three public datasets for comparative experiments. In addition, in order to verify the effect of the representation similarity evaluation function proposed in the present application, the present application uses the BERT model as the query encoder and the ComplEx model as the knowledge base encoder, respectively, and performs experiments on the MetaQA single-hop dataset, trains and tests the feature matching module. The baseline models used are GraftNet model, PullNet model, KV-Mem model, EmbedKGQA model, LEGO model and GenBoost model.

[0070] The experimental results are shown in Table 1, and the evaluation index used by the present application is the Hit@1 index, and the experimental conclusions are as follows:

[0071] (1) MetaQA dataset: On the MetaQA dataset, the model has some progress on the 1-hop task, slightly lower than the EmbedKGQA method, and the GenBoost+MCRIM model achieves the best performance (63.2%) on the 2-hop task, which is 3% higher than the original GenBoost model, which shows that by introducing MCRIM, the ability of the GenBoost model to handle more complex problems has been significantly improved. On the 3-hop problem, GenBoost+MCRIM also shows the best performance (70.3%), which is 2.2% higher than the original GenBoost model. This further confirms the effectiveness of MCRIM in handling complex problems. On the single-hop problem, GenBoost+MCRIM is slightly inferior to the performance of EmbedKGQA.

[0072] Table 1 Comparison of MCRIM on the remaining 50% tuple dataset (%)

[0073]

[0074] (2) WebQSP dataset: The performance of GenBoost model on WebQSP task is also quite good, and after introducing MCRIM as improvement, the accuracy of GenBoost+MCRIM is further improved to 58.5%, which is increased by 1.9% compared with the original GenBoost model. This shows that the introduction of MCRIM is effective for improving the performance of the model on WebQSP task. By comparing the performance of GenBoost and GenBoost+MCRIM, it can be observed that MCRIM has a positive effect on improving the performance of the model. On WebQSP task, the addition of MCRIM increases the accuracy of the model, which may be because MCRIM can better capture and integrate information in different channels, thus improving the understanding of the model and the accuracy of the answer to the question.

[0075] (3) ComplexWebQSP dataset: Due to the limited number of submissions of CWQ test set, the present application compares the results on the validation set. On the CWQ dataset with 50% missing tuples, EmbedKGQA and GenBoost+MCRIM perform outstandingly on the CWQ dataset, especially EmbedKGQA on 1-hop questions and GenBoost+MCRIM on 2-hop and 3-hop questions, both of which achieve high accuracy. These results show that combining effective model improvement and feature fusion strategy can significantly improve the performance on complex question answering tasks.

[0076] (4) EmbedKGQA enhancement experiment: The present application enhances the EmbedKGQA model and conducts corresponding experiments. On 1-hop questions, the accuracy of EmbedKGQA+MCRIM decreases slightly to 67.8%, which is decreased by 2.8% compared with the original EmbedKGQA model. This may indicate that when dealing with direct, single-hop problems, the introduction of MCRIM does not bring performance improvement, and even may lead to performance degradation due to the increase of model complexity. EmbedKGQA+MCRIM shows good generalization ability on different multi-hop questions. Whether on MetaQA, WebQSP or CWQ dataset, this model can adapt to different types of questions and knowledge graph structures, and provide accurate answers. This shows that the fusion of semantic features and knowledge base structure information is significantly beneficial to knowledge base multi-hop reasoning task.

Claims

1. A multi-hop question answering method for characterizing interactive augmented incomplete knowledge base, characterized in that, Comprising the following steps: Step 1: obtaining the query question input by the user; Step 2: in the query side of the double tower architecture, using the pre-trained language model BERT to map the input query question to a low-dimensional vector space, adding a representation interaction node and extracting the initial representation of the query: Enc(Q) = {w0, w1,..., w n , [cls], q int} wherein [cls] represents a representation fusion character on the query side, w i represents a query vector representation, q int represents a representation interaction node on the query side, Q = {w1, w2,..., w n} represents a query question, w i represents a constituent word of the query question; Step 3: On the knowledge base side of the dual tower architecture, use the PullNet model to construct a query subgraph G related to the query question and the query question center entity sub , the query subgraph G sub is composed of at most three-hop neighbor entities related to the query question center entity: G sub = {e1, e2,..., en} (1) n , e int} wherein G sub represents a query subgraph, e j represents an entity node constituting the query subgraph, e int represents a representation interaction node on the knowledge base side; Step 4: complete the initial construction of the query representation and the knowledge base entity representation in the double tower architecture, input to the fine-grained multi-round representation fusion module for further processing; Step 5, comprising the following steps: Step 5.1: Semantic fusion of the interacting nodes on the query side and the knowledge base side by the shallow characterizing fusion network FuNet(q, e) to obtain and concatenated into the fusion layer, mixed and then divided into wherein, is a representation interaction node at the query side intermediate layer, is a representation interaction node after semantic fusion at the current layer, and MLP[·] represents a multi-layer perception, is a representation interaction node at the knowledge base side intermediate layer, is a representation interaction node after semantic fusion at the current layer. Step 5.2: in the query side of the double tower architecture, the query representation is updated layer by layer using the Transformer model, and each layer of representation update is completed through the self-attention mechanism, which specifically follows the formula: wherein, denotes the representation of the query side intermediate layer, j denotes the position of the word in the sentence, I denotes the current layer, and [cls]I denotes the intermediate layer state of the representation fusion character, is the semantic interaction node after semantic fusion at the current layer; Step 5.3: Knowledge base side of the double-tower architecture, first calculate the attention weight of node m and adjacent node n through the attention mechanism, and then weight the representation of the adjacent node to obtain the neighbor aggregation representation of node m, and then use a multi-layer perception and a weight matrix to linearly transform it, and generate the updated node representation through the activation function Softmax(·) In which the representation update of the knowledge base is carried out by using a graph attention mechanism, in each layer of the graph convolution network GCN, the representation of the current node embedding is input into the graph attention layer, and a pre-fusion node of each entity is generated, and the node embedding is: where e m and e n denote the representations of node m and node n, respectively, MLP(·) denotes a multi-layer perceptron, W node , W edge , and W out denote weight matrices, d denotes the dimension of the representation, denotes the set of nodes adjacent to node m, the symbol [·] in the formula denotes vector concatenation, the symbol [·] denotes matrix multiplication, Attention(·) denotes an attention mechanism, r nm denotes the relationship between node m and node n; and the final subgraph is obtained after updating the knowledge base embedding at the GCN layer: Step 5.4: complete the construction of the query representation and the knowledge base entity representation in the double tower architecture, and input to the multi-channel double tower matching module for representation similarity evaluation; Step 6, comprising the following steps: Step 6.1: transform the representation for which the representation similarity is evaluated: where L is the number of layers of the interaction layer stack, τ θ is a hyperparameter that controls the raw information retention rate, denotes the initial embedding vector of the representation vector e x . Step 6.2: using the CompIEx algorithm, define the representation similarity evaluation function of the knowledge base as: where e h , e r , e t denote the vector representation of the head entity node, the vector representation of the knowledge base relation and the vector representation of the tail entity node respectively, and the symbol denotes the conjugate of e t , and Re(·) denotes the real part of a complex vector, corresponding to e h , e r , is the complex representation in the k-th dimension, and d denotes the dimension of the embedding vector; for all existing triple relations (h, r, t), f(h, r, t) > 0, if f(h, r, t) < 0, it means that there is no such knowledge in the knowledge base; Step 6.3: three parallel representation processing channels are constructed, which are the original representation channel, the real part representation information and the imaginary part representation information; among them, the original representation channel contains all the information of the query question, the real part representation channel processes the real part of the complex vector representation, and the imaginary part representation channel processes the imaginary part of the complex vector representation; when evaluating the semantic similarity between the query and the relationship, the similarity evaluation function is represented as: where e h , e q , e a denote the vector representation of the head entity node, the vector representation of the knowledge base query and the vector representation of the answer entity node, respectively, and the symbol denotes the conjugate of e a , Re(·) denotes the real part of a complex vector and Im(·) denotes the imaginary part of a complex vector; if f(h, q, a) > 0, it indicates that the answer to the knowledge base query is correct, and if f(h, q, a) < 0, it indicates that the answer to the knowledge base query is incorrect; Step 6.4: feature interaction is performed at the interaction layer, the original representation channel is used as a complex vector to calculate the dot product, and the vectors of the real part channel and the imaginary part channel are used as real vectors to calculate the dot product, and the representation similarity evaluation scalars calculated by each are integrated through linear regression: F(h, q, a) = f(h, q, a) + θ1 · f Re (h, q, a) + θ2 · f Im (h, q, a) wherein h, q, a respectively represent head entity, query question, answer entity, f(h, q, a) represents a representation similarity evaluation function based on real part and imaginary part of complex vector of the original representation channel, f Re (h, q, a) represents a representation similarity evaluation function based on real part of complex vector of the real part channel, f Im (h, q, a) represents a representation similarity evaluation function based on imaginary part of complex vector of the imaginary part channel, θ1, θ2 are weighting coefficients, and F(h, q, a) represents a representation similarity evaluation function based on overall information. Step 7: in the answer reasoning stage, for the query question input by the user, the related answer selection is based on the result of the representation similarity evaluation function F(h, q, a); the value of F(h, q, a) is used as the answer score, and the answer nodes with high scores are selected as the candidate answers, and the higher the value of F(h, q, a), the higher the relevance of the candidate answer node to the query question.

2. The interactive enhanced representation of an incomplete knowledge base multi-hop question answering method according to claim 1, further comprising optimizing the representation similarity evaluation function in step 6.2, characterized in that, Comprising the following steps: Step A1: on the basis of the representation similarity evaluation function f(h, q, a) described in step 6.2, an evaluation function for measuring the relevance of the query representation and the knowledge base relationship is added: wherein h, q, r represent head entity, query question, relation respectively, e h , e q , e r represent vector representation of head entity node, vector representation of knowledge base query, vector representation of knowledge base relation respectively, symbol represents taking conjugate of e r , Re(·) represents real part of complex vector, corresponding to e h , e q , complex representation in the kth dimension, d represents dimension of embedding vector; value of f(h, q, r) is used to evaluate matching degree between triplets (h, q, r), the higher the value, the more likely the triplet exists; Step A2: add query question related loss term and negative sample, and realize the contrast loss function by selecting strong negative example and randomly selecting weak negative example: Loss = L pos + ω1 · L neg-hard + ω2 · L neg-soft wherein R pos is the set of positive examples, selected from the relations between the center entity and the answer in the single-hop question answering, R nh is the set of strong negative examples, selected from the relations of non-pointing answer entities around the center entity, R ns is the set of weak negative examples, selected from the relations not involving the center entity, γ represents a fixed boundary parameter for controlling the score difference between positive samples and negative samples, σ1, σ2, σ3 are activation functions, L pos represents the loss function of the positive sample, L neg-hard represents the loss function of the strong negative example, L neg-soft represents the loss function of the weak negative example, ω1 and ω2 are hyperparameters for controlling the loss weights of the strong negative example and the weak negative example, and Loss is the total loss function, which comprehensively considers the losses of the positive sample, the strong negative example, and the weak negative example. Step A3: optimize the representation similarity evaluation function f(h, q, a) by the total loss function Loss described in step A2.

3. A computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method according to claim 1 or 2 when executing the program.

Images