A Secure Multi-hop Question Answering Method and System Combining Internal and External Knowledge Graphs

Abstract

The present invention provides a secure multi-hop question answering method and system that combines internal and external knowledge graphs, belonging to the field of multi-hop question answering. It performs joint training based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem; based on the joint representation learning subsystem, obtains entity representations and relationship representations; based on the internal knowledge graph, constructs a question encoding and decoding subsystem through a language encoding model and a sequence encoding model; obtains multi-hop inference paths; based on the multi-hop inference paths, and the entity representations and the relationship representations obtained by the joint representation learning subsystem, constructs a secure knowledge retrieval subsystem; when a multi-hop question is input from the private end to the secure knowledge retrieval subsystem, obtains the correct answer corresponding to the multi-hop question. Through the method provided by the present invention, not only can question answering reasoning be completed on multiple knowledge graphs while ensuring privacy, but also the query efficiency can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of multi-hop question answering, and particularly relates to a secure multi-hop question answering method and system combining internal and external knowledge graphs. Background Art

[0002] A knowledge graph stores a vast amount of information in a structured form through triples, expressing Internet data as an objectively recognizable semantic representation, and has powerful semantic expression capabilities. A question answering system based on a knowledge graph utilizes its rich structured information to give accurate answers by understanding users' natural language questions, showing important application values in multiple fields such as medical care, finance, and e-commerce. In actual application scenarios, users tend to express complex multi-hop questions, so the research on multi-hop question answering is an important task.

[0003] Privately constructed knowledge graphs are often incomplete because building a knowledge graph itself is a highly challenging task. For example, ontology design and knowledge annotation require expert knowledge as support, and training a knowledge extraction model incurs certain costs. Therefore, when reasoning about users' complex questions, it is easily restricted. For example, when the correct reasoning path for locating an answer or the answer expected by the user is not included in a single knowledge graph, it is difficult to obtain the correct answer relying on a single knowledge graph. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a secure multi-hop question answering method, system, electronic device, and storage medium combining internal and external knowledge graphs to overcome or at least partially solve the above problems.

[0005] In the first aspect of the embodiments of the present invention, a secure multi-hop question answering method combining internal and external knowledge graphs is provided. The method includes:

[0006] Jointly train based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem;

[0007] Based on the joint representation learning subsystem, obtain entity representations and relationship representations; wherein, the entity representations include internal entity representations of the internal knowledge graph and external entity representations of the external knowledge graph, and the relationship representations include internal relationship representations of the internal knowledge graph and external relationship representations of the external knowledge graph;

[0008] Based on the internal knowledge graph, construct a question encoding and decoding subsystem through a language encoding model and a sequence encoding model;

[0009] Based on the question encoding and decoding subsystem, obtain a multi-hop reasoning path;

[0010] Construct a security knowledge retrieval subsystem based on the multi-hop reasoning path and the entity representation and relationship representation obtained by the joint representation learning subsystem;

[0011] When a multi-hop question is input from the private end to the security knowledge retrieval subsystem, obtain the correct answer corresponding to the multi-hop question.

[0012] Optionally, the constructing the joint representation learning subsystem by jointly training based on the internal knowledge graph and the external knowledge graph includes:

[0013] Obtain the internal relationship list of the internal knowledge graph and the external relationship list of the external knowledge graph;

[0014] Based on the internal relationship list and the external relationship list, obtain the common relationship list;

[0015] Based on the common relationship list, construct a joint representation learning subsystem through an orthogonal Procrustes problem.

[0016] Optionally, the obtaining the entity representation and relationship representation based on the joint representation learning subsystem includes:

[0017] Based on the joint representation learning subsystem, construct the constraint conditions of the entity and the common relationship;

[0018] Based on the constraint conditions, obtain the relationship generation factor;

[0019] Based on the relationship generation factor, obtain the calculated relationship embedding matrix;

[0020] Based on the relationship embedding matrix and the constraint conditions for training, obtain the entity representation and the relationship representation.

[0021] Optionally, the constructing the question encoding and decoding subsystem based on the internal knowledge graph through a language encoding model and a sequence encoding model includes:

[0022] Based on the internal knowledge graph, obtain the question and answer training data;

[0023] Based on the question and answer training data, extract natural language questions and corresponding topic entities and answers;

[0024] Based on the natural language questions, obtain the prediction path of the natural language questions through the language encoding model and the sequence encoding model;

[0025] Based on the subject entity and the answer, using the shortest path in the internal knowledge graph as a weak supervision label, generate a relationship sequence between the subject entity and the answer; wherein, in the case where there are multiple identical shortest paths in the internal knowledge graph, use the shortest path with the highest occurrence frequency as the weak supervision label;

[0026] Based on the predicted path and the relationship sequence, update the language encoding model and the sequence encoding model;

[0027] Based on the updated language encoding model and sequence encoding model, construct the question encoding and decoding subsystem.

[0028] Optionally, based on the multi-hop inference path, and the entity representation and relationship representation obtained by the joint representation learning subsystem, construct a security knowledge retrieval subsystem, including:

[0029] Perform product quantization compression on the entity representation and the relationship representation to obtain a product quantization codebook;

[0030] Based on the entity representation, obtain an inverted index corresponding to the entity representation;

[0031] Based on the inverted index, obtain a set of vector list corresponding to the inverted index;

[0032] Based on the multi-hop inference path, as well as the set of vector lists, the inverted index, and the product quantization codebook, construct the security knowledge retrieval subsystem.

[0033] Optionally, when a multi-hop question is input from the private end to the security knowledge retrieval subsystem, the security knowledge retrieval subsystem outputs the correct answer corresponding to the multi-hop question, including:

[0034] Based on the security knowledge retrieval subsystem, convert the multi-hop question into a plurality of query vectors;

[0035] By performing distance calculation and asymmetric distance calculation on the plurality of query vectors, obtain the nearest neighbor entity embedding representations corresponding to the plurality of query vectors;

[0036] Based on the nearest neighbor entity embedding representation, obtain the correct answer.

[0037] In the second aspect of the embodiments of the present invention, a secure multi-hop question answering system combining internal and external knowledge graphs is provided, and the system includes:

[0038] A first construction module, configured to perform joint training according to an internal knowledge graph and an external knowledge graph to construct a joint representation learning subsystem;

[0039] The first acquisition module is used to acquire entity representations and relationship representations based on the joint representation learning subsystem; wherein, the entity representations include internal entity representations of the internal knowledge graph and external entity representations of the external knowledge graph, and the relationship representations include internal relationship representations of the internal knowledge graph and external relationship representations of the external knowledge graph;

[0040] The second construction module is used to construct a question encoding and decoding subsystem based on the internal knowledge graph through a language encoding model and a sequence encoding model;

[0041] The second acquisition module is used to acquire multi-hop inference paths based on the question encoding and decoding subsystem;

[0042] The third construction module is used to construct a security knowledge retrieval subsystem based on the multi-hop inference paths and the entity representations and relationship representations obtained by the joint representation learning subsystem;

[0043] The third acquisition module is used to acquire the correct answer corresponding to the multi-hop question when a multi-hop question is input from the private side to the security knowledge retrieval subsystem.

[0044] Optionally, the first construction module includes:

[0045] The first construction sub-module is used to acquire the internal relationship list of the internal knowledge graph and the external relationship list of the external knowledge graph;

[0046] Based on the internal relationship list and the external relationship list, acquire a public relationship list;

[0047] Based on the public relationship list, construct a joint representation learning subsystem through an orthogonal Procrustes problem.

[0048] Optionally, the first acquisition module includes:

[0049] The first acquisition sub-module is used to construct constraint conditions for entities and public relationships based on the joint representation learning subsystem;

[0050] Based on the constraint conditions, acquire relationship generation factors;

[0051] Based on the relationship generation factors, acquire a calculated relationship embedding matrix;

[0052] Based on the relationship embedding matrix and the constraint conditions for training, acquire the entity representations and the relationship representations.

[0053] Optionally, the second construction module includes:

[0054] Based on the internal knowledge graph, acquire question and answer training data;

[0055] Extract natural language questions, corresponding topic entities, and answers based on the Q&A training data;

[0056] Based on the natural language questions, obtain the prediction paths of the natural language questions through the language encoding model and the sequence encoding model;

[0057] Based on the topic entity and the answer, generate a relationship sequence between the topic entity and the answer using the shortest path in the internal knowledge graph as a weak supervision label; where, in the case of multiple identical shortest paths in the internal knowledge graph, use the shortest path with the highest occurrence frequency as the weak supervision label;

[0058] Update the language encoding model and the sequence encoding model based on the prediction path and the relationship sequence;

[0059] Construct the question encoding and decoding subsystem based on the updated language encoding model and sequence encoding model.

[0060] Optionally, the third construction module includes:

[0061] A third construction sub-module for performing product quantization compression on the entity representation and the relationship representation to obtain a product quantization codebook;

[0062] Based on the entity representation, obtain the inverted index corresponding to the entity representation;

[0063] Based on the inverted index, obtain the set of vector list corresponding to the inverted index;

[0064] Construct the security knowledge retrieval subsystem based on the multi-hop inference path, as well as the vector list set, the inverted index, and the product quantization codebook.

[0065] Optionally, the third acquisition module includes:

[0066] A third acquisition sub-module for converting the multi-hop question into multiple query vectors based on the security knowledge retrieval subsystem;

[0067] Obtain the nearest neighbor entity embedding representations corresponding to the multiple query vectors by performing distance calculation and asymmetric distance calculation on the multiple query vectors;

[0068] Obtain the correct answer based on the nearest neighbor entity embedding representation.

[0069] In a third aspect of the embodiments of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, the steps in the secure multi-hop question answering that combines internal and external knowledge graphs as described in the first aspect of the embodiments of the present invention are implemented.

[0070] In a fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the secure multi-hop question answering that combines internal and external knowledge graphs as described in the first aspect of the embodiments of the present invention are implemented.

[0071] Through the method provided by the present invention, first, joint training is performed based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem, and entity representations and relationship representations are obtained. The internal knowledge graph and the external knowledge graph can be combined to expand the retrieval scope of multi-hop question answers, improve the accuracy and precision. Then, through the internal knowledge graph, a question encoding and decoding subsystem is constructed through a language encoding model and a sequence encoding model to obtain a multi-hop reasoning path. Further, through the multi-hop reasoning path, as well as the entity representations and relationship representations, a secure knowledge retrieval subsystem is constructed; the privacy of the private end can be guaranteed. Finally, when a multi-hop question is input from the private end to the secure knowledge retrieval subsystem, the correct answer corresponding to the multi-hop question is obtained.

[0072] Through the method provided by the present invention, not only can knowledge graph question answering reasoning be performed on multiple knowledge graphs, improving the accuracy of the knowledge graph question answering system, but also the query efficiency can be improved during the query process, and the privacy of the query data and the query results of the private end can be protected. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0074] Figure 1 is a flowchart of the steps of a secure multi-hop question answering method that combines internal and external knowledge graphs provided by the embodiments of the present invention;

[0075] Figure 2 is a schematic structural diagram of a system of a secure multi-hop question answering method that combines internal and external knowledge graphs provided by the embodiments of the present invention;

[0076] Figure 3 is a schematic diagram of a joint representation learning subsystem provided by the embodiments of the present invention;

[0077] Figure 4 It is a schematic diagram of a problem encoding and decoding subsystem provided by an embodiment of the present invention;

[0078] Figure 5 It is a schematic diagram of a security knowledge retrieval subsystem provided by an embodiment of the present invention;

[0079] Figure 6 It is a schematic diagram of a security multi-hop question answering system that combines internal and external knowledge graphs provided by an embodiment of the present invention;

[0080] Figure 7 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0081] The exemplary embodiments of the present invention will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present invention. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present invention can be more thoroughly understood and the scope of the present invention can be fully conveyed to those skilled in the art.

[0082] Embodiment 1

[0083] Refer to Figure 1 , Figure 1 It is a flowchart of the steps of a security multi-hop question answering method that combines internal and external knowledge graphs provided by an embodiment of the present invention; the method step flow includes:

[0084] Step S101: Perform joint training based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem.

[0085] In this embodiment, mainly aiming at the problem of the incompleteness of the internal knowledge graph, there are two ways in the related art to solve this problem. The first is from the perspective of the data source. By means of entity linking, documents associated with the entities in the knowledge graph are collected to establish an entity-document heterogeneous graph, thereby enhancing the data source and introducing new connections between entities. The second is from the perspective of the model. Utilizing the link prediction ability of knowledge graph embedding, nodes and relationships are represented by knowledge graph embedding, and the questions raised by users, the topic entities included in the questions, and the answers are modeled into triplets for embedding representation to make up for the lack of relationships in the knowledge graph. However, in the related art, only the situation of relationship loss is considered, ignoring the possibility of entity loss. In this embodiment, an open-source knowledge graph is introduced as data supplement, that is, an external knowledge graph, to solve the situation of entity loss. Considering the huge scale of the open-source knowledge graph, it is infeasible to download it to the local to expand the local knowledge base. Instead, it is retrieved according to the query requirements. First, based on the user's question, a set of entities related to the question is retrieved locally, and then each entity element of this entity set is used as a query to input to the open-source party to obtain the corresponding neighbor entities of the query entity as supplement. Then, all candidate entities are sorted. Therefore, a joint representation learning subsystem is constructed through joint training of the internal knowledge graph and the external knowledge graph. The joint representation learning subsystem aims to learn the joint representation of the private internal knowledge graph entity data and the public external knowledge graph entity data while protecting the privacy of the private internal knowledge graph entity data, laying a foundation for the question encoding and decoding subsystem. This part uniformly updates the representation of the relationship through the method of federated learning. Additionally, based on the orthogonal Procrustes problem, the representation of the knowledge graph is learned for subsequent knowledge search.

[0086] Step S102: Based on the joint representation learning subsystem, obtain entity representations and relationship representations; wherein, the entity representations include internal entity representations of the internal knowledge graph and external entity representations of the external knowledge graph, and the relationship representations include internal relationship representations of the internal knowledge graph and external relationship representations of the external knowledge graph.

[0087] In this embodiment, through the joint representation learning subsystem, entity representations and relationship representations are obtained. Since the joint representation learning subsystem is a joint representation learning subsystem jointly constructed by the internal knowledge graph and the external knowledge graph, and it is a joint representation learning subsystem in which the internal knowledge graph and the external knowledge graph have a common relationship. Refer to Figure 2 , Figure 2 is a schematic structural diagram of a system for a secure multi-hop question-answering method combining internal and external knowledge graphs provided by an embodiment of the present invention. By putting the entities and relationships in the internal knowledge graph and the entities and relationships in the external knowledge graph into the joint representation learning subsystem for training, entity representations and relationship representations can be obtained, that isFigure 2 The outputs 1-4 in

[0088] Step S103: Based on the internal knowledge graph, construct a question encoding and decoding subsystem through a language encoding model and a sequence encoding model.

[0089] In this embodiment, through the internal knowledge graph, a question encoding and decoding subsystem is constructed through a language encoding model and a sequence encoding model. The natural language question raised by the private end is encoded by the large-scale language model BERT to obtain the corresponding vector representation. Then, the inference path corresponding to the question is used as the supervision signal, that is, the shortest path between the question and the answer in the internal knowledge graph is used to generate weak supervision labels. The gradient is calculated through the binary cross-entropy loss between the predicted sequence and the pseudo-label by a recurrent neural network, and the parameters of the encoding model and the decoding model are updated. The updated language encoding model and sequence encoding model are used to construct a question encoding and decoding subsystem.

[0090] Step S104: Based on the question encoding and decoding subsystem, obtain a multi-hop inference path.

[0091] In this embodiment, through the question encoding and decoding subsystem, a multi-hop inference path is obtained. Combining Figure 2 , through the question decoding and encoding subsystem, the natural language question can be encoded into a vector representation, and then a recurrent neural network is used to decode the vector representation of the question into the corresponding inference path, that is, the multi-hop inference path. The multi-hop inference path means that there are multiple relationships between the head entity and the tail entity, rather than just the head entity directly obtaining the tail entity through one relationship. For example, the multi-hop question is how tall is Yao Ming's wife? The first relationship is to determine that Yao Ming's wife is Ye Li, and the second relationship is that Ye Li's height is 1.9 meters. Then there will be two relationships, and the correct tail entity corresponding to the head entity can be obtained through two relationships. Here, wife and height are both relationships, which can be understood as a two-hop relationship. The first-hop relationship is wife, and the second-hop relationship is height.

[0092] Step S105: Based on the multi-hop inference path, and the entity representation and relationship representation obtained by the joint representation learning subsystem, construct a security knowledge retrieval subsystem.

[0093] In this embodiment, a security knowledge retrieval subsystem is constructed through multi-hop inference paths and the entity relationships and relationship representations obtained by the joint representation learning subsystem. This is because when retrieving answers from the internal knowledge graph to the external knowledge graph, the head entity included in the retrieved question may expose the user's true thoughts and there is a possibility of disclosure. Therefore, a security knowledge retrieval subsystem needs to be constructed to play a role in confidentiality. The security knowledge retrieval subsystem in this embodiment is a semi-honest adversary model. Although in the semi-honest adversary security model, the participating parties will correctly execute the protocol without violating it, they may be monitored by malicious attackers to obtain their own input and output during the protocol execution and the information obtained during the protocol operation. Therefore, in this embodiment, starting from the topic entity included in the question, the inference path output by the question encoding and decoding subsystem is used to search for existing triples or potential triples using the entity and relationship representations obtained by the joint representation learning subsystem. This subsystem reduces the triple query of the knowledge graph to a nearest neighbor search problem based on the Euclidean squared distance, and significantly improves the search efficiency by combining product quantization. During the search for the public-end entity set, it is ensured that the distance metrics between the multiple query vectors of the private end and the public-end entity list and the multiple query vectors cannot be obtained by the public end to ensure the query privacy of the private end. To construct the security knowledge retrieval subsystem, it should be noted that in this embodiment, the private end corresponds to the internal knowledge graph end, and the public end corresponds to the external knowledge graph end. Similarly, the private end and the public end in other places are referred to here and will not be elaborated further.

[0094] Step S106: When a multi-hop question is input from the private end to the security knowledge retrieval subsystem, obtain the correct answer corresponding to the multi-hop question.

[0095] In this embodiment, when a multi-hop question is output from the private end to the security knowledge retrieval subsystem, the security knowledge retrieval will output the corresponding correct answer. For example, when inputting "What is the height of Yao Ming's wife?" to the security knowledge retrieval subsystem, the security knowledge retrieval subsystem will output 1.9 meters.

[0096] In one embodiment, the construction of the joint representation learning subsystem by jointly training the internal knowledge graph and the external knowledge graph includes: obtaining the internal relationship list of the internal knowledge graph and the external relationship list of the external knowledge graph; obtaining the common relationship list based on the internal relationship list and the external relationship list; and constructing the joint representation learning subsystem through the orthogonal Procrustes problem based on the common relationship list.

[0097] In this embodiment, the purpose of constructing a joint representation learning subsystem is to input questions from the private end, and to unite the external knowledge graph through the joint representation learning subsystem, so that more accurate or more complete answers can be found in the external knowledge graph than in the internal knowledge graph. The joint representation learning subsystem is constructed to serve the private end, so it is necessary to obtain the internal relationship list of the internal knowledge graph and the external relationship list of the external knowledge graph. Through the internal relationship list and the external relationship list, a public relationship list is obtained, that is, a public relationship list with the same relationship in the internal knowledge graph and the external knowledge graph. Based on the public relationship list, a joint representation learning subsystem is constructed through the orthogonal Prucker problem.

[0098] For example, the internal knowledge graph held by the private end and external knowledge graphs held by the public side Answer the natural language question q raised by the private end, where the answer to the question may be located in the internal knowledge graph or the external knowledge graph, where ε in ≠ε ex ; ε is the entity set, is a relationship set, that is, a list of public relationships, for The private and public ends use orthogonal Pluck analysis to model entities and relationships, that is, a joint representation learning subsystem is constructed through the orthogonal Pluck problem.

[0099] In one embodiment, the obtaining of entity representation and relationship representation based on the joint representation learning subsystem includes: constructing constraints on entities and public relationships based on the joint representation learning subsystem; obtaining relationship generation factors based on the constraints; obtaining a calculated relationship embedding matrix based on the relationship generation factors; and training based on the relationship embedding matrix and the constraints to obtain the entity representation and the relationship representation.

[0100] In this embodiment, referring to Figure 3 , Figure 3 is a schematic diagram of a joint representation learning subsystem provided by an embodiment of the present invention. According to the definition of the orthogonal Prucker problem, if the head entity representation H, the relationship representation R, and the tail entity representation T should satisfy the following constraints: The private and public ends divide the triples locally by relationship, and for each relationship The corresponding entity matrix H i and T i Calculate the generation factor H of the relationship i T Ti The private end and the public end respectively generate the generated factors calculated by them according to the relationship and upload them to the server end. The server end calculates the relationship embedding matrix according to the generated factors: where generate() is the relationship embedding generation function, t is the current update round, and the superscripts in and ex represent the entity embeddings belonging to the private end or the public end. Then the server distributes the relationship embedding matrices belonging to the two parties respectively. The private end and the public end use the updated relationship matrices and update the representations of the entities. Repeat this process until the training converges, and the private end and the public end will obtain the final knowledge graph embedding representations. Repeat this process until the constraint conditions of the private end and the public end no longer change and the training converges, and the private end and the public end will obtain the final knowledge graph embedding representations and where H * , T * represent the final embedding representation matrices of the head entity and the tail entity, R * is the final embedding representation matrix of the relationship. The subscripts in and ex respectively refer to the private end and the public end. For the same relationship r i ∈R, it is necessary to ensure that the relationship embedding matrix R i simultaneously satisfies and According to the properties of matrix operations, the solution to this problem can be equivalently transformed into performing a singular value decomposition on , that is where t is the current update round, and the superscripts in and ex represent the entity embeddings belonging to the private end or the public end. Then the server distributes the relationship embedding matrices belonging to the two parties respectively. The private end and the public end use the updated relationship matrices and update the representations of the entities. Repeat this process until the training converges. Combining Figure 3 , Output 1 and Output 3 are relationship representations, and Output 2 and Output 4 are entity representations.

[0101] In one embodiment, based on the internal knowledge graph, a question encoding and decoding subsystem is constructed through a language encoding model and a sequence encoding model, including: obtaining Q&A training data based on the internal knowledge graph; extracting natural language questions and corresponding topic entities and answers based on the Q&A training data; obtaining the predicted path of the natural language question through the language encoding model and the sequence encoding model based on the natural language question; generating a relationship sequence between the topic entity and the answer with the shortest path in the internal knowledge graph as the weak supervision label based on the topic entity and the answer; where, in the case of multiple identical shortest paths in the internal knowledge graph, the shortest path with the highest occurrence frequency is used as the weak supervision label; updating the language encoding model and the sequence encoding model based on the predicted path and the relationship sequence; constructing the question encoding and decoding subsystem based on the updated language encoding model and sequence encoding model.

[0102] In this embodiment, according to the internal knowledge graph, Q&A training data is obtained. The Q&A training data can be directly obtained from other channels, as long as the entities or relationships included in the Q&A training data exist in the internal knowledge graph. It can also be Q&A training data generated through some existing software according to the entities and relationships existing in the internal knowledge graph. The Q&A training data includes natural language questions, and the Q&A data and the internal knowledge graph are two parallel inputs. The Q&A training data itself already contains natural language questions and their topic entities and answers, so there is no need to extract them. In fact, the logic here should be to label the weak supervision label in the internal knowledge graph according to the topic entities and answers in the Q&A data. Refer to Figure 4 , Figure 4 is a schematic diagram of a question encoding and decoding subsystem provided by an embodiment of the present invention; training process: The natural language question q provided by the private end is encoded by the large-scale language model BERT to obtain the corresponding vector representation v q , and then using the inference path corresponding to the question q as the supervision signal, v q is output through a Long Short-Term Memory (LSTM) network to obtain the corresponding relationship sequence. The initial input of the LSTM model is h0 = FFN h (v q ), c0 = FFN c (v q ), x0 = v q . The hidden layer vector h t at the t-th time step is h t-1 , c t-1 , x t-1 ) = LSTM(h t-1 , c t-1 , x t-1 ), and the corresponding output vector a t= softmax(MLP(h t ))), where the dimension size of a t is equal to the size of the relation list , meaning that the output vector at each time step t predicts the t-th relation r in the relation sequence t in a t as the dimension index value with the highest score in the dimension.

[0103] Inference process: Consistent with the forward propagation process of the training process, the natural language question q proposed by the private end is encoded by the large-scale language model BERT to obtain the corresponding vector representation v q , and then using the inference path corresponding to the question q as the supervision signal, v q is output through the LSTM to obtain the corresponding relation sequence.

[0104] Finally, when a corresponding inference path PATH = (r1, r2,..., r n ) is generated as the pseudo-label for each natural language question q, the gradient is calculated through the binary cross-entropy loss between the prediction sequence and the pseudo-label, and the parameters of the language encoding model and the sequence decoding model are updated. Through the updated language encoding model and sequence decoding model, a question encoding and decoding subsystem is constructed.

[0105] In one embodiment, based on the multi-hop inference path, and the entity representation and the relation representation obtained by the joint representation learning subsystem, a security knowledge retrieval subsystem is constructed, including: performing product quantization compression on the entity representation and the relation representation to obtain a product quantization codebook; obtaining an inverted index corresponding to the entity representation based on the entity representation; obtaining a set of vector list corresponding to the inverted index based on the inverted index; constructing the security knowledge retrieval subsystem based on the multi-hop inference path, the set of vector lists, the inverted index, and the product quantization codebook.

[0106] In this embodiment, referring to Figure 5 , Figure 5 is a schematic diagram of a security knowledge retrieval subsystem provided by an embodiment of the present invention; constructing a security knowledge retrieval subsystem, respectively performing quantization compression on the entity embeddings of the knowledge graphs of the public end and the private end, and converting the triple query problem of finding the tail entity given the known head entity and relation into a nearest neighbor search problem of finding the nearest neighbor vector given multiple query vectors in the Euclidean space, so as to achieve efficient triple retrieval, and at the same time, the search process is performed on the private end to ensure the query privacy of the private end.

[0107] Exemplarily, taking the public-end entity embedding set as an example, product quantization compression is performed on the entity embedding set. First, for each entity's embedding representation, the nearest coarse-grained clustering center (obtained using the K-means algorithm) is calculated, and then, for the entity's inverted index value, the difference r(e) between the embedding representation of entity e and the corresponding clustering center representation c i is calculated, and then the difference vector is product quantized to obtain the quantized vector q p (r(e)). The quantized encoding of the vector q p (r(e)) and the id of entity e are stored in the vector list with index value i in the inverted index . A secure knowledge retrieval subsystem is constructed. According to this method, this operation is performed on each entity embedding representation in the public-end entity embedding set, so that the entire embedding set is quantized and encoded. After product quantization, the data storage amount is greatly reduced. The public-end sends the quantized codebook and inverted index to the private-end. The private-end uses the clustering center representation of the inverted index to calculate the nearest clustering center j to multiple query vectors x, and then calculates the difference r(x) between x and the clustering center representation c j . The private-end retrieves from the public-end the vector list with inverted index value j . Then the public-end sends the corresponding vector list and the product quantization codebook to the private-end. The private-end uses the list and the product quantization codebook to calculate the vector closest to r(x), and calculates the entity closest to the query vector according to the product quantization codebook

[0108] In one embodiment, when a multi-hop question is input from the private-end to the secure knowledge retrieval subsystem, the secure knowledge retrieval subsystem outputs the correct answer corresponding to the multi-hop question, including: based on the secure knowledge retrieval subsystem, converting the multi-hop question into multiple query vectors; obtaining the nearest neighbor entity embedding representations corresponding to the multiple query vectors by performing distance calculation and asymmetric distance calculation on the multiple query vectors; and obtaining the correct answer based on the nearest neighbor entity embedding representations

[0109] In this embodiment, to obtain the correct answer for a multi-hop question, the multi-hop question is sequentially split into single-hop questions in order for calculation. Each single-hop question corresponds to a query vector, and the multi-hop question is queried for the answer in a recursive order. The correct answer is obtained through the entity embedding representation of the last-hop question. Exemplarily, assume the multi-hop question is "What is the height of Yao Ming's wife?" This question is a two-hop question. According to the method provided by the present invention, first, it will be confirmed that the first-hop question is "Who is Yao Ming's wife?" Then the first-hop question will be converted into a query vector, and an answer of Ye Li will be obtained. After obtaining the answer to the first hop, the second-hop question that appears is "What is Ye Li's height?" At this time, the second-hop question will be converted into a query vector and continue to be queried, and the answer to the second hop is obtained as 1.9 meters.

[0110] Combined with Figure 5 , when a multi-hop question is input from the private end to the security knowledge retrieval subsystem and queried to the public end, the security knowledge retrieval subsystem will convert the multi-hop question into multiple query vectors. If the multi-hop question is a one-hop question, then there is only one query vector x = hR j , (that is, at this time, the head entity is h and the relationship is r u ), both the head entity and the relationship belong to the content in the final relationship representation and entity representation of the relationship joint representation learning subsystem. The public end sends the inverted index to the private end, and the private end uses the cluster center representation of the inverted index to calculate the cluster center j closest to the query vector x. Then calculate the difference r(x) between x and the cluster center representation c j . The private end retrieves from the public end the vector list with the inverted index value of j Then the public end sends the corresponding vector list and the product quantization codebook to the private end, and the private end uses the list and the product quantization codebook to calculate the vector closest to r(x), and the corresponding entity is used as the final query result. If the multi-hop question is a two-hop question, then two relationships r1 and r2 are extracted from the multi-hop question, and the subject entity is e1. First, use E1*R1 (representing the vector representations of e1 and r1 respectively) as the first query vector to find the most suitable intermediate vector e2, and then use E2*R2 as the second query vector, which is also the last query vector, to find the most suitable vector denoted as E3. At this time, it is found that the relationship sequences r1 and r2 corresponding to the multi-hop question have all been used, indicating that the query has ended. Then at this time, E3 is considered the correct answer sought by the user. Therefore, for a natural language question q, given its subject entity e topic and the inference path PATH = (r1, r2,..., r n ), the security knowledge retrieval subsystem is used iteratively. From e topic and r1, find e1 such that the triple (e topic, r1, e1) exists in the internal knowledge graph or the external knowledge graph, and so on. Finally, (e n-1 , r n , e n ) is retrieved, where e n is regarded as the answer to the question q. That is Figure 5 the nearest neighbor entity embedding finally output in

[0111] Since the knowledge graph query technology of the RDF database or the graph database can only retrieve the triples explicitly represented in the knowledge graph, due to the incompleteness of the knowledge graph, the traditional knowledge retrieval will lead to a decrease in the answerability of complex questions. The present invention utilizes the characteristic of the knowledge graph to implicitly model knowledge, maps the entity embedding into the Euclidean space, models the triple query as the nearest neighbor retrieval in the Euclidean space, and further quantizes and compresses the entity embedding by using product quantization, making the triple query more efficient.

[0112] Embodiment 2

[0113] In the second aspect of the embodiments of the present invention, a secure multi-hop question answering system combining internal and external knowledge graphs is provided. The system includes: a first construction module 601, a first acquisition module 602, a second construction module 603, a second acquisition module 604, a third construction module 605, and a third acquisition module 606.

[0114] The first construction module 601 is used to construct a joint representation learning subsystem through joint training based on the internal knowledge graph and the external knowledge graph.

[0115] The first acquisition module 602 is used to obtain entity representations and relationship representations based on the joint representation learning subsystem; wherein, the entity representations include internal entity representations of the internal knowledge graph and external entity representations of the external knowledge graph, and the relationship representations include internal relationship representations of the internal knowledge graph and external relationship representations of the external knowledge graph.

[0116] The second construction module 603 is used to construct a question encoding and decoding subsystem based on the internal knowledge graph through a language encoding model and a sequence encoding model.

[0117] [[ID=3L]]The second acquisition module 604 is used to obtain multi-hop inference paths based on the question encoding and decoding subsystem.

[0118] The third construction module 605 is used to construct a secure knowledge retrieval subsystem based on the multi-hop inference paths, and the entity representations and the relationship representations obtained by the joint representation learning subsystem.

[0119] A third acquisition module 606, configured to obtain the correct answer corresponding to the multi-hop question when a multi-hop question is input from the private end to the security knowledge retrieval subsystem.

[0120] Through the secure multi-hop question-answering system provided by the present invention, applied in the field of multi-hop question answering, it can not only perform knowledge graph question-answering reasoning on multiple knowledge graphs, improve the accuracy of the knowledge graph question-answering system, but also improve the query efficiency during the query process, and can also protect the privacy of the query data at the private end and the privacy of the query results.

[0121] Embodiment III

[0122] In the third aspect of the embodiments of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, it implements the steps in the secure multi-hop question answering that combines internal and external knowledge graphs as described in the first aspect of the embodiments of the present invention.

[0123] In this embodiment, refer to Figure 7 , Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present invention; as Figure 7 shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are communicatively connected via a bus. A computer program is stored in the memory 110, and the computer program can run on the processor 120, thereby implementing the steps of the secure multi-hop question answering that combines internal and external knowledge graphs as described in the first aspect of the embodiments of the present application.

[0124] Embodiment IV

[0125] In the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the secure multi-hop question answering that combines internal and external knowledge graphs as described in the first aspect of the embodiments of the present invention.

[0126] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same and similar parts among the embodiments can be referred to each other.

[0127] Embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods and apparatuses according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0130] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

[0131] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the said element.

[0132] The above has introduced in detail a secure multi-hop question answering method and system that combines internal and external knowledge graphs. In this text, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A secure multi-hop question answering method combining internal and external knowledge graphs, characterized in that The method includes: Jointly training based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem; Based on the joint representation learning subsystem, obtaining entity representations and relationship representations; wherein, the entity representations include internal entity representations of the internal knowledge graph and external entity representations of the external knowledge graph, and the relationship representations include internal relationship representations of the internal knowledge graph and external relationship representations of the external knowledge graph; Based on the internal knowledge graph, constructing a question encoding and decoding subsystem through a language encoding model and a sequence encoding model; wherein, constructing the question encoding and decoding subsystem based on the internal knowledge graph through the language encoding model and the sequence encoding model includes: Based on the internal knowledge graph, obtaining question and answer training data; Based on the question and answer training data, extracting natural language questions and corresponding topic entities and answers; Based on the natural language questions, obtaining the prediction paths of the natural language questions through the language encoding model and the sequence encoding model; Based on the topic entity and the answer, generating a relationship sequence between the topic entity and the answer with the shortest path in the internal knowledge graph as the weak supervision label; wherein, in the case of multiple identical shortest paths in the internal knowledge graph, the shortest path with the highest occurrence frequency is used as the weak supervision label; Based on the prediction paths and the relationship sequences, updating the language encoding model and the sequence encoding model; Based on the updated language encoding model and sequence encoding model, constructing the question encoding and decoding subsystem; Based on the question encoding and decoding subsystem, obtaining multi-hop inference paths; Based on the multi-hop inference paths, and the entity representations and the relationship representations obtained by the joint representation learning subsystem, constructing a security knowledge retrieval subsystem; When inputting a multi-hop question from the private side to the security knowledge retrieval subsystem, obtaining the correct answer corresponding to the multi-hop question; wherein, when inputting a multi-hop question from the private side to the security knowledge retrieval subsystem and obtaining the correct answer corresponding to the multi-hop question, it includes: based on the security knowledge retrieval subsystem, converting the multi-hop question into multiple query vectors.

2. The method according to claim 1, characterized in that, The jointly training based on the internal knowledge graph and the external knowledge graph to construct a joint representation learning subsystem includes: Obtaining the internal relationship list of the internal knowledge graph and the external relationship list of the external knowledge graph; Based on the internal relationship list and the external relationship list, obtaining a common relationship list; Based on the common relationship list, constructing a joint representation learning subsystem through an orthogonal Procrustes problem.

3. The method according to claim 1, wherein The obtaining entity representations and relationship representations based on the joint representation learning subsystem includes: Based on the joint representation learning subsystem, constructing constraint conditions for entities and common relationships; Based on the constraint conditions, obtaining relationship generation factors; Based on the relationship generation factors, obtaining a calculated relationship embedding matrix; Based on the relationship embedding matrix and the constraint conditions for training, obtaining the entity representations and the relationship representations.

4. The method according to claim 1, characterized in that, Construct a security knowledge retrieval subsystem based on the multi-hop inference path and the entity representation and relationship representation obtained by the joint representation learning subsystem, including: Perform product quantization compression on the entity representation and the relationship representation to obtain a product quantization codebook; Based on the entity representation, obtain an inverted index corresponding to the entity representation; Based on the inverted index, obtain a set of vector list corresponding to the inverted index; Based on the multi-hop inference path, as well as the set of vector lists, the inverted index, and the product quantization codebook, construct the security knowledge retrieval subsystem.

5. The method according to claim 1, characterized in that, When a multi-hop question is input from the private end to the security knowledge retrieval subsystem, the security knowledge retrieval subsystem outputs the correct answer corresponding to the multi-hop question, and further includes: Obtain the nearest neighbor entity embedding representations corresponding to the multiple query vectors by calculating distances and asymmetric distances for the multiple query vectors; Based on the nearest neighbor entity embedding representations, obtain the correct answer.

6. A secure multi-hop question answering system that combines internal and external knowledge graphs, characterized in that, The system includes: A first construction module for jointly training according to an internal knowledge graph and an external knowledge graph to construct a joint representation learning subsystem; A first acquisition module for obtaining an entity representation and a relationship representation based on the joint representation learning subsystem; wherein, the entity representation includes an internal entity representation of the internal knowledge graph and an external entity representation of the external knowledge graph, and the relationship representation includes an internal relationship representation of the internal knowledge graph and an external relationship representation of the external knowledge graph; A second construction module for constructing a question encoding and decoding subsystem based on the internal knowledge graph through a language encoding model and a sequence encoding model; wherein, the second construction module includes: Obtain question-answering training data based on the internal knowledge graph; Extract natural language questions and corresponding topic entities and answers based on the question-answering training data; Obtain the predicted path of the natural language question through the language encoding model and the sequence encoding model based on the natural language question; Based on the topic entity and the answer, generate a relationship sequence between the topic entity and the answer with the shortest path in the internal knowledge graph as a weak supervision label; wherein, in the case of multiple identical shortest paths in the internal knowledge graph, use the shortest path with the highest occurrence frequency as the weak supervision label; Update the language encoding model and the sequence encoding model based on the predicted path and the relationship sequence; Construct the question encoding and decoding subsystem based on the updated language encoding model and sequence encoding model; A second acquisition module for obtaining a multi-hop inference path based on the question encoding and decoding subsystem; A third construction module for constructing a security knowledge retrieval subsystem based on the multi-hop inference path and the entity representation and relationship representation obtained by the joint representation learning subsystem; A third acquisition module, configured to obtain the correct answer corresponding to the multi-hop question when a multi-hop question is input from the private end to the security knowledge retrieval subsystem; wherein, the third acquisition module includes: based on the security knowledge retrieval subsystem, converting the multi-hop question into a plurality of query vectors.

7. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When executed by the processor, it implements the steps in the secure multi-hop question answering method combining internal and external knowledge graphs according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the secure multi-hop question answering method combining internal and external knowledge graphs according to any one of claims 1-5.

Images