A method for constructing a knowledge graph question answering model

Abstract

The application provides a knowledge graph question answering model construction method, the knowledge graph question answering model is used for deducing the answer of a question according to a knowledge graph, and the method comprises the following steps: S1, constructing an initial knowledge graph question answering model; wherein the initial knowledge graph question answering model comprises a question decomposition module, an embedding module and a question answering module; S2, obtaining a first training set, the first training set comprises a plurality of first training samples and a first label, the plurality of first training samples are questions, and the label is the answer true value of the first training sample; S3, training the initial knowledge question answering model by using the first training set, and updating the parameters of the knowledge graph question answering model by using a preset loss function in the training.

Description

Technical Field

[0001] This invention relates to the field of knowledge graphs, specifically to the question-answering domain within the knowledge graph field, and more specifically, to a method for constructing a knowledge graph question-answering model. Background Technology

[0002] Knowledge graphs (KGs) can store numerous facts from the real world and facilitate the development of downstream applications. As a promising direction for inferring new knowledge from existing knowledge, reasoning within knowledge graphs has attracted increasing attention. Specifically, a typical reasoning task is query-answering complex first-order logic (FOL) queries on knowledge graphs, which involves existential quantifiers. Conjunction (∧), disjunction (∨), and negation Logical operations, etc.

[0003] Over the past few decades, traditional methods have achieved significant success in link prediction using inductive reasoning. These methods learn the embeddings of knowledge graphs and queries, and map logical operations and queries to a latent space for query-response through inductive computation. However, knowledge graphs are often incomplete and open-ended, and unseen entities may be encountered during knowledge graph reasoning. This inductive reasoning approach requires the embeddings of all entities to perform reasoning, severely limiting the generalization and reasoning capabilities of knowledge graphs. Therefore, inductive reasoning cannot handle unseen entities in this context. To address this, researchers have made efforts to improve inductive reasoning methods. Specifically, some researchers have proposed learning inductive representations of entities for prediction, while others focus on learning inductive representations of relationships to leverage the relationships between queries and responses.

[0004] Since knowledge graphs are composed of entities and relations, it is inappropriate to use only the representation of any one entity to learn the embedding of entities in the knowledge graph and predict new entities. This will not only cause a loss of semantic value, but also affect the reasoning of open knowledge graphs. In particular, when faced with unknown entities, the loss of semantic value will be further aggravated, thereby reducing the reasoning ability of open knowledge graphs. Summary of the Invention

[0005] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for constructing a knowledge graph question-answering model.

[0006] According to one aspect of the present invention, a method for constructing a knowledge graph question-answering model is provided. The method includes: S1, constructing an initial knowledge graph question-answering model, wherein the initial knowledge graph question-answering model includes: a question decomposition module, used to decompose a question into a logical answer form to obtain multiple entities representing the question, wherein the logical question answer includes: conjunction, disjunction, and / or negation; and searching for multi-hop neighbor entities that are related to the multiple entities of the question in a preset knowledge graph to expand the multiple entities of the question to obtain a knowledge structure related to the question; an embedding module, used to perform short-range structure calculation on each entity of the question according to the knowledge structure related to the question to obtain a first embedding representation and a long-range structure. The system calculates a second embedding representation and aggregates the first and second embedding representations to obtain the embedding representation of the entity; the question answering module calculates the probability that each entity in the knowledge structure related to the question is the answer based on the embedding representation corresponding to each entity of the question, and selects the entity with the highest probability as the answer to the question; S2, obtains a first training set, which includes multiple first training samples and a first label, wherein the multiple first training samples are questions and the label is the truth value of the answer of the first training sample; S3, trains the initial knowledge question answering model using the first training set, and updates the parameters of the knowledge graph question answering model using a preset loss function during training.

[0007] In some embodiments of the present invention, the problem decomposition module is configured to: decompose the problem into a logical answer form based on suffix notation to obtain multiple entities representing the problem; based on the multiple entities of the problem, search for neighboring entities within a preset number of hops that are related to the multiple entities of the problem in the knowledge graph and expand the multiple entities of the problem to obtain a knowledge structure related to the problem; based on the knowledge structure related to the problem, initialize each entity of the problem using an initialization function to obtain an initial vector representation of each entity.

[0008] In some embodiments of the present invention, the embedding module includes a multi-layer graph convolutional network, each layer of which includes: a short-range inference unit, used to perform short-range structure calculation based on the input vector of each entity and the input vector of its one-hop neighbor entities to obtain a first embedding representation of each entity in the current layer of the graph convolutional network; a long-range inference unit, used to perform long-range structure calculation based on the input vector of each entity, its initial vector representation, and the sum of the input vectors of its neighbor entities within a preset number of hops to obtain a second embedding representation of each entity in the current layer of the graph convolutional network; and an aggregation unit, used to assign weights to the first and second embedding representations of each entity in the graph convolutional network using a preset aggregation function and aggregate them to obtain the output vector of each entity in the current layer of the graph convolutional network; wherein, in the multi-layer graph convolutional network, the input vector of the first layer of the graph convolutional network is the initial vector representation, the output vector of the previous layer of the graph convolutional network is used as the input vector of the next layer of the graph convolutional network, and the output of the last layer of the graph convolutional network is the final embedding representation.

[0009] In some embodiments of the present invention, the step of performing short-range structure calculation based on the input vector of each entity and the input vector of its one-hop neighbor entities to obtain the first embedding representation of each entity in the current layer of the graph convolutional network includes: mapping the input vector of each entity's one-hop neighbor entity to the geometric space using a relation mapping matrix to obtain the mapped vector representation of the one-hop neighbor entity; fine-tuning the dimension of the mapped vector representation of the one-hop neighbor entity using a translation matrix of the relationship between each entity and its one-hop neighbor entity to obtain the vector representation of each entity and its one-hop neighbor entity in the geometric space; and calculating the vector representation of each entity and its one-hop neighbor entity in the geometric space using an activation function to obtain the first embedding representation of each entity in the current layer of the graph convolutional network; wherein, the first embedding representation is as follows:

[0010]

[0011] Where v represents an entity, u represents a one-hop neighbor entity of v, r represents the relationship between entity v and its neighbor u, and k represents the number of layers in the graph convolutional network. This represents the set of neighboring entities of entity v. This represents the relational mapping matrix in the graph convolutional network at layer k. This represents the embedding representation of neighboring entities in the graph convolutional network at layer k. Let f(·) represent the translation matrix in the graph convolutional network at layer k, φ(·) represent the matrix computation function, and f(·) represent the activation function.

[0012] In some embodiments of the present invention, the step of performing long-range structure calculations based on the input vector of each entity, its initial vector representation, and the sum of the input vectors of its neighboring entities within a preset number of hops to obtain the second embedding representation of each entity in the current layer of the graph convolutional network includes: calculating the weighted sum of the paths between each entity and its neighboring entities within a preset number of hops using an attention mechanism; assigning corresponding weight coefficients to the paths between each entity and its neighboring entities within a preset number of hops based on the calculated weighted sum to obtain the path representations between each entity in the current layer of the graph convolutional network and its neighboring entities; and calculating the second embedding representation of each entity based on the initial vector representation of each entity, the path representations between each entity and its neighboring entities, and the embedding representations of each entity output from the previous layer; wherein the second embedding representation is as follows:

[0013]

[0014] Where v represents an entity, u represents a neighboring entity within a preset number of hops of entity v, m represents an intermediate entity between entity v and its neighbor u, and λ represents the path weight coefficient. This represents the path representation between entity v and its neighbor u in a graph convolutional network at layer k. The initial vector representation of entity v.

[0015] In some embodiments of the present invention, the step of calculating the probability that each entity in the knowledge structure related to the question is the answer based on the embedding representation corresponding to each entity of the question includes: calculating the probability that each entity in the knowledge structure related to the question is the answer using a fuzzy set method based on the embedding representation corresponding to each entity of the question and the relationship between each entity of the question and each entity in the knowledge structure related to the question.

[0016] In some embodiments of the present invention, the preset loss function is expressed as follows:

[0017]

[0018] in, This represents the set of all positive samples that are predicted as the answer. Represents the set of answers. This represents the set of all negative samples that are predicted as the answer. This represents the probability that a positive sample is predicted as the answer. This indicates the probability that a negative sample is predicted as the answer; the positive sample is the entity in the knowledge structure related to the question that belongs to the entity corresponding to the label, and the negative sample is the entity in the knowledge structure related to the question that does not belong to the entity corresponding to the label.

[0019] According to a second aspect of the present invention, a question answering method based on a knowledge graph question answering model is provided. The method includes the following steps: T1, obtaining a knowledge graph and a target question; T2, inputting the knowledge graph and the target question into a knowledge graph question answering model constructed using the method described in the first aspect of the present invention, obtaining the probability that each entity in the knowledge graph is predicted to be the answer to the target question, and selecting the entity corresponding to the highest probability as the answer to the question based on the probability.

[0020] Compared with the prior art, the advantages of the present invention are as follows:

[0021] 1) By breaking down a problem into a logical question-and-answer format, the relationships between entities can be expressed in a structured way, which is beneficial for subsequent related calculations.

[0022] 2) The short-range structure of an entity is represented by information from its neighboring entities, rather than by directly using information from the entity itself. This can effectively mine the one-hop relationship between an entity and its neighboring entities, ensuring the semantic relationship between entities and overcoming the problem of insufficient mining capabilities of existing models.

[0023] 3) The final long-range structural knowledge representation of the entity is obtained by using the initial vector representation of the entity itself, the path information representation between the entity and its neighboring entities, and the embedding representation of the entity obtained from the previous layer. This not only ensures that the obtained knowledge representation is neither biased towards the entity itself nor towards its neighboring entities, but also preserves the original information of the entity and can also uncover the potential relationships between the entity and its multi-hop neighboring entities, thus improving the performance of the model in uncovering the potential relationships between entities and fully guaranteeing the semantic relationships between entities.

[0024] 4) By using aggregation functions to assign weight coefficients to short-range and long-range structures respectively, a unified representation can be formed, which can effectively capture long-range information of entities, thereby mining richer interactions between entities. This allows the model to focus on the rich semantic associations between entities in open knowledge graphs, take into account different knowledge structures of entities, and overcome the weakness of existing models that only mine neighborhood information, thus improving the model's ability to reason about open knowledge graphs. Attached Figure Description

[0025] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0026] Figure 1 This is a flowchart illustrating a method for constructing a knowledge graph question-answering model according to an embodiment of the present invention.

[0027] Figure 2 This is a schematic diagram of the structure of the embedded module according to an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the knowledge structure related to the problem described in an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is further described in detail below through specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0030] As described in the background section, while existing technologies have achieved significant success in link prediction using inductive reasoning, such methods require the embeddings of all entities during reasoning. However, knowledge graphs are typically incomplete and open. Therefore, when an entity not found in the knowledge graph appears, existing methods cannot process it accordingly. Furthermore, existing methods only utilize the representations of its one-hop neighboring entities to learn the entity embeddings in the knowledge graph and predict new entities. This not only ignores the rich semantic relationships (knowledge structures) between facts in the open knowledge graph, resulting in a lack of semantic quantity, but also affects reasoning on the open knowledge graph. Especially when facing unknown entities, the loss of semantic quantity is further aggravated, thereby reducing the reasoning ability of the open knowledge graph, and the model cannot fully explore the potential relationships between entities.

[0031] To address the aforementioned problems, this invention proposes a knowledge graph question-answering model construction method. This method learns representations of knowledge structures, rather than just entity or relation representations, to answer questions about unseen entities using a reasoning approach similar to human reasoning. In summary, as... Figure 1As shown, the method includes: S1, constructing an initial knowledge graph question-answering model, wherein the initial knowledge graph question-answering model includes: a question decomposition module, used to decompose the question into a logical answer form to obtain multiple entities representing the question, wherein the logical question answer includes: conjunction, disjunction, and / or negation; and searching for multi-hop neighbor entities that are related to the multiple entities of the question in a preset knowledge graph to expand the multiple entities of the question to obtain the knowledge structure related to the question; an embedding module, used to perform short-range structure calculation to obtain a first embedding representation and long-range structure calculation to obtain a second embedding representation for each entity of the question according to the knowledge structure related to the question, and The first and second embedding representations are aggregated to obtain the embedding representation of the entity; the question answering module is used to calculate the probability that each entity in the knowledge structure related to the question is the answer based on the embedding representation corresponding to each entity of the question, and select the entity with the highest probability as the answer to the question; S2, a first training set is obtained, the first training set includes multiple first training samples and a first label, the multiple first training samples are questions, and the label is the truth value of the answer of the first training sample; S3, the initial knowledge question answering model is trained using the first training set, and the parameters of the knowledge graph question answering model are updated using a preset loss function during training. The knowledge graph question model constructed based on the method of the present invention can not only learn the short-range structure between an entity and its one-hop neighbor entities, but also learn the long-range structure between an entity and its multi-hop neighbor entities, and work together to correctly learn the knowledge structure in the knowledge graph, realize the rich semantic association (knowledge structure) between facts in the open knowledge graph, increase the semantic quantity in the knowledge graph, improve the model's reasoning ability on the open knowledge graph, and enhance the model's ability to mine potential relationships between entities.

[0032] To better understand the knowledge graph question answering model construction method of the present invention, the technical solution of the present invention will be described in detail below with reference to specific embodiments, from four parts: model structure, training samples, training process and application scenarios.

[0033] I. Model Structure

[0034] First, let me introduce the initial knowledge graph question-answering model structure constructed in this invention, which includes a question decomposition module, an embedding module, and a question-answering module, wherein:

[0035] The problem decomposition module is used to decompose the problem into a logical answer to obtain multiple entities representing the problem. The logical question answer includes conjunction, disjunction, and / or negation. The module also searches for multi-hop neighbor entities that are related to the multiple entities of the problem in a preset knowledge graph to expand the multiple entities of the problem and obtain the knowledge structure related to the problem.

[0036] The embedding module is used to perform short-range structure calculation to obtain a first embedding representation and long-range structure calculation to obtain a second embedding representation for each entity of the problem based on the knowledge structure related to the problem, and to aggregate the first embedding representation and the second embedding representation to obtain the embedding representation of the entity.

[0037] The question-answering module is used to calculate the probability that each entity in the knowledge structure related to the question is the answer based on the embedded representation corresponding to each entity of the question, and select the entity with the highest probability as the answer to the question.

[0038] According to one embodiment of the present invention, the problem decomposition module is configured to: decompose the problem into a logical answer form based on suffix notation to obtain multiple entities representing the problem; based on the multiple entities of the problem, search for neighboring entities within a preset number of hops that are related to the multiple entities of the problem in a knowledge graph and expand the multiple entities of the problem to obtain a knowledge structure related to the problem; based on the knowledge structure related to the problem, initialize each entity of the problem using an initialization function to obtain an initial vector representation of each entity. By decomposing the problem into a logical question-and-answer form, the relationships between entities can be expressed in a structured way, which is beneficial for subsequent related calculations.

[0039] According to one example of the invention, the question is decomposed into a logical answer form (FOL logical question-and-answer) based on suffix notation to obtain multiple entities representing the question, wherein the logical question-and-answer form is composed of quantifiers. Conjunction (∧), disjunction (∨), and negation Composed of logical operations, for example: the question is: which citizens of developed countries have won the Nobel Prize? This can be represented as: Among them, developed countries, citizens, and Nobel Prizes represent entities related to this issue. The postfix notation, also known as Reverse Polish Notation (Lukasiewicz, 1951), actually uses operators (quantifiers)... Conjunction (∧), disjunction (∨), and negation This is a notation that is written after the operands in an expression. For more information, see: Lukasiewicz, J. Aristotle's syllogistic from the standpoint of modern formallogic. 1951.

[0040] According to one example of the present invention, based on the entities obtained from the above problem decomposition, neighboring entities within a preset number of hops that are related to multiple entities of the problem are searched in the knowledge graph, and the multiple entities of the problem are expanded. For example, if the entity is a developed country, then entities related to developed countries (e.g., the United Kingdom, France, Spain, etc.) are searched in the knowledge graph; if the entity is the Nobel Prize, then entities related to the Nobel Prize (e.g., the Nobel Prize in Literature, the Nobel Prize in Physics, the Nobel Prize in Chemistry, etc.) are searched in the knowledge graph. Based on the entities found in the knowledge graph, the multiple entities of the problem are expanded to obtain a knowledge structure related to the problem.

[0041] To facilitate subsequent calculations, each entity needs to be initialized. According to one embodiment of the present invention, an initialization function is used to initialize each entity in the problem and each entity in the knowledge structure related to the problem, to obtain an initial vector representation for each entity. For example, through initialization, each entity is assigned a multi-dimensional vector code representing entity identity information as its initial vector representation. According to an example of the present invention, the INDICATOR initialization function is used to initialize the entities in the problem, for example:

[0042]

[0043] in, Let v be the initial vector representation of entity v, q be the problem, and u and v be entities in problem q.

[0044] According to one embodiment of the present invention, the embedding module includes a multi-layer graph convolutional network, the structure of which is as follows: Figure 2 As shown, each layer of the graph convolutional network includes: a short-range inference unit, used to perform short-range structure calculation based on the input vector of each entity and the input vector of its one-hop neighbor entities to obtain the first embedding representation of each entity in the current layer of the graph convolutional network; a long-range inference unit, used to perform long-range structure calculation based on the input vector of each entity, its initial vector representation, and the sum of the input vectors of its neighbor entities within a preset number of hops to obtain the second embedding representation of each entity in the current layer of the graph convolutional network; and an aggregation unit, used to assign weights to the first and second embedding representations of each entity in the graph convolutional network using a preset aggregation function and aggregate them to obtain the output vector of each entity in the current layer of the graph convolutional network; wherein, in the multi-layer graph convolutional network, the input vector of the first layer of the graph convolutional network is the initial vector representation of the entity, the output vector of the previous layer of the graph convolutional network is used as the input vector of the next layer of the graph convolutional network, and the output of the last layer of the graph convolutional network is the final embedding representation.

[0045] According to an embodiment of the present invention, in the short-range inference unit, the method for performing short-range structure calculation based on the input vector of each entity and the input vector of its one-hop neighbor entities to obtain a first embedding representation of each entity in the graph convolutional network of the current layer includes: mapping the input vector of each entity's one-hop neighbor entity to a geometric space using a relation mapping matrix to obtain a mapped vector representation of the one-hop neighbor entity; fine-tuning the dimension of the mapped vector representation of the one-hop neighbor entities using a translation matrix of the relationship between each entity and its one-hop neighbor entities to obtain a vector representation of each entity and its one-hop neighbor entities in the geometric space; and calculating the vector representation of each entity and its one-hop neighbor entities in the geometric space using an activation function to obtain a first embedding representation of each entity in the graph convolutional network of the current layer; wherein, the first embedding representation is as follows:

[0046]

[0047] Where v represents an entity, u represents a one-hop neighbor entity of v, r represents the relationship between entity v and its neighbor u, and k represents the number of layers in the graph convolutional network. This represents the set of one-hop neighbor entities of entity v. This represents the relational mapping matrix in the graph convolutional network at layer k. This represents the embedding representation of neighboring entities in the graph convolutional network at layer k. Let f(·) represent the translation matrix in the k-th layer of the graph convolutional network, φ(·) represent the matrix computation function, and f(·) represent the activation function. In this invention, the sigmoid activation function can be used. The technical solution of this embodiment can achieve at least the following beneficial technical effects: This embodiment does not directly use the information of the entity itself for representation, but uses the entity's neighboring entities for representation. This can effectively mine the one-hop relationship between the entity and its neighboring entities, ensuring the semantic relationship between entities and overcoming the problem of insufficient mining capabilities in existing models.

[0048] According to one embodiment of the present invention, in the short-range inference unit, the matrix calculation function can be either a matrix multiplication function or a matrix addition function. Optionally, if the matrix calculation function is a matrix multiplication function, then... If the matrix calculation function is a matrix addition function, then

[0049] According to an embodiment of the present invention, in the long-range inference unit, the method for performing long-range structure calculation based on the input vector of each entity, its initial vector representation, and the sum of the input vectors of its neighboring entities within a preset number of hops to obtain a second embedding representation of each entity in the graph convolutional network of the current layer includes: calculating a weighted sum of the paths between each entity and its neighboring entities within a preset number of hops using an attention mechanism; assigning corresponding weight coefficients to the paths between each entity and its neighboring entities within a preset number of hops based on the calculated weighted sum to obtain the path representations between each entity in the graph convolutional network of the current layer and its neighboring entities; and calculating a second embedding representation for each entity based on the initial vector representation of each entity, the path representations between each entity and its neighboring entities, and the embedding representations corresponding to each entity output from the previous layer; wherein the second embedding representation is as follows:

[0050]

[0051] Where v represents an entity, u represents a neighboring entity within a preset number of hops of entity v, m represents an intermediate entity between entity v and its neighbor u, and λ represents the path weight coefficient. This represents the path representation between entity v and its neighbor u in a graph convolutional network at layer k. This represents the initial vector representation of entity v. The technical solution of this embodiment can achieve at least the following beneficial technical effects: by comprehensively utilizing the initial vector representation of the entity itself and the path information representation to obtain the knowledge representation of the entity, the obtained knowledge representation is neither biased towards the entity itself nor towards neighboring entities. It can not only preserve the original information of the entity, but also mine the potential relationships between the entity and multi-hop neighboring entities, thus improving the performance of the model in mining the potential relationships between entities.

[0052] More specifically, the second embedding representation can also be represented as:

[0053]

[0054] in, Let g(·) represent the path between v and u in the graph convolutional network at layer k, and g(·) represent the path selection function.

[0055] The path between v and u in the k-th layer of the graph convolutional network can be represented by intermediate entities, as shown below:

[0056]

[0057] Where m represents the intermediate entity between entity v and its neighboring entity u. The path represents the set of neighboring entities of entity v. It is obtained by sequentially splicing the embedded representations of all entities at the k-th level on the path connecting entity v and its neighboring entity u through the intermediate entity m.

[0058] Then its path selection function can be expressed as:

[0059]

[0060] Here, λ represents the weighting coefficient.

[0061] According to one embodiment of the present invention, its short-range structure represents the relationship between an entity and a one-hop neighbor entity, for example: Haidian District -> (located) in Asia; its long-range structure represents the multi-hop relationship between different entities, for example: Haidian District -> (located) in Beijing -> (belongs) to China -> (belongs) to Asia, that is, the relationship from Haidian District to Asia is a multi-hop relationship, and it is necessary to connect Haidian District and Asia through a multi-hop relationship.

[0062] According to an embodiment of the present invention, in the aggregation unit, a preset aggregation function is used to assign weights to the first and second embedding representations of each entity in the graph convolutional network and then aggregate them to obtain the embedding representation of each entity, which is as follows:

[0063]

[0064] in, This represents the first embedding representation of entity v in the graph neural network at layer k. Let v represent the second embedding representation of entity v in the graph neural network at layer k, and let ψ(·) represent the predefined aggregation function.

[0065] Its aggregation function can be expressed as:

[0066]

[0067] Wherein, α represents the weight coefficient of the first embedding representation, and β represents the weight coefficient of the second embedding representation. The technical solution of this embodiment can achieve at least the following beneficial technical effects: by using aggregation functions to assign weight coefficients to short-range and long-range structures respectively for aggregation, a unified representation can be formed, which can effectively capture long-range information of entities, thereby mining richer interactions between entities. This allows the model to take into account different knowledge structures of entities, overcoming the weakness of existing models that only mine neighborhood information.

[0068] According to one embodiment of the present invention, the question-answering module is configured to: calculate the probability that each entity in the knowledge structure related to the question is an answer based on the embedded representations corresponding to each entity of the question and the relationship between each entity of the question and each entity in the knowledge structure related to the question using fuzzy set theory; and select the entity with the highest probability as the answer to the question. According to an example of the present invention, based on fuzzy set theory, the probability that each entity in the knowledge structure related to the question is an answer and the relationship between each entity in the knowledge structure related to the question and each entity of the question are introduced, the probability that each entity in the knowledge structure related to the question is an answer can be calculated based on the embedded representations of each entity obtained in the embedding module and the relationship between each entity in the knowledge structure related to the question and each entity of the question. The calculation method is as follows:

[0069]

[0070] x vi ∈[0,1] ε ;

[0071] in, x represents the embedding representation of each entity. vi r represents the probability that the i-th entity in the knowledge structure related to the question is the answer, and its value ranges from 0 to 1. i This represents the relationship between the i-th entity in the knowledge structure related to the problem and the entity v of the problem.

[0072] Based on the above calculation method, a probability vector related to the entities in the question can be obtained. This probability vector is composed of the probabilities that each entity in the knowledge structure related to the question is the answer. Therefore, for questions involving multiple entities, it is necessary to calculate the probability vector of each entity separately using the above probability calculation method. Then, based on the logical question-and-answer format of the question, appropriate logical operations are selected to perform the final calculation on the obtained probability vectors of each entity. For example, when the question is decomposed into a conjunction form, the calculation method is as follows: When the problem is decomposed into a disjunctive form, its calculation method is as follows: When the problem is decomposed into a negated form, the calculation method is as follows: Here, a and b represent the probability vectors of each entity in the problem.

[0073] II. Training Samples

[0074] Secondly, in order to enable the model to have better performance in mining relationships between entities, the initial knowledge graph question answering model constructed above needs to be trained. Before training, training samples need to be organized.

[0075] According to one embodiment of the present invention, a first training set is obtained, the first training set including a plurality of first training samples and a first label, wherein the plurality of first training samples are questions, and the label is the truth value of the answer of the first training sample. According to an example of the present invention, if the question is: What fruit hit Newton on the head and caused Newton to discover gravity? Then the label is: apple.

[0076] III. Training Process

[0077] Based on the training samples of the aforementioned organization, the loss value is calculated using the loss function to back-update the parameters of the initial knowledge graph question answering model, so as to obtain a knowledge graph question answering model for practical application scenarios.

[0078] According to an embodiment of the present invention, the initial knowledge graph model is trained using the first training set. During training, the parameters of the knowledge graph question-answering model are updated using a preset loss function. The process includes: inputting training samples from the first training set into the question decomposition module of the initial knowledge graph question-answering model to obtain the knowledge structure related to the training samples and the initial vector representation of each entity; using an embedding module to perform short-range structure calculations on each entity in the training samples based on the knowledge structure related to the training samples to obtain a first embedding representation and long-range structure calculations to obtain a second embedding representation, and aggregating the first and second embedding representations to obtain the embedding representation corresponding to each entity in the training samples; using a question-answering module to calculate the probability that each entity in the knowledge structure related to the training samples is an answer based on the embedding representation corresponding to each entity in the training samples; calculating a loss value using a preset loss function based on the calculated probability that each entity in the knowledge structure related to the training samples is an answer and the label, and then using the gradient to back-update the parameters of the initial knowledge graph question-answering model based on the loss value. The preset loss function is as follows:

[0079]

[0080] in, This represents the set of all positive samples that are predicted as the answer. Represents the set of answers. This represents the set of all negative samples that are predicted as the answer. This represents the probability that a positive sample is predicted as the answer. This indicates the probability that a negative sample is predicted as the answer; the positive sample is the entity in the knowledge structure related to the question that belongs to the entity corresponding to the label, and the negative sample is the entity in the knowledge structure related to the question that does not belong to the entity corresponding to the label.

[0081] IV. Application Scenarios

[0082] In practical application scenarios: acquire the target knowledge graph and the target question; input the knowledge graph and the target question into the knowledge graph question answering model obtained by the above method, obtain the probability that each entity in the knowledge graph is predicted to be the answer to the target question, and select the entity with the highest probability as the answer to the question based on the probability.

[0083] According to one embodiment of the present invention, the target question is decomposed using a question decomposition module in the constructed knowledge graph question-answering model, and multiple entities in the target knowledge graph that are related to multiple entities of the target question are searched to expand the multiple entities of the target question, so as to obtain the knowledge structure related to the target question; the embedding module performs short-range structure calculation and long-range structure calculation on each entity of the target question according to the knowledge structure related to the target question, and aggregates the calculation results to obtain the embedding representation of the entity; the question-answering module calculates the probability that each entity in the knowledge structure related to the target question is the answer according to the embedding representation corresponding to each entity of the target question, and selects the entity with the highest probability as the answer to the target question.

[0084] According to one example of the present invention, the target question is: What fruit hit Newton on the head? Decomposing the question yields its logical question-and-answer form, which can be represented as: Based on the logical question-and-answer format of the above problem, we can obtain two entities: fruit and Newton. We then expand the knowledge graph by searching for multi-hop neighbor entities that are related to multiple entities in the target problem, thus obtaining a knowledge structure related to the target problem. For example, for the entity "fruit," there might be multi-hop neighbor entities in the knowledge graph that are related to fruit (such as apples, bananas, pears, mangoes, strawberries, vitamins, etc.); for the entity "Newton," there might be multi-hop neighbor entities in the knowledge graph that are related to Newton (such as the UK, Cambridge University, Galileo, Newton's three laws, Earth's gravity, etc.). Their knowledge structure can be as follows: Figure 3 As shown, solid arrows indicate a relationship between any two pairs of arrows, while dashed arrows indicate a possible relationship between any two pairs of arrows.

[0085] Based on the obtained knowledge structure related to the target problem, a first embedding representation is obtained by performing short-range structure calculation on each entity of the target problem, that is, the first embedding representation of entity v is: Accordingly, the first embedding of entity u is represented as:

[0086] For each entity in the target problem, a second embedding representation is obtained by performing long-range structure computation, that is, the second embedding representation of entity v is: Accordingly, the second embedding of entity u is represented as:

[0087] Aggregating the first embedding representation and the second embedding representation of each entity yields the embedding representation of that entity, i.e., the embedding representation of entity v is: Accordingly, the embedding representation of entity u is:

[0088] Based on the embedding representations corresponding to each entity in the target question, the probability of each entity in the relevant knowledge structure being the answer is calculated, and the entity with the highest probability is selected as the answer to the question. Since the target question is "What fruit fell on Newton's head?", what needs to be calculated is the probability between the fruit and Newton, i.e., the degree of correlation between the various entities representing the fruit in the knowledge graph and Newton. For entity v, its embedding representation is... Using calculation methods in fuzzy sets Calculate the probability that each entity in the knowledge structure related to the target problem is associated with entity v, for example: This represents the probability that an apple is associated with entity v. This represents the probability that a mango is associated with entity v. This represents the probability that a strawberry is associated with entity v. This represents the probability that a pear is associated with entity v. Let v represent the probability that banana is associated with entity v. Therefore, we can obtain a probability vector associated with entity v of the target question: (0.8, 0.06, 0.04, 0.08, 0.02). The entity corresponding to each probability in the probability vector is (apple, mango, strawberry, pear, banana). Therefore, we choose the entity with the highest probability - apple - as the answer to the target question.

[0089] According to other embodiments of the present invention, the problem is: which citizens of developed countries have won the Nobel Prize? This problem can be broken down into a logical question-and-answer format, as follows:

[0090]

[0091] The entities include developed countries, citizens, and Nobel Prizes. Expanding the problem based on the target knowledge graph, "developed countries" needs to include all entities from developed countries, and "Nobel Prizes" may include Nobel Prizes from various fields. For this type of problem, it's necessary to calculate not only the first probability vector between citizens and Nobel laureates but also the second probability vector between developed countries and citizens. Since this problem is decomposed into a conjunction form, the final probability calculation method is as follows: Where a represents the first probability vector and b represents the second probability vector; based on the finally calculated probability vectors, the entity with the highest probability in the probability vectors is selected as the answer to the question.

[0092] In summary, compared with existing technologies, the knowledge graph question-answering model construction method proposed in this invention has the following advantages:

[0093] 1) By breaking down a problem into a logical question-and-answer format, the relationships between entities can be expressed in a structured way, which is beneficial for subsequent related calculations.

[0094] 2) The short-range structure of an entity is represented by information from its neighboring entities, rather than by directly using information from the entity itself. This can effectively mine the one-hop relationship between an entity and its neighboring entities, ensuring the semantic relationship between entities and overcoming the problem of insufficient mining capabilities of existing models.

[0095] 3) The final long-range structural knowledge representation of the entity is obtained by using the initial vector representation of the entity itself, the path information representation between the entity and its neighboring entities, and the embedding representation of the entity obtained from the previous layer. This not only ensures that the obtained knowledge representation is neither biased towards the entity itself nor towards its neighboring entities, but also preserves the original information of the entity and can also uncover the potential relationships between the entity and its multi-hop neighboring entities, thus improving the performance of the model in uncovering the potential relationships between entities and fully guaranteeing the semantic relationships between entities.

[0096] 4) By using aggregation functions to assign weight coefficients to short-range and long-range structures respectively, a unified representation can be formed, which can effectively capture long-range information of entities, thereby mining richer interactions between entities. This allows the model to focus on the rich semantic associations between entities in open knowledge graphs, take into account different knowledge structures of entities, and overcome the weakness of existing models that only mine neighborhood information, thus improving the model's ability to reason about open knowledge graphs.

[0097] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0098] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0099] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0100] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a knowledge graph question-answering model, wherein the knowledge graph question-answering model is used to derive answers to questions based on a knowledge graph, characterized in that, The method includes: S1. Construct an initial knowledge graph question-answering model, wherein the initial knowledge graph question-answering model includes: The problem decomposition module is used to decompose a problem into a logical question-and-answer format to obtain multiple entities representing the problem, wherein the logical question-and-answer includes conjunction, disjunction, and / or negation; and to expand the multiple entities of the problem by searching for multi-hop neighbor entities that are related to the multiple entities of the problem in a preset knowledge graph to obtain the knowledge structure related to the problem. An embedding module is used to perform short-range structure computation to obtain a first embedding representation and long-range structure computation to obtain a second embedding representation for each entity in the problem based on the knowledge structure related to the problem, and to aggregate the first and second embedding representations to obtain the embedding representation of the entity. The embedding module includes a multi-layer graph convolutional network, each layer of which includes: a short-range inference unit, used to perform short-range structure computation based on the input vector of each entity and the input vector of its one-hop neighbor entities to obtain the first embedding representation of each entity in the current layer of the graph convolutional network; and a long-range inference unit, used to perform short-range structure computation based on the input vector of each entity and its initial vector table. The graph convolutional network (GCNN) is used to calculate the second embedding representation of each entity in the current layer by performing long-range structure calculations on the input vectors of its neighboring entities within a preset hop count. An aggregation unit is used to assign weights to the first and second embedding representations of each entity in the GCNN using a preset aggregation function and then aggregate them to obtain the output vector of each entity in the current layer. In the multi-layer GCNN, the input vector of the first layer is the initial vector representation, the output vector of the previous layer is used as the input vector of the next layer, and the output of the last layer is the final embedding representation. The question-answering module is used to calculate the probability that each entity in the knowledge structure related to the question is the answer based on the embedded representation corresponding to each entity of the question, and select the entity with the highest probability as the answer to the question; S2. Obtain a first training set, which includes multiple first training samples and a first label. The multiple first training samples are questions, and the label is the truth value of the answer of the first training sample. S3. The initial knowledge question answering model is trained using the first training set, and the parameters of the knowledge graph question answering model are updated using a preset loss function during training.

2. The method according to claim 1, characterized in that, The problem decomposition module is configured as follows: The problem is decomposed into a logical question-and-answer form based on suffix notation to obtain multiple entities representing the problem; Based on the multiple entities of the problem, the knowledge graph is searched for neighboring entities within a preset number of hops that are related to the multiple entities of the problem, and the multiple entities of the problem are expanded to obtain the knowledge structure related to the problem. Based on the knowledge structure related to the problem, an initialization function is used to initialize each entity of the problem to obtain an initial vector representation of each entity.

3. The method according to claim 2, characterized in that, The step of performing short-range structure computation based on the input vector of each entity and the input vectors of its one-hop neighbor entities to obtain the first embedding representation of each entity in the graph convolutional network of the current layer includes: The input vector of each entity's one-hop neighbor entity is mapped to the geometric space using a relation mapping matrix to obtain the mapping vector representation of the one-hop neighbor entity; By fine-tuning the dimension of the mapping vector representation of the one-hop neighbor using the translation matrix of the relationship between each entity and its one-hop neighbor entities, the vector representation of each entity and its one-hop neighbor entities in the geometric space is obtained. The vector representations of each entity and its one-hop neighbor entities in the geometric space are calculated using an activation function to obtain the first embedding representation of each entity in the graph convolutional network of the current layer; wherein, the first embedding representation is as follows: ； in, Represents an entity, Representing entities One-hop neighbor entity, Representing entities with neighboring entities The relationship between them This indicates the number of layers in the graph convolutional network. Representing entities The set of neighboring entities, Indicates the first The relational mapping matrix in a layered graph convolutional network. Indicates the first Embedded representations of neighboring entities in a layered graph convolutional network. Indicates the first Translation matrices in a layered graph convolutional network Represents matrix calculation functions. This represents the activation function.

4. The method according to claim 2, characterized in that, The step of performing long-range structure calculations based on the input vector of each entity, its initial vector representation, and the input vectors of its neighboring entities within a preset number of hops to obtain the second embedding representation of each entity in the current layer of the graph convolutional network includes: The attention mechanism is used to calculate the weighted sum of the paths between each entity and its neighboring entities within a preset number of hops; Based on the calculated weighted sum, a corresponding weight coefficient is assigned to the path between each entity and its neighboring entities within a preset number of hops, so as to obtain the path representation between each entity and its neighboring entities in the current layer of the graph convolutional network. The second embedding representation for each entity is calculated based on the initial vector representation of each entity, the path representation between each entity and its neighboring entities, and the embedding representation of each entity output from the previous layer; wherein the second embedding representation is as follows: ； in, Represents an entity, Representing entities The neighboring entities within the preset number of hops, Representing entities with neighboring entities Intermediate entities between The weight coefficients representing the path. Indicates the first Entities in a layered graph convolutional network with neighboring entities Path representation between, Representing entities The initial vector representation.

5. The method according to claim 1, characterized in that, The embedding representation based on each entity corresponding to the question calculates the probability that each entity in the knowledge structure related to the question is the answer, including: Based on the embedded representations of each entity in the problem and the relationships between each entity in the problem and each entity in the knowledge structure related to the problem, the probability of each entity in the knowledge structure related to the problem being the answer is calculated using fuzzy set methods.

6. The method according to claim 1, characterized in that, The preset loss function is expressed as follows: ； in, This represents the set of all positive samples that are predicted as the answer. Represents the set of answers. This represents the set of all negative samples that are predicted as the answer. This represents the probability that a positive sample is predicted as the answer. This indicates the probability that a negative sample is predicted as the answer; the positive sample is the entity in the knowledge structure related to the question that belongs to the entity corresponding to the label, and the negative sample is the entity in the knowledge structure related to the question that does not belong to the entity corresponding to the label.

7. A question-answering method based on a knowledge graph question-answering model, characterized in that, The method includes the following steps: T1. Obtain the knowledge graph and the target question; T2. Input the knowledge graph and the target question into the knowledge graph question answering model constructed using any one of the methods described in claims 1-6, obtain the probability that each entity in the knowledge graph is predicted to be the answer to the target question, and select the entity with the highest probability as the answer to the question based on the probability.

8. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the electronic device to perform the steps of the method as described in any one of claims 1 to 7.

Images