A knowledge graph-based answer retrieval method

Through deep learning technology and Digestella algorithm to extract path information from the knowledge graph, combined with GPT-2 and graph inference network, the problem of difficulty in capturing deep paths and implicit relationships in the existing technology is solved, and more accurate and complete information acquisition.

CN114896377BActive Publication Date: 2024-11-08SOUTHEAST UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202210360978.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-07
Publication Date
2024-11-08
Estimated Expiration
2042-04-07

AI Technical Summary

Technical Problem

Existing methods for obtaining answers based on knowledge graphs are difficult to capture and effectively utilize deep paths and implicit relationships, resulting in inaccurate and complete information acquisition.

Method used

Using deep learning technology, path information is extracted from the knowledge graph through keyword extraction and Digestella algorithm, evidence aggregation is carried out in combination with GPT-2, graph inference network and pre-trained language model, and graph processing and pooling operations are obtained. Final answer.

Benefits of technology

It improves the accuracy and completeness of information acquisition, can explore implicit relationships and deep paths more effectively, enhances the use of knowledge, and overcomes the problems of information sparseness and topic inconsistencies in traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114896377B_ABST
    Figure CN114896377B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for obtaining answers based on a knowledge graph, which is as follows: extract relevant keyword information from questions and candidate answers; extract the path from the keywords in the question to the keywords in the options in the knowledge graph; input the path into the path evidence aggregator PEF and the graph evidence aggregator GEF, and obtain relevant path evidence information and graph evidence information respectively; combine the question and the options into a state vector and input it into the text evidence aggregator TEF, use the pre-trained language model in TEF to capture the semantic association between texts, and obtain features outside the knowledge graph; fuse the evidence from the three different sources obtained above into a feature vector and finally obtain a selection score. Make full use of the information given by the question, capture the semantic relationship of words in the question, and use the knowledge graph to obtain relevant evidence information for answer selection, combine structured and unstructured evidence, and have high recommendation accuracy and good interpretability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for obtaining answers based on a knowledge graph, and belongs to the technical field of Internet and artificial intelligence. Background Art

[0002] With the rapid development of the Internet, data information is showing an explosive growth trend. For example, there are tens of thousands of products on e-commerce platforms such as Taobao, JD.com, and Pinduoduo; there are tens of thousands of film and television resources on video websites such as iQiyi, Tencent Video, and Youku; there are hundreds of millions of news information on news websites such as Toutiao, Sina.com, and Tencent News. Faced with such a large amount of data, it is impossible for users to browse all of them, so it is also very difficult for users to find content that interests them. How to efficiently obtain truly useful information, that is, to achieve efficient acquisition, has become one of the topics that attracts many researchers. We are currently in an era of explosive growth in information. This phenomenon not only brings convenience to information acquisition, but also leads to the difficulty and complexity of accurate information acquisition. It is shown that people often spend 80% of their time looking for relevant information in their daily work and information processing, and the actual time for processing and using data is often less than 20%. At present, people mainly use search engines based on keyword retrieval to find the information they need. Such engines can return highly relevant information based on the input keywords by comparing with the information previously obtained by crawlers. However, this method only relies on the input text information to obtain answer documents sorted by relevance, and cannot obtain semantic context information related to the input text, and the specific answer still needs to be manually checked and found. Moreover, this method is very dependent on the information input by the user. Different people have different ways of describing the same thing, and the information they obtain through search engines may also be very different.

[0003] Many different types of external knowledge have been introduced to solve this type of problem. Methods such as KagNet and BERT_CS extract static graphs from external knowledge graphs (KG) and combine them with pre-trained language models to obtain the final prediction. After extracting the static graph, MHGRN and RGCN use graph neural networks to pass messages from neighbors to central nodes, thereby improving the transferability and circulation of information in the static graph extracted from KG. PG uses a generative model and optimizes the original path to deal with the problem of sparse information in static graphs.

[0004] Although the application of knowledge graphs brings many benefits, its shortcomings cannot be ignored. It limits the exploration of implicit relationships and deep paths. Many previous research works only use the original relationships in KG instead of exploring the implicit relationships between different entities for prediction. In addition, since the extracted subgraphs are often too large but lack information, the sparsity of the adjacency matrix is ​​a common problem of GNN-based models and usually cannot provide enough evidence for reasoning. This is because the node update mechanism of GNN focuses more on using various methods to calculate different forms of weights and to update node representations, rather than exploring implicit relationships between different nodes. Summary of the invention

[0005] In view of the problem that the current answer acquisition method based on knowledge graph is difficult to capture and effectively utilize the deep paths and implicit relationships in the knowledge graph, the present invention proposes an answer acquisition method based on knowledge graph. By using deep learning technology, keywords are extracted by using keyword extraction technology in different combinations of existing questions and options, and then these keywords are used to extract relevant paths in the knowledge graph. The relationships in these paths are then processed by a path-based knowledge aggregator, and finally evidence C from the path is obtained through a pooling operation. p From the perspective of the graph, the extracted paths are processed into graphs to construct a graph network. Then, the graph reasoning network is used to update the entity vector representation by information interaction between nodes. Then, the node representation of the last layer is concatenated and passed through the pooling layer to complete the graph evidence C. g Aggregation. The text evidence aggregator processes the state vector and finally obtains the text evidence C t .

[0006] To achieve the above-mentioned purpose of the invention, the answer acquisition method based on the knowledge graph described in the present invention first uses the keyword extraction method to extract the keywords of the text; then extracts the corresponding path information from the knowledge graph according to the keywords; then constructs a path evidence aggregator based on GPT-2, a graph evidence aggregator based on the graph reasoning network, and a text evidence aggregator based on the pre-trained model. Finally, the evidence aggregated by the three evidence aggregators is processed to obtain score information and select the final answer. The method mainly includes four steps, as follows:

[0007] A method for obtaining answers based on knowledge graph, the method comprising the following steps: Step 1: Based on a given question Q i and its candidate answer A ij , this article starts from Q i Extract n keywords from From A ij Extract m keywords And apply these keywords to different evidence extractors below;

[0008] Step 2: For the keyword information extracted in step 1, use the Dijkstra algorithm to find the path information of the two keyword corresponding nodes in the knowledge graph; the idea of ​​the Dijkstra algorithm is: suppose that the graph G = (V, E) is a directed graph, where V is the vertex set and E is the edge set. Divide the vertex set V in the graph into two parts. The first group is the vertex set for which the shortest path has been found (represented by S. Initially, there are only source points in S. After each shortest path is found, the vertex is added to S until all vertices are added to S and the algorithm ends). The second group is the remaining vertex set for which the shortest path has not been found (represented by U). The vertices in the second group are added to the first group in order of the length of the shortest path. The specific method is as follows:

[0009] (1) The entity nodes corresponding to the keywords in the knowledge graph are regarded as nodes in the directed graph, and the relationship between entities is regarded as the edge in the directed graph. The distance between adjacent nodes is 1;

[0010] (2) Select a question keyword as the source point v. Initially, there is only the source point in S, that is, S = {v}, and the distance of v is 0 (the distance to itself is 0). U contains all other vertices except v, and the distance between v and vertex u in U is 1 (if there is an edge from v to u) or ∞ (there is no edge from v to u);

[0011] (3) Select a vertex k with the smallest distance v from U and add it to S;

[0012] (4) Take k as the new intermediate point and modify the distances of each vertex in U. If the distance from the source point v through vertex k to vertex u is shorter than the original distance (not passing through vertex k), modify the distance of vertex u to be the distance of vertex k plus 1;

[0013] (5) Repeat (3) and (4) until all vertices are added to S.

[0014] (6) Repeat (2) to (5) until all question keyword nodes have completed the query of the shortest path to the option keyword nodes.

[0015] Step 3: Train the answer acquisition method based on the knowledge graph. First, use the keywords extracted in step 1 and the path in step 2 to build three evidence aggregators to aggregate the evidence information from the path, graph, and text respectively. Then, concatenate the evidence information obtained above to obtain the selected score information and give the answer. Finally, use the cross entropy loss function to realize back propagation.

[0016] Step 4: Obtain answers to the questions to be answered based on the best verification model obtained in step 3. For the questions to be answered, first process them according to steps 1 and 2, input the original question text, keywords and path into the answer acquisition method trained in step 3, and obtain the final answer information. This solution can extract key information from multiple dimensions, overcome the problems of inconsistent themes and incomplete information in traditional text summary methods, and can be applied to the accurate extraction of key information in media texts, improving the effect of media text summarization.

[0017] As an improvement of the present invention, step 1 is as follows: first, the text is preprocessed using Natural Language Toolkit (NLTK), including operations such as word segmentation, part-of-speech tagging, and word form restoration, and special characters such as stop words and punctuation marks in the sentence are removed to achieve the effect of improving the quality of text features. Then, the text processes the words in the text through n-gram to generate word tuples, and uses underscores "_" to connect the words in the same tuple into combinations, and then evaluates the importance of these combinations in the sentence according to the TF-IDF algorithm (as shown in the following formula), selects the top n and top m combinations in importance, and splits them as the final keywords.

[0018] The TF-IDF algorithm is:

[0019]

[0020]

[0021] tfidf i,j =tf i,j ×idf i (3)

[0022] Where n i,j is a tuple t i In path d j The number of occurrences, ∑ k n k,j For path d j The sum of the number of triplets, |{j:t i ∈d j}| is a triplet t i The total number of paths that appear, |D| is the total number of all paths.

[0023] As an improvement of the present invention, step 3, using the answer acquisition method based on the knowledge graph processed in step 2 for training, the implementation of this step can be divided into the following sub-steps:

[0024] Sub-step 3-1, construct a path evidence aggregator. In order to solve the sparsity problem of the adjacency matrix, this method introduces PEF to find paths with fewer hops but more information. In addition, this method believes that the path evidence extracted directly from the knowledge graph is not optimal, because such a method knowledge is implemented by retrieving relevant information but does not explore the factual information stored in the knowledge graph. Compared with explicit relationship information, implicit relationships are used to describe the connection between two specific entities and cannot be directly retrieved from the knowledge graph. Therefore, this method uses PEF with GPT-2 as the core to obtain question-option keyword pairs. The optimal k (1≤k≤K) jump path p′ with implicit relations, and GPT-2 is believed to be able to find more relevant structured evidence in static graphs. This method uses the triples in the path to fine-tune the pre-trained GPT-2, and uses Byte-Pair Encoding (BPE) to convert the path into text sequence information, using the prefix "" to represent the opposite relationship. These new paths can make full use of the information in the knowledge graph. When training the model for reasoning, the last word in the path should be input into the model at the beginning to indicate that the model should use this word as the end of the path. The calculation process is as follows: as shown in Formula 4.

[0025] {E0, R′0, E1}=GPT-2({E0, R0, E1}) (4)

[0026] Among them, {E0, R0, E1} is the triple in the original path p, and {E0, R′0, E1} is the triple in the new path p′ containing implicit relations. The relation entities in the path are upgraded to implicit relations containing a lot of reasoning information. When generating the paths {p1, p2, ..., p n}, the attention mechanism is used to obtain the representation of the overall knowledge as the evidence aggregation obtained from the path. The attention calculation formula is shown in the following formula:

[0027] u i =tanh(W i p′ i +b i ) (5)

[0028]

[0029]

[0030] In the above formula, W i , b i is the learnable model parameter, p′ iis the newly generated path information, tanh(·) is the activation function, u i is the representation of the generated path, α i is the path weight, C p For evidence from the path.

[0031] Sub-step 3-2, constructing a graph evidence aggregator. This method selects a Bert-based vector representation method to initialize nodes. The specific initialization method is shown in the formula.

[0032]

[0033] where x i Represents the vector representation of node i,

[0034] As shown in Formula 9, this method uses linear transformation to calculate the value of the node The information flowing to node i through the path,

[0035]

[0036] Where W k are learnable model parameters, is the information transmitted to node i through the k-hop path, is the node attribute or hidden state of node j in layer l, Represents the set of starting points of the k-hop path from node i.

[0037] In the l+1th layer, the attention weight e ik It represents the importance and relevance of the information transmitted to node i through the k-hop path. This paper only considers the influence of this information for k ≤ K. In order to obtain the final attention weight, this method uses the softmax function to normalize all attention coefficients, as shown in formulas 10 and 11.

[0038]

[0039]

[0040] Where W attn are learnable model parameters, s is the state vector, is the information transmitted to node i through the k-hop path, e ik is the attention coefficient, β ik is the attention weight.

[0041] Here, this method uses the attention mechanism to measure s and The correlation between them and the corresponding weight β ik, as shown in Formula 12, this paper believes that the state vector s contains relevant information about the problem and options. A larger weight indicates that the information is closer to the current option and can provide more convincing information for reasoning.

[0042]

[0043] Among them, m i represents the information after aggregation, β ik is the attention weight.

[0044] Next, as shown in Formula 13, this method transmits the information m to node i i The hidden state of the current layer is linearly combined to obtain the node representation of the next layer.

[0045]

[0046] in is the parameter learned by the model, σ is the sigmoid activation function;

[0047] Finally, after the information is transmitted through the l+1 network, the final representation of each node can be obtained, as shown in Formula 14. This method finally obtains the graph-level evidence representation by inputting the attributes of all nodes into the pooling layer.

[0048]

[0049] Among them C g represents the evidence results aggregated from the graph,

[0050] Sub-step 3-3, constructing a text evidence aggregator. This method first uses a lexical analyzer to analyze the question Q i and its candidate answer A ij The words are separated and then formed into a state vector s and input into TEF to obtain evidence information, as shown in Formula 15. When processing the input, this paper uses a pre-trained language model (PLMs) to process the text and uses the hidden state of the last hidden layer as evidence extracted from the text.

[0051] C t =PLMs(s) (15)

[0052] Sub-steps 3-4, construct the decision layer, as shown in formula 16, and finally the evidence C from the text t , Evidence C from Fig. g And evidence from path C p After splicing, it is sent to the decision layer to calculate the score and obtain the final prediction result.

[0053] ρ(Q i , A ij )=W mlp ([C p :C t :C g ])+b mlp (16)

[0054] Among them, W mlp and b mlp is the model parameter to be learned, [:] is the concatenation function, ρ(Q i , A ij ) is the question Q i Option A ij score;

[0055] Substep 3-5, construct a loss function layer. This layer uses cross entropy loss as the training loss function of the model, as shown in Formula 17. During the training phase, this method maximizes the correct answer by minimizing the cross entropy loss. score;

[0056]

[0057] Substeps 3-6, joint training loss. This example uses random initialization to initialize all parameters. During the training process, the Adam optimizer is used for gradient backpropagation to update the model parameters. The initial learning rate is set to 0.001. When the training loss no longer decreases or the number of training rounds exceeds 50, the model training ends and the model with the best performance on the validation set is saved.

[0058] Compared with the prior art, the advantages of the present invention are as follows: 1) The present invention uses a path evidence aggregator based on the generative model GPT-2, and uses the GPT-2 model's good performance in generation tasks to explore the deep paths and implicit relationships between different entity nodes, thereby enhancing the utilization of knowledge; 2) The present invention uses a graph evidence aggregator based on a graph reasoning network (GRN), which can collect the most direct and effective evidence from the knowledge graph, and integrate the relationship information between entities into the update of entity representation, thereby improving the representation of graph evidence from another level; 3) The present invention uses a text evidence aggregator based on a pre-trained language model, which can capture the tiny but important semantic and contextual relationships between different words in the same sentence as evidence from text, as a supplement to the above two evidence aggregators. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A method framework diagram of an embodiment of the present invention.

[0060] Figure 2 A flow chart of a method according to an embodiment of the present invention

[0061] Figure 3 Schematic diagram of data flow of the evidence aggregator in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to deepen the recognition and understanding of the present invention, the present invention is further illustrated below in conjunction with specific embodiments.

[0063] Example 1: See Figure 1-Figure 3 , a method for obtaining answers based on knowledge graph, the method comprising the following steps:

[0064] Step 1: Based on the given question Q i and its candidate answer A ij , this method requires i Extract n keywords from From A ij Extract m keywords And apply these keywords to the different evidence extractors described below. First, this method uses the Natural Language Toolkit (NLTK) to preprocess these texts, including word segmentation, part-of-speech tagging, word form restoration and other operations, and removes special characters such as stop words and punctuation marks in the sentences to achieve the effect of improving the quality of text features. Then this paper processes the words in the text through n-gram to generate word tuples, and uses underscores "_" to connect the words in the same tuple into combinations, and then evaluates the importance of these combinations in the sentence according to the TF-IDF algorithm (as shown in the following formula), and selects the top n and top m combinations of importance and splits them as the final keywords.

[0065]

[0066]

[0067] tfidf i,j =tf i,j ×idf i (3)

[0068] Where n i,j is a tuple t i In path d j The number of occurrences, ∑ k n k,j For path d j The sum of the number of triplets, |{j:t i ∈d j}| is a triplet t iThe total number of paths that appear, |D| is the total number of all paths.

[0069] Step 2. For the keyword information extracted in step 1, use the Dijkstra algorithm to find the path information of the two keyword corresponding nodes in the knowledge graph. The idea of ​​the Dijkstra algorithm is: suppose that the graph G = (V, E) is a directed graph, where V is the vertex set and E is the edge set. Divide the vertex set V in the graph into two parts. The first group is the vertex set for which the shortest path has been found (represented by S. Initially, there are only source points in S. After each shortest path is found, the vertex is added to S until all vertices are added to S and the algorithm ends). The second group is the remaining vertex set for which the shortest path has not been found (represented by U). The vertices in the second group are added to the first group in order of the length of the shortest path. The specific method is as follows:

[0070] (1) The entity nodes corresponding to the keywords in the knowledge graph are regarded as nodes in the directed graph, and the relationship between entities is regarded as the edge in the directed graph. The distance between adjacent nodes is 1;

[0071] (2) Select a question keyword as the source point v. Initially, there is only the source point in S, that is, S = {v}, and the distance of v is 0 (the distance to itself is 0). U contains all other vertices except v, and the distance between v and vertex u in U is 1 (if there is an edge from v to u) or ∞ (there is no edge from v to u);

[0072] (3) Select a vertex k with the smallest distance v from U and add it to S;

[0073] (4) Take k as the new intermediate point and modify the distances of each vertex in U. If the distance from the source point v through vertex k to vertex u is shorter than the original distance (not passing through vertex k), modify the distance of vertex u to be the distance of vertex k plus 1;

[0074] (5) Repeat (3) and (4) until all vertices are added to S.

[0075] (6) Repeat (2) to (5) until all question keyword nodes have completed the query of the shortest path to the option keyword nodes.

[0076] Step 3: Use the knowledge graph-based answer acquisition method processed in step 2 for training. The implementation of this step can be divided into the following sub-steps:

[0077] Sub-step 3-1, construct a path evidence aggregator. In order to solve the sparsity problem of the adjacency matrix, this method introduces PEF to find paths with fewer hops but more information. In addition, this method believes that the path evidence extracted directly from the knowledge graph is not optimal, because such a method knowledge is implemented by retrieving relevant information but does not explore the factual information stored in the knowledge graph. Compared with explicit relationship information, implicit relationships are used to describe the connection between two specific entities and cannot be directly retrieved from the knowledge graph. Therefore, this method uses PEF with GPT-2 as the core to obtain question-option keyword pairs. The optimal k (1≤k≤K) jump path p′ with implicit relations, and it is believed that GPT-2 can find more relevant structured evidence in static graphs. This method uses the triplets in the path to fine-tune the pre-trained GPT-2, and uses Byte-Pair Encoding (BPE) to convert the path into text sequence information, using the prefix "_" to represent the opposite relationship. These new paths can make full use of the information in the knowledge graph. When training the model for reasoning, the last word in the path should be input into the model at the beginning to indicate that the model should use this word as the end of the path. The calculation process is shown in Formula 4.

[0078] {E0, R′0, E1}=GPT-2({E0, R0, E1}) (4)

[0079] Among them, {E0, R0, E1} is the triple in the original path p, and {E0, R′0, E1} is the triple in the new path p′ containing implicit relations. The relation entities in the path are upgraded to implicit relations containing a lot of reasoning information. When generating the path {p1, p2, …, p n}, the attention mechanism is used to obtain the representation of the overall knowledge as the evidence aggregation obtained from the path. The attention calculation formula is shown in the following formula.

[0080] u i =tanh(W i p′ i +b i ) (5)

[0081]

[0082]

[0083] In the above formula, W i , b i is the learnable model parameter, p′ iis the newly generated path information, tanh(·) is the activation function, u i is the representation of the generated path, α i is the path weight, C p For evidence from the path.

[0084] Sub-step 3-2, constructing a graph evidence aggregator. This method uses a Bert-based vector representation method to initialize nodes. The specific initialization method is shown in the formula.

[0085]

[0086] where x i Represents the vector representation of node i.

[0087] As shown in Formula 9, this method uses linear transformation to calculate the value of the node Information flowing to node i through the path.

[0088]

[0089] Where W k are learnable model parameters, is the information transmitted to node i through the k-hop path, is the node attribute or hidden state of node j in layer l, Represents the set of starting points of the k-hop path from node i.

[0090] In the l+1th layer, the attention weight e ik It represents the importance and relevance of the information transmitted to node i through the k-hop path. This paper only considers the influence of this information for k≤K. In order to obtain the final attention weight, this method uses the softmax function to normalize all attention coefficients, as shown in formulas 10 and 11.

[0091]

[0092]

[0093] Where W attn are learnable model parameters, s is the state vector, is the information transmitted to node i through the k-hop path, e ik is the attention coefficient, β ik is the attention weight.

[0094] Here, this method uses the attention mechanism to measure s and The correlation between them and the corresponding weight β ikAs shown in Formula 12, this paper believes that the state vector s contains relevant information about the problem and options. A larger weight indicates that the information is closer to the current option and can provide more convincing information for reasoning.

[0095]

[0096] Among them, m i represents the information after aggregation, β ik is the attention weight.

[0097] Next, as shown in Formula 13, this method transmits the information m to node i i The hidden state of the current layer is linearly combined to obtain the node representation of the next layer.

[0098]

[0099] in is the parameter learned by the model, and σ is the sigmoid activation function.

[0100] Finally, after the information is transmitted through the l+1 network, the final representation of each node can be obtained. As shown in Formula 14, this method finally obtains the graph-level evidence representation by inputting the attributes of all nodes into the pooling layer.

[0101]

[0102] Among them C g Represents the evidence results aggregated from the graph.

[0103] Sub-step 3-3, constructing a text evidence aggregator. This method first uses a lexical analyzer to analyze the question Q i and its candidate answer A ij The words are separated and then formed into a state vector s and input into TEF to obtain evidence information. As shown in Formula 15, when processing the input, this paper uses pre-trained language models (PLMs) to process the text and uses the hidden state of the last hidden layer as evidence extracted from the text.

[0104] C t =PLMs(S) (15)

[0105] Sub-steps 3-4, construct the decision layer. As shown in Formula 16, the final evidence C from the text t , Evidence C from Fig. g And evidence from path C p After splicing, it is sent to the decision layer to calculate the score and obtain the final prediction result.

[0106] ρ(Q i , A ij )=W mlp ([C p :C t :C g ])+b mlp (16)

[0107] Among them, W mlp and b mlp is the model parameter to be learned, [:] is the concatenation function, ρ(Q i , A ij ) is the question Q i Option A ij score.

[0108] Substep 3-5, construct a loss function layer, which uses cross entropy loss as the training loss function of the model. As shown in Formula 17, during the training phase, this method minimizes the cross entropy loss component to maximize the correct answer score.

[0109]

[0110] Substeps 3-6, joint training loss. This example uses random initialization to initialize all parameters. During the training process, the Adam optimizer is used for gradient backpropagation to update the model parameters. The initial learning rate is set to 0.001. When the training loss no longer decreases or the number of training rounds exceeds 50, the model training ends and the model with the best performance on the validation set is saved.

[0111] Step 4: Use the trained model to build an output model to obtain the answer. The output model does not need to build data in advance, but only needs the question to be answered as input. This method will first extract keywords, then use different evidence aggregators to perform evidence aggregation operations, and finally output the obtained options as the answer to the input question.

[0112] Based on the same inventive concept, an example of the present invention also provides an answer acquisition device based on a knowledge graph, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned answer acquisition method based on the knowledge graph when loaded into the processor.

[0113] It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

Claims

1. A method for obtaining answers based on knowledge graph, It is characterized in that The method comprises the following steps: Step 1: Based on the given question Q i and its candidate answer A ij , this method starts from Q i Extract n keywords from From A ij Extract m keywords And apply these keywords to different evidence extractors; Step 2: For the keyword information extracted in step 1, use the Dijkstra algorithm to find the path information of the two keyword corresponding nodes in the knowledge graph. Step 3: Train the answer acquisition method based on the knowledge graph. First, use the keywords extracted in step 1 and the path in step 2 to build three evidence aggregators to aggregate the evidence information from the path, graph, and text respectively. After splicing the evidence information obtained above, the selected score information is obtained and the answer is given. Finally, the cross entropy loss function is used to realize back propagation. Step 4: Obtain answers to the questions to be answered based on the best verification model obtained in step 3. For the questions to be answered, first process them according to steps 1 and 2, input the original question text, keywords and path into the answer acquisition method trained in step 3, and obtain the final answer information; Step 3: Use the knowledge graph-based answer acquisition method processed in step 2 for training. The implementation of this step is divided into the following sub-steps: Sub-step 3-1, construct a path evidence aggregator, using the decoder module stacking-based generation method GPT-2 as the core PEF to obtain question-option keyword pairs The optimal k, 1≤k≤K, jump path p′ with implicit relations, and it is believed that GPT-2 can find more relevant structured evidence in static graphs. The pre-trained GPT-2 is fine-tuned using the triples in the path, and the path is converted into text sequence information using byte pair encoding. The prefix "_" is used to represent the opposite relationship. When training the model for reasoning, the last word in the path should be input into the model at the beginning to indicate that the model should use this word as the end of the path. The calculation process is as follows: {BY 0 ,R′ 0 ,BY 1 }=GPT-2({E 0 ,R 0 ,BY 1 }) (4) where {E 0 , R 0 , E 1 } is the triple in the original path p, {E 0 , R′ 0 , E 1 } is a triple in the new path p′ containing implicit relations. The relation entities in the path are upgraded to implicit relations containing a lot of reasoning information. When generating the path {p 1 , p 2 ,…,p n }, the attention mechanism is used to obtain the representation of the overall knowledge as the evidence aggregation obtained from the path. The attention calculation formula is shown in the following formula: u i =tanh(W i p′ i +b i ) (5) In the above formula, W i , b i is the learnable model parameter, p′ i is the newly generated path information, tanh(·) is the activation function, u i is the representation of the generated path, α i is the path weight, C p For evidence from the path, Sub-step 3-2, build a graph evidence aggregator, select the Bert-based vector representation method to initialize the nodes. The specific initialization method is shown in the formula: where x i Represents the vector representation of node i, is the hidden state of node i at layer 0, As shown in Formula 9, a linear transformation is used at layer l to calculate the value from node The information flowing to node i through the path, Where W k are learnable model parameters, is the information transmitted to node i through the k-hop path, is the node attribute or hidden state of node j in layer l, represents the set of starting points of the k-hop path from node i, At the l+1th layer, the attention weight e ik It represents the importance and relevance of information transmitted to node i through the k-hop path. All attention coefficients are normalized using the softmax function, as shown in formulas 10 and 11. Where W attn are learnable model parameters, s is the state vector, is the information transmitted to node i through the k-hop path, e ik is the attention coefficient, β ik is the attention weight; The attention mechanism is used to measure s and The correlation between them and the corresponding weight β ik , as shown in formula 12, Among them, β ik is the attention weight, m i Represents the information after aggregation; As shown in Formula 13, the information m transmitted to node i i The hidden state of the current layer is linearly combined to obtain the node representation of the next layer; in is the parameter learned by the model, σ is the sigmoid activation function, m i Represents the information after aggregation; Finally, after the information is transmitted through the l+1 network, the final representation of each node can be obtained, as shown in Formula 14. By inputting the attributes of all nodes into the pooling layer, the graph-level evidence representation is finally obtained; Among them C g represents the evidence results aggregated from the graph, Sub-step 3-3, build a text evidence aggregator, first use the lexical analyzer to analyze the question Q i and its candidate answer A ij The words in it are separated, and then they are combined into a state vector s and input into TEF to obtain evidence information, as shown in Formula 15. When processing the input, this paper uses a pre-trained language model to process the text and uses the hidden state of the last hidden layer as evidence extracted from the text. C t =PLMs(s)(15) Where s is the state vector; Sub-steps 3-4, construct the decision layer, as shown in formula 16, and finally the evidence C from the text t , Evidence C from Fig. g And evidence from path C p After splicing, it is sent to the decision layer to calculate the score and obtain the final prediction result. ρ(Q i ,A ij )=W mlp ([C p :C:C g ])+b mlp (16) Among them, W mlp and b mlp is the model parameter to be learned, [:] is the concatenation function, ρ(Q i , A ij ) is the question Q i Option A ij score; Substep 3-5, construct a loss function layer. This layer uses cross entropy loss as the training loss function of the model, as shown in Formula 17. During the training phase, this method maximizes the correct answer by minimizing the cross entropy loss. score; Substeps 3-6, joint training loss, use random initialization to initialize all parameters, use Adam optimizer to perform gradient backpropagation to update model parameters during training, and set the initial learning rate to 0.

001. When the training loss no longer decreases or the number of training rounds exceeds 50, the model training ends and the best performing model on the validation set is saved.

2. According to the answer acquisition method based on knowledge graph according to claim 1, It is characterized in that Step 1 is as follows: First, the Natural Language Toolkit is used to preprocess the text, including word segmentation, part-of-speech tagging, and word form restoration operations, and special characters such as stop words and punctuation marks in the sentence are removed to improve the quality of text features. Then, this article processes the words in the text through n-gram to generate word tuples, and uses underscores "_" to connect words in the same tuple into combinations. Then, the TF-IDF algorithm is used to evaluate the importance of these combinations in the sentence, and the top n and top m combinations in importance are selected and split as the final keywords. The TF-IDF algorithm is: tfidf i,j =tf i,j ×idf i (3) where n i,j is a tuple t i The number of times it appears in path j, ∑ k n k,j For path d j The sum of the number of triplets, |{j:t i ∈d j }| is a tuple t i The total number of paths that appear, |D| is the total number of all paths.

3. According to the answer acquisition method based on knowledge graph according to claim 1, It is characterized in that Step 2: The idea of ​​Dijkstra's algorithm is as follows: (1) The entity nodes corresponding to the keywords in the knowledge graph are regarded as nodes in the directed graph, and the relationship between entities is regarded as the edge in the directed graph. The distance between adjacent nodes is 1; (2) Select a question keyword as the source point v. Initially, there is only the source point in S, that is, S = {v}. The distance of v is 0, and the distance to itself is 0. U contains all other vertices except v. If there is an edge from v to u, the distance between v and vertex u in U is 1. If there is no edge from v to u, the distance between v and vertex u in U is ∞. (3) Select a vertex k with the smallest distance v from U and add it to S; (4) Take k as the new intermediate point and modify the distances of each vertex in U. If the distance from the source point v to the vertex u through vertex k is shorter than the distance without passing through vertex k, modify the distance of vertex u to be the distance of vertex k plus 1. (5) Repeat (3) and (4) until all vertices are added to S; (6) Repeat (2) to (5) until all question keyword nodes have completed the query of the shortest path to the option keyword nodes.

Citation Information

Cited By

  • Method and server for generating, on basis of language model, questions of personality aptitude test by using question and answer network

    US20240331811A1