Text enhancement multi-turn question answering method and system based on knowledge graph
By employing a knowledge graph-based text enhancement method that integrates structured and unstructured knowledge bases, and utilizing entity mapping and graph neural networks to optimize the multi-turn question-answering process, the system addresses the issues of low efficiency and resource waste in existing technologies, achieving a highly efficient and accurate multi-turn question-answering system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILIN UNIVERSITY
- Filing Date
- 2024-02-02
- Publication Date
- 2026-07-21
Smart Images
Figure CN117909481B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent question answering technology, and in particular relates to a text-enhanced multi-turn question answering method and system based on knowledge graphs. Background Technology
[0002] With societal development and changes in human-computer interaction methods, interest in and demand for question-answering models are increasing. Limited by the nature of digital computing machines, question-answering models read and parse text using vector nodes, or provide more accurate and convenient information services with the assistance of structured knowledge bases such as knowledge graphs. Multi-turn question answering refers to multi-turn interactive question-and-answer sessions between humans and robots or artificial intelligence systems, where each round of dialogue can contain multiple questions and answers. Considering that users need to communicate step-by-step or cannot accurately describe their needs in one go, most question-answering models are developing towards multi-turn question answering.
[0003] Currently, there are many multi-turn question answering models on the market. Their design typically involves constructing a candidate entity node graph based on the question-and-answer context, and then calculating the content of pronouns and missing words in the current question. This imputation-based approach limits the model's knowledge acquisition methods, and more importantly, it cannot infer the user's current intent from the context. Text-based multi-turn question answering models can only extract limited knowledge from unstructured knowledge bases, performing encoding matching and semantic similarity calculations based on the user's question, resulting in an excessively large retrieval scope and low efficiency. Furthermore, knowledge graph-based multi-turn question answering models on the market analyze the topic of the current question based on weights; these parameters are not only difficult to present to users but also difficult for developers to understand. On the other hand, structured and unstructured knowledge bases on the market each have their strengths and weaknesses. Structured knowledge bases can provide high accuracy and easy-to-read / write knowledge nodes, but require significant maintenance costs and often suffer from missing relationships. Unstructured knowledge bases are widely available on the internet, in media, and in forums, containing a wealth of knowledge with extremely high information mining value, but are difficult to read and contain errors. Currently, there is still a lack of cross-modal knowledge bases that truly integrate the two, making it difficult to leverage their respective strengths. To address this, we propose a text-enhanced multi-turn question-answering method and system based on knowledge graphs. Summary of the Invention
[0004] The purpose of this invention is to provide a text-enhanced multi-turn question answering method and system based on knowledge graphs, aiming to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A knowledge graph-based text-enhanced multi-turn question answering method includes the following steps:
[0007] Step S1: Read the questions and historical Q&A, and initialize an empty candidate entity node graph and JSON file;
[0008] Step S2: Select unprocessed issues. For each entity in the issue, use the entity mapping model to find its number in the knowledge graph and retrieve the entity's description document. Put the entity's number and description document into the candidate entity node graph.
[0009] Step S3: Retrieve the question from the description document of each entity node, extract candidate answer entities from the retrieved target statements, and put the candidate answer entities into the candidate entity node graph;
[0010] Step S4: Calculate the encoding of the question and candidate answer entities;
[0011] Step S5: Receive the calculated candidate answer entity codes, multiply the candidate answer entity codes with the question codes, sort the multiplication results, calculate and adjust the loss function based on the sorting results, select the largest multiplication result as the answer entity, and use the answer entity and the description document as the answer to the current question;
[0012] Step S6: Repeat steps S2-S5 until all questions are answered, and the process ends.
[0013] Furthermore, the specific operation of step S2 is as follows: use the pre-trained entity mapping model ELQ to read the entities in the question, find the entity's number in the knowledge graph, use the number as the key and the description document as the value to construct a dictionary, and store the dictionary in a file using JSON.
[0014] Furthermore, the specific operation of step S3 is as follows: input the question into the Elasticsearch retrieval tool, configure Elasticsearch to the local documentation, and then retrieve the documentation of all entity nodes in the candidate entity node graph; call the entity mapping model ELQ again to extract the entities in the description statement, and use the extracted entities as candidate answer entities; finally, construct forward and backward edges between the candidate answer entities and the entity nodes from which the candidate answer entities originate, and put them into the candidate entity node graph.
[0015] Furthermore, the knowledge distillation process in step S3 includes:
[0016] Training the teacher model: retain all candidate answer entities and do not accept training parameters;
[0017] Training the student model: Do not retain candidate answer entities, only retain expected answer entities, accept training parameters and pass the path of the teacher model as a parameter to the student model.
[0018] Furthermore, the specific operation of step S4 is as follows: input the question into the sequence prediction model LSTM to obtain the code of the question and the hidden parameters of LSTM; then use GCN graph neural network and HGCN graph neural network to calculate the code of the candidate answer entity respectively.
[0019] Furthermore, the GCN graph neural network has a dimension of 300, and the weight parameter W is multiplied by the hidden parameters of the sequence prediction model LSTM each time; the input of GCN is the entity encoding result of the previous training and the edge set R of the current entity node, and the entity encoding is adjusted and output through the weight parameter; wherein, if the new node does not have the previous encoding, it is initialized with Gaussian distribution encoding.
[0020] Furthermore, the dimension and weight parameters of the HGCN graph neural network are set the same as those of the GCN graph neural network. The HGCN graph neural network is used to process heterogeneous graphs. The construction of the heterogeneous graph includes: encoding the description document using the pre-trained large language model BERT to obtain BERT encoding, and inputting the edge set R of the entity nodes into the HGCN graph neural network using the BERT encoding.
[0021] A knowledge graph-based text-enhanced multi-turn question answering system includes:
[0022] The reading module is used to read questions and historical Q&A, and initialize an empty candidate entity node graph and a JSON file;
[0023] The knowledge collector is used to select unprocessed issues. For each entity in the issue, the entity mapping model is used to find its number in the knowledge graph and retrieve the entity's description document. The entity's number and description document are then placed into the candidate entity node graph.
[0024] The candidate answer retriever is used to retrieve the description document of each entity node for the question, extract candidate answer entities from the retrieved target statements, and put the candidate answer entities into the candidate entity node graph;
[0025] Graph neural network models are used to compute the encodings of question and candidate answer entities;
[0026] The answer selector receives the calculated candidate answer entity codes, multiplies the candidate answer entity codes by the question codes, sorts the multiplication results, calculates and adjusts the loss function based on the sorting results, selects the largest multiplication result as the answer entity, and uses the answer entity and the description document as the answer to the current question.
[0027] Furthermore, the specific steps for calculating the loss function and making adjustments based on the ranking results are as follows:
[0028] The loss function of the sequence prediction model LSTM is calculated by subtracting the largest multiplication result from the expected answer and then returned to the graph neural network model for parameter adjustment. Then, the documentation of the answer entity is searched, and forward and backward edges between the answer entity and the entity node from which the answer entity originates are constructed and added to the candidate entity node graph.
[0029] Compared with the prior art, the beneficial effects of the present invention are:
[0030] 1. This invention introduces the model and concept of cross-modal knowledge graphs (a type of knowledge graph that integrates multiple information sources, such as text, images, and videos; this invention uses text enhancement) into a multi-turn question-answering system. It fully leverages the strengths of both structured and unstructured knowledge bases. Even when faced with missing relationships in the knowledge graph, the system can still complete and answer questions using knowledge from the unstructured knowledge base. Furthermore, the question-answering system can autonomously retrieve and supplement information during the question-answering process, overcoming the limitations of question-answering history and making it easier to infer user intent.
[0031] 2. This invention redesigns the reasoning process of a multi-turn question-answering system, transforming the complex graph reasoning process into a linear reasoning process of retrieval, reasoning, and retrieval. This significantly improves both logic and presentation, and the information can be presented to the user in the documentation to verify the correctness of the reasoning path.
[0032] 3. This invention designs a knowledge distillation process for the question-answering system, controlling the number of nodes and reducing resource waste. While ensuring accuracy, it provides methods and ideas for reasoning optimization, preventing time consumption caused by too many reasoning paths and improving the interactivity of the question-answering system. Attached Figure Description
[0033] Figure 1 This is a flowchart of the present invention.
[0034] Figure 2 This is a system flowchart of the present invention.
[0035] Figure 3 This is a flowchart of the knowledge distillation process of the present invention.
[0036] Figure 4 This is an architecture diagram of the knowledge collector and candidate answer retrieval device in this invention.
[0037] Figure 5 This is an architecture diagram of the neural network model in this invention. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0039] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.
[0040] An embodiment of the present invention provides a text-enhanced multi-turn question-answering method based on knowledge graphs, comprising the following steps:
[0041] Step S1: The reading module reads the questions and historical Q&A, and initializes an empty candidate entity node graph and a JSON (JavaScript Object Notation, a lightweight data exchange format that can be understood as a data format stored in the form of a dictionary, used in this invention to store test items and their corresponding test results) file;
[0042] Step S2: The knowledge collector selects unprocessed questions. For each entity in the question, it uses the entity mapping model to find the number in the knowledge graph and retrieves the entity's description document. The entity's number and description document are then placed into the candidate entity node graph.
[0043] The specific operation of step S2 is as follows: use the pre-trained entity mapping model ELQ to read the entities in the question, find the entity's number in the knowledge graph, use the number as the key and the description document as the value to build a dictionary, and store the dictionary in a file using JSON.
[0044] Step S3: The candidate answer retriever retrieves the description document of each entity node for the question, extracts candidate answer entities from the retrieved target statements, and puts the candidate answer entities into the candidate entity node graph;
[0045] The specific operation of step S3 is as follows: input the question into the Elasticsearch retrieval tool, configure Elasticsearch to the local documentation, and then retrieve the documentation of all entity nodes in the candidate entity node graph; call the entity mapping model ELQ again to extract the entities in the description statement, and use the extracted entities as candidate answer entities; finally, construct forward and backward edges between the candidate answer entities and the entity nodes from which the candidate answer entities originate, and put them into the candidate entity node graph.
[0046] Step S4: The graph neural network model calculates the encoding of the question and candidate answer entities;
[0047] The specific operation of step S4 is as follows: The question is input into the sequence prediction model LSTM (a variant of a recurrent neural network used to process and model sequence data), yielding the question's encoding and the LSTM's hidden parameters (the weights and biases in the model, learned and adjusted during training but not directly exposed to the user. Hidden parameters connect the various layers of the neural network to capture complex features and patterns in the input data); then, GCN (a deep learning model for graph-structured data, used for graph analysis and node classification tasks) and HGCN (a graph neural network model for processing heterogeneous graph data. Compared to traditional graph neural networks, HGCN is specifically designed to handle heterogeneous graph structures with different types of nodes and edges) graph neural networks are used to compute the encodings of the candidate answer entities, respectively.
[0048] The GCN graph neural network has a dimension of 300, and the weight parameter W is multiplied with the hidden parameters of the sequence prediction model LSTM in each iteration. The input of GCN is the entity encoding result of the previous training and the edge set R of the current entity node. The entity encoding is adjusted and output through the weight parameter (which represents the different importance or influence assigned to each element and is used to adjust the data or calculation results). If the new node does not have the previous encoding, it is initialized with a Gaussian distribution encoding.
[0049] The dimensions and weight parameters of the HGCN graph neural network are set the same as those of the GCN graph neural network. The HGCN graph neural network is used to process heterogeneous graphs. The construction of the heterogeneous graph includes: encoding the description document using the pre-trained large language model BERT to obtain BERT encoding, and inputting the edge set R of the entity nodes into the HGCN graph neural network using the BERT encoding.
[0050] Step S5: The answer selector receives the calculated candidate answer entity codes, multiplies the candidate answer entity codes with the question codes, sorts the multiplication results, subtracts the largest multiplication result from the expected answer, calculates the loss function of the sequence prediction model LSTM, and returns it to the graph neural network model for parameter adjustment; then, it searches for the description document of the answer entity, constructs forward and backward edges between the answer entity and the entity node from which the answer entity originates, and puts them into the candidate entity node graph; the largest multiplication result is selected as the answer entity, and the answer entity and description document are used as the answer to the current question;
[0051] Step S6: Repeat steps S2-S5 until all questions are answered, and the process ends.
[0052] An embodiment of the present invention provides a knowledge distillation method for a question-answering system, comprising:
[0053] The training steps for the teacher model are as follows: train the question-answering system, retain all candidate answers during the process, and put all candidate answers into the candidate entity node graph; the model trained in this way consumes more resources but has higher accuracy.
[0054] The training steps for the student model are as follows: The question-answering system is trained, retaining only the expected answers and adding them to the candidate entity node graph; simultaneously, the teacher model is run, and the difference between the paths chosen by the teacher and student models is compared in the answer selector, serving as the loss function for the LSTM sequence prediction model. Through these two steps, the student model can maintain a high accuracy rate in the early stages of training.
[0055] Example 1: The steps for configuring the question-and-answer system environment are as follows:
[0056] (1) Install the Python language compilation environment;
[0057] (2) Use the pip install command to download the required Python packages;
[0058] (3) After all packages are installed successfully, test the network environment;
[0059] (4) The environment configuration is complete when the knowledge graph can be queried.
[0060] The training steps for the neural network part of the question-answering system are as follows:
[0061] (1) Set the neural network parameters. Generally, use the default values;
[0062] (2) Prepare the corpus. The data needed for training can be obtained from a large number of online dialogues. Download the data, package it into a .txt file, and upload it.
[0063] (3) Training the teacher model: Set to retain all candidate answer entities and set not to accept training parameters;
[0064] (4) Train the student model: Set it to not retain candidate answer entities, set it to accept training parameters and pass the path of the teacher model as a parameter to the student model;
[0065] (5) Wait for the training to complete and enter the test question on the user terminal.
[0066] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.
Claims
1. A text-enhanced multi-turn question-answering method based on knowledge graphs, characterized in that, Includes the following steps: Step S1: Read the questions and historical Q&A, and initialize an empty candidate entity node graph and JSON file; Step S2: Select unprocessed issues. For each entity in the issue, use the entity mapping model to find its number in the knowledge graph and retrieve the entity's description document. Put the entity's number and description document into the candidate entity node graph. Step S3: Retrieve the question from the description document of each entity node, extract candidate answer entities from the retrieved target statements, and put the candidate answer entities into the candidate entity node graph; Step S4: Calculate the encoding of the question and candidate answer entities; Step S5: Receive the calculated candidate answer entity codes, multiply the candidate answer entity codes with the question codes, sort the multiplication results, calculate and adjust the loss function based on the sorting results, select the largest multiplication result as the answer entity, and use the answer entity and the description document as the answer to the current question; Step S6: Repeat steps S2-S5 until all questions are answered, and the process ends.
2. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 1, characterized in that, The specific operation of step S2 is as follows: use the pre-trained entity mapping model ELQ to read the entities in the question, find the entity's number in the knowledge graph, use the number as the key and the description document as the value to build a dictionary, and store the dictionary in a file using JSON.
3. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 1, characterized in that, The specific operation of step S3 is as follows: input the question into the Elasticsearch retrieval tool, configure Elasticsearch to the local documentation, and then retrieve the documentation of all entity nodes in the candidate entity node graph; call the entity mapping model ELQ again to extract the entities in the description statement, and use the extracted entities as candidate answer entities; finally, construct forward and backward edges between the candidate answer entities and the entity nodes from which the candidate answer entities originate, and put them into the candidate entity node graph.
4. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 1, characterized in that, The knowledge distillation process in step S3 includes: Training the teacher model: retain all candidate answer entities and do not accept training parameters; Training the student model: Do not retain candidate answer entities, only retain expected answer entities, accept training parameters and pass the path of the teacher model as a parameter to the student model.
5. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 1, characterized in that, The specific operation of step S4 is as follows: input the question into the sequence prediction model LSTM to obtain the code of the question and the hidden parameters of LSTM; then use GCN graph neural network and HGCN graph neural network to calculate the code of the candidate answer entity respectively.
6. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 5, characterized in that, The GCN graph neural network has a dimension of 300, and the weight parameter W is multiplied by the hidden parameters of the sequence prediction model LSTM each time. The input of GCN is the entity encoding result of the previous training and the edge set R of the current entity node. The entity encoding is adjusted by the weight parameter and output. If the new node does not have the previous encoding, it is initialized with Gaussian distribution encoding.
7. The knowledge graph-based text-enhanced multi-turn question-answering method according to claim 5, characterized in that, The dimensions and weight parameters of the HGCN graph neural network are set the same as those of the GCN graph neural network. The HGCN graph neural network is used to process heterogeneous graphs. The construction of the heterogeneous graph includes: encoding the description document using the pre-trained large language model BERT to obtain BERT encoding, and inputting the edge set R of the entity nodes into the HGCN graph neural network using the BERT encoding.
8. A text-enhanced multi-turn question-answering system based on knowledge graphs, characterized in that: include: The reading module is used to read questions and historical Q&A, and initialize an empty candidate entity node graph and a JSON file; The knowledge collector is used to select unprocessed issues. For each entity in the issue, the entity mapping model is used to find its number in the knowledge graph and retrieve the entity's description document. The entity's number and description document are then placed into the candidate entity node graph. The candidate answer retriever is used to retrieve the description document of each entity node for the question, extract candidate answer entities from the retrieved target statements, and put the candidate answer entities into the candidate entity node graph; Graph neural network models are used to compute the encodings of question and candidate answer entities; The answer selector receives the calculated candidate answer entity codes, multiplies the candidate answer entity codes by the question codes, sorts the multiplication results, calculates and adjusts the loss function based on the sorting results, selects the largest multiplication result as the answer entity, and uses the answer entity and the description document as the answer to the current question.
9. The knowledge graph-based text-enhanced multi-turn question-answering system according to claim 8, characterized in that, The specific steps for calculating and adjusting the loss function based on the sorting results are as follows: The loss function of the sequence prediction model LSTM is calculated by subtracting the largest multiplication result from the expected answer and then returned to the graph neural network model for parameter adjustment. Then, the documentation of the answer entity is searched, and forward and backward edges between the answer entity and the entity node from which the answer entity originates are constructed and added to the candidate entity node graph.