Question and answer method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to a question and answer method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a target question, and determining a reference document corresponding to the target question from a plurality of alternative documents; obtaining a knowledge graph corresponding to the target question, and determining a target sub-graph corresponding to the target question in the knowledge graph; generating node reference information matched with nodes in the target sub-graph according to the reference document, and generating relation reference information matched with relations in the target sub-graph; and inputting the target sub-graph, the node reference information, the relation reference information and the target question into a question and answer model to obtain a target answer corresponding to the target question. By adopting the method, the accuracy of the target answer can be improved.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a question-answering method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] In the field of information retrieval, question-answering systems are an advanced form of information retrieval. They utilize natural language processing to understand the intended meaning of user-entered questions, extract knowledge from a knowledge base, and return the corresponding answer to the user. Question-answering systems can use question-answering corpora, knowledge graphs, public and private knowledge bases, and network knowledge as knowledge bases. Compared to search engines, question-answering systems offer more flexible question-answering knowledge bases, more accurate responses, easier deployment, and a superior question-answering experience.

[0003] Related technologies include using a large language model as the question-answering model in a question-answering system and providing input to the large language model through retrieval-augmented generation (RAG). This technology queries an external knowledge base for multiple documents related to the user's question, then feeds both the document and the question into the large language model to generate an answer.

[0004] However, the large language model has a limitation on input length. The current retrieval enhancement generation technology cannot input all retrieved documents into the large language model, resulting in most documents being discarded. As a result, the information referenced by the large language model when giving answers is not comprehensive enough and the accuracy is not high. Summary of the Invention

[0005] Based on this, it is necessary to provide a question-answering method, apparatus, computer device, storage medium and computer program product to address the above technical issues.

[0006] In a first aspect, the present application provides a question-answering method. The method comprises:

[0007] Obtaining a target question and determining a reference document corresponding to the target question from a plurality of candidate documents;

[0008] Obtain a knowledge graph corresponding to the target question, and determine a target subgraph corresponding to the target question in the knowledge graph;

[0009] generating node reference information matching nodes in the target subgraph according to the reference document, and generating relationship reference information matching relationships in the target subgraph;

[0010] The target subgraph, the node reference information, the relationship reference information and the target question are input into a question-answering model to obtain a target answer corresponding to the target question.

[0011] In one embodiment, determining a reference document corresponding to the target question from a plurality of candidate documents includes:

[0012] Perform word segmentation processing on the target question to obtain target matching words;

[0013] A reference document is determined from the candidate documents according to the relevance between each target matching word and each candidate document, and the relevance between the target question and each candidate document.

[0014] In one embodiment, determining a reference document from each candidate document based on the relevance between the target matching term and each candidate document, and the relevance between the target question and each candidate document, includes:

[0015] Determining a first reference document from each candidate document according to the occurrence frequency of each target matching word in each candidate document;

[0016] Determining a word vector for each of the target matching words, fusing the word vectors to obtain a target word vector, and determining a second reference document from each of the candidate documents based on a similarity between the target word vector and the text vector of each of the candidate documents;

[0017] Determining a third reference document from each of the candidate documents based on the similarity between the text vector of the target question and the text vector of each of the candidate documents;

[0018] A reference document is determined according to the first reference document, the second reference document, and the third reference document.

[0019] In one embodiment, determining the target subgraph corresponding to the target question in the knowledge graph includes:

[0020] Performing entity extraction on the target question to obtain target matching entities;

[0021] For any of the target matching entities, based on the similarity between the target matching entity and the entities corresponding to each node in the knowledge graph, determine the target node corresponding to the target matching entity from the knowledge graph, and obtain the nodes in the knowledge graph that have a direct connection relationship or an indirect connection relationship with the target node to obtain a subgraph corresponding to the target matching entity;

[0022] According to the subgraph corresponding to each of the target matching entities, the target subgraph corresponding to the target question in the knowledge graph is determined.

[0023] In one embodiment, generating node reference information matching a node in the target subgraph according to the reference document includes:

[0024] Extracting node information matching each node in the target subgraph from the reference document, wherein the node information is used to describe the entity corresponding to the node;

[0025] For any node in the target subgraph, a reference node is determined from the nodes in the target subgraph that have a direct or indirect connection relationship with the node, and node reference information corresponding to the node is generated based on the node information corresponding to each reference node and the node information of the node.

[0026] In one embodiment, generating relationship reference information matching the relationship in the target subgraph includes:

[0027] Extracting, from the reference document, relationship information matching each relationship in the target subgraph, the relationship information being used to describe the cause of the relationship;

[0028] For any of the relationships in the target subgraph, the relationship strength information corresponding to the relationship is determined according to the relationship information corresponding to the relationship, and the relationship reference information corresponding to the relationship is generated according to the relationship information and the relationship strength information.

[0029] In one embodiment, inputting the target subgraph, the node reference information, the relationship reference information, and the target question into a question-answering model to obtain a target answer corresponding to the target question includes:

[0030] Obtaining multiple previous round questions corresponding to the target question and the previous round answers corresponding to each of the previous round questions;

[0031] Summarize the questions and answers of the previous rounds to obtain summary information of the previous rounds;

[0032] The previous round summary information, the target subgraph, the node reference information, the relationship reference information and the target question are input into the question-answering model to obtain a target answer corresponding to the target question.

[0033] In a second aspect, the present application further provides a question-answering device. The device comprises:

[0034] A first determination module is configured to obtain a target question and determine a reference document corresponding to the target question from a plurality of candidate documents;

[0035] A second determination module is used to obtain the knowledge graph corresponding to the target question and determine the target subgraph corresponding to the target question in the knowledge graph;

[0036] a generating module, configured to generate node reference information matching nodes in the target subgraph and relationship reference information matching relationships in the target subgraph based on the reference document;

[0037] An input module is used to input the target subgraph, the node reference information, the relationship reference information and the target question into the question-answering model to obtain a target answer corresponding to the target question.

[0038] In one embodiment, the first determining module is further configured to:

[0039] Perform word segmentation processing on the target question to obtain target matching words;

[0040] A reference document is determined from the candidate documents according to the relevance between each target matching word and each candidate document, and the relevance between the target question and each candidate document.

[0041] In one embodiment, the first determining module is further configured to:

[0042] Determining a first reference document from each candidate document according to the occurrence frequency of each target matching word in each candidate document;

[0043] Determining a word vector for each of the target matching words, fusing the word vectors to obtain a target word vector, and determining a second reference document from each of the candidate documents based on a similarity between the target word vector and the text vector of each of the candidate documents;

[0044] Determining a third reference document from each of the candidate documents based on the similarity between the text vector of the target question and the text vector of each of the candidate documents;

[0045] A reference document is determined according to the first reference document, the second reference document, and the third reference document.

[0046] In one embodiment, the second determining module is further configured to:

[0047] Performing entity extraction on the target question to obtain target matching entities;

[0048] For any of the target matching entities, based on the similarity between the target matching entity and the entities corresponding to each node in the knowledge graph, determine the target node corresponding to the target matching entity from the knowledge graph, and obtain the nodes in the knowledge graph that have a direct connection relationship or an indirect connection relationship with the target node to obtain a subgraph corresponding to the target matching entity;

[0049] According to the subgraph corresponding to each of the target matching entities, the target subgraph corresponding to the target question in the knowledge graph is determined.

[0050] In one embodiment, the generating module is further configured to:

[0051] Extracting node information matching each node in the target subgraph from the reference document, wherein the node information is used to describe the entity corresponding to the node;

[0052] For any node in the target subgraph, a reference node is determined from the nodes in the target subgraph that have a direct or indirect connection relationship with the node, and node reference information corresponding to the node is generated based on the node information corresponding to each reference node and the node information of the node.

[0053] In one embodiment, the generating module is further configured to:

[0054] Extracting, from the reference document, relationship information matching each relationship in the target subgraph, the relationship information being used to describe the cause of the relationship;

[0055] For any of the relationships in the target subgraph, the relationship strength information corresponding to the relationship is determined according to the relationship information corresponding to the relationship, and the relationship reference information corresponding to the relationship is generated according to the relationship information and the relationship strength information.

[0056] In one embodiment, the input module is further configured to:

[0057] Obtaining multiple previous round questions corresponding to the target question and the previous round answers corresponding to each of the previous round questions;

[0058] Summarize the questions and answers of the previous rounds to obtain summary information of the previous rounds;

[0059] The previous round summary information, the target subgraph, the node reference information, the relationship reference information and the target question are input into the question-answering model to obtain a target answer corresponding to the target question.

[0060] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above methods when executing the computer program.

[0061] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above methods when executed by a processor.

[0062] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements any of the above methods when executed by a processor.

[0063] The above-mentioned question-answering method, device, computer equipment, storage medium and computer program product obtain multiple reference documents according to the target question, and determine the target subgraph corresponding to the target question in the pre-constructed knowledge graph, and extract reference information related to the nodes and relationships in the target subgraph from the reference document; and then input the target subgraph, the extracted reference information and the target question into the question-answering model to obtain the target answer corresponding to the target question. The embodiment of the present application adopts the knowledge graph as the knowledge framework that the question-answering model needs to refer to, and enhances the information carried by the knowledge graph by extracting information related to the nodes and relationships in the knowledge graph from the reference document, while taking advantage of the characteristics of the knowledge graph having clear logical relationships and rich information in the reference document. It can solve the problem that when the reference document is used as the reference knowledge of the question-answering model alone, the reference document is too long to be fully input into the question-answering model, and only part of the information of the reference document is input and the large language model cannot capture the overall logic of the document. It can also solve the problem that when the knowledge graph is used as the reference knowledge of the question-answering model alone, the knowledge graph carries too little information through the nodes and their relationships. It can provide the question-answering model with clear and rich information in the logical relationship, thereby improving the accuracy of the answer generated by the question-answering model. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 1 is a flow chart of a question-answering method according to an embodiment;

[0065] Figure 2 102 is a flow chart of step 102 in one embodiment;

[0066] Figure 3 204 is a flow chart of step 204 in one embodiment;

[0067] Figure 4 104 is a flow chart of step 104 in one embodiment;

[0068] Figure 5106 is a flow chart of step 106 in one embodiment;

[0069] Figure 6 106 is a flow chart of step 106 in one embodiment;

[0070] Figure 7 108 is a flow chart of step 108 in one embodiment;

[0071] Figure 8 1 is a flow chart of a question-answering method according to an embodiment;

[0072] Figure 9 Schematic diagram of a process for enhancing a target subgraph in one embodiment;

[0073] Figure 10 is a structural block diagram of a question-answering device in one embodiment;

[0074] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0076] In one embodiment, Figure 1 As shown, a question-answering method is provided. This embodiment uses the method applied to a server as an example for illustration. It is understandable that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0077] Step 102: Obtain a target question and determine a reference document corresponding to the target question from a plurality of candidate documents.

[0078] In the embodiment of the present application, the target question is the question input by the user into the question-answering system, and the alternative document is the document stored in the knowledge base of the question-answering system. The alternative document may include but is not limited to text, images, audio, video, etc. Information such as work orders handled by front-line operation and maintenance personnel, documents issued by the enterprise, important files, etc. can be collected regularly, and the above information can be updated to the knowledge base as alternative documents. In order to improve the quality of documents in the knowledge base, before updating a document to the knowledge base, the similarity between the document and the various alternative documents already in the knowledge base can also be determined, and the document will be updated to the knowledge base only when the similarity is lower than a preset threshold.

[0079] After receiving the target question, the server can query the knowledge base and obtain reference documents related to the target question from each alternative document. Exemplarily, the target question can be converted into a text vector, and each alternative document can also be converted into a text vector. Then, the similarity between the text vector of the target question and the text vector of the alternative document is calculated, and a preset number of alternative documents with the highest similarity are selected from each alternative document as reference documents. For alternative documents in the form of images, the text vector of the alternative document can be obtained by generating a description text for the image and then converting the description text into a text vector. For alternative documents in the form of audio or video, the audio in these alternative documents can be converted into text, and then the text can be converted into a text vector to obtain the text vector of the alternative document.

[0080] Alternatively, the target question may be segmented to obtain multiple target matching words, and then the frequency of occurrence of each target matching word in different candidate documents may be calculated, and a preset number of candidate documents with the highest frequency of occurrence may be selected as reference documents. In addition to the above two methods, other methods may be used to determine reference documents from candidate documents, which are not specifically limited in the present embodiment.

[0081] Step 104: obtain the knowledge graph corresponding to the target question and determine the target subgraph corresponding to the target question in the knowledge graph.

[0082] In the embodiment of the present application, a knowledge graph represents a knowledge domain. For example, a knowledge graph can be constructed for wired networks, a knowledge graph can be constructed for wireless networks, etc. The knowledge graph corresponding to the target problem is the knowledge graph corresponding to the domain to which the target problem belongs.

[0083] The domain to which the target question belongs can be determined by training a classification model. For example, a text vector can be trained with the target question as input. Based on the text vector, a classification model can be used to classify the target question into various domains with corresponding knowledge graphs. This classification model can then be used to determine the domain to which the target question belongs, and thus the knowledge graph corresponding to the target question.

[0084] The target subgraph corresponding to the target question in the knowledge graph is a subgraph consisting of a subset of nodes in the knowledge graph related to the target question and the relationships between these nodes. Since each node in the knowledge graph has a corresponding entity, the nodes in the knowledge graph related to the target question can be identified by determining the similarity between the text vector of the target question and the text vectors corresponding to these entities. Furthermore, the relationships between these nodes can be obtained to obtain the target subgraph.

[0085] Step 106 : Generate node reference information that matches the nodes in the target subgraph according to the reference document, and generate relationship reference information that matches the relationships in the target subgraph.

[0086] In an embodiment of the present application, since the target subgraph carries limited information through the relationships between nodes, the reference documents obtained from the knowledge base can be used to enhance the information of each node and relationship in the target subgraph, so that the question-answering model can refer to more information when generating answers.

[0087] Node reference information is information related to a node in a reference document. For example, entity extraction can be performed on the reference document, and then the entity corresponding to the node can be converted into a text vector. The extracted entity in the reference document can also be converted into a text vector. By calculating the similarity between the text vectors, it can be determined whether the entity corresponding to the node has appeared in the reference document. If so, the portion of the reference document related to the entity (e.g., the sentence or paragraph containing the entity) can be obtained and used as the node reference information for the node. Alternatively, the portion can be summarized using a large language model or a pre-trained neural network model, and the summarized information can be used as the node reference information for the node.

[0088] Relationship reference information is information related to a relationship in a reference document. For example, after performing entity extraction on the reference document, it can be determined whether any two nodes with a relationship in the target subgraph both appear in the reference document. If so, the portion of the reference document related to the two entities (e.g., sentences or paragraphs where the two entities appear simultaneously) can be obtained and used as the relationship reference information for the relationship between the two entities. Alternatively, this portion can be summarized using a large language model or a pre-trained neural network model, and the resulting information can be used as the relationship reference information for the relationship.

[0089] Step 108: Input the target subgraph, node reference information, relationship reference information and target question into the question-answering model to obtain the target answer corresponding to the target question.

[0090] In the embodiment of the present application, after enhancing the information contained in the target subgraph, the target subgraph, the enhanced information, and the target question are input into the question-answering model, so that the question-answering model outputs the target answer. The question-answering model is constructed based on the large language model, and the question-answering model can be obtained by fine-tuning the large language model based on the corpus corresponding to the domain of the question-answering system.

[0091] Since the input of a large language model is generally natural language, the target subgraph needs to be converted into natural language before being input into the question-answering model. For example, the target subgraph can be converted into a list of triplets.

[0092] The prompt words that are input into the large language model can be constructed based on the target subgraph, node reference information, relationship reference information, and target question. The prompt words can include a description of the target question, a description of the target subgraph and reference information, and a description of the task that the question-answering model needs to answer the target question based on the target subgraph and its reference information. An exemplary prompt word can be as follows:

[0093] “Task Description:

[0094] Given a question and a knowledge graph, the answer to the question is generated based on the knowledge graph. The knowledge graph is given in the form of triples, with the format of <entity, entity, relationship>. The brackets after the triple carry more information about the triple.

[0095] Question: Target question;

[0096] Knowledge Graph:

[0097] <Entity 1, Entity 2, Relationship 1> (node ​​reference information of entity 1, node reference information of entity 2, relationship reference information of relationship 1);

[0098] <Entity 2, Entity 3, Relationship 2> (node ​​reference information of entity 2, node reference information of entity 3, relationship reference information of relationship 2).”

[0099] After constructing the prompt word, it is input into the question-answering model to obtain the target answer. The server can then display the target answer to the user.

[0100] The question-answering method provided in the embodiment of the present application obtains multiple reference documents according to the target question, determines the target subgraph corresponding to the target question in the pre-constructed knowledge graph, extracts reference information related to the nodes and relationships in the target subgraph from the reference document, and then inputs the target subgraph, the extracted reference information and the target question into the question-answering model to obtain the target answer corresponding to the target question. The embodiment of the present application adopts the knowledge graph as the knowledge framework that the question-answering model needs to refer to, and enhances the information carried by the knowledge graph by extracting information related to the nodes and relationships in the knowledge graph from the reference document, while taking advantage of the characteristics of the clear logical relationship of the knowledge graph and the rich information of the reference document. It can solve the problem that when the reference document is used as the reference knowledge of the question-answering model alone, the reference document is too long to be fully input into the question-answering model, and only part of the information of the reference document is input and the large language model cannot capture the overall logic of the document. It can also solve the problem that when the knowledge graph is used as the reference knowledge of the question-answering model alone, the knowledge graph carries too little information through the nodes and their relationships. It can provide the question-answering model with clear and rich information in the logical relationship, thereby improving the accuracy of the answer generated by the question-answering model.

[0101] In one embodiment, Figure 2 As shown, in step 102, a reference document corresponding to the target question is determined from a plurality of candidate documents, including:

[0102] Step 202: segment the target question to obtain target matching words;

[0103] Step 204 : Determine a reference document from each candidate document based on the correlation between each target matching word and each candidate document, and the correlation between the target question and each candidate document.

[0104] In an embodiment of the present application, the target question may be segmented to obtain multiple target matching words, and then a reference document may be determined from the candidate documents based on the relevance between each target matching word and the target question and the candidate documents.

[0105] The embodiments of this application do not specifically limit the word segmentation algorithm, and any method is applicable to the embodiments of this application. To prevent the word segmentation algorithm from splitting professional vocabulary into different words, when using a dictionary-based word segmentation algorithm, multiple professional vocabulary can be pre-set and added to the dictionary; when using a deep learning-based word segmentation algorithm, training samples containing professional vocabulary can be used to train the word segmentation model so that the word segmentation model can recognize professional vocabulary as a single word.

[0106] After obtaining the individual words output by the word segmentation algorithm, each word can be used as a target matching word. Furthermore, since the purpose of obtaining target matching words is to match relevant reference documents from candidate documents, after obtaining the individual words output by the word segmentation algorithm, each word can be further screened to obtain the target matching word that best represents the user's question intent represented by the target question and can produce the best matching effect.

[0107] For example, because nouns and verbs are more capable of representing the user's question intent, the part of speech of each word output by the word segmentation algorithm can be determined, and words with a part of speech of noun, verb, gerund, or user-created words can be used as target matching words. At the same time, considering that single-character words are less semantically clear than multi-character words, using these words as target matching words may result in poor matching results. Therefore, the number of characters contained in each word output by the word segmentation algorithm can be determined, and words with a character count greater than 1 can be used as target matching words. A stop word library can also be set. If a word output by the word segmentation algorithm matches a stop word in the stop word library, the word will not be used as a target matching word.

[0108] After obtaining the target matching term, the correlation between the target matching term and each candidate document can be determined. Combined with the correlation between the target question and each candidate document, a reference document can be determined from each candidate document. The correlation can be characterized by parameters such as semantic similarity and word frequency.

[0109] For example, you can count the number of times each target matching term appears in each candidate document and use that number as the correlation between the target matching term and the candidate document. Alternatively, you can use a text matching algorithm, such as the BM25L (Best Match 25 with Length Normalization) algorithm or the Tf-IDF (Term Frequency-Inverse Document Frequency) algorithm, to calculate the correlation between the target matching term and the candidate document.

[0110] The relevance between the target question and the candidate document may also be determined for each candidate document by calculating the similarity between the text vector of the target question and the text vector of the candidate document.

[0111] Then, the reference documents can be determined based on the correlation between the target matching words and the candidate documents, and the correlation between the target question and the candidate documents. For example, the candidate documents can be sorted from high to low according to the correlation between the target matching words and the candidate documents, and the candidate documents can be sorted from high to low according to the correlation between the target question and the candidate documents, and the candidate documents can be obtained as reference documents if they are ranked in the top preset positions (for example, the top 10) in the results obtained after the two sortings. That is, if an alternative document is ranked in the top preset positions in both sortings, the alternative document can be used as a reference document. Alternatively, the alternative documents ranked in the top preset positions in the results obtained after the two sortings can be taken separately, and these alternative documents can be used as reference documents. That is, if an alternative document is ranked in the top preset positions in either of the two sortings, the alternative document can be used as a reference document.

[0112] The question-answering method provided in the embodiments of the present application determines a reference document from the candidate documents based on the correlation between each target matching term in the target question and the candidate documents, as well as the correlation between the target question itself and each candidate document. Therefore, when determining the reference document, the relevance of the target question to the document at the sentence level and the relevance at the word level can be considered simultaneously, thereby improving the accuracy of the reference document determination.

[0113] In one embodiment, Figure 3As shown, in step 204, based on the correlation between the target matching word and each candidate document, and the correlation between the target question and each candidate document, a reference document is determined from each candidate document, including:

[0114] Step 302: determining a first reference document from each candidate document based on the frequency of occurrence of each target matching word in each candidate document;

[0115] Step 304: Determine the word vector for each target matching word, fuse the word vectors to obtain a target word vector, and determine a second reference document from each candidate document based on the similarity between the target word vector and the text vector of each candidate document.

[0116] Step 306 , determining a third reference document from the candidate documents based on the similarity between the text vector of the target question and the text vectors of the candidate documents;

[0117] Step 308: Determine a reference document based on the first reference document, the second reference document, and the third reference document.

[0118] In an embodiment of the present application, reference documents can be matched from various candidate documents using the word frequency of the target matching word, the word vector of the target matching word (that is, the text vector of the target matching word), and the text vector of the target question, and then the reference documents obtained by the three methods are screened to obtain the final reference document.

[0119] When determining the first reference document based on word frequency, a word frequency score can be calculated for a candidate document based on the number of times each target matching word appears in the candidate document. The candidate documents are then sorted from high to low according to the word frequency score, and the candidate document ranked at the top of a preset ranking is selected as the first reference document. The word frequency score can be obtained by adding the number of times each target matching word appears, or it can be obtained using an algorithm such as the BM25L algorithm or the Tf-IDF algorithm, which is not specifically limited in this embodiment of the application.

[0120] When determining the second reference document based on the word vector, the word vector of each target matching word can be generated separately by any natural language processing model, and then the word vectors can be fused. The fusion method can be to find the average value of each word vector, to sequentially splice each word vector, etc., and the embodiment of the present application does not specifically limit this. By calculating the similarity between the target word vector obtained after fusion and the text vector of the candidate document, the word vector score of the candidate document can be obtained, and then the candidate documents are sorted from high to low according to the word vector score, and the candidate document ranked in the top preset ranking (the preset ranking may be different from the preset ranking for the word frequency score) is selected as the second reference document.

[0121] When determining the second reference document based on the target question, the text vector of the target question can be generated through any natural language processing model. By calculating the similarity between the text vector of the target question and the text vector of the alternative document, the text vector score of the alternative document can be obtained, and then the alternative documents are sorted from high to low according to the text vector score, and the alternative document ranked in the top preset position (this preset position may be different from the previous preset position for word frequency score and word vector score) is selected as the third reference document.

[0122] After obtaining the first, second, and third reference documents, the final reference document can be further determined. For example, the intersection of the first, second, and third reference documents can be used as the final reference document. Specifically, if a candidate document is simultaneously the first, second, and third reference documents, then the candidate document is the final reference document. Alternatively, the union of the first, second, and third reference documents can be used as the final reference document. Specifically, if a candidate document is any one of the first, second, and third reference documents, then the candidate document is the final reference document.

[0123] The question-answering method provided in the embodiment of the present application determines reference documents from candidate documents in multiple steps based on the word frequency and word vector of the target matching word, as well as the text vector of the target text, to obtain the final reference document. When determining the reference document, the relevance of the target question to the document at the sentence semantic vector level, the relevance to the document at the word semantic vector level, and the relevance to the document in terms of word frequency can be considered simultaneously, thereby improving the accuracy of determining the reference document.

[0124] In one embodiment, Figure 4 As shown, in step 104, determining the target subgraph corresponding to the target question in the knowledge graph includes:

[0125] Step 402: Perform entity extraction on the target question to obtain target matching entities;

[0126] Step 404: For any target matching entity, based on the similarity between the target matching entity and the entities corresponding to each node in the knowledge graph, determine the target node corresponding to the target matching entity from the knowledge graph, and obtain the nodes in the knowledge graph that have a direct or indirect connection relationship with the target node to obtain the subgraph corresponding to the target matching entity;

[0127] Step 406: Determine the target subgraph corresponding to the target question in the knowledge graph based on the subgraph corresponding to each target matching entity.

[0128] In the embodiment of the present application, by determining the entities contained in the target question, a target subgraph corresponding to the target question in the knowledge graph is obtained. The embodiment of the present application does not specifically limit the method for extracting entities from the target question, and any entity extraction method is applicable to the embodiment of the present application.

[0129] After obtaining each target matching entity in the target question, the subgraph corresponding to each target matching entity can be retrieved from the knowledge graph. For a target matching entity, the subgraph corresponding to the target matching entity consists of the target node corresponding to the target matching entity, the nodes that have direct or indirect connections between the target nodes, and the relationships between these nodes.

[0130] The target node corresponding to the target matching entity can be obtained by calculating the similarity between the target matching entity and the entity corresponding to each node in the knowledge graph. For example, a similarity threshold can be set in advance. After calculating the similarity between the target matching entity and the entity corresponding to each node in the knowledge graph, if there is a node whose similarity to the target matching entity is greater than the similarity threshold, the node with the highest similarity among such nodes is selected as the target node corresponding to the target matching entity; if there is no node whose similarity is greater than the similarity threshold, it can be considered that the target matching entity has no corresponding target node in the knowledge graph, and accordingly, the target matching entity has no corresponding subgraph in the knowledge graph.

[0131] After obtaining the target node, all nodes that have a direct connection relationship with the target node (direct connection means that the node and the target node are directly connected through a relationship), as well as all nodes that have an indirect connection relationship with the target node (indirect connection means that the node is connected to another node that has a direct or indirect connection relationship with the target node) can be obtained. These nodes and the relationships between these nodes can be used as the subgraph corresponding to the target matching entity. Alternatively, these nodes can be further screened, for example, from the nodes that have a direct or indirect connection relationship with the target node, the nodes whose distance to the target node is within a preset number of hops (the distance refers to the number of nodes on the shortest path between the node and the target node) are determined, and these nodes are used as the relationships between these nodes as the subgraph corresponding to the target matching entity.

[0132] After obtaining the subgraphs corresponding to all target matching entities, we can integrate the subgraphs by taking the union of the subgraphs to obtain the target subgraph corresponding to the target problem in the knowledge graph.

[0133] In one embodiment, Figure 5 As shown, in step 106, generating node reference information matching the node in the target subgraph according to the reference document includes:

[0134] Step 502: extracting node information matching each node in the target subgraph from the reference document, where the node information is used to describe the entity corresponding to the node;

[0135] Step 504: for any node in the target subgraph, determine a reference node from the nodes in the target subgraph that have a direct or indirect connection relationship with the node, and generate node reference information corresponding to the node based on the node information corresponding to each reference node and the node information of the node.

[0136] In an embodiment of the present application, in addition to information related to the node itself, the node reference information may also include information related to nodes adjacent to the node, so as to enrich the node reference information.

[0137] After obtaining the target subgraph, the node information corresponding to each node can be extracted from the reference document. The node information extraction method can refer to the aforementioned embodiment, which involves performing entity extraction on the reference document, determining the entity corresponding to the node in the reference document, and then obtaining the description related to the entity. This embodiment of the application will not be repeated here.

[0138] Alternatively, this step can be accomplished using a large language model. This involves generating prompts for the large language model, including referencing the target subgraph, extracting entities and their corresponding descriptions from the reference document, and then inputting the prompts into the large language model to obtain node information output by the large language model. An example prompt might be as follows:

[0139] “Task Description:

[0140] Given a text document and a knowledge graph, refer to the part of the document content that is related to the entities in the knowledge graph and supplement the attributes of each entity in the knowledge graph according to the given task steps.

[0141] Task steps:

[0142] ① Extract the following information of each entity from the knowledge graph:

[0143] -Entity-name: Entity name

[0144] -Entity-type: Entity type, in the following format: [{Entity-type}]

[0145] ② Based on the content related to the entity name of each entity in the given text document, generate the following information for each entity:

[0146] -Entity-description: Information description of the entity

[0147] ③Output the information of steps ① and ② in the following format:

[0148] "Entity" {tuple format} <Entity-name> {tuple format} <Entity-type> {tuple format} <entity-description>{tuple format}"

[0149] After obtaining the node information for each node, a reference node for each node can be determined. For example, a reference node can be selected as a node that is within a preset number of hops from the node, or a node that has a specific relationship with the node or a relationship strength that meets requirements (the relationship strength can be evaluated using preset rules when the relationship is generated, or using a trained model).

[0150] Based on the node information of each reference node and the node information of each node, node reference information of the node can be generated. For example, the node information of the node and the node information of each reference node can be summarized to obtain the node reference information. This summary can be performed using a large language model or a trained natural language processing model.

[0151] The question-and-answer method provided in the embodiment of the present application generates node information for each node in the target subgraph respectively, and then generates node reference information of the node based on the node information of the node and the node information of the node's reference node. This allows the node reference information of each node to include information about other nodes associated with the node, thereby improving the richness of the node reference information and thereby improving the accuracy of the answers generated by the question-and-answer model based on the node reference information.

[0152] In one embodiment, Figure 6 As shown, in step 106, generating relationship reference information that matches the relationship in the target subgraph includes:

[0153] Step 602: extracting relationship information matching each relationship in the target subgraph from the reference document, where the relationship information is used to describe the cause of the relationship.

[0154] Step 604 : for any relationship in the target subgraph, determine the relationship strength information corresponding to the relationship according to the relationship information corresponding to the relationship, and generate the relationship reference information corresponding to the relationship according to the relationship information and the relationship strength information.

[0155] In the embodiment of the present application, the relationship reference information may include relationship strength information in addition to information related to the cause of the relationship. The relationship strength information is used to characterize the closeness of the connection between the entities represented by the two nodes, thereby enriching the relationship reference information.

[0156] After obtaining the target subgraph, the relationship information corresponding to each relationship can be extracted from the reference document. Referring to the aforementioned embodiment, the description of the portion related to the two entities corresponding to the relationship is obtained from the reference document. The portion of the reference document related to the relationship is then summarized using a pre-trained neural network model to obtain the cause of the relationship and score the relationship to obtain relationship strength information. A higher score indicates a closer connection between the entities represented by the two nodes.

[0157] Alternatively, this step can be accomplished using a large language model. A prompt word can be generated that includes the target subgraph that the large language model needs to reference, extracts the relationship and the corresponding cause of the relationship from the reference document, and then inputs the prompt word into the large language model to obtain the relationship information output by the large language model. An exemplary prompt word can be as follows:

[0158] "Task Description:

[0159] Given a text document and a knowledge graph, refer to the part of the document content that is related to the relationship in the knowledge graph and supplement the attributes of each relationship in the knowledge graph according to the given task steps.

[0160] Task steps:

[0161] ① Identify entity pairs that have relationships with each other from the knowledge graph and extract the following information about the entity pairs:

[0162] -Source_entity: the name of the head entity in the entity pair;

[0163] -Target_entity: the name of the tail entity in the entity pair;

[0164] ② Based on the content related to each entity pair in the given text document, extract the following information of the entity pair:

[0165] -Relationship_description: Explain why the head entity and the tail entity are related to each other;

[0166] -Relationship_strength: A numeric score indicating the strength of the relationship between the head entity and the tail entity;

[0167] ③Output the information of steps ① and ② in the following format:

[0168] ("relation" {tuple format}<Source_entity> {tuple format}<Target_entity> {tuple format}<Relationship_description> {tuple format}<Relationship_strength> )”

[0169] After obtaining the relationship information and relationship strength information of each relationship, the relationship information and the relationship strength information can be used together as relationship reference information.

[0170] The question-answering method provided in the embodiment of the present application generates relationship information for each relationship in the target subgraph respectively, and then generates relationship reference information of the relationship based on the relationship information and relationship strength information of the relationship, which can improve the richness of the relationship reference information and thus improve the accuracy of the answers generated by the question-answering model based on the relationship reference information.

[0171] In one embodiment, Figure 7 As shown, in step 108, the target subgraph, node reference information, relationship reference information and target question are input into the question-answering model to obtain the target answer corresponding to the target question, including:

[0172] Step 702: Obtain multiple previous round questions corresponding to the target question, and the previous round answers corresponding to each previous round question;

[0173] Step 704: Summarize the questions and answers of each previous round to obtain summary information of the previous round;

[0174] Step 706: Input the previous round summary information, target subgraph, node reference information, relationship reference information and target question into the question-answering model to obtain the target answer corresponding to the target question.

[0175] In an embodiment of the present application, in order to improve the performance of the question-answering model in multiple rounds of question-answering, previous round summary information can be generated based on the questions and answers in the previous rounds in the multiple rounds of question-answering, and then the previous round summary information and other information can be input into the question-answering model together so that the question-answering model outputs the target answer.

[0176] In current related technologies, multi-round question-answering in a large language model is typically implemented by adding a memory module. The memory module stores the text information of the questions and answers from the previous round. During the current round of question-answering, the information stored in the memory module is fed into the large language model along with the information from the current round. However, when the answers or questions in the previous round are long, due to the input word limit of the large language model, the information from the previous round will squeeze out the space for other input information (especially reference information obtained from reference documents). Therefore, the embodiments of the present application require compressing and summarizing the information from the previous round to enable more reference information to be input into the question-answering model.

[0177] A round of Q&A refers to a text pair consisting of a question input by the user and the answer output by the Q&A model. Previous rounds can be a preset number of rounds before the current round, such as the first four or three rounds. If the number of rounds before the current round is less than the preset number (for example, if the preset number is four and the current round is the first round), all previous rounds can be considered as previous rounds.

[0178] After each round of question-and-answering, the server may store text pairs consisting of the question and answer from that round. After the number of stored rounds reaches a preset number, the server deletes the oldest text pair from the currently stored text pairs each time it stores a new text pair, so that the server always stores text pairs from the preset number of rounds prior to the current round.

[0179] The server can further vectorize the above information and then store it to save server storage space. The vectorized information can be stored in the Qdrant vector database to facilitate subsequent further operations and reading of the information.

[0180] After the current round of Q&A begins, the server retrieves the questions (previous round questions) and answers (previous round answers) from the cache for each previous round. This information can then be summarized based on these questions and answers to obtain summary information for the previous round. This summary information can be obtained using any natural language processing model, or this step can be performed using a large language model. When this step is performed using a large language model, an exemplary prompt word might be as follows:

[0181] "Task Description:

[0182] Given the questions and answers from the first three rounds, merge the same questions and answers from these three rounds and summarize the merged questions and answers.

[0183] After obtaining the summary information of the previous round, the summary information of the previous round as well as the target subgraph, node reference information, relationship reference information and target question are input into the question-answering model to obtain the target answer corresponding to the target question.

[0184] The question-answering method provided in the embodiment of the present application summarizes the questions and answers of the previous rounds in multiple rounds of question-answering, which can enable the question-answering model to have the ability to remember multiple rounds of conversations while streamlining the input length required for this ability. This allows the saved input space to be allocated more to reference information, thereby improving the richness of the information that the question-answering model can reference, and thereby improving the accuracy of the answers generated by the question-answering model.

[0185] In one embodiment, Figure 8 As shown, a question-answering method is provided, including:

[0186] S1, establish a private domain database.

[0187] The private database is used to store candidate documents. These candidate documents are sourced from public databases, work orders processed by frontline operations and maintenance personnel, group-issued documents, and important files. To ensure data currency, the private database can be updated monthly. To maintain the quality of the private database, data with a high degree of similarity to existing candidate documents is deleted during updates.

[0188] S2, accepts the target question and searches for reference documents.

[0189] First, the target question is segmented using the Jieba tokenizer. The results are then evaluated for part of speech. If a segmented result is not a user-defined word, common noun, gerund, or common verb, it is filtered out. If a segmented result appears in a stop word, it is also filtered out. If a segmented result is just a single word, it is also filtered out. After filtering, the remaining words are used as target matching words.

[0190] The BM25L algorithm is used to calculate the relevance score between each target matching term and candidate documents, and the three candidate documents with the highest relevance scores are recalled as reference documents. The SBERT (Sentence-BERT) pre-trained language model is used to vectorize each target matching term. Average pooling is performed on each target matching term to obtain the final target word vector. The similarity between the target word vector and the candidate documents is calculated, and the three candidate documents with the highest similarity are recalled as reference documents. The SBERT model is used to vectorize the target question, and the similarity between the target question text vector and the candidate documents is calculated. The three candidate documents with the highest similarity are recalled as reference documents. The three reference documents obtained in each of the three steps above are then combined as the final reference document.

[0191] S3, enhances the nodes and relationships in the target subgraph with reference documents.

[0192] See also Figure 9 As shown, in the embodiment of the present application, the knowledge graph is stored in the ES (Elasticsearch) database. First, the entity can be identified from the target question, and then the target subgraph corresponding to the target question can be obtained by obtaining the nodes associated with the entity in the target question from the knowledge graph. Then, the prompt words for the large language model are constructed, and the nodes and relationships in the target subgraph are enhanced based on the large language model. The enhanced object is the relationship between each two nodes ( Figure 9 Entity-relationship-relationship in entity), and the attribute value of the entity corresponding to each node ( Figure 9 Specific enhancement steps can be found in the above embodiments, which will not be described in detail in the embodiments of this application.

[0193] S4, memory enhancement for multiple rounds of question answering.

[0194] Among them, a large language model is used to summarize the summary information of the previous rounds in multiple rounds of question and answer. The specific summarization method can be found in the aforementioned embodiment, and will not be repeated in the embodiments of this application.

[0195] S5: Input the summary information of the previous round, the target subgraph and its reference information into the question-answering model built based on the large language model to obtain the target answer.

[0196] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0197] Based on the same inventive concept, the present application also provides a question-answering device for implementing the question-answering method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more of the following question-answering device embodiments can be found in the limitations of the question-answering method described above and will not be repeated here.

[0198] In one embodiment, Figure 10 As shown, a question-answering device 1000 is provided, comprising: a first determination module 1002, a second determination module 1004, a generation module 1006, and an input module 1008, wherein:

[0199] A first determining module 1002 is configured to obtain a target question and determine a reference document corresponding to the target question from a plurality of candidate documents;

[0200] The second determination module 1004 is configured to obtain a knowledge graph corresponding to the target question and determine a target subgraph corresponding to the target question in the knowledge graph;

[0201] A generating module 1006 is configured to generate node reference information matching nodes in the target subgraph and relationship reference information matching relationships in the target subgraph based on the reference document;

[0202] The input module 1008 is used to input the target subgraph, the node reference information, the relationship reference information and the target question into the question-answering model to obtain a target answer corresponding to the target question.

[0203] The question-answering device provided in the embodiment of the present application obtains multiple reference documents according to the target question, determines the target subgraph corresponding to the target question in the pre-constructed knowledge graph, extracts reference information related to the nodes and relationships in the target subgraph from the reference document, and then inputs the target subgraph, the extracted reference information and the target question into the question-answering model to obtain the target answer corresponding to the target question. The embodiment of the present application adopts the knowledge graph as the knowledge framework that the question-answering model needs to refer to, and enhances the information carried by the knowledge graph by extracting information related to the nodes and relationships in the knowledge graph from the reference document, while taking advantage of the characteristics of the knowledge graph having clear logical relationships and rich information in the reference document. It can solve the problem that when the reference document is used as the reference knowledge of the question-answering model alone, the reference document is too long to be fully input into the question-answering model, and only part of the information of the reference document is input and the large language model cannot capture the overall logic of the document. It can also solve the problem that when the knowledge graph is used as the reference knowledge of the question-answering model alone, the knowledge graph carries too little information through the nodes and their relationships. It can provide the question-answering model with clear and rich information in the logical relationship, thereby improving the accuracy of the answer generated by the question-answering model.

[0204] In one embodiment, the first determining module 1002 is further configured to:

[0205] Perform word segmentation processing on the target question to obtain target matching words;

[0206] A reference document is determined from the candidate documents according to the relevance between each target matching word and each candidate document, and the relevance between the target question and each candidate document.

[0207] In one embodiment, the first determining module 1002 is further configured to:

[0208] Determining a first reference document from each candidate document according to the occurrence frequency of each target matching word in each candidate document;

[0209] Determining a word vector for each of the target matching words, fusing the word vectors to obtain a target word vector, and determining a second reference document from each of the candidate documents based on a similarity between the target word vector and the text vector of each of the candidate documents;

[0210] Determining a third reference document from each of the candidate documents based on the similarity between the text vector of the target question and the text vector of each of the candidate documents;

[0211] A reference document is determined according to the first reference document, the second reference document, and the third reference document.

[0212] In one embodiment, the second determining module 1004 is further configured to:

[0213] Performing entity extraction on the target question to obtain target matching entities;

[0214] For any of the target matching entities, based on the similarity between the target matching entity and the entities corresponding to each node in the knowledge graph, determine the target node corresponding to the target matching entity from the knowledge graph, and obtain the nodes in the knowledge graph that have a direct connection relationship or an indirect connection relationship with the target node to obtain a subgraph corresponding to the target matching entity;

[0215] According to the subgraph corresponding to each of the target matching entities, the target subgraph corresponding to the target question in the knowledge graph is determined.

[0216] In one embodiment, the generating module 1006 is further configured to:

[0217] Extracting node information matching each node in the target subgraph from the reference document, wherein the node information is used to describe the entity corresponding to the node;

[0218] For any node in the target subgraph, a reference node is determined from the nodes in the target subgraph that have a direct or indirect connection relationship with the node, and node reference information corresponding to the node is generated based on the node information corresponding to each reference node and the node information of the node.

[0219] In one embodiment, the generating module 1006 is further configured to:

[0220] Extracting, from the reference document, relationship information matching each relationship in the target subgraph, the relationship information being used to describe the cause of the relationship;

[0221] For any of the relationships in the target subgraph, the relationship strength information corresponding to the relationship is determined according to the relationship information corresponding to the relationship, and the relationship reference information corresponding to the relationship is generated according to the relationship information and the relationship strength information.

[0222] In one embodiment, the input module 1008 is further configured to:

[0223] Obtaining multiple previous round questions corresponding to the target question and the previous round answers corresponding to each of the previous round questions;

[0224] Summarize the questions and answers of the previous rounds to obtain summary information of the previous rounds;

[0225] The previous round summary information, the target subgraph, the node reference information, the relationship reference information and the target question are input into the question-answering model to obtain a target answer corresponding to the target question.

[0226] Each module in the above-mentioned apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0227] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 11 As shown. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a question-and-answer method.

[0228] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0229] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0230] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0231] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0232] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0233] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0234] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0235] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A question-answering method, characterized in that: The method comprises: Obtaining a target question, and determining a reference document corresponding to the target question from a plurality of candidate documents; Obtain a knowledge graph corresponding to the target question, and determine a target subgraph corresponding to the target question in the knowledge graph; Generate node reference information matching the nodes in the target subgraph according to the reference document, and generate relationship reference information matching the relationships in the target subgraph; The target subgraph, the node reference information, the relationship reference information and the target question are input into a question-answering model to obtain a target answer corresponding to the target question.

2. The method according to claim 1, characterized in that The step of determining a reference document corresponding to the target question from a plurality of candidate documents includes: Perform word segmentation processing on the target question to obtain target matching words; According to the correlation between each of the target matching words and each of the candidate documents, and the correlation between the target question and each of the candidate documents, a reference document is determined from each of the candidate documents.

3. The method according to claim 2, characterized in that Determining a reference document from each candidate document according to the correlation between the target matching word and each candidate document, and the correlation between the target question and each candidate document, includes: Determining a first reference document from each candidate document according to the occurrence frequency of each target matching word in each candidate document; Determine the word vector of each target matching word, fuse the word vectors to obtain a target word vector, and determine a second reference document from each candidate document according to the similarity between the target word vector and the text vector of each candidate document; Determining a third reference document from each of the candidate documents according to the similarity between the text vector of the target question and the text vector of each of the candidate documents; A reference document is determined according to the first reference document, the second reference document and the third reference document.

4. The method according to claim 1, characterized in that: Determining a target subgraph corresponding to the target problem in the knowledge graph includes: Performing entity extraction processing on the target question to obtain target matching entities; For any of the target matching entities, according to the similarity between the target matching entity and the entities corresponding to each node in the knowledge graph, determine the target node corresponding to the target matching entity from the knowledge graph, and obtain the nodes in the knowledge graph that have a direct connection relationship or an indirect connection relationship with the target node, and obtain the subgraph corresponding to the target matching entity; According to the subgraphs corresponding to the target matching entities, the target subgraph corresponding to the target question in the knowledge graph is determined.

5. The method according to claim 1 or 4, characterized in that: The step of generating node reference information matching a node in the target subgraph according to the reference document includes: Extracting node information matching each node in the target subgraph from the reference document, respectively, wherein the node information is used to describe an entity corresponding to the node; For any node in the target subgraph, a reference node is determined from the nodes in the target subgraph that have a direct or indirect connection relationship with the node, and node reference information corresponding to the node is generated based on the node information corresponding to each of the reference nodes and the node information of the node.

6. The method according to claim 1 or 4, characterized in that: The generating of the relation reference information matching the relation in the target subgraph comprises: Extracting relationship information matching each relationship in the target subgraph from the reference document, respectively, wherein the relationship information is used to describe the cause of the relationship; For any of the relationships in the target subgraph, the relationship strength information corresponding to the relationship is determined according to the relationship information corresponding to the relationship, and the relationship reference information corresponding to the relationship is generated according to the relationship information and the relationship strength information.

7. The method according to claim 1, characterized in that The step of inputting the target subgraph, the node reference information, the relationship reference information, and the target question into a question-answering model to obtain a target answer corresponding to the target question includes: Obtaining a plurality of previous round questions corresponding to the target question, and previous round answers corresponding to each of the previous round questions; Summarize and obtain summary information of the previous rounds according to the questions of the previous rounds and the answers of the previous rounds; The previous round summary information, the target subgraph, the node reference information, the relationship reference information and the target question are input into the question-answering model to obtain a target answer corresponding to the target question.

8. A question-answering device, characterized in that: The device comprises: A first determination module is used to obtain a target question and determine a reference document corresponding to the target question from a plurality of candidate documents; A second determination module is used to obtain the knowledge graph corresponding to the target question and determine the target subgraph corresponding to the target question in the knowledge graph; A generating module, configured to generate node reference information matching nodes in the target subgraph and generate relationship reference information matching relationships in the target subgraph according to the reference document; An input module is used to input the target subgraph, the node reference information, the relationship reference information and the target question into the question-answering model to obtain a target answer corresponding to the target question.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images