Multi-document question and answer retrieval method combining generative language model and semantic document atlas

Abstract

The invention discloses a multi-document question and answer retrieval method combining a generative language model and a semantic document atlas, which further perfects and optimizes the construction and traversal mode of a document knowledge atlas, and respectively designs a bert-class model-based atlas construction and a KGP3 algorithm-based retrieval traversal mode optimization. In the graph traversal process, a main problem and an obtained node are combined, a sub-problem related to next needed node information is generated, the most appropriate document node is accurately selected from neighbor nodes, the correlation between the node and an initial problem is judged, and if the node and the initial problem are not related, the node and the initial problem are not listed in a follow-up retrieval list. The method is efficient, has traceability and interpretability, and abstractly summarizes the whole process of selecting the next hop node in graph traversal by the model. In order to reduce the cost, the small encoder-decoder T5 model is improved, and a contrast learning mechanism is introduced into the model, so that the retrieval performance of the small model is further improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a multi-document question-answering retrieval method, and specifically to a multi-document question-answering retrieval method that combines a generative language model with a semantic document graph. Background Art

[0002] Currently, large language models have been widely used in various fields, especially in question answering (QA) tasks. However, despite the power of these models, they have never seen content outside of pre-training data and are prone to hallucinations. Especially in open domains, the importance of retrieval-enhanced large models has become increasingly prominent. The output of such models can be constructed based on real and reliable external knowledge.

[0003] Previous retrieval-enhanced large-model methods usually adopt a one-shot retrieval strategy, that is, only using user input to retrieve relevant knowledge. For simple questions, this single retrieval method can effectively meet information needs and is suitable for single-hop question answering or single-hop fact verification. However, for complex information needs, such as multi-hop reasoning and long-form question answering, a single retrieval is obviously not enough. In order to solve complex multi-hop tasks, recent studies have improved the effect by collecting knowledge multiple times throughout the answering process. For example, using intermediate information generation or using predecessor search information as query conditions. Such methods perform well in multi-hop question answering tasks, but the knowledge base they rely on is often based on the traditional heuristic BM25 method or only uses deep dense embedding, resulting in problems such as high retrieval repetition rate and insufficient diversity. In addition, although the knowledge graph (KGQA) method based on triple construction has potential, it is time-consuming and difficult to scale in triple extraction.

[0004] In summary, the problems existing in the prior art are:

[0005] 1. Limitations of knowledge bases. Existing knowledge bases mostly rely on traditional BM25 methods or deep dense embedding, resulting in high duplication and insufficient diversity of retrieval results.

[0006] 2. Difficulty in constructing knowledge graphs. The triple-based knowledge graph (KGQA) method is time-consuming and difficult to scale in triple extraction, which limits its practical application.

[0007] 3. Dependence on multiple knowledge collections. Although the effect is improved through multiple knowledge collections, the quality and diversity of the basic knowledge base are still not resolved. Summary of the invention

[0008] In view of the above-mentioned problems existing in the prior art, the present invention provides a multi-document question-answering retrieval method combining a generative language model with a semantic document graph. The method further improves and optimizes the construction and traversal methods of the document knowledge graph, respectively designs a graph construction based on a bert-like model, and designs a KGP3 algorithm to optimize the retrieval traversal method. In the process of graph traversal, the present invention combines the main question and the obtained nodes to generate a sub-question involving the next required node information, accurately selects the most suitable document node from the neighboring nodes, and judges the relevance of the node to the initial question. If it is not relevant, it will not be included in the subsequent retrieval list. This method is not only efficient and traceable and explainable, but also abstractly summarizes the overall process of the model selecting the next hop node in the graph traversal. In order to reduce costs, the present invention also improves the small encoder-decoder T5 model, and introduces a comparative learning mechanism in the model, which further improves the retrieval performance of the small model.

[0009] The objective of the present invention is achieved through the following technical solutions:

[0010] A multi-document question answering retrieval method combining a generative language model with a semantic document graph comprises the following steps:

[0011] Step 1: Construction of document-based knowledge graph:

[0012] Step 1.1, document library acquisition: obtain a document library containing a large number of relevant documents in a certain field;

[0013] Step 1.2, text segmentation: After obtaining the document library, perform word segmentation on the document and divide the document into independent blocks;

[0014] Step 1.3: Embed the document chunks based on the pre-trained language model: Use the pre-trained language model to extract the semantic features of each document chunk;

[0015] Step 1.4, calculate similarity: calculate the similarity between blocks based on the document block embedding vector to form edges. For each document node, connect the neighboring document nodes based on the similarity threshold and top-k algorithm to form a document knowledge graph;

[0016] Step 2: Document knowledge graph multi-hop retrieval generative question answering based on generative language model:

[0017] Step 21, generating an initial candidate set: using the user's question as the query condition, perform a TF-IDF algorithm search in the document library obtained in step 1 to obtain a set of nodes related thereto, i.e., the initial candidate set;

[0018] Step 22: Iterate and expand the document set:

[0019] Step 221, traversing the initial node set: traversing each node in the initial candidate set generated in step 21, and taking it as the initial node of the current path;

[0020] Step 222: Input the question and the retrieved nodes into the generative language model: Input the question and the node information on the current path into the generative language model, and obtain the document knowledge graph multi-hop question answering based on the KGP3 algorithm. The specific steps are as follows:

[0021] (1) Generate sub-problems: Combine the main problem and the nodes that have been obtained to generate a sub-problem involving the information that the next required node should contain;

[0022] (2) Get the next hop node based on the generated sub-questions: Use the generated sub-questions to further retrieve new nodes in the document-based knowledge graph constructed in step 1. These nodes are the targets to be visited in the next step.

[0023] (3) Determine the relevance of the selected node: If the relevance of the node is judged to be "support" or "fuzzy", the node is added to the current path and the next hop selection is continued based on this node; if the relevance of the node is judged to be "rejected", the current node is skipped and the initial node set is continued to be traversed; support: means that the current node highly supports the seed node when answering the query; fuzzy: means that the current node supports the seed node to a certain extent; reject: means that there is no valuable information for the main question;

[0024] (4) Determine whether the current path length reaches a threshold: Check whether the current path length reaches a preset threshold. If not, continue to execute steps (1) to (3). If it reaches the threshold, add the document of the current path to the retrieved document set.

[0025] (5) Determine whether the initial node set has been traversed completely: Check whether the initial node set has been completely traversed. If not, continue to traverse the initial node set; if the traversal is complete, end the entire search process and obtain the search document set;

[0026] The generative language model is a generative large language model or a small encoder-decoder model T5 model. When the generative language model is a small encoder-decoder model T5 model, it is fine-tuned, and negative samples are added to each fine-tuning data while introducing a contrastive learning head: the input of the T5 model includes a question and related retrieval evidence, where each candidate node represents a potential relevant information fragment. Through the encoder of the T5 model, these inputs are converted into a rich continuous vector representation and then sent to the decoder to generate or predict text. During training, two negative samples are provided for each positive sample to form a comparison between positive and negative samples;

[0027] Step 23: After step 21 and step 22, the retrieved document set is subjected to a deduplication post-processing step to obtain a final retrieved document set, which is then incorporated into the user question and input into the generative language model to obtain an answer response.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The present invention proposes the KGP3 method to further improve the retrieval process of the model. This method is efficient and traceable and explainable. It also abstractly summarizes the overall process of the model in selecting the next hop node during graph traversal. It has certain universality, and the present invention can easily simulate this common scenario to construct a fine-tuning data set to further enhance the retrieval ability of the model. In addition, the present invention fine-tunes the T5 model, and introduces a contrast learning head based on its encoder-decoder architecture and the characteristics of the traversal method of the KGP3 method. This method encourages the model to select the correct document by adding contextual comparison targets, while avoiding the selection of other easily confused documents, thereby enhancing the judgment ability of the model. Finally, it is connected to WeChat to realize the application of robot question and answer. This method reduces the loss of manpower and material resources and promotes the high-quality development of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the improved T5 model architecture of the present invention;

[0031] Figure 2 Schematic diagram of the KGP3 algorithm flow chart designed for the present invention. DETAILED DESCRIPTION

[0032] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0033] The present invention provides a multi-document question-answering retrieval method combining a generative language model with a semantic document graph, comprising the following steps:

[0034] Step 1: Definition and construction of document-based knowledge graph.

[0035] Step 1.1, definition of document knowledge graph: Document knowledge graph is a method of converting text information into structured knowledge. It extracts the semantics, keywords and other information of the document and organizes it into the form of a graph so that machines can better understand and use this information.

[0036] Step 1.2: Construction of document-based knowledge graph:

[0037] Step 1.2.1, document library acquisition: First, there needs to be a document library containing a large number of relevant documents in a certain field, which is the basis for building a knowledge graph. Therefore, the present invention selects two public data sets for verification. Among them, HotpotQA is a multi-hop reasoning question and answering data set developed by researchers at Stanford University in the United States. This data set aims to promote the understanding and solution capabilities of complex problems in the field of natural language processing, especially those that require the integration of information from multiple sources to answer. HotpotQA requires the model to not only understand a single document, but also effectively integrate information from different documents to complete multi-step reasoning. MuSiQue (Multihop Questions via Single-hop Question Composition) is a multi-hop reasoning question and answering data set introduced by the Knowledge Engineering Research Laboratory of Tsinghua University. The purpose of the design of this data set is to overcome the problems that can be solved by shortcuts in existing data sets. MuSiQue adopts a bottom-up question synthesis method to enhance the logical relevance between questions by ensuring that the answer to each sub-question is necessary for the answer to subsequent questions.

[0038] Step 1.2.2, text segmentation: perform word segmentation on the document. After obtaining the document library, the document needs to be segmented to divide the long document into independent blocks (such as sentences, paragraphs or chapters). The purpose of this step is to break the document into smaller units that are easier to process and analyze. Word segmentation can use existing natural language processing (NLP) tools such as jieba (Chinese word segmentation) or NLTK (English word segmentation).

[0039] Text segmentation in this step, especially for the processing of long documents, is an important step in natural language processing (NLP). This process usually includes word segmentation and segmenting long documents into independent After the word segmentation, the long document is still a continuous text stream. In order to facilitate subsequent processing and analysis, it is usually necessary to further segment the document into independent blocks. The definition of blocks can be determined according to specific needs, such as segmenting by paragraphs, sentences or specific tags (such as titles, line breaks, etc.). The segmented blocks can be used as independent processing units for tasks such as information extraction and summary generation. In actual operation, text segmentation usually involves the following steps:

[0040] Step 1.2.2.1. Preprocessing: remove irrelevant characters in the text (such as punctuation marks, numbers, etc.) and perform necessary text cleaning.

[0041] Step 1.2.2.2, word segmentation: Use word segmentation tools or algorithms to segment the text.

[0042] Step 1.2.2.3, segmentation: Segment the segmented text into independent blocks according to the defined rules or tags.

[0043] Step 1.2.2.4, post-processing (optional): further clean or format the segmented text blocks.

[0044] Step 1.2.3: Embed document chunks based on pre-trained language models: Use pre-trained language models such as BERT to extract the semantic features of each document chunk for subsequent similarity calculations. The specific steps are as follows:

[0045] Step 1.2.3.1. Pre-trained language model: Language models (such as BERT) are pre-trained on large amounts of text data to learn general language representations. BERT is a typical example that processes input sequences through a Transformer architecture and is able to understand the meaning of words in context.

[0046] Step 1.2.3.2, document chunk embedding: Embed the document chunk based on the pre-trained language model. For a given piece of text or "document chunk", the embedding process can be expressed as shown in formula (1):

[0047] e i =Embed(chunk i ) (1)

[0048] Among them, e i is the embedding vector of document chunk i, Embed is the embedding function, usually a pre-trained deep neural network, chunk i Indicates a small unit block after a long text is segmented.

[0049] Step 1.2.3.3, semantic feature extraction: Use a pre-trained language model to convert it into a fixed-length vector. This process usually involves inputting the text into the language model and extracting the output feature vector from a layer of the language model (usually the last layer). The output feature vector obtained through the above process contains the semantic information of the original text. This means that the output feature vector not only reflects the literal meaning of the words, but also reflects more complex features such as their role in the sentence and the logical relationship between sentences, so as to facilitate subsequent similarity calculations.

[0050] Step 1.2.4, calculate similarity: calculate the similarity between document blocks based on the document block embedding vector to form edges. For each document node, connect the neighboring document nodes based on the similarity threshold and top-k algorithm to form a document knowledge graph, which actually describes the process of building a document graph. This process involves the following key steps:

[0051] Step 1.2.4.1. Document node representation.

[0052] First, each document is considered as a node in the graph. These document nodes are usually converted into vector representations through some embedding techniques (such as word embedding, sentence embedding, or paragraph embedding) for use in subsequent similarity calculations.

[0053] Step 1.2.4.2: Similarity calculation.

[0054] Use some similarity measure (such as cosine similarity, Euclidean distance, etc.) to calculate the similarity between each pair of document nodes. The choice of similarity measure depends on the nature of the embedding vector and the specific requirements of the task.

[0055] Taking cosine similarity as an example, suppose there are two document blocks A and B, and their embedding vectors are and Then the cosine similarity between them can be calculated by the following formula:

[0056]

[0057] in, is the dot product of vector A and vector B, It is the modulus of vector A and vector B. The range of cosine similarity is [-1,1], where 1 means that the two vectors are exactly the same (same direction and equal length), and 0 means that the two vectors are orthogonal (i.e. unrelated).

[0058] Step 1.2.4.3, neighbor node connection.

[0059] Based on the similarity calculation results, the neighboring document nodes are connected.

[0060] Step 1.2.4.4, similarity threshold.

[0061] First, set a similarity threshold. If the similarity between two document nodes exceeds this threshold, they are considered neighbors and connected by edges in the graph. Second, use the top-k algorithm: for each document node, find the k most similar document nodes and connect them as neighbors. Here k is a pre-set positive integer.

[0062] Steps 1.2.4.5: Form a document knowledge graph.

[0063] Through the above steps, the present invention constructs a network consisting of document nodes and edges connecting them. This network can be regarded as a document knowledge graph, in which nodes represent documents and edges represent similar relationships between documents.

[0064] Step 1.2.4.6, Subsequent applications.

[0065] After completing the above steps, the formed document knowledge graph can be used for various NLP tasks, such as document clustering, topic detection, information retrieval, recommendation systems, etc. By analyzing the nodes and edges in the graph, potential connections between documents can be discovered, revealing the internal structure of the data.

[0066] Step 2, Multi-hop retrieval generative question answering for document knowledge graph based on generative language model.

[0067] Step 21, Generate the initial candidate set: First, the user's question needs to be used as a query condition to perform TF-IDF algorithm search in the document corpus to obtain a set of related nodes, that is, the initial candidate set.

[0068] TF-IDF algorithm search is a widely used weighting technique in information retrieval and text mining, used to evaluate the importance of a word for a document in a document set or corpus. The TF-IDF search process mainly includes the following steps:

[0069] Step 211, Preprocessing stage:

[0070] (1) Text tokenization: Tokenize each document in the document collection to obtain a list composed of words.

[0071] (2) Stop word processing: Remove stop words (such as common but meaningless words like "de", "shi", etc.) in the document.

[0072] (3) Lemmatization (optional): Restore the vocabulary to its basic form (such as restoring "running" to "run") to better handle morphological changes.

[0073] Step 212, Construct the term frequency matrix:

[0074] (1) Calculate the term frequency (TF): For each word, calculate the number of times it appears in each document and divide it by the total number of words in that document to obtain the TF of the word in that document. Its expression is shown as follows:

[0075]

[0076] (2) Construct a matrix where the rows represent documents and the columns represent vocabulary. Each element in the matrix represents the term frequency of the corresponding vocabulary in the corresponding document.

[0077] Step 213, Calculate the inverse document frequency (IDF):

[0078] For each word in the vocabulary, calculate its IDF. IDF reflects the general importance of a word. The higher the IDF value of a word, the more representative it is. Its expression is as follows:

[0079]

[0080] Note: The denominator is added with 1 to avoid the situation where the denominator is 0.

[0081] Step 214, calculate the TF-IDF value:

[0082] Multiply the TF of each word in each document by its IDF value to get the TF-IDF value of the word in the document. The expression is as follows:

[0083] TF-IDF(t, d)=TF(t,d0×IDF(t) (5)

[0084] Step 215, construct TF-IDF vector:

[0085] For each document, a vector is constructed using the TF-IDF values ​​of all its words. This vector can be regarded as the representation of the document in the vocabulary space.

[0086] Step 216, search and sort:

[0087] When a user enters a query, the query is also preprocessed by word segmentation and stop word processing. The TF-IDF value of each word in the query is calculated in the document collection (usually the IDF value of the word in the query is used, because the query itself is not a complete document). The similarity between the query vector and the document vector is calculated using methods such as cosine similarity. The documents are sorted according to the similarity score, and the document with the highest score is returned to the user as the search result.

[0088] Step 22: Iterate and expand the document set:

[0089] Step 221, traverse the initial node set: traverse each node in the initial node set and use it as the initial node of the current path.

[0090] Step 222: Input the question and the retrieved nodes into the generative large language model: Based on the KGP3 algorithm designed by the present invention, input the question and the node information on the current path into the generative language model to form a document knowledge graph multi-hop question answering. The specific implementation process is as follows:

[0091] (1) Generate sub-questions: Combine the main question and the nodes that have been obtained to generate a sub-question involving the information that the next node should contain. For example, when asked who is older, Bob Bryan or Mariaan de Swardt, and with information about Bob Bryan's age, our intuition is to ask Mariaan de Swardt's age. This intuition forms the basis for the generative language model's reasoning, enabling it to generate subsequent questions. The generative language model outputs the generated sub-questions to guide the selection of the next hop node.

[0092] (2) Get the next hop node based on the generated sub-question: Use the generated sub-question to further retrieve new nodes in the knowledge graph. These nodes are the targets to be visited in the next step. The specific method is as follows: After generating the sub-question, the generative language model uses this sub-question as reasoning information and carefully selects the next most suitable document node from the neighbors. In this process, the generative language model always needs to select an optimal node in the current path to facilitate the next step of exploration on this node. The node selected is the node that is most helpful in answering the initial question Q among the candidate neighbors.

[0093] (3) Determine the relevance of the selected node: In the step of selecting neighbor nodes, all neighbor nodes of the current node may not necessarily contain the information required to answer the question, or may even have no relevance. If this node is included in the retrieval results, it will interfere with the answer to the question to a certain extent, so it is necessary to determine the relevance of the selected node to the question. Specifically, the selected nodes are evaluated in the following three ways: support: indicating that the current node highly supports the seed node when answering the query; fuzzy: indicating that the current node supports the seed node to a certain extent; and rejection: indicating that there is no valuable information for the main question. If the relevance of the node is judged to be "support" or "fuzzy", the node is added to the current path and the next hop selection is continued based on this node. If the relevance of the node is judged to be "rejected", the current node is skipped and the initial node set is continued to be traversed. This method is efficient, traceable and explainable. It also abstractly summarizes the overall process of the generative language model in graph traversal to select the next hop node. It has a certain degree of universality and can easily simulate this common scenario to build a fine-tuning dataset to further improve the retrieval ability of the model.

[0094] (4) Determine whether the current path length reaches a threshold: Check whether the current path length reaches a preset threshold. If it does not reach the threshold, continue to execute steps (1) to (3); if it reaches the threshold, add the document of the current path to the retrieved document set.

[0095] (5) Determine whether the initial node set has been traversed completely: Check whether the initial node set has been completely traversed. If it has not been traversed completely, continue to traverse the initial node set; if it has been traversed completely, end the entire retrieval process.

[0096] After completing the above content, you can obtain the retrieval document set after the document knowledge graph multi-hop question and answer training.

[0097] In the present invention, the generative language model is a generative large language model or a small encoder-decoder model T5 model. When the generative language model is a small encoder-decoder model T5 model, it is fine-tuned, and the model is improved according to the architecture of its encoder-decoder and the characteristics of the traversal method of the KGP3 method, and negative samples are added to each fine-tuning data while introducing a contrastive learning head. This method encourages the model to select the correct document by adding contextual contrast targets, while avoiding selecting other easily confused documents, thereby enhancing the model's judgment ability. In this process, the input of the model includes a question and related retrieval evidence, where each candidate node represents a potential relevant information fragment. Through the encoder of T5, these inputs are converted into a rich continuous vector representation, and then fed into the decoder to generate or predict text. During training, two negative samples are provided for each positive sample. Negative samples are generated by replacing the correct node with an irrelevant node or by changing the judgment result (for example, changing "accept" to "reject"). This contrastive learning method helps the model to be more sensitive to the subtle differences between useful information and misleading information. In this way, a comparison between positive samples and negative samples is formed. In addition to the traditional cross entropy loss, the present invention also uses this contrastive learning method to further improve the performance of the model.

[0098]

[0099] Among them, formulas (5) and (6) represent the output probabilities of positive samples and negative samples, where y + and represents the decoder hidden state of the positive sample and k negative samples, respectively, W y and b y is a learnable parameter, and δ is a Sigmoid function used to convert the input into a value between 0 and 1. Formula (7) defines the contrast loss L cl , which is obtained by dividing the probability of a positive sample by the sum of the probabilities of all samples. r is a temperature hyperparameter used to adjust the steepness of the softmax function, and Avg(*) represents the average pooling function based on the length of the target sequence. Formula (8) represents the cross entropy loss L mim , which is used to measure the difference between the model prediction result and the actual label. Finally, formula (9) gives the total loss Ltotal , which is the contrast loss L_cl and the cross entropy loss L mim The weighted sum of , λ is the weight coefficient. In general, these formulas describe how a machine learning model can improve its performance by optimizing these two loss functions.

[0100] Step 23: After step 21 and step 22, the retrieved document set is subjected to post-processing steps such as deduplication to obtain a final retrieved document set, which is then incorporated into the user question and input into the large model to obtain an answer.

[0101] Example:

[0102] Answer the question: "Who is the father of the current husband of the famous British writer "Harry Potter"?

[0103] Initial situation:

[0104] Initial search results:

[0105] Document [1]: JK Rowling is a famous British author. Her masterpiece, the Harry Potter series, has sold over 500 million copies worldwide. Her current husband is Neil Murray.

[0106] Neighbor nodes in the graph:

[0107] Document [2]: Neil Murray is a Scottish anesthesiologist. He married JK Rowling in Edinburgh in 2001 and they have two children together.

[0108] Document [3]: Neil Murray's father, John Murray, is a retired professor at the University of Glasgow, and his mother, Eileen Anne Hoey, is an educator.

[0109] Document [4]: ​​Introduction to the history of the city of Edinburgh.

[0110] Document [5]: History of the development of medicine in Scotland.

[0111] Step 1: Subproblem generation and neighbor node selection

[0112] Sub-problem generation:

[0113] Based on the initial document [1], the first sub-question generated by KGP3 is: “Who is JK Rowling’s current husband?”

[0114] Neighbor node selection:

[0115] KGP3 passes the sub-question "Who is JK Rowling's current husband?" to the large model, which evaluates the relevance of neighbor nodes:

[0116] Document [2]: Because it mentions Neil Murray's marriage to JK Rowling.

[0117] Node Verification:

[0118] Document [2] is highly relevant and is marked as a supporting node because it directly answers the sub-question.

[0119] Generate new sub-problems:

[0120] Based on the information in document [2], KGP3 generates a new sub-question: "Who is Neil Murray's father?"

[0121] Neighbor nodes in the graph:

[0122] Document [3]: Neil Murray's father, John Murray, is a retired professor at the University of Glasgow.

[0123] Document [4]: ​​Introduction to the history of the city of Edinburgh.

[0124] Document [5]: History of the development of medicine in Scotland.

[0125] Step 2: Further neighbor node selection and verification

[0126] Neighbor node selection:

[0127] Based on the new sub-question "Who is Neil Murray's father?", KGP3 again evaluates the relevance of the neighbor nodes:

[0128] Document [3]: Because it mentions Neil Murray's father

[0129] Node Verification:

[0130] Document [3] is highly relevant and is marked as a supporting node because it directly answers the new sub-question.

[0131] The final answer is generated:

[0132] After the above steps, the corresponding final retrieval document set is obtained, and finally the final answer is generated by the large model responsible for document reading comprehension: "The father of the current husband of the famous British writer "Harry Potter" is John Murray."

Claims

1. A multi-document question answering retrieval method combining a generative language model and a semantic document graph, characterized in that The method comprises the following steps: Step 1: Construction of document-based knowledge graph: Step 1.1, document library acquisition: obtain a document library containing a large number of relevant documents in a certain field; Step 1.2, text segmentation: After obtaining the document library, perform word segmentation on the document and divide the document into independent blocks; Step 1.3: Embed the document chunks based on the pre-trained language model: Use the pre-trained language model to extract the semantic features of each document chunk; Step 1.4, calculate similarity: calculate the similarity between blocks based on the document block embedding vector to form edges. For each document node, connect the neighboring document nodes based on the similarity threshold and top-k algorithm to form a document knowledge graph; Step 2: Document knowledge graph multi-hop retrieval generative question answering based on generative language model: Step 21, generating an initial candidate set: using the user's question as the query condition, perform a TF-IDF algorithm search in the document library obtained in step 1 to obtain a set of nodes related thereto, i.e., the initial candidate set; Step 22: Iterate and expand the document set: Step 221, traversing the initial node set: traversing each node in the initial candidate set generated in step 21, and taking it as the initial node of the current path; Step 222: Input the question and the retrieved nodes into the generative language model: Input the question and the node information on the current path into the generative language model, and obtain the document knowledge graph multi-hop question answering based on the KGP3 algorithm. The specific steps are as follows: (1) Generate sub-problems: Combine the main problem and the nodes that have been obtained to generate a sub-problem involving the information that the next required node should contain; (2) Get the next hop node based on the generated sub-questions: Use the generated sub-questions to further retrieve new nodes in the document-based knowledge graph constructed in step 1. These nodes are the targets to be visited in the next step. (3) Determine the relevance of the selected node: If the relevance of the node is judged to be "supported" or "fuzzy", the node is added to the current path and the next hop selection is continued based on this node; if the relevance of the node is judged to be "rejected", the current node is skipped and the initial node set is continued to be traversed; Support: indicates that the current node highly supports the seed node when answering the query, Fuzzy: indicates that the current node supports the seed node to a certain extent, Reject: indicates that there is no valuable information for the main question; (4) Determine whether the current path length reaches a threshold: Check whether the current path length reaches a preset threshold. If not, continue to execute steps (1) to (3). If it reaches the threshold, add the document of the current path to the retrieved document set. (5) Determine whether the initial node set has been traversed completely: Check whether the initial node set has been completely traversed. If not, continue to traverse the initial node set; if the traversal is complete, end the entire search process and obtain the search document set; Step 23: After step 21 and step 22, the retrieved document set is subjected to a deduplication post-processing step to obtain a final retrieved document set, which is then incorporated into the user question and input into the generative language model to obtain an answer response.

2. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The step 1.2 comprises the following steps: Step 1.2.1, preprocessing: remove irrelevant characters in the text and perform necessary text cleaning; Step 1.2.2: Word segmentation: Use word segmentation tools or algorithms to segment the text; Step 1.2.3, segmentation: Segment the segmented text into independent blocks according to the defined rules or tags.

3. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 2, characterized in that The step 1.2 also includes the following steps: Step 1.2.4, post-processing: further clean or format the segmented blocks.

4. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The step 1.3 comprises the following steps: Step 1.2.3.

1. Pre-training language model: Pre-train the language model on a large amount of text data; Step 1.2.3.2, document chunk embedding: Embed the document chunk based on the pre-trained language model. For a given piece of text or "document chunk", the embedding process is expressed as: e i =Embed(chunk i ) Among them, e i is the embedding vector of document chunk i, Embed is the embedding function, chunk i Indicates a small unit block after the long text is segmented; Step 1.2.3.3, semantic feature extraction: extract the output feature vector from a certain layer of the language model.

5. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The step 1.4 comprises the following steps: Step 1.4.1, document node representation: Each document is regarded as a node in the graph, and these document nodes are converted into vector representations through embedding technology; Step 1.4.2, similarity calculation: use the similarity metric to calculate the similarity between each pair of document nodes; Step 1.4.3, neighbor node connection: connect neighbor document nodes based on similarity calculation results; Step 1.4.4, Similarity threshold: First, set a similarity threshold. If the similarity between two document nodes exceeds this threshold, they are considered neighbors and connected with edges in the graph. Second, use the top-k algorithm: for each document node, find the k most similar document nodes to it and connect them as neighbors. Step 1.4.5, forming a document knowledge graph: Through steps 1.4.1 to 1.4.4, a network consisting of document nodes and edges connecting them is constructed. This network is regarded as a document knowledge graph, in which nodes represent documents and edges represent similarity relationships between documents.

6. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The TF-IDF algorithm includes the following steps: Step 211, pre-processing stage: (1) Text segmentation: Each document in the document collection is segmented to obtain a list of words; (2) Stop word processing: remove stop words in the document; Step 212: construct a word frequency matrix: (1) Calculate the word frequency TF: For each word, calculate the number of times it appears in each document and divide it by the total number of words in the document to obtain the TF of the word in the document. The expression is as follows: (2) Construct a matrix where rows represent documents, columns represent words, and each element in the matrix represents the frequency of the corresponding word in the corresponding document; Step 213, calculate the inverse file frequency IDF: For each word in the vocabulary, calculate its IDF, which is expressed as follows: Step 214, calculate the TF-IDF value: Multiply the TF of each word in each document by its IDF value to get the TF-IDF value of the word in the document. The expression is as follows: TF-IDF(t,d)=TF(t,d)×IDF(t ) Step 215, construct TF-IDF vector: For each document, a vector is constructed using the TF-IDF values ​​of all its words. This vector is regarded as the representation of the document in the vocabulary space. Step 216, search and sort: When a user enters a query, the query is also preprocessed, the TF-IDF value of each word in the query in the document collection is calculated, the similarity between the query vector and the document vector is calculated, the documents are sorted according to the similarity score, and the document with the highest score is returned to the user as the search result.

7. The multi-document question-answering retrieval method combining a generative language model and a semantic document graph according to claim 6, characterized in that The step 211 further includes the following steps: (3) Lemma restoration: restoring vocabulary to its basic form to better handle morphological changes.

8. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The generative language model is a generative large language model or a small encoder-decoder model T5 model.

9. The multi-document question answering retrieval method combining a large language model and a semantic document graph according to claim 8, characterized in that When the generative language model is a small encoder-decoder model T5 model, it is fine-tuned, negative samples are added to each fine-tuning data, and a contrastive learning head is introduced: the input of the T5 model includes a question and related retrieval evidence, where each candidate node represents a potential piece of relevant information. Through the encoder of the T5 model, these inputs are converted into a rich continuous vector representation and then sent to the decoder to generate or predict text. During training, two negative samples are provided for each positive sample to form a comparison between positive and negative samples.

Images