A Multi-Document Question Answering Retrieval Method Combining Generative Language Model and Semantic Document Graph

Abstract

The present invention discloses a multi-document question-answering retrieval method combining a generative language model and a semantic document graph. The method further improves and optimizes the construction and traversal methods of the document knowledge graph, designs the construction of the graph based on a bert-like model, and designs the KGP3 algorithm to optimize the retrieval traversal method. During the graph traversal process, combined with the main question and the obtained nodes, sub-questions related to the information of the next required node are generated, the most suitable document node is accurately selected from the neighbor nodes, and the relevance of the node to the initial question is judged. If it is not relevant, it is not included in the subsequent retrieval list. This method is not only efficient but also traceable and interpretable, and at the same time abstracts and generalizes the overall process of the model selecting the next-hop node during graph traversal. In order to reduce costs, the small encoder-decoder T5 model is also improved, and a contrastive learning mechanism is introduced into the model to further improve the retrieval performance of the small model.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and relates to a multi-document question answering retrieval method, specifically to a multi-document question answering retrieval method combining a generative language model and a semantic document graph. Background Art

[0002] Currently, large language models have been widely used in various fields, especially showing significant progress in question answering (QA) tasks. However, despite their power, since they have not seen content outside the pre-trained data and are prone to hallucinations, especially in the open domain, the importance of retrieval-enhanced large models has become increasingly prominent. The output of such models can be constructed based on real and reliable external knowledge.

[0003] Previous retrieval-enhanced large model methods usually adopt a one-time retrieval strategy, that is, only using the user input to retrieve relevant knowledge. For simple questions, this single retrieval method can effectively meet the information needs and is suitable for single-hop question answering or single-hop fact verification. However, for complex information needs, such as multi-hop reasoning and long-form question answering, single retrieval is obviously insufficient. To solve complex multi-hop tasks, recent research has improved the effect through multiple knowledge collections during the entire answering process. For example, using intermediate information generation or using the precursor search information as a query condition. Such methods perform well in multi-hop question answering tasks, but the knowledge bases they rely on often are based on traditional heuristic BM25 methods or only use deep dense embeddings, resulting in problems such as high retrieval repetition rate and lack of diversity. In addition, the knowledge graph (KGQA) method based on triples has potential, but it is time-consuming and difficult to scale in triple extraction.

[0004] In summary, the problems existing in the prior art are as follows:

[0005] 1. Limitations of the knowledge base. Existing knowledge bases mostly rely on traditional BM25 methods or deep dense embeddings, resulting in a high retrieval result repetition rate and lack of diversity.

[0006] 2. Difficulties in knowledge graph construction. The knowledge graph (KGQA) method based on triples is time-consuming and difficult to scale in triple extraction, restricting its practical application.

[0007] 3. Dependence on multiple knowledge collections. Although the effect is improved through multiple knowledge collections, the quality and diversity problems of the basic knowledge base have not been solved yet. Summary of the Invention

[0008] In view of the above problems existing in the prior art, the present invention provides a multi-document question-answering retrieval method combining a generative language model and a semantic document graph. This method further improves and optimizes the construction and traversal of the document knowledge graph, designs a graph construction based on a bert-like model, and designs the KGP3 algorithm to optimize the retrieval traversal method. During the graph traversal process, the present invention combines the main question and the obtained nodes to generate a sub-question involving the information of the next required node, accurately selects the most suitable document node from the neighbor nodes, and judges the relevance of the node to the initial question. If it is not relevant, it is not included in the subsequent retrieval list. This method is not only efficient but also traceable and interpretable, and at the same time abstracts the overall process of the model selecting the next-hop node during the graph traversal. To reduce costs, the present invention also improves the small encoder-decoder T5 model and introduces a contrastive learning mechanism into the model to further improve the retrieval performance of the small model.

[0009] The object of the present invention is achieved by the following technical solutions:

[0010] A multi-document question-answering retrieval method combining a generative language model and a semantic document graph, comprising the following steps:

[0011] Step 1, construction of the document-based knowledge graph:

[0012] Step 1.1, document library acquisition: Obtain a document library containing a large number of relevant documents in a certain field;

[0013] Step 1.2, text cutting: After obtaining the document library, perform word segmentation on the documents to divide the documents into independent text blocks;

[0014] Step 1.3, embedding the document blocks based on a pre-trained language model: Use a pre-trained language model to extract the semantic features of each document block;

[0015] Step 1.4, calculating similarity: Calculate the similarity between the document blocks based on the document block embedding vectors to form edges, and connect the neighbor document nodes for each document node based on the similarity threshold and the top-k algorithm to form a document knowledge graph;

[0016] Step 2, multi-hop retrieval generative question-answering of the document knowledge graph based on the generative language model:

[0017] Step 21, generating an initial candidate set: Use the user's question as a query condition and perform TF-IDF algorithm search in the document library obtained in step 1 to obtain a set of related nodes, that is, the initial candidate set;

[0018] Step 22, iteratively expanding the document set:

[0019] Step 221. Traverse the initial node set: Traverse each node in the initial candidate set generated in Step 21 and use it as the initial node of the current path;

[0020] Step 222. Input the question and the retrieved node into the generative language model: Input the question and the node information on the current path into the generative language model, and obtain the multi-hop question answering of the document knowledge graph based on the KGP3 algorithm. The specific steps are as follows:

[0021] (1) Generate sub-questions: Combine the main question and the nodes that have been obtained to generate a sub-question regarding the information that the next required node should contain;

[0022] (2) Obtain the next-hop nodes based on the generated sub-questions: Use the generated sub-questions to further retrieve new nodes in the document-based knowledge graph constructed in Step 1. These nodes are the targets to be visited in the next step;

[0023] (3) Judge the relevance of the selected nodes: If the relevance of a node is judged as "support" or "ambiguous", add the node to the current path and continue to select the next hop based on this node; if the relevance of a node is judged as "reject", skip the current node and continue to traverse the initial node set; Support: indicates that the current node highly supports the seed node when answering the query, Ambiguous: indicates that the current node supports the seed node to a certain extent, Reject: indicates that there is no valuable information for the main question;

[0024] (4) Judge whether the length of the current path reaches the threshold: Check whether the length of the current path has reached the pre-set threshold. If it has not reached the threshold, continue to execute Steps (1) to (3); if it has reached the threshold, add the document of the current path to the retrieved document set;

[0025] (5) Judge whether the initial node set has been traversed completely: Check whether the initial node set has been completely traversed. If it has not been traversed completely, continue to traverse the initial node set; if it has been traversed completely, end the entire retrieval process and obtain the retrieved document set;

[0026] The generative language model is a generative large language model or the small encoder-decoder model T5 model. When the generative language model is the small encoder-decoder model T5 model, it is fine-tuned by adding negative samples to each fine-tuning data and introducing a contrastive learning head at the same time: The input of the T5 model includes a question and the retrieved evidence related to it. Each candidate node represents a potential relevant information segment. Through the encoder of the T5 model, these inputs are converted into a rich continuous vector representation and then fed into the decoder to generate or predict text. During training, two negative samples are provided for each positive sample to form a contrast between the positive sample and the negative sample;

[0027] Step 23: After Steps 21 and 22, perform a deduplication process on the retrieved document set to obtain the final retrieved document set, and incorporate the user's question into it and input it into the generative language model to obtain an answer reply.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The present invention proposes the KGP3 method for further improving the retrieval process of the model. This method is efficient, traceable, and interpretable. At the same time, it abstractly generalizes the overall process of the model selecting the next-hop node in graph traversal, and has a certain universality. Moreover, the present invention can conveniently simulate this general scenario to construct a fine-tuning data set to further improve the retrieval ability of the model. In addition, the present invention fine-tunes the T5 model, and in view of the characteristics of its encoder-decoder architecture and the traversal method of the KGP3 method, a contrastive learning head is introduced. This method encourages the model to select the correct document and avoid selecting other easily confused documents by adding a context contrast objective, thereby strengthening the judgment ability of the model. Finally, it is connected to WeChat to realize the practical application of robot question answering. This method reduces the consumption of manpower and material resources and promotes the high-quality development of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the improved T5 model architecture of the present invention;

[0031] Figure 2 Schematic diagram of the KGP3 algorithm process designed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The technical solutions of the present invention will be further described below with reference to the accompanying drawings, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention shall be covered by the protection scope of the present invention.

[0033] The present invention provides a multi-document question answering retrieval method combining a generative language model and a semantic document graph, including the following steps:

[0034] Step 1: Definition and construction of a document-based knowledge graph.

[0035] Step 1.1: Definition of a document-based knowledge graph: A document knowledge graph is a method of converting text information into structured knowledge. It refines information such as the semantics and keywords of a document and organizes them into a graph form, enabling machines to better understand and utilize this information.

[0036] Step 1.2: Construction of a document-based knowledge graph:

[0037] Step 1.2.1, Document Library Acquisition: First, a document library containing a large number of relevant documents in a certain field is required, which is the basis for constructing a knowledge graph. Therefore, the present invention selects two publicly available datasets for verification. Among them, HotpotQA is a multi-hop reasoning question-answering dataset developed by researchers at Stanford University in the United States. This dataset aims to promote the understanding and solution of complex problems in the field of natural language processing, especially those questions that require integrating information from multiple sources to answer. HotpotQA requires the model to not only understand a single document but also effectively integrate information from different documents to complete multi-step reasoning. MuSiQue (Multihop Questions via Single-hop Question Composition) is a multi-hop reasoning question-answering dataset introduced by the Knowledge Engineering Laboratory of Tsinghua University. The design purpose of this dataset is to overcome the problems that can be answered by shortcuts existing in existing datasets. MuSiQue adopts a bottom-up question composition method, which enhances the logical relevance between questions by ensuring that the answer to each sub-question is necessary for answering subsequent questions.

[0038] Step 1.2.2, Text Cutting: Perform word segmentation on the documents. After obtaining the document library, it is necessary to perform word segmentation on the documents to split the long documents into independent text blocks (such as sentences, paragraphs, or chapters). The purpose of this step is to decompose the documents into smaller, more manageable, and analyzable units. Word segmentation can use existing natural language processing (NLP) tools, such as jieba (Chinese word segmentation) or NLTK (English word segmentation).

[0039] The text cutting in this step, especially for the processing of long documents, is an important step in natural language processing (NLP). This process usually includes word segmentation and splitting long documents into independent ones. After completing word segmentation, the long document is still a continuous text stream. For the convenience of subsequent processing and analysis, it is usually necessary to further split the document into independent text blocks. The definition of text blocks can be determined according to specific needs, such as splitting by paragraphs, sentences, or specific markers (such as headings, line breaks, etc.). The split text blocks can be used as independent processing units for tasks such as information extraction and summary generation. In actual operation, text cutting usually involves the following steps:

[0040] Step 1.2.2.1, Preprocessing: Remove irrelevant characters (such as punctuation marks, numbers, etc.) from the text and perform necessary text cleaning.

[0041] Step 1.2.2.2, Word Segmentation: Use a word segmentation tool or algorithm to perform word segmentation on the text.

[0042] Step 1.2.2.3, Segmentation: According to the defined rules or tags, segment the tokenized text into independent text chunks.

[0043] Step 1.2.2.4, Post-processing (optional): Further clean or format the segmented text chunks.

[0044] Step 1.2.3, Embed document chunks based on a pre-trained language model: Use a pre-trained language model such as BERT to extract the semantic features of each document chunk for subsequent similarity calculation. The specific steps are as follows:

[0045] Step 1.2.3.1, Pre-trained language model: The language model (such as BERT) is pre-trained on a large amount of text data with the aim of learning general language representations. BERT is a typical example. It processes the input sequence through the Transformer architecture and can understand the meanings of context-related words.

[0046] Step 1.2.3.2, Document chunk embedding: Embed the document chunks based on the pre-trained language model. For a given piece of text or "document chunk", the embedding process can be represented as shown in formula (1):

[0047] (1)

[0048] where is the embedding vector of document chunk i, Embed is the embedding function, usually a pre-trained deep neural network, and chunk i represents a small unit block after splitting the long text.

[0049] Step 1.2.3.3, Semantic feature extraction: Use the pre-trained language model to convert it into a fixed-length vector. This process usually involves inputting the text into the language model and extracting the output feature vector from a certain layer (usually the last layer) of the language model. The output feature vector obtained through the above process contains the semantic information of the original text. This means that the output feature vector not only reflects the literal meaning of the words but also reflects more complex features such as their roles in the sentence and the logical relationships between sentences for subsequent similarity calculation.

[0050] Step 1.2.4, Calculate similarity: Calculate the similarity between text chunks based on the document chunk embedding vectors to form edges. For each document node, connect neighbor document nodes based on the similarity threshold and the top-k algorithm to form a document knowledge graph, which actually describes a process of constructing a document graph. This process involves the following key steps:

[0051] Step 1.2.4.1, Document node representation.

[0052] First, each document is considered as a node in the graph. These document nodes are usually transformed into vector representations through some embedding techniques (such as word embedding, sentence embedding, or paragraph embedding) for use in subsequent similarity calculations.

[0053] Step 1.2.4.2, Similarity calculation.

[0054] Use a certain similarity metric (such as cosine similarity, Euclidean distance, etc.) to calculate the similarity between each pair of document nodes. The choice of similarity metric depends on the nature of the embedding vectors and the specific requirements of the task.

[0055] Taking cosine similarity as an example, assume there are two document chunks A and B, and their embedding vectors are and , respectively. Then the cosine similarity between them can be calculated by the following formula:

[0056] (2)

[0057] where, is the dot product of vector A and vector B, and is the norm of vector A and vector B. The value range of cosine similarity is [-1, 1], where 1 indicates that the two vectors are exactly the same (same direction and equal length), and 0 indicates that the two vectors are orthogonal (i.e., not related).

[0058] Step 1.2.4.3, Connecting neighbor nodes.

[0059] Based on the similarity calculation results, connect the neighbor document nodes.

[0060] Step 1.2.4.4, Similarity threshold.

[0061] First, set a similarity threshold. If the similarity between two document nodes exceeds this threshold, they are considered neighbors and connected by an edge in the graph. Second, use the top-k algorithm: for each document node, find the k most similar document nodes to it and connect them as neighbors. Here, k is a pre-set positive integer.

[0062] Step 1.2.4.5, Forming the document knowledge graph.

[0063] Through the above steps, the present invention constructs a network composed of document nodes and the edges connecting them. This network can be regarded as a document knowledge graph, where the nodes represent documents and the edges represent the similarity relationships between documents.

[0064] Step 1.2.4.6, Subsequent applications.

[0065] After completing the above steps, a document knowledge graph can be formed and used for various NLP tasks, such as document clustering, topic detection, information retrieval, recommendation systems, etc. By analyzing the nodes and edges in the graph, potential connections between documents can be discovered, revealing the internal structure of the data.

[0066] Step 2: Multi-hop retrieval generative question answering for the document knowledge graph based on the generative language model.

[0067] Step 21: Generate the initial candidate set: First, the user's question needs to be used as a query condition to perform a TF-IDF algorithm search in the document corpus to obtain a set of related nodes, that is, the initial candidate set.

[0068] The TF-IDF algorithm search is a widely used weighting technique in information retrieval and text mining, used to evaluate the importance of a word for a document in a document set or corpus. The TF-IDF search process mainly includes the following steps:

[0069] Step 211: Preprocessing stage:

[0070] (1) Text tokenization: Tokenize each document in the document collection to obtain a list of words.

[0071] (2) Stop word processing: Remove stop words (such as common but meaningless words like "of", "is", etc.) from the documents.

[0072] (3) Lemmatization (optional): Lemmatize the words to their basic forms (e.g., change "running" to "run") to better handle morphological variations.

[0073] Step 212: Construct the term frequency matrix:

[0074] (1) Calculate the term frequency (TF): For each word, calculate the number of times it appears in each document and divide it by the total number of words in that document to obtain the TF of the word in that document. Its expression is as follows:

[0075] (3)

[0076] (2) Construct a matrix where the rows represent documents and the columns represent words. Each element in the matrix represents the term frequency of the corresponding word in the corresponding document.

[0077] Step 213: Calculate the inverse document frequency (IDF):

[0078] For each word in the vocabulary, calculate its IDF. The IDF reflects the general importance of a word. The higher the IDF value of a word, the more representative it is. Its expression is as follows:

[0079] (4)

[0080] Note: Adding 1 to the denominator is to avoid the case where the denominator is 0.

[0081] Step 214, Calculate the TF-IDF value:

[0082] Multiply the TF of each term in each document by its IDF value to obtain the TF-IDF value of the term in the document. Its expression is as follows:

[0083] (5)

[0084] Step 215, Construct the TF-IDF vector:

[0085] For each document, construct a vector using the TF-IDF values of all its terms. This vector can be regarded as the representation of the document in the term space.

[0086] Step 216, Search and sort:

[0087] When the user enters a query, preprocess the query in the same way, such as word segmentation and stop word processing. Calculate the TF-IDF value of each term in the query in the document collection (usually use the IDF value of the terms in the query because the query itself is not a complete document). Calculate the similarity between the query vector and the document vectors using methods such as cosine similarity. Sort the documents according to the similarity scores and return the document with the highest score to the user as the search result.

[0088] Step 22, Iteratively expand the document set:

[0089] Step 221, Traverse the initial node set: Traverse each node in the initial node set and use it as the initial node of the current path.

[0090] Step 222, Input the question and the retrieved nodes into the generative large language model: Based on the KGP3 algorithm designed by the present invention, input the question and the node information on the current path into the generative language model to form a multi-hop question and answer of the document knowledge graph. The specific implementation process is as follows:

[0091] (1)Generate sub - questions: Combining the main question and the nodes already obtained, generating a sub - question involves the information that the next required node should contain. For example, when asked who is older, Bob Bryan or Mariaan de Swardt, and having the information about Bob Bryan's age, our intuition is to ask about Mariaan de Swardt's age. This intuition forms the reasoning basis of the generative language model, enabling it to generate subsequent questions. The generative language model outputs the generated sub - questions to guide the selection of the next - hop node.

[0092] (2)Get the next - hop node according to the generated sub - question: Use the generated sub - question to further retrieve new nodes in the knowledge graph, and these nodes are the targets to be accessed in the next step. The specific method is as follows: After generating the sub - question, the generative language model uses this sub - question as reasoning information and carefully selects the next most suitable document node from the neighbors. In this process, the generative language model always needs to select an optimal node in the current path for subsequent exploration at this node. The selection of this node is the node that is most helpful for answering the initial question Q among the candidate neighbors.

[0093] (3)Judge the relevance of the selected node: In the step of selecting neighbor nodes, it is possible that not all neighbor nodes of the current node contain the information required to answer the question, or there is no association at all. If this node is included in the retrieval result, it will interfere with the question - answering to a certain extent. Therefore, it is necessary to judge the relevance of the selected node to the question. Specifically, the following three evaluations are made on the selected node: Support: indicating that the current node highly supports the seed node when answering the query; Ambiguous: indicating that the current node supports the seed node to a certain extent; Reject: indicating that there is no valuable information for the main question. If the relevance of the node is judged as "Support" or "Ambiguous", the node is added to the current path, and the selection of the next - hop is continued based on this node. If the relevance of the node is judged as "Reject", the current node is skipped, and the traversal of the initial node set is continued. This method is efficient, traceable, and interpretable. At the same time, it abstractly summarizes the overall process of the generative language model in traversing the graph to select the next - hop node, has a certain universality, and can conveniently simulate this general scenario to construct a fine - tuning dataset to further improve the retrieval ability of the model.

[0094] (4)Judge whether the length of the current path reaches the threshold: Check whether the length of the current path has reached the pre - set threshold. If it has not reached the threshold, continue to execute steps (1) to (3); if it has reached the threshold, add the documents of the current path to the retrieved document set.

[0095] (5) Determine whether the initial node set has been traversed: Check whether the initial node set has been completely traversed. If it has not been traversed, continue to traverse the initial node set; if it has been traversed, end the entire retrieval process.

[0096] Completing the above content can obtain the retrieved document set after multi-hop Q&A training of the document knowledge graph.

[0097] In the present invention, the generative language model is a generative large language model or a small encoder-decoder model, the T5 model. When the generative language model is the small encoder-decoder model T5, it is fine-tuned. At the same time, for the architecture of its encoder-decoder and the characteristics of the traversal method of the KGP3 method, the model is improved. Negative samples are added to each fine-tuning data and a contrastive learning head is introduced. This method encourages the model to select the correct document and avoid selecting other confusing documents by adding a context contrast objective, thereby strengthening the model's judgment ability. In this process, the input of the model includes a question and the retrieved evidence related to it, where each candidate node represents a potential relevant information segment. Through the encoder of T5, these inputs are converted into a rich continuous vector representation and then fed into the decoder to generate or predict text. During training, two negative samples are provided for each positive sample. The negative samples are generated by replacing the correct node with an irrelevant node or by changing the judgment result (for example, changing "accept" to "reject"). This contrastive learning method helps the model to be more sensitive to the subtle differences between useful information and misleading information. In this way, a contrast between positive samples and negative samples is formed. In addition to the traditional cross-entropy loss, the present invention also uses this contrastive learning method to further improve the performance of the model.

[0098] (6)

[0099] (7)

[0100] (8)

[0101] (9)

[0102] (10)

[0103] Among them, formulas (6) and (7) represent the output probabilities of positive samples and negative samples, where y + and respectively represent the decoder hidden states of the positive sample and k negative samples, W y and b yare learnable parameters, and δ is a Sigmoid function used to convert the input into a value between 0 and 1. Equation (8) defines the contrastive loss L cl , which is obtained by calculating the probability of the positive sample divided by the sum of the probabilities of all samples. r is a temperature hyperparameter used to adjust the steepness of the softmax function, and Avg( ) represents the average pooling function based on the target sequence length. Equation (9) represents the cross-entropy loss L mim , which is used to measure the difference between the model's prediction result and the actual label. Finally, Equation (10) gives the total loss L total , which is the weighted sum of the contrastive loss L_cl and the cross-entropy loss L mim , and λ is the weight coefficient. Overall, these equations describe how a machine learning model can improve its performance by optimizing these two loss functions.

[0104] Step 23: After Steps 21 and 22, post-processing steps such as deduplication are performed on the retrieved document set to obtain the final retrieved document set, which is combined with the user's question and input into the large model to obtain the answer reply.

Claims

1. A multi-document question answering retrieval method combining a generative language model and a semantic document graph, characterized in that The method includes the following steps: Step 1, construction of a document-based knowledge graph: Step 1.1, document library acquisition: Obtain a document library containing a large number of relevant documents in a certain field; Step 1.2, text segmentation: After obtaining the document library, perform word segmentation on the documents to split them into individual text chunks; Step 1.3, embedding of document chunks based on a pre-trained language model: Use a pre-trained language model to extract the semantic features of each document chunk; Step 1.4, calculation of similarity: Calculate the similarity between text chunks based on the document chunk embedding vectors to form edges. For each document node, connect neighbor document nodes based on the similarity threshold and the top-k algorithm to form a document knowledge graph; Step 2, multi-hop retrieval generative question answering for the document knowledge graph based on a generative language model: Step 21, generation of an initial candidate set: Use the user's question as a query condition and perform TF-IDF algorithm search in the document library obtained in Step 1 to obtain a set of related nodes, i.e., the initial candidate set; Step 22, iterative expansion of the document set: Step 221, traversal of the initial node set: Traverse each node in the initial candidate set generated in Step 21 and use it as the initial node of the current path; Step 222, input of the question and the retrieved nodes into a generative language model: Input the question and the node information on the current path into a generative language model to obtain multi-hop question answering for the document knowledge graph based on the KGP3 algorithm. The specific steps are as follows: (1) Generation of sub-questions: Combine the main question and the nodes already obtained to generate a sub-question involving the information contained in the next required node; (2) Obtaining the next-hop nodes based on the generated sub-question: Use the generated sub-question to further retrieve new nodes in the document-based knowledge graph constructed in Step 1. These nodes are the targets to be visited in the next step; (3) Judgment of the relevance of the selected nodes: If the relevance of a node is judged as "support" or "ambiguous", add the node to the current path and continue to select the next hop based on this node; if the relevance of a node is judged as "reject", skip the current node and continue to traverse the initial node set; Support: indicates that the current node highly supports the seed node in answering the query; Ambiguous: indicates that the current node supports the seed node to a certain extent; Reject: indicates that there is no valuable information for the main question; (4) Judgment of whether the length of the current path reaches the threshold: Check whether the length of the current path has reached the pre-set threshold. If it has not reached the threshold, continue to execute steps (1) to (3); if it has reached the threshold, add the documents on the current path to the retrieved document set; (5) Judgment of whether the initial node set has been traversed completely: Check whether the initial node set has been completely traversed. If it has not been traversed completely, continue to traverse the initial node set; if it has been traversed completely, end the entire retrieval process to obtain the retrieved document set; Step 23: After Steps 21 and 22, perform a deduplication process on the retrieved document set to obtain the final retrieved document set, and incorporate it together with the user's question into the generative language model to obtain an answer response.

2. The multi-document question-answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The said Step 1.2 includes the following steps: Step 1.2.1: Preprocessing: Remove irrelevant characters from the text and perform necessary text cleaning. Step 1.2.2: Word Segmentation: Use a word segmentation tool or algorithm to perform word segmentation on the text. Step 1.2.3: Splitting: According to the defined rules or tags, split the word-segmented text into independent text chunks.

3. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 2, characterized in that The said Step 1.2 further includes the following steps: Step 1.2.4: Postprocessing: Perform further cleaning or formatting on the split text chunks.

4. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The said Step 1.3 includes the following steps: Step 1.2.3.1: Pre-trained Language Model: Pre-train a language model on a large amount of text data. Step 1.2.3.2: Document Chunk Embedding: Based on the pre-trained language model, perform embedding on the document chunks. For a given piece of text or "document chunk", the embedding process is represented as: Among them, is the embedding vector of document block i, Embed is the embedding function, and chunk i represents a small unit block after splitting the long text; Step 1.2.3.3: Semantic Feature Extraction: Extract the output feature vector from a certain layer of the language model.

5. The multi-document question-answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The said Step 1.4 includes the following steps: Step 1.4.1: Document Node Representation: Each document is regarded as a node in the graph, and these document nodes are converted into vector representations through embedding technology. Step 1.4.2: Similarity Calculation: Use a similarity metric to calculate the similarity between each pair of document nodes. Step 1.4.3: Neighbor Node Connection: Based on the similarity calculation results, connect the neighbor document nodes. Step 1.4.4: Similarity Threshold: First, set a similarity threshold. If the similarity between two document nodes exceeds this threshold, they are regarded as neighbors and connected by an edge in the graph. Second, use the top-k algorithm: For each document node, find the k most similar document nodes and connect them as neighbors. Step 1.4.5: Form a Document Knowledge Graph: Through Steps 1.4.1 to 1.4.4, a network composed of document nodes and the edges connecting them is constructed. This network is regarded as a document knowledge graph, where the nodes represent documents and the edges represent the similarity relationships between documents.

6. The multi-document question-answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The said TF-IDF algorithm includes the following steps: Step 211: Preprocessing Phase: (1) Text Word Segmentation: Perform word segmentation on each document in the document set to obtain a list composed of words. (2) Stop Word Processing: Remove stop words from the documents. Step 212: Construct a Term Frequency Matrix: (1) Calculate Term Frequency TF: For each word, calculate the number of times it appears in each document and divide it by the total number of words in that document to obtain the TF of the word in that document. Its expression is as shown in the following formula: (2) Construct a matrix where the rows represent documents and the columns represent vocabulary. Each element in the matrix represents the term frequency of the corresponding vocabulary in the corresponding document. Step 213: Calculate Inverse Document Frequency IDF: For each vocabulary in the vocabulary list, calculate its IDF. Its expression is as shown in the following formula: Step 214, Calculate the TF-IDF value: Multiply the TF of each term in each document by its IDF value to obtain the TF-IDF value of the term in the document. The expression is as follows: Step 215, Construct a TF-IDF vector: For each document, use the TF-IDF values of all its terms to construct a vector, which is regarded as the representation of the document in the term space. Step 216, Search and sort: When the user enters a query, preprocess the query in the same way, calculate the TF-IDF value of each term in the query in the document collection, calculate the similarity between the query vector and the document vectors, sort the documents according to the similarity scores, and return the document with the highest score as the search result to the user.

7. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 6, characterized in that The said step 211 further includes the following steps: (3) Lemmatization: Restore the terms to their basic forms to better handle morphological variations.

8. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 1, characterized in that The generative language model is a generative large language model or the small encoder-decoder model T5 model.

9. The multi-document question answering retrieval method combining a generative language model and a semantic document graph according to claim 8, characterized in that When the generative language model is the small encoder-decoder model T5 model, fine-tune it by adding negative samples for each fine-tuning data and introducing a contrastive learning head: The input of the T5 model includes a question and the associated retrieval evidence, where each candidate node represents a potential relevant information segment. Through the encoder of the T5 model, these inputs are converted into a rich continuous vector representation and then fed into the decoder to generate or predict text. During training, two negative samples are provided for each positive sample to form a contrast between positive and negative samples.

Images