Enhanced retrieval generation method supporting dynamic knowledge clipping

Abstract

The invention relates to the technical field of artificial intelligence and natural language processing, in particular to an enhanced retrieval generation method supporting dynamic knowledge clipping, which is based on a retrieval enhanced generation framework optimized by a knowledge graph and comprises the following steps: firstly, extracting (h, r, t) triples from an unstructured knowledge text through a design template and cleaning and storing the triples; then, the constructed knowledge graph is optimized; further calculating semantic similarity between user query and knowledge graph nodes by using an embedded model, preliminarily screening out related nodes, specifically, carrying out hybrid similarity candidate screening and dynamic candidate set generation, then constructing a multi-hop associated sub-graph by using breadth-first search, cutting redundant edges by using a maximum spanning tree, retaining high-score nodes, and finally constructing a multi-hop associated sub-graph by using breadth-first search; realizing dynamic context truncation; and finally, combining the retrieved and pruned context information with the user question, promoting the LLM to generate an answer for the user question by constructing a cue word, continuously optimizing the LLM, and generating a more accurate and deeper answer.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, specifically to an enhanced retrieval generation method that supports dynamic knowledge trimming. Background Technology

[0002] Large language models, such as GPT-4, Llama3, and DeepSeek-R1, can generate fluent and coherent text through large-scale pre-training and self-supervised learning, and perform excellently in tasks such as question answering systems, text generation, and code generation. However, large language models face the illusion problem in knowledge-intensive tasks, i.e., generating text containing inaccurate or fictitious information, which is particularly prominent in fields with extremely high accuracy requirements (such as biomedicine and law).

[0003] To address the problem of illusion, Retrieval Augmentation (RAG) technology has emerged. RAG significantly improves the accuracy and reliability of generated content by combining external knowledge bases (such as Wikipedia and knowledge graphs) with large language models (LLMs). Its core idea is to retrieve knowledge fragments relevant to the user's query and use them as contextual input to LLMs to generate fact-based answers.

[0004] However, existing RAG methods exhibit certain limitations in knowledge-intensive tasks, especially in scenarios requiring multi-hop reasoning and understanding complex relationships. Specifically, existing RAG methods primarily rely on semantic retrieval, that is, retrieving relevant content by calculating the semantic similarity between the user query and document fragments. While this method is simple and effective, it ignores the inherent relationships between knowledge fragments, resulting in potentially isolated and redundant retrieved content that fails to fully utilize the structured information in the knowledge graph. Furthermore, existing RAG methods often face problems of high computational resource consumption and low retrieval efficiency when processing large-scale knowledge graphs. Summary of the Invention

[0005] The purpose of this invention is to provide an enhanced retrieval generation method that supports dynamic knowledge pruning, aiming to improve the performance of large language models in knowledge-intensive tasks and optimize the diversity and coherence of retrieval results in scenarios involving multi-hop reasoning and complex relationship understanding.

[0006] To achieve the above objectives, the present invention provides an enhanced retrieval generation method that supports dynamic knowledge pruning, comprising the following steps:

[0007] Step 1: Knowledge Graph Construction;

[0008] Step 2: Dynamic contextual retrieval and pruning;

[0009] Step 3: Large-scale language model enhancement generation.

[0010] Optionally, the knowledge graph construction process in step 1 includes the following steps:

[0011] Step 1.1: Document segmentation and preprocessing, specifically using a sliding window-based text segmentation algorithm to preprocess the document;

[0012] Step 1.2: Entity and relation extraction. Using prompting engineering, extract (h,r,t) triples from unstructured knowledge text by designing templates.

[0013] Step 1.3: Select a suitable graph database to store the constructed knowledge graph and optimize the constructed knowledge graph.

[0014] Optionally, during the storage of the constructed knowledge graph, entities are stored as nodes in the graph database, and relationships are stored as edges in the graph database, connecting the corresponding nodes; methods for optimizing the constructed knowledge graph include removing duplicate nodes and edges and supplementing missing information.

[0015] Optionally, the execution process of step 2 includes the following steps:

[0016] Step 2.1: Calculate the semantic similarity between the user query and the knowledge graph nodes using the embedding model to initially filter out relevant nodes;

[0017] Step 2.2: Generate a candidate set by combining similarity candidate filtering and dynamic candidate set generation;

[0018] Step 2.3: Construct a multi-hop association subgraph using breadth-first search, and prune redundant edges using maximum spanning tree to retain high-scoring nodes, thus achieving dynamic context truncation.

[0019] Optionally, the process of generating large-scale language models in step 3 includes the following steps:

[0020] Step 3.1: Combine the retrieved and pruned contextual information with the user's question to construct prompt words;

[0021] Step 3.2: Select a large language model to optimize and generate answers to user questions.

[0022] This invention provides an enhanced retrieval generation method supporting dynamic knowledge pruning. Based on a knowledge graph-optimized retrieval enhancement generation framework, it first extracts (h,r,t) triples from unstructured knowledge text using a designed template, cleans and saves them, and then optimizes the constructed knowledge graph. Next, it utilizes an embedding model to calculate the semantic similarity between the user query and knowledge graph nodes, initially filtering relevant nodes through a hybrid similarity candidate filtering and dynamic candidate set generation. Then, it uses breadth-first search to construct a multi-hop association subgraph and uses maximum spanning tree pruning to remove redundant edges and retain high-scoring nodes, achieving dynamic context truncation. Finally, it combines the retrieved and pruned context information with the user question, constructing prompts to encourage the LLM (Limited Learning Model) to generate answers tailored to the user question, and continuously optimizes the LLM to better utilize structured knowledge graph information, generating more accurate and in-depth answers. This invention can integrate knowledge graphs and dynamic pruning techniques to capture fact-level relationships and optimize the diversity and coherence of retrieval results. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an enhanced retrieval generation method that supports dynamic knowledge trimming according to the present invention. Detailed Implementation

[0025] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0026] This invention provides an enhanced retrieval generation method that supports dynamic knowledge pruning, comprising the following steps:

[0027] Step 1: Knowledge Graph Construction;

[0028] Step 2: Dynamic contextual retrieval and pruning;

[0029] Step 3: Large-scale language model enhancement generation.

[0030] Please see Figure 1 The following provides further explanation with reference to specific embodiments and execution processes:

[0031] I. Knowledge Graph Construction

[0032] (I) Document Segmentation and Preprocessing

[0033] Blocking Strategy: A sliding window-based text blocking algorithm is used to preprocess the document. Let the window length L = 256 words, the step size S = (1-α)L, and the overlap rate α = 0.2. For a document d∈D, its block sequence C(d) is represented as:

[0034]

[0035] Where |d| represents the number of words in the document, and wi:j represents the sequence from the i-th word to the j-th word. Each block ci is associated with metadata.<ti,pi> , respectively, indicate the title and paragraph position.

[0036] Example: Suppose there is a document with a length of 512 words. If it is divided into blocks according to the above parameters, two blocks can be obtained. The first block contains words 1 to 256, and the second block contains words 205 to 460 (because the step size S = 204), and so on, until the entire document is covered.

[0037] Data source: Taking the supporting documents of the HotpotQA dataset (such as Wikipedia paragraphs) as an example, these documents usually contain a large number of long sentences, which need to be processed by the above-mentioned block segmentation algorithm to meet the sentence length requirements of the embedding model.

[0038] Tool selection: Python's NLTK library can be used for text chunking. These libraries provide convenient text processing functions and can easily implement sliding window-based chunking algorithms.

[0039] (II) Entity and Relation Extraction

[0040] Method Selection: Prompt engineering is employed to extract (h,r,t) triples from unstructured knowledge text by designing templates. Large language models such as LLaMA can be used to accomplish this task.

[0041] Template Design:

[0042] Example template: For the text "Scott Derrickson (born July 16, 1966) is an American director, screenwriter, and producer.", the template design is as follows:

[0043] Input: Extract entities and their relation triples from the following text: Scott Derrickson (born July 16, 1966) is an American director, screenwriter, and producer.

[0044] Output:<Scott Derrickson,date ofbirth,July 16,1966> ,<Scott Derrickson,nationality,United States> ,<Scott Derrickson,occupation,film director> ,<ScottDerrickson,occupation,screenwriter> ,<Scott Derrickson,occupation,producer> .

[0045] Data cleaning: Clean the generated triples to remove unreasonable triples, such as triples with unclear relationships or non-existent entities.

[0046] Save format: Save the cleaned triples to a file for later processing. JSON format can be used for easier knowledge graph construction and data exchange.

[0047] (III) Knowledge Graph Construction

[0048] Graph database selection: Choose a suitable graph database to store the constructed knowledge graph. Neo4j is a powerful graph database that supports efficient graph traversal and query operations, making it suitable for building and managing large-scale knowledge graphs. If you do not need to visualize the knowledge graph, you can skip this step.

[0049] Node creation: Entities are stored as nodes in the graph database. For example, "ScottDerrickson", "July 16, 1966", and "United States" from the example above are stored as nodes in the database.

[0050] Edge creation: Relationships are stored as edges in the graph database, connecting the corresponding nodes. For example, creating an edge.<ScottDerrickson,date ofbirth,July 16,1966> This indicates that there is a "date of birth" relationship between the entities "ScottDerrickson" and "July16,1966".

[0051] Graph optimization: This involves optimizing the constructed knowledge graph, such as removing duplicate nodes and edges, and adding missing information. Duplicate nodes can be found and processed by writing query statements for the graph database.

[0052] II. Dynamic Contextual Retrieval and Pruning

[0053] (I) Entity Extraction and Linking

[0054] Entity extraction: Extract the entity set E(q) = {eq1, eq2, ..., eqm} from the user question q. This can be achieved by configuring a local LLaMA model using Ollama.

[0055] Example: For the question "Are Scott Derrickson and Ed Wood of the same nationality?", extract the entities "Scott Derrickson" and "Ed Wood".

[0056] Hybrid similarity candidate selection:

[0057] Similarity calculation: The cosine similarity between entity eqi and knowledge graph node vj is calculated using an embedding model. The top k (e.g., k=6) candidate nodes based on similarity are initially selected, represented as:

[0058]

[0059] Where veqi and vvj represent the embedding vectors of entity eqi and node vj, respectively.

[0060] Reordering: For the initially screened candidate nodes, optimize the score based on the overall graph structure density (shortest path length between nodes) and reorder them. The specific formula is:

[0061] Where λ = 0.7, cos(vq,vci) represents the cosine similarity between the user question embedding vector and the candidate node embedding vector, and path_length(ci) represents the shortest path length from the user question entity to the candidate node. Finally, nodes satisfying s(q,ci) ≥ 0.5 are retained.

[0062] Dynamic candidate set generation: A dual-selection mechanism controls node growth, providing high-quality input for subsequent subgraph expansion. The final candidate set is defined as follows:

[0063] Candidates finil (e qi )={v j ∈Candidates base ∣s(q,v j )≥0.5}

[0064] (ii) Context extension based on graph traversal

[0065] Breadth-First Search (BFS): Starting from the final candidate node set Candidatesfinal(eq), a breadth-first search (BFS) is performed along the knowledge graph relation edges to construct a multi-hop relational subgraph Gqm. The maximum search depth is set to m=1 to avoid long path noise. Specifically, it is represented as follows:

[0066]

[0067] Here, path(eq,j) represents the shortest path length from the query entity eq to node h.

[0068] Example: Suppose the query entity is "Scott Derrickson". In the knowledge graph, the nodes directly connected to it are "American" (nationality), "Director" (occupation), etc. These nodes and their relationships will be included in the subgraph Gqm.

[0069] Limit the number of nodes expanded: To prevent the subgraph from exploding, the number of nodes expanded per hop is limited to no more than Nmax = 60. This can be achieved by recording the visited depth of each node during the BFS process and stopping the search when the maximum depth or the node limit is reached.

[0070] Subgraph structured compression:

[0071] Convert to an undirected weighted graph: Convert the resulting subgraph Gqm into an undirected weighted graph with edge weights equal to the mixed similarity scores.

[0072] wij=s(q,cij)

[0073] Calculate the Maximum Spanning Tree (MST): Use the MST algorithm to prune redundant edges. The specific formula is:

[0074] T = MST(G qm ,w ij )

[0075] Dynamic context truncation: Based on the LLM context window limit (e.g., 8192 tokens in GPT-4), high-scoring nodes are retained to achieve dynamic context truncation. The set of retained nodes is represented as follows:

[0076] C retained ={c i ∣s(q,c i )≥τ}(τ=0.5)

[0077] Where ci represents the original text block associated with the subgraph node; cij represents the relation description text corresponding to edge (vi, vj).

[0078] III. Enhanced Generation of Large Language Models (LLM)

[0079] (I) Prompt Generation

[0080] Constructing suggestion words: Suggestion words are constructed by combining the retrieved and pruned contextual information with the user's question. The structure of the suggestion words is as follows:

[0081] The instruction section guides the LLM on how to utilize the provided context. For example: "Answer using the provided context".

[0082] The context section (CONTEXT) contains synthesized narrative passages that provide relevant background information. For example: "Scott Derrickson directed Doctor Strange. Christopher Nolan directed The Dark Knight Rises. These directors each created a different superhero franchise."

[0083] The QUESTION section presents the question posed by the user's original query. For example: "Are Scott Derrickson and Christopher Nolan of the same nationality?"

[0084] Optimize prompts: Improve the quality of LLM responses by optimizing the structure and content of prompts. For example, ensure that the information in the context is closely related to the question and avoid irrelevant information interfering with the LLM's judgment.

[0085] (II) Answer Generation

[0086] Choose an LLM model: Select a suitable large-scale language model, such as GPT-4 or LLaMA3-8B. These models have powerful language generation and contextual understanding capabilities, and can generate accurate and coherent answers based on prompts.

[0087] Answer generation: Input the constructed prompts into the selected LLM to generate an answer to the user's question. This can be achieved by calling the LLM's API interface, such as using OpenAI's GPT-4 API.

[0088] Results processing: Post-processing of the generated answers, such as removing irrelevant content and optimizing language expression, is performed to improve the quality and readability of the answers. This can be achieved using techniques such as text summarization and keyword extraction.

[0089] Furthermore, the present invention has been experimentally verified and illustrated as follows:

[0090] Experimental environment: The experiment was conducted on a server equipped with an NVIDIA GTX 3090 GPU and an Intel(R) Xeon(R) Gold 6133 CPU.

[0091] Evaluation metrics: Answer quality was assessed using F1 score (0.661), precision (0.692), and recall (0.677), while retrieval quality was assessed using supporting fact F1 score (s-F1) (0.371), supporting fact precision (s-Precision) (0.236), and supporting fact recall (s-Recall) (0.926).

[0092] The experimental results are as follows:

[0093] Ablation experiment:

[0094] Removing the BFS mechanism: When BFS was replaced with a local similarity expansion algorithm, the answer F1 score decreased by 2.3%, the supporting fact F1 score decreased by 3.8%, and the supporting fact precision decreased by 3.8%. This indicates that BFS is crucial for maintaining semantic coherence in multi-hop reasoning.

[0095] Removing the MST compression mechanism: When the MST is replaced with a direct weight-based threshold, token consumption increases by 3.2%, the answer F1 score decreases by 1.2%, and the supporting fact F1 score decreases by 2.4%. This demonstrates that topology optimization of the MST can effectively remove redundant information and improve information density.

[0096] Removing both BFS and MST simultaneously leads to further performance degradation, with the answer F1 score decreasing by 2.4%, the supporting fact F1 score decreasing by 4.0%, and token consumption increasing by 3.5%. This indicates that BFS and MST have a synergistic effect: BFS ensures the coherence of evidence collection, while MST optimizes information density.

[0097] Experimental results show that the GraphFlow-RAG framework developed in this invention outperforms traditional RAG methods in both response and retrieval quality on HotpotQA and its variant datasets. Furthermore, it provides a scalable solution for cross-domain RAG applications, particularly suitable for scenarios requiring high accuracy and real-time knowledge updates.

[0098] In summary, the enhanced retrieval generation method supporting dynamic knowledge pruning of the present invention has the following advantages:

[0099] Knowledge Graph Enhancement: By constructing a dynamic semantic pruning module and utilizing the fact-level relationships and context organization optimization techniques of knowledge graphs, the performance of LLMs in knowledge-intensive tasks is significantly improved.

[0100] Efficient retrieval and pruning: A graph traversal-based context pruning and reorganization method is proposed to provide a complete knowledge chain context within an acceptable token consumption range, assisting LLMs in answering queries.

[0101] Hint generation optimization: By constructing a knowledge graph through hint engineering, we can capture fact-level relationships in documents and combine them with structured knowledge to achieve hint generation.

[0102] The above description discloses only one or more preferred embodiments of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. An enhanced retrieval generation method supporting dynamic knowledge pruning, characterized in that, Comprising the following steps: Step 1: knowledge graph construction; Step 2: dynamic context retrieval and pruning; Step 3: large language model enhanced generation.

2. The enhanced retrieval generation method supporting dynamic knowledge pruning according to claim 1, wherein The process of step 1 knowledge graph construction comprises the following steps: Step 1.1: document blocking and preprocessing, specifically using a sliding window-based text blocking algorithm to preprocess the document; Step 1.2: entity and relationship extraction, using prompt engineering to extract (h, r, t) triples from unstructured knowledge text through designed templates; Step 1.3: selecting a suitable graph database to store the constructed knowledge graph, and optimizing the constructed knowledge graph.

3. The enhanced retrieval generation method supporting dynamic knowledge pruning according to claim 2, wherein In the process of storing the constructed knowledge graph, the entity is stored as a node in the graph database, the relationship is stored as an edge in the graph database, and the corresponding nodes are connected; the method of optimizing the constructed knowledge graph includes removing duplicate nodes and edges, and supplementing missing information.

4. The enhanced retrieval generation method supporting dynamic knowledge pruning according to claim 3, wherein The execution process of step 2 comprises the following steps: Step 2.1: calculate the semantic similarity between user query and knowledge graph node using embedding model, and preliminarily filter out relevant nodes; Step 2.2: through hybrid similarity candidate filtering and dynamic candidate set generation; Step 2.3: use breadth-first search to construct multi-hop association subgraph, and through maximum spanning tree pruning to retain high-score nodes, realize dynamic context truncation.

5. The enhanced retrieval generation method supporting dynamic knowledge pruning according to claim 4, wherein The process of large language model enhanced generation in step 3 comprises the following steps: Step 3.1: combine the context information after retrieval and pruning with the user question to construct prompt words; Step 3.2: select a large language model to optimize the generation of answers to user questions.

Images