A dynamic retrieval enhancement generation method and system based on semantic contradiction driving

Abstract

The application belongs to the technical field of natural language processing and large language model application, and relates to a dynamic retrieval enhancement generation method and system based on semantic contradiction driving. The method comprises the following steps: performing multi-path sampling on an input question to generate multiple candidate answers; generating a formal answer sentence by sentence according to the input question, performing contradiction identification on the generated answer sentence and the candidate answers after each generation, and obtaining a contradiction pair; generating a follow-up query based on the identified contradiction pair; retrieving relevant documents from an external knowledge base by using the follow-up query, regenerating the current sentence based on the relevant documents, and continuing the subsequent generation process until the answer is completed. The application actively identifies semantic contradictions in the generation process, accurately judges knowledge gaps, and traces the root cause of the contradiction to generate high-quality retrieval queries, thereby significantly improving the accuracy and reliability of the large language model in knowledge-intensive tasks.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and large language model application technology, and in particular to a method and system for dynamic retrieval-Augmented Generation (DynamicRAG) for complex multi-hop question answering tasks. Background Technology

[0002] In recent years, Large Language Models (LLMs) have demonstrated powerful capabilities in various natural language tasks. However, limited by the timeliness of training data and the scale of model parameters, LLMs are prone to "hallucination" when faced with questions beyond their internal knowledge base, i.e., generating factually incorrect or unfounded content. To alleviate this problem, Retrieval-Augmented Generation (RAG) has been widely adopted, supplementing the model's knowledge by introducing external knowledge bases.

[0003] For simple question-answering tasks, single-round RAG (i.e., retrieval only once during the question input stage) can effectively improve performance. However, in complex tasks such as multi-hop QA, questions often require multiple steps of reasoning, and relying solely on the initial retrieval is insufficient to cover all necessary knowledge. Therefore, a dynamic RAG strategy is required—triggering retrieval multiple times during text generation to achieve progressive knowledge supplementation.

[0004] Existing dynamic RAG methods mainly suffer from the following technical shortcomings: 1. Crude retrieval timing judgment: Most methods rely on static rules (such as retrieval every N tokens generated or at the end of each sentence) or set thresholds based on token probabilities (such as FLARE, DRAGIN), ignoring the semantic uncertainty of the generated content. Such methods are prone to redundant retrieval (retrieval when not needed) or missed detections (failure to trigger when needed), and the thresholds need to be manually tuned, resulting in poor generalization.

[0005] 2. Poor quality of search query generation: Existing methods either use only the latest generated sentences as search queries, ignoring context and resulting in unclear references; or they use attention mechanisms to extract keywords, introducing a large amount of irrelevant information, diluting the search focus, and reducing the accuracy of recalling relevant documents.

[0006] 3. Lack of proactive detection mechanism for model knowledge state: Existing methods passively respond to generated signals and fail to proactively assess whether the model currently "knows" a certain fact. When the model generates inconsistent content due to knowledge gaps, the system cannot intervene in a timely manner.

[0007] 4. High deployment complexity: Some dynamic RAG solutions rely on the model's internal attention scores, gradient information, or additional training modules, making them difficult to use plug and play and limiting their versatility across different LLMs.

[0008] Therefore, there is an urgent need for a dynamic RAG framework that requires no fine-tuning, does not rely on internal signals, and can proactively trigger retrieval and generate accurate queries based on semantic consistency, in order to improve the accuracy and robustness of complex question answering tasks. Summary of the Invention

[0009] To address the aforementioned problems, this invention provides a dynamic retrieval enhancement generation method and system based on semantic contradictions. By actively identifying semantic contradictions during the generation process, accurately determining knowledge gaps, and tracing the root causes of contradictions to generate high-quality retrieval queries, this invention significantly improves the accuracy and reliability of large language models in knowledge-intensive tasks.

[0010] The technical solution adopted in this invention is as follows: A dynamic retrieval enhancement generation method driven by semantic contradictions includes the following steps: Multi-path sampling is performed on the input question to generate multiple candidate answers; Generate a formal answer sentence by sentence based on the input question. After each sentence is generated, identify contradictions between it and the candidate answers to obtain contradiction pairs. Generate follow-up queries based on the identified contradictions; The system uses follow-up queries to retrieve relevant documents from an external knowledge base, regenerates the current sentence based on the relevant documents, and continues the subsequent generation process until the answer is completed.

[0011] Furthermore, the step of performing multi-path sampling on the input question to generate multiple candidate answers includes: Based on the input question, multiple candidate answers are generated through random sampling using a large language model; Each candidate answer is stored in a structured manner at the sentence level to form a semantic sample set.

[0012] Furthermore, the step of generating a formal answer sentence by sentence based on the input question, and identifying contradictions between each generated sentence and the candidate answers, includes: Using a large language model, formal answers are generated sentence by sentence based on the input question and the already generated content; Perform a semantic consistency comparison between the currently generated sentence and the semantic sample set; If the current sentence is found to contradict any sample in the semantic sample set in terms of core facts, then the sample is a conflicting sample, the sentence is a contradictory sentence, the sample and the sentence form a contradictory pair, and it is determined that external knowledge retrieval needs to be triggered.

[0013] Furthermore, the core, in fact, contradictory judgment focuses on the consistency of key information, ignoring irrelevant details; the key information includes subject, event, time, and attribute.

[0014] Furthermore, the step of generating a follow-up query based on the identified contradictory pairs includes: inputting the contradictory pairs into a large language model, and guiding the large language model to generate a follow-up query that clarifies the contradictions and integrates the semantics of the currently generated content and the sample context.

[0015] Furthermore, the generation of the probing search query adopts a prompting engineering template to ensure that the query semantics are clear and directly correspond to the knowledge gap.

[0016] Furthermore, when regenerating the current sentence, the original question, the generated prefix text, and the retrieved relevant documents are used together as the input context of the large language model.

[0017] A dynamic retrieval enhancement generation system driven by semantic contradictions, comprising: The semantic sampling module is used to perform multi-path sampling on the input question and generate multiple candidate answers; The contradiction identification module is used to generate a formal answer sentence by sentence based on the input question. After each sentence is generated, it is compared with the candidate answers to identify contradictions and obtain contradiction pairs. The contradiction tracing module is used to generate follow-up queries based on the identified contradiction pairs; The dynamic retrieval and regeneration module is used to retrieve relevant documents from an external knowledge base using follow-up queries, regenerate the current sentence based on the relevant documents, and continue the subsequent generation process until the answer is completed.

[0018] Compared with the prior art, the beneficial effects of the present invention are: 1. More accurate retrieval timing (based on semantics rather than probability).

[0019] The shortcomings of existing technologies: Existing technologies typically rely on static rules (such as retrieving once every N tokens generated) or set thresholds based on token probability / entropy values ​​(such as FLARE and DRAGIN). These methods ignore the semantic uncertainty of the generated content, which can easily lead to "frequent retrieval when no retrieval is needed" (redundancy) or "failure to trigger when retrieval is needed" (missed detection), and the thresholds are difficult to generalize.

[0020] The advantages of this invention are: it introduces a "semantic contradiction-driven" mechanism. Candidate answers are generated through multi-path sampling, and the semantic consistency between the current sentence and candidate samples is compared in real time during the formal generation process.

[0021] Beneficial effects: Retrieval is triggered only when there are contradictions (i.e., semantic inconsistencies) in the internal knowledge generated by the model. This triggering method, based on high-order semantic indicators, can more accurately locate knowledge gaps in the model, avoid invalid retrievals, and improve the system's response efficiency and accuracy.

[0022] 2. Higher quality search results (contradictory tracing and probing queries).

[0023] The shortcomings of existing technologies: Existing methods often use the latest generated sentences as queries directly, which can easily lead to retrieval failures due to unclear references; or they use attention mechanisms to extract keywords, which can easily introduce a lot of irrelevant noise and dilute the search focus.

[0024] The advantages of this invention are: a "contradiction tracing" mechanism is designed. When a contradiction is identified, the "formally generated sentence" and the "candidate sample of the conflict" are input into the large model together, guiding the model to generate a "follow-up query" (e.g., "What is the exact year that Einstein won the Nobel Prize?").

[0025] Beneficial effects: The generated queries naturally integrate the current generation intent with the contextual semantics, directly addressing knowledge gaps. This effectively solves the problems of ambiguous referencing and missing information, significantly improving the relevance and accuracy of documents retrieved from external knowledge bases.

[0026] 3. Possesses the ability to proactively detect the knowledge state of the model.

[0027] The shortcomings of existing technologies: Most existing dynamic RAG methods generate signals in a passive response manner, and cannot actively assess whether the model truly "knows" a certain fact at present.

[0028] The advantages of this invention are: utilizing the diversity generated by a large language model (obtaining multiple candidate paths through random sampling) as a detection signal.

[0029] Beneficial effects: This mechanism transforms generative diversity into a knowledge state detection signal. If the model is absolutely certain about a fact, the generation results from different paths should tend to be consistent; conversely, if contradictions occur, it indicates that the model has a knowledge blind spot in that area. This mechanism enables proactive and real-time evaluation of the model's knowledge state.

[0030] 4. High versatility and low deployment cost (plug and play).

[0031] The shortcomings of existing technologies: Some dynamic RAG solutions rely on the attention scores and gradient information inside the model, or require additional fine-tuning and training of the model, making it difficult to transfer to different large language models (LLMs) and resulting in high deployment complexity.

[0032] Advantages of this invention: This method is a plug-and-play framework. It does not require fine-tuning of large language models, nor does it rely on attention weights or gradient information within the model. It can complete multipath sampling and contradiction identification solely through a standard generation interface (API).

[0033] Beneficial effects: It greatly reduces the technical threshold and deployment cost, can be widely adapted to open source or closed source models of different scales and versions such as Llama2 and Llama3, and has strong robustness and generalization ability. Attached Figure Description

[0034] Figure 1 This is a flowchart of the steps of the dynamic retrieval enhancement generation method based on semantic contradiction driving according to the present invention.

[0035] Figure 2 The overall processing flow of the dynamic retrieval enhancement generation method based on semantic contradiction driving of the present invention is illustrated, and the closed-loop mechanism from question input, multi-path sampling, contradiction identification, contradiction tracing to retrieval regeneration is demonstrated. Figure 3 This is a module composition diagram of the dynamic retrieval enhancement generation system based on semantic contradiction driven by the present invention, including a semantic sampling module, a contradiction identification module, a contradiction tracing module, and a dynamic retrieval and regeneration module. Detailed Implementation

[0036] The present invention will be further illustrated below with specific implementation examples. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0037] The innovation of this invention lies in transforming the generative diversity of large language models into knowledge state detection signals. By using semantic contradiction as a high-order indicator, it achieves joint optimization of "when to retrieve" and "what to retrieve", thus constructing a lightweight, general, and efficient dynamic RAG framework.

[0038] This invention provides a dynamic retrieval enhancement generation method based on semantic contradiction-driven approaches, such as... Figure 1 As shown, it includes the following steps: Multi-path sampling is performed on the input question to generate multiple candidate answers; Generate a formal answer sentence by sentence based on the input question. After each sentence is generated, identify contradictions between it and the candidate answers to obtain contradiction pairs. Generate follow-up queries based on the identified contradictions; The system uses follow-up queries to retrieve relevant documents from an external knowledge base, regenerates the current sentence based on the relevant documents, and continues the subsequent generation process until the answer is completed.

[0039] In one embodiment, the overall processing flow of the dynamic retrieval enhancement generation method based on semantic contradiction driven by the present invention is as follows: Figure 2 As shown, it includes the following steps: (1) Perform multi-path sampling on the input question to generate multiple candidate answer samples.

[0040] (1.1) Based on the input question, n candidate answers are generated by random sampling using a large language model; (1.2) Each candidate answer is stored in a structured manner at the sentence level to form a semantic sample set.

[0041] (2) Perform the formal generation process and perform contradiction identification after each sentence is generated.

[0042] (2.1) Using a large language model, generate formal answers sentence by sentence based on the input question and the generated content; (2.2) Perform a semantic consistency comparison between the currently generated sentence and the semantic sample set obtained in step (1); (2.3) If the current sentence is found to contradict any sample in core facts, then the sample is a conflict sample and it is determined that external knowledge retrieval needs to be triggered.

[0043] (3) Based on the identified contradictory pairs, generate targeted search queries (follow-up queries).

[0044] (3.1) Input the generated sentences that conflict with each other and the corresponding conflict samples into the large language model; (3.2) Guide the large language model to generate a follow-up retrieval query to clarify the contradiction, which integrates the currently generated content with the contextual semantics.

[0045] (4) Use the generated search query to retrieve relevant documents from the external knowledge base, and regenerate the current sentence based on the search results, and continue the subsequent generation process until the answer is completed.

[0046] In one embodiment, the random sampling described in step (1.1) does not require fine-tuning or additional training; it can be completed by directly calling the generation interface of the original large language model.

[0047] In one embodiment, the core factual contradiction judgment described in step (2.3) focuses on the consistency of key information such as subject, event, time, and attributes, ignoring irrelevant details and differences.

[0048] In one embodiment, the generation of the probing search query in step (3.2) uses a prompting engineering template to ensure that the query semantics are clear and directly correspond to the knowledge gap.

[0049] In one embodiment, when regenerating the current sentence in step (4), the original question, the generated prefix text, and the retrieved relevant documents are used together as the input context of the large language model.

[0050] The method described above in this invention is a plug-and-play framework that does not rely on internal model attention weights, token probability thresholds, or fixed retrieval intervals.

[0051] In one embodiment, a dynamic retrieval enhancement generation method based on semantic contradiction driving according to the present invention includes the following steps: (1) Multi-path candidate answer sampling. To probe the model's knowledge of the current question, multi-path generation is first performed. Given the user input question q, the target large language model (such as Llama3-8B) is invoked to generate n = 3 candidate answers a1, a2, a3 through random sampling (temperature > 0). Each answer is segmented according to sentence boundaries to form a structured sample set A. s = {a1, a2, ..., a n This is used for subsequent contradiction comparison. This process requires no model fine-tuning, only a standard generation interface, and has strong versatility.

[0052] (2) Formal Generation and Contradiction Identification. During formal response generation, semantic consistency is checked sentence by sentence. The LLM model is based on q and the generated prefix X. i-1 The i-th sentence x that generates the intermediate answer i x i With sample set A s For each sample, the corresponding sentence (or paragraph covering the same topic) is input into a contradiction identification prompt template. The prompt template explicitly requires LLM to focus only on core factual conflicts (such as "Are the years the same?" "Are the subjects the same?"), ignoring non-critical differences such as style and details. If sample set A... s Any sample a f With x i If it is determined to be contradictory, then sample a f This is called a conflicting sample, x i This is called a contradictory sentence, a f With x i These are called contradictory pairs, triggering the retrieval process.

[0053] (3) Contradiction Tracing and Retrieval Query Generation. To accurately locate knowledge gaps, this invention designs a contradiction tracing mechanism. Contradictory pairs, i.e., contradictory sentences x...i Its conflicting sample a f Enter LLM along with the prompt: "Please ask a question that clarifies the contradictory facts in the two statements above." Then, LLM generates a follow-up query q. r For example, a query like "What was the exact year Einstein received the Nobel Prize?". This query naturally integrates the current generation intent with the sample context, avoiding missing information or the introduction of noise.

[0054] (4) Dynamic retrieval and sentence generation. Using q r Retrieve Top-K related documents from external knowledge bases (such as Wikipedia). Let q and X... i-1 The retrieved relevant documents are concatenated into a new context, and the i-th sentence of the answer is regenerated. This process is repeated until an end marker is generated or the maximum length of the answer is reached.

[0055] This invention has been validated on three multi-hop question-answering datasets: HotpotQA, 2WikiMultihopQA, and IIRC. It significantly outperforms existing dynamic RAG methods such as FLARE and DRAGIN, and maintains stable gains on models of different sizes, from Llama2-7B to Llama3-8B, demonstrating its effectiveness and robustness.

[0056] Another embodiment of the present invention provides a dynamic retrieval enhancement generation system based on semantic contradiction driving, such as... Figure 3 As shown, it includes: The semantic sampling module is used to perform multi-path sampling on the input question and generate multiple candidate answers; The contradiction identification module is used to generate a formal answer sentence by sentence based on the input question. After each sentence is generated, it is compared with the candidate answers to identify contradictions and obtain contradiction pairs. The contradiction tracing module is used to generate follow-up queries based on the identified contradiction pairs; The dynamic retrieval and regeneration module is used to retrieve relevant documents from an external knowledge base using follow-up queries, regenerate the current sentence based on the relevant documents, and continue the subsequent generation process until the answer is completed.

[0057] The contradiction identification module employs a "one-vote veto" mechanism, meaning that a search is triggered as soon as a single sample contradicts the currently generated sentence. The contradiction tracing module automatically retains contextual information from conflicting samples when generating search queries, preventing ambiguity or missing information.

[0058] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be referred to the corresponding process in the aforementioned method embodiments, and will not be repeated here. Each of the above modules can be implemented entirely or partially through software, hardware, or a combination thereof.

[0059] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0060] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0061] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0062] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A dynamic retrieval enhancement generation method based on semantic contradiction-driven approach, characterized in that, Includes the following steps: Multi-path sampling is performed on the input question to generate multiple candidate answers; Generate a formal answer sentence by sentence based on the input question. After each sentence is generated, identify contradictions between it and the candidate answers to obtain contradiction pairs. Generate follow-up queries based on the identified contradictions; The system uses follow-up queries to retrieve relevant documents from an external knowledge base, regenerates the current sentence based on the relevant documents, and continues the subsequent generation process until the answer is completed.

2. The method according to claim 1, characterized in that, The process of multi-path sampling of the input question to generate multiple candidate answers includes: Based on the input question, multiple candidate answers are generated through random sampling using a large language model; Each candidate answer is stored in a structured manner at the sentence level to form a semantic sample set.

3. The method according to claim 2, characterized in that, The process of generating a formal answer sentence by sentence based on the input question, and identifying contradictions between each generated sentence and the candidate answers, includes: Using a large language model, formal answers are generated sentence by sentence based on the input question and the already generated content; Perform a semantic consistency comparison between the currently generated sentence and the semantic sample set; If the current sentence is found to contradict any sample in the semantic sample set in terms of core facts, then the sample is a conflicting sample, the sentence is a contradictory sentence, the sample and the sentence form a contradictory pair, and it is determined that external knowledge retrieval needs to be triggered.

4. The method according to claim 3, characterized in that, The core argument that there is a contradiction focuses on the consistency of key information while ignoring irrelevant details; the key information includes subject, event, time, and attributes.

5. The method according to claim 1, characterized in that, The process of generating follow-up queries based on identified contradictory pairs includes: inputting contradictory pairs into a large language model, and guiding the large language model to generate follow-up queries that clarify the contradictions and integrate the semantics of the currently generated content with the sample context.

6. The method according to claim 1, characterized in that, The generation of the probing search query uses a prompting engineering template to ensure that the query semantics are clear and directly correspond to the knowledge gap.

7. The method according to claim 1, characterized in that, When regenerating the current sentence, the original question, the generated prefix text, and the retrieved relevant documents are used together as the input context of the large language model.

8. A dynamic retrieval enhancement generation system based on semantic contradiction-driven approach, characterized in that, include: The semantic sampling module is used to perform multi-path sampling on the input question and generate multiple candidate answers; The contradiction identification module is used to generate a formal answer sentence by sentence based on the input question. After each sentence is generated, it is compared with the candidate answers to identify contradictions and obtain contradiction pairs. The contradiction tracing module is used to generate follow-up queries based on the identified contradiction pairs; The dynamic retrieval and regeneration module is used to retrieve relevant documents from an external knowledge base using follow-up queries, regenerate the current sentence based on the relevant documents, and continue the subsequent generation process until the answer is completed.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images