Fine tuning method and system for retrieval enhancement generative model based on causal reasoning

By modeling causal reasoning and defining knowledge gain scores, constructing a causal enhancement dataset, and fine-tuning the model, we solved the problems of accuracy and coherence in the dialogue system's answers in the context of long documents, and improved the naturalness and fluency of human-computer interaction.

CN120725148APending Publication Date: 2025-09-30FUZHOU UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510896272.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-30
Publication Date
2025-09-30

AI Technical Summary

Technical Problem

Existing dialogue systems find it difficult to effectively combine retrieval-enhanced generation technology when processing long document contexts, resulting in insufficient answer accuracy and coherence, affecting the natural fluency of human-computer interaction.

Method used

Through causal reasoning modeling, the causal relationship between the conversation context, knowledge documents and related unrelated knowledge is clarified, the knowledge benefit score is defined, a causal enhancement dataset is constructed, and the model is fine-tuned through intervention and counterfactual reasoning to optimize the robustness and response quality of the model under different knowledge input conditions.

Benefits of technology

This significantly improves the dialogue system's ability to understand context, enhances communication coherence and response quality, and ensures that generated answers are more natural and fluent.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120725148A_ABST
    Figure CN120725148A_ABST
Patent Text Reader

Abstract

The invention relates to a causal reasoning-based retrieval enhancement generative model fine tuning method and system, and the method comprises the following steps: A, modeling a causal relationship to reveal an influence mechanism of an input variable on related knowledge extraction and irrelevant knowledge filtering; step B, defining a knowledge revenue score (KGS), and calculating the knowledge revenue score through expert labeling or automatic evaluation to construct a causal enhancement data set (CED); c, designing a fine tuning strategy based on intervention and anti-factual reasoning, systematically performing knowledge variable intervention and anti-factual condition simulation, and enhancing the sensitivity of the model to related knowledge and the robustness of the model to irrelevant knowledge; and D, dynamically integrating the context and the knowledge to generate a high-quality response in a reasoning stage. The method is beneficial to improving the accuracy and stability of the dialogue generation model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically relates to a retrieval enhancement generation model fine-tuning method and system based on causal reasoning. Background Art

[0002] In recent years, dialogue systems have garnered widespread attention as a key research area in natural language processing (NLP). Building dialogue systems capable of natural and continuous interaction with humans has become a key research topic. Driven by deep learning technologies, dialogue systems have made significant progress.

[0003] Conversational Question Answering (CQA) is a type of natural language processing task that allows users to interact with the system through multiple rounds of dialogue to obtain information or solve problems. Unlike traditional single-round question-answering models, conversational question answering can understand context, maintain dialogue coherence, and dynamically adjust answers, making the interaction more natural and smooth. Currently, many well-known question-answering systems (such as those from OpenAI, Google, and Anthropic) use conversational question answering methods based on large language models (LLMs), which take into account the contextual information of multiple rounds of dialogue when generating answers. When the context length of the dialogue and the reference document exceeds the context window of the LLM, it is necessary to combine retrieval-augmented generation (RAG) technology to improve the accuracy and relevance of the answers.

[0004] Retrieval-augmented generation (RAG) technology combines information retrieval and generative artificial intelligence. By retrieving relevant information from an external knowledge base and providing it to the LLM in the form of prompts, RAG significantly enhances the model's performance in handling knowledge-intensive tasks such as language modeling and open-domain question answering. This technology was first proposed by the Facebook AI Research team in 2020 and quickly became a popular solution for large-scale model applications. For example, Borgeaud et al. proposed the Retrieval-Augmented Transformer (Retro) model, which integrates a large-scale retrieval mechanism with an autoregressive language model, enabling the language model to effectively utilize relevant information in the retrieval database. In addition, Adolphs et al. proposed a Knowledge to Response (K2R) model, the goal of which is to improve factual accuracy. K2R first generates a sequence of knowledge related to the conversation context, and then integrates this knowledge to synthesize the final response. Izacard and Grave developed a Fusion-in-Decoder (FiD) model that combines a generative model with paragraph retrieval. FiD encodes the question and multiple retrieved paragraphs separately, and combines the question with each paragraph independently during the decoding process, thereby effectively integrating multi-source information.

[0005] Building on retrieval enhancement, Xu et al. focused on the problem of dynamic interaction in long-term conversational contexts. They proposed a retrieval-enhanced generation model that integrates conversational context and model mechanisms, and further explored memory-based and summarization-based methods that summarize past conversation content to assist in response generation. Komeili et al. proposed an "internet-enhanced conversation generation" system that integrates real-time internet information. The system consists of two modules: a Transformer-based search query generation module that generates web search queries from the conversational context; and a response generation module that constructs the final response based on the retrieval results. In addition, Wu et al. proposed the SEG-CKRG model, which uses the Generative Knowledge Selection (GenSel) method for knowledge selection. GenSel not only selects existing knowledge from the retrieved candidate set but also generates new, extended knowledge, improving the interpretability of the knowledge selection process.

[0006] In addition, fine-tuning, as a method of transfer learning in deep learning, can be applied to all or part of the layers of a neural network. In recent years, researchers have developed parameter-efficient fine-tuning (PEFT) technology to reduce the computational cost of fine-tuning large language models and improve their adaptability to different tasks. At the same time, there are also studies that attempt to fine-tune pre-trained large language models in the RAG framework to further improve their performance on retrieval-enhanced generation tasks. The core goal of these studies is to optimize the adaptability of LLM in complex information retrieval and generation tasks so that it can be efficiently applied to diverse tasks in different fields. For example, Liu et al. proposed a two-stage instruction tuning method and designed a data construction strategy to significantly improve LLM's ability to integrate user-provided or retrieved context in conversational question-answering and retrieval-enhanced generation tasks. In addition, Zhang et al. proposed a retrieval-augmented fine-tuning (RAFT) strategy, which improved the performance of the model on RAG tasks in specific domains by effectively integrating domain knowledge. Summary of the Invention

[0007] The purpose of the present invention is to provide a causal reasoning-based retrieval enhancement generation model fine-tuning method and system, which can significantly improve the dialogue system's ability to understand the context, maintain the coherence of communication, and dynamically adjust the generated answers to make human-computer interaction more natural and smooth.

[0008] To achieve the above objectives, the present invention proposes a retrieval enhancement generation model fine-tuning method based on causal reasoning, which includes the following specific steps:

[0009] Step A: Causal modeling;

[0010] Through causal relationship modeling, the conversation context C, knowledge document KD, and related knowledge K are clearly defined. R and irrelevant knowledge K I The causal relationship between them; where the dialogue context C and the knowledge document KD are input variables, jointly affecting the relevant knowledge K R Extraction and irrelevant knowledge K I Filtering, thereby affecting the response quality of the generated model;

[0011] Step B: Definition of knowledge gain score and dataset construction;

[0012] We define a knowledge gain score (KGS) to quantify the contribution of each knowledge item to improving the quality of responses generated by the model. We collect and preprocess multiple datasets for retrieval-enhanced question answering tasks as the original dataset D. We then calculate the knowledge gain score for each knowledge item using expert annotation or automated evaluation methods. This allows us to select high-quality knowledge content and construct a causal enhancement dataset (CED). The CED includes explicit causal annotations to ensure that relevant and irrelevant knowledge can be distinguished during model fine-tuning.

[0013] Step C: Model fine-tuning based on causal reasoning;

[0014] Based on the causal relationship established in step A, a model fine-tuning strategy combining intervention reasoning and counterfactual reasoning is constructed. This strategy specifically includes the following three stages:

[0015] (1) Intervention reasoning stage:

[0016] The do operator of causal reasoning is used to intervene in the model input knowledge to create diverse training scenarios. By intervening in knowledge items, the influence of relevant and irrelevant knowledge on the model response generation is evaluated, thereby improving the sensitivity and robustness of the model. The overall intervention reasoning is as follows:

[0017]

[0018] Where y represents the response of the model, K represents the knowledge variable, do(K) represents the intervention state of the knowledge, z represents the potential context understanding state, C represents the dialogue context, and p(|) represents the conditional probability;

[0019] (2) Counterfactual reasoning stage:

[0020] Based on intervention reasoning, counterfactual reasoning is used to construct hypothetical scenarios and analyze the differences in the model's response quality under different knowledge input conditions. In other words, different knowledge input scenarios are simulated to evaluate and improve the model's efficiency in utilizing relevant knowledge and suppress the negative impact of irrelevant knowledge. The overall counterfactual reasoning is as follows:

[0021]

[0022] where k′ represents the counterfactual state of the hypothetical knowledge;

[0023] (3) Model optimization stage of causal enhancement:

[0024] During the model fine-tuning process, the robustness of the model under intervention reasoning and counterfactual reasoning conditions is enhanced through training data construction strategies and loss function optimization methods. Specifically, data samples containing different intervention and counterfactual conditions are designed, and the corresponding loss function is defined in combination with the knowledge gain score KGS to improve the model's ability to predict relevant knowledge K. Rsensitivity, reducing irrelevant knowledge K I interference;

[0025] Step D: Model inference and response generation;

[0026] In the actual reasoning stage, the preset system prompts, knowledge document KD and current dialogue context C are spliced ​​and input into the fine-tuned model as a whole to generate dialogue responses that meet the context requirements.

[0027] Furthermore, the step A specifically includes the following steps:

[0028] Step A1: Definition of causal variables and clarification of scenarios;

[0029] Define the following three causally related core variables:

[0030] (1) Dialogue context C: It consists of multiple rounds of historical dialogue content and provides the necessary background context for the model to understand the current dialogue intent and generate responses;

[0031] (2) Knowledge Document KD: It is a pre-built external knowledge collection, mainly including the following two types of knowledge fragments: 1) Related knowledge KD R : It can be combined with the conversation context to directly derive the knowledge fragments of the real response, which has a positive causal contribution to the quality of model generation; 2) irrelevant knowledge K I : Knowledge fragments that have no clear causal relationship with the real response may interfere with the model response;

[0032] (3) True response R: Given the dialogue context C and related knowledge K R Under these conditions, the correct answer or optimal answer generated by the model;

[0033] Through the above definitions, the model can distinguish the different roles of knowledge fragments in the knowledge document KD;

[0034] Step A2: Causal modeling and path analysis;

[0035] Based on the causal variables defined in step A1, analyze the causal relationships and impact paths between different variables, including:

[0036] (1) C→KD→K R :According to the dialogue context C, select relevant knowledge K in the knowledge document KD R , which is used to guide the model's subsequent response generation; the relevant knowledge acquisition process is as follows:

[0037] K R =f R (C,KD)+∈ R

[0038] Among them, f R (·) is the function of model selection knowledge, ∈ R is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness;

[0039] (2)KD→K I : Not affected by the context explicitly, but not related to knowledge K I Exists in the knowledge document KD as a potential interference factor;

[0040] (3) C, K R →R:Related knowledge K R Together with the dialogue context C, the response R of the generated model is determined, reflecting a direct causal path. The response generation process is as follows:

[0041] R=f resp (C,K R ,K I )+∈ resp

[0042] Among them, f resp (·) is the model response generating function, ∈ resp is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness;

[0043] (4)K I →R:Irrelevant knowledge K I It may affect the response R generation of the model in a potential interference way. This path is the interference path.

[0044] The above causal path clarifies the goal that needs to be focused on during model fine-tuning, namely maximizing the context C and related knowledge K R Positive causal influence on the true response R while minimizing irrelevant knowledge K I negative interference effects.

[0045] Furthermore, the step B specifically includes the following steps:

[0046] Step B1: Definition and calculation of knowledge benefit score;

[0047] The knowledge gain score (KGS) is defined to quantify the contribution of knowledge to the generation of high-quality responses by the model. The knowledge gain comprehensively considers the following three dimensions:

[0048] (1) Semantic similarity S sem ;

[0049] Calculate the semantic proximity between knowledge and true responses, use vector representation to obtain the embedding vectors of the two, and then evaluate them through cosine similarity to ensure the semantic accuracy of the model in knowledge selection;

[0050]

[0051] Among them, v K and v R They represent the semantic vector representation of knowledge K and true response R respectively, and |||| represents the Euclidean norm of the vector;

[0052] (2) Word coverage S cov ;

[0053] Calculate word-level overlap between the knowledge fragment and the true response, specifically defined as the ratio of the number of words shared between the two to the number of words in the true response. This dimension complements the semantic similarity metric and reflects the direct connection between the knowledge and the true response from the perspective of word overlap.

[0054]

[0055] Among them, || represents the cardinality of the set, that is, the number of elements in the set; W K With W R Represents the vocabulary set of knowledge fragment K and true response R, |W K ∩W R | represents the number of words in common between the two, |W R | represents the number of words in the true response, which is used for normalization so that the coverage is in the interval [0, 1];

[0056] (3) Conditional interaction information S cxmi ;

[0057] Evaluate the degree of conditional dependency between knowledge K, conversation context C, and true response R to ensure that the knowledge introduced meets the needs of the conversation context and makes a real contribution to generating true responses, rather than being redundant or irrelevant information.

[0058]

[0059] Where R is the true response, C is the conversation context, P(R|C) represents the probability of the model generating a true response based only on the conversation context C, and P(R|C,K) represents the probability of generating a true response R based on both the conversation context C and the knowledge K.

[0060] The final knowledge gain score (KGS) is a linear weighted combination of the above three dimensional indicators:

[0061] KGS(K,C,R)=α·S sem (K,R)+β·S cov (K,R)+γ·S cxmi (K,C,R)

[0062] Among them, α, β, and γ are weight coefficients, satisfying α+β+γ=1;

[0063] Step B2: Construction of causal enhancement dataset (CED);

[0064] We collect, process, and fuse multiple public datasets involving retrieval-enhanced question answering tasks as the original dataset D to construct a causal-enhanced dataset (CED). The general structure of a single sample in the causal-enhanced dataset includes:

[0065] (1) Complete knowledge document KD: external knowledge information outside the dialogue context, used to assist the model in generating responses;

[0066] (2) Complete conversation context C: includes the current conversation turn and all historical conversation turns, namely:

[0067]

[0068] in, Indicates the text concatenation symbol, t indicates the current dialogue turn number, u i 、r i are the user utterance and system response in round i respectively;

[0069] (3) True response R: The accurate answer text that the model should generate in the current round t; for true responses whose answers come from knowledge documents, the actual text of the span extracted from the document is used as the true response;

[0070] The causal enhancement dataset is processed and integrated, including the following steps:

[0071] Step B21: Conversation Segmentation:

[0072] Split a single multi-round dialogue sample in the original dataset D into multiple independent training samples. For a single sample (u1, r1, u2, r2, u3, r3) containing three rounds of dialogue, split it into three independent samples: (u1), (u1, r1, u2), (u1, r1, u2, r2, u3), and their corresponding true responses are r1, r2, and r3 respectively.

[0073] Step B22: Data enhancement and knowledge annotation:

[0074] For each segmented training sample, in addition to the complete knowledge document, the following knowledge information is additionally annotated:

[0075] 1) Related knowledge K R : If the knowledge document contains a true response, the document fragment containing the true response is directly used as the relevant knowledge; otherwise, the knowledge fragment with the highest knowledge benefit score is selected as the relevant knowledge;

[0076] 2) Irrelevant knowledge K I : Select knowledge fragments from knowledge documents that are mutually exclusive with related knowledge fragments and have low knowledge benefits;

[0077] 3) Replaced related knowledge fragment K RR : Another knowledge fragment that is mutually exclusive with the above two knowledge fragments and has high knowledge benefits;

[0078] 4) Replaced irrelevant knowledge fragment K RI : Additional knowledge fragments that are mutually exclusive with the above three knowledge fragments and have low knowledge benefits;

[0079] Therefore, a single training sample in the final constructed causal enhancement dataset CED is recorded as:

[0080] (C,K,R,K R ,K I ,K RR ,K RI )

[0081] Step B3: The division and purpose of the data set are clearly defined;

[0082] The constructed causal enhancement dataset is divided into training set and validation set by random sampling.

[0083] Furthermore, the step C specifically includes the following steps:

[0084] Step C1: intervention reasoning based on the do operator;

[0085] For the samples in the causal enhancement dataset, the input knowledge fragments are used as intervention variables. By combining, splicing, and replacing the knowledge document content, multiple intervention scenarios are constructed to measure the impact of changes in knowledge variables on the model generation response. Specifically, the knowledge benefit scores of the model response under different input conditions are calculated respectively, so as to obtain the relevant knowledge benefits under the intervention conditions. and irrelevant knowledge gains

[0086] Step C2: Reasoning based on counterfactual reasoning;

[0087] Based on the variables defined in the causal enhancement dataset, different counterfactual input scenarios are constructed by retaining or deleting the knowledge document content. The knowledge gain scores of the model under different knowledge provision scenarios are calculated to analyze the changes in the model response generation quality, thereby obtaining the relevant knowledge gains under the corresponding conditions. and irrelevant knowledge gains

[0088] Step C3: Comprehensive evaluation of knowledge benefits;

[0089] Relevant knowledge gains calculated based on steps C1 and C2 and irrelevant knowledge gains The following comprehensive indicators are further defined and calculated: Comprehensive knowledge gain (KGS) R ; Comprehensive irrelevant knowledge gain (KGS I ; Relevant knowledge benefit gap ΔKGS R ; Irrelevant knowledge benefit gap ΔKGS I ;

[0090] Step C4: auxiliary task construction;

[0091] Based on the comprehensive relevant knowledge gain and irrelevant knowledge gain results calculated in step C3, targeted auxiliary tasks are constructed. The auxiliary tasks are designed to improve the model's ability to effectively utilize relevant knowledge during training, while suppressing the model's interference effect on irrelevant knowledge, thereby improving the robustness and quality of response generation.

[0092] Step C5: multi-task fine-tuning optimization strategy;

[0093] During fine-tuning, the tasks are divided into:

[0094] Main task: the difference between the predicted response and the true response, calculating the main task loss;

[0095] Auxiliary task: Calculate the auxiliary task loss based on the comprehensive relevant knowledge gain, comprehensive irrelevant knowledge gain, relevant knowledge gain gap, and irrelevant knowledge gain gap;

[0096] During fine-tuning, the back-propagation algorithm is used to calculate the gradient of network parameters, and the parameters are updated by the stochastic gradient descent algorithm;

[0097] Step C6: fine-tune the stopping criterion;

[0098] When the value of the loss function during fine-tuning is less than a preset threshold or the maximum number of iterations is reached, fine-tuning training of the large language model is stopped.

[0099] Furthermore, the step C1 specifically includes the following steps:

[0100] Step C11: Design the intervention scenarios of the do operator; specifically, design the following three intervention scenarios:

[0101] Scenario 1 (no intervention): No modification is made to the original knowledge document. This serves as a baseline control group to observe the impact of the original knowledge document.

[0102] Scenario ② (related knowledge intervention): Only the content of the related knowledge fragments is intervened, and the impact of the changes on the model-generated response R is evaluated by modifying the related knowledge fragments;

[0103] Scenario ③ (irrelevant knowledge intervention): Only the content of irrelevant knowledge fragments is intervened, and the interference effect of irrelevant knowledge on the model-generated response R is evaluated by modifying irrelevant knowledge fragments;

[0104] Step C12: knowledge intervention;

[0105] In fine-tuning training, relevant knowledge fragments are replaced with spare relevant knowledge fragments, and irrelevant knowledge fragments are replaced with spare irrelevant knowledge fragments, and then the knowledge benefit scores of the corresponding scenarios are calculated;

[0106] Step C13: calculation of knowledge utility of do operator;

[0107] DO operator related knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ② (related knowledge intervention):

[0108]

[0109] DO operator irrelevant knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ③ (irrelevant knowledge intervention):

[0110]

[0111] in, represents text concatenation, KD represents complete knowledge document, K R Represents relevant knowledge, K I represents irrelevant knowledge, and do(·) represents intervention in knowledge.

[0112] Furthermore, the step C2 specifically includes the following steps:

[0113] Step C21: Counterfactual Scenario Design

[0114] Design the following three typical counterfactual scenarios:

[0115] Scenario 1 (no knowledge provided): No knowledge input, used to evaluate the response quality of the model in the absence of external knowledge, and to observe whether false or invalid responses are generated;

[0116] Scenario 2 (providing only relevant knowledge): Only clear relevant knowledge is provided to analyze the response quality of the model under non-interference conditions;

[0117] Scenario 3 (providing only irrelevant knowledge): Provide only irrelevant knowledge to assess the model's sensitivity and interference to irrelevant knowledge and determine whether it produces erroneous or misleading responses.

[0118] Step C22: Explore the following questions:

[0119] ① What kind of response will the model generate if no knowledge input is provided?

[0120] ② What is the quality of the model’s response if only relevant knowledge documents are referenced?

[0121] ③ What is the quality of the model’s response if it only refers to irrelevant knowledge documents?

[0122] Step C23: Counterfactual Knowledge Utility Calculation

[0123] Counterfactual knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ② (providing relevant knowledge):

[0124]

[0125] Counterfactual irrelevant knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ③ (providing irrelevant knowledge):

[0126]

[0127] Among them, null means no knowledge fragment input.

[0128] Furthermore, the step C3 specifically includes the following steps:

[0129] Step C31: Calculation of comprehensive relevant knowledge benefits

[0130] Based on the relevant knowledge gains calculated by do operator intervention and counterfactual reasoning, the comprehensive relevant knowledge gain KGS is defined R is the weighted average of the two:

[0131]

[0132] Among them, λ∈[0,1] is an adjustable weight parameter used to control the contribution ratio of the do operator and counterfactual reasoning; Step C32: Calculation of comprehensive irrelevant knowledge benefits

[0133] Based on the irrelevant knowledge benefits calculated by do operator intervention and counterfactual reasoning, the comprehensive irrelevant knowledge benefit KGS is defined I is the weighted average of the two:

[0134]

[0135] Among them, μ∈[0,1] is an adjustable weight parameter, which is used to weigh the role of intervention reasoning and counterfactual reasoning in the evaluation of irrelevant knowledge utility;

[0136] Step C33: Calculation of relevant knowledge benefit gap

[0137] Based on the relevant knowledge gains obtained from the do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated to measure the consistency of the two causal reasoning methods:

[0138]

[0139] Step C34: Calculation of irrelevant knowledge benefit gap

[0140] Based on the irrelevant knowledge benefits obtained by do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated:

[0141]

[0142] Furthermore, the step C4 specifically includes the following steps:

[0143] Step C41: Maximize the comprehensive relevant knowledge gain KGS R ;

[0144] Step C42: Minimize the comprehensive irrelevant knowledge gain KGS I ;

[0145] Step C43: Minimize the knowledge gain gap ΔKGS between the do operator and counterfactual reasoning R ;

[0146] Step C44: Minimize the irrelevant knowledge benefit gap ΔKGS between the do operator and counterfactual reasoning I .

[0147] Furthermore, the step C5 specifically includes the following steps:

[0148] Step C51: Construction of contrastive loss

[0149] Construct a contrast loss based on the comprehensive relevant knowledge gain and the comprehensive irrelevant knowledge gain to significantly improve the discrimination between relevant knowledge and irrelevant knowledge:

[0150]

[0151] Among them, m is the interval parameter of contrast loss, which ensures that the benefits of relevant knowledge are significantly higher than those of irrelevant knowledge;

[0152] Step C52: Construction of consistency loss

[0153] The consistency loss is constructed based on the relevant knowledge benefit gap and the irrelevant knowledge benefit gap to ensure the consistency of the model's knowledge utilization under intervention and counterfactual conditions:

[0154]

[0155] Step C53: Construction of cross entropy loss

[0156] The original loss function of the model when predicting the response is defined as the standard cross entropy loss, which is used to ensure that the generated response matches the true answer:

[0157]

[0158] Among them, y i represents the true probability distribution of the i-th token of the true response, and its value range is i=1,2,…,T, where T is the length of the true response sequence, represents the probability distribution predicted by the model;

[0159] Step C54: Comprehensive design of final loss

[0160] The above loss terms are combined and weighted to form the final loss function, which is used for overall optimization:

[0161]

[0162] Among them, ω1 and ω2 are weight coefficients, which are determined by adjusting the validation set to optimize the overall performance and generalization ability of the model in the response generation task.

[0163] The present invention also provides a retrieval enhancement generation model fine-tuning system based on causal reasoning, which includes a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0164] Compared with the prior art, the present invention has the following beneficial effects:

[0165] 1) This invention clearly defines the conversation context and the causal paths of various knowledge through causal relationship modeling, which can accurately distinguish the key factors affecting the quality of model generation, improve the effectiveness of knowledge extraction, and thus enhance the model's reasoning ability and response quality.

[0166] 2) This paper defines a knowledge gain score (KGS) and constructs a causal enhancement dataset (CED) to ensure that the training data has clear causal annotations, enabling the model to accurately identify high-value knowledge during fine-tuning, improve the utilization of relevant knowledge, and effectively filter out irrelevant knowledge, thereby enhancing the model's learning ability and generalization performance.

[0167] 3) The present invention fine-tunes the model based on intervention reasoning and counterfactual reasoning, systematically intervenes in knowledge input and constructs hypothetical scenarios, optimizes the robustness of the model under different knowledge input conditions, enhances sensitivity to relevant knowledge, and reduces the interference of irrelevant knowledge on response quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0168] Figure 1 This is a flowchart of an implementation method of a retrieval enhancement generation model fine-tuning method based on causal reasoning provided by an embodiment of the present invention;

[0169] Figure 2 This is an architectural diagram of a retrieval enhancement generation model fine-tuning method based on causal reasoning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0170] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0171] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0172] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0173] like Figure 1-2 As shown, this embodiment provides a retrieval enhancement generation model fine-tuning method based on causal reasoning, including the following steps:

[0174] Step A: Causal modeling.

[0175] Through causal relationship modeling, the conversation context (C), knowledge documents (KD), and relevant knowledge (KD) are clearly defined. R ) and irrelevant knowledge (IrrelevantKnowledge, K I ) between them; where the dialogue context C and knowledge document KD serve as input variables and jointly influence the relevant knowledge K R Extraction and irrelevant knowledge K I This step aims to clarify the causal path and influence mechanism between various factors and provide a theoretical basis for subsequent fine-tuning.

[0176] Step B: Knowledge gain score definition and dataset construction.

[0177] A knowledge gain score (KGS) is defined to quantify the contribution of each knowledge item to improving the quality of responses generated by the model. Specifically, multiple datasets for retrieval-enhanced question-answering tasks are collected and preprocessed as the original dataset D. Combined with expert annotation or automated evaluation methods, the KGS is calculated for each knowledge item. This allows for the selection of high-quality knowledge content and the construction of a causally enriched dataset (CED). The CED includes explicit causal annotations to ensure that relevant and irrelevant knowledge can be clearly distinguished during model fine-tuning, thereby improving the model's effective learning ability.

[0178] Step C: Model fine-tuning based on causal inference.

[0179] Based on the causal relationship established in step A, a model fine-tuning strategy combining interventional reasoning and counterfactual reasoning is constructed. This strategy specifically includes the following three stages:

[0180] (1) Intervention reasoning stage:

[0181] The do-operator of causal reasoning is used to intervene in the model input knowledge, that is, to artificially manipulate the knowledge combination to create diverse training scenarios; by intervening in knowledge items, the influence of relevant and irrelevant knowledge on the model response generation is evaluated, thereby improving the sensitivity and robustness of the model. The overall intervention reasoning is as follows:

[0182]

[0183] Among them, y represents the response of the model, K represents the knowledge variable, do(K) represents the intervention state of knowledge, z is the potential context understanding state, C is the dialogue context, and p(|) represents the conditional probability.

[0184] (2) Counterfactual reasoning stage:

[0185] Based on intervention reasoning, counterfactual reasoning is used to construct hypothetical scenarios to analyze the differences in the model's response quality under different knowledge input conditions. This is done by simulating different knowledge input scenarios (e.g., "How would the model's response change if different knowledge were provided"), thereby evaluating and improving the model's efficiency in utilizing relevant knowledge and suppressing the negative impact of irrelevant knowledge. The overall counterfactual reasoning is as follows:

[0186]

[0187] where k′ represents the counterfactual state of the assumed knowledge.

[0188] (3) Model optimization stage of causal enhancement:

[0189] During the model fine-tuning process, the robustness of the model under intervention reasoning and counterfactual reasoning conditions is enhanced through training data construction strategies and loss function optimization methods. Specifically, data samples containing different intervention and counterfactual conditions are designed, and the corresponding loss function is defined in combination with the knowledge gain score KGS to improve the model's ability to predict relevant knowledge K. R sensitivity, reducing irrelevant knowledge K I interference, thus ensuring that the model can generate consistent, high-quality responses in different knowledge scenarios.

[0190] Step D: Model inference and response generation.

[0191] During the actual reasoning phase, the preset system prompt, knowledge document KD, and current conversation context C are combined and fed into the fine-tuned model as a whole to generate a conversational response that meets the contextual requirements. By combining the improved model performance of causal reasoning fine-tuning methods, the system can dynamically integrate context and related knowledge to generate highly coherent, high-quality natural language responses.

[0192] In this embodiment, step A specifically includes the following steps:

[0193] Step A1: Definition of causal variables and clarification of scenarios.

[0194] In the Retrieval-Augmented Generation (RAG) task, the model first searches and identifies knowledge fragments closely related to the current conversation in external knowledge documents based on the given conversation context. These fragments serve as the causal basis ("cause") for generating subsequent conversation responses, while effectively ignoring or reducing the interference effects of irrelevant knowledge fragments. Therefore, this paper defines the following three causally related core variables:

[0195] (1) Conversation Context (C): It consists of multiple rounds of historical conversation content and provides the necessary background context for the model to understand the current conversation intent and generate responses;

[0196] (2) Knowledge Documents (KD): A pre-built external knowledge collection, mainly including the following two types of knowledge fragments: 1) Relevant Knowledge (KD) R):It can be combined with the conversation context to directly derive the knowledge fragments of the real response, which has a positive causal contribution to the quality of model generation; 2) Irrelevant Knowledge (K I ): Knowledge fragments that have no clear causal relationship with the real response may interfere with the model response;

[0197] (3) Ground-truth Response (R): Given a conversation context C and relevant knowledge K R Under these conditions, the correct answer or optimal answer generated by the model.

[0198] Through the above definition, the model can accurately distinguish the different roles of knowledge fragments in the knowledge document KD to enhance the utilization of relevant knowledge and suppress the interference caused by irrelevant knowledge.

[0199] Step A2: Causal modeling and path analysis.

[0200] Based on the causal variables defined in step A1, the present invention establishes a clear and specific causal graph model, and then analyzes the causal relationship and impact path between different variables, specifically including:

[0201] (1) C→KD→K R :According to the dialogue context C, select relevant knowledge K in the knowledge document KD R , which is used to guide the model’s subsequent response generation; the relevant knowledge acquisition process (based on context selection) is as follows:

[0202] K R =f R (C,KD)+∈ R

[0203] Among them, f R (·) is the function of model selection knowledge, ∈ R is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness.

[0204] (2)KD→K I : Not affected by the context explicitly, but not related to knowledge K I It exists in the knowledge document KD as a potential interference factor.

[0205] (3) C, K R →R:Related knowledge K R Together with the dialogue context C, the response R of the generated model is determined, reflecting a direct causal path. The response generation process is as follows:

[0206] R=f resp (C,K R ,KI )+∈ resp

[0207] Among them, f resp (·) is the model response generating function, ∈ resp is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness.

[0208] (4)K I →R:Irrelevant knowledge K I The path that may affect the model's response R generation in a potential interference manner is called the interference path.

[0209] The above causal path clarifies the goal that needs to be focused on during model fine-tuning, namely maximizing the context C and related knowledge K R Positive causal influence on the true response R while minimizing irrelevant knowledge K I negative interference effects.

[0210] In this embodiment, step B specifically includes the following steps:

[0211] Step B1: Definition and calculation of knowledge benefit score.

[0212] A knowledge gain score (KGS) is defined to quantify the contribution of knowledge to the generation of high-quality responses by the model. The knowledge gain considers the following three dimensions:

[0213] (1) Semantic Similarity sem );

[0214] Calculate the semantic proximity between knowledge and true responses, use vector representation to obtain their embedding vectors, and then evaluate them through cosine similarity to ensure the semantic accuracy of the model in knowledge selection;

[0215]

[0216] Among them, v K and v R They represent the semantic vector representation of knowledge K and true response R respectively, and |||| represents the Euclidean norm of the vector.

[0217] (2) Word Coverage cov );

[0218] Calculate word-level overlap between the knowledge fragment and the true response, specifically defined as the ratio of the number of words shared between the two to the number of words in the true response. This dimension complements the semantic similarity metric and reflects the direct connection between the knowledge and the true response from the perspective of word overlap.

[0219]

[0220] Among them, || represents the cardinality of the set, that is, the number of elements in the set; W K With W R Represents the vocabulary set of knowledge fragment K and true response R, |W K ∩W R | represents the number of words in common between the two, |W R | represents the number of words in the true response, which is used for normalization so that the coverage is in the interval [0, 1].

[0221] (3) Conditional Cross Mutual Information (S cxmi );

[0222] Evaluate the degree of conditional dependency between knowledge K, conversation context C, and true response R to ensure that the introduced knowledge meets the needs of the conversation context and contributes to the generation of true responses, rather than being redundant or irrelevant information. CXMI uses a probabilistic model to calculate the degree of change in the probability of true responses before and after knowledge introduction.

[0223]

[0224] Where R is the true response, C is the conversation context, P(R|C) represents the probability that the model generates a true response based only on the conversation context C, and P(R|C,K) represents the probability of generating a true response R based on both the conversation context C and the knowledge K.

[0225] The final knowledge gain score (KGS) is a linear weighted combination of the above three dimensional indicators:

[0226] KGS(K,C,R)=α·S sem (K,R)+β·S cov (K,R)+γ·S cxmi (K,C,R)

[0227] Among them, α, β, and γ are weight coefficients, satisfying α+β+γ=1.

[0228] Step B2: Causal Enhancement Dataset (CED) construction.

[0229] We collect, process, and fuse multiple public datasets related to retrieval-enhanced question answering tasks as the original dataset D to construct a causally enriched dataset (CED). The general structure of a single sample in a causally enriched dataset includes:

[0230] (1) Complete knowledge document KD: external knowledge information outside the dialogue context, used to assist the model in generating responses;

[0231] (2) Complete conversation context C: includes the current conversation turn and all historical conversation turns, namely:

[0232]

[0233] in, Indicates the text concatenation symbol, t indicates the current dialogue turn number, u i 、r i are the user utterance and system response in round i respectively;

[0234] (3) True response R: The accurate answer text that the model should generate in the current round t; for true responses whose answers come from knowledge documents, the actual text of the span extracted from the document is used as the true response.

[0235] The causal enhancement dataset is processed and integrated, including the following steps:

[0236] Step B21: Conversation Segmentation:

[0237] The single multi-round dialogue sample in the original dataset D is divided into multiple independent training samples. For a single sample (u1, r1, u2, r2, u3, r3) containing three rounds of dialogue, it is divided into three independent samples: (u1), (u1, r1, u2), (u1, r1, u2, r2, u3), and their corresponding true responses are r1, r2, and r3 respectively.

[0238] Step B22: Data enhancement and knowledge annotation:

[0239] For each segmented training sample, in addition to the complete knowledge document, the following knowledge information is additionally annotated:

[0240] 1) Related knowledge K R : If the knowledge document contains a true response, the document fragment containing the true response is directly used as the relevant knowledge; otherwise, the knowledge fragment with the highest knowledge benefit score is selected as the relevant knowledge;

[0241] 2) Irrelevant knowledge K I : Select knowledge fragments from knowledge documents that are mutually exclusive with related knowledge fragments and have low knowledge benefits;

[0242] 3) Replaced related knowledge fragment K RR : Another knowledge fragment that is mutually exclusive with the above two knowledge fragments and has high knowledge benefits;

[0243] 4) Replaced irrelevant knowledge fragment K RI : Additional knowledge fragments that are mutually exclusive with the above three knowledge fragments and have low knowledge benefits.

[0244] Therefore, a single training sample in the final constructed causal enhancement dataset CED is recorded as:

[0245] (C,K,R,K R ,K I ,K RR ,K RI )

[0246] Step B3: The division and purpose of the data set are clearly defined;

[0247] The dataset is divided into training set and validation set by random sampling;

[0248] Training Set: used for training and optimizing model parameters;

[0249] Validation Set: used for model performance evaluation and hyperparameter adjustment;

[0250] This partitioning strategy ensures the adequacy of model training while guaranteeing the effectiveness and fairness of verification and evaluation.

[0251] In this embodiment, step C specifically includes the following steps:

[0252] Step C1: Intervention reasoning based on the do operator.

[0253] The do operator is a symbolic operator used to represent intervention operations, which can be used to explore how the artificial manipulation of certain variables affects the changes in other variables. In this invention, for the samples in the causal enhancement dataset, the input knowledge fragments are used as intervention variables. By combining, splicing, and replacing the knowledge document content, a variety of intervention scenarios are constructed to measure the degree of influence of changes in knowledge variables on the model generation response; specifically, the knowledge gain scores of the model response under different input conditions are calculated respectively, so as to obtain the relevant knowledge gain under the intervention conditions. and irrelevant knowledge gains

[0254] In this embodiment, step C1 specifically includes the following steps:

[0255] Step C11: Design the intervention scenarios of the do operator; specifically, design the following three intervention scenarios:

[0256] Scenario 1 (no intervention): No modification is made to the original knowledge document. This serves as a baseline control group to observe the impact of the original knowledge document.

[0257] Scenario ② (related knowledge intervention): Only the content of the related knowledge fragments is intervened, and the impact of the changes on the model-generated response R is evaluated by modifying the related knowledge fragments;

[0258] Scenario ③ (Irrelevant knowledge intervention): Only the content of irrelevant knowledge fragments is intervened, and the interference effect of irrelevant knowledge on the model generation response R is evaluated by modifying irrelevant knowledge fragments.

[0259] Step C12: knowledge intervention;

[0260] In fine-tuning training, relevant knowledge fragments are replaced with spare relevant knowledge fragments, and irrelevant knowledge fragments are replaced with spare irrelevant knowledge fragments, and then the knowledge benefit scores of the corresponding scenarios are calculated.

[0261] Step C13: calculation of knowledge utility of do operator;

[0262] DO operator related knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ② (related knowledge intervention):

[0263]

[0264] DO operator irrelevant knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ③ (irrelevant knowledge intervention):

[0265]

[0266] in, represents text concatenation, KD represents complete knowledge document, K R Represents relevant knowledge, K I represents irrelevant knowledge, and do(·) represents intervention in knowledge.

[0267] Step C2: Reasoning based on counterfactual reasoning.

[0268] Counterfactual reasoning is a method of exploring possible outcomes based on known facts by setting hypothetical conditions that are contrary to the facts. Its key components are as follows:

[0269] (1) Factual Event: a fact or event that has occurred;

[0270] (2) Counterfactual Condition: an assumed situation that is opposite to or different from the actual event;

[0271] (3) Expected Outcome: The possible outcome derived based on the assumed conditions.

[0272] Based on the variables defined in the causal enhancement dataset, this paper systematically retains or deletes the content of knowledge documents, constructs different counterfactual input scenarios, and calculates the knowledge benefit scores of the model under different knowledge provision scenarios to analyze the changes in the model response generation quality, thereby obtaining the relevant knowledge benefits under the corresponding conditions. and irrelevant knowledge gains

[0273] In this embodiment, step C2 specifically includes the following steps:

[0274] Step C21: Counterfactual Scenario Design

[0275] Design the following three typical counterfactual scenarios:

[0276] Scenario 1 (no knowledge provided): No knowledge input, used to evaluate the response quality of the model in the absence of external knowledge, and to observe whether false or invalid responses are generated;

[0277] Scenario 2 (providing only relevant knowledge): Only clear relevant knowledge is provided to analyze the response quality of the model under non-interference conditions;

[0278] Scenario ③ (providing only irrelevant knowledge): Only irrelevant knowledge is provided to evaluate the model’s sensitivity and interference to irrelevant knowledge and determine whether it produces erroneous or misleading responses.

[0279] Step C22: Counterfactual reasoning intuitive explanation, that is, exploring the following questions:

[0280] ① What kind of response will the model generate if no knowledge input is provided?

[0281] ② What is the quality of the model’s response if only relevant knowledge documents are referenced?

[0282] ③ What is the response quality of the model if it only refers to irrelevant knowledge documents?

[0283] Step C23: Counterfactual Knowledge Utility Calculation

[0284] Counterfactual knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ② (providing relevant knowledge):

[0285]

[0286] Counterfactual irrelevant knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ③ (providing irrelevant knowledge):

[0287]

[0288] Among them, null means no knowledge fragment input.

[0289] Step C3: Comprehensive evaluation of knowledge benefits.

[0290] Relevant knowledge gains calculated based on steps C1 and C2 and irrelevant knowledge gains The following comprehensive indicators are further defined and calculated: Comprehensive knowledge gain (KGS) R ; Comprehensive irrelevant knowledge gain (KGS I ; Relevant knowledge benefit gap ΔKGS R ; Irrelevant knowledge benefit gap ΔKGS I .

[0291] These indicators help to accurately quantify the model's sensitivity and robustness to knowledge fragments, and clearly distinguish the positive contribution of relevant knowledge from the negative interference of irrelevant knowledge.

[0292] In this embodiment, step C3 specifically includes the following steps:

[0293] Step C31: Calculation of comprehensive relevant knowledge benefits

[0294] Based on the relevant knowledge gains calculated by do operator intervention and counterfactual reasoning, the comprehensive relevant knowledge gain KGS is defined R is the weighted average of the two:

[0295]

[0296] Among them, λ∈[0, 1] is an adjustable weight parameter used to control the contribution ratio of the do operator and counterfactual reasoning.

[0297] Step C32: Calculation of comprehensive irrelevant knowledge benefits

[0298] Based on the irrelevant knowledge benefits calculated by do operator intervention and counterfactual reasoning, the comprehensive irrelevant knowledge benefit KGS is defined I is the weighted average of the two:

[0299]

[0300] Among them, μ∈[0, 1] is an adjustable weight parameter, which is used to weigh the role of intervention reasoning and counterfactual reasoning in the evaluation of irrelevant knowledge utility.

[0301] Step C33: Calculation of relevant knowledge benefit gap

[0302] Based on the relevant knowledge gains obtained from the do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated to measure the consistency of the two causal reasoning methods:

[0303]

[0304] Step C34: Calculation of irrelevant knowledge benefit gap

[0305] Based on the irrelevant knowledge benefits obtained by do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated:

[0306]

[0307] Step C4: Auxiliary task construction.

[0308] Based on the comprehensive relevant knowledge gain and irrelevant knowledge gain results calculated in step C3, targeted auxiliary tasks are constructed; the auxiliary tasks are designed to improve the model's ability to effectively utilize relevant knowledge during training, while suppressing the model's interference effect on irrelevant knowledge, thereby improving the robustness and quality of response generation.

[0309] In this embodiment, step C4 specifically includes the following steps:

[0310] Step C41: Maximize the comprehensive relevant knowledge gain KGS R ;

[0311] Step C42: Minimize the comprehensive irrelevant knowledge gain KGS I ;

[0312] Step C43: Minimize the knowledge gain gap ΔKGS between the do operator and counterfactual reasoning R ;

[0313] Step C44: Minimize the irrelevant knowledge benefit gap ΔKGS between the do operator and counterfactual reasoning I .

[0314] Step C5: Multi-task fine-tuning optimization strategy.

[0315] During fine-tuning, the tasks are divided into:

[0316] Main Task: The difference between the predicted response and the true response, and the calculation of the main task loss;

[0317] Auxiliary Task: Calculate the auxiliary task loss based on the comprehensive relevant knowledge gain, comprehensive irrelevant knowledge gain, relevant knowledge gain gap, and irrelevant knowledge gain gap.

[0318] During the fine-tuning process, the backpropagation algorithm is used to calculate the network parameter gradients, and the parameters are updated through the stochastic gradient descent algorithm (SGD).

[0319] In this embodiment, step C5 specifically includes the following steps:

[0320] Step C51: Construction of Contrastive Loss

[0321] Construct a contrast loss based on the comprehensive relevant knowledge gain and the comprehensive irrelevant knowledge gain to significantly improve the discrimination between relevant knowledge and irrelevant knowledge:

[0322]

[0323] Among them, m is the margin parameter of contrast loss, which ensures that the gain of relevant knowledge is significantly higher than that of irrelevant knowledge.

[0324] Step C52: Construction of Consistency Loss

[0325] The consistency loss is constructed based on the relevant knowledge benefit gap and the irrelevant knowledge benefit gap to ensure the consistency of the model's knowledge utilization under intervention and counterfactual conditions:

[0326]

[0327] Step C53: Construction of model original loss (cross entropy loss, Cross-Entropy Loss)

[0328] The original loss function of the model when predicting the response is defined as the standard cross entropy loss, which is used to ensure that the generated response matches the true answer:

[0329]

[0330] Among them, y i represents the true probability distribution of the i-th token of the true response, and its value range is i=1,2,…,T, where T is the length of the true response sequence, represents the probability distribution predicted by the model.

[0331] Step C54: Comprehensive design of final loss

[0332] The above loss terms are combined and weighted to form the final loss function, which is used for overall optimization:

[0333]

[0334] Among them, ω1 and ω2 are weight coefficients, which are determined by adjusting the validation set to optimize the overall performance and generalization ability of the model in the response generation task.

[0335] Step C6: Fine-tune the stopping criterion.

[0336] When the value of the loss function during fine-tuning is less than a preset threshold or the maximum number of iterations is reached, fine-tuning training of the large language model is stopped.

[0337] In this embodiment, the performance of this method and other baseline methods are compared on datasets such as PopQA after being applied to the Llama-3-Instruct8B pre-trained language model, and the results are shown in Table 1. As can be seen from Table 1, the performance of this method is better than other methods.

[0338] Table 1 Performance comparison of this method and the baseline method on PopQA and other datasets after applying the Llama-3-Instruct8B pre-trained language model

[0339]

[0340] This embodiment also provides a retrieval enhancement generation model fine-tuning system based on causal reasoning, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0341] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0342] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0343] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0344] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0345] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A retrieval enhancement generative model fine-tuning method based on causal reasoning, characterized by: The following steps are involved: Step A: Causal modeling; Through causal relationship modeling, the conversation context C, knowledge document KD, and related knowledge K are clearly defined. R and irrelevant knowledge K I The causal relationship between them; where the dialogue context C and the knowledge document KD are input variables, jointly affecting the relevant knowledge K R Extraction and irrelevant knowledge K I Filtering, thereby affecting the response quality of the generated model; Step B: Definition of knowledge gain score and dataset construction; We define a knowledge gain score (KGS) to quantify the contribution of each knowledge item to improving the quality of responses generated by the model. We collect and preprocess multiple datasets for retrieval-enhanced question answering tasks as the original dataset D. We then calculate the knowledge gain score for each knowledge item using expert annotation or automated evaluation methods. This allows us to select high-quality knowledge content and construct a causal enhancement dataset (CED). The CED includes explicit causal annotations to ensure that relevant and irrelevant knowledge can be distinguished during model fine-tuning. Step C: Model fine-tuning based on causal reasoning; Based on the causal relationship established in step A, a model fine-tuning strategy combining intervention reasoning and counterfactual reasoning is constructed. This strategy specifically includes the following three stages: (1) Intervention reasoning stage: The do operator of causal reasoning is used to intervene in the model input knowledge to create diverse training scenarios. By intervening in knowledge items, the influence of relevant and irrelevant knowledge on the generation of model responses is evaluated, thereby improving the sensitivity and robustness of the model. The overall intervention reasoning is as follows: Where y represents the response of the model, K represents the knowledge variable, do(K) represents the intervention state of the knowledge, z represents the potential context understanding state, C represents the dialogue context, and p(|) represents the conditional probability; (2) Counterfactual reasoning stage: Based on intervention reasoning, counterfactual reasoning is used to construct hypothetical scenarios and analyze the differences in the model's response quality under different knowledge input conditions. In other words, different knowledge input scenarios are simulated to evaluate and improve the model's efficiency in utilizing relevant knowledge and suppress the negative impact of irrelevant knowledge. The overall counterfactual reasoning is as follows: where k′ represents the counterfactual state of the hypothetical knowledge; (3) Model optimization stage of causal enhancement: During the model fine-tuning process, the robustness of the model under intervention reasoning and counterfactual reasoning conditions is enhanced through training data construction strategies and loss function optimization methods. Specifically, data samples containing different intervention and counterfactual conditions are designed, and the corresponding loss function is defined in combination with the knowledge gain score KGS to improve the model's ability to predict relevant knowledge K. R sensitivity, reducing irrelevant knowledge K I interference; Step D: Model inference and response generation; In the actual reasoning stage, the preset system prompts, knowledge document KD and current dialogue context C are spliced ​​and input into the fine-tuned model as a whole to generate dialogue responses that meet the context requirements.

2. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 1 is characterized in that: The step A specifically comprises the following steps: Step A1: Definition of causal variables and clarification of scenarios; Define the following three causally related core variables: (1) Dialogue context C: It consists of multiple rounds of historical dialogue content and provides the necessary background context for the model to understand the current dialogue intent and generate responses; (2) Knowledge Document KD: It is a pre-built external knowledge collection, mainly including the following two types of knowledge fragments: 1) Related knowledge KD R : It can be combined with the conversation context to directly derive the knowledge fragments of the real response, which has a positive causal contribution to the quality of model generation; 2) irrelevant knowledge K I : Knowledge fragments that have no clear causal relationship with the real response may interfere with the model response; (3) True response R: Given the dialogue context C and related knowledge K R Under these conditions, the correct answer or optimal answer generated by the model; Through the above definitions, the model can distinguish the different roles of knowledge fragments in the knowledge document KD; Step A2: Causal modeling and path analysis; Based on the causal variables defined in step A1, analyze the causal relationships and impact paths between different variables, including: (1) C→KD→K R :According to the dialogue context C, select relevant knowledge K in the knowledge document KD R , which is used to guide the model's subsequent response generation; the relevant knowledge acquisition process is as follows: K R =f R (C,KD)+∈ R Among them, f R (·) is the function of model selection knowledge, ∈ R is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness; (2)KD→K I : Not affected by the context explicitly, but not related to knowledge K I Exists in the knowledge document KD as a potential interference factor; (3) C, K R →R:Related knowledge K R Together with the dialogue context C, the response R of the generated model is determined, reflecting a direct causal path. The response generation process is as follows: R=f resp (C,K R ,K I )+∈ resp Among them, f resp (·) is the model response generating function, ∈ resp is an exogenous random error term, representing factors not explicitly modeled or model-intrinsic randomness; (4)K I →R:Irrelevant knowledge K I It may affect the response R generation of the model in a potential interference way. This path is the interference path. The above causal path clarifies the goal that needs to be focused on during model fine-tuning, namely maximizing the context C and related knowledge K R Positive causal influence on the true response R while minimizing irrelevant knowledge K I negative interference effects.

3. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 1 is characterized in that: The step B specifically comprises the following steps: Step B1: Definition and calculation of knowledge benefit score; The knowledge gain score (KGS) is defined to quantify the contribution of knowledge to the generation of high-quality responses by the model. The knowledge gain comprehensively considers the following three dimensions: (1) Semantic similarity S sem ; Calculate the semantic proximity between knowledge and true responses, use vector representation to obtain the embedding vectors of the two, and then evaluate them through cosine similarity to ensure the semantic accuracy of the model in knowledge selection; Among them, v K and v R They represent the semantic vector representation of knowledge K and true response R respectively, and |||| represents the Euclidean norm of the vector; (2) Word coverage S cov ; Calculate word-level overlap between the knowledge fragment and the true response, specifically defined as the ratio of the number of words shared between the two to the number of words in the true response. This dimension complements the semantic similarity metric and reflects the direct connection between the knowledge and the true response from the perspective of word overlap. Among them, || represents the cardinality of the set, that is, the number of elements in the set; W K With W R Represents the vocabulary set of knowledge fragment K and true response R, |W K ∩W R | indicates the number of words in common between the two, |W R | represents the number of words in the true response, which is used for normalization so that the coverage is in the interval [0, 1]; (3) Conditional interaction information S cxmi ; Evaluate the degree of conditional dependency between knowledge K, conversation context C, and true response R to ensure that the knowledge introduced meets the needs of the conversation context and makes a real contribution to generating true responses, rather than being redundant or irrelevant information. Where R is the true response, C is the conversation context, P(R|C) represents the probability of the model generating a true response based only on the conversation context C, and P(R|C,K) represents the probability of generating a true response R based on both the conversation context C and the knowledge K. The final knowledge gain score (KGS) is a linear weighted combination of the above three dimensional indicators: KGS(K,C,R)=α·S sem (K,R)+β·S cov (K,R)+γ·S cxmi (K,C,R) Among them, α, β, and γ are weight coefficients, satisfying α+β+γ=1; Step B2: Construction of causal enhancement dataset (CED); We collect, process, and fuse multiple public datasets involving retrieval-enhanced question answering tasks as the original dataset D to construct a causal-enhanced dataset (CED). The general structure of a single sample in the causal-enhanced dataset includes: (1) Complete knowledge document KD: external knowledge information outside the dialogue context, used to assist the model in generating responses; (2) Complete conversation context C: includes the current conversation turn and all historical conversation turns, namely: in, Indicates the text concatenation symbol, t indicates the current dialogue turn number, u i 、r i are the user utterance and system response in round i respectively; (3) True response R: The accurate answer text that the model should generate in the current round t; for true responses whose answers come from knowledge documents, the actual text of the span extracted from the document is used as the true response; The causal enhancement dataset is processed and integrated, including the following steps: Step B21: Conversation Segmentation: Split a single multi-round dialogue sample in the original dataset D into multiple independent training samples. For a single sample (u1, r1, u2, r2, u3, r3) containing three rounds of dialogue, split it into three independent samples: (u1), (u1, r1, u2), (u1, r1, u2, r2, u3), and their corresponding true responses are r1, r2, and r3 respectively. Step B22: Data enhancement and knowledge annotation: For each segmented training sample, in addition to the complete knowledge document, the following knowledge information is additionally annotated: 1) Related knowledge K R : If the knowledge document contains a true response, the document fragment containing the true response is directly used as the relevant knowledge; otherwise, the knowledge fragment with the highest knowledge benefit score is selected as the relevant knowledge; 2) Irrelevant knowledge K I : Select knowledge fragments from knowledge documents that are mutually exclusive with related knowledge fragments and have low knowledge benefits; 3) Replaced related knowledge fragment K RR : Another knowledge fragment that is mutually exclusive with the above two knowledge fragments and has high knowledge benefits; 4) Replaced irrelevant knowledge fragment K RI : Additional knowledge fragments that are mutually exclusive with the above three knowledge fragments and have low knowledge benefits; Therefore, a single training sample in the final constructed causal enhancement dataset CED is recorded as: (C,K,R,K R ,K I ,K RR ,K RI ) Step B3: The division and purpose of the data set are clearly defined; The constructed causal enhancement dataset is divided into training set and validation set by random sampling.

4. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 1, characterized in that: The step C specifically comprises the following steps: Step C1: intervention reasoning based on the do operator; For the samples in the causal enhancement dataset, the input knowledge fragments are used as intervention variables. By combining, splicing, and replacing the knowledge document content, multiple intervention scenarios are constructed to measure the impact of changes in knowledge variables on the model generation response. Specifically, the knowledge benefit scores of the model response under different input conditions are calculated respectively, so as to obtain the relevant knowledge benefits under the intervention conditions. and irrelevant knowledge gains Step C2: Reasoning based on counterfactual reasoning; Based on the variables defined in the causal enhancement dataset, different counterfactual input scenarios are constructed by retaining or deleting the knowledge document content. The knowledge gain scores of the model under different knowledge provision scenarios are calculated to analyze the changes in the model response generation quality, thereby obtaining the relevant knowledge gains under the corresponding conditions. and irrelevant knowledge gains Step C3: Comprehensive evaluation of knowledge benefits; Relevant knowledge gains calculated based on steps C1 and C2 and irrelevant knowledge gains The following comprehensive indicators are further defined and calculated: Comprehensive knowledge gain (KGS) R ; Comprehensive irrelevant knowledge gain (KGS I ; Relevant knowledge benefit gap ΔKGS R ; Irrelevant knowledge benefit gap ΔKGS I ; Step C4: auxiliary task construction; Based on the comprehensive relevant knowledge gain and irrelevant knowledge gain results calculated in step C3, targeted auxiliary tasks are constructed. The auxiliary tasks are designed to improve the model's ability to effectively utilize relevant knowledge during training, while suppressing the model's interference effect on irrelevant knowledge, thereby improving the robustness and quality of response generation. Step C5: multi-task fine-tuning optimization strategy; During fine-tuning, the tasks are divided into: Main task: the difference between the predicted response and the true response, calculating the main task loss; Auxiliary task: Calculate the auxiliary task loss based on the comprehensive relevant knowledge gain, comprehensive irrelevant knowledge gain, relevant knowledge gain gap, and irrelevant knowledge gain gap; During fine-tuning, the back-propagation algorithm is used to calculate the gradient of network parameters, and the parameters are updated by the stochastic gradient descent algorithm; Step C6: fine-tune the stopping criterion; When the value of the loss function during fine-tuning is less than a preset threshold or the maximum number of iterations is reached, fine-tuning training of the large language model is stopped.

5. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 4 is characterized in that: The step C1 specifically includes the following steps: Step C11: Design the intervention scenarios of the do operator; specifically, design the following three intervention scenarios: Scenario 1 (no intervention): No modification is made to the original knowledge document. This serves as a baseline control group to observe the impact of the original knowledge document. Scenario ② (related knowledge intervention): Only the content of the related knowledge fragments is intervened, and the impact of the changes on the model-generated response R is evaluated by modifying the related knowledge fragments; Scenario ③ (irrelevant knowledge intervention): Only the content of irrelevant knowledge fragments is intervened, and the interference effect of irrelevant knowledge on the model-generated response R is evaluated by modifying irrelevant knowledge fragments; Step C12: knowledge intervention; In fine-tuning training, relevant knowledge fragments are replaced with spare relevant knowledge fragments, and irrelevant knowledge fragments are replaced with spare irrelevant knowledge fragments, and then the knowledge benefit scores of the corresponding scenarios are calculated; Step C13: calculation of knowledge utility of do operator; DO operator related knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ② (related knowledge intervention): DO operator irrelevant knowledge utility: Calculate the difference in response knowledge utility between scenario ① (no intervention) and scenario ③ (irrelevant knowledge intervention): in, represents text concatenation, KD represents complete knowledge document, K R Represents relevant knowledge, K I represents irrelevant knowledge, and do(·) represents intervention in knowledge.

6. The causal reasoning-based retrieval enhancement generation model fine-tuning method according to claim 4 is characterized in that: The step C2 specifically includes the following steps: Step C21: Counterfactual Scenario Design Design the following three typical counterfactual scenarios: Scenario 1 (no knowledge provided): No knowledge input, used to evaluate the response quality of the model in the absence of external knowledge, and to observe whether false or invalid responses are generated; Scenario 2 (providing only relevant knowledge): Only clear relevant knowledge is provided to analyze the response quality of the model under non-interference conditions; Scenario 3 (providing only irrelevant knowledge): Provide only irrelevant knowledge to assess the model's sensitivity and interference to irrelevant knowledge and determine whether it produces erroneous or misleading responses. Step C22: Explore the following questions: ① What kind of response will the model generate if no knowledge input is provided? ② What is the quality of the model’s response if only relevant knowledge documents are referenced? ③ What is the quality of the model’s response if it only refers to irrelevant knowledge documents? Step C23: Counterfactual Knowledge Utility Calculation Counterfactual knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ② (providing relevant knowledge): Counterfactual irrelevant knowledge utility: Calculate the difference in response utility between scenario ① (no knowledge input) and scenario ③ (providing irrelevant knowledge): Among them, null means no knowledge fragment input.

7. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 4, characterized in that: The step C3 specifically includes the following steps: Step C31: Calculation of comprehensive relevant knowledge benefits Based on the relevant knowledge gains calculated by do operator intervention and counterfactual reasoning, the comprehensive relevant knowledge gain KGS is defined R is the weighted average of the two: Among them, λ∈[0,1] is an adjustable weight parameter used to control the contribution ratio of the do operator and counterfactual reasoning; Step C32: Calculation of comprehensive irrelevant knowledge benefits Based on the irrelevant knowledge benefits calculated by do operator intervention and counterfactual reasoning, the comprehensive irrelevant knowledge benefit KGS is defined I is the weighted average of the two: Among them, μ∈[0,1] is an adjustable weight parameter, which is used to weigh the role of intervention reasoning and counterfactual reasoning in the evaluation of irrelevant knowledge utility; Step C33: Calculation of relevant knowledge benefit gap Based on the relevant knowledge gains obtained from the do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated to measure the consistency of the two causal reasoning methods: Step C34: Calculation of irrelevant knowledge benefit gap Based on the irrelevant knowledge benefits obtained by do operator intervention and counterfactual reasoning, the benefit gap between the two is calculated:

8. The causal reasoning-based retrieval enhancement generation model fine-tuning method according to claim 4 is characterized in that: The step C4 specifically includes the following steps: Step C41: Maximize the comprehensive relevant knowledge gain KGS R ; Step C42: Minimize the comprehensive irrelevant knowledge gain KGS I ; Step C43: Minimize the knowledge gain gap ΔKGS between the do operator and counterfactual reasoning R ; Step C44: Minimize the irrelevant knowledge benefit gap ΔKGS between the do operator and counterfactual reasoning I .

9. The causal reasoning-based retrieval enhancement generative model fine-tuning method according to claim 4, characterized in that: The step C5 specifically includes the following steps: Step C51: Construction of contrast loss Construct a contrast loss based on the comprehensive relevant knowledge gain and the comprehensive irrelevant knowledge gain to significantly improve the discrimination between relevant knowledge and irrelevant knowledge: Among them, m is the interval parameter of contrast loss, which ensures that the benefits of relevant knowledge are significantly higher than those of irrelevant knowledge; Step C52: Construction of consistency loss The consistency loss is constructed based on the relevant knowledge benefit gap and the irrelevant knowledge benefit gap to ensure the consistency of the model's knowledge utilization under intervention and counterfactual conditions: Step C53: Construction of cross entropy loss The original loss function of the model when predicting the response is defined as the standard cross entropy loss, which is used to ensure that the generated response matches the true answer: Among them, y i represents the true probability distribution of the i-th token of the true response, and its value range is i=1,2,…,T, where T is the length of the true response sequence, represents the probability distribution predicted by the model; Step C54: Comprehensive design of final loss The above loss terms are combined and weighted to form the final loss function, which is used for overall optimization: Among them, ω1 and ω2 are weight coefficients, which are determined by adjusting the validation set to optimize the overall performance and generalization ability of the model in the response generation task.

10. A retrieval-enhanced generative model fine-tuning system based on causal reasoning, characterized by: The method comprises a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method according to any one of claims 1 to 9 can be implemented.

Citation Information

Cited By

  • Two-stage document filtering and robust fine tuning method based on graph attention network

    CN121997918A

  • A Two-Stage Document Filtering and Robust Fine-Tuning Method Based on Graph Attention Networks

    CN121997918B