Fine-tuning and joint task-based large language model explanation generation method and system

Abstract

The application discloses a fine-tuning and joint task-based large language model explanation generation method and system, relates to the field of natural language processing, and comprises the following steps: preprocessing text data and designing a prompt template suitable for zero-shot and few-shot scenarios; using an improved mmBERT encoder to perform semantic representation learning on samples to obtain text semantic vectors suitable for statement detection and explanation generation tasks; using QLoRA parameter efficient fine-tuning technology to fine-tune a large model for the statement detection task and the explanation generation task respectively; further training the two tasks jointly, optimizing the joint model through QLoRA, and finally obtaining an end-to-end explanation generation model capable of judging the authenticity of a statement and generating corresponding explanations, which is used for automatically outputting authenticity judgment and explainability of new text data. The application realizes explainability of text data by using parameter efficient fine-tuning technology, a joint task training mode and a unified prompt template to guide a large language model.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, specifically to a method and system for generating interpretations of large language models based on fine-tuning and joint tasks. Background Technology

[0002] Explanation generation can be realized as an important component to improve the credibility of online rumor verification tasks. In recent years, it has made some progress. Existing explanation generation methods mainly include: extractive explanation generation method, question-answering explanation generation method, and natural language explanation (NLE) generation method.

[0003] In the current era of rapid development in artificial intelligence technology, large language models, with their powerful language understanding and generation capabilities, have demonstrated enormous application potential in the field of natural language processing and have been widely applied in various scenarios such as text generation, question answering systems, and translation. However, large language models are often regarded as "black boxes," as their complex internal decision-making processes are difficult for humans to understand. This makes it difficult for users to trust the model's output, especially in critical fields such as medicine, finance, and law, where interpretability is extremely important. This lack of transparency severely limits the practical application of large language models. Explanation-based generation methods for large language models have emerged in this context. These methods are significant because they can provide reasonable, clear, and understandable explanations for the output of large language models, allowing users to understand the basis and reasoning process behind the model's conclusions. This enhances user trust in the model, promotes the application of large language models in more high-risk and high-requirement fields, and facilitates the deep integration of artificial intelligence with various industries.

[0004] Most methods treat statement detection and interpretation generation tasks separately, lacking effective integration of joint task datasets. This approach results in weak inter-task correlation; the model often processes statement detection and interpretation generation independently, failing to share valuable information between the two tasks, thus limiting the model's overall performance. For example, although statement authenticity judgment and interpretation generation tasks are inherently related, in existing methods, their training data and optimization objectives are usually separate, making it difficult to fully exploit their synergistic effects during training.

[0005] Existing methods for training large language models typically rely on full parameter tuning, which requires substantial computational resources, resulting in high time and computational costs during training. While some efficient fine-tuning methods have been proposed, most still have room for optimization, especially in terms of training efficiency under limited resources. Furthermore, traditional fine-tuning methods usually require large amounts of labeled data and long training times, posing challenges for model training in practical applications. Against this backdrop, QLoRA (Quantized Low-Rank Adaptation) emerges as an efficient fine-tuning strategy that can effectively reduce computational costs and improve training efficiency even with limited resources.

[0006] While the joint task model can theoretically handle both statement authenticity assessment and interpretation generation simultaneously, existing methods often fall short in terms of information exchange and collaborative optimization between tasks. Single-task optimization strategies frequently lead to models struggling to balance the demands of statement detection and interpretation generation, particularly when weighing multiple task objectives. For example, some methods overemphasize the accuracy of statement detection while neglecting the logical coherence of interpretation generation, resulting in insufficient background information to support the generated interpretations, thus affecting the reasonableness and comprehensibility of the final results. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a method and system for generating interpretations of large language models based on fine-tuning and joint tasks. By combining improved mmBERT text encoding, multi-type prompt template design, QLoRA efficient parameter fine-tuning, and a joint training mechanism for declaration detection and interpretation generation, it achieves accurate judgment of the authenticity of text declarations and generates interpretable reasons in zero-sample and few-sample scenarios.

[0008] The specific plan is as follows:

[0009] On the one hand, large language model interpretation generation methods based on fine-tuning and joint tasks include:

[0010] S1. Preprocess the collected text data to obtain preprocessed data. Extract text statements and corresponding text evidence from the preprocessed data and match them to form structured input and output samples.

[0011] S2 guides large language models to process statements and evidence based on structured input and output samples and specified format prompts, generating prompt templates suitable for zero-shot and few-shot scenarios.

[0012] S3, based on prompt templates and structured input and output samples, uses improved mmBERT text encoding to encode the structured input and output samples, obtaining text semantic representations suitable for declaration detection tasks and text semantic representations suitable for interpretation generation tasks;

[0013] S4. The text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task are input into the large language model through the QLoRA parameter fine-tuning technique to fine-tune the declaration detection task and the interpretation generation task.

[0014] S5. Jointly train the fine-tuned statement detection task and explanation generation task, and fine-tune the joint task using QLoRA to obtain the final explanation generation model.

[0015] S6 uses the final interpretation and generation model to interpret and generate interpretations of the newly collected text data, thereby obtaining a true interpretation of the text.

[0016] Furthermore, the prompt template applicable to zero-sample and few-sample scenarios includes a single-task prompt word template for whether evidence supports the claim in a zero-sample prompt scenario. Single-task prompt templates for scenarios with zero-sample prompts, and for situations where there is insufficient supporting evidence. How can the generation of evidence in zero-sample scenarios support joint task prompt templates for statements and interpretations? How can we support joint task prompt word templates for statements and interpretations when directly generating evidence in scenarios with few samples? They are defined as follows: ;

[0017] in, A single-task prompt template indicating whether evidence supports a claim in a zero-sample scenario; This indicates a declaration; Present evidence; This indicates the instructions input to the large model in this scenario; Indicates text concatenation;

[0018] ;

[0019] in, A single-task prompt template indicating whether relevant supporting evidence is available in zero-sample prompt scenarios and when there is insufficient information to support the evidence. Indicate how the evidence supports the statement. This indicates the command to input a large model in this scenario.

[0020] ;

[0021] in, How can the direct generation of evidence in zero-sample scenarios support joint task prompt word templates for statements and interpretations? This indicates the instructions input to the large model in this scenario;

[0022] ;

[0023] in, This section describes a joint task prompt template for how directly generated evidence can support declarations and interpretations in scenarios with few samples. This indicates that a small number of samples have been provided. This indicates the instructions input to the large model in this scenario.

[0024] Furthermore, in S3, the improved mmBERT text encoding is defined as follows:

[0025] ;

[0026] in, Indicates text input; Indicates word embedding; Indicates positional encoding; This represents attention encoding; Indicates the number of layers in the Transformer; in this scenario, it will... The number of layers was increased to 8 to improve efficiency; This means compressing the matrix output by the encoder into a fixed-dimensional semantic vector; This is represented as the final generated text semantic vector; Cosine similarity is used to measure semantic consistency. This indicates minimizing the number of tokens; This represents the balance coefficient, prioritizing ensuring that the semantics of the low-resource language are not lost; This represents the token sequence after preprocessing text fragmentation; This represents the preprocessed text; A temporary semantic vector representing the preprocessed text; A temporary semantic vector representing a sequence of tokens.

[0027] Furthermore, the calculation formula for fine-tuning the declaration detection task is as follows:

[0028] ;

[0029] ;

[0030] ;

[0031] in, For the Sigmoid function; This indicates that QLoRA is used for fine-tuning of the large model; This indicates that parameter adjustments were made under the declaration detection task, and the result is... and ; and These are the trainable parameters of the model; This is the loss function, used to measure the reasonableness of the explanations generated by the model at each step during model fine-tuning, further improving the effectiveness of model fine-tuning; function Used to determine a declaration Whether evidence has been obtained The support, among which and By function generate; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement.

[0032] Furthermore, in S4, the calculation formula for fine-tuning the interpretation generation task is as follows:

[0033] ;

[0034] ;

[0035] ;

[0036] in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This indicates the task to be interpreted and generated; This indicates that the model output is being decoded; The loss function; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; The training set represents the explanation; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; This represents the prediction set used for explanation.

[0037] Furthermore, in S5, the fine-tuned declaration detection task and the fine-tuned interpretation generation task are jointly trained, and the calculation formula is as follows:

[0038] ;

[0039] ;

[0040]

[0041] in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This represents the generated declaration and its interpretation, where and For function generate; This is the loss function.

[0042] On the other hand, large language model interpretation and generation systems based on fine-tuning and joint tasks include:

[0043] The matching module preprocesses the collected text data to obtain preprocessed data, extracts text statements and corresponding text evidence from the preprocessed data and matches them to form structured input and output samples.

[0044] The template generation module guides the large language model to process statements and evidence based on structured input and output samples and specified format prompts, generating prompt templates suitable for zero-shot and few-shot scenarios.

[0045] The semantic representation generation module, based on prompt templates and structured input and output samples, performs encoding processing on the structured input and output samples through improved mmBERT text encoding to obtain text semantic representations suitable for declaration detection tasks and text semantic representations suitable for interpretation generation tasks.

[0046] The fine-tuning module inputs the text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task into the large language model through the QLoRA parameter fine-tuning technique, and fine-tunes the declaration detection task and the interpretation generation task.

[0047] The joint training module performs joint training on the fine-tuned statement detection task and explanation generation task, and fine-tunes the joint task using QLoRA to obtain the final explanation generation model.

[0048] The interpretation and generation module interprets and generates interpretations of newly collected text data using the final interpretation and generation model, thereby obtaining a realistic interpretation of the text.

[0049] The present invention adopts the above technical solution and has the following beneficial effects:

[0050] (1) This invention designs a structured prompt template suitable for zero-sample and few-sample scenarios, combined with improved mmBERT text encoding, enabling large language models to effectively understand the relationship between statements and evidence when labeled data is limited, significantly enhancing the generalization ability and deployment flexibility of the model in practical applications.

[0051] (2) The present invention uses the QLoRA parameter high-efficiency fine-tuning technology to finely tune the declaration detection task and the interpretation generation task respectively, and further introduces a joint training mechanism so that the two tasks can promote each other on the basis of sharing semantic representation, thereby improving the accuracy of authenticity judgment and the relevance and consistency of interpretation content.

[0052] (3) This invention improves the structure of the mmBERT encoder (such as compressing the number of Transformer layers) and combines it with the QLoRA efficient fine-tuning strategy to reduce computational overhead while ensuring performance; at the same time, it generates end-to-end natural language explanations containing the authenticity judgment and its basis, which enhances the transparency and credibility of the model decision-making process. Attached Figure Description

[0053] Figure 1 This is a flowchart of a large language model interpretation generation method based on fine-tuning and joint tasks, as described in an embodiment of the present invention.

[0054] Figure 2 This is a diagram of a large language model interpretation and generation system based on fine-tuning and joint tasks, as described in an embodiment of the present invention. Detailed Implementation

[0055] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0056] like Figure 1 As shown, the present invention provides a method for generating large language model interpretations based on fine-tuning and joint tasks, comprising:

[0057] S1. Preprocess the collected text data to obtain preprocessed data. Extract text statements and corresponding text evidence from the preprocessed data and match them to form structured input and output samples.

[0058] Specifically, in this embodiment, data preprocessing is performed to remove irrelevant information, conduct necessary text processing, and design a unified prompt template. This step aims to improve input quality and standardize model generation logic through data preprocessing, thereby ensuring the accuracy and consistency of interpretation results and reducing engineering implementation costs.

[0059] S2 guides large language models to process statements and evidence based on structured input and output samples and specific formatted prompts, generating prompt templates suitable for zero-sample and few-sample scenarios.

[0060] Specifically, the prompt template applicable to zero-sample and few-sample scenarios is defined as follows: ;

[0061] in, A single-task prompt template for whether evidence supports a claim in a zero-sample scenario; This indicates a declaration; Present evidence; This indicates the instructions input to the large model in this scenario; Indicates text concatenation;

[0062] ;

[0063] in, This indicates a single-task prompt word template that generates an explanation when the statement is supported by relevant evidence, is not supported by relevant evidence, or does not have enough information to support the evidence in a zero-sample prompt scenario. Indicate how the evidence supports the statement. This indicates the command to input a large model in this scenario.

[0064] ;

[0065] in, This section explains how directly generated evidence can support claims and interpret joint task prompt word templates in zero-sample prompt scenarios. This indicates the instructions input to the large model in this scenario;

[0066] ;

[0067] in This indicates how directly generating evidence can support claims and interpretations in scenarios with few samples; This indicates that a small number of samples have been provided. This indicates the instructions input to the large model in this scenario.

[0068] Specifically, in a zero-hint scenario, each task uses the following hints.

[0069] First, there's the statement detection task prompt template. Input a statement and evidence, and the model determines the relevance of the statement and evidence, using "Supported," "Refuted," and "NEI," representing that the statement is supported by relevant evidence, not supported by relevant evidence, and lacks sufficient information to support the evidence, respectively.

[0070] Verification Task

[0071] Claim: X

[0072] Evidence: Y

[0073] Based only on the context, categorize

[0074] the claim as:

[0075] - Supported (supported by context)

[0076] - Refuted (refuted by context)

[0077] - NEI (not enough info).

[0078] Secondly, for the prompt template for explaining the generation task, the input includes a statement, evidence, and a judgment result, allowing the model to explain why the judgment result is given based on the statement and evidence. ,Right now:

[0079] Explanation generation

[0080] Claim: X

[0081] Evidence: Y

[0082] The claim veracity: Z

[0083] Using only the context provided, explain why the claim veracity isZ.

[0084] Finally, the goal of the joint task is to allow the model to determine whether a claim is supported by evidence and then provide a relevant explanation. This differs from the previous step in that a standard label for the truthfulness of a claim is not provided to the model.

[0085] Joint Task

[0086] Claim: X

[0087] Evidence: Y

[0088] Based only on the context, categorize

[0089] the claim as:

[0090] - Supported (supported by context)

[0091] - Refuted (refuted by context)

[0092] - NEI (not enough information)

[0093] And explain your reasoning. Provide the response in JSON format with the following keys: veracity ,explanation.

[0094] In few-shot cueing scenarios, this chapter provides a reference example for each task, derived from the dataset itself. Taking the interpretation task as an example, an example is provided to help the large model understand the specification, namely:

[0095] Claim: X

[0096] Evidence: Y

[0097] The claim veracity: Z

[0098] Instruction: Using only the provided context, explain why the claim'sveracity is Z.

[0099] Hint: Please complete the task based on the following example:

[0100] Claim: [Claim text]

[0101] Evidence: [Evidence text]

[0102] The claim veracity: [Veracity label]

[0103] Explanation: [Explanation text].

[0104] S3, based on prompt templates and structured input and output samples, uses improved mmBERT text encoding to encode the structured input and output samples, obtaining text semantic representations suitable for declaration detection tasks and text semantic representations suitable for interpretation generation tasks.

[0105] Specifically, in S3, the improved mmBERT text encoding is defined as follows:

[0106] ;

[0107] in Indicates text input; Indicates word embedding; Indicates positional encoding; This represents attention encoding; Indicates the number of layers in the Transformer; in this scenario, it will... The number of layers was increased to 8 to improve efficiency; This means compressing the matrix output by the encoder into a fixed-dimensional semantic vector; This is represented as the final generated text semantic vector; Cosine similarity is used to measure semantic consistency. This indicates minimizing the number of tokens; This represents the balance coefficient, prioritizing ensuring that the semantics of the low-resource language are not lost; This represents the token sequence after preprocessing text fragmentation; This represents the preprocessed text; A temporary semantic vector representing the preprocessed text; A temporary semantic vector representing a sequence of tokens.

[0108] S4 inputs the text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task into the large language model through the QLoRA parameter efficient fine-tuning technique, and fine-tunes the declaration detection task and the interpretation generation task.

[0109] Specifically, the calculation formula for fine-tuning the declaration detection task is as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] in, For the Sigmoid function; This indicates that QLoRA is used for fine-tuning of the large model; This indicates that parameter adjustments were made under the declaration detection task, and the result is... and ; and These are the trainable parameters of the model; This is the loss function, used to measure the reasonableness of the explanations generated by the model at each step during model fine-tuning, further improving the effectiveness of model fine-tuning; function Used to determine a declaration Whether evidence has been obtained The support, among which and By function generate; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement.

[0114] The calculation formula for fine-tuning the interpretation generation task is as follows:

[0115] ;

[0116] ;

[0117] ;

[0118] in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This indicates the task to be interpreted and generated; This indicates that the model output is being decoded; This is the loss function used to measure the rationality of the explanations generated by the model at each step during the model fine-tuning process, thereby further improving the effectiveness of model fine-tuning. express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; The training set represents the explanation; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; This represents the prediction set used for explanation.

[0119] Specifically, the parameter-efficient fine-tuning method mentioned in this embodiment aims to reduce the number of trainable parameters and has become the standard paradigm for fine-tuning LLMs. Specifically, the QLoRA method of PEFT is chosen for fine-tuning.

[0120] The large model is primarily trained using the Qlora fine-tuning paradigm of quantization and LoRa tuning. When training the model using Qlora, the 7B model consumes approximately 17GB of GPU memory.

[0121] The goal of the statement detection task is to determine a given statement Can it be supported by evidence, that is, given a statement? and corresponding evidence The goal is to derive a function To determine the statement Whether evidence has been obtained Support.

[0122] Input indicates: Declaration With evidence .

[0123] Objective: Calculate the function , where 0 indicates that the statement is not supported by evidence, 1 indicates that the statement is supported by evidence, and 2 indicates that the evidence is insufficient.

[0124] Model architecture: given a declaration and evidence The two are encoded into vector representations through a pre-trained model, that is:

[0125]

[0126]

[0127] ;

[0128] in, For the Sigmoid function; and The trainable parameters of the model are obtained using the cross-entropy loss function. Perform model training.

[0129] QLoRA Fine-tuning: To improve model performance, QLoRA technology is used to fine-tune large language models. Through quantization and low-rank adaptation techniques, the computational cost and memory requirements of model fine-tuning can be significantly reduced, making the fine-tuning process more efficient. Specifically, by adjusting the model's low-rank parameters, the complex interaction between claims and evidence is captured, thereby improving the accuracy of claim detection.

[0130] The interpretation and generation task aims to provide a given statement Provide an explanation as to why the veracity of the statement was determined to be... This requires deriving a function. To generate the corresponding explanation.

[0131] Input: Declaration ,evidence And the declaration of test results .

[0132] Objective: Generate an explanation Explain why the statement was judged as .

[0133] Model architecture: Using a large language model, given a declaration ,evidence and the judgment result Generate an explanation, that is:

[0134] ;

[0135]

[0136] in, This is the loss function.

[0137] QLoRA Fine-tuning: Similarly, QLoRA fine-tuning is used in the explanation generation task. Through low-rank adaptation, the generative model is able to more accurately generate explanations that conform to the truth of the claims. The fine-tuned model can better capture the semantic relationship between the claims and the evidence, and generate persuasive explanations.

[0138] S5, jointly trains the fine-tuned statement detection task and the fine-tuned explanation generation task, and fine-tunes the joint task using QLoRA to obtain the final explanation generation model.

[0139] Specifically, the fine-tuned statement detection task and the fine-tuned interpretation generation task are jointly trained, and the calculation formula is as follows:

[0140] ;

[0141] ;

[0142] ;

[0143] in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This represents the generated declaration and its interpretation, where and For function generate; This is the loss function.

[0144] In a joint task setting, both declaration detection and interpretation generation must be handled simultaneously. The goal of the joint task is to address a given declaration... and evidence Derive a function This function can simultaneously output the veracity of a statement and its corresponding explanation.

[0145] Input: Declaration and evidence .

[0146] Objective: Calculate the function It also outputs a judgment and explanation of the authenticity of the statement.

[0147] Model Architecture: A multi-task learning framework is employed, combining classification and generation tasks. Given a declaration... and evidence The representation of the claims and evidence is obtained through a shared encoder, namely:

[0148]

[0149] Multi-task learning and QLoRA fine-tuning: In the joint task, a shared multi-task model is introduced to learn both statement detection and interpretation generation. QLoRA is used to fine-tune the joint task model to improve overall model performance. The model output consists of two parts: statement detection results. and explanation ,Right now:

[0150] ;

[0151] ;

[0152] in, For the output of the model, This is the loss function.

[0153] Compared with existing extractive interpretation generation methods, question-answering interpretation generation methods, and natural language interpretation generation methods, the fine-tuned interpretation generation under a joint task effectively solves the black-box problem and weak interpretability of previous methods in interpretation generation. Furthermore, traditional methods train declaration detection and interpretation generation tasks separately, resulting in a lack of logical support for interpretation generation. This method combines the training of these two tasks, effectively solving this problem. Experiments demonstrate a significant improvement in interpretation generation tasks.

[0154] S6 uses the final interpretation and generation model to interpret and generate interpretations of the newly collected text data, thereby obtaining a true interpretation of the text.

[0155] Specifically, this invention uses the Ru22Fact dataset, an interpretable rumor verification dataset. The main features of this dataset include:

[0156] (1) Real-world claims. The dataset contains 16,033 samples, each of which includes a real-world claim, optimized evidence, and reference interpretation.

[0157] (2) Explainability. In addition to the rumor-checking labels, human-understandable explanatory tasks are provided to illustrate the adjudication process and prove the correctness of the labels.

[0158] To better compare model performance, this invention selected three representative methods as comparison methods, as follows:

[0159] (1) FCS: Uses large language models (LLMs) and combines evidence gathered from the network to generate more accurate explanations.

[0160] (2) EXPLAINERFC: Uses a pre-trained BERT summarization model on news articles and summary datasets to generate explanations.

[0161] (3) MKL: A multi-task learning model that can perform fact verification and text summarization simultaneously. During training, the model is fully optimized to improve the classification and summarization tasks, thereby generating explanations of the decision-making process.

[0162] In this embodiment, the evaluation criteria used are as follows: the accuracy (ACC) and F1 score are used as evaluation indicators to assess the accuracy of predictions.

[0163] ACC: Represents the proportion of correct results among all model outcomes. The calculation formula is as follows:

[0164]

[0165] in, This represents the number of correctly classified positive samples. This represents the number of correctly classified negative samples. This represents the number of positive samples that were misclassified. This represents the number of negative samples that were misclassified.

[0166] F1 score: This metric measures the quality of the model's predictions for the positive class. The calculation formula is as follows:

[0167]

[0168] For the evaluation of explanations, this chapter employs both automatic and manual evaluation methods. It is important to note that standard explanations are typically more abstract than those generated by large language models. The introduction of manual evaluation aims to overcome the difficulties that automatic evaluation may encounter when handling abstract explanations, thereby providing more comprehensive evaluation results. In automatic evaluation, the correlation between manual and automatic evaluation metrics is often low when evaluating explanation generation. ROUGE, based on exact matching, presents problems when comparing abstract explanations, as abstract explanations often involve more flexible linguistic expressions and content summaries that may not be adequately captured by exact matching evaluation methods. Nevertheless, following recent related work, this chapter still uses ROUGE to compare generated explanations with standard explanations, expressed as:

[0169]

[0170] ROUGE-L (Longest Common Subsequence) is primarily used to evaluate the longest common subsequence (LCS) between generated and reference text. It measures the quality of the generated text by calculating the length of the LCS. Human evaluation focuses on the following seven criteria: linguistic consistency. This refers to whether the generated explanation uses its own language, rather than directly repeating the original statement. "Yes" indicates that the explanation is linguistically innovative and does not directly repeat the original statement; "No" indicates that the explanation directly repeats the language of the original statement.

[0171] The information reuse section addresses whether unnecessary repetitive information is included, affecting clarity of understanding and the truthfulness category. Based on the explanation, it assesses whether the statement can be categorized (true, false, unverified); evaluating the annotator's ability to accurately classify the statement and ensure the generated explanation helps in accurately determining its truthfulness. Logical consistency is assessed, checking for logical contradictions within the explanation and ensuring internal logical consistency. A 0-4 scale is used, where 0 represents complete inconsistency and 4 represents complete consistency. Higher scores indicate better logical consistency. Contextual consistency is assessed, ensuring the generated explanation matches the surrounding context and avoiding contradictions. A 0-4 scale is used, where 0 represents complete inconsistency and 4 represents complete consistency. Higher scores indicate a better match between the explanation and the context. Relevant information is assessed, marking it as (yes / no). "Yes" indicates the explanation contains additional information; "No" indicates the explanation is based solely on the statement and context, without additional information. Key information is also assessed. Whether the explanation is complete and contains all the important information necessary to prove the truthfulness of the statement is assessed using a three-part system: "1" indicates serious missing information; "2" indicates that the information is basically complete but partially missing; and "3" indicates that the information is complete and there are no major missing items.

[0172] In this embodiment, the comparative experimental results show that: (1) The explanation generation method based on the large language model achieves the best performance on both ROUGE-1 and ROUGE-L metrics, improving by 1.6 and 0.3 percentage points respectively compared to the best baseline. (2) Compared with EXPLAINERFC, the explanation generation method based on the large language model improves the ROUGE-1 and ROUGE-L scores by 8.2 and 7.5 respectively, indicating that by integrating the joint task dataset of statement detection and explanation generation, and optimizing the fine-tuning strategy, the quality of the explanation generated by the model can be effectively improved. (3) The improvement of the explanation generation method based on the large language model on MKL and FCS, especially the Qlora efficient fine-tuning strategy used in the model fine-tuning process, further enhances the performance of the model, ensuring that the generated explanation not only has high language fluency, but also better conforms to the actual content of the statement. (4) The explanation generation method based on the large language model improves the performance of explanation generation. In particular, by jointly training the statement detection and explanation generation tasks, the model can not only more accurately identify the association between the statement and the evidence, but also generate more coherent explanations that conform to the actual content. This section compares the performance differences between different large language models, as well as the performance differences between joint tasks and single tasks. Representative large language models are selected for comparison, specifically: Flan-T5 (7B). This model is a pre-trained language model proposed by Google, which transforms all natural language problems into text-to-text forms and solves them using a unified model. T5 is based on the Transformer architecture and has a self-attention mechanism, enabling it to understand the global dependencies of the input sequence. Flan-T5 is based on the pre-trained T5 model and undergoes fine-tuning for better zero-shot and few-shot performance through instructions. Qwen-7B (7B). This model is one of the Tongyi Qianwen large model series developed by Alibaba Cloud, boasting 7 billion parameters. It is based on the Transformer architecture and trained on a massive pre-training dataset. This pre-training data includes high-quality Chinese, English, multilingual, code, and mathematical content, covering both general and professional domains. It performs excellently on multiple downstream evaluation tasks in both Chinese and English, even surpassing larger-sized models in some metrics. Bloom-7B (7B).

[0173] Specifically, in this embodiment, under the Windows 10 system, Intel i7-8700K CPU, 24GB memory, and NVIDIA RTX 3090 GPU environment, the PyCharm development tool was used, and representative interpretation generation methods were selected for comparison.

[0174] Table 1. Comparative experimental results of the declaration test (%);

[0175]

[0176] Specifically, as shown in Table 1, the comparative experimental results of claim detection demonstrate that few-shot hint learning achieves better results compared to zero-shot hint methods. For example, the Qwen model's ACC score improved by 18.73%, and its F1 score improved by 24.38%. This may be because few-shot hint methods can provide the model with more contextual information, enabling it to better understand the task requirements, thereby improving the accuracy and consistency of predictions. Furthermore, few-shot hints can also help the model better capture patterns and regularities in the data, further enhancing its performance.

[0177] After training with QLoRA, the table shows a further improvement in model performance. For example, compared to methods with few-shot hints, the QLoRA model achieves an ACC score improvement of approximately 122.39% and an F1 score improvement of approximately 116.11%. This indicates that the QLoRA method not only significantly reduces GPU memory usage through quantization techniques, making it possible to fine-tune large models on a single GPU, but also further improves model performance while maintaining training efficiency.

[0178] In summary, the QLoRA fine-tuning method demonstrates significant advantages in improving model performance. Furthermore, experimental results show that, in most cases, joint tasks outperform single tasks, although the specific results may be influenced by multiple factors such as model architecture and task configuration.

[0179] Table 2 Explanation of the generated comparative experimental results (%);

[0180]

[0181] Specifically, as shown in Table 2, the results of the comparative experiments are explained as follows:

[0182] With zero-shot hints, the Bloom model achieved the highest ROUGE-1 score in the joint task. In the single task of interpreting generated data, Bloom and Flan-t5-xl performed exceptionally well. Qwen achieved a ROUGE-L score of 15.4 in the joint task, slightly higher than other models, which may be attributed to Qwen's strength in handling complex tasks.

[0183] With limited-sample cues, the table demonstrates the rapid learning ability of LLMs to learn from examples. For instance, the Bloom model performs exceptionally well in the joint task, achieving a ROUGE-1 score of 26.4 and a ROUGE-L score of 17.2, showcasing its superiority under limited-sample conditions. Qwen achieves a ROUGE-1 score of 24.1 in the interpretive generation task and reaches 24.3 in the joint task. All models show a certain degree of performance improvement compared to zero cues.

[0184] After fine-tuning with Qlora, all models showed significant improvements in ROUGE scores for both the explanation / generation and joint tasks. Flan-t5-xl, in particular, achieved outstanding performance, with a ROUGE-1 score of 30.1 and a ROUGE-L score of 26.3 in the explanation / generation task, and a ROUGE-1 score of 30.7 and a ROUGE-L score of 27.3 in the joint task.

[0185] In addition, it can be observed from the table that the ROUGE score of joint tasks is higher than that of single tasks. The reasons for this can be analyzed in conjunction with the calculation method of ROUGE score as follows.

[0186] First, the joint task, by simultaneously learning two tasks—claim detection and interpretation generation—can share feature representations, thereby improving the model's understanding of the relationship between claims and evidence and thus increasing the ROUGE score.

[0187] Secondly, features in the claim detection task can enhance the model's understanding of the consistency between claims and evidence, while features in the explanation generation task can help the model generate more detailed and logical explanations. Information sharing and fusion enable the model to generate more comprehensive and accurate explanations, thereby increasing the overlap between the generated text and the reference text.

[0188] Finally, the joint task model can directly consider the results of claim detection when generating explanations, which means that the generated explanations can respond more specifically to the veracity judgment of claims, thereby obtaining higher scores in the ROUGE evaluation.

[0189] Overall, few-shot prompts generally achieve higher ROUGE scores than zero-shot prompts, reflecting that few-shot prompts significantly improve the model's understanding of the task by providing more specific task examples. Furthermore, the ROUGE score for joint tasks is slightly higher than for single tasks, which may indicate that the model maintains performance better when handling multi-task or more complex tasks. Qlora fine-tuning significantly improves the performance of all models, showing excellent results in both interpreting and generating tasks and joint tasks, demonstrating that fine-tuning can effectively improve model performance on specific tasks.

[0190] Specifically, to more comprehensively evaluate the quality of the model-generated explanations, this embodiment also designed three progressively higher human evaluation criteria: basic quality criteria (covering linguistic consistency, information reuse, and authenticity), enhanced consistency criteria (further requiring logical consistency and contextual consistency on top of the basic criteria), and comprehensive completeness criteria (requiring all seven evaluation dimensions to be met simultaneously). The experiment randomly selected 30 samples from the test set for human scoring. Among the models fine-tuned using QLoRA, Flan-T5-XL and Qwen-7B showed stable performance and high overall scores. Qwen-7B, in particular, achieved a comprehensive completeness score of 28.5 in the joint task, demonstrating strong explanation generation capabilities. In contrast, while Bloom-7B and GPT-Neo performed reasonably well on some metrics, their comprehensive completeness scores declined, reflecting shortcomings in the accuracy, coherence, and completeness of their explanations. For example, they still have limitations in internal / external consistency, duplicate content control, and information completeness processing. It is worth noting that the manually written "standard explanations" scored only 25.0 on various metrics, significantly lower than the results of most large models. This indicates that the expert explanations collected in practice are not always of high quality. This may stem from ambiguities in the label mapping during the construction of the RU22Fact dataset (unifying multi-source rumor verification labels into three categories: "true / false / unverified") and the fact that journalists relied on diverse information sources and even subjective knowledge when writing explanations, rather than solely on given evidence, thus affecting the consistency and standardization of the explanations. In summary, large language models fine-tuned with QLoRA, especially Qwen-7B, demonstrate the potential to outperform human benchmarks in generating high-quality, multi-dimensionally consistent explanations.

[0191] like Figure 2 As shown, this embodiment also discloses a large language model interpretation and generation system based on fine-tuning and joint tasks, including:

[0192] The matching module 21 preprocesses the collected text data to obtain preprocessed data, extracts text statements and corresponding text evidence from the preprocessed data and matches them to form structured input and output samples.

[0193] Template generation module 22 guides the large language model to process statements and evidence based on structured input and output samples and specific format prompts, generating prompt templates suitable for zero-sample and few-sample scenarios;

[0194] The semantic representation generation module 23, based on the prompt template and structured input and output samples, performs encoding processing on the structured input and output samples through improved mmBERT text encoding to obtain text semantic representations suitable for the declaration detection task and text semantic representations suitable for the interpretation generation task.

[0195] The fine-tuning module 24 inputs the text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task into the large language model through the QLoRA parameter fine-tuning technique, and fine-tunes the declaration detection task and the interpretation generation task.

[0196] The joint training module 25 performs joint training on the fine-tuned statement detection task and explanation generation task, and fine-tunes the joint task using QLoRA to obtain the final explanation generation model.

[0197] The interpretation generation module 26 interprets and generates interpretations of the newly collected text data through the final interpretation generation model, thereby obtaining a true interpretation of the text.

[0198] The specific implementation of the large language model interpretation and generation system based on fine-tuning and joint tasks is the same as that of the large language model interpretation and generation method based on fine-tuning and joint tasks, and will not be described again in this embodiment.

[0199] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.

Claims

1. A method for generating interpretations of large language models based on fine-tuning and joint tasks, characterized in that, include: S1. Preprocess the collected text data to obtain preprocessed data. Extract text statements and corresponding text evidence from the preprocessed data and match them to form structured input and output samples. S2 guides large language models to process statements and evidence based on structured input and output samples and specified format prompts, generating prompt templates suitable for zero-shot and few-shot scenarios. S3, based on prompt templates and structured input and output samples, uses improved mmBERT text encoding to encode the structured input and output samples, obtaining text semantic representations suitable for declaration detection tasks and text semantic representations suitable for interpretation generation tasks; S4. The text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task are input into the large language model through the QLoRA parameter fine-tuning technique to fine-tune the declaration detection task and the interpretation generation task. S5. Jointly train the fine-tuned statement detection task and explanation generation task, and fine-tune the joint task using QLoRA to obtain the final explanation generation model. S6, the newly collected text data is interpreted and generated through the final interpretation and generation model to obtain a true interpretation of the text; The calculation formula for fine-tuning the declaration detection task is as follows: ； ； ； in, For the Sigmoid function; This indicates that QLoRA is used for fine-tuning of the large model; This indicates that parameter adjustments were made under the declaration detection task, and the result is... and ; and These are the trainable parameters of the model; The loss function; function Used to determine a declaration Whether evidence has been obtained The support, among which and By function generate; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set representing the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; express All predictions prior to the given time; The set of predictions represents the declaration; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; The calculation formula for fine-tuning the interpretation generation task is as follows: ； ； ； in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This indicates the task to be interpreted and generated; This indicates that the model output is being decoded; The loss function; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; The training set represents the explanation; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; The prediction set represents the explanation; The fine-tuned statement detection task and the fine-tuned interpretation generation task are jointly trained, and the calculation formula is as follows: ； ； ； in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This represents the generated declaration and its interpretation, where and For function generate; This is the loss function.

2. The method for generating large language model interpretations for fine-tuning and joint tasks according to claim 1, characterized in that, In S2, the prompt template applicable to zero-sample and few-sample scenarios includes a single-task prompt word template for whether the evidence supports the claim in a zero-sample prompt scenario. Single-task prompt templates for scenarios with zero-sample prompts, and for situations where there is insufficient supporting evidence. How can the generation of evidence in zero-sample scenarios support joint task prompt templates for statements and interpretations? How can we support joint task prompt word templates for statements and interpretations when directly generating evidence in scenarios with few samples? They are defined as follows: ; in, A single-task prompt template indicating whether evidence supports a claim in a zero-sample scenario; This indicates a declaration; Present evidence; This indicates the instructions input to the large model in this scenario; Indicates text concatenation; ； in, A single-task prompt template indicating whether relevant supporting evidence is available in zero-sample prompt scenarios and when there is insufficient information to support the evidence. Indicate how the evidence supports the statement. This indicates the command to input a large model in this scenario. ； in, How can the direct generation of evidence in zero-sample scenarios support joint task prompt word templates for statements and interpretations? This indicates the instructions input to the large model in this scenario; ； in, This section describes a joint task prompt template for how directly generated evidence can support declarations and interpretations in scenarios with few samples. This indicates that a small number of samples have been provided. This indicates the instructions input to the large model in this scenario.

3. The method for generating large language model interpretations based on fine-tuning and joint tasks according to claim 1, characterized in that, In S3, the improved mmBERT text encoding is defined as follows: ； in, Indicates text input; Indicates word embedding; Indicates positional encoding; This represents attention encoding; Indicates the number of layers in the Transformer; in this scenario, it will... The number of layers was increased to 8 to improve efficiency; This means compressing the matrix output by the encoder into a fixed-dimensional semantic vector; This is represented as the final generated text semantic vector; Cosine similarity is used to measure semantic consistency. This indicates minimizing the number of tokens; This represents the balance coefficient, prioritizing ensuring that the semantics of the low-resource language are not lost; This represents the token sequence after preprocessing text fragmentation; This represents the preprocessed text; A temporary semantic vector representing the preprocessed text; A temporary semantic vector representing a sequence of tokens.

4. A large language model interpretation and generation system based on fine-tuning and joint tasks, characterized in that, include: The matching module preprocesses the collected text data to obtain preprocessed data, extracts text statements and corresponding text evidence from the preprocessed data and matches them to form structured input and output samples. The template generation module guides the large language model to process statements and evidence based on structured input and output samples and specified format prompts, generating prompt templates suitable for zero-shot and few-shot scenarios. The semantic representation generation module, based on prompt templates and structured input and output samples, performs encoding processing on the structured input and output samples through improved mmBERT text encoding to obtain text semantic representations suitable for declaration detection tasks and text semantic representations suitable for interpretation generation tasks. The fine-tuning module inputs the text semantic representations suitable for the declaration detection task and the text semantic representations suitable for the interpretation generation task into the large language model through the QLoRA parameter fine-tuning technique, and fine-tunes the declaration detection task and the interpretation generation task. The joint training module performs joint training on the fine-tuned statement detection task and explanation generation task, and fine-tunes the joint task using QLoRA to obtain the final explanation generation model. The interpretation and generation module interprets and generates interpretations of newly collected text data through the final interpretation and generation model, thereby obtaining a realistic interpretation of the text. The calculation formula for fine-tuning the declaration detection task is as follows: ； ； ； in, For the Sigmoid function; This indicates that QLoRA is used for fine-tuning of the large model; This indicates that parameter adjustments were made under the declaration detection task, and the result is... and ; and These are the trainable parameters of the model; The loss function; function Used to determine a declaration Whether evidence has been obtained The support, among which and By function generate; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; The calculation formula for fine-tuning the interpretation generation task is as follows: ； ； ； in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This indicates the task to be interpreted and generated; This indicates that the model output is being decoded; The loss function; express The prediction result at any given time; This represents the semantic vector of the text generated using mmBERT; Represents the calculation of conditional probability; The training set is used to indicate the declaration; The training set representing the evidence; A training set that shows how the evidence supports the statement; The training set represents the explanation; express All predictions prior to the given time; The set of predictions declared; A prediction set representing evidence; A set of predictions indicating how the evidence supports the statement; The prediction set represents the explanation; The fine-tuned statement detection task and the fine-tuned interpretation generation task are jointly trained, and the calculation formula is as follows: ； ； ； in, This indicates that the parameter adjustments for the generation task were explained, and the result is... and ; and These are the trainable parameters of the model; This represents the generated declaration and its interpretation, where and For function generate; This is the loss function.

Images