Model training method, validation model, data validation method, and program product

Abstract

This application provides a model training method, a verification model, a data verification method, and a program product, relating to the field of data analysis technology. The model training method includes: within a debate framework, constructing an adjudicator role based on the verification results output by the first and second roles, based on a first role supporting the target claim and a second role refuting the target claim; and post-training the adjudicator role based on a training dataset to obtain a verification model. The verification model includes: a first role supporting the target claim; a second role refuting the target claim; and an adjudicator role outputting verification results based on the first and second roles. The data verification method includes: acquiring the target data to be verified; conducting debate based on the target data using the first and second roles in the verification model; and making a judgment using the adjudicator role in the verification model to obtain the target verification result.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and more specifically, to a model training method, a model validation method, a data validation method, and a program product. Background Technology

[0002] Currently, in various data processing applications, such as content review and fact-checking for platforms and media to curb the spread of misinformation, enhancing search engines and question-answering systems to provide users with more reliable information, and conducting data analysis, learning research, and processing large amounts of text information, data is typically processed using a single large language model (LLM) based on Retrieval Enhanced Generation (RAG) and the authenticity of the data is verified through claim verification schemes.

[0003] However, when using a single large language model (LLM) based on retrieval augmentation to process data and perform tasks such as claim verification, it suffers from reasoning limitations. For example, when there are contradictions between pieces of evidence or when deep reasoning is required, a single LLM is prone to errors. In complex issues involving ethics, legal interpretation, and scientific controversies that require weighing multiple perspectives, the model may tend to make guesses rather than honest admissions in cases of "insufficient evidence." Therefore, existing large language models struggle to handle complex and contradictory evidence, exhibit low analytical precision, and produce results with low accuracy and interpretability, failing to meet current processing needs. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a model training method, a model verification method, a data verification method, and a program product to improve the problem of poor data verification effect in the prior art.

[0005] To address the aforementioned problems, firstly, embodiments of this application provide a model training method, the method comprising: In the debate framework, an adjudicating role is constructed based on the first role supporting the target claim and the second role refuting the target claim, and the adjudicating role is constructed based on the verification results output by the first role and the second role. Based on the training dataset, the adjudicator role is post-trained to obtain a validation model.

[0006] In the aforementioned implementation process, an intelligent debate framework can be constructed. Based on the first and second roles with different objectives, an adjudicator role is built that can output verification results based on the debate between the two roles, simulating a human-like, debate-driven statement verification process. Furthermore, to further improve the effectiveness of the adjudicator role's output verification results and reduce the problem of the adjudicator role tending to make neutral (vague) rulings, which is inconsistent with the behavior of human verifiers making clear judgments based on sufficient evidence, the adjudicator role can be post-trained using a pre-set training dataset to obtain the corresponding verification model. Post-training can improve the effectiveness of the verification model in handling complex and contradictory evidence, improve the accuracy of data analysis, and improve the accuracy and interpretability of verification results, meeting the processing needs of various application scenarios.

[0007] Optionally, the training dataset is determined in the following ways: The zero-shot-based debate framework processes a historical dataset with annotation information to obtain initial data; wherein, the initial data includes multiple rounds of debate records of the first role and the second role, as well as the judgment rules of the adjudicator role; Based on the historical dataset, identify the samples of the adjudicator role that were incorrectly determined; The initial data is optimized using a corrector and the samples to be corrected to obtain the training dataset.

[0008] In the above implementation process, considering the tendency of the adjudicator to be overly neutral in the zero-sample scenario—that is, to tend to give ambiguous verification results such as insufficient evidence or "conflicting evidence"—which is inconsistent with the behavior of human fact-checkers who make clear judgments based on evidence, and taking into account the scarcity of human debate data, we can process historical datasets with annotation information based on a template-based debate framework to obtain initial data containing multiple rounds of debate records of the first and second roles, as well as the adjudicator's judgment rules. Since there are erroneous judgment data in the historical dataset, we can first identify the erroneous judgment samples in the historical dataset that need to be corrected. We can then use a corrector to determine the reasons for the correction after identifying the erroneous samples, and optimize the initial data accordingly to simulate the process of human experts making correct inferences based on the same arguments, thus obtaining a training dataset with high effectiveness. We can then post-train the adjudicator based on the synthesized training dataset, effectively solving the disadvantage of overly neutral verification results in the zero-sample scenario.

[0009] Optionally, the historical dataset is determined in the following ways: Acquire historical debate data in multiple languages; Based on preset evidence conditions, the historical debate data is classified and filtered to obtain classified data; wherein, the evidence conditions include: golden evidence conditions, searchable evidence conditions, and no evidence conditions; Each of the categorized data points is labeled to obtain the historical dataset.

[0010] In the above implementation process, to achieve the verification function for data in multiple languages, historical debate data in various languages ​​can be acquired. Based on preset evidence conditions, the historical debate data is classified and filtered to obtain categorized data. Semantic annotation processing is then performed on each categorized data to label the various processes and logic within the data, resulting in a historical dataset with semantic annotation information. The ability to classify historical data in multiple languages ​​to obtain annotated historical datasets effectively improves the diversity and authenticity of historical datasets.

[0011] Optionally, the step of post-training the adjudicator role based on the training dataset to obtain a validation model includes: Based on the training dataset, the adjudicator role is trained in a forward manner to obtain an intermediate model; Based on the training dataset, the intermediate model is optimized for preferences to obtain the validation model.

[0012] In the above implementation process, when post-training the adjudicator role based on the training dataset, considering the different situations of correct and incorrect judgments, to achieve comprehensive debate-driven post-training, the adjudicator role can be positively trained based on the training dataset. This allows the adjudicator role to learn how to output correct judgments and reasons based on the debate context, resulting in a corresponding intermediate model. Furthermore, the intermediate model can be optimized using preferences based on the training dataset, encouraging the model to generate responses consistent with human judgment and penalizing incorrect responses, resulting in the final validation model. This allows the adjudicator role to be trained from different perspectives, enabling the final validation model to make judgments and reasoning operations closer to those of human experts in complex debate contexts, thereby improving the effectiveness and accuracy of the validation results.

[0013] Optionally, the step of forward training the adjudicator role based on the training dataset to obtain an intermediate model includes: Identify the correct samples in the training dataset that were correctly judged; Based on the correct samples, the adjudicator role is trained to obtain the intermediate model with a first type of loss.

[0014] In the above implementation process, correct samples that are judged correctly can be identified first from the training dataset. Then, based on the correct samples, the adjudicator role is forward-trained to obtain an intermediate model with Type I loss. By forward-training with correct samples, the adjudicator role can learn to generate correct logic that conforms to the correct result, effectively improving the verification performance of the intermediate model.

[0015] Optionally, the step of performing preference optimization on the intermediate model based on the training dataset to obtain the validation model includes: Identify the erroneous samples in the training dataset that were incorrectly identified; Based on the erroneous samples, and in conjunction with the corrector, preference pairs are determined; Based on the preference pair, the intermediate model is trained to obtain the validation model with the second type of loss.

[0016] In the above implementation process, erroneous samples in the training dataset can be identified first. These erroneous samples and the corrector are then combined to generate corresponding preference pairs. Based on these preference pairs, the intermediate model is further trained to obtain a validation model with a second type of loss. This training using erroneous samples and the corrector's preferences enables the adjudicator to make correct responses to different situations, effectively improving the validation performance of the validation model.

[0017] Optionally, the preference pair includes: a preferred response and a rejected response; The priority response is determined based on the true determination of the erroneous sample and the correction rationale of the corrector; The rejection response is determined based on the error judgment and error reason of the error sample.

[0018] In the above implementation process, the constructed preference pairs include preferred responses and rejection responses. Preferred responses can be determined based on the true judgment of erroneous samples and the correction reasons generated by the corrector based on them, while rejection responses can be determined based on the erroneous judgment and erroneous reasons of erroneous samples. Based on the constructed preference pairs, the validation model can be trained to maximize the probability of generating preferred responses and minimize the probability of generating rejection responses, further reducing the disadvantage of the validation model being too neutral in zero-shot scenarios.

[0019] Optionally, the method further includes: Configure the debate rules between the first role and the second role; wherein, the debate rules include: the number of debates between the first role and the second role; Based on the debate rules, the termination conditions for the adjudicator role are determined; wherein, the termination conditions include: determining whether the first role and / or the second role outputs a new argument.

[0020] In the above implementation process, the working parameters of multiple roles in the intelligent debate framework can be configured, the debate rules between the first and second roles with different objectives can be configured, and based on the debate rules, the termination conditions for the adjudicator role to control the two roles to stop debating can be configured to control the two roles to stop debating after analysis. It is possible to configure the working parameters of multiple different types of roles in the debate framework accordingly, so that multiple roles in the debate framework can simulate the human debate process and obtain corresponding verification results.

[0021] Optionally, the method further includes: Determine the accuracy threshold based on usage requirements; If the accuracy of the verification result of the verification model is lower than the accuracy threshold, the verification model shall be adjusted.

[0022] In the above implementation process, in order to determine that the obtained verification model has high verification performance, an accuracy threshold can be determined based on the actual usage requirements of the application scenario in which the verification model is located. The accuracy threshold is compared with the accuracy of the verification results generated by the verification model. When the accuracy of the verification results is lower than the accuracy threshold, it indicates that the accuracy of the verification model is low and the verification performance is poor. The verification model can continue to be adjusted or retrained until the accuracy of the verification results is higher than or equal to the accuracy threshold. This indicates that the verification model can meet the usage requirements of the application scenario and can be put into use.

[0023] Secondly, embodiments of this application also provide a verification model, which is trained based on the training method described in any one of the first aspects above, and the verification model includes: Based on the primary role of supporting the target claims; A second role in refuting the stated objective claims; The adjudicating role is determined based on the verification results output by the first role and the second role.

[0024] In the above implementation process, the verification model can conduct multiple rounds of debate around a given statement and a set of evidence. The first and second roles can propose and defend their own arguments and challenge the opposing arguments based on the evidence, while the adjudicator role can evaluate each round and make a final ruling, outputting the corresponding verification results.

[0025] Thirdly, embodiments of this application also provide a data verification method, the method comprising: Obtain the target data to be verified; The first and second roles in the verification model perform debate processing based on the target data, and the adjudicator role in the verification model makes a judgment to obtain the target verification result; wherein, the verification model is obtained based on the model training method described in any one of the first aspects above.

[0026] In the above implementation process, in practical application scenarios, the target data to be verified can be obtained first, and then input into the verification model for processing. The verification model can conduct multiple rounds of debate around the given target data. The first and second roles can propose and defend their own arguments and challenge the opponent's arguments based on the target data, and the adjudicator role can evaluate each round and make a final judgment, outputting the corresponding target verification results.

[0027] Fourthly, embodiments of this application provide a computer program product, the computer program product including a computer program / instruction, which, when executed by a processor, implements the steps in the method described in any of the first aspects above.

[0028] In summary, the embodiments of this application provide a model training method, a verification model, a data verification method, and a program product, which can improve the effectiveness of the verification model in processing complex and contradictory evidence through post-training processing, improve the accuracy of data analysis, and improve the accuracy and interpretability of verification results, thus meeting the processing needs of various application scenarios. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application; Figure 2 A flowchart illustrating the first model training method provided in this application embodiment; Figure 3 A flowchart illustrating the second model training method provided in this application embodiment; Figure 4 A flowchart illustrating the third model training method provided in this application embodiment; Figure 5 A detailed flowchart of step S220 provided for an embodiment of this application; Figure 6A detailed flowchart of step S221 provided for an embodiment of this application; Figure 7 A detailed flowchart of step S222 provided for an embodiment of this application; Figure 8 A flowchart illustrating the fourth model training method provided in this application embodiment; Figure 9 A flowchart illustrating the fifth model training method provided in this application embodiment; Figure 10 A schematic diagram of an evaluation result provided for an embodiment of this application; Figure 11 This application provides a performance diagram illustrating the differences between different models. Figure 12 This is a schematic diagram of the structure of a verification model provided in an embodiment of this application; Figure 13 This is a flowchart illustrating a data verification method provided in an embodiment of this application.

[0031] Icons: 100 - Electronic device; 111 - Memory; 112 - Memory controller; 113 - Processor; 114 - Peripheral interface; 115 - Input / output unit; 116 - Display unit; 410 - First role; 420 - Second role; 430 - Decision role. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0033] In today's digital environment where misinformation spreads widely and rapidly online, claim verification has become a core technology for assessing the authenticity of information. It can mitigate the impact of misinformation and improve digital literacy. Existing claim verification frameworks typically utilize Large Language Models (LLMs) to collect relevant evidence from external sources (such as the internet) through Retrieval Augmentation Generation (RAG) techniques. They then analyze the consistency or contradiction between the claim and the evidence to assess the claim's authenticity. In this framework, the target claim and the collected evidence are integrated into structured prompts, guiding a single LLM to generate predicted conclusions and corresponding reasons, clarifying the basis for its authenticity assessment. However, a single LLM has limitations in its reasoning ability when verifying complex, multi-dimensional claims. For example, the HerO model using the above framework incorrectly refutes the claim that "52% of Nigeria's current population lives in urban areas," relying solely on outdated evidence from 2008 showing an urbanization rate of 36%, while ignoring more recent evidence showing an urbanization rate of 53.52%. This highlights the inadequacy of a single LLM in reconciling inconsistencies among the collected evidence, likely due to a mismatch between its pre-training objectives (learning language patterns and autoregressively predicting the most likely next token) and the demands of claim verification for refined analysis of evidence and robust reasoning.

[0034] Therefore, in processing data using a single large language model based on retrieval enhancement to achieve tasks such as claim verification, there are limitations in reasoning. For example, when there are contradictions between pieces of evidence or when deep reasoning is required, a single LLM is prone to errors. This involves complex issues such as ethical considerations, legal interpretations, and scientific controversies that require weighing multiple perspectives. In cases of "insufficient evidence," the model may tend to make guesses rather than honest admissions. Consequently, existing large language models struggle to handle complex and contradictory evidence, exhibit low analytical precision, and produce results with low accuracy and interpretability, failing to meet current processing needs.

[0035] To address the aforementioned issues, this application provides a model training method, a verification model, a data verification method, and a program product. These methods can improve the effectiveness of the verification model in processing complex and contradictory evidence through post-training processing, enhance the accuracy of data analysis, and improve the accuracy and interpretability of verification results, thus meeting the processing needs of various application scenarios.

[0036] This application provides a model training method and a data verification method that can be applied to electronic devices, such as servers, personal computers (PCs), tablets, smartphones, personal digital assistants (PDAs), and other electronic devices with logical computing functions.

[0037] Optionally, please refer to Figure 1 , Figure 1 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0039] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0040] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0041] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0042] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and a keyboard.

[0043] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display various data such as the final output verification result and the target verification result.

[0044] The electronic device in this embodiment can be used to execute various steps in the model training and data validation methods provided in the embodiments of this application. The implementation process of the model training and data validation methods is described in detail below through several embodiments.

[0045] Please see Figure 2 , Figure 2 This is a flowchart illustrating a first model training method provided in an embodiment of this application. The method may include steps S210-S220.

[0046] Step S210: In the debate framework, based on the first role supporting the target claim and the second role refuting the target claim, construct an adjudicator role based on the output verification results of the first and second roles.

[0047] Among them, an intelligent debate framework can be constructed, based on the first and second roles with different objectives, and an adjudicator role can be constructed to output verification results based on the debate situation of the two roles, so as to simulate a human-like, debate-driven statement verification process.

[0048] It is important to note that, compared to existing claim verification methods, in the best practices of real-world fact-checking organizations, human fact-checkers employ a debate-driven approach for claim verification. Specifically, human fact-checkers critically evaluate each other's judgments of truthfulness, identifying potential flaws, particularly logical errors and misunderstandings in the presented evidence (such as ignoring the latest urbanization statistics in the example above). Conversely, fact-checkers need to refine their judgments by addressing identified flaws and re-analyzing the evidence from a more nuanced perspective to strengthen their arguments. This iterative process can be repeated multiple times, continuously optimizing until a consensus is reached on the conclusion. This framework thus enhances the depth and rigor of evidence analysis, facilitating a comprehensive evaluation of claims. Inspired by real-world fact-checking practices, this application proposes the first evidence-based multi-agent debate framework for claim verification, which can be named DebateCV. This framework employs multiple agents based on a Large Language Model (LLM), namely, a first role, a second role, and an adjudicator role, to simulate a human-like debate-driven claim verification process, achieving refined evidence analysis and reasoning.

[0049] Optionally, the first role can support a claim, while the second role can refute it. Both sides challenge each other's positions in multiple rounds of debate and use collected evidence to defend themselves. After each round, the adjudicator evaluates the arguments presented by both debaters, summarizes the content of the round, and decides whether to continue the debate. If no further debate is needed, the adjudicator will provide a final conclusion and generate reasons as verification. By simulating the evaluation process driven by human debate, the debate framework can examine evidence from different perspectives, iteratively optimize reasoning, and ultimately arrive at a well-founded conclusion.

[0050] It should be noted that the first and second roles in the debate framework are the debaters, divided into the affirmative debater (first role) and the negative debater (second role), with the moderator as the arbitrator. The core responsibility of the affirmative debater is to support the target claim, constructing arguments using provided data, such as a set of evidence, and critically analyzing and refuting the opposing viewpoint based on the evidence. The core responsibility of the negative debater is to refute the target claim, presenting counter-evidence using the same set of evidence, questioning the sufficiency of the affirmative evidence, or providing alternative explanations for the refutation. The arbitrator is responsible for monitoring the debate process, assessing the strength of both sides' evidence and arguments, summarizing key points of contention, and determining whether further debate is necessary. After the debate, the arbitrator outputs a final verdict on the veracity of the claims (support, refute, insufficient evidence, conflicting evidence / selective presentation) and the corresponding reasons, serving as the verification result.

[0051] Optionally, the debate framework's workflow may include: the first role presents arguments supporting claim c based on evidence, followed by a rebuttal from the second role. Both sides continuously defend their positions and question the opposing viewpoints; after each round, the adjudicator summarizes new insights and points out gaps in the evidence. If both sides merely repeat their viewpoints without providing new information, the debate terminates; otherwise, it continues, ultimately generating a verification result, which includes the final predicted conclusion. and the corresponding reasons for their generation. .

[0052] Step S220: Based on the training dataset, post-train the adjudicator role to obtain the validation model.

[0053] In order to further improve the effectiveness of the verification results output by the adjudicator and reduce the problem that the adjudicator tends to make neutral (vague) decisions and is inconsistent with the behavior of human inspectors who make clear judgments based on sufficient evidence, the adjudicator can be post-trained using a pre-set training dataset to obtain the corresponding verification model.

[0054] Optionally, to overcome the inherent neutrality of the adjudicator in heated debates, debate can be used to align their reasoning with the human decision-making process in claim verification. To this end, this application designs a novel debate-driven post-training strategy that fully utilizes synthetic debate data, i.e., the training dataset, to post-train the adjudicator. Experimental results show that, under different evidence conditions, this framework outperforms existing claim verification methods in both zero-shot and supervised scenarios, providing an accurate and interpretable solution.

[0055] exist Figure 2In the illustrated embodiment, post-training processing can improve the effectiveness of the verification model in processing complex and contradictory evidence, improve the accuracy of data analysis, and improve the accuracy and interpretability of verification results, thus meeting the processing needs of various application scenarios.

[0056] Optionally, in zero-shot scenarios, even with sufficient evidence for truth assessment, the adjudicator tends to give a neutral conclusion (neither supporting nor refuting the claim). This behavior stems from the inherent neutrality of the adjudicator—LLMs are typically trained to remain impartial during alignment, rather than taking a strong stance like human fact-checkers in claim verification debates. To mitigate this overly neutral behavior, it is necessary to post-train the adjudicator to align with the judgment methods of human fact-checkers. However, this requires human debate-driven claim verification labeled data, which is scarce and costly to obtain. To enable effective training, this application proposes a novel debate-driven post-training strategy utilizing synthetic debate data, i.e., the training dataset. The zero-shot DebateCV framework can be used to simulate claim debates with human-labeled true conclusions, thereby synthesizing debate transcripts (i.e., debaters' multi-round arguments) and the conclusions and reasons given by the adjudicator. To ensure the quality of the synthesized reasons, this application introduces another LLM-based corrector agent capable of supervising the generation of reasons based on the true conclusions of the claim. By using reinforcement learning methods, the adjudicator role can more accurately align with the judgments of human fact-checkers, thereby improving its performance in debate-driven claim verification.

[0057] It should be noted that although the debate framework of this application can effectively simulate debate-driven claim verification, preliminary experiments show that, with zero samples, the adjudicator tends to over-balance and favor neutral conclusions. Specifically, despite higher accuracy, the adjudicator has a higher false positive rate on conclusions in the "insufficient evidence" (0.105 vs. 0.047) and "conflicting evidence / selective presentation" (0.355 vs. 0.144) categories compared to the RAG-based baseline model. This behavior is inconsistent with human fact-checkers—humans take a stronger stance based on stronger reasoning and evidence. However, records of human debate-driven claim verification are scarce, and these records are crucial for post-training the adjudicator to align with human fact-checkers in debates. Therefore, this application also discloses a debate-driven post-training strategy based on synthetic debate data, which includes two steps: debate data synthesis and debate-driven post-training.

[0058] Optionally, please refer to Figure 3 , Figure 3 This is a flowchart illustrating a second model training method provided in an embodiment of this application. The method may further include steps S311-S313.

[0059] Step S311: Process the historical dataset with annotation information based on the zero-shot debate framework to obtain the initial data.

[0060] The initial data can include multi-round debate records of the first and second roles, as well as the adjudicator's judgment rules. Considering the tendency of the adjudicator to be overly neutral in the zero-sample scenario, i.e., to give ambiguous verification results such as insufficient evidence or "conflicting evidence," which is inconsistent with the behavior of human fact-checkers who make clear judgments based on evidence, and taking into account the scarcity of human debate data, the historical dataset with annotation information can be processed based on the leading-sample debate framework to obtain the initial data containing multi-round debate records of the first and second roles and the adjudicator's judgment rules.

[0061] Optionally, the judgment rules may include various types of data such as the judgment process and judgment reasons of the adjudicator.

[0062] It should be noted that existing research has designed multi-agent debate frameworks to solve specific downstream tasks, such as machine translation, hallucination mitigation, weak-to-strong supervision, and position detection. Compared with existing frameworks, the claim verification task in this application simulates a scenario closer to real-world fact-checking, emphasizing the identification of contradictions or consistency between collected evidence and claims. Furthermore, existing research focuses on applying pre-trained LLMs to multi-agent debate, with only three previous studies utilizing multi-agent debate to train LLMs. For example, multi-agent debate is used to refine the training corpus of LLMs; persuasive argument corpora are generated through multi-agent debate to enhance the persuasiveness of LLMs; and a debate game based on LLMs among agent teams is designed to improve their teamwork capabilities. In contrast, this application utilizes multi-agent debate to synthesize informative reasons based on debate-driven evidence analysis and uses the synthesized debate context to post-train the LLM for downstream tasks. This is the first proposed strategy for post-training LLMs for downstream tasks in multi-agent debate.

[0063] Step S312: Based on the historical dataset, identify the samples that have incorrectly determined the adjudicator role.

[0064] Step S313: Optimize the initial data using a corrector and the samples to be corrected to obtain the training dataset.

[0065] Since there are erroneous judgments in the historical dataset, we can first identify the erroneous judgment samples in the historical dataset that need to be corrected. By combining the corrector with the samples to be corrected, we can determine the reasons for the correction and optimize the initial data accordingly. This will simulate the process of human experts making correct inferences based on the same arguments, and obtain a training dataset with high effectiveness.

[0066] For example, the zero-shot DebateCV framework can be used to simulate debates on a historical dataset (3068 claims) of the AVeriTeC dataset, generating debate records for the first and second roles, as well as the initial predicted verdict and reasons for the adjudicator. A corrector agent can be introduced: for samples to be corrected where the initial predicted verdict is inconsistent with the true verdict, the debate record and the true verdict are input, guiding the corrector to generate corrective reasons "derived from the debate to the true verdict," ensuring that the reasons conform to the reasoning logic of human fact-checking. Each sample contains: claim, debate record, true verdict, initial predicted verdict, initial reason, and corrective reason (only present in samples to be corrected), constructing a synthetic dataset, denoted as SynDeC, as the corresponding training dataset.

[0067] Optionally, a debate record is first generated using a debate framework, and the conclusion and reasons are output. Specifically, given n claims... and the corresponding collection of evidence The DebateCV framework sparks debate. To verify these claims, the base adjudicator then outputs a truth label for the initial prediction. and the corresponding reasons Because predicted conclusions and reasons may contain errors, and because human fact-checkers may have inconsistent preferences in debate scenarios, this application introduces another LLM-based corrector agent. This agent synthesizes alternative reasons for the same claim based on the debate record and the actual conclusion. Specifically, for each incorrect predicted conclusion... The claim Input debate records into the corrector and the true conclusion Guide them to generate reasons for correction. Explain how to come from the debate Derive the true conclusion from the middle It can guide the analysis of the corrector. The debate examines the strengths and weaknesses of the arguments presented by both sides, and attempts to provide a coherent reasoning basis for a valid conclusion. (Reasoning for synthesis) It approximates the process of a human fact checker verifying facts through this debate. The reasoning process.

[0068] Optionally, each sample in the constructed training dataset SynDeC contains: claims ( Debate records ), true conclusion ( ), Prediction conclusions ( ), Reasons for prediction ( ) and the reasons for the amendment (if Then includes The SynDeC dataset provides support for post-validation training of argument-driven claims in LLM.

[0069] Optionally, this application proposes an innovative debate-driven post-training strategy that uses synthetic debate data, i.e., the training dataset, to train the adjudicator role.

[0070] Optionally, prompts can be used to guide the corrector in generating corrective reasons, explaining how to derive the true conclusion from the debate. The corrector's prompts may include: "Debate record: [debate record content]; core insight: [core insight content]; task: based on the debate context, please provide reasons for the [true conclusion] of this claim. The output format is JSON: {"reasons for conclusion":"..."}.

[0071] exist Figure 3 In the illustrated embodiment, the adjudicator role can be post-trained based on a synthetic training dataset, effectively addressing the unfavorable situation where the verification results are too neutral in the case of zero samples.

[0072] Optionally, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a third model training method provided in an embodiment of this application. The method may further include steps S321-S323.

[0073] Step S321: Obtain historical debate data in multiple different languages.

[0074] In order to enable the verification function of data in multiple languages, historical debate data in multiple languages ​​can be obtained.

[0075] Optionally, different languages ​​may include multiple languages ​​such as Chinese, English, Korean, and Japanese.

[0076] Step S322: Based on preset evidence conditions, classify and filter historical debate data to obtain classified data.

[0077] Step S323: Label each category of data to obtain the historical dataset.

[0078] The evidentiary conditions can include: the golden evidence condition, the retrieval evidence condition, and the no-evidence condition. Based on these pre-defined evidentiary conditions, historical debate data can be categorized and filtered to obtain classified data. Each category of data can then undergo corresponding semantic annotation processing to label the various processes and logic within the data, resulting in a historical dataset with semantically annotated information.

[0079] Optionally, the gold standard condition is evidence used for human fact-checking, with a performance ceiling; the evidence retrieval condition is automatically retrieved evidence simulating real-world scenarios, containing noise; and the no-evidence condition is an ablation experiment to verify the evidentiary value of the debate.

[0080] exist Figure 4 In the described embodiments, historical data in multiple languages ​​can be classified and processed to obtain labeled historical datasets, effectively improving the diversity and authenticity of historical datasets.

[0081] Optionally, please refer to Figure 5 , Figure 5 The following is a detailed flowchart of step S220 provided in an embodiment of this application. Step S220 may include steps S221-S222.

[0082] Step S221: Based on the training dataset, perform forward training on the adjudicator role to obtain an intermediate model.

[0083] In the post-training of the adjudicator role based on the training dataset, considering the different situations of incorrect and correct judgments, in order to achieve comprehensive debate-driven post-training, the adjudicator role can be forward-trained based on the training dataset so that the adjudicator role can learn how to output correct judgments and reasons based on the debate context, thus obtaining the corresponding intermediate model.

[0084] Step S222: Based on the training dataset, perform preference optimization on the intermediate model to obtain the validation model.

[0085] Furthermore, the intermediate model can be optimized based on the training dataset to encourage the model to generate responses consistent with human judgment, while penalizing incorrect responses, thus obtaining the final validation model.

[0086] exist Figure 5 In the illustrated embodiments, the adjudicator role can be trained from different perspectives so that the final verification model can make adjudications and reasoning operations that are closer to those of human experts in complex debate contexts, thereby improving the effectiveness and accuracy of the verification results.

[0087] Optionally, please refer to Figure 6 , Figure 6 The following is a detailed flowchart of step S221 provided in an embodiment of this application. Step S221 may include steps S2211-S2212.

[0088] Step S2211: Determine the correct samples that were correctly judged in the training dataset.

[0089] Step S2212: Based on the correct samples, train the adjudicator role to obtain an intermediate model with the first type of loss.

[0090] In this process, the correct samples that make the correct judgments can be identified from the training dataset first, and then the adjudicator role can be trained in a positive direction based on the correct samples to obtain an intermediate model with the first type of loss.

[0091] Optionally, a debate-driven supervised fine-tuning (SFT) model can be used to first select correct samples from the initial predictions in SynDeC, use the debate records as the context of multi-turn dialogues, train the adjudicator to generate correct reasons that conform to the real verdict, and obtain an intermediate model. The first type of loss of the intermediate model is the SFT loss.

[0092] exist Figure 6 In the illustrated embodiment, the adjudicator can learn to generate the correct logic that conforms to the correct result through positive training with correct samples, effectively improving the verification performance of the intermediate model.

[0093] Optionally, please refer to Figure 7 , Figure 7 The following is a detailed flowchart of step S222 provided in an embodiment of this application. Step S222 may include steps S2221-S2223.

[0094] Step S2221: Identify the erroneous samples in the training dataset that were incorrectly judged.

[0095] Step S2222: Based on the erroneous samples, and in conjunction with the corrector, determine the preference pairs.

[0096] Step S2223: Based on preference pairs, train the intermediate model to obtain a validation model with a second type of loss.

[0097] First, erroneous samples in the training dataset can be identified. Then, erroneous samples and correctors can be combined to generate corresponding preference pairs. Based on these preference pairs, the intermediate model can be further trained to obtain a validation model with a second type of loss.

[0098] Optionally, for the erroneous samples with initial prediction errors in the training dataset, the "true verdict + reason for correction" can be used as the preferred response and the "initial error verdict + initial reason" can be used as the rejection response to construct a preference pair. The final validation model is obtained by training with DPO (Direct Preference Optimization) to maximize the probability of the preferred response and minimize the probability of the rejection response. The second type of loss of the validation model is the DPO loss.

[0099] It should be noted that the preference pairs can include preferred responses and rejection responses. Preferred responses are determined based on the true judgment of erroneous samples and the corrector's reasoning, while rejection responses are determined based on the erroneous judgment and reasoning of erroneous samples. Based on the constructed preference pairs, the validation model can be trained to maximize the probability of generating preferred responses and minimize the probability of generating rejection responses, further reducing the disadvantage of the validation model being overly neutral in zero-shot scenarios.

[0100] exist Figure 7 In the illustrated embodiment, the adjudicator can be trained to make the correct response to different situations through error samples and corrector preference training, which effectively improves the verification performance of the verification model.

[0101] It should be noted that, utilizing the training dataset, this application proposes a debate-driven post-training strategy to ensure that the adjudicator's judgment aligns with that of a human fact-checker in claim verification debates. A typical LLM post-training process can be followed, including supervised fine-tuning (SFT) and reinforcement learning (RL) procedures, and a dedicated post-training strategy is designed for the debate framework. First, based on the accuracy of the predictions made by the basic adjudicator, claims in the SynDeC dataset are categorized into two classes. (Predicting the correct claims, i.e., the correct samples), and (Predicting incorrect claims, i.e., incorrect samples). Then, using... Supervisory fine-tuning (SFT) of the basic adjudication role is implemented to improve its claim verification performance. Specifically, the debate record... Treating it as a multi-turn dialogue context, train the adjudicator role in each claim The final round of the debate generated the correct conclusion. and the reasons The response. This process is called debate-driven SFT, or forward training, which yields an intermediate model, denoted as θ_SFT=argmin_θ. _SFT^correct(θ); where, _SFT stands for SFT, i.e., the first loss, and θ is the adjudicator role. Subsequently, in the reinforcement learning (RL) phase, to correct previous errors by the adjudicator role, the following can be used... Direct preference optimization (DPO) is widely adopted for θ_SFT. Specifically, for Each of them A response that includes both the true conclusions and the rationale for the synthesis ( ) is defined as the preferred response The priority response will include the initial erroneous conclusion and reasoning. Defined as a rejection response Forming preference pairs ( DPO maximizes the optimal response. The probability (this response reflects the probability of human fact-checkers asserting verification arguments) (the reasoning process in the text), while minimizing the rejection response. The probability of this is used to improve the claim verification performance of the final adjudicator in the final verification model. The parameter is: θ_SFTw / DPO=argmin_θ _DPO(θ_SFT); where, _DPO represents the DPO loss, also known as the second type of loss. The debate-driven post-training strategy progressively trains the advanced adjudicator to analyze claims, validate arguments, accurately predict conclusions, and generate arguments based on those claims. It should be noted that although this strategy was designed for claim validation tasks, it can be applied to other multi-agent debate scenarios with minor modifications.

[0102] Optionally, please refer to Figure 8 , Figure 8 This is a flowchart illustrating the fourth model training method provided in this application embodiment. The method may further include steps S331-S332.

[0103] Step S331: Configure the debate rules between the first role and the second role.

[0104] This allows for the configuration of operational parameters for multiple roles within the intelligent debate framework, as well as the configuration of debate rules between the first and second roles with different objectives. Debate rules can include the number of debates between the first and second roles, for example, three debates per debate session.

[0105] Optionally, the debate rules may also include meta-catch words and other data to guide the behavior of each role during the debate. The meta-catch words for the first and second roles of debaters may include: "You are a debater in a fact-checking scenario. Your task is to verify the accuracy of a specific claim based on credible evidence provided, by supporting or refuting it. Your role: Defend your assigned position using credible evidence. Respect the facts, critically analyze and question the opposing side's arguments. Objective: Determine the truthfulness of the claim. Provide persuasive evidence to support your position. Respond to and critique the opposing side's evidence to strengthen your position while remaining concise. Claim: [Claim content]; Evidence set: [Evidence set content]." The meta-catch words for the adjudicator role may include: "You are the moderator of the fact-checking debate." Two debaters will test the veracity of a particular claim by presenting credible evidence to support or refute it. Your goal is to facilitate a fact-based assessment of the claim, ensuring both sides maintain a strong stance supported by sufficient evidence. Responsibilities: Direct each round of debate, ensuring arguments are always based on evidence. Evaluate the relevance and strength of the credible evidence presented by both sides. Determine if more rounds of debate are needed based on new insights provided. Terminate the debate if either side merely repeats previous arguments without bringing substantial new information. Claim: [Claim content]; Evidence set: [Evidence set content]; Conclusion criteria: Supported: The claim is adequately supported by the presented credible evidence. Refuted: The claim directly contradicts the presented credible evidence. Not Enough Evidence: There is insufficient credible evidence to confirm or refute the claim. Conflicting Evidence / Selective Presentation (Evidence / Cherry-picking): The claim is misleading due to conflicting evidence or selective information. After pre-debate configuration, the first role can use the following prompts to initiate the debate: "claim: [claim content]; You need to support the truthfulness of this claim. Use the provided credible evidence to construct a persuasive argument to support the claim. Method: Break down the claim into its core components and clearly explain their meaning. Present the most relevant evidence to support your position, each piece of evidence labeled (evidence content, source link). Critically analyze and refute the opposing viewpoint using credible evidence (optional in the first round)." When refuting, the second role can use the following prompts to present their counter-arguments: "Opponent's Argument (Affirmative): [Affirmative Argument Content]; You need to refute this claim and demonstrate its falsity. Use the provided credible evidence to construct a persuasive argument against the claim. Method: Present the most relevant evidence to support your position, each piece of evidence labeled (evidence content, source link). Critically analyze and refute the opposing viewpoint using credible evidence." After each round of debate, the adjudicator's prompts to monitor the debate process may include: "Fact-checking debate round [round number] has ended."Affirmative Argument: [Content of Affirmative Argument]; Negative Argument: [Content of Negative Argument]; As the moderator, please evaluate both sides' arguments by analyzing the relevance and sufficiency of the presented credible evidence, while keeping it concise. Follow these steps: 1. Summarize the main new insights gained in this round compared to previous rounds. 2. Point out any missing evidence or arguments in either side's argument. 3. Assess whether further debate is necessary, or whether either side is merely repeating previous viewpoints without adding substantial new information. Conclusion: If there is a clear conclusion to support it, or further debate is unnecessary: ​​Provide the reasons for this conclusion; Choose one of the following conclusion labels: "Support," "Refute," "Insufficient Evidence," or "Conflicting Evidence / Selective Presentation"; Set "Need to Continue?" to "No." If further debate is necessary: ​​Explain why an additional round is needed; Set "Need to Continue?" to "Yes"; Provide "Reasons for Continued Debate," outlining the basis for needing further evidence; Leave "Reasons for Conclusion" and "Conclusion" blank. Output your findings in JSON format: {"Core Insights":"...","Evidence Gap":"...","Reasons to Continue the Debate":"...","Need to Continue":"Yes / No","Reasons for Conclusion":"...","Conclusion":"Support / Refute / Insufficient Evidence / Conflicting Evidence / Selective Presentation"}” If the moderator decides to continue the debate, the following interactive prompts can be provided to the first debater: "Opponent's Argument: [Opponent's Argument Content]; Do you agree with this viewpoint? Provide your response, explaining your reasoning process, supporting evidence, and pointing out any shortcomings in the opponent's argument." If the moderator determines that the debate between the two debaters has reached a consensus, the debate is terminated. If the debate lasts for more than three rounds, the following final prompts are provided to the moderator to guide them in outputting their conclusions and generating corresponding reasons: "Affirmative Argument: [Affirmative Argument Content]; Negative Argument: [Negative Argument Content]; Briefly summarize the core insights gained throughout the debate. After reviewing both sides' arguments regarding the claim [claim content]: Based on credible evidence, choose one of the following conclusion labels: "Support", "Refute", "Insufficient Evidence", or "Conflicting Evidence / Selective Presentation". Provide step-by-step reasons for your choice, and then present the conclusion in JSON format: {"Reason for conclusion":"...","Conclusion":"Support / Refute / Insufficient evidence / Conflicting evidence / Selective presentation"}.

[0106] Step S332: Based on the debate rules, determine the termination conditions for the adjudicator role.

[0107] Based on the debate rules, the system analyzes the adjudicator roles and configures termination conditions to stop the debate between the two roles. Termination conditions may include: determining whether the first and / or second roles have presented new arguments; after each round of debate, the adjudicator can summarize key points of contention and new insights, and determine whether the debate has converged (i.e., neither side presents new arguments). If converged, the adjudicator terminates the debate and outputs a final verdict and justification based on the entire debate content; otherwise, the next round begins. To prevent infinite loops, a maximum number of rounds can be set based on the number of debates.

[0108] exist Figure 8 In the illustrated embodiment, the working parameters of multiple different types of roles in the debate framework can be configured accordingly, so that multiple roles in the debate framework can simulate the human debate process and obtain corresponding verification results.

[0109] Optionally, please refer to Figure 9 , Figure 9 This is a flowchart illustrating the fifth model training method provided in the embodiments of this application. The method may further include steps S341-S342.

[0110] Step S341: Determine the accuracy threshold based on usage requirements.

[0111] In order to ensure that the obtained verification model has high verification performance, the accuracy threshold can be determined based on the actual usage requirements of the application scenario in which the verification model is located.

[0112] For example, the accuracy threshold can be determined based on actual usage requirements. When the usage requirement is high accuracy, the accuracy threshold can be set to a higher value, such as 90%. When the usage requirement is medium accuracy, the accuracy threshold can be set to a medium-high value, such as 80%.

[0113] Step S342: If the accuracy of the validation result of the validation model is lower than the accuracy threshold, then the validation model is adjusted.

[0114] Specifically, the accuracy of the validation results generated by the validation model can be compared with the accuracy threshold. If the accuracy of the validation results is lower than the accuracy threshold, it indicates that the accuracy of the validation model is low and the validation performance is poor. The validation model can be adjusted or retrained until the accuracy of the validation results is higher than or equal to the accuracy threshold. Then, it indicates that the validation model can meet the usage requirements of the application scenario and can be put into use.

[0115] exist Figure 9In the illustrated embodiment, the performance of the trained validation model can be validated to ensure that the validation model can meet the usage requirements of the application scenario.

[0116] Optionally, the performance of the validation model can be determined experimentally. Experiments can be uniformly conducted on a 40GB NVIDIA A100 GPU. For all LLMs, the maximum number of generated tokens, the temperature parameter, and the top_p parameter are set to 512, 0.7, and 1.0, respectively. For post-training methods involving training, the Adam optimizer is used with a learning rate of... The training run lasts for 2 epochs. For LoRA, the rank is set to 128 and the alpha value to 256. All hyperparameters are consistent with existing research to ensure fair comparison. Several evaluation questions can be posed: EQ1: In zero-shot claim validation, how does the validation model perform compared to existing single-LLM-based RAG methods? EQ2: In terms of interpretability, how does the validation model perform compared to existing single-LLM-based RAG methods? EQ3: Can a debate-driven post-training strategy based on synthetic debate data improve the performance of LLMs in claim validation debates? The experiments can use the AVReeTeC dataset, currently the largest and most up-to-date real-world claim validation dataset, as the validation set. This dataset contains English text claims from 50 fact-checking bodies, annotated by human fact-checkers. The AVReeTeC training set (containing 3068 claims) is used during the training phase, and the publicly available development set (containing 500 claims) is used during the evaluation phase. It is important to note that the evaluation data is completely invisible during training to avoid potential bias in the evaluation results. The AVeriTeC dataset provides a knowledge base containing approximately 1000 online sources of evidence. Two types of evidence can be selected from this knowledge base to systematically evaluate the performance of the proposed method under different evidence conditions: **Golden Evidence:** This uses golden evidence employed by human fact checkers, representing the upper limit of performance achievable by any method. **Retrieved Evidence:** To simulate a real end-to-end claim verification process, an automated evidence retrieval method that may introduce retrieval noise (such as irrelevant evidence) is employed. Specifically, the best-performing retrieval strategy from the AVeriTeC challenge can be used to simulate the process of retrieving evidence from the open network.

[0117] It should be noted that evidence retrieval is not within the scope of this study. Its impact on experimental results can be controlled by consistently using the same set of evidence in all experiments. Furthermore, to explore the importance of debate-driven evidence analysis in the validation model, an ablation experiment without evidence can be conducted—providing no evidence to the debaters and judges. The widely used AVRIETC score can be used as the primary evaluation metric for claim validation performance, i.e., the accuracy metric. This metric is initially evaluated using the METEOR score. Assess the quality of evidence (including) To compare the collected evidence With real evidence (scoring function), then setting a threshold The AVeriTeC score is used to determine whether sufficient evidence has been collected. It is defined as the predictive accuracy of the asserted conclusion when the evidence score exceeds this threshold. Additionally, standard accuracy is used as a supplementary metric to assess the effectiveness of the method when the collected evidence is deemed insufficient (i.e., the evidence score is below 0.25). It is worth noting that under the gold standard of evidence, accuracy is equal to the AVeriTeC score; under the condition of no evidence, the AVeriTeC score is not applicable.

[0118] It should be noted that, to evaluate the performance of the proposed method, the following baseline models and ablation experiments can be set up, divided into two categories: zero-shot methods and post-training methods. Other implementation details are as follows: Zero-shot methods: Training the LLM without using any data for claim verification. The following two methods are considered in the experiment: RAG: This baseline model inputs claims and evidence into a single LLM to guide its prediction of claim truthfulness. It is the state-of-the-art method for zero-shot claim verification and was adopted by the top three solutions in the AVRiteC challenge. The only differences are in evidence retrieval strategy, hints, and the LLM backbone model. In the experiment, historical hints can be used in conjunction with the above evidence conditions for evaluation. To comprehensively evaluate the performance of different LLMs, the proprietary model GPT-4o (OpenAI, 2024) and the open-source model Llama-3.1-8B-Instruct (Meta, 2023, hereinafter referred to as Llama3.1) can be used as backbone models simultaneously. Due to cost and inference efficiency considerations, the first and second roles use GPT-4o-mini as the backbone model; to ensure a fair comparison with the RAG baseline model, the adjudicator role uses both GPT-4o and Llama-3.1 as backbone models. Post-training method: Supervised data is used to optimize the claim verification performance of the LLM. In the experiments, the post-trained baseline model uses the open-source Llama-3.1 as the LLM backbone model (because GPT-4o is a proprietary model). All LLMs are fine-tuned using LoRA to improve computational efficiency. This category includes the following three baseline methods: RAG+: This baseline model uses standard SFT to post-train a single LLM on the training set for claim verification, representing an existing solution for claim verification. It can directly use open-source checkpoints provided by existing technologies, with other configurations consistent with RAG. DebateCV+: The final verification model generated has the same configuration as the DebateCV baseline model. DebateCV+w / oCorrector: An ablation variant of DebateCV+ that performs debate-driven SFT for the adjudicator role without utilizing the corrector to build reinforcement learning (RL) preference pairs. This ablation experiment can explore the effectiveness of error correction designs.

[0119] Please see Figure 10 , Figure 10 This application provides a schematic diagram of an evaluation result, representing the evaluation results of claim verification of a zero-sample baseline model under different evidence conditions. Figure 10The results show that the proposed debate framework consistently outperforms the RAG baseline model based on a single LLM under all evidentiary conditions. Furthermore, ablation experiments under no-evidence conditions demonstrate a significant performance decrease in the debate framework, indicating that the performance improvement stems from the comprehensive evidence analysis during the debate process. Based on EQ1: The proposed debate framework not only surpasses the performance ceiling (performance proof under the gold evidence condition) but also exhibits strong robustness in the real end-to-end claim verification process with imperfect evidence containing retrieval noise (performance proof under the retrieval evidence condition). This superior performance is attributed to the advantages of the debate-driven approach compared to single-model reasoning. By simulating the deliberate thinking process of a human fact-checker, the reasoning structure of the debate framework ensures that conclusions are based on contextual details from different perspectives, enabling a more comprehensive analysis of the consistency and contradictions between evidence and claims.

[0120] Please see Figure 11 , Figure 11 This illustration, provided for embodiments of this application, represents the performance distribution of correct and incorrect claim predictions across debate rounds in the debate framework and validation model (x-axis represents debate rounds, y-axis represents the number of claims). It compares the performance of the base adjudicator role (a) DebateCV with zero samples in the debate framework and the advanced adjudicator role (b) DebateCV+ in the validation model under evidence retrieval conditions. For DebateCV, the proportion of correct conclusions is high in the first round of debate, but the error rate increases significantly with each round. In contrast, the advanced adjudicator role in DebateCV+ achieves a significantly higher proportion of correct conclusions in all rounds, indicating that the proposed debate-driven post-training strategy effectively improves the reasoning ability of the adjudicator role in complex and contentious debates involving differing viewpoints. Furthermore, the ablation variant DebateCV+ w / oCorrector performs worse than DebateCV+, with accuracy decreasing by more than 3.6% in both metrics. This highlights the crucial role of the corrector in the proposed post-training strategy: the corrector constructs preference pairs, rewarding the adjudicator for generating responses consistent with the human-labeled true labels and corrective rationale, while penalizing the generation of initial incorrect responses. Nevertheless, the baseline model consistently outperforms zero-shot DebateCV. These results validate the effectiveness of the proposed debate data synthesis method and the SynDeC dataset in improving the performance of debate-driven claim verification adjudicators, particularly under evidence retrieval conditions.

[0121] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of a verification model provided in an embodiment of this application. The verification model is trained based on the training method of any of the above embodiments, and the verification model may include: Based on the first role 410 that supports the objective claim; The second role, 420, refutes the target's claims; The adjudicator role 430 is determined based on the verification results output by the first and second roles.

[0122] The verification model utilizes an LLM-based agent to simulate the prudent verification process of human fact-checking. Furthermore, this application proposes a debate data synthesis method and a debate-driven post-training strategy to further optimize the performance of the adjudicator role in claim verification debates. The proposed method significantly improves the accuracy and interpretability of zero-sample and supervised claim verification under different evidence sufficiency conditions, contributing to enhanced digital literacy and the fight against misinformation.

[0123] exist Figure 12 In the illustrated embodiment, the verification model can conduct multiple rounds of debate around a given statement and a set of evidence. First and second roles can present and defend their own arguments and challenge the opposing side's arguments based on the evidence, while the adjudicator role can evaluate each round and make a final ruling, outputting the corresponding verification results.

[0124] Please see Figure 13 , Figure 13 This is a flowchart illustrating a data verification method provided in an embodiment of this application. The method may include steps S510-S520.

[0125] Step S510: Obtain the target data to be verified; Step S520: The first and second roles in the verification model conduct debate based on the target data, and the adjudicator role in the verification model makes a judgment to obtain the target verification result.

[0126] The verification model is obtained based on the model training method of any one of the above embodiments.

[0127] exist Figure 13 In the illustrated embodiment, in practical application scenarios, the target data to be verified can be obtained first as an evidence set, and then input into the verification model for processing. The verification model can conduct multiple rounds of debate around the given target data. The first and second roles can propose and defend their own arguments and challenge the opponent's arguments based on the target data. The adjudicator role can evaluate each round and make a final judgment, outputting the corresponding target verification result.

[0128] This application also provides a computer program product, which includes a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the steps of any of the model training methods or data verification methods described above.

[0129] It should be noted that the solution provided in this application is primarily applied to claim verification and does not involve the evidence retrieval stage. However, considering that evidentiary conditions have a significant impact on performance—the method using "golden evidence" is superior to methods relying on evidence retrieved from real-world sources—evidence retrieval and claim verification can be jointly optimized through debate-driven evidence analysis.

[0130] In summary, this application proposes the first debate-driven claim verification framework, DebateCV, which draws on the real-world verification practices of human fact-checkers. This framework overcomes the inference limitations of a single LLM (Legal Management Model) and enables in-depth and detailed analysis of contradictions or consistency between retrieved evidence and claims, yielding more accurate and interpretable results compared to existing solutions. A novel debate-driven post-training strategy utilizing synthetic debate data is introduced. This strategy addresses the scarcity of human debate data in claim verification, ensuring that the adjudicator role in the DebateCV framework aligns with the behavior of human fact-checkers. Experimental results demonstrate that, in scenarios with varying levels of evidence quality, the verification model provided in this application outperforms existing claim verification methods in both zero-sample scenarios (accuracy improvement ≥5.2%) and supervised scenarios (14.8% improvement compared to zero-sample scenarios and ≥1.8% improvement compared to non-debate post-training methods).

[0131] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0132] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0133] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0135] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0136] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A model training method, characterized in that, The method includes: In the debate framework, an adjudicating role is constructed based on the first role supporting the target claim and the second role refuting the target claim, and the adjudicating role is constructed based on the verification results output by the first role and the second role. Based on the training dataset, the adjudicator role is post-trained to obtain a validation model.

2. The method according to claim 1, characterized in that, in, The methods for determining the training dataset include: The zero-shot-based debate framework processes a historical dataset with annotation information to obtain initial data; wherein, the initial data includes multiple rounds of debate records of the first role and the second role, as well as the judgment rules of the adjudicator role; Based on the historical dataset, identify the samples of the adjudicator role that were incorrectly determined; The initial data is optimized using a corrector and the samples to be corrected to obtain the training dataset.

3. The method according to claim 2, characterized in that, in, The methods for determining the historical dataset include: Acquire historical debate data in multiple languages; Based on preset evidence conditions, the historical debate data is classified and filtered to obtain classified data; wherein, the evidence conditions include: golden evidence conditions, searchable evidence conditions, and no evidence conditions; Each of the categorized data points is labeled to obtain the historical dataset.

4. The method according to claim 1, characterized in that, The step of post-training the adjudicator role based on the training dataset to obtain a validation model includes: Based on the training dataset, the adjudicator role is trained in a forward manner to obtain an intermediate model; Based on the training dataset, the intermediate model is optimized for preferences to obtain the validation model.

5. The method according to claim 4, characterized in that, The step of performing forward training on the adjudicator role based on the training dataset to obtain an intermediate model includes: Identify the correct samples in the training dataset that were correctly judged; Based on the correct samples, the adjudicator role is trained to obtain the intermediate model with a first type of loss.

6. The method according to claim 4, characterized in that, The step of optimizing the intermediate model based on the training dataset to obtain the validation model includes: Identify the erroneous samples in the training dataset that were incorrectly identified; Based on the erroneous samples, and in conjunction with the corrector, preference pairs are determined; Based on the preference pair, the intermediate model is trained to obtain the validation model with the second type of loss.

7. The method according to claim 6, characterized in that, in, The preference pairs include: preferred response and rejected response; The priority response is determined based on the true determination of the erroneous sample and the correction rationale of the corrector; The rejection response is determined based on the error judgment and error reason of the error sample.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Configure the debate rules between the first role and the second role; wherein, the debate rules include: the number of debates between the first role and the second role; Based on the debate rules, the termination conditions for the adjudicator role are determined; wherein, the termination conditions include: determining whether the first role and / or the second role outputs a new argument.

9. The method according to any one of claims 1-7, characterized in that, The method further includes: Determine the accuracy threshold based on usage requirements; If the accuracy of the verification result of the verification model is lower than the accuracy threshold, the verification model shall be adjusted.

10. A verification model, characterized in that, The verification model is trained based on the training method of any one of claims 1-9, and the verification model includes: Based on the primary role of supporting the target claims; A second role in refuting the stated objective claims; The adjudicating role is determined based on the verification results output by the first role and the second role.

11. A data verification method, characterized in that, The method includes: Obtain the target data to be verified; The first and second roles in the verification model perform debate processing based on the target data, and the adjudicator role in the verification model makes a judgment to obtain the target verification result; wherein, the verification model is obtained based on the model training method of any one of claims 1-9.

12. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the steps of the method according to any one of claims 1-9.

Images