Automatic fact evidence obtaining method based on self-organization and Lag-DPO and electronic equipment

Abstract

The invention discloses an automatic fact evidence obtaining method based on self-organization and Lag-DPO and electronic equipment. The automatic fact evidence obtaining method comprises the following steps: S1, two-dimensional data enhancement; s2, constructing a preference data pair; s3, carrying out Lag-DPO preference alignment training; and S4, factual evidence obtaining and prediction. According to the automatic factual evidence obtaining method, the interpretability automatic factual evidence obtaining method combining self-inductance and Lag-DPO is achieved, the self-inductance is used for generating a preference data set, and the Lag-DPO is used for fine adjustment of the Lag-DPO. The method has the advantage of high accuracy of a traditional fine tuning method and the advantage of high interpretability of a large language model. In addition, the experiment shows that the method has certain generalization.

Description

Technical Field

[0001] The invention belongs to the field of software, and in particular relates to an automated fact-collecting method based on self-instruction and Lag-DPO. Background Art

[0002] Fact-finding is an important and widely used natural language processing task. It aims to determine the truth of a statement given relevant facts. A high-quality fact-finding system is crucial to the healthy development of society. It can: 1. reduce the spread of false information and increase public trust in information; 2. help media practitioners identify the authenticity of reported content; 3. provide reliable information decision-making support for government departments and enterprises; 4. cultivate the public's critical thinking and enhance media literacy.

[0003] Automatic fact-finding refers to the use of artificial intelligence and other technologies to complete automated fact-finding, thereby saving a lot of manpower and material resources. It is an important frontier in the field of natural language processing and is also related to the public's right to know and information security. Current automatic fact-finding methods can be roughly divided into two categories. One is based on pre-trained models, which treat the fact-finding problem as a multi-classification task. This method is low-cost and high-accuracy, but lacks interpretability and generalization capabilities. The other is based on large language model prompt engineering, which is an emerging research direction. This method has strong interpretability, but relatively low accuracy and high cost.

[0004] Glossary:

[0005] Self-instruction: is a method to align pre-trained language models with instructions. The model can generate data by itself without the need for a lot of manual annotation.

[0006] Lag-DPO: An improved version of DPO that uses Lagrangian constraints.

[0007] Zero-shot: refers to the model completing the task through its own generalization ability without any training or learning samples for a specific task.

[0008] Prompt: A text input used to guide or direct a large language model to perform a specific task. With a well-designed prompt, we can make the model produce the expected output. Prompts can include task descriptions, background information, example input / output, etc. Prompt engineering is an important area of ​​using prompts to optimize and control the output of large language models. A well-designed prompt can make the model produce high-quality, relevant output.

[0009] DPO: is a preference alignment algorithm that directly optimizes the language model, avoiding the reward model construction and reinforcement learning steps in traditional RLHF to better meet human preferences. DPO simplifies the preference learning process and improves efficiency and accuracy Summary of the invention

[0010] In order to solve the above technical problems, the technical solution of the present invention is:

[0011] An automated fact-collecting method based on self-instruction and Lag-DPO includes the following steps:

[0012] S1. Two-dimensional data enhancement: construct a training set data set, and then perform data enhancement on the training set data set to obtain the enhanced training data set D;

[0013] S2. Construction of preference data pairs: Use a large language model to test the difficulty of each piece of data in the enhanced training dataset in a zero-shot manner, determine the data sampling rate, and then sample the preference data from the enhanced training dataset according to the determined data sampling rate;

[0014] S3.Lag-DPO preference alignment training: Input the sampling preference data of the enhanced training dataset into Lag-DPO to fine-tune the large language model, regard the explainable fact-finding task as preference alignment training, and use the improved direct preference alignment algorithm - Lag-DPO to fine-tune the large language model to obtain the fine-tuned large language model;

[0015] S4. Factual forensic prediction: Input the statement and related evidence into the fine-tuned large language model, which generates the true label of the statement and uses the reasoning process of judging the authenticity of the statement as the explainable content.

[0016] For further improvement, the data enhancement method in S1 is as follows:

[0017] S1_1. Counterfactual data enhancement: By modifying the statements or given evidence of the original data in the training set, counterfactual data with similar semantics and opposite labels are generated to form synthetic data, and the original data and synthetic data are merged to increase the size of the data set;

[0018] S1_2. Self-guided explainable content generation. Given different labels, h explanatory contents are generated as samples according to the given evidence, statement, and label information of the original data. If the label is correct, the generated content is a positive sample, otherwise it is a negative sample.

[0019] For further improvement, the specific method of step S1_2 is as follows:

[0020] a. If the original data in the training set is a given evidence-claim pair, multiple data are generated in a zero-shot manner, where each data is in the form of a label-explanation pair, as shown in formula (1), where π() represents the large language model, c represents the claim, E represents the evidence, I() represents the designed prompt; e represents the explanatory content under the given label y;

[0021] b. If the original data in the training set is a given evidence-claim-label triple, h explanatory contents are generated as samples in a self-instructed manner. If the label is correct, the generated content is a positive sample, otherwise it is a negative sample, as shown in formulas (2) and (3); where y no represents a negative label, that is, there is no contradiction between the statement and evidence, y yes Indicates a positive label, that is, there is a contradiction between the claim and evidence, e no Indicates that y is given no The explanatory content generated by the model under the premise of yes Indicates that y is given yes The explanatory content generated by the model under the premise of , P() represents the probability distribution;

[0022] c. If the data is counterfactual data generated by S1_1, the data will contain anti-editing information. Then, given the evidence-claim-positive label-editing information quadruple, generate positive samples containing more accurate information, as shown in formula (4), where tips represents editing information; e tips Represents the explanatory content generated by the model given the correct label y and editing information tips;

[0023] P(y,e|c,E)=π(y,e|I(c,E))#(1)

[0024] P(e no |c,E,y no )=π(e no |I(c,E,y no ))#(2)

[0025] P(e yes |c,E,y yes )=π(e no |I(c,E,y yes ))#(3)

[0026] P(e tips |c, E, y, tips) = π(e tips |I(c,E,tips,y))#(4).

[0027] For further improvement, the specific steps of step S2 are as follows:

[0028] S2_1. Data difficulty evaluation: Use a large language model to perform zero-shot evaluation on the enhanced training dataset, repeat k times, statistically calculate the correct results n, and determine the quantization difficulty d of the data, as shown in formula (5):

[0029]

[0030] S2_2. Positive and negative sample sampling: According to formula (6), determine the number of sampling times t of the enhanced training data set, where base is the number of basic sampling times and max represents the operation of taking the maximum value from two numbers. Finally, randomly sample t times from positive and negative samples, randomly combine positive and negative samples in pairs, and match their corresponding labels to generate t preference data pairs:

[0031] t=2*max(base,d*h)#(6)

[0032] h represents the number of positive samples or negative samples;

[0033] S2_3. Preference data set construction. For each data entry of the enhanced training data set D, the operation of step S2_2 is performed to obtain data preference data pairs of size |D|*h. All data pairs are shuffled and stored to obtain the preference data set; where |D| represents the size of the enhanced training data set D.

[0034] As a further improvement, in step S3, the specific method of using the improved direct preference alignment algorithm, Lag-DPO, to fine-tune the large language model is as follows:

[0035] S3_1. Construct the loss function of DPO, the loss function of DPO L DPO As shown in formula (7), where π ref () represents the original model, π θ () represents the fine-tuning model, y chosen is a positive example, y rejected is a negative example, x is the input statement-evidence pair, β is a hyperparameter, and σ represents the sigmoid function;

[0036]

[0037] Reference two constraints C chosen and C rejected The loss function of constrained DPO is shown in formula (8) and formula (9), where a1 and a2 are both hyperparameters;

[0038]

[0039] During the training process, formula (10) must be satisfied:

[0040] argmin D L DPO

[0041] C chosen ≤0#(10)

[0042] C rejected ≤0

[0043] argmin θ Represents solving the model parameter θ so that the loss function L DPO Reach a minimum value;

[0044] Introducing μ1 and μ2 as Lagrange multipliers, using the Lagrange method to solve the above problem, formulas (7)-(10) are transformed into an unconditional optimization problem, as shown in formula (11).

[0045] Loss = L DPO +logσ[β(μ1C chosen +μ2C rejected )]#(11)

[0046] S3_2. Preference alignment fine-tuning based on preference dataset: The interpretable fact-finding task is regarded as preference alignment training, where the positive examples are the correct label-explanation pairs y chosen , negative examples are wrong labels-explanations of y rejected , the input evidence-statement pair is x; for the loss function Loss, the gradient descent method is used to optimize until the loss function Loss converges to obtain the fine-tuned large language model. For μ1 and μ2, the formulas (12) and (13) are used for optimization, where lr μ is the learning rate of the Lagrange multiplier; max means taking the maximum value;

[0047] μ1=max(μ1-lr μ *C chosen ,0)#(12)

[0048] μ2=max(μ2-lr μ *C rejected ,0)#(13).

[0049] A further improvement is that the large language model is Llama2-7B.

[0050] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any of the above methods.

[0051] Advantages of the present invention:

[0052] The present invention realizes an interpretable automated fact-collection method combining self-instruction and Lag-DPO, wherein the former is used to generate a preference dataset and the latter is used to fine-tune Lag-DPO. The invention combines the advantages of traditional fine-tuning methods: high accuracy, and the advantages of large language models: high interpretability. In addition, our experiments show that the method has a certain degree of generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is an overall flow chart of the automated fact-collecting method based on self-instruction and Lag-DPO.

[0054] Figure 2 It is a two-dimensional data enhancement model diagram based on self-instruction and counterfactual data enhancement.

[0055] Figure 3 It is a flowchart of the preference dataset construction.

[0056] Figure 4 This is an example of Prompt's design and large language models generating labels and interpretable content. DETAILED DESCRIPTION

[0057] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0058] Example

[0059] The present invention implements an automated fact-collecting method combining self-instruction and Lag-DPO, which is used to construct a preference data set and fine-tune the preference of the model, and can improve the performance of a small-scale large language model in fact reasoning ability. The specific implementation method is as follows:

[0060] S1. Two-dimensional data enhancement, which enhances the dataset size and dataset label features from the label side and the data side. This step can effectively expand the size of the dataset, and the generated interpretable content will be used to enhance the model's reasoning ability, rather than fitting labels like traditional binary classification networks, which ensures the generalization and interpretation performance of the model. Specifically, it is divided into the following steps:

[0061] S1_1. Counterfactual data enhancement: Using traditional counterfactual data enhancement methods, by modifying the statement items or fact items in the data, generate semantically similar, oppositely labeled, highly confusing counterfactual data, and merge the original data with the synthetic data. This step is implemented using the existing research method RACE.

[0062] S1_2. Self-guided generation of interpretable content. Given different labels, the model is required to generate interpretable content according to the following settings, such as Figure 2 As shown:

[0063] a. If the original data in the training set is a given evidence-claim pair, multiple data are generated in a zero-shot manner, where each data is in the form of a label-explanation pair, as shown in formula (1), where π() represents the large language model, c represents the claim, E represents the evidence, I() represents the designed prompt; e represents the explanatory content under the given label y;

[0064] b. If the original data in the training set is a given evidence-claim-label triple, h explanatory contents are generated as samples in a self-instructed manner. If the label is correct, the generated content is a positive sample, otherwise it is a negative sample, as shown in formulas (2) and (3); where y no represents a negative label, that is, there is no contradiction between the statement and evidence, y yes Indicates a positive label, that is, there is a contradiction between the claim and evidence, e no Indicates that y is given no The explanatory content generated by the model under the premise of yes Indicates that y is given yes The explanatory content generated by the model under the premise of , P() represents the probability distribution;

[0065] c. If the data is counterfactual data generated by S1_1, the data will contain anti-editing information. Then, given the evidence-claim-positive label-editing information quadruple, generate positive samples containing more accurate information, as shown in formula (4), where tips represents editing information; e tips Represents the explanatory content generated by the model given the correct label y and editing information tips;

[0066] P(y,e|c,E)=π(y,e|I(c,E))#(1)

[0067] P(e no |c,E,y no )=π(e no |I(c,E,y no ))#(2)

[0068] P(e yes |c,E,y yes )=π(e no |I(c,E,y yes ))#(3)

[0069] P(e tips|c, E, y, tips) = π(e tips |I(c,E,tips,y))#(4).

[0070] S2. Preference data pair construction. This step first evaluates the performance of the large language model zero-shot, determines the data sampling rate, and then samples the preference data from the data generated by S1. The overall process is as follows: Figure 3 As shown, the following are the detailed sub-steps.

[0071] S2_1. Data difficulty assessment: Use a large language model to perform zero-shot assessment on the data, repeat k times, calculate the correct results n statistically, and determine the quantization difficulty d of the data, as shown in formula (5). The data evaluated in this step can be obtained from S1_2.a.

[0072]

[0073] S2_2. Positive and negative sample sampling. For data entry D, after executing step S1, we generate h positive and negative samples respectively. Then, we determine the number of samplings t according to formula (6), where base is the number of basic samplings and max represents the operation of taking the maximum value from two numbers. Finally, we randomly sample t times from the positive and negative samples, randomly combine the positive and negative samples in pairs, and match their corresponding labels to generate t preference data pairs.

[0074] t=2*max(base,d*h)#(6)

[0075] S2_3. Construction of preference data set. For each data entry in data set D, the operation of S2_2 is performed to obtain data preference data pairs of O(|D|*h) scale. All data pairs are shuffled and stored to obtain a large-scale preference data set.

[0076] S3.Lag-DPO preference alignment training, regard the explainable fact-finding task as preference alignment training, and use the improved direct preference alignment algorithm - Lag-DPO to fine-tune the large language model. The following are the detailed sub-steps.

[0077] S3_1. Reference two constraints: C chosen and C rejected Improved DPO loss function L DPO The traditional DPO algorithm achieves low-cost preference alignment by directly increasing and decreasing the probability of generating positive and negative samples in the training dataset. The loss function of DPO is shown in formula (7), where π ref is the original model, π θ is the fine-tuned model, y chosen is a positive example, y rejectedis a negative example, x is the input statement-evidence pair, and β is a hyperparameter. However, due to the defects in the loss function, DPO will cause the probability of generating positive and negative examples in the test data to decrease simultaneously.

[0078]

[0079] To alleviate the above problems, we introduce two constraints C chosen and C rejected To improve DPO, as shown in formula (8) and formula (9), a * is a hyperparameter.

[0080]

[0081] During the training process, two constraints must satisfy formula (10):

[0082] argmin θ L DPO

[0083] C chosen ≤0#(10)

[0084] C rejected ≤0

[0085] Therefore, we introduce μ1 and μ2 as Lagrange multipliers and use the Lagrange method to solve the above problem, transforming the above conditional optimization problem into an unconditional optimization problem, as shown in formula (11).

[0086] Loss = L DPO +logσ[β(μ1C chosen +μ2C rejected )]#(11)

[0087] For the loss function Loss, we use the gradient descent method to optimize, and for μ1 and μ2, we use formula (12) and formula (13) to optimize, where lr μ is the learning rate of the Lagrange multiplier.

[0088] μ1=max(μ1-lr μ *C chosen ,0)#(12)

[0089] μ2=max(μ2-lr μ *C rejected ,0)#(13)

[0090] S3_2. Based on the preference dataset, the preference alignment fine-tuning is performed, and the explainable fact-finding task is regarded as preference alignment training, where the positive examples are the correct label-explanation pairs y chosen , and the negative sample is the opposite, which is yrejected , the input evidence-claim pair is x. We use gradient descent to fine-tune the model, with the goal of increasing the probability of generating a positive example π θ (y chosen |x), reducing the probability of generating negative samples π θ (y rejected |x).

[0091] S4. Factual evidence prediction, after fine-tuning the large language model, you only need to fill in the statement and evidence into the designed prompt words and input them to the model. The model will output labels and explanatory content at the same time, such as Figure 4 shown.

[0092] Training and testing datasets. We use three datasets, FEVEROUS-S, HOVER, and FEVER to evaluate the performance of our model. FEVEROUS-S is a fact-finding dataset proposed by Yang et al. that contains various challenges such as numerical reasoning and entity disambiguation. HOVER is a more challenging multi-hop fact-finding dataset proposed by Yang et al. with a maximum number of hops of 4 and higher difficulty. FEVER was proposed by Thorne et al. Although FEVER and FEVEROUS-S have some similarities, neither of them is as difficult as HOVER. We train and test models on FEVEROUS-S and HOVER, and additionally use FEVER to evaluate the generalization performance of our method.

[0093] Deployment tools and parameter configuration. In the specific implementation process, we used the following tool libraries to implement our method: (1) pytorch deep learning framework is used to build the model network; (2) huggingface's model fine-tuning framework trl implements model preference fine-tuning and process monitoring; (3) accelerate library implements half-precision efficient fine-tuning; (4) Lora in the PEFT library implements efficient parameter fine-tuning of large language models; (5) DeepSpeed ​​library is used for video memory optimization management, unloading some parameters of the optimizer to the memory to relieve video memory pressure.

[0094] In the S1. 2D data augmentation stage, we set the maximum number of tokens to 512 to ensure diversity and set k = h = 10. In the S2. Preference data pair construction stage, we set base to 3. The above settings enable us to sample 148, 232 and 221,373 preference data pairs on FEVEROUS-S and HOVER, respectively. In the S3. Lag-DPO preference alignment training stage, we fine-tune the half-precision open source large language model Llama2-7b, using 2 Nvidia RTX 4090 GPUs to fine-tune the half-precision model, setting the learning rate to 1e-4, the warm-up to 200, the learning decay rate to 0.999, β to 0.1, and a1 and a2 to 0.1. We set the batch size to 20, the iteration to 30,000 times, and the early stopping strategy. In order to implement Lag-DPO, we modified the Adam Optimizer and added the necessary hook functions to update μ1 and μ2.

[0095] Performance comparison settings. We used the following two types of data for evaluation:

[0096] Traditional binary classification prediction: These methods only provide binary classification results based on the input data to determine whether there is a contradiction between the claim and the evidence. We fine-tune these methods using the full dataset including a subset of counterfactual data to evaluate the advantages of our methods over traditional methods. We selected the following four methods: Bert-FC and T5 use pre-trained models of different sizes to complete the binary factual forensics classification task. RoBERTa-NLI and Muser are fine-tuned on other textual entailment tasks. Zero-shot prediction based on large language models: These methods use large language models to generate prediction results and related explanations, including commercial large language models that are difficult to fine-tune due to their large number of parameters. To solve this problem, we adopted a few-shot learning method, selecting 10 training data examples and using their labels and explanations as prompts. We selected the following methods for evaluation: FLAN-T5, Llama2, ChatGPT, and ProgramFC, which converts the reasoning process into a program.

[0097] Performance evaluation. We use accuracy and macro-f1 to evaluate the performance of the method and evaluate the generalization and accuracy of the model.

[0098] Our proposed method performs better than traditional methods in generalization performance, as shown in Table 1. Table 1 shows the performance of various methods on other datasets after fine-tuning on FEVROUS-S or HOVER (gray font indicates that the model is fine-tuned on the corresponding dataset). On the FEVEROUS dataset, our model achieved 89% accuracy and 88.83% f1 (fine-tuned on the HOVER dataset). On the HOVER dataset, our model achieved 73.30% accuracy and 73.39% f1. On the FEVER dataset, our model achieved 87.22% accuracy and 87.21% f1 (fine-tuned on the FEVEROUS dataset). We believe that these excellent results are due to the DPO method we adopted. Compared with traditional full-scale fine-tuning, DPO only fine-tunes a few parameters (only 0.99%), thereby maintaining the generation and reasoning capabilities of the pre-trained model. Joint modeling labels and explanations also enable the model to improve reasoning capabilities and generalize better to other datasets.

[0099] Table 1 Generalization performance evaluation

[0100]

[0101] Regarding the comparative experiments, we used fewer training parameters and achieved performance comparable to or even better than traditional SFT, as shown in Tables 2 and 3. Table 2 shows the accuracy and f1 performance of FEVEROUS-S. In addition to showing the overall result (Total), we also show the three subcategories of mathematical reasoning (NR), entity confusion (NR) and multi-hop reasoning (MhR) in the dataset. Table 2 shows the accuracy and f1 performance of HOVER. We show the two-hop (2_HOP), three-hop (3_HOP), four-hop (4_HOP) and overall results (Total) in the dataset.

[0102] The trainable parameters of our method are one order of magnitude less than those of traditional SFT methods (only 61.5% of Bert-FC and 18.9% of Ro-NLI). Our method has achieved SOTA in most cases, with only a 1.08% and 1.64% drop in accuracy compared to DeBERTa-NLI in the Multi-hop Reasoning task. On the HOVER dataset, our performance did not surpass RoBERTa-NLI overall, but was only close to the level of such methods.

[0103] We analyzed the experimental results and summarized three reasons: 1) We used a larger learning rate (1e-4) for fine-tuning. 2) For complex multi-hop reasoning, the traditional SFT method may learn the bias in the dataset and take shortcuts instead of learning the true reasoning logic. In addition, the 4_HOP data length in HOVER exceeds the maximum processing length of the SFT method, causing the model to be more biased towards data bias. 3) The solution we proposed requires the model to take into account both labels and interpretability, which has a certain compromise on accuracy.

[0104] Table 2 FEVEROUS-S comparative experimental performance

[0105]

[0106] Table 3 HOVER comparison experimental performance

[0107]

[0108] Regarding the interpretability analysis, we used specific cases to evaluate the interpretability of the model. Our method enables the large language model to accurately capture the contradictions in the statement. In the example in Table 4, the model needs to find the corresponding information from three pieces of evidence to refute the statement based on the given statement, which requires filtering and sorting the information, and performing at least three hops of reasoning in multiple different information to get the correct answer. From the final conclusion of the generated content, our method correctly points out the main contradiction of the statement, that is, organization-1 (a political group) does not contain Boris Souvarine. The reasoning process of the model successively presents the information required to complete the reasoning and gives the view that the statement and evidence are contradictory. In contrast, Chatgpt-3.5 does not filter a large amount of interference information and generates content that is irrelevant to the contradiction (for example, the party to which Pierre-Josep belongs). Secondly, although the model digs out important information about the founder of organization-2, it only retains the correct part of the statement and ignores the wrong part (Person-4 is not the founder of organization-1). The fine-tuning method we proposed can accurately point out the contradictions between the statement and the evidence.

[0109] Table 4 Interpretability analysis

[0110]

[0111] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the embodiments. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. An automated fact-collecting method based on self-instruction and Lag-DPO, characterized in that: The steps include: S1. Two-dimensional data enhancement: construct a training set data set, and then perform data enhancement on the training set data set to obtain the enhanced training data set D; S2. Construction of preference data pairs: Use a large language model to test the difficulty of each piece of data in the enhanced training dataset in a zero-shot manner, determine the data sampling rate, and then sample the preference data from the enhanced training dataset according to the determined data sampling rate; S3.Lag-DPO preference alignment training: Input the sampling preference data of the enhanced training dataset into Lag-DPO to fine-tune the large language model, regard the explainable fact-finding task as preference alignment training, and use the improved direct preference alignment algorithm - Lag-DPO to fine-tune the large language model to obtain the fine-tuned large language model; S4. Factual forensic prediction: Input the statement and related evidence into the fine-tuned large language model, which generates the true label of the statement and uses the reasoning process of judging the authenticity of the statement as the explainable content.

2. The automated fact-collecting method based on self-instruction and Lag-DPO as claimed in claim 1, characterized in that: The data enhancement method in S1 is as follows: S1_1. Counterfactual data enhancement: By modifying the statements or given evidence of the original data in the training set, counterfactual data with similar semantics and opposite labels are generated to form synthetic data, and the original data and synthetic data are merged to increase the size of the data set; S1_2. Self-guided explainable content generation. Given different labels, h explanatory contents are generated as samples according to the given evidence, statement, and label information of the original data. If the label is correct, the generated content is a positive sample, otherwise it is a negative sample.

3. The automated fact-collecting method based on self-instruction and Lag-DPO as claimed in claim 2, characterized in that: The specific method of step S1_2 is as follows: a. If the original data in the training set is a given evidence-claim pair, multiple data are generated in a zero-shot manner, where each data is in the form of a label-explanation pair, as shown in formula (1), where π() represents the large language model, c represents the claim, E represents the evidence, I() represents the designed prompt; e represents the explanatory content under the given label y; b. If the original data in the training set is a given evidence-claim-label triple, h explanatory contents are generated as samples in a self-instructed manner. If the label is correct, the generated content is a positive sample, otherwise it is a negative sample, as shown in formulas (2) and (3); where y no represents a negative label, that is, there is no contradiction between the statement and evidence, y yes Indicates a positive label, that is, there is a contradiction between the claim and evidence, e no Indicates that y is given no The explanatory content generated by the model under the premise of yes Indicates that y is given yes The explanatory content generated by the model under the premise of , P() represents the probability distribution; c. If the data is counterfactual data generated by S1_1, the data will contain anti-editing information. Then, given the evidence-claim-positive label-editing information quadruple, generate positive samples containing more accurate information, as shown in formula (4), where tips represents editing information; e tips Represents the explanatory content generated by the model given the correct label y and editing information tips; P(y,e|c,E)=π(y,e|I(c,E))#(1) Instead no |c,E,y no )=π(e no |I(c,e,y no ))#(2) Instead yes |c,E,y yes )=π(e no |I(c,e,y yes ))#(3) P(e tips |c,E,y,tips)=π(e tips |I(c,E,tips,y))#(4)。 4. The automated fact-collecting method based on self-instruction and Lag-DPO as claimed in claim 1, characterized in that: The specific steps of step S2 are as follows: S2_1. Data difficulty evaluation: Use a large language model to perform zero-shot evaluation on the enhanced training data set, repeat k times, statistically calculate the correct results n, and determine the quantization difficulty d of the data, as shown in formula (5: S2_2. Positive and negative sample sampling: According to formula (6), the number of sampling times t of the enhanced training data set is determined, where base is the number of basic sampling times and max represents the operation of taking the maximum value from two numbers; finally, randomly sample t times from positive and negative samples, randomly combine positive and negative samples in pairs, and match their corresponding labels to generate t preference data pairs: t=2*max(base,d*h)#(6) h represents the number of positive samples or negative samples; S2_3. Preference data set construction. For each data entry of the enhanced training data set D, the operation of step S2_2 is performed to obtain data preference data pairs of size |D|*h. All data pairs are shuffled and stored to obtain the preference data set; where |D| represents the size of the enhanced training data set D.

5. The automated fact-collecting method based on self-instruction and Lag-DPO as claimed in claim 2, characterized in that: In step S3, the specific method of using the improved direct preference alignment algorithm, Lag-DPO, to fine-tune the large language model is as follows: S3_1. Construct the loss function of DPO, the loss function of DPO L DPO As shown in formula (7), where π ref () represents the original model, π θ () represents the fine-tuning model, y chosen is a positive example, y rejected is a negative example, x is the input statement-evidence pair, β is a hyperparameter, and σ represents the sigmoid function; Reference two constraints C chosen and C rejected The loss function of constrained DPO is shown in formula (8) and formula 9), where a1 and a2 are both hyperparameters; During the training process, formula (10) must be satisfied: argmin θ L DPO C chosen ≤0#(10) C rejected ≤0 argmin θ Represents solving the model parameter θ so that the loss function L DPO Reach a minimum value; Introducing μ1 and μ2 as Lagrange multipliers, using the Lagrange method to solve the above problem, formulas (7)-(10) are transformed into unconditional optimization problems, as shown in formula (11): Loss=L DPO +logσ[β(μ1C chosen +μ2C rejected )]#(11) S3_2. Preference alignment fine-tuning based on preference dataset: The interpretable fact-finding task is regarded as preference alignment training, where the positive examples are the correct label-explanation pairs y chosen , negative examples are wrong label-explanation pairs y rejected , the input evidence-statement pair is x; for the loss function Loss, the gradient descent method is used to optimize until the loss function Loss converges to obtain the fine-tuned large language model. For μ1 and μ2, the formulas (12) and (13) are used for optimization, where lr μ is the learning rate of the Lagrange multiplier; max means taking the maximum value; μ1=max(μ1-lr μ *C chosen ,0)#(12) μ2=max(μ2-lr μ *C rejected ,0)#(13)。 6. The automated fact-collecting method based on self-instruction and Lag-DPO as claimed in claim 1, characterized in that: The large language model is Llama2-7B.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.

Images