Large model knowledge distillation method based on counterfactual data augmentation and multi-perspective thinking chain

Abstract

This invention discloses a large-model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains. First, this invention utilizes topic terms and parse trees to obtain key semantic features of text, and then automatically generates high-quality counterfactual data using a large language model. Next, it leverages multi-perspective thought chains to enhance the diversity of the counterfactual data, thereby improving the reasoning ability of the small language model. Compared with existing technologies, the large-model counterfactual distillation framework proposed in this invention can enhance the robustness and generalization ability of the small model's reasoning, achieving excellent performance on small language models with different architectures and datasets with different distributions.

Description

Technical Field

[0001] This invention belongs to the field of large-scale model knowledge distillation, specifically involving how to efficiently transfer the reasoning ability of large language models to small language models. Background Technology

[0002] Knowledge distillation of large models is the process of transferring the knowledge and reasoning capabilities of large models to smaller models. It is significant in promoting the popularization of artificial intelligence and optimizing resources. First, through distillation, smaller models can retain the core knowledge and complex reasoning capabilities of larger models, enabling high-performance intelligent applications even in environments with limited computing resources. This not only reduces the computational and storage costs of the models but also allows AI applications to be deployed more widely to edge devices and mobile terminals, improving the versatility of models in real-world scenarios.

[0003] However, in existing technologies, efficiently transferring the knowledge and reasoning capabilities of large models to small models presents certain challenges. Currently, the performance of small models after knowledge distillation often falls short of expectations, primarily due to the nature of the knowledge distillation process. Therefore, there is an urgent need to provide a more effective method for distilling the capabilities of large models into small models. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing large model knowledge distillation techniques and to provide a large model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains.

[0005] The specific technical solution adopted in this invention is as follows:

[0006] In a first aspect, the present invention provides a large-scale model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains, which includes the following steps:

[0007] S1: Input the question-and-answer text consisting of questions, options, and answers into the large language model, and combine the large language model and syntactic analysis to identify phrases related to causal features from the question text;

[0008] S2: Replace phrases related to causal features in the question text with masking characters, and modify the answer as the task label. Then, drive the large language model with example prompts to complete all masking characters in the question text based on the modified answer, thereby generating a new question and answer text that is highly similar to the original question and answer text but with modified task labels, as a counterfactual example.

[0009] S3: For different question and answer texts, repeat S1 to S2 continuously to obtain a dataset to be filtered consisting of a series of counterfactual examples; use the self-consistency of the large language model to filter the dataset to be filtered. For each counterfactual example in the dataset to be filtered, the large language model generates a specified number of inference paths. Counterfactual examples whose inference results are consistent with the actual task labels in proportion to a threshold are retained in the new dataset, and the rest of the counterfactual examples are removed.

[0010] S4: For each question-and-answer text in the new dataset, generate a multi-perspective thought chain for each question-and-answer text using a large language model. The multi-perspective thought chain includes a positive thought chain for the correct answer option and a negative thought chain for each incorrect answer option. Finally, expand the question-and-answer text into a dataset containing multi-perspective thought chains and use it to train a small language model.

[0011] As a preferred embodiment of the first aspect above, in S1, the method for identifying phrases related to causal features from the question text is as follows:

[0012] S11: By leveraging the context learning capabilities of the prompt-driven large language model, the topic words in the question text can be obtained;

[0013] S12: Use syntactic analysis tools to obtain all noun phrases in the problem text, remove personal pronouns and possessive pronouns, and take the remaining noun phrases together with the topic words as phrases related to causal features.

[0014] As a preferred embodiment of the first aspect above, in S2, each phrase in the problem text related to causal features is replaced with the character "[MASK]".

[0015] As a preferred embodiment of the first aspect above, in S2, the answer used as the task label is modified to any option in the original question and answer text other than the correct answer option.

[0016] As a preferred embodiment of the first aspect above, in S2, the prompt with examples includes examples and instructions, wherein the example consists of an exemplary question text, options, expected answer, and question completion result, wherein phrases related to causal features in the question text are pre-replaced with masking characters; the instructions are task introductions for inputting a large language model, used to prompt the large language model to complete the masking characters in the context of a given option, so that the correct answer corresponding to the completed question text in the given option is consistent with the expected answer.

[0017] As a preferred embodiment of the first aspect above, in step S3, the method for filtering the dataset to be screened using the self-consistency of the large language model to generate a new dataset is as follows:

[0018] For each counterfactual example in the dataset to be filtered, the question and options in the corresponding new question-and-answer text are input into the large language model. The large language model performs multiple rounds of reasoning. Each reasoning round generates a reasoning path and selects the answer option from the options. It is determined whether the answer option selected in each reasoning result is consistent with the task label in the new question-and-answer text. If the consistency rate of all reasoning rounds is not lower than a preset threshold, the counterfactual example is retained in the new dataset; otherwise, the counterfactual example is removed.

[0019] As a preferred embodiment of the first aspect above, in S3, when filtering the dataset to be screened using the self-consistency of the large language model, each counterfactual example is inferred repeatedly by the large language model at least 5 times.

[0020] As a preferred option for the first aspect mentioned above, the consistency threshold for determining whether counterfactual examples are retained in the new dataset is 60%.

[0021] As a preferred embodiment of the first aspect above, in S4, when generating a positive thought chain for each question-and-answer text using the large language model, it is necessary to use prompts to drive the large language model to output the thought chain that it reasoned for the correct answer to the current question; when generating a negative thought chain for each question-and-answer text using the large language model, it is necessary to use prompts to drive the large language model to output the thought chain that it reasoned for the current question that each non-correct answer option is not the correct answer.

[0022] Secondly, the present invention provides a small language model training method, which utilizes the large language model knowledge distillation method based on counterfactual data augmentation and multi-perspective thinking chain as described in any of the solutions in the first aspect above to generate the new dataset as training data, and trains the small language model with the assistance of the multi-perspective thinking chain so that it can correctly realize question-answering reasoning.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] This invention provides a large-model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains. The method first utilizes topic terms and parse trees to extract key semantic features of text, and then automatically generates high-quality counterfactual data using a large language model. Next, it leverages multi-perspective thought chains to enhance the diversity of training data and improve the reasoning ability of small language models. Compared to existing technologies, the large-model counterfactual distillation framework proposed in this invention enhances the robustness and generalization ability of small model reasoning, achieving excellent performance on small language models with different architectures and datasets with different distributions. On datasets with the same distribution, this invention significantly outperforms the baseline, with an average improvement in inference accuracy of 11.43%. On three datasets with different distributions, this invention achieves an average improvement in inference accuracy of 14.9% compared to the baseline. This invention consistently demonstrates superior performance across various parameter scales (from 120M to 770M) and model architectures (from decoder-only to encoder-decoder). Attached Figure Description

[0025] Figure 1 The overall architecture diagram of the large-model knowledge distillation method based on counterfactual data augmentation and multi-perspective thinking chain provided by the present invention;

[0026] Figure 2 This is an experimental result of an embodiment of the present invention on the out-of-distribution dataset CSQA;

[0027] Figure 3 This is an experimental result of an embodiment of the present invention on the out-of-distribution dataset QuaRel;

[0028] Figure 4 This is a graph showing the experimental results of an embodiment of the present invention on the out-of-distribution dataset QASC;

[0029] Figure 5 This is an experimental result of an embodiment of the present invention on the out-of-distribution dataset ARC. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.

[0031] like Figure 1 As shown, in one embodiment of the present invention, a large-scale model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains is provided, the steps of which are shown in S1 to S4:

[0032] S1: Input the question-and-answer text consisting of questions, options, and answers into the Large Language Model (LLM), and combine the Large Language Model and Syntax Analysis to identify phrases related to causal features from the question text.

[0033] Through step S1 above, phrases related to causal features can be identified, and then the causal relationship in the original text can be disrupted through subsequent MASK operations. To achieve the identification of phrases related to causal features, this invention proposes a method involving topic terms and syntactic analysis to extract phrases related to causal features from the text. The specific steps are as follows:

[0034] S11: By leveraging the context learning capabilities of the large language model driven by prompts, the topic words in the question text are obtained.

[0035] S12: Use syntactic analysis tools to obtain all noun phrases in the problem text, remove personal pronouns and possessive pronouns, and take the remaining noun phrases together with the topic words as phrases related to causal features.

[0036] It should be noted that noun phrases are used in text to introduce, describe, and connect concepts, thereby enhancing the expressiveness and accuracy of the text. Therefore, noun phrases are important elements constituting causal features. This invention uses syntactic analysis tools to obtain noun phrases from the text, but in order to preserve elements from the original text as much as possible, personal pronouns and possessive pronouns are retained.

[0037] S2: Replace phrases related to causal features in the question text with masking characters, modify the answer used as the task label, and then drive the large language model with example prompts to complete all masking characters in the question text based on the modified answer, thereby generating a new question-and-answer text that is highly similar to the original question-and-answer text but with modified task labels, as a counterfactual example.

[0038] In step S2 above, since the large language model needs to be driven by example-based prompts, it is necessary to pre-construct prompts for the large model's contextual learning. In an embodiment of the present invention, in step S2, the example-based prompt includes an example and an instruction. The example consists of an exemplary question text, options, expected answer, and question completion result, wherein phrases related to causal features in the question text are pre-replaced with masking characters. The instruction is a task introduction for inputting into the large language model, used to prompt the large language model to complete the masking characters in the context of the given options, so that the correct answer corresponding to the completed question text in the given options is consistent with the expected answer. Therefore, the final constructed prompt consists of five parts: question, options, expected answer, completed text, and instruction.

[0039] In an embodiment of the present invention, in step S2 above, each phrase related to causal features in the question text is replaced with the character "[MASK]". Additionally, the answer serving as the task label can be modified to any option in the original question-and-answer text other than the correct answer. Therefore, after replacing the phrases related to causal features in the question text with mask characters, it can be concatenated with the constructed hints and input into the large language model. Since the answer serving as the task label has been replaced, the large language model will complete the question text after the mask operation, following the example, that is, replacing the text at each position of the character "[MASK]" according to the current answer serving as the task label, so that the correct answer of the completed question text can match the current answer serving as the task label. The resulting new question-and-answer text can then serve as a counterfactual example.

[0040] For example, see [link to previous article] Figure 1 As shown, in step S1 above, the original question-and-answer text consists of a question, options, and an answer. The question is "What might be the result if one is convicted of obstructing justice?", and there are four options: "A: injustice; B: criminal charges; C: fear; D: going to jail". The answer is "D: going to jail". After extracting phrases related to causal features from the text using the method of combining topic words and syntactic analysis in S1, the topic word is "obstructing justice" and two other noun phrases, "the result" and "one". These three phrases are then replaced with mask characters to obtain the mask text "Whatmight be [MASK] if [MASK] is convicted of [MASK]?". Then, a hint with examples is constructed, consisting of five parts: the question text, options, expected answer, question completion result, and hint, as follows:

[0041] Question: Aside from MASK, what does your MASK need?

[0042] Options: A:bone B:charm C:lots of attention D:walked

[0043] Expected answer: lots of attention

[0044] Question completion result: Aside from water and nourishment, what does your dog need?

[0045] Tip: Multiple choice questions consist of questions, options and answers. Based on the above example, please complete the [MASK] part of the question to make it a multiple-choice question with smooth semantics and clear logic.

[0046] Therefore, after concatenating the above Mask text and the example hints and inputting them into the large language model, the final question completion result will be obtained in the following format:

[0047] Question: What might be [MASK] if [MASK] is convicted of [MASK]?

[0048] Options:A:injustice B:criminal charges C:fear D:going to jail

[0049] Expected answer: injustice

[0050] Question completion result: What might be a concern for society if an innocent person is convicted of a crime?

[0051] The completed answer can be combined with the options and expected answer to form a new question-and-answer text, which can then be added to the dataset of counterfactual examples.

[0052] S3: For different question-and-answer texts, repeat S1 to S2 continuously to obtain a dataset to be filtered, consisting of a series of counterfactual examples; use the self-consistency of the large language model to filter the dataset to be filtered. For each counterfactual example in the dataset to be filtered, the large language model generates a specified number of inference paths. Counterfactual examples whose inference results are consistent with the actual task labels in proportion to a threshold are retained in the new dataset, and the rest of the counterfactual examples are removed.

[0053] It should be noted that the process of generating a reasoning path for each counterfactual example using the large language model only requires inputting the question text and options of the counterfactual example into the large language model. The large language model itself generates a reasoning path and makes a judgment, that is, selects the answer it considers to be correct from all options.

[0054] To obtain high-quality counterfactual data, this invention designs an evaluation strategy aimed at selecting the most promising counterfactual examples, thereby eliminating potential errors. This strategy is used to verify whether the generated completed questions match the new answers. Large language models possess self-consistency, meaning their selection of the correct answer to a question is generally consistent. Based on the self-consistency principle of large language models, this invention can generate a series of reasoning paths for each counterfactual example, then judge the degree of consistency of their answers, thereby filtering the counterfactual examples.

[0055] Therefore, in the embodiments of the present invention, the method for filtering the dataset to be screened using the self-consistency of the large language model to generate a new dataset in step S3 above is as follows:

[0056] For each counterfactual example in the dataset to be filtered, the question and options from the corresponding new question-and-answer text are input into a large language model. The large language model performs multiple rounds of reasoning. In each round of reasoning, the large language model generates a reasoning path and selects an answer option from the options. It is then determined whether the answer option selected in each round of reasoning is consistent with the task label (i.e., the aforementioned expected answer) in the new question-and-answer text. If the consistency rate across all rounds of reasoning is not lower than a preset threshold, the counterfactual example is retained in the new dataset; otherwise, it is removed.

[0057] After each reasoning iteration, the selected answer option can be compared with the task label (i.e., the aforementioned expected answer) in the new question-and-answer text. If they match, the answer is considered correct; otherwise, it is considered incorrect. Therefore, the consistency rate across all reasoning iterations can be considered the correctness rate of the answer. In embodiments of this invention, when using the self-consistency of the large language model to filter the dataset, the number of reasoning iterations performed by the large language model for each counterfactual example can be adjusted according to actual needs. Furthermore, the consistency rate threshold used to determine whether a counterfactual example is retained in the new dataset can be adjusted according to actual needs. Generally, each counterfactual example should be reasoned through at least 5 times by the large language model; fewer iterations are not advisable. A higher consistency rate threshold indicates a stricter retention standard for counterfactual examples, resulting in fewer retained examples of higher quality; conversely, a lower threshold indicates a more lenient retention standard, resulting in more retained examples of lower quality. In embodiments of this invention, the number of reasoning iterations can be set to 5, and the consistency rate threshold can be set to 60%.

[0058] For example, see [link to previous article] Figure 1 As shown, the completed question "What might be the result if one is convicted of obstructing justice?" has the following options: "A: injustice B: criminal charges C: fear D: going to jail". The expected answer (i.e., the correct answer) is: injustice. For this new question-and-answer text, the question and options can be input into a large language model, which will output its perceived answer. The model will then determine if the chosen answer matches the expected answer; this is considered one round. After five rounds, if the consistency rate exceeds 60% (meaning the answer matches the expected answer more than three times), the new question-and-answer text can be retained in the new dataset; otherwise, it needs to be deleted.

[0059] S4: For each question-and-answer text in the new dataset, generate a multi-perspective thought chain for each question-and-answer text using a large language model. The multi-perspective thought chain includes a positive thought chain for the correct answer option and a negative thought chain for each incorrect answer option. Finally, expand the question-and-answer text into a dataset containing multi-perspective thought chains and use it to train a small language model.

[0060] In an embodiment of the present invention, in step S4 above, when generating a positive thought chain for each question-and-answer text using the large language model, it is necessary to drive the large language model to output the thought chain that it reasoned to obtain the correct answer for the current question through prompts; when generating a negative thought chain for each question-and-answer text using the large language model, it is necessary to drive the large language model to output the thought chain that it reasoned to obtain the correct answer for each of the options through prompts, indicating that each non-correct answer option is not the correct answer.

[0061] The multi-perspective thought chain generated for each question-and-answer text consists of two parts: a positive thought chain supporting the correct option and a negative thought chain representing refutational evidence for the incorrect option. Both are directly generated by the large language model through context learning. The goal of the positive thought chain is to explore relevant information about the correct answer, while the negative thought chain focuses more on generating a negation reasoning path for each option (except the correct answer). This method can help small models distinguish and eliminate inappropriate options among multiple choices, enabling them to select the correct answer more selectively.

[0062] Continuing with the example of the new question and answer text mentioned above, see [link to example]. Figure 1As shown, in the new question-and-answer text, the completed question "What might be the result if one is convicted of obstructing justice?" has the following options: "A: injustice B: criminal charges C: fear D: going to jail". The expected answer (i.e., the correct answer) is: injustice. For this new question-and-answer text, the large language model can output its reasons for believing option A is the correct answer as a positive thought chain: "If an innocent person is convicted of a crime, one of the main concerns for society would be the issue of injustice. This can erode public trust in the legal system and have far-reaching consequences for both the individual wrongfully convicted and society as a whole. Therefore, the answer is A." Simultaneously, it can output its reasons for believing options B, C, and D are not correct answers as a negative thought chain. Taking the negative thought chain B as an example, its output is "The question is asking about a concern for society if an innocent person is convicted of a crime. Criminal charges would not be a concern in this scenario because the person has already been convicted. Therefore, the answer is not B." Thus, this invention can incorporate both positive and negative thought chains into new question-and-answer text to create a sample for training a small language model. This method helps the small model distinguish and eliminate insufficient options among multiple choices, thereby enabling it to more effectively select the correct answer.

[0063] The new dataset generated by the above-mentioned knowledge distillation method for large language models can be used as training data to train small language models, enabling the small language models to be trained with the assistance of multi-perspective thinking chains, so that they can correctly achieve question-answering reasoning.

[0064] The following example demonstrates the effectiveness of the new dataset generated by the above-described large language model knowledge distillation method on training a small language model on different datasets.

[0065] Example

[0066] This embodiment validates the performance on four multi-step inference datasets: CommonsenseQA (CSQA), QuaRel, ARC, and QASC. The ARC dataset is divided into a challenge set and an easy set. In the experiments of this embodiment, these two subsets are combined for model training. Since the test labels for the CSQA and QASC datasets are not publicly available, the official development sets are used as the test sets. Experimental results are as follows: Figure 2 , Figure 3 , Figure 4 , Figure 5 and as shown in Table 1

[0067] Table 1

[0068]

[0069]

[0070] Table 1 summarizes the accuracy comparison between small models using the proposed method and existing FT and FT-CoT methods. Across models with varying parameter counts from 120 million to 770 million, the proposed method outperforms FT-CoT. Specifically, the proposed method achieves performance improvements ranging from 4.17% to 23.22% at these scales, with an average improvement of 11.43%. On the QASC and ARC datasets, the proposed method outperforms large-parameter models trained using FT-CoT in distillation performance on small-parameter models. For example, on the ARC dataset, the proposed method achieves 40.42% performance on GPT2-base, outperforming GPT2-medium trained using FT-CoT by 3.27%. Furthermore, the proposed method outperforms GPT2-large trained using FT-CoT by 8.57% on GPT2-medium.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the implementation process of the present invention has been described in detail above, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A large-scale model knowledge distillation method based on counterfactual data augmentation and multi-perspective thought chains, characterized in that, Includes the following steps: S1: Input the question-and-answer text consisting of questions, options, and answers into the large language model, and combine the large language model and syntactic analysis to identify phrases related to causal features from the question text; S2: Replace phrases related to causal features in the question text with masking characters, and modify the answer as the task label. Then, drive the large language model with example prompts to complete all masking characters in the question text based on the modified answer, thereby generating a new question and answer text that is highly similar to the original question and answer text but with modified task labels, as a counterfactual example. S3: For different question and answer texts, repeat S1~S2 continuously to obtain a dataset to be filtered consisting of a series of counterfactual examples; use the self-consistency of the large language model to filter the dataset to be filtered. For each counterfactual example in the dataset to be filtered, the large language model generates a specified number of inference paths. Counterfactual examples whose inference results are consistent with the actual task labels in proportion to a threshold are retained in the new dataset, and the rest of the counterfactual examples are removed. S4: For each question-and-answer text in the new dataset, a multi-perspective thought chain is generated for each question-and-answer text using a large language model. The multi-perspective thought chain includes a positive thought chain for the correct answer option and a negative thought chain for each incorrect answer option. Finally, the new dataset with multi-perspective thought chains in the question-and-answer text is used as the knowledge distillation result of the large language model to train the small language model. In step S1, the method for identifying phrases related to causal features from the question text is as follows: S11: By leveraging the context learning capabilities of the prompt-driven large language model, the topic words in the question text can be obtained; S12: Use syntactic analysis tools to obtain all noun phrases in the problem text, remove personal pronouns and possessive pronouns, and take the remaining noun phrases together with the topic words as phrases related to causal features; In S2, the answer used as the task label is modified to any option in the original question and answer text other than the correct answer option; In S2, the example prompt includes an example and an instruction. The example consists of an exemplary question text, options, expected answer, and question completion result. Phrases related to causal features in the question text are pre-replaced with masking characters. The instruction is a task introduction for the input large language model, used to prompt the large language model to complete the masking characters in the context of the given options, so that the correct answer corresponding to the completed question text in the given options is consistent with the expected answer. In S3, the self-consistency of the large language model is used to filter the dataset to be screened, and the method to generate a new dataset is as follows: For each counterfactual example in the dataset to be filtered, the question and options in the corresponding new question-and-answer text are input into the large language model. The large language model performs multiple rounds of reasoning. Each time the large language model makes a reasoning, it generates a reasoning path and selects the answer option from the options. It is determined whether the answer option selected in each reasoning result is consistent with the task label in the new question-and-answer text. If the consistency rate of all reasoning rounds is not lower than a preset threshold, the counterfactual example is retained in the new dataset; otherwise, the counterfactual example is removed.

2. The knowledge distillation method for a large language model based on counterfactual data augmentation and multi-perspective thought chains as described in claim 1, characterized in that, In step S2, each phrase in the problem text related to causal features is replaced with the character "[MASK]".

3. The knowledge distillation method for a large language model based on counterfactual data augmentation and multi-perspective thought chains as described in claim 1, characterized in that, In S3, when filtering the dataset to be screened using the self-consistency of the large language model, each counterfactual example is inferred repeatedly by the large language model at least 5 times.

4. The knowledge distillation method for a large language model based on counterfactual data augmentation and multi-perspective thought chains as described in claim 3, characterized in that, The consistency threshold for determining whether counterfactual examples are retained in the new dataset is 60%.

5. The knowledge distillation method for a large language model based on counterfactual data augmentation and multi-perspective thought chains as described in claim 1, characterized in that, In step S4, when generating a positive thought chain for each question-and-answer text using the large language model, it is necessary to use prompts to drive the large language model to output the thought chain that it reasoned to arrive at the correct answer for the current question; when generating a negative thought chain for each question-and-answer text using the large language model, it is necessary to use prompts to drive the large language model to output the thought chain that it reasoned to arrive at the correct answer for each of the options, which is not the correct answer.

6. A method for training a small language model, characterized in that, Using the knowledge distillation method for large language models based on counterfactual data augmentation and multi-perspective thinking chains as described in any one of claims 1 to 5, the new dataset is generated and used as training data. With the assistance of the multi-perspective thinking chains, the small language model is trained so that it can correctly realize question-and-answer dialogue.

Images