Large language model knowledge extraction method and device based on difficulty perception

Abstract

The invention discloses a big language model knowledge extraction method and device based on difficulty perception, and aims to solve the problems that an existing knowledge extraction method is high in training cost and lack of pertinence in sample selection. According to the method, the difficulty of a distillation difficulty score evaluation sample is introduced, a difficult sample with learning value for a student model is identified, a distillation data set is dynamically adjusted in combination with a hierarchical data updating strategy, the difficult sample is preferentially reserved, a simple sample is removed, and meanwhile, data diversity is kept. Besides, a bidirectional difference loss function is provided, KL divergence and inverse KL divergence are combined, the optimization process is stabilized, more attention is paid to difficult samples, and gradient explosion or disappearance is avoided. Experimental results show that the performance of a student model is effectively improved, the method even exceeds a teacher model under some conditions, meanwhile, the training cost is remarkably reduced, and the method is suitable for task-independent instruction following and specific tasks and has wide application prospects.

Description

Technical Field

[0001] The present invention relates to a large language model knowledge extraction method based on difficulty perception, and also relates to a corresponding large language model knowledge extraction device, belonging to the field of computational model technology. Background Art

[0002] In recent years, large language models (LLMs) have attracted widespread attention due to their remarkable capabilities and intelligent performance. These models have made revolutionary progress in the field of natural language processing and are able to achieve state-of-the-art performance in a range of complex tasks. Early works such as GPT laid the foundation for subsequent innovations by introducing stacked Transformer decoders. Building on this foundation, models such as Llama and Qwen further improved efficiency and performance by improving the Transformer architecture. However, although these models have made significant progress in performance, their large parameter size also brings huge computational and storage requirements, which poses challenges to practical deployment. To address this problem, knowledge distillation has emerged as an effective method, whose goal is to transfer the knowledge of a large teacher model to a small student model to achieve efficient reasoning.

[0003] Knowledge distillation provides an effective method for building smaller and more efficient neural networks by transferring knowledge from a higher-performing teacher model to a compact student model. Although knowledge distillation has been successful in improving the performance of student models, it still suffers from the problem of high training cost. Existing knowledge distillation methods typically require hundreds of GPU hours to distill models with billions of parameters, which makes the distillation process very time-consuming and expensive. In addition, existing knowledge distillation methods have limitations in sample selection. They usually ignore the difficulty differences between different samples, resulting in unnecessary distillation of simple samples, which further increases the distillation cost. Although many efficient distillation methods have been proposed in recent years, such as knowledge caching, self-distillation, and dataset concentration techniques, these methods mainly focus on traditional downstream tasks (such as computer vision or language processing), and few methods have conducted in-depth research on how to select samples with high information content for generative large language models.

[0004] How to effectively select distillation samples remains an unsolved problem, making efficient distillation of large generative language models a nontrivial task. Existing methods lack specificity in sample selection and fail to fully leverage the interaction between the teacher and student models to optimize the distillation process. This not only limits improvements in distillation efficiency but also hinders the performance of the student model on complex tasks. Summary of the Invention

[0005] The primary technical problem to be solved by the present invention is to provide a large language model knowledge extraction method based on difficulty perception.

[0006] Another technical problem to be solved by the present invention is to provide a large language model knowledge extraction device based on difficulty perception.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a method for extracting knowledge from a large language model based on difficulty perception is provided, comprising the following steps:

[0009] Step 1: Set the initial selection ratio r = 1, which means that the entire data set D is used in the first iteration;

[0010] Step 2: Use the entire dataset D for the first iteration training and calculate the initial distillation difficulty score;

[0011] Step 3: From the second iteration until the total training cycle is completed, perform the following operations:

[0012] 31) Update the selection ratio r and gradually reduce the amount of selected data;

[0013] 32) Calculate the cross entropy loss of the teacher model and the student model on the dataset D;

[0014] 33) Update the distillation difficulty score of each sample based on the cross entropy loss of the teacher model and the student model;

[0015] 34) Arrange the dataset D in descending order according to the updated distillation difficulty score to obtain an ordered dataset;

[0016] 35) Divide the sorted dataset into two parts: the high distillation difficulty score part D high and low distillation difficulty score part D low ;

[0017] 36) From the low distillation difficulty score part D low Randomly select a part of the samples so that its size is τ×|D high | From the high distillation difficulty score part D high Randomly select a part of the samples so that its size is (1-τ)×|D high ∣; where τ is the selection ratio;

[0018] 37) Merge the samples selected from the high distillation difficulty score part and the low distillation difficulty score part to form a new distillation dataset;

[0019] 38) Update the teacher model and student model using the new distilled dataset;

[0020] 39) calculating a bidirectional difference loss function, using the bidirectional difference loss function as a loss function, and updating the parameters of the student model by back propagation;

[0021] Step 4: After completing all training cycles, return the trained student model.

[0022] Preferably, the two-way difference loss function is obtained by the following steps:

[0023] In each training step, the probability distribution of the teacher model and the student model on the current sample is first calculated; then the teacher probability distribution and the student probability distribution are combined to form the bidirectional difference loss function.

[0024] Preferably, the two-way difference loss function is calculated by the following formula:

[0025] D BDL (p,q θ )=D KL (((1-λ)p+λq θ )||(λp+(1-λ)q θ ))

[0026] Among them, D BDL Denotes the bidirectional difference loss function, D KL represents the standard KL divergence, p and q θ They represent the probability distributions of the teacher model and the student module respectively, and λ is a parameter used to balance the respective contributions of the teacher probability distribution and the student probability distribution.

[0027] Preferably, when λ=0.9, the bidirectional difference loss function performs best on difficult samples.

[0028] Preferably, the distillation difficulty score is calculated by the following formula:

[0029]

[0030] Wherein, DDS(x) is the distillation difficulty score, Lstudent(x) and Lteacher(x) represent the cross entropy loss of the student model and the teacher model on sample x, respectively.

[0031] Preferably, the distillation difficulty score is used to identify difficult samples that are difficult for the student model to learn but can be processed by the teacher model; the difficult samples are retained and the simple samples are removed to dynamically adjust the distillation dataset.

[0032] Preferably, in the sub-step 31), the selection ratio r of each training cycle is gradually reduced according to a linear decay plan or a cosine decay plan.

[0033] According to a second aspect of an embodiment of the present invention, a large language model knowledge extraction device based on difficulty perception is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the above-mentioned large language model knowledge extraction method.

[0034] Compared with the prior art, the large language model knowledge extraction method provided by the embodiment of the present invention significantly improves the efficiency and effect of knowledge extraction from large language models through difficulty-aware data update (DiffUp) and bidirectional difference loss function (BDL). Among them, the difficulty-aware data update strategy optimizes the distillation process by dynamically adjusting the distillation data set, giving priority to retaining difficult samples and eliminating simple samples. It introduces the distillation difficulty score (DDS) to measure sample complexity, identifies samples that are difficult for the student model to learn but can be easily processed by the teacher model, and adopts the layered data update (SDU) strategy to maintain data diversity. The bidirectional difference loss function combines the teacher probability distribution and the student probability distribution to stabilize the optimization process and pay more attention to difficult samples to avoid gradient explosion or disappearance. Experimental results show that the present invention effectively improves the performance of the student model in multiple tasks, even surpassing the teacher model in some cases, while significantly reducing the training cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A logical framework diagram of the large language model knowledge extraction method provided by an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the performance of the Distillation Difficulty Score (DDS) in different situations;

[0037] Figure 3 A flowchart of a large language model knowledge extraction method provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of a large language model knowledge extraction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] The technical concept of the embodiments of the present invention is to filter out simple samples by dynamically adjusting the distillation dataset and give priority to samples with high difficulty for training, thereby reducing computational costs. In addition, the present invention also proposes a new bidirectional discrepancy loss function (BDL), which combines the teacher probability distribution and the student probability distribution, and constrains the distribution difference between the student model and the teacher model, making the optimization process more stable, thereby effectively improving the efficiency and effectiveness of knowledge extraction from the large language model (LLM).

[0041] Figure 1 The logical framework of the knowledge extraction method provided by an embodiment of the present invention is demonstrated, which includes two parts: a bidirectional difference loss function and difficulty-aware data update. In the bidirectional difference loss function part, the probability distributions of the teacher model and the student model are represented as P and Q respectively. The bidirectional difference loss function is calculated by combining the teacher probability distribution and the student probability distribution to stabilize the optimization process and focus on difficult samples. In the difficulty-aware data update part, starting from the initial data set D, the cross entropy loss Lp and the distillation difficulty score (DDS) of each sample are calculated. The data set is divided into a high DDS part and a low DDS part after being sorted by DDS. A new distillation data set D′ is constructed by stratified sampling, and the amount of data is gradually reduced to optimize the distillation process. Below, a specific embodiment is used to explain in detail:

[0042] First, given a large teacher model and a small student model, knowledge extraction is a technique that transfers knowledge from the large teacher model to the small student model, aiming to achieve faster inference. However, existing knowledge extraction methods typically require a lot of training time due to the need to process large amounts of training data.

[0043] In order to improve the efficiency of distillation, the embodiment of the present invention first proposes a data update strategy based on difficulty awareness (Diff Up). The basic idea of ​​this data update strategy is to dynamically update the distillation data based on the difficulty of the sample, thereby reducing the computational cost. Specifically, in order to quantitatively evaluate the difficulty of the sample, the embodiment of the present invention introduces a new indicator, namely the distillation difficulty score (DDS). DDS is defined as the ratio of the teacher model loss to the student model loss. The specific formula is as follows:

[0044]

[0045] Among them, Lstudent(x) and Lteacher(x) represent the cross entropy loss of the student model and the teacher model on sample x, respectively.

[0046] This simple but effective distillation difficulty score captures the distillation difficulty based on the difference between the teacher model and the student model. Figure 2 The figure shows the performance of the Distillation Difficulty Score (DDS) in different situations. When the student model has a high loss on a certain example and the teacher model has a low loss on the same example, the DDS value will be large. This indicates that the teacher model has high confidence in the example, while the student model is having difficulty learning the example and therefore requires additional supervision from the teacher model. Conversely, when both the student and teacher models have low losses on a certain example, the DDS value is small, indicating that the example has been well learned by both models and further distillation is unnecessary. In addition, when both the student and teacher models have high losses on a certain example, the DDS value is also small, indicating that the teacher model has insufficient understanding of the example and cannot provide effective guidance to the student model. DDS can identify examples that the student model has difficulty learning but the teacher model can handle easily. These examples are considered the most valuable in the distillation process because they represent areas where the student model needs additional supervision. In contrast, when both the teacher and student models have low losses on a certain example, the example has been well learned and no further distillation is necessary. When the losses of both the teacher model and the student model on a certain sample are high, it means that the teacher model does not have enough understanding of the sample and therefore cannot provide effective guidance to the student model.

[0047] Based on the Distillation Difficulty Score (DDS), we can identify difficult examples that the student model struggles to learn but that the teacher model can handle. During the subsequent distillation process, difficult examples are prioritized and simple examples are discarded, dynamically adjusting the distillation dataset. Therefore, the DiffUp strategy provided by this embodiment of the present invention optimizes the distillation process, not only improving distillation efficiency but also ensuring that the student model obtains the most useful information from the teacher model.

[0048] In this embodiment of the present invention, the cross-entropy loss is first calculated for each sample in the initial dataset, and the DDS for each sample is calculated based on this cross-entropy loss. Next, a hierarchical data update is performed based on the DDS, and the distilled dataset is gradually reduced during this process. During the distillation process, a bidirectional difference loss function is used as the loss function to stabilize the optimization process and focus on difficult samples.

[0049] It should be noted that data selection is crucial in optimizing the LLM fine-tuning process. It aims to prioritize the data subset that is most valuable for training, reduce computational costs, and maintain or improve model performance. Although recent studies have explored data selection methods for LLM fine-tuning, its application in knowledge extraction has not been fully studied, especially the interaction between the teacher model and the student model needs to be considered. Different from existing research, the embodiments of the present invention propose a hierarchical data update strategy during the distillation process, dynamically selecting data for each training cycle based on DDS. The core idea is to prioritize difficult samples with high DDS values ​​and gradually reduce the overall data volume in the next training cycle to achieve efficient distillation. Assume there is a training dataset of size N, and the data selection ratio for each training cycle is r, that is, rN samples are used for distillation in each cycle. Initially, the selection ratio r is set to 1, which means that all available data is used at the beginning. As distillation progresses, r gradually decreases according to a linear decay schedule or a cosine decay schedule. If the total number of training cycles is E, the selection ratio of the e-th training cycle is:

[0050] Linear decay schedule:

[0051] Cosine decay scheme:

[0052] This strategy enables the distillation process to focus on the most valuable data, thereby reducing training costs without sacrificing model performance.

[0053] In order to enhance data diversity and alleviate the forgetting problem when only difficult samples are used, the embodiment of the present invention further proposes a stratified sampling strategy to mix various samples when updating the distilled data. Specifically, at the beginning of each training cycle, the distillation difficulty score of the entire dataset is first calculated and sorted in descending order. Then, the dataset is divided into two parts r by a number threshold r, and a low DDS part D containing (1-r)% of samples is created. low and the high DDS portion D containing r% of the sample high Next, samples are randomly drawn from these two parts to construct the distilled data subset for the current training cycle.

[0054] Formally, we denote the entire dataset as set D. And the power set P(D) denotes all subsets. Obviously, D high ,D low ∈P(D) and D high ∪D low =D. Next, Indicates that D is included in the power set high The set of subsets of elements, namely:

[0055]

[0056] Where |·| is the number of elements in the returned set, and τ is used to balance the selection ratio in the high and low partitions. Before each training cycle, calculate and sort to obtain D high , then from Randomly select an element D′ high ,Right now and Where ∈R represents random selection from a finite set. low The same procedure applies to the collection, where Randomly select an element D′ low ,Right now in Indicates that it contains τ|D high | element D low Therefore, the activation data of this training cycle is D′=D′ high ∪D′ low , and it is easy to know |D′ low |+|D′ high |=τ|D high |+(1-τ)|D high |=r|D|.

[0057] This stratified sampling strategy ensures that the distilled dataset not only prioritizes challenging samples but also retains representative easier samples, thus avoiding potential bias towards high-difficulty samples (high-DDS) and enabling the student model to generalize effectively across the entire data distribution.

[0058] On the other hand, when the above-mentioned DiffUp strategy is used to construct a distillation dataset, the retained training samples are usually more challenging, and the difference between the teacher model and the student model is larger. In order to achieve robust and stable distillation, the embodiment of the present invention provides a new distillation loss function called Bidirectional Discrepancy Loss. This function is built on the KL divergence, but further integrates the probability distribution of the two measurements. Its formal expression is as follows:

[0059] D BDL (p,q θ )=D KL (((1-λ)p+λq θ )||(λp+(1-λ)q θ )) (4)

[0060] Among them, D BDL Denotes the bidirectional difference loss function, D KL represents the standard KL divergence. p and q θdenote the probability distributions of the teacher model and the student model respectively. λ is a parameter used to balance the contributions of the teacher probability distribution and the student probability distribution.

[0061] The bidirectional difference loss function, a loss function that combines KL divergence and inverse KL divergence, is used in knowledge extraction to stabilize the optimization process and focus more on difficult examples. It effectively limits the range of gradients, thereby avoiding the problem of exploding or vanishing gradients. The gradient range of the bidirectional difference loss function is determined only by predefined coefficients and is independent of the distribution of the teacher or student models. This property ensures the stability of the optimization process and maintains robust updates even when the distributions of the teacher and student models differ significantly. Through mathematical derivation, the gradient of the bidirectional difference loss function is constrained to a limited range, thus avoiding gradient anomalies caused by excessive distribution differences.

[0062] Compared with the existing technology, the bidirectional difference loss function has the following characteristics:

[0063] (1) Combining teacher probability distribution and student probability distribution: The bidirectional difference loss function comprehensively considers the probability distribution differences between the teacher model and the student model by using the teacher probability distribution and the student probability distribution at the same time.

[0064] (2) Stable Optimization Process: The bidirectional divergence loss function ensures the stability of the optimization process by limiting the range of the gradient. Because it combines two divergences, the range of the gradient is determined only by the predefined coefficients and is independent of the distribution of the teacher model or the student model. This prevents gradient anomalies caused by large distribution differences.

[0065] (3) Focus on difficult samples: The bidirectional difference loss function can pay more attention to difficult samples where the output of the teacher model and the student model are significantly different. The distribution difference of these samples will amplify the gradient, thus forcing the model to prioritize these samples during training. This feature enables the student model to better learn the knowledge of the teacher model, especially when dealing with difficult samples.

[0066] In one embodiment of the present invention, the bidirectional difference loss function is obtained by the following steps:

[0067] In each training step, the probability distribution of the teacher model and the student model on the current sample is first calculated. Then, the teacher probability distribution and the student probability distribution are combined to form the bidirectional difference loss function using the above formula (4).

[0068] Then, the bidirectional difference loss function is used as the loss function to update the parameters of the student model through backpropagation. The gradient of the bidirectional difference loss function guides the optimization direction of the student model, making it perform better on difficult samples.

[0069] In practical applications, the contributions of the teacher and student probability distributions can be balanced by adjusting the weight coefficients of the two. This adjustment can be optimized according to the needs of specific tasks to achieve the best distillation effect.

[0070] In the above way, the bidirectional difference loss function not only improves the performance of the student model, but also ensures the stability and efficiency of the training process.

[0071] It's important to note that a notable feature of the bidirectional discrepancy loss function is its ability to focus more on difficult examples where the outputs of the teacher and student models differ significantly. The difference in the distribution of these examples amplifies the gradient, forcing the model to prioritize these examples during training. Experimental observations show that the bidirectional discrepancy loss function performs better when handling difficult examples, especially when the distribution of the teacher and student models differs significantly. This property enables the student model to better learn from the teacher model, especially when handling the most challenging examples for the student model.

[0072] The parameter λ in the bidirectional difference loss function is used to balance the contributions of the teacher probability distribution and the student probability distribution. Different values ​​of λ affect the behavior of the bidirectional difference loss function. When λ = 0, the bidirectional difference loss function behaves similarly to the standard KL divergence, with a small gradient coefficient, which may cause the gradient to vanish. When λ = 0.9, the bidirectional difference loss function performs best, with a moderate gradient coefficient and a stable optimization process. When λ = 1, the bidirectional difference loss function behaves similarly to the inverse KL divergence, with a large gradient coefficient, which may cause gradient explosion. Experimental results show that when λ = 0.9, the bidirectional difference loss function performs best on difficult samples, verifying the correctness of the theoretical derivation.

[0073] In summary, the bidirectional difference loss function, by combining the teacher and student probability distributions, not only stabilizes the optimization process but also pays more attention to difficult examples. This property makes the bidirectional difference loss function excellent for knowledge extraction, especially when dealing with examples where the output of the teacher model and the student model differ significantly. By adjusting the parameter λ, the performance of the bidirectional difference loss function can be further optimized to achieve optimal results in different distillation tasks. These characteristics of the bidirectional difference loss function make it a powerful tool that can effectively improve the performance of the student model while ensuring the stability and efficiency of the training process.

[0074] Next, combine Figure 3 The specific steps of the knowledge extraction method provided by the embodiment of the present invention are described as follows:

[0075] Step 1: Initialize the selection ratio

[0076] The initial selection ratio r is set to 1, which means that the entire data set D is used in the first iteration.

[0077] Step 2: Iterative training

[0078] The first iteration of training is performed using the entire dataset D and the initial distillation difficulty score (DDS) is calculated.

[0079] Step 3: Loop Update

[0080] From the 2nd iteration to the total training cycle E:

[0081] Update the selection ratio: Update the selection ratio r according to formula (2) or formula (3), and gradually reduce the amount of selected data.

[0082] Calculate loss: Calculate the teacher model θ T and the student model θ S Cross entropy loss Lq on dataset D θ (D) and Lp(D).

[0083] Update DDS: Update the DDS of each sample based on the cross entropy loss of the teacher model and the student model.

[0084] Sorting the dataset: Arrange the dataset D in descending order according to the updated DDS to obtain the ordered dataset D ord .

[0085] Divide the data set: Divide the sorted data set into two parts: the high DDS part D high and low DDS part D low It should be noted that the DDS threshold used to distinguish the high DDS portion from the low DDS portion is not fixed and can be flexibly set according to actual conditions.

[0086] · Stratified sampling: from the low DDS part D low Randomly select a part of the samples so that its size is τ×|D high ∣; From the high DDS part D high Randomly select a part of the samples so that its size is (1-τ)×|D high ∣.

[0087] Construct a new distilled dataset: Merge the samples selected from the high DDS part and the low DDS part to form a new distilled dataset D′.

[0088] Model update: Update the teacher model and student model using the new distilled dataset D′. Calculate BDL loss: Calculate the bidirectional difference loss function (BDL).

[0089] Backpropagation: Use the bidirectional difference loss function as the loss function and update the parameters of the student model through backpropagation.

[0090] Step 4: End the loop

[0091] After completing all training cycles, return the trained student model θ S .

[0092] Through the above steps, the knowledge extraction method provided by the embodiment of the present invention can dynamically adjust the distillation dataset, prioritize difficult samples with high DDS, and maintain data diversity through stratified sampling. This method not only improves distillation efficiency but also ensures that the student model obtains the most useful information from the teacher model, thereby improving the distillation effect.

[0093] To verify the effectiveness of the knowledge extraction method provided by the embodiment of the present invention, we conducted extensive experiments with the following specific settings:

[0094] The experiments cover both task-independent instruction following and task-specific experiments. The instruction following experiments were distilled using the data-bricksdolly-15k dataset and evaluated on five instruction following datasets, including Dolly evaluation, Self-Instruct, Super-Natural Instructions, Unnatural Instruction, and Vicuna evaluation, with the evaluation metric being the ROUGE-L score. Task-specific experiments included text summarization (using the SAM Sum dataset) and mathematical reasoning (using the GSM8K dataset), evaluated using the ROUGE-L score and zero-shot accuracy, respectively. The teacher models were LLaMA2-7B and Qwen2.5-7B, and the student models were Sheared-LLaMA2-2.7B / 1.3B and Qwen2.5-1.5B / 0.5B. Training was performed for 10 epochs with a batch size of 8, using the AdamW optimizer and a cosine learning rate scheduler with an initial learning rate of 1e-5. The parameters τ = 0.1, λ = 0.9 were set, and a cosine decay schedule was used to gradually reduce the data selection ratio r. The compared methods include SFT, KD-KL, KD-RKL, SeqKD, GKD and Distillm.

[0095] In the task-independent instruction following experiments shown in Table 1, the present invention achieves the highest ROUGE-L scores on most evaluation datasets. For example, on the Llama2-2.7B model, the present invention achieves an average ROUGE-L score of 28.11, surpassing KD-RKL (27.70) and GKD (26.85). On the Qwen2.5-0.5B model, the present invention achieves an average ROUGE-L score of 30.44, surpassing Distillm (30.24) and other baseline methods. It is worth noting that the present invention even surpasses the teacher model in some cases.

[0096] Table 1

[0097]

[0098] In the specific task experiments shown in Table 2, the present invention also performed well. In the text summarization task of the SAMSum dataset, the present invention achieved ROUGE-L scores of 39.20 and 32.92 on Llama2-1.3B and Llama3.2-1B, respectively, surpassing Distillm (38.73 and 32.53). In particular, on Qwen2.5, the student model even surpassed the 7B teacher model, reaching a score of 40.05. In the mathematical reasoning task of the GSM8K dataset, the present invention achieved a zero-shot accuracy of 54.66% on Qwen2.5, 14.56% on Llama2, and 22.37% on Llama3.2, significantly outperforming its competitors.

[0099] Table 2

[0100]

[0101] As can be seen in Table 3, our method significantly improves training efficiency. For example, our method only requires 1,963 training iterations, a 55% reduction compared to other methods. Training time is only 26.1% of GKD (106.35 minutes vs. 408.24 minutes) and 49.9% of Distillm (106.35 minutes vs. 213.34 minutes).

[0102] Table 3

[0103]

[0104] The ablation studies shown in Tables 4 and 5 demonstrate that DiffUp and BDL are key factors in the success of the present invention. In the ablation study of DiffUp, the average ROUGE-L score decreased by 1.07 in the “w / o DDS” setting, demonstrating the importance of DDS in selecting informative and challenging samples. Similarly, the average ROUGE-L score decreased by 0.91 in the “w / o SDU” setting, demonstrating the importance of the stratified data update (SDU) strategy in maintaining data diversity. In the ablation study of BDL, BDL performed best among different loss functions, with an average ROUGE-L score of 26.15, exceeding SRKL (24.99) and other loss functions. In addition, by analyzing the influence of the parameter λ in BDL, it was found that BDL performed best when λ = 0.9, verifying the correctness of the theoretical derivation.

[0105] Table 4

[0106]

[0107] Table 5

[0108]

[0109] In summary, by dynamically adjusting the distillation dataset and using the BDL loss function, this paper achieves excellent performance on multiple benchmark datasets while significantly reducing training costs. Experimental results show that this paper not only improves the performance of the student model but also surpasses the teacher model in some cases, demonstrating its effectiveness in efficient large language model distillation.

[0110] Based on the above-mentioned large language model knowledge extraction method based on difficulty perception, the present invention also provides a large language model knowledge extraction device. Figure 3 As shown, the apparatus includes one or more processors and a memory. The memory is coupled to the processor and is configured to store one or more spiking neural networks. The processor implements the difficulty-aware large language model knowledge extraction method described in the above embodiment to prune the spiking neural network.

[0111] The processor is used to control the overall operation of the device to complete all or part of the steps of the above-mentioned difficulty-aware large language model knowledge extraction method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support the operation of the difficulty-aware large language model knowledge extraction method. Such data may include, for example, instructions for any application or method operating on the device, as well as application-related data.

[0112] The memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, etc.

[0113] In an exemplary embodiment, the apparatus may be implemented as a computer chip or entity, or as a product having certain functions, for executing the above method and achieving the same technical effect as the above method. A typical embodiment is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-computer interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0114] In another exemplary embodiment, the present invention further provides a computer-readable storage medium comprising program instructions, which, when executed by a processor, implement the steps of the difficulty-aware large language model knowledge extraction method described in any of the aforementioned embodiments. For example, the computer-readable storage medium may be a memory comprising program instructions, which may be executed by a processor of a large language model knowledge extraction device to perform the difficulty-aware large language model knowledge extraction method described above and achieve the same technical effects as the aforementioned method.

[0115] It should be noted that the above embodiments are merely examples, and the technical solutions of the various embodiments may be combined and are all within the scope of protection of the present invention.

[0116] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0117] The above describes in detail the difficulty-aware large language model knowledge extraction method and apparatus provided by the present invention. For those skilled in the art, any obvious modification to this invention without departing from its essence would constitute an infringement of the present invention's patent rights and would incur corresponding legal liability.

Claims

1. A large language model knowledge extraction method based on difficulty perception, characterized by The steps include: Step 1: Set the initial selection ratio r = 1, which means that the entire data set D is used in the first iteration; Step 2: Use the entire dataset D for the first iteration training and calculate the initial distillation difficulty score; Step 3: From the second iteration until the total training cycle is completed, perform the following operations: 31) Update the selection ratio r and gradually reduce the amount of selected data; 32) Calculate the cross entropy loss of the teacher model and the student model on the dataset D; 33) Update the distillation difficulty score of each sample based on the cross entropy loss of the teacher model and the student model; 34) Arrange the dataset D in descending order according to the updated distillation difficulty score to obtain an ordered dataset; 35) Divide the sorted dataset into two parts: the high distillation difficulty score part D high and low distillation difficulty score part D low ; 36) From the low distillation difficulty score part D low Randomly select a part of the samples so that its size is τ×|D high | From the high distillation difficulty score part D high Randomly select a part of the samples so that its size is (1-τ)×|D high ∣; where τ is the selection ratio; 37) Merge the samples selected from the high distillation difficulty score part and the low distillation difficulty score part to form a new distillation dataset; 38) Update the teacher model and student model using the new distilled dataset; 39) calculating a bidirectional difference loss function, using the bidirectional difference loss function as a loss function, and updating the parameters of the student model by back propagation; Step 4: After completing all training cycles, return the trained student model.

2. The large language model knowledge extraction method according to claim 1, characterized in that The bidirectional difference loss function is obtained by the following steps: In each training step, the probability distribution of the teacher model and the student model on the current sample is first calculated; then the teacher probability distribution and the student probability distribution are combined to form the bidirectional difference loss function.

3. The large language model knowledge extraction method according to claim 2, characterized in that The bidirectional difference loss function is calculated by the following formula: D BDL (p,q θ )=D KL (((1-λ)p+λq θ )||(λp+(1-λ)q θ )) Among them, D BDL Denotes the bidirectional difference loss function, D KL represents the standard KL divergence, p and q θ denote the probability distributions of the teacher model and the student module respectively, and λ is a parameter used to balance the respective contributions of the teacher probability distribution and the student probability distribution.

4. The large language model knowledge extraction method according to claim 3, wherein: When λ=0.9, the bidirectional difference loss function performs best on difficult samples.

5. The large language model knowledge extraction method according to claim 1, characterized in that The distillation difficulty score is calculated by the following formula: Wherein, DDS(x) is the distillation difficulty score, Lstudent(x) and Lteacher(x) represent the cross entropy loss of the student model and the teacher model on sample x, respectively.

6. The large language model knowledge extraction method according to claim 1, wherein: The distillation difficulty score is used to identify difficult samples that are difficult for the student model to learn but can be handled by the teacher model; the difficult samples are retained and the simple samples are removed to dynamically adjust the distillation dataset.

7. The large language model knowledge extraction method according to claim 1, characterized in that In the sub-step 31), the selection ratio r of each training cycle is gradually reduced according to a linear decay plan or a cosine decay plan.

8. The large language model knowledge extraction method according to claim 7, characterized in that In the linear decay schedule, the selection ratio r of the e-th training cycle is: Where E is the total training cycle.

9. The large language model knowledge extraction method according to claim 7, characterized in that In the cosine decay scheme, the selection ratio r of the e-th training cycle is: Where E is the total training cycle.

10. A large language model knowledge extraction device based on difficulty perception, characterized in that It includes a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the large language model knowledge extraction method described in any one of claims 1 to 9.

Images