Comparative self-adaptive knowledge distillation method oriented to large language model lightweight

Abstract

The invention relates to a contrast type self-adaptive knowledge distillation method oriented to large language model lightweight. The method comprises the following steps of selecting a general teacher model and a student model and performing initialization; training sets X constructed by using the same type of data are respectively used as inputs of a teacher model and a student model, and respective outputs of the two models are combined with original inputs to construct a mixed training sample; and constructing a double-branch contrast type loss function for training the student model, and finally obtaining the trained student model. The student model training method provided by the invention has the advantages of stable training, excellent performance, efficient resources and the like, is suitable for lightweight deployment scenes of language, code, vision and other multi-modal large models in a resource-constrained environment, and has good engineering application prospects and popularization values.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence and deep learning model compression, and particularly relates to a knowledge distillation method for general large model light weight. BACKGROUND

[0002] In recent years, general large language models (LLM) centered on Transformer have made significant progress in natural language processing, code generation, multi-modal understanding, and other tasks. Typical large models such as GPT-4, LLaMA-3, and Qwen2 series have parameter scales of tens of billions (Billion) or even hundreds of billions (Trillion). Although such models have achieved excellent performance in task performance, the problem of high computational resources and deployment costs caused by large-scale parameters has become increasingly prominent. In practical applications, large models face challenges such as high memory occupancy, large inference delay, and difficult control of energy consumption, especially in mobile terminals, edge devices, and industrial embedded environments.

[0003] To alleviate the above problems, the industry generally adopts model compression techniques, mainly including pruning, quantization, and knowledge distillation (KD). Among them, KD, as a method with strong universality and high transferability, has been widely used in academia and industry. Traditional knowledge distillation usually adopts the "teacher-student" paradigm, that is, using a superior large model (teacher) to generate soft labels, measuring the difference between the output distributions of the teacher and student through Kullback-Leibler (KL) divergence, and guiding the small model (student) to learn the behavior pattern of the teacher model. In the fields of image classification and speech recognition, KL distillation has achieved certain results.

[0004] However, in the task of large language model distillation, traditional KD methods are gradually revealing multiple limitations. First, KL divergence is prone to amplify the gradient of small probability when dealing with low confidence output of the teacher model, thereby introducing training instability and even leading to overfitting of the student model in unimportant areas. Second, traditional methods mostly use token-level or intermediate layer alignment strategies, which are difficult to capture the semantic consistency and structural integrity of the entire sequence in generation tasks, especially in high-consistency tasks such as code generation and mathematical reasoning. Third, when the parameter scale of the teacher model is much larger than that of the student model (e.g., 7B vs. 1.5B), the difference between their representation spaces is significant, and the alignment effect in the distillation process is not good, making it difficult for the student model to effectively inherit the teacher's knowledge, and the distillation converges slowly and unstably.

[0005] To solve the above problems, some researches have tried to introduce reverse KL divergence, bidirectional alignment loss, sequence-level distillation and other means for improvement, but most of the schemes are limited to small models or specific tasks, and lack a unified framework for general large models that can adapt to multi-task scenarios. The existing public patent schemes also have the problem of incomplete technical coverage. For example, CN112418343A and CN114611670A mainly target image classification and speech recognition tasks, and use traditional KL divergence to construct a static distillation device, without addressing the sequence structure consistency problem in text generation tasks; CN112967088A introduces a segmented temperature to smooth the soft label, but its distillation is still based on the static KL architecture, which is difficult to dynamically distinguish between the teacher high-confidence region and the student tail noise, and lacks the ability to model the differences in model structure and dynamic features in the distillation process.

[0006] In summary, there is currently no set of knowledge distillation methods that can simultaneously consider general large models, high confidence modeling, low confidence region suppression, and resource controllable deployment. Especially in the cross-scale migration process from large models to small models, how to balance knowledge transfer and training stability has become an important technical bottleneck in the field of knowledge distillation. SUMMARY

[0007] To solve the above problems of the prior art, the technical problems solved by the present application are: when the parameter size difference between the teacher model and the student model is large, how to improve the training stability and generalization ability in the knowledge distillation process and avoid gradient noise interference caused by the low confidence region; and how to significantly reduce the frequency of calling the teacher model through mixed sampling and adaptive scheduling mechanism, thereby reducing the resource consumption and deployment cost of the overall training process.

[0008] To solve the above technical problems, the present application adopts the following technical solutions:

[0009] A contrastive adaptive knowledge distillation method for large language model lightening, comprising the following steps:

[0010] S100: selecting a general teacher model p and a general student model and initializing the parameters of p and ;

[0011] S200: constructing a training data set X, the training data set X comprising code class prompts and natural language instruction prompts;

[0012] inputting X into p , outputting a teacher high-confidence response set TGOs, and storing TGOs as offline supervision data;

[0013] freezing initialization parameters from any optional one training sample x in X output student immediate response y s and y s corresponding logits data, and write into Replay Buffer; and y s corresponding teacher high-confidence response is y t ;

[0014] traverse all data in X, get student immediate response corresponding to each training sample, and collect all student immediate responses to get SGOs;

[0015] S300: construct mixed training sample Y:

[0016] randomly extract part of student immediate responses from SGOs, all extracted student immediate responses, input x i corresponding to each of the extracted student immediate responses, and teacher high-confidence response corresponding to each of the extracted student immediate responses form a triplet All triplets form mixed training sample Y, Y=(B1,…,B i ,…,B |Y| )}, where |Y| represents the total number of data in Y;

[0017] S400: dynamically calculate y t and y s according to the probability difference: y t corresponds to adaptive skew coefficient α s and y s corresponds to adaptive skew coefficient α t , α t and α s are parameters in logits, when logits is obtained, information about α t and α s is obtained, α t and α s are calculated as follows:

[0018]

[0019]

[0020] wherein, |p-q|=|p(y|x)-q θ (y|x)|, when |p-q|<0.02, α t and α s are limited to 0.05-0.10; when |p-q|>0.20, α t and α sThe upper limit is 0.40; wherein, m represents a normalization constant, and a0 represents an initial value of an adaptive skewness coefficient;

[0021] S500: Constructing a student model total loss function L total total The calculation expression is as follows:

[0022] L total =(1-λ)L LM +λL CAKD

[0023] Wherein, L LM represents a standard language modeling loss, L CAKD represents a contrastive distillation loss, and λ represents a weight parameter;

[0024] S600: Taking Y as input, training the student model by using L total , and using an AdamW optimizer, and stopping the training when the training reaches a maximum number of iterations or a loss function reaches a minimum, so as to obtain a trained student model

[0025] S700: Selecting a model to be used M, and obtaining a student model corresponding to M by using the method described in S200-S600.

[0026] As a preferred, the teacher model p and the student model Further comprising: the parameter size of the teacher model is not less than 5B, the parameter size of the student model is not greater than 2B, and the ratio of the parameter quantities of the two is not higher than 0.3.

[0027] As a preferred, the step of constructing the student model total loss function L total in S500 is as follows:

[0028] S510: Measuring a high confidence area of a teacher branch by using a skew KL divergence SKL The calculation formula is as follows:

[0029]

[0030]

[0031] Wherein, represents a teacher branch soft interpolation distribution;

[0032] S520: Measuring a tail probability of a student branch by using a skew reverse KL divergence SRKL The calculation formula is as follows:

[0033]

[0034] ​

[0035] wherein, denotes the student branch soft interpolation distribution;

[0036] S530: calculate the contrastive total loss function L CAKD , the calculation formula is as follows:

[0037]

[0038] wherein, β denotes a loss weight;

[0039] The calculation formula of β is as follows:

[0040]

[0041] wherein, e denotes a current training cycle serial number, E denotes a total cycle number, τ denotes a step number in a current cycle, T denotes an upper limit of the cycle step number, and β0 denotes an initial loss weight;

[0042] S540: jointly L CAKD and the standard language modeling loss L LM construct L total , the calculation formula is as follows:

[0043] L total =(1-λ)L LM +λL CAKD

[0044] wherein, λ∈[0,1] is a distillation mixing coefficient;

[0045] The calculation expression of L LM is as follows:

[0046]

[0047] wherein, y * denotes a real target sequence corresponding to x, t denotes a sequence position index, denotes a mask function, and skips padding or unsupervised positions marked as -100.

[0048] Compared with the prior art, the application has at least the following advantages:

[0049] The contrastive adaptive knowledge distillation method proposed in the application can effectively improve the generation quality and stability of the student model, has significant performance advantages and practical application potential in the task of lightweight large model, and promotes the large-scale landing of general large model technology in resource-limited environments.

[0050] 1. Improve training stability and enhance the generalization ability of the student model.

[0051] The present application first introduces a double-branch contrastive loss function structure in the large language model distillation scene, and the teacher branch and the student branch correspond to "skewed KL divergence (SKL)" and "skewed reverse KL divergence (SRKL)", respectively. Through asymmetric modeling, the high-confidence area is strengthened and the low-confidence area is suppressed, effectively alleviating the problems such as gradient explosion and training shock in the low-confidence area in the traditional KL distillation. This mechanism makes the student model more easily capture the structural knowledge of the teacher model in the early training stage, and gradually improves its autonomous generation ability in the later training stage, thereby significantly improving the convergence, training stability and generalization performance of the distillation process as a whole.

[0052] 2. Significantly reduce the consumption of computing resources and optimize the training efficiency. In view of the problem that the large model teacher needs to be frequently called in the traditional online distillation method, the present application proposes an offline-online hybrid sampling mechanism, which combines the high-confidence response (TGOs) generated by the teacher offline once and the immediate response (SGOs) generated by the student online periodically, constructs a Replay Buffer buffer pool, and randomly extracts sample combinations of training data in the training process. On the premise of not sacrificing performance, the frequency of calling the teacher model is reduced by about 98%, and the computing resources are significantly saved. This mechanism cooperates with resource optimization technologies such as gradient accumulation and FP16 hybrid precision training, so that the whole distillation process can run efficiently in a single machine multi-card environment, the memory consumption can be controlled within 60GB, the training time is only increased by about 30% compared with the traditional KD, but the performance improvement is more than 9% (such as HumanEval).

[0053] 3. Dynamic adaptation ability, supporting task complexity self-adjustment. The present application proposes a dynamic loss weight scheduling mechanism based on the idea of curriculum learning, which dynamically adjusts the weight coefficient β in the contrastive loss through the training period (epoch) and step (step), thereby realizing the smooth transition process of "teacher guidance → student dominance". This strategy adaptively adjusts the learning pace according to the task difficulty and model performance, effectively reduces the risk of overfitting of the student model to the teacher in training, and improves its adaptability to long sequences, complex logic and low-confidence prediction areas. It is especially suitable for generation tasks such as code generation and function completion that require strict syntax structures.

[0054] 4. Strong structural versatility, easy to deploy and migrate. The present method is a modular distillation framework, which is independent of upstream and downstream tasks and model architectures, and can complete loss construction and training optimization based on the logarithmic probability distribution of model output. It can seamlessly connect with current mainstream language models (such as Qwen, LLaMA, Baichuan, etc.), and support open-source training frameworks (such as DeepSpeed, vL LM, FSDP) integrated deployment. At the same time, the core algorithm used in the application is platform-independent and can be deployed on general GPU servers or even part of the edge devices, greatly reducing the inference delay and deployment threshold while ensuring accuracy, providing a practical solution for the landing of general large models in enterprise-level, government affairs and privacy protection scenarios.

[0055] 5. Training data organization is more efficient, and the distillation target is more contrastive and different. By designing the data structure of "same input, different response" between the teacher output and the student output (TGOs vs. SGOs), a contrastive distillation sample triple is constructed across models and confidence distributions, so that each round of training not only focuses on imitating the teacher, but also explicitly models the student's bias and shortcomings, enhances the contrastiveness and directionality of the distillation process, and improves the targeting of knowledge transfer. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The figure is a schematic diagram of the overall framework of the inventive method.

[0057] Figure 2 The figure is a coding schematic for calling move_to_memory() to write to the Replay Buffer.

[0058] Figure 3 The figure is a coding schematic for the specific implementation of the contrastive distillation loss. DETAILED DESCRIPTION

[0059] The application will be further described in detail below.

[0060] The application proposes a large model lightweight distillation method for code generation tasks. Specifically, the overall architecture of the CAKD method includes four core modules: teacher branch, student branch, course-like hyperparameter scheduler, and offline-online hybrid sampling module (cooperating with Replay Buffer). The system takes the training corpus X in the form of prompt-response as input, generates response data through the teacher and student branches, and then realizes stable and efficient training of the student model through the specially designed contrastive loss function and dynamic hyperparameter adjustment strategy. Finally, the lightweight student model q θ .

[0061] Referring to Figures 1-3 , a contrastive adaptive knowledge distillation method for lightweight large language models includes the following steps:

[0062] S100: Select a general teacher model p and a general student model and p and Parameter initialization is performed; parameter initialization includes but is not limited to setting temperature T, skewness coefficient a0 and contrast weight b0; the teacher model and the student model use a unified vocabulary and a tokenizer, and all weights are converted into FP16 mixed precision when loaded to reduce memory usage and avoid convergence instability caused by inconsistent embedding layers; p and The parameter initialization includes: the value range of the distillation temperature T is 0.7-1.3, the value range of the skewness coefficient a0 is 0.1-0.4, and the value range of the contrast weight b0 is 0-0.3

[0063] The teacher model p and the student model Further comprising: the parameter size of the teacher model is not less than 5B, the parameter size of the student model is not greater than 2B, and the ratio of the parameter sizes of the two models is not higher than 0.3. The parameter size is used to form sufficient capacity difference between the two models and improve data contrast;

[0064] S200: constructing a training data set X, the training data set X includes code class prompts and natural language instruction prompts; in the training data set X, generally the same type of data, the proportion of code class prompts in X is not less than 60%, the proportion of natural language instruction prompts is not higher than 40%, and the length of a single prompt is not more than 256 tokens, so as to improve the pertinence and efficiency of the distillation process to the code generation task;

[0065] Input X into p, output the teacher high-confidence response set TGOs, and store TGOs as offline supervision data;

[0066] Freeze the initialization parameters, input from any selected training sample x in X, output the student immediate response y s and the corresponding logits data of y s , and write into the Replay Buffer; the corresponding teacher high-confidence response of y s is y t ;

[0067] Traverse all data in X to obtain the student immediate response corresponding to each training sample, and collect all student immediate responses to obtain SGOs;

[0068] S300: constructing a mixed training sample Y:

[0069] Randomly extract part of the student immediate responses from SGOs, all the extracted student immediate responses, the input x i corresponding to each of the extracted student immediate responses, and the teacher high-confidence response corresponding to each of the extracted student immediate responses form a triple All triplets form a mixed training sample Y, Y = (B1,..., B i ,..., B |Y| )}, where |Y| denotes the total number of data in Y;

[0070] vLLM is a prior art; using batch inference framework vLLM, one-time inference is performed in batch 64 to obtain teacher confidence Token and the log probability corresponding to the Token; selecting Token with confidence greater than 0.3 and the corresponding log probability as the teacher high-confidence response TGOs; storing TGOs in the form of "prompt, generated text, log-prob vector" as offline supervision data;

[0071] Set the sampling strategy to top-p (0.95) + temperature (1.0), and freeze the student model parameters θ e-1 during calculation; sampling inference is performed in batch 32-64; the generated y s and the corresponding logits have a byte length of no more than 128 tokens; y s and the corresponding logits are written to the Replay Buffer through the move_to_memory(·) interface, and the time stamp is recorded and managed in a first-in, first-out manner when writing;

[0072] S400: According to y t and the probability difference between y s , dynamically calculate: y t corresponds to the adaptive skew coefficient α s and y s corresponds to the adaptive skew coefficient α t , α t and α s are parameters in logits, and the calculation formula is as follows:

[0073]

[0074] where |p-q| = |p(y|x)-q θ (y|x)|, when |p-q| < 0.02, α t and α s are limited to 0.05-0.10; when |p-q| > 0.20, α t and α s are limited to 0.40; where m represents a normalization constant, generally taking a value of 0.5, and α0 represents the initial value of the adaptive skew coefficient;

[0075] Let p(y|x) be the conditional probability given by the teacher model p for the target token y given the input x; let qθ (y|x) is the student model q θ The conditional probability given to the same token y when the input x is given; let |p-q| = |p(y|x)-q θ (y|x)|, that is, the absolute value of the single-point confidence difference between the teacher and the student on this token, is used to characterize the sample difficulty; the skewing coefficient α of the teacher branch and the student branch is dynamically updated according to |p-q| t , α s ; α t and α s are parameters in logits, and logits data are output at the same time when the student instant response is output, and the parameters α t and α s are attached in logits;

[0076] An offline-online hybrid data sampling strategy and a Replay Buffer mechanism are designed, which can reduce the calling cost of the teacher model, effectively reduce the inference resource consumption and improve the diversity of training samples.

[0077] S500: Constructing a student model total loss function L total , L total The calculation expression is as follows:

[0078] L total =(1-λ)L LM +λL CAKD

[0079] Wherein, L LM represents a standard language modeling loss, L CAKD represents a contrastive distillation loss, and λ represents a weight parameter, generally taking 0.2-0.5;

[0080] The steps of constructing the student model total loss function L total in the S500 are as follows:

[0081] S510: Using the skewing KL divergence SKL to measure the high-confidence region of the teacher branch The calculation formula is as follows:

[0082]

[0083]

[0084] Wherein, represents the soft interpolation distribution of the teacher branch; α t ∈(0,1] is the skewing coefficient of the teacher branch, p(y t |x) is the conditional probability of the teacher model on token y t ; and qθ (y t ∣x) is the conditional probability of the student model on the same token y t .

[0085] S520: The skew reverse KL divergence SRKL is used to measure the tail probability of the student branch The calculation formula is as follows:

[0086]

[0087]

[0088] wherein, p s ∈(0,1] is the skew coefficient of the student branch, p s (y s ∣x) is the conditional probability of the teacher model on token y θ ; t (y s ∣x) is the conditional probability of the student model on the same token y

[0089] S530: The contrastive total loss function L CAKD is calculated, and the calculation formula is as follows:

[0090]

[0091] wherein, β represents the loss weight; generally, β ∈ [0,1];

[0092] To avoid numerical underflow, when calculating the logarithmic term of S510-S530, a smoothing term of 1×10 -6 is added to the probability, and all operations use the FP16 method, and FP16 represents the model calculation precision;

[0093] The calculation formula of β is as follows:

[0094]

[0095] wherein, e represents the current training cycle number, E represents the total cycle number, τ represents the step number in the current cycle, T represents the upper limit of the step number in the cycle, and β0 represents the initial loss weight; β0 is generally 0.05-0.30;

[0096] S540: L CAKD is constructed by combining L LM and the standard language modeling loss L total , and the calculation formula is as follows:

[0097] L total =(1-λ)L LM+λL CAKD

[0098] Where λ∈[0,1] is the distillation mixing coefficient. In the early stage of training, λ is usually set to ≤ 0.3 to focus on real label supervision. As the training progresses, λ can be slowly increased to a maximum of 0.7 to increase the weight of the CAKD contrast distillation signal.

[0099] L LM The calculation expression is:

[0100]

[0101] Among them, y * represents the true target sequence corresponding to x, t represents the sequence position index, represents a mask function that skips the padding or unsupervised positions marked as –100. LM Calculating cross-entropy for all unmasked tokens is a common indicator for language model training, which is used to ensure the basic generation correctness of the student model on the real label.

[0102] A dual-branch contrastive distillation structure is introduced to apply skewed KL divergence (SKL) and skewed reverse KL divergence (SRKL) to the high confidence region of the teacher model and the tail probability region of the student model, respectively, to construct an asymmetric loss function; at the same time, the loss weight coefficient is dynamically adjusted, so as to achieve a smooth transition from teacher guidance to student autonomous learning.

[0103] S600: Take Y as input and use L total , use AdamW optimizer to train the student model. When the training reaches the maximum number of iterations or the loss function reaches the minimum, stop training and get the trained student model The parameters of the AdamW optimizer are generally configured as follows: β1 = 0.9, β2 = 0.95, weight_decay = 0.1, and the learning rate is scheduled using cosine annealing with a linear warm-up in the first 500 steps.

[0104] S700: Select a model M to be used, and use the method described in S200-S600 to obtain a student model corresponding to M.

[0105] Example

[0106] The application of the CAKD method of the present invention in the code generation scenario is as follows according to the specific implementation method:

[0107] S101 initialization phase

[0108] Hardware Environment: 4x NVIDIA RTX 3090 (24GB) + Intel Xeon 4410x2 + 512GB RAM.

[0109] Software Dependency: PyTorch 2.3, DeepSpeed 0.14 (ZeRO-2+FP16), vLLM 0.4, Python 3.10.

[0110] Model Loading:

[0111] Teacher Model p: Qwen-2.5-Coder-7B (7B parameters).

[0112] Student Model q0: Qwen-2.5-Coder-1.5B (1.5B parameters).

[0113] Hyperparameter Initial Values (Recommended Value Range):

[0114] Temperature T = 1.0 (0.7 - 1.3);

[0115] Bias Coefficient a0= 0.2 (0.1 - 0.4);

[0116] Contrast Weight b0= 0.1 (0 - 0.3).

[0117] Training Corpus: WizardCoder (350K) + Evol-Instruct-Code (250K); prompt maximum length 128 tokens.

[0118] S102 Offline Teacher Output Generation

[0119] All prompts are batch-inferred by vLLM to obtain TGOs, and stored (log-prob + text). This process only occurs once.

[0120] S103 Batch Online Data Generation Stage

[0121] At the beginning of each epoch, freeze the current student weights e-1 , generate SGOs in batches of 32, call move_to_memory() as shown in the figure below to write to the Replay Buffer, see Figure 2 .

[0122] S104 Constructing a Mixed Training Sample Set

[0123] According to the random sampling of TGOs:SGOs = 1:1 to form a mini-batch B = {(x,y t ,y s )}, and send it to the forward calculation.

[0124] S105 adaptive skew coefficient update

[0125] For each (x, y t , y s ) in B, dynamically update according to formulas (1)-(2)

[0126] α t , α s ∈(0, 1]

[0127] where m = 0.5 is a normalization constant. If |p-q|<0.02, then α t is reduced to 0.05-0.1 to reduce overfitting of simple samples; otherwise, it is increased to 0.3-0.4.

[0128] S106 contrastive distillation loss calculation

[0129] The teacher branch calculates SKL, focusing on high-confidence tokens;

[0130] The student branch calculates SRKL, suppressing tail probabilities;

[0131] The specific implementation of cakd_loss is shown in Figure 3 ;

[0132] S107 dynamic scheduling of contrastive loss weight coefficient

[0133] Linearly increasing: β is increased from 0.1 to 0.4 at the 1st epoch and stabilized at 0.7 at the 3rd epoch, embodying the "teacher first, student second" teaching strategy.

[0134] S108 model parameter optimization update

[0135] DeepSpeed ZeRO-2 is used for backpropagation and synchronous update of θ. Single-step memory ≈19GB, and training 3 rounds of epochs takes 18h.

[0136] Experimental verification

[0137] To verify the effectiveness and practicality of the proposed contrastive adaptive knowledge distillation method (CAKD), two representative datasets, HumanEval and MBPP, are selected for evaluation in the code generation task. Two mainstream distillation methods, GKD and DistiLLM, are selected as the control group, and the student model uses Qwen2.5-Coder-1.5B.

[0138] Under the premise of keeping the experimental environment consistent - using NVIDIA RTX 3090 graphics card, uniform batch size and training parameter configuration, this experiment counted the video memory usage and training time of different methods in the complete training process, and the results are shown in Table 1:

[0139] Table 1 Resource consumption and training time comparison of different distillation methods

[0140] Training method GPU memory usage (unit: MB) Training time (unit: seconds) SFT 47062 1002 GKD 54737 2082 DistiLLM 54737 2055 CAKD (the present invention) 57809(↑6％) 2698(↑30％)

[0141] From the results, it can be seen that the video memory usage of CAKD increases only about 6% compared with GKD and DistiLLM, and the training time increases about 30%. The main reason for the slight increase in resources is that CAKD introduces an asymmetric double-branch loss function (SKL+SRKL) and a dynamic hyperparameter scheduling mechanism. However, due to the use of gradient accumulation and offline caching optimization methods, the overall resource consumption is still within a reasonable range.

[0142] To fairly evaluate the real effect of each method under the same time budget, this experiment conducted an "equal time training" experiment, which unified the training time of GKD and DistiLLM to 2698 seconds, the same as CAKD, and compared the pass@1 performance and precision loss amplitude of the three on HumanEval and MBPP. The results are shown in Table 2:

[0143] Table 2 Performance comparison of different methods under the same training time

[0144]

[0145]

[0146] The results show that the pass@1 of CAKD on the HumanEval dataset reaches 42.24%, which is 1.4 and 2.6 percentage points higher than GKD and DistiLLM, respectively; on MBPP, it also leads slightly. At the same time, the precision loss of the CAKD model is controlled at 29.3%, which is significantly better than GKD (37.0%) and DistiLLM (52.0%), indicating that it can more effectively preserve the predictive ability of the teacher model.

[0147] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and they should be included in the scope of the claims of the present application.

Claims

1. A lightweight, contrastive adaptive knowledge distillation method for large language models, characterized by: The steps include: S100: Select the general teacher model p and the general student model And for p and Initialize parameters; S200: Constructing a training dataset X, wherein the training dataset X includes code prompts and natural language instruction prompts; Input X into p, output the teacher’s high confidence response set TGOs, and store TGOs as offline supervision data; freeze Initialization parameters, select any training sample x from X and input Output is the student's immediate response y s and y s The corresponding logits data is written into the Replay Buffer; s The corresponding teacher's high confidence response is y t ; Traverse all the data in X, get the student's immediate response corresponding to each training sample, and collect all the student's immediate responses to get SGOs; S300: Construct mixed training sample Y: Randomly extract some students' immediate responses from SGOs, and the input x corresponding to all the extracted students' immediate responses and each of the extracted students' immediate responses i , and the teacher's high-confidence response corresponding to each student's immediate response constitutes a triplet All triples constitute mixed training samples Y, Y = (B1, ..., B i ,…,B |Y| )}, where |Y| represents the total number of data in Y; S400: According to y t with y s Dynamic calculation of probability difference: y t The corresponding adaptive skew coefficient α s and y s The corresponding adaptive skew coefficient α t , the calculation formula is as follows: Among them, α t and α s is the parameter in logits, |pq|=|p(y|x)-q θ (y|x)|, when |pq|<0.02, α t and α s Limit to 0.05–0.10; when |pq|>0.20, α t and α s The upper limit is 0.40; where m represents the normalization constant and α0 represents the initial value of the adaptive skew coefficient; S500: Construct the total loss function L of the student model total , L total The calculation expression is as follows: THE total =(1-λ)L LM +λL CAKD Among them, L LM represents the standard language modeling loss, L CAKD represents the contrastive distillation loss, and λ represents the weight parameter; S600: Take Y as input and use L total , use AdamW optimizer to train the student model. When the training reaches the maximum number of iterations or the loss function reaches the minimum, stop training and get the trained student model S700: Select a model M to be used, and use the method described in S200-S600 to obtain a student model corresponding to M.

2. The lightweight, contrastive, adaptive knowledge distillation method for large language models according to claim 1, characterized in that: In S100, the teacher model p and the student model are selected. It also includes: the parameter scale of the teacher model is not less than 5B, the parameter scale of the student model is not greater than 2B, and the ratio of the two parameters is not higher than 0.

3.

3. The lightweight, contrastive adaptive knowledge distillation method for large language models according to claim 2, characterized in that: The total loss function L of the student model is constructed in S500 total The steps are as follows: S510: Use skewed KL divergence SKL to measure the high confidence area of ​​the teacher branch The calculation formula is as follows: in, represents the soft interpolation distribution of teacher branches; S520: Using skewed reverse KL divergence SRKL to measure the student branch tail probability The calculation formula is as follows: in, represents the student branch soft interpolation distribution; S530: Calculate the total loss function L of the contrast formula CAKD , the calculation formula is as follows: Among them, β represents the loss weight; The calculation formula for β is as follows: Where e represents the current training cycle number, E represents the total number of cycles, τ represents the number of steps in the current cycle, T represents the upper limit of the number of steps in the cycle, and β0 represents the initial loss weight; S540: Joint L CAKD and the standard language modeling loss L LM Build L total , the calculation formula is as follows: THE total =(1-λ)L LM +λL CAKD Where λ∈[0,1] is the distillation mixing coefficient; L LM The calculation expression is: Among them, y * represents the true target sequence corresponding to x, t represents the sequence position index, represents a mask function that skips padded or unsupervised positions marked as –100.

Images