A method and system for selecting data for large language model recovery after pruning based on uncertain signals from student models.

Abstract

This invention discloses a method and system for selecting data for recovering a pruned large language model based on uncertainty signals in the student model. It solves the technical problem of low computational efficiency in the data selection stage caused by existing methods for recovering data from pruned large language models. The method includes acquiring a large-scale instruction fine-tuning dataset, dividing it into multiple capability clusters through S2RIC clustering, performing NLL sample-by-sample scoring in conjunction with the pruned student model, and generating a sample-NLL score mapping table corresponding to each capability cluster. An NTM noise token masking operation is performed based on a preset noise token masking threshold, producing masked sample data and corresponding score mapping tables. Then, IES sorting and sample subset selection are performed to obtain a recovery training subset. Finally, this subset is used to perform LoRA model recovery training on the pruned student model, ultimately outputting the recovered student model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for selecting data for recovering a large language model after pruning based on uncertain signals from a student model. Background Technology

[0002] In recent years, large language models based on the Transformer architecture have made groundbreaking progress in the field of natural language processing. Models such as the LLaMA (Large Language Model Meta AI) series and the GPT (Generative Pre-trained Transformer) series have demonstrated powerful text generation, reasoning, and task-solving capabilities. However, these models typically have billions or even hundreds of billions of parameters, resulting in a large demand for computational resources and memory during inference, which limits their deployment in resource-constrained environments such as edge devices and mobile terminals.

[0003] Therefore, model compression techniques have emerged, aiming to reduce the number of parameters and computational cost of a model without significantly sacrificing model performance. The main model compression techniques include: 1) Quantization: Convert model weights from high-precision floating-point numbers (such as 32-bit) to low-precision representations (such as 8-bit, 4-bit or even lower) to reduce storage and computational overhead.

[0004] 2) Knowledge Distillation: By having a small model (student model) mimic the output distribution or intermediate features of a large model (teacher model), knowledge from the large model is transferred to the small model.

[0005] 3) Pruning: Removes unimportant weights or structures from the model, including unstructured pruning (removing individual weights) and structured pruning (removing weights by entire rows / columns).

[0006] 4) Parameter-Efficient Fine-Tuning (PEFT): Methods such as LoRA (Low-Rank Adaptation) can adapt the model to a specific task by fine-tuning only a small number of parameters.

[0007] Structured pruning is widely studied and applied due to its ability to directly reduce model dimensionality and its hardware-friendly nature. However, pruning inevitably impairs model performance. To recover the performance of the pruned model, fine-tuning is usually required on a carefully selected subset of data; this is the "post-pruning recovery" problem.

[0008] Existing methods for data selection in large language model recovery after pruning employ JSD (Jensen-Shannon Divergence) as the scoring function. Calculating JSD requires simultaneously loading both the teacher model (the original large language model before pruning) and the student model (the large language model after pruning), performing two forward propagations for each sample (once for the teacher model and once for the student model), obtaining the output probability distributions of both models across the entire vocabulary, and then calculating the JSD value between the two distributions. This process makes the data selection stage the computational bottleneck of the entire pipeline, severely limiting the scalability of the method on large datasets and resulting in low computational efficiency in the data selection stage. Summary of the Invention

[0009] This invention provides a method and system for selecting data for large language model recovery after pruning based on uncertainty signals of student models, which solves the technical problem of low computational efficiency in the data selection stage caused by existing data selection methods for large language model recovery after pruning.

[0010] The first aspect of this invention provides a method for selecting large language model recovery data after pruning based on student model uncertainty signals, comprising: Obtain a large-scale instruction fine-tuning dataset, and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters; Perform NLL per-sample scoring on the multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster; Based on a preset noise token mask threshold, NTM noise token masking is performed on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster, and the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster are output. Perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset; The LoRA model is used to perform recovery training on the pruned student model using the recovered training subset, and the recovered student model is output.

[0011] Furthermore, the S2RIC clustering is performed on the large-scale instruction fine-tuning dataset to output multiple capability clusters, including: The input instructions of each sample in the large-scale instruction fine-tuning dataset are embedded and encoded using a pre-set S-BERT model, and the fixed-dimensional sentence embedding vector corresponding to each sample is output. Based on the fixed-dimensional sentence embedding vectors corresponding to each sample, the cosine similarity between each pair of samples is calculated, thereby constructing a sample similarity matrix; The sample similarity matrix is ​​smoothed using a preset diffusion kernel, and the smoothed similarity matrix is ​​output. The smoothed similarity matrix is ​​decomposed using a preset nonnegative matrix factorization algorithm, and a coefficient matrix is ​​output. Based on the membership degree of each sample in the coefficient matrix, each sample in the coefficient matrix is ​​assigned to the cluster with the highest membership degree, and multiple mutually exclusive capability clusters are output.

[0012] Further, the step of performing NLL per-sample scoring on the multiple capability clusters and the pruned student model, and outputting a sample-NLL score mapping table corresponding to each capability cluster, includes: The sample data from each of the capability clusters are input into the pruned student model for a single forward propagation process, and the conditional probability distribution of each of the sample data from each of the capability clusters is output for multiple time steps. Extract the correct reference token probability value from the conditional probability distribution at each time step; The negative logarithmic probability value of each time step is obtained by taking the natural logarithm of the correct reference token probability value in turn. The negative logarithmic probability values ​​of each time step corresponding to each sample data are summed and then divided by the length of the output sequence of the expected output sequence in the corresponding sample data to obtain the NLL score corresponding to each sample data. Based on the sample data and corresponding NLL scores in each capability cluster, construct a sample-NLL score mapping table for each capability cluster. The sample-NLL score mapping table retains the negative logarithmic probability component of each sample data at each time step.

[0013] Further, the step of performing NTM noise token masking on each capability cluster and the corresponding sample-NLL score mapping table based on a preset noise token masking threshold, and outputting the masked sample data and the masked sample-NLL score mapping table for each capability cluster, includes: Based on the expected output sequence of the sample data in each of the capability clusters, a concept consistency graph of each capability cluster is constructed. A graph structure analysis is performed based on the concept consistency graph of each capability cluster using a preset noise token mask threshold, and the noise token labeling results of each capability cluster are output. Based on the noise token marking results of each capability cluster, the noise tokens in the expected output sequence of the sample data in each capability cluster are masked to obtain the masked sample data corresponding to each capability cluster. Remove the negative log probability component of the time step corresponding to the noise token from each of the sample-NLL score mapping tables, and recalculate the sample NLL score based on the negative log probability component of the remaining time steps, and output the masked sample-NLL score mapping table corresponding to each capability cluster.

[0014] Further, the step of performing IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and outputting the recovered training subset, includes: The sample-NLL score mapping table after masking of each capability cluster is normalized according to the capability cluster dimension, and the normalized sample-NLL score mapping table of each capability cluster is output. Based on the normalized NLL scores in the normalized sample-NLL score mapping table of each capability cluster, the masked sample data in the corresponding capability cluster are sorted from high to low scores, and a sorted list of masked samples in each capability cluster is output. The sample quantity allocation is performed based on the importance weight of each capability cluster, and the number of samples to be selected for each capability cluster is output. From the masked sample sorting list within each capability cluster, select the corresponding target samples according to the corresponding number of samples to be selected, thereby constructing the recovery training subset; The importance weights are determined based on the NLL statistical distribution characteristics of each capability cluster or the capability priority of a preset model.

[0015] Further, the step of performing LoRA model recovery training on the pruned student model using the recovered training subset, and outputting the recovered student model, includes: After injecting a low-rank adaptation matrix of a preset rank into the attention layer and feedforward network layer of the pruned student model, the main parameters of the student model are frozen, the restored training subset is input into the student model, and only the parameters of the low-rank adaptation matrix of the preset rank are updated to obtain the intermediate model after training. The low-rank adaptation matrix of the intermediate model after training is merged with the main parameters of the pruned student model, or kept as an independent adapter module, and the restored student model is output.

[0016] The second aspect of this invention provides a data selection system for the recovery of large language models after pruning based on student model uncertainty signals, comprising: The acquisition module is used to acquire a large-scale instruction fine-tuning dataset and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters; The scoring module is used to perform NLL per-sample scoring on the multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster. The masking module is used to perform NTM noise token masking on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster based on a preset noise token masking threshold, and output the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster. The filtering module is used to perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset; The training module is used to perform LoRA model recovery training on the pruned student model using the recovered training subset, and output the recovered student model.

[0017] A third aspect of the present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above-described method for selecting large language model recovery data after pruning based on student model uncertainty signals.

[0018] The fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, it implements the above-described method for selecting large language model recovery data after pruning based on student model uncertainty signals.

[0019] The fifth aspect of the present invention provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the above-described method for selecting large language model recovery data after pruning based on student model uncertainty signals.

[0020] As can be seen from the above technical solutions, the present invention has the following advantages: The above-mentioned technical solution of the present invention provides a method for selecting data to recover large language models after pruning based on the uncertainty signal of the student model. It obtains a large-scale instruction fine-tuning dataset, performs S2RIC clustering on the dataset, and outputs multiple capability clusters; performs NLL per-sample scoring on the multiple capability clusters and the pruned student model, and outputs a sample-NLL score mapping table corresponding to each capability cluster; based on a preset noise token masking threshold, performs NTM noise token masking on each capability cluster and its corresponding sample-NLL score mapping table, and outputs the masked sample data and the masked sample-NLL score mapping table for each capability cluster; performs IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table for each capability cluster, and outputs a recovered training subset; uses the recovered training subset to perform LoRA model recovery training on the pruned student model, and outputs the recovered student model; based on the above solution, the present invention utilizes S2RIC clustering... The large-scale instruction fine-tuning dataset is divided into multiple capability clusters to avoid repetitive and redundant calculations on massive amounts of undifferentiated data. Secondly, NLL per-sample scoring relies solely on a single forward propagation of the pruned student model to quantify sample value based on model uncertainty, eliminating the need for additional models or multiple rounds of forward propagation or complex distribution difference calculations, significantly reducing the computational cost of the core scoring stage. Next, NTM noise token masking removes noise tokens from the expected output sequence of samples and simultaneously cleans up invalid score components, further reducing the amount of invalid data processed. Subsequently, IES sorting accurately sorts the masked samples based on the normalized NLL scores and allocates the number of samples by cluster, selecting only high-value samples to construct a restored training subset. Finally, LoRA model restoration training freezes the main parameters of the student model, updating only the low-rank adaptation matrix to restore model performance with minimal parameter updates, eliminating the need for iterative optimization of all model parameters and significantly reducing computational costs during the training phase. The overall process is optimized through a series of progressive steps: data clustering to reduce redundancy, single-model scoring to reduce overhead, noise filtering to reduce ineffectiveness, precise screening to improve efficiency, and lightweight training to save computing power. This systematically reduces ineffective computation and resource consumption in each stage of data selection, and significantly improves the overall computational efficiency of the data selection stage for restoring large language models after pruning. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1This is a flowchart of the steps of a method for selecting data to recover a large language model after pruning based on uncertainty signals of a student model, provided in Embodiment 1 of the present invention. Figure 2 A comparison diagram of the implementation process of the present invention and the prior art provided in Embodiment 1 of the present invention; Figure 3 This is an overview diagram of the PASER pipeline provided in Embodiment 1 of the present invention; Figure 4 This is a comparison chart of accuracy improvement provided in Embodiment 1 of the present invention; Figure 5 This is a time decomposition diagram provided in Embodiment 1 of the present invention; Figure 6-Figure 10 This is a confidence-quality analysis chart provided in Embodiment 1 of the present invention; Figure 11-Figure 14 This is a scatter plot of NLL and JSD provided in Embodiment 1 of the present invention; Figure 15 This is a rank correlation comparison diagram provided in Embodiment 1 of the present invention; Figure 16 This is a structural block diagram of a large language model recovery data selection system based on student model uncertainty signals provided in Embodiment 2 of the present invention. Detailed Implementation

[0023] This invention provides a method and system for selecting data for recovering a large language model after pruning based on the uncertainty signal of the student model, which solves the technical problem of low computational efficiency in the data selection stage caused by existing methods for recovering large language models after pruning.

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. It should be noted that in the optional embodiments of the present invention, the object information and other related data involved require the permission or consent of the object when the embodiments of the present invention are applied to specific products or technologies, and the collection, use, and processing of related data must comply with relevant laws, regulations, and standards. That is to say, if the embodiments of the present invention involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of relevant departments, and in compliance with relevant laws, regulations, and standards. If personal information is involved in the embodiments, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required, and the embodiments also need to be implemented with the authorization and consent of the object.

[0025] Terminology Explanation: LLM (Large Language Model) refers to a pre-trained language model based on the Transformer architecture with a huge number of parameters (usually billions to hundreds of billions).

[0026] NLL (Negative Log-Likelihood) measures the negative log-value of the probability that a model assigns to the correct output sequence. A higher value indicates that the model is less certain about that sample.

[0027] JSD (Jensen-Shannon Divergence) is a symmetric distance metric that measures the difference between two probability distributions.

[0028] PASER (Pruning-Aware Selection for Efficient Recovery) is a five-stage data selection framework in the prior art.

[0029] S2RIC (Sentence-BERT Similarity-based Refinement with Intra-similarityClustering) is the first-stage clustering method of the PASER framework.

[0030] NTM (Noisy Token Masking) is a mechanism in the PASER framework used to filter low-quality samples.

[0031] IES (Importance-Efficiency Sorting) is a mechanism in the PASER framework for sorting data by normalized scores and allocating budgets accordingly.

[0032] LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that reduces the number of trainable parameters through low-rank matrix factorization.

[0033] Singular Value Decomposition (SVD) is a matrix factorization technique often used in structured pruning.

[0034] PPL (Perplexity) is a metric that measures the predictive ability of a language model; a lower value indicates better model performance.

[0035] Spearman Spearman's Rank Correlation Coefficient is a nonparametric statistical measure of the correlation between two variables in terms of their ranking.

[0036] CM (Confident Misalignment) refers to the phenomenon where a student model assigns a high probability to a token but actually predicts it incorrectly.

[0037] S-BERT / Sentence-BERT (Sentence Bidirectional Encoder Representations from Transformers) is a pre-trained model used to generate sentence embedding representations.

[0038] pp (percentage point) is an absolute unit used to measure changes in metrics such as accuracy.

[0039] Structured pruning removes model weights according to a specific structure (such as an entire row or column), rather than randomly removing individual weights.

[0040] Instruction Tuning uses instruction-formatted data to fine-tune a large language model, making it better follow human instructions.

[0041] Forward propagation (also known as forward pass) is the computational process from input to output in a neural network.

[0042] A vocabulary is the set of all possible output tokens used by the model.

[0043] Reference Token: The correct output term at position t in the sample annotation.

[0044] A capability cluster is a subset of instruction data divided according to the model's capability dimension. Each cluster corresponds to a type of similar instruction capability.

[0045] Cohen's dz (Cohen's dz Effect Size) is a measure of paired sample effect size. The calculation formula is: ,in and These represent the mean of the pairwise difference between the two methods (e.g., NLL and JSD). The standard deviation of the paired differences is used to quantify the average performance improvement of NLL compared to JSD.

[0046] Statistical power is the probability that a statistical test will correctly reject the null hypothesis given a significance level and effect size. The higher the power value (usually ≥80%), the more reliable the experimental conclusion. This invention uses Cohen's dz as the effect size index to calculate statistical power.

[0047] T (Sequence Length) refers to the number of tokens in the sample output sequence, and is the time step factor in complexity analysis.

[0048] TV distance (Total Variation Distance) is a distance metric that measures the difference between two probability distributions.

[0049] The products in which the technology of this invention is applied include, but are not limited to: 1) Large Language Model Compression and Deployment System: Used to deploy structured pruned large language models in resource-constrained environments (such as edge computing devices, mobile terminals, and embedded systems) and restore model performance through efficient data selection strategies.

[0050] 2) Model Distillation and Fine-tuning Platform: Used to build the data selection module in the model knowledge distillation pipeline, especially in the teacher-student framework, to replace complex distribution difference measures and simplify the deployment process.

[0051] 3) Artificial Intelligence Model Optimization Service Platform: For enterprise-level model compression services, it provides a fast model recovery solution after pruning, reducing model inference latency and storage overhead while maintaining model inference capabilities.

[0052] 4) Automated model compression toolchain: Integrated into the automated machine learning (AutoML) process, it serves as a data subset selection component in the post-pruning recovery phase, reducing manual intervention and computational resource consumption.

[0053] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a method for selecting data to recover a large language model after pruning based on uncertainty signals in a student model, as provided in Embodiment 1 of the present invention.

[0054] This invention provides a method for selecting data for recovering a large language model after pruning based on uncertain signals from a student model, comprising: Step 101: Obtain a large-scale instruction fine-tuning dataset and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters.

[0055] Large-scale instruction fine-tuning datasets refer to collections containing a massive number of instruction-output pairs of samples, with a total of N or more. Each sample consists of a natural language input instruction and a tokenized expected output sequence. The input instructions cover various types of tasks such as question answering, translation, summarization, reasoning, and code generation. The expected output sequence is the standard and compliant target output text for the corresponding task. The data covers multiple fields, including general, industry, and professional fields, providing comprehensive, diverse, and high-quality basic training data support for instruction fine-tuning of large language models.

[0056] It should be noted that a large-scale instruction fine-tuning dataset is obtained, which contains multiple samples consisting of input instructions and expected output sequences. S2RIC clustering is performed on the large-scale instruction fine-tuning dataset, and multiple capability clusters are output through sentence embedding encoding, similarity construction, matrix smoothing decomposition, and sample membership assignment.

[0057] Further, step 101 may include the following sub-steps: S11. The input instructions of each sample in the large-scale instruction fine-tuning dataset are embedded and encoded using the pre-set S-BERT model, and the fixed-dimensional sentence embedding vector corresponding to each sample is output. S12. Based on the fixed-dimensional sentence embedding vectors corresponding to each sample, calculate the cosine similarity between each pair of samples, and thus construct a sample similarity matrix. S13. Use a preset diffusion kernel to smooth the sample similarity matrix and output the smoothed similarity matrix; S14. Use a preset non-negative matrix factorization algorithm to perform matrix factorization on the smoothed similarity matrix and output the coefficient matrix. S15. Based on the membership degree of each sample in the coefficient matrix, assign each sample in the coefficient matrix to the cluster with the highest membership degree, and output multiple mutually exclusive capability clusters.

[0058] It should be noted that, for the S2RIC clustering stage: this invention uses S-BERT embeddings and nonnegative matrix factorization to fine-tune the large-scale instruction dataset. Divide into K mutually exclusive capability clusters C1, C2, ..., C K , For the first i One sample data, For the first i A desired output sequence, N This represents the total number of samples.

[0059] Specifically, 1) each instruction sample in the large-scale instruction fine-tuning dataset is encoded into a fixed-dimensional dense vector representation (sentence embedding) using a pre-trained Sentence-BERT (S-BERT, sentence bidirectional encoder representation) model.

[0060] 2. Based on the sentence embedding vectors of all samples, calculate the cosine similarity between each pair of samples and construct a similarity matrix.

[0061] 3. Apply a diffusion kernel to smooth the similarity matrix to enhance the structural information of local neighborhoods, making similar samples more similar and dissimilar samples less similar.

[0062] 4. Use non-negative matrix factorization to decompose the smoothed matrix into the product of two non-negative matrices.

[0063] 5. Based on the membership degree of each sample in each cluster in the decomposition results, assign each sample to the cluster with the highest membership degree, forming the final K capability clusters C1, C2, ..., C6. K .

[0064] It is worth mentioning that NLL scoring can also be combined with other clustering methods: instruction clustering based on K-Means; clustering based on topic models (such as LDA); and manual grouping based on capability labels.

[0065] In this embodiment, a pre-defined S-BERT model is used to embed and encode the input instructions for each sample in the large-scale instruction fine-tuning dataset, converting natural language instructions into fixed-dimensional sentence embedding vectors that can represent semantic information. Based on the fixed-dimensional sentence embedding vectors corresponding to each sample, the cosine similarity between vectors is calculated to quantify the semantic similarity of the samples, thereby constructing a sample similarity matrix. A pre-defined diffusion kernel is used to smooth the sample similarity matrix, weakening random noise interference and strengthening the correlation features between similar samples, outputting a smoothed similarity matrix. A pre-defined non-negative matrix factorization algorithm is then used to further refine the smoothed matrix. The similarity matrix is ​​subjected to matrix factorization to mine the potential capability dimension features of the data and output a coefficient matrix. Based on the cluster membership values ​​corresponding to each sample in the coefficient matrix, the coefficient matrix is ​​traversed row by row to extract all cluster membership values ​​corresponding to a single sample. The target cluster corresponding to the maximum value is selected by numerical comparison, and the sample is assigned to the target cluster. After traversing all samples, the sample sets under each cluster are summarized, cross-cluster duplicate samples are removed, and multiple mutually exclusive capability clusters that are independent and do not overlap are output. This process achieves data grouping through semantic clustering, reduces the subsequent repeated calculation of the full data, and reduces the computational cost of the data selection stage.

[0066] Step 102: Perform NLL per-sample scoring on multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster.

[0067] It should be noted that NLL per-sample scoring is performed on multiple capability clusters and the pruned student model. Each capability cluster is traversed and the samples within the cluster are sequentially input into the pruned student model. The NLL score corresponding to each sample is calculated through forward propagation, and then the correspondence between the sample and the NLL score is established, and the sample-NLL score mapping corresponding to each capability cluster is output.

[0068] Furthermore, step 102 may include the following sub-steps: S21. Input multiple sample data from each capability cluster into the pruned student model and perform a single forward propagation to output the conditional probability distribution of multiple time steps corresponding to each sample data in each capability cluster. S22. Extract the correct reference token probability value from the conditional probability distribution at each time step; S23. Take the natural logarithm of the correct reference token probability value at each time step and then take the negative value to obtain the negative logarithmic probability value at each time step. S24. After summing the negative log probability values ​​of each time step corresponding to each sample data, divide by the length of the output sequence of the expected output sequence in the corresponding sample data to obtain the NLL score corresponding to each sample data. S25. Based on the sample data and their corresponding NLL scores in each capability cluster, construct a sample-NLL score mapping table for each capability cluster. S26, where the sample-NLL score mapping table retains the negative logarithmic probability component of each sample data at each time step.

[0069] It should be noted that this invention calculates a score for each sample in the dataset, reflecting the uncertainty of the student model. A higher score indicates greater value for the sample in retraining the student model. For the NLL scoring phase, for each cluster C... k Each sample in ( ),in y i =( y i,1 , y i,2 ,…, y i , Ti) Input the sample into the pruned student model M p Perform one forward propagation; for each time step t=1,2,… T i : Obtain the student model in the correct reference token y i,t conditional probability P Mp ( y i,t | y i,<t ,x i ); Calculate the NLL score for this sample: s N LL ( )=-(1 / T i ) .

[0070] Specifically, for a given sample (input instruction) x Expected output sequence y =( y 1, y 2,…, y T )),in T The length of the output sequence. y t Indicates the first t The correct reference token at each time step.

[0071] Subsequently, the sample ( x,yInput the pruned student model M p Perform one forward propagation. During the forward propagation, record each time step. t ( t =1,2,…, T The conditional probability distribution of the model output. P Mp ( y i,t | y i,<t ,x i ), that is, given an input instruction x And all the previous correct output tokens y 1, y 2,…, y t-1 Under the given conditions, the model predicts the probability distribution of the next token. From each time step... t Extract the correct reference token from the probability distribution. y t Corresponding probability value P Mp ( y t | y <t ,x).

[0072] Finally, for each time step t Calculate the negative logarithm of the probability of the correct token: -log P Mp ( y t | y <t The final NLL score for a sample is obtained by averaging the negative logarithmic values ​​of all time steps (x, y). s N LL =-(1 / T ) .

[0073] in: T The length of the sample output sequence (number of tokens); y t For the first t The correct reference token at each time step; y <t For the first t The sequence of all correct output tokens prior to time step n, i.e. ( y 1, y 2,…, y t-1 ),x For input instructions, M p The student model after pruning; P Mp ( y t | y <t x) represents the student model given input. x and historical output y <t Under the condition of predicting the correct token y t The conditional probability; log It is the natural logarithm.

[0074] It is worth mentioning that the key differences between the NLL scoring in this invention and the existing JSD scoring (including complexity analysis of the T factor) are shown in Table 1: Table 1

[0075] Among them, | D | represents the dataset size. T The average sequence length, F f w.d. For the computational cost of a single forward propagation, | V | represents the vocabulary size. The complexity of the entire JSD dataset. O (2| D |· T · F f w.d. +| D |· T ·| V In |), the first item O (2| D |· T · F f w.d. ) One forward propagation from both the teacher and student models (each propagation round contains T (time step), second item O (| D |· T ·| V |) From the vocabulary list V Calculate the JSD for the complete probability distribution. The complexity of NLL across the entire dataset. O (| D |· T · F f w.d.It only involves a single forward propagation from the student model and does not require traversing the entire vocabulary.

[0076] Furthermore, in addition to NLL, other metrics that rely solely on the student model output and reflect model uncertainty can be used as scoring functions, such as entropy: the entropy value of the student model output distribution. H ( P Mp ( | y <t ,x))= As a measure of uncertainty, a higher entropy value indicates greater uncertainty about the model's output. Compared to NLL, entropy considers the distribution shape across the entire vocabulary, thus potentially capturing richer structural information, but its computational complexity is correspondingly higher (requiring traversal of all lexical units in vocabulary V). v The computational complexity of entropy at each time step is O(|V|), and the complexity for the entire dataset is O(|V|). O (| D |· T ·| V |), higher than NLL O (2| D |· T · F f w.d. (but still lower than JSD) O (2| D |· T · F f w.d. +| D |· T ·| V |); Max Probability Gap: The difference between the maximum probability of the student model and the probability of the correct token is used as a measure of uncertainty. Specifically, for each time step... t ,calculate gap t = Then, the average is taken to obtain the sample score. The larger the difference, the greater the uncertainty of the model regarding the correct output. This method only needs to find the probability of the word with the highest probability and the probability of the correct word, and the computational complexity is between NLL and entropy; Predictive Confidence: directly uses the probability value of the student model for the correct output. As a measure of certainty, the score is converted to a measure of uncertainty by taking its reciprocal or a negative value. s c onf =-(1 / T ) This method is similar to NLL, but it performs a linear rather than logarithmic transformation on the probability values.

[0077] Secondly, a scoring function based on the perplexity ratio can be constructed: calculating the student's perplexity PPL(x) in the "conditional mode" (given input instruction x) and in the "reference mode" (given input x and correct output)... y <t The perplexity PPL(x,y) is used as the perplexity ratio IFD = PPL(x) / PPL(x,y) as a measure of sample difficulty. The higher the IFD value, the more difficult the sample is for students, i.e., the more valuable it is. Compared with NLL, IFD makes additional use of perplexity information in the conditional pattern, but requires two forward propagations (one for the conditional pattern and one for the reference pattern), and its complexity is between that of NLL and JSD.

[0078] Furthermore, gradient information can be used to construct a scoring function: for each sample, calculate the loss gradient of the student model on that sample. Using gradient norm || The inner product of the gradient and a reference direction can be used as a measure of sample importance. A sample with a large gradient norm indicates that the student model has a stronger need for parameter updates on that sample, i.e., it is more "confused" about that sample. Compared with NLL, the gradient similarity method has higher computational complexity (requiring backpropagation to calculate the gradient), but it can capture more fine-grained model uncertainty information.

[0079] Furthermore, NLL is combined with other scoring signals to construct a hybrid scoring function: NLL + intra-cluster frequency weight: The NLL score is multiplied by the frequency weight of the sample within its ability cluster, giving higher scores to low-frequency but important samples; NLL + sample length normalization: The NLL scores of samples of different lengths are normalized according to length to avoid long sequence samples having higher scores due to more accumulated terms; NLL + hard example mining: A threshold is set for the NLL score, and only samples exceeding the threshold are further analyzed, or combined with other hard example mining strategies.

[0080] Meanwhile, on large-scale datasets, the NLL scoring process can be distributed across multiple computing nodes for parallel execution. Each node is responsible for processing a subset of the dataset, and the scoring results from all nodes are finally aggregated. Since NLL scoring does not involve the teacher model, each node only needs to load the student model to work independently, facilitating distributed deployment.

[0081] It is worth noting that the NLL scoring method of this invention can be applied not only to recovery after structured pruning, but also extended to: recovery after unstructured pruning: for model recovery after unstructured pruning (removing individual weights instead of entire rows / columns), NLL scoring can also be used for data selection; recovery after quantization: for models whose performance degrades after weight quantization (such as 4-bit or 8-bit quantization), NLL scoring can be used to select a subset of data for quantization-aware fine-tuning; recovery after joint compression: for models that have undergone multiple compression operations such as pruning and quantization, NLL scoring can also be used for data selection.

[0082] In this embodiment, multiple sample data from each capability cluster are input one by one into the pruned student model. Only one forward propagation is performed, and the model outputs the conditional probability distribution step by step according to the token generation sequence. The probability value corresponding to the correct reference token is accurately extracted from the conditional probability distribution of each time step. The probability value of the correct reference token at each time step is then negatively calculated by performing the natural logarithm operation on the probability value of the correct reference token at each time step to obtain the negative logarithmic probability corresponding to each time step. The negative logarithmic probabilities of a single sample at all time steps are summed and then divided by the length of the expected output sequence of the sample. After normalization, the NLL score corresponding to the sample is obtained. Each sample and its corresponding NLL score are summarized according to the capability cluster dimension to construct a sample-NLL score mapping table. The negative logarithmic probability component of each sample at each time step is fully preserved to support subsequent processing. This process only relies on a single forward propagation of the pruned student model to complete the sample uncertainty quantification, without the need for additional model participation or complex distribution calculations, effectively reducing the computational overhead of the data selection stage.

[0083] Step 103: Based on the preset noise token mask threshold, perform NTM noise token masking on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster, and output the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster.

[0084] It should be noted that, based on the preset noise token masking threshold, a concept consistency graph is first constructed for each capability cluster. Then, noise tokens are identified and marked through graph structure analysis. Subsequently, the noise tokens in the sample data are masked. Simultaneously, the negative log probability component corresponding to the noise token is removed from the sample-NLL score mapping table, and the sample NLL score is recalculated. Finally, the masked sample data and the masked sample-NLL score mapping table are output.

[0085] Furthermore, step 103 may include the following sub-steps: S31. Based on the expected output sequence of each sample data in each capability cluster, construct a concept consistency diagram for each capability cluster; S32. Perform graph structure analysis based on the concept consistency graph of each capability cluster using a preset noise token mask threshold, and output the noise token labeling results of each capability cluster. S33. Based on the noise token marking results of each capability cluster, the noise tokens in the expected output sequence of the sample data in each capability cluster are masked to obtain the masked sample data corresponding to each capability cluster. S34. Remove the negative log probability component of the time step corresponding to the noise token from each sample-NLL score mapping table, and recalculate the sample NLL score based on the negative log probability component of the remaining time steps, and output the masked sample-NLL score mapping table corresponding to each capability cluster.

[0086] Noise tokens belong to the lexical units within the expected output sequence of a single sample. The semantic features of this type of token deviate from the core semantics of the overall capability cluster to a preset judgment threshold, and cannot carry effective task expression information. Not only will it destroy the semantic integrity of the sample itself, but it will also interfere with the calculation of the sample uncertainty score and the subsequent sample screening judgment. It is an invalid lexical unit that needs to be filtered out.

[0087] The noise token labeling result is a set of identification information formed by the statistical summary of a single capability cluster. The content includes the corresponding capability cluster affiliation information, associated sample number, time step position of the noise token in the output sequence, specific token content, and semantic deviation judgment criteria. It can clearly indicate the specific distribution and judgment reasons of all invalid tokens to be masked, providing accurate reference for subsequent sequence masking and fraction component removal.

[0088] It should be noted that, for the NTM noise token masking stage, this invention applies noise token masking to each cluster to filter low-quality tokens and prevent noise interference from resuming training. Specifically, this invention constructs a Conceptual Consistency Graph for each capability cluster, identifies noise tokens that are semantically inconsistent with other tokens within the cluster through graph structure analysis, and masks these noise tokens in subsequent processing.

[0089] Specifically, based on the expected output sequence corresponding to each sample within each capability cluster, the sequence is first split into discrete independent token units. The semantic association between pairs of tokens is then calculated using semantic representation. Using individual tokens as graph nodes and semantic associations between tokens as node connections, a concept consistency graph corresponding to each capability cluster is constructed layer by layer. A preset noise token mask threshold is used as the criterion. All nodes in the concept consistency graph are traversed one by one, and the degree of fit between each token node and the mainstream semantic nodes within the cluster is compared. Tokens with semantic deviations exceeding the threshold are filtered out. After regularization and summarization, the noise token labeling results for each capability cluster are obtained. The expected output of each sample is then accurately located based on the labeling results. The system identifies the location of noisy tokens in the sequence and performs masking on these tokens to remove semantically invalid content, resulting in masked sample data. Simultaneously, based on the correspondence between time steps and tokens, the negative logarithmic probability components corresponding to the time periods of the noisy tokens are removed from the sample-NLL score mapping table. The remaining valid probability components are accumulated and statistically analyzed, and then re-divided evenly based on the actual remaining valid sequence length after noise removal. The NLL score values ​​corresponding to the samples are updated, and finally, the masked sample-NLL score mapping table after processing for each capability cluster is output. This process removes interfering noise content, compresses the actual data size to be processed, reduces the amount of invalid computation in subsequent sorting and filtering processes, and effectively alleviates the problem of high computational consumption in the data selection stage.

[0090] Step 104: Perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset.

[0091] It should be noted that IES sorting and sample subset selection are carried out based on the masked sample data processed by each capability cluster and the matching masked sample-NLL score mapping table. First, the score normalization operation is completed on a single capability cluster basis. The samples within the cluster are sorted and arranged in sequence according to the normalized scores. The size of the selectable samples for a single cluster is determined by combining the importance weights of each capability cluster. High-quality samples are selected and integrated according to the determined number, and finally combined to generate the restored training subset.

[0092] Furthermore, step 104 may include the following sub-steps: S41. Normalize the sample-NLL score mapping table after masking for each capability cluster according to the capability cluster dimension, and output the normalized sample-NLL score mapping table for each capability cluster. S42. Based on the normalized NLL scores in the normalized sample-NLL score mapping table of each capability cluster, sort the masked sample data in the corresponding capability cluster from high to low scores, and output the sorted list of masked samples in each capability cluster. S43. Allocate the number of samples according to the importance weight of each capability cluster, and output the number of samples to be selected for each capability cluster. S44. From the masked sample sorting list within each capability cluster, select the corresponding target samples according to the corresponding number of samples to be selected, thereby constructing the recovered training subset; S45. The importance weight is determined based on the NLL statistical distribution characteristics of each capability cluster or the capability priority of the preset model.

[0093] It should be noted that within each cluster, samples are normalized and sorted according to NLL scores; based on data budget... B Samples are selected from each cluster to form a recovery training subset. D s elected .

[0094] Specifically, the present invention performs efficiency normalization on the obtained NLL scores by cluster, sorts the normalized NLL scores from high to low within each cluster, allocates the number of samples according to the importance weight of each cluster based on the preset data budget, and selects a corresponding number of samples from each cluster to form the final recovery training subset.

[0095] In this invention, by replacing the JSD score with the NLL score, the meaning of the NLL score changes (from "distribution difference" to "uncertainty"), but the direction of the score remains consistent with the JSD—a higher score indicates a more "valuable" sample. Therefore, the specific implementation of the IES ranking (sorting by score from high to low) remains unchanged.

[0096] In this embodiment, each capability cluster is defined as a separate computational unit. All score data from the masked sample-NLL score mapping table of each cluster are extracted. Normalization rules are used to correct differences in score intervals, eliminating evaluation scale bias caused by data characteristics of different clusters. After unifying the cross-cluster score comparison standard, a normalized sample-NLL score mapping table for each capability cluster is generated. The standardized scores stored in the normalized mapping table are retrieved, and the normalized NLL scores of all masked samples within the cluster are compared one by one. The samples are rearranged according to a decreasing score value rule, and the sorted results are used to obtain the masked samples corresponding to each capability cluster. The system sorts the NLL scores of individual clusters and analyzes their distribution range and discrete characteristics to obtain statistical distribution features. It then combines this with a pre-defined model capability priority level to calculate the importance weight of each capability cluster. The overall sample quota is then allocated according to the weight ratio to determine the total number of samples selectable for each cluster. Based on the determined selection quantity, target samples are sequentially extracted from the high-ranking end of each cluster's sorting list. All samples selected from all clusters are then aggregated and integrated to form a final training subset. This screening method accurately selects high-quality, effective samples while discarding low-value, useless samples, thus reducing the data processing scale of subsequent steps.

[0097] Step 105: Use the restored training subset to perform LoRA model restoration training on the pruned student model, and output the restored student model.

[0098] It should be noted that the selected recovered training subset is used as training material, and the pruned student model is iteratively trained using a low-rank adaptation fine-tuning method. The model's inference performance is corrected by using high-quality samples within the subset. After completing the training process, a recovered student model with fully repaired performance is obtained.

[0099] Furthermore, step 105 may include the following sub-steps: S51. After injecting a low-rank adaptation matrix of a preset rank into the attention layer and feedforward network layer of the pruned student model, freeze the main parameters of the student model, input the restored training subset into the student model, and only update the parameters of the low-rank adaptation matrix of the preset rank to obtain the intermediate model after training. S52. Merge the low-rank adaptation matrix of the intermediate model after training with the main parameters of the pruned student model, or keep them as independent adapter modules, and output the restored student model.

[0100] It should be noted that this invention uses a subset of data. D s elected right M p Perform LoRA fine-tuning and output the restored model. M p '.

[0101] Specifically, this invention uses a selected subset of data to fine-tune and restore the pruned student model: a low-rank adaptation matrix is ​​injected into the Transformer layers (including attention layers and feedforward network layers) of the pruned model, with the rank r typically set to 8 or 16. Then, standard autoregressive language model training is performed using the selected restored training subset, with the training objective being to minimize the cross-entropy loss. During training, only the parameters of the low-rank adaptation matrix are updated, while the parameters of the main model remain frozen. After training, the low-rank matrix can be merged with the main model or kept as an independent adapter module for inference.

[0102] It is worth mentioning that NLL scoring can also be combined with other efficient parameter fine-tuning methods: prefix tuning; prompt tuning; adapter layers; and full fine-tuning, when computational resources allow.

[0103] In this embodiment, the attention layer and feedforward network layer in the internal structure of the pruned student model are accurately located. Low-rank adaptation matrices are embedded in the corresponding network layer structures according to a pre-set rank dimension specification. After the matrix is ​​mounted, all the original main parameters of the model are locked, constraining these parameters to not change in value in subsequent calculations. The integrated restored training subset is imported into the model in batches for calculation derivation. Based on the backpropagation path of the error generated by the calculation results, iterative optimization is continuously performed only on the embedded preset rank low-rank adaptation matrix parameters. After the parameters tend to stabilize, the trained intermediate model is generated. The processing method is selected according to the actual application deployment scenario. The low-rank adaptation matrix with the completed parameter tuning can be integrated with the original main parameters of the model, or the low-rank adaptation matrix can be kept separately as an external independent adapter module. Finally, the restored student model is output. This training mode only updates and optimizes a small number of low-rank matrix parameters, without iterative calculations on a massive number of main parameters of the model, which greatly reduces the computational resource consumption of the model restoration process.

[0104] As a comparison of technical effectiveness, existing techniques can be used as a reference. In the post-pruning recovery process, data selection plays a crucial role. Research shows that not all training samples contribute equally to model recovery: some samples provide key recovery signals, while others may contain noise or redundant information, or even interfere with the recovery process. Therefore, selecting the most valuable subset from a large-scale instruction fine-tuning dataset for recovery training becomes key to improving recovery efficiency and effectiveness.

[0105] The core challenges of data selection include: 1) Design of indicator selection: How to quantify the "value" of each sample to the model recovery.

[0106] 2) Computational efficiency: The cost of calculating the selected metrics on a large-scale dataset.

[0107] 3) Selection bias: Whether the selection method systematically favors certain types of samples while ignoring the samples that are truly needed.

[0108] 4) Adaptation to pruning scenarios: Whether the selected method is robust to different pruning ratios and model architectures.

[0109] Based on the above, the present invention mainly solves the following technical problems: 1) The problem of low computational efficiency in the data selection stage of existing technologies: The existing PASER framework uses JSD (Janssen-Shannon divergence) as the scoring function in the second-stage sample-by-sample scoring. Calculating JSD requires simultaneously loading both the teacher model (the original large language model before pruning) and the student model (the large language model after pruning), performing two forward propagations for each sample (once for the teacher model and once for the student model) to obtain the output probability distributions of both models across the entire vocabulary, and then calculating the JSD value between the two distributions. This process makes the data selection stage the computational bottleneck of the entire pipeline, severely limiting the scalability of the method on large-scale datasets.

[0110] 2) Performance crashes of existing technologies under aggressive pruning scenarios

[0111] When the model pruning ratio is high (e.g., 50% structured pruning), the existing JSD scoring method leads to a sharp deterioration in the perplexity (PPL) of the recovered model (experimental observation value as high as 11,435.67), meaning that the model experiences a severe performance collapse after recovery. This phenomenon stems from the fact that JSD's symmetric midpoint averaging mechanism introduces a systematic selection bias when the distributions of the teacher and student models differ too much, selecting inappropriate samples for recovery training.

[0112] 3) Existing technologies suffer from sample quality problems due to selection bias.

[0113] The JSD scoring method tends to select samples where the student model appears confident but actually makes incorrect predictions (i.e., high-confidence error samples where the student model is "confident but wrong"). These samples can mislead the recovery fine-tuning process, causing the model to solidify in the wrong direction and reducing recovery performance.

[0114] 4) Reduce the dependence of the model recovery process on the teacher model after pruning.

[0115] The existing PASER framework's entire data selection process heavily relies on the existence of the teacher model. This means that during post-pruning recovery, the original, unpruned teacher model must always be maintained, increasing deployment costs and system complexity. This invention aims to reduce reliance on the teacher model by using only the uncertainty signals from the student model itself for data selection.

[0116] Specifically, the first drawback is the huge computational overhead (two forward propagations + calculation of the complete vocabulary).

[0117] The existing PASER framework uses JSD scoring in its second stage, which suffers from serious efficiency issues: It requires two forward propagations: for each sample, a complete forward propagation must be performed separately for both the teacher and student models (each propagation consists of T time steps) to obtain the complete output probability distributions of both models at each time step. It requires calculating the distribution over the entire vocabulary: JSD involves operating on the probability distributions of all words in the vocabulary V. For large language models, the vocabulary size is typically 32,000 to over 100,000 words, meaning that tens of thousands of dimensional probability vectors need to be processed at each time step. It requires loading both models simultaneously: during the scoring process, the teacher and student models must reside in memory (or GPU memory) simultaneously, creating enormous memory pressure for models with billions of parameters. Time cost: the combination of these factors makes the data selection stage the bottleneck of the entire pipeline. Specifically, JSD scoring on the entire dataset... D The computational complexity of the above is O (2| D |· T · F f w.d. +| D |· T ·| V |), where | D | represents the dataset size. T The average sequence length, F f w.d. For the computational cost of a single forward propagation, | V | represents the vocabulary size. This complexity severely limits the scalability of the method on large-scale datasets.

[0118] Disadvantage 2: The symmetrical midpoint averaging mechanism of JSD introduces selection bias—which has a quantitative lower bound.

[0119] The calculation of JSD involves the midpoint average of two distributions. M=(P Mo +P Mp ) / 2. This midpoint averaging mechanism introduces systematic selection bias when there is a large difference in teacher-student distribution, and this bias has a strict quantitative lower bound: Lemma 1 (Midpoint Bias): Let the teacher model distribution be... P Mo The student model distribution is P Mp The total variation distance between the two is || P Mo - P Mp || TV =ε. Then the midpoint is evenly distributed. M Teacher model distribution P Mo The total variation distance between them is: || M - P Mo || T V = e / 2; Proof Summary: By the definition of midpoint average M =( P Mo + P Mp ) / 2, which can be directly calculated to get || M - P Mo || TV =||( P Mp - P Mo ) / 2|| TV =(1 / 2)|| P Mp - P Mo || T V = e / 2.

[0120] This lemma shows that when the teacher-student distribution difference is... e At that time, the average of the midpoint M The distance from the correct reference distribution of the teacher model is exactly [missing information]. e / 2. Distribution of student models under aggressive pruning scenarios (e.g., 50% structured pruning). P Mp Distribution of the teacher model P Mo The difference is extremely large (i.e.) e (Very large), at this time the midpoint average MIt deviates significantly from the correct distribution of the teacher model, becoming an unreliable reference point.

[0121] Theorem 1 (Quantitative Lower Bound of JSD Selection Bias): Let the loss function for regression training be... L r ec When using JSD scoring for data selection, the expected loss for resuming training satisfies the lower bound: E [ L r ec ]≥ c · e ² / (2+2 e ); in c For positive constants related to the task, e =|| P Mo - P Mp || T V The total variation distance of the teacher-student distribution.

[0122] Proof Summary: By Lemma 1, the midpoint average M The distance from the teacher distribution is e / 2. The symmetry of JSD allows the teacher model to... M divergence and student model direction M The divergence is treated equally. When the distribution differences are large, the JSD score systematically favors samples that result in larger JSD values, and the expected recovery loss for these samples has the aforementioned lower bound. Specifically, by Pinsker's inequality and the definition of JSD, JSD( P || Q )≥(1 / 2)|| P - Q ||² T V Combining Lemma 1 e / 2 deviation, after expectation calculation and algebraic simplification, yields: E [ L r ec ]≥ c · e ² / (2+2 e ).

[0123] The physical meaning of this theorem is: selection bias is proportional to the square of the distribution difference; when the teacher-student distribution difference ε increases (such as in aggressive pruning scenarios), the selection bias introduced by JSD increases. e ² / (2+2 e The increased rate of pruning leads to a higher lower bound on the expected loss during recovery training. The performance crash at high pruning ratios can be theoretically explained: when pruning is 50%, e The lower bound given by Theorem 1 indicates that the recovery loss cannot be lower than a certain positive number, which explains the performance crash observed in the experiment where the perplexity spiked to 11,435.67. JSD symmetry is an unavoidable theoretical flaw: regardless of the specific implementation, JSD's symmetric midpoint averaging mechanism inevitably introduces at least [a certain number] when the distribution differences are large. e The deviation of 2 / 2 is an inherent theoretical flaw in the JSD method.

[0124] Disadvantage 3: Performance crashes under aggressive pruning.

[0125] Experimental observations show that when 50% structured pruning is applied, the perplexity (PPL) of the model after data selection and restoration using JSD scoring soars to 11,435.67 (the normal value should be in the range of 10-20), indicating that the model has completely lost its language ability. Combined with the theoretical analysis of Theorem 1, the fundamental reason for this phenomenon is the difference in teacher-student distribution under a high pruning ratio. e The JSD midpoint averaging mechanism is introduced to maximize this. e The systematic bias of / 2 leads to the selection of inappropriate samples for recovery training, causing the recovery loss to meet the requirements. E [ L r ec ]≥ c · e ² / (2+2 e The lower bound of ) cannot achieve effective recovery.

[0126] Disadvantage 4: Bias towards selecting "confident but wrong" samples.

[0127] Confidence-Mass Analysis (CM analysis) reveals that the Judgment Scale (JSD) score tends to select samples that the student model assigns high probabilities to but actually predicts incorrectly. The high JSD score for these samples is due to the stark contrast between the student model's confidence (peak distribution) and the teacher model's correct distribution. However, using these samples for retraining reinforces the student model's incorrect beliefs rather than correcting its errors.

[0128] Disadvantage 5: Strong dependence on the existence of the teacher model

[0129] JSD scoring requires the participation of a teacher model, meaning that in actual deployments, even after pruning, the original teacher model must still be retained only for data selection. This increases system complexity and resource overhead, and makes it unusable in certain scenarios (such as when the original teacher model is unavailable or has been deleted to save space).

[0130] In view of the shortcomings of the prior art, the objectives of the present invention include: 1. Improve computational efficiency in the data selection phase: By calculating scores using only a single forward propagation of the student model, the dependency on the teacher model is eliminated, increasing data selection speed by approximately 2.25 times. The computational complexity of NLL scoring is... O (| D |· T · F f w.d. Compared to JSD O (2| D |· T · F f w.d. +| D |· T ·| V |) significantly reduced.

[0131] 2. Eliminate selection bias introduced by the symmetric midpoint averaging of JSD: Employ an asymmetric one-sided uncertainty measure (NLL) to avoid the systematic bias of the symmetric averaging mechanism when distribution differences are too large (Theorem 1 shows that the lower bound of the JSD bias is...). e / 2).

[0132] 3. Improve recovery robustness under aggressive pruning scenarios: Ensure that the recovered model maintains stable performance under 50% equal-height pruning, avoiding perplexity collapse. NLL does not involve midpoint averaging, therefore it is not subject to the lower bound in Theorem 1. E [ L r ec ]≥ c · e ² / (2+2 e () constraints.

[0133] 4. Avoid selecting “confident but wrong” samples: Select samples where the student model is truly “confused” rather than “confident but wrong” samples by directly measuring the uncertainty of the student model (rather than teacher-student differences).

[0134] 5. Simplify deployment process: Remove the dependency on the teacher model, so that the recovery process after pruning can be completed independently by only maintaining the student model.

[0135] like Figure 2 As shown, the core idea of ​​this invention is to replace the scoring function in the second stage of the five-stage pipeline of the PASER framework with JSD (Jensen-Shannon divergence), which requires the participation of both teacher and student models, with NLL (Negative Log-Likelihood), which only relies on the student model for forward propagation. This significantly improves data selection efficiency while maintaining or improving the recovery effect and eliminates the selection bias problem of JSD under high pruning ratios.

[0136] The theoretical basis for NLL as an uncertainty measure is that NLL directly measures the overall uncertainty of the student model for the correct output sequence. The higher the NLL value, the more uncertain the student model is about the correct output of the sample. That is, the sample contains more knowledge that the student model has not yet mastered. Therefore, including these samples in the recovery training can more effectively make up for the knowledge loss caused by pruning.

[0137] Furthermore, the effectiveness of choosing NLL as an alternative scoring function in this invention is based on the following technical principles: 1. The principle of uncertainty as information content: Non-Limited Range (NLL) measures the uncertainty of the student model regarding the correct output. Samples with high NLL values ​​are precisely those samples where the student model is "least confident," containing key knowledge lost during pruning. Including these samples in the recovery training can maximize the compensation for the knowledge gaps caused by pruning.

[0138] 2. Avoid the "Confident but Wrong" Trap: Unlike JSD, NLL does not rely on the distribution of the teacher model. JSD tends to select samples where the student model is confident but makes incorrect predictions (because the peak distribution of students differs most from the teacher's correct distribution in these samples), while NLL selects samples where the student model is truly uncertain—these samples often contain knowledge that the student model has not yet mastered and are more suitable for recovery training.

[0139] 3. Robustness when distribution differences are too large—Theoretical guarantee of Theorem 1: When the pruning ratio is very high, the distribution differences between the teacher model and the student model are extremely large. The symmetric midpoint average of JSD will produce a misleading intermediate distribution in this case (Lemma 1 shows that the midpoint M deviates from the teacher distribution by a distance of...). e / 2), which causes the selection bias to satisfy the lower bound of Theorem 1. E [ L r ec ]≥ c · e ² / (2+2 eNLL, on the other hand, only focuses on the uncertainty of the student model itself, does not involve any midpoint average, and is not affected by the degree of difference in teacher-student distribution. Therefore, it does not have the aforementioned lower bound constraint and is more stable under aggressive pruning.

[0140] 4. Computational Advantages: NLL only needs to extract the probability values ​​corresponding to the correct tokens from the student model's output, without needing to calculate the distribution over the entire vocabulary or compare it with the teacher model. The total dataset complexity of NLL is O(n log n). O (| D |· T · F f w.d. Compared to JSD O (2| D |· T · F f w.d. +| D |· T ·| V |), in vocabulary size| V The computational advantage is extremely significant in scenarios with large language models, typically ranging from 32,000 to over 100,000.

[0141] To assess the statistical reliability of the experimental conclusions of this invention, Cohen's dz effect size and statistical power analysis were used to quantitatively evaluate the experimental results.

[0142] Cohen's dz is the effect size index for paired-sample designs, and its calculation formula is: ; in, This represents the mean of the paired differences of the NLL method on the benchmark. This represents the mean of the paired differences of the JSD method on the benchmark. This is the standard deviation of the paired difference (NLL accuracy; JSD accuracy).

[0143] As shown in Table 2, the results of the statistical power analysis are as follows: Table 2

[0144] For the 50% pruning setting: Cohen's dz > 8, statistical power > 99.9%, far exceeding the commonly required 80% threshold. This indicates that in aggressive pruning scenarios, the advantage of NLL over JSD is highly statistically significant, and the experimental conclusions are highly reliable. For the 25% pruning setting: Cohen's dz ≈ 1.1, statistical power is only about 18%, below the standard 80% threshold. This indicates that in mild pruning scenarios, although a difference exists between NLL and JSD (an experimentally observed accuracy improvement of +0.26 to +0.94 percentage points), due to the small effect size and large variance, larger-scale experiments are needed to confirm the statistical significance of this difference. Caution should be exercised when interpreting the conclusions of the 25% pruning setting.

[0145] Furthermore, regarding baseline fairness, all experimental comparisons were performed under the same LoRA configuration (rank r=8 or 16) to ensure a fair comparison between the JSD and NLL methods. The only change was the second-stage scoring function (JSD vs NLL); all other hyperparameters and training configurations remained consistent.

[0146] like Figure 3 As shown in the diagram, this diagram illustrates the overall five-stage pipeline architecture of the PASER framework. The five stages are displayed from top to bottom: The first stage (S2RIC clustering) demonstrates the process of Sentence-BERT embedding, diffusion kernel smoothing, nonnegative matrix factorization, and the formation of K capability clusters. The clustering results are represented by schematic diagrams of multiple circular clusters in the figure.

[0147] The second stage (sample-by-sample scoring) compares the existing JSD scoring method with the NLL scoring method of this invention. The JSD scoring subgraph illustrates the process requiring two forward passes from both the teacher and student models to obtain the complete vocabulary distribution before calculating the JSD value. The NLL scoring subgraph demonstrates the process requiring only one forward pass from the student model to directly extract the probability of the correct token. The comparison between the two subgraphs clearly illustrates the difference in computational complexity.

[0148] Phase 3 (NTM Noise Token Masking): Demonstrates the construction of the concept consistency graph and the process of identifying and masking noise tokens.

[0149] Phase 4 (IES Sorting): Demonstrates the process of sorting by normalized score and budget allocation.

[0150] Phase 5 (LoRA Recovery): Demonstrates the process of fine-tuning the low-rank adaptation using a selected subset and outputting the recovered model.

[0151] like Figure 4As shown in the figure, the graph, in the form of a bar chart or line graph, illustrates the comparison of the recovery effects of the NLL scoring method of the present invention and the existing JSD scoring method under different experimental settings.

[0152] The horizontal axis represents different experimental settings (including different models LLaMA2-7B / LLaMA3-8B and different pruning ratios of 25% / 50%), and the vertical axis represents the mean (percentage) of paired differences on common sense reasoning benchmarks after recovery.

[0153] The figure shows two bars or two curves for each experimental setting, representing the JSD method and the NLL method, respectively. It is clear from the figure that in all four experimental settings, the NLL method achieved or surpassed the recovery performance of the JSD method, with improvements ranging from +0.26 to +0.94 percentage points (pp).

[0154] This figure demonstrates that the present invention is no less effective than, and may even be superior to, existing technologies in terms of restoration effect. Combined with statistical power analysis (Section 9.3), the conclusions of the 50% pruning setting are highly reliable (power > 99.9%), while the conclusions of the 25% pruning setting require careful interpretation (power ≈ 18%).

[0155] like Figure 5 As shown in the figure, the stacked bar chart displays the time consumption of each stage of the PASER framework under different scoring methods.

[0156] The horizontal axis represents the JSD and NLL methods, and the vertical axis represents time (in seconds or relative proportions). Each bar is divided into multiple segments, representing the time consumption of the first stage (S2RIC), the second stage (scoring), the third stage (NTM), the fourth stage (IES), and the fifth stage (LoRA).

[0157] As clearly shown in the graph, the second stage (scoring) accounts for the majority of the time under the JSD method, while it is significantly shortened under the NLL method. Overall, the NLL method reduces the total data selection time from the baseline of the JSD method to approximately 44.4%, which is a speedup of approximately 2.25 times. This speedup is consistent with the complexity analysis: the complexity of JSD is O(2|D|·T·F_fwd + |D|·T·|V|), and the complexity of NLL is O(|D|·T·F_fwd). When |V| >> 1, the speedup is mainly contributed by the |V| factor.

[0158] The figure visually demonstrates the significant improvement in computational efficiency achieved by this invention.

[0159] like Figure 6-Figure 10 As shown in the figure, this set of figures illustrates the confidence-quality distribution patterns of the JSD and NLL scoring methods when selecting samples.

[0160] Each graph is a two-dimensional heatmap or scatter plot, with the horizontal axis representing the confidence level of the sample (the degree of confidence the student model has in its predictions) and the vertical axis representing the quality of the sample (whether it makes correct predictions).

[0161] The graph corresponding to JSD shows that samples with high JSD scores are concentrated in the "high confidence, low quality" region (i.e., samples where the student model is confident but makes incorrect predictions), which confirms that JSD tends to select "confident but incorrect" samples.

[0162] The graph corresponding to NLL shows that samples with high NLL scores are concentrated in the "low confidence" region (i.e., samples that the student model is uncertain about), and these samples are exactly what the student model needs to learn.

[0163] This figure explains, mechanistically, why the NLL scoring method is superior to the JSD scoring method.

[0164] like Figure 11-Figure 14 As shown, this set of graphs presents the relationship between JSD scores and NLL scores in the form of a scatter plot.

[0165] In each graph, the horizontal axis represents the JSD score, and the vertical axis represents the NLL score. Each point represents a sample. Spearman's notation is also included in the graphs. (Spearman rank correlation coefficient) value.

[0166] As can be observed from the figure, the Spearman ρ values ​​of JSD and NLL range only from -0.22 to 0.52, indicating a low correlation between the rankings of the two scoring methods. The points in the scatter plot are relatively dispersed and do not show a clear linear relationship, indicating a significant difference in the two methods' judgment of sample importance.

[0167] This figure demonstrates that JSD and NLL do indeed have different preference patterns when selecting samples.

[0168] like Figure 15 As shown in the figure, the Spearman ρ values ​​of JSD and NLL under different experimental settings are displayed in the form of a bar chart.

[0169] The horizontal axis represents different experimental settings (combinations of different models and pruning ratios), and the vertical axis represents the Spearman ρ value (ranging from -1 to 1).

[0170] As can be seen from the figure, the Spearman ρ values ​​were low (-0.22 to 0.52) in all experimental settings, far lower than that of perfect positive correlation (1.0). This indicates a significant difference in the ranking of samples between NLL and JSD, further confirming that the two scoring methods have different selection preferences.

[0171] This figure provides statistical evidence for understanding the differences in choice behavior between the two scoring methods.

[0172] In summary, all the above figures together constitute the complete experimental evidence chain of this invention: framework_overview.png illustrates the method architecture and modification locations; accuracy_gap.png proves that NLL achieves or surpasses JSD in recovery performance (confidence is assessed in conjunction with statistical power analysis); cost_breakdown.png proves the computational efficiency advantage of NLL (consistent with complexity analysis); scatter_.png and rho_comparison.png prove the ranking difference between the two methods; cm_.png explains the mechanistic reason why NLL is superior to JSD.

[0173] Furthermore, the key points of this invention are as follows: Key point 1: Replace the JSD scoring function in the PASER framework with an NLL scoring function. In the data selection framework for recovering large language models after pruning, NLL (Negative Log-Likelihood) scoring, which relies solely on a one-way forward propagation from the student model, is used instead of JSD (Jensen-Shannon Divergence) scoring, which requires bidirectional forward propagation between the teacher and students. The formula for calculating the NLL score is as follows: s N LL =-(1 / T ) ; in M p For the student model after pruning, y t For the first t The correct reference token at each time step; y <t For the first t The sequence of all correct output tokens up to the specified time step. x For input instructions, T The length of the sample output sequence (number of tokens).

[0174] Key Point 2: Utilizing the inherent uncertainty signal of the student model as a basis for data selection

[0175] Instead of using the distributional difference between the teacher and student models, the negative log-likelihood (NLL) value of the student model for the correct output sequence is used directly as a measure of sample importance. A higher NLL value indicates greater uncertainty in the student model regarding that sample, meaning that the sample is more valuable for retraining.

[0176] Key Point 3: Eliminating the dependence of the data selection process on teacher models

[0177] NLL scoring only requires the student model and does not require the forward propagation output of the teacher model. Therefore, the teacher model does not need to be loaded during the data selection phase, reducing memory usage and system complexity, and allowing the post-pruning recovery process to be completed independently. The computational complexity of NLL across the entire dataset is O(n log n). O (| D |· T · F f w.d. Compared to JSD O (2| D |· T · F f w.d. +| D |· T ·| V |) significantly reduced.

[0178] Key Point 4: Avoid selection bias (including quantitative lower bound) of the JSD symmetric midpoint averaging mechanism under high pruning ratios.

[0179] As an asymmetric one-sided metric, NLL does not involve the midpoint mean of the distributions of the teacher and student models. M =( P Mo + P Mp ) / 2. In scenarios with a high pruning ratio (e.g., 50% structured pruning) and significant differences in teacher-student distribution, Lemma 1 (midpoint deviation) indicates that the midpoint mean M deviates from the correct distribution of the teacher model. P Mo The distance is e / 2, where e =|| P Mo - P Mp || T V Let be the total variation distance of the teacher-student distribution; Theorem 1 (lower bound of JSD selection bias) states that the expected loss for recovery training using JSD scoring satisfies E [ L r ec ]≥ c · e ² / (2+2 e The lower bound increases as ε increases, explaining the performance crash of JSD under high pruning ratios; NLL does not involve midpoint averaging and is therefore not subject to the above lower bound constraint, thus avoiding the systematic selection bias of JSD under high pruning ratios.

[0180] Key Point 5: Reduce computational complexity and improve data selection speed (including complexity analysis of the T factor)

[0181] NLL scoring only requires calculating the probability value of the correct reference token, without needing to traverse the probability distribution of all tokens in the complete vocabulary. In the complexity analysis including the sequence length T factor, the complexity of the entire JSD dataset is: O (2| D |· T · F f w.d. +| D |· T ·| V |), where the first term comes from a forward propagation of T time steps for both the teacher and student models, and the second term comes from the JSD calculation of the complete probability distribution on vocabulary V; NLL full dataset complexity: O (| D |· T · F f w.d. It only involves a single forward propagation from the student model and does not require traversing the entire vocabulary; the overall data selection speed is improved by about 2.25 times, and the speedup increases with the vocabulary size |V|.

[0182] Compared with the prior art, the present invention has the following advantages: Advantage 1: Significantly improved computational efficiency (data selection speed increased by approximately 2.25 times). JSD scoring requires two forward propagations for each sample, one for the teacher model and one for the student model (each propagation consists of T time steps), and each forward propagation requires calculating and storing the probability distribution over the complete vocabulary. NLL scoring only requires one forward propagation for the student model and only needs to extract the probability value of the correct reference token, without processing the complete vocabulary distribution. From a complexity perspective, JSD is... O (2| D |· T · F f w.d. +| D |· T ·| V |), NLL is O (| D |· T · F f w.d.Experimental results show that NLL scoring reduces the time consumption of the data selection phase in the PASER framework to approximately 44.4% of the original time, resulting in an overall speed improvement of approximately 2.25 times. This means that approximately 2.25 times the amount of data can be processed in the same amount of time, or approximately 55.6% of the time can be saved when processing the same amount of data.

[0183] Advantage 2: Recovery effect reaches or surpasses existing technology (accuracy improved by +0.26~+0.94 percentage points)

[0184] Under four experimental settings—two model architectures, LLaMA2-7B and LLaMA3-8B, and two structured pruning ratios of 25% and 50%—the mean pairwise discrepancies of the restored model using the NLL scoring method on seven commonsense reasoning benchmarks all reached or exceeded those of the JSD scoring method, with improvements ranging from +0.26 to +0.94 percentage points. This indicates that the present invention achieves faster performance without sacrificing restoration quality, and in fact, improves results.

[0185] Advantage 3: Better robustness under high pruning ratios (avoids confusion collapse)

[0186] When 50% structured pruning was applied, the perplexity (PPL) of the recovery model using JSD scoring soared to 11,435.67 (the normal range should be 10-20), indicating that the model had completely lost its linguistic ability. Theorem 1 (lower bound of JSD selection bias) theoretically explains this phenomenon: when teacher-student distribution differences... e When the value is large, the midpoint averaging mechanism of JSD is introduced. e A deviation of 2 / 2 results in the recovery loss satisfying the condition. E [ L r ec ]≥ c · e ² / (2+2 e The lower bound of the above-mentioned lower bound cannot achieve effective recovery. However, the recovery model using NLL scoring maintains a stable perplexity level under the same conditions without performance crashes because NLL does not involve midpoint averaging and is not subject to the aforementioned lower bound constraint. This advantage is crucial for practical deployments, as resource-constrained scenarios often require a high proportion of pruning.

[0187] Advantage 4: Avoid selecting "confident but wrong" samples

[0188] Confidence-Quality Analysis (CM analysis) shows that JSD scoring tends to select samples that the student model is confident in but makes incorrect predictions, which can mislead retraining. NLL scoring, on the other hand, selects samples that the student model is truly uncertain about. These samples contain key knowledge that the student model has not yet mastered, and their use in retraining can more effectively compensate for the knowledge gaps caused by pruning.

[0189] Advantage 5: Reduced deployment complexity and resource requirements

[0190] Since NLL scoring does not require the participation of the teacher model, there is no need to maintain the original unpruned teacher model in actual deployment. This reduces the overall resource requirements and deployment complexity of the system, and is particularly suitable for scenarios where it is impossible to obtain or store the teacher model (such as obtaining a pruned model from a third party, or deleting the teacher model to save storage space).

[0191] Advantage 6: The experimental conclusions have statistical reliability assurance (power analysis and baseline fairness).

[0192] This invention performs statistical power analysis on the experimental conclusions to ensure their reliability: 50% pruning setting: Cohen's dz > 8, statistical power > 99.9%, conclusions are highly reliable, NLL is significantly better than JSD; 25% pruning setting: Cohen's dz ≈ 1.1, statistical power ≈ 18%, conclusions need to be interpreted with caution, differences need to be confirmed by larger-scale experiments; Baseline fairness: all comparative experiments were conducted under the same LoRA configuration (rank r = 8 or 16) to ensure the comparison between JSD and NLL is fair, the only change is the second-stage scoring function.

[0193] Advantage 7: Achieves plug-and-play compatibility

[0194] This invention replaces only the second-stage scoring function of the PASER framework, while the other four stages (S2RIC clustering, NTM noise token masking, IES sorting, and LoRA recovery) remain completely unchanged. This minimal modification allows the invention to be easily integrated into existing PASER framework implementations without any adjustments to other modules, exhibiting excellent backward compatibility.

[0195] Furthermore, the present invention has been fully verified in the following experimental settings: 1) Basic models: LLaMA2-7B (approximately 7 billion parameters), LLaMA3-8B (approximately 8 billion parameters) 2) Pruning method: LLM-Pruner (Ma et al., 2023) structured pruning 3) Pruning ratio: 25% and 50% 4) Fine-tuning method: LoRA (rank r=8 or 16) 5) Evaluation Benchmarks: 7 common sense reasoning benchmark tests (including but not limited to PIQA, HellaSwag, WinoGrande, ARC-easy, ARC-challenge, OpenBookQA, BoolQ, etc.) 6) Training data: General instruction fine-tuning dataset The statistical power analysis table is shown in Table 3: Table 3

[0196] Statistical power was calculated using Cohen's dz as the effect size indicator at a significance level of α=0.05, under a two-tailed test. A power greater than 80% is generally considered the threshold for a reliable conclusion.

[0197] Summary of the proofs for the key theoretical results—Lemma 1 and Theorem 1: Lemma 1 (Midpoint Bias): Statement: Let the teacher model distribution be... P Mo The student model distribution is P Mp The total variation distance between the two is || P Mo - P Mp || TV = e Then the midpoint is evenly distributed. M =( P Mo + P Mp ) / 2 and teacher model distribution P Mo The total variation distance between them is || M - P Mo || T V = e / 2.

[0198] Proof summary: 1. By the definition of midpoint average, M - P Mo =( P Mp - P Mo ) / 2. 2. Take the total variation distance on both sides: || M - P Mo|| TV =||( P Mp - P Mo ) / 2|| TV 3. Due to the homogeneity of the total variation distance: || M - P Mo || TV =||( P Mp - P Mo ) / 2|| TV =(1 / 2)|| P Mp - P Mo || T V = e / 2.4. Q.E.D.

[0199] Theorem 1 (Quantitative Lower Bound of JSD Selection Bias): Statement: Let the loss function for retraining be... L r ec When using JSD scoring for data selection, the expected loss for resuming training satisfies the lower bound: E [ L r ec ]≥ c · e ² / (2+2 e ); in c For positive constants related to the task, e =|| P Mo - P Mp || T V .

[0200] Proof summary: 1. By Lemma 1, the midpoint average M The distance from the teacher distribution is e / 2.

[0201] 2. By Pinsker's inequality, for any two distributions... P and Q There is JSD ( P || Q )≥(1 / 2)|| P - Q ||²T V .

[0202] 3. For the samples selected by JSD scoring, the teacher model... M KL divergence KL( P Mo || M )satisfy: KL ( P Mo || M )≥(1 / 2)|| P Mo - M ||² T V (1 / 2)( e / 2)²= e ² / 8; 4. Similarly, the student model... M KL divergence KL( P Mo || M It satisfies the corresponding lower bound.

[0203] 5. The value of JSD is the average of the two: JSD = (1 / 2)[KL( P Mo || M )+KL( P Mp || M )).

[0204] 6. JSD scoring tends to select samples with high JSD values, and the recovery loss corresponding to these samples has a lower bound.

[0205] 7. After expectation calculation and algebraic simplification (transforming the lower bound of KL divergence into the lower bound of loss expectation, and utilizing...), e (By simplifying the definition), we can obtain E [ L r ec ]≥ c · e ² / (2+2 e ).

[0206] 8. The constant c depends on the specific form of the recovery loss function and the characteristics of the task.

[0207] 9. The proof is complete.

[0208] Key findings: 1) The Spearman rank correlation coefficients of NLL and JSD are only -0.22 to 0.52, indicating that there is a significant difference in the ranking of sample importance between the two scoring methods.

[0209] 2) With a 50% pruning ratio, the JSD score led to a perplexity of 11,435.67 (severe collapse) in the recovery model, while the NLL score maintained a stable perplexity. Statistical power analysis showed that this conclusion was highly reliable (>99.9%).

[0210] 3) With a 25% pruning rate, the accuracy of NLL improved by +0.26 to +0.94 percentage points, but the statistical power was only about 18%, and the conclusions should be interpreted with caution.

[0211] 4) NLL achieved or surpassed the recovery performance of JSD in all experimental settings.

[0212] 5) Data selection speed is improved by approximately 2.25 times, consistent with complexity analysis (JSD: O (2| D |· T · F f w.d. +| D |· T ·| V |), NLL: O (| D |· T · F f w.d. )).

[0213] 6) Theorem 1 theoretically proves that performance degradation of JSD under high pruning ratios is inevitable. E [ L r ec ]≥ c · e ² / (2+2 e NLL is not bounded by this lower bound.

[0214] The core code logic for NLL scoring can be summarized as follows: 1. Load the pruned student model and tokenizer.

[0215] 2. Segment each sample in the dataset to obtain the input token sequence and the output token sequence.

[0216] 3. Pass the input token sequence into the student model, perform forward propagation, and obtain the output logits (log odds).

[0217] 4. For each time step, use the Softmax function to convert the log odds into a probability distribution.

[0218] 5. Extract the probability value corresponding to the correct output token from the probability distribution.

[0219] 6. Take the logarithm of the probability values ​​for all time steps and sum them, then take the negative values ​​and average them to obtain the NLL score.

[0220] 7. Sort all samples from highest to lowest NLL score.

[0221] For the commercial application scenarios of this invention

[0222] 1) Deployment of large models on mobile devices: Deploy pruned large language models on mobile devices such as smartphones and tablets, and quickly restore model performance using the data selection method of this invention.

[0223] 2) Edge computing: Deploy compressed large language models on edge servers to reduce data transmission latency.

[0224] 3) Enterprise-level model compression service: Provides large language model compression and recovery solutions for enterprise customers.

[0225] 4) Multi-model management platform: Simplify data selection and recovery processes on platforms that require simultaneous management of multiple compressed model versions.

[0226] In this embodiment of the invention, a method for selecting data to recover a pruned large language model based on uncertainty signals of a student model is provided. This method involves acquiring a large-scale instruction fine-tuning dataset, performing S2RIC clustering on the dataset to output multiple capability clusters, performing NLL per-sample scoring on the capability clusters and the pruned student model, and outputting a sample-NLL score mapping table for each capability cluster. Based on a preset noise token mask threshold, NTM noise token masking is performed on each capability cluster and its corresponding sample-NLL score mapping table, outputting masked sample data and the masked sample-NLL score mapping table for each capability cluster. IES sorting and sample subset filtering are then performed on the masked sample data and the masked sample-NLL score mapping table for each capability cluster, outputting a recovered training subset. The recovered training subset is used to perform LoRA model recovery training on the pruned student model, outputting the recovered student model. Based on the above scheme, this invention utilizes S2RIC clustering... The system divides the large-scale instruction fine-tuning dataset into multiple capability clusters, avoiding repetitive and redundant calculations on massive amounts of undifferentiated data. Secondly, NLL per-sample scoring relies solely on a single forward propagation of the pruned student model to quantify sample value based on model uncertainty, eliminating the need for additional models or multiple rounds of forward propagation or complex distribution difference calculations, significantly reducing the computational cost of the core scoring stage. Next, NTM noise token masking removes noise tokens from the expected output sequence of samples, simultaneously cleaning up invalid score components, further reducing the amount of invalid data processed. Subsequently, IES sorting accurately sorts the masked samples based on the normalized NLL scores and allocates sample numbers by cluster, selecting only high-value samples to construct a restored training subset. Finally, LoRA model restoration training freezes the main parameters of the student model, updating only the low-rank adaptation matrix to restore model performance with minimal parameter updates, eliminating the need for iterative optimization of all model parameters and significantly reducing computational costs during training. The overall process is optimized through a series of progressive steps: data clustering to reduce redundancy, single-model scoring to reduce overhead, noise filtering to reduce ineffectiveness, precise screening to improve efficiency, and lightweight training to save computing power. This systematically reduces ineffective computation and resource consumption in each stage of data selection, and significantly improves the overall computational efficiency of the data selection stage for restoring large language models after pruning.

[0227] Please see Figure 16 , Figure 16 This is a structural block diagram of a large language model recovery data selection system based on student model uncertainty signals provided in Embodiment 2 of the present invention.

[0228] This invention provides a data selection system for recovering large language models after pruning based on student model uncertainty signals, comprising: The acquisition module 1601 is used to acquire a large-scale instruction fine-tuning dataset and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters; The scoring module 1602 is used to perform NLL per-sample scoring on multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster. The masking module 1603 is used to perform NTM noise token masking on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster based on a preset noise token masking threshold, and output the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster. The filtering module 1604 is used to perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset. Training module 1605 is used to perform LoRA model recovery training on the pruned student model using the recovered training subset, and output the recovered student model.

[0229] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0230] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the data selection method for pruning large language model recovery based on student model uncertainty signal as described in the above embodiments.

[0231] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the data selection method for pruning large language model recovery based on student model uncertainty signals as described in the above embodiments.

[0232] This invention also provides a computer program product, including a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer performs the steps of the data selection method for pruning large language model recovery based on student model uncertainty signals as described in the above embodiments.

[0233] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0234] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0235] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0236] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for selecting data for recovering a large language model after pruning based on uncertain signals from a student model, characterized in that, include: Obtain a large-scale instruction fine-tuning dataset, and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters; Perform NLL per-sample scoring on the multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster; Based on a preset noise token mask threshold, NTM noise token masking is performed on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster, and the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster are output. Perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset; The LoRA model is used to perform recovery training on the pruned student model using the recovered training subset, and the recovered student model is output.

2. The method for selecting large language model recovery data after pruning based on student model uncertainty signals according to claim 1, characterized in that, The S2RIC clustering of the large-scale instruction fine-tuning dataset is performed to output multiple capability clusters, including: The input instructions of each sample in the large-scale instruction fine-tuning dataset are embedded and encoded using a pre-set S-BERT model, and the fixed-dimensional sentence embedding vector corresponding to each sample is output. Based on the fixed-dimensional sentence embedding vectors corresponding to each sample, the cosine similarity between each pair of samples is calculated, thereby constructing a sample similarity matrix; The sample similarity matrix is ​​smoothed using a preset diffusion kernel, and the smoothed similarity matrix is ​​output. The smoothed similarity matrix is ​​decomposed using a preset nonnegative matrix factorization algorithm, and a coefficient matrix is ​​output. Based on the membership degree of each sample in the coefficient matrix, each sample in the coefficient matrix is ​​assigned to the cluster with the highest membership degree, and multiple mutually exclusive capability clusters are output.

3. The method for selecting large language model recovery data after pruning based on student model uncertainty signals according to claim 1, characterized in that, The step involves performing NLL per-sample scoring on the multiple capability clusters and the pruned student model, outputting a sample-NLL score mapping table corresponding to each capability cluster, including: The sample data from each of the capability clusters are input into the pruned student model for a single forward propagation process, and the conditional probability distribution of each of the sample data from each of the capability clusters is output for multiple time steps. Extract the correct reference token probability value from the conditional probability distribution at each time step; The negative logarithmic probability value of each time step is obtained by taking the natural logarithm of the correct reference token probability value in turn. The negative logarithmic probability values ​​of each time step corresponding to each sample data are summed and then divided by the length of the output sequence of the expected output sequence in the corresponding sample data to obtain the NLL score corresponding to each sample data. Based on the sample data and corresponding NLL scores in each capability cluster, construct a sample-NLL score mapping table for each capability cluster. The sample-NLL score mapping table retains the negative logarithmic probability component of each sample data at each time step.

4. The method for selecting large language model recovery data after pruning based on student model uncertainty signals according to claim 1, characterized in that, The step of performing NTM noise token masking on each capability cluster and the corresponding sample-NLL score mapping table based on a preset noise token masking threshold, and outputting the masked sample data and the masked sample-NLL score mapping table for each capability cluster, includes: Based on the expected output sequence of the sample data in each of the capability clusters, a concept consistency graph of each capability cluster is constructed. A graph structure analysis is performed based on the concept consistency graph of each capability cluster using a preset noise token mask threshold, and the noise token labeling results of each capability cluster are output. Based on the noise token marking results of each capability cluster, the noise tokens in the expected output sequence of the sample data in each capability cluster are masked to obtain the masked sample data corresponding to each capability cluster. Remove the negative log probability component of the time step corresponding to the noise token from each of the sample-NLL score mapping tables, and recalculate the sample NLL score based on the negative log probability component of the remaining time steps, and output the masked sample-NLL score mapping table corresponding to each capability cluster.

5. The method for selecting large language model recovery data after pruning based on student model uncertainty signals according to claim 1, characterized in that, The step of performing IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and outputting the recovered training subset includes: The sample-NLL score mapping table after masking of each capability cluster is normalized according to the capability cluster dimension, and the normalized sample-NLL score mapping table of each capability cluster is output. Based on the normalized NLL scores in the normalized sample-NLL score mapping table of each capability cluster, the masked sample data in the corresponding capability cluster are sorted from high to low scores, and a sorted list of masked samples in each capability cluster is output. The sample quantity allocation is performed based on the importance weight of each capability cluster, and the number of samples to be selected for each capability cluster is output. From the masked sample sorting list within each capability cluster, select the corresponding target samples according to the corresponding number of samples to be selected, thereby constructing the recovery training subset; The importance weights are determined based on the NLL statistical distribution characteristics of each capability cluster or the capability priority of a preset model.

6. The method for selecting large language model recovery data after pruning based on student model uncertainty signals according to claim 1, characterized in that, The step of performing LoRA model recovery training on the pruned student model using the recovered training subset, and outputting the recovered student model, includes: After injecting a low-rank adaptation matrix of a preset rank into the attention layer and feedforward network layer of the pruned student model, the main parameters of the student model are frozen, the restored training subset is input into the student model, and only the parameters of the low-rank adaptation matrix of the preset rank are updated to obtain the intermediate model after training. The low-rank adaptation matrix of the intermediate model after training is merged with the main parameters of the pruned student model, or kept as an independent adapter module, and the restored student model is output.

7. A data selection system for recovering a large language model after pruning based on uncertain signals from a student model, characterized in that, include: The acquisition module is used to acquire a large-scale instruction fine-tuning dataset and perform S2RIC clustering on the large-scale instruction fine-tuning dataset to output multiple capability clusters; The scoring module is used to perform NLL per-sample scoring on the multiple capability clusters and the pruned student model, and output the sample-NLL score mapping table corresponding to each capability cluster. The masking module is used to perform NTM noise token masking on each capability cluster and the sample-NLL score mapping table corresponding to each capability cluster based on a preset noise token masking threshold, and output the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster. The filtering module is used to perform IES sorting and sample subset filtering on the masked sample data and the masked sample-NLL score mapping table corresponding to each capability cluster, and output the recovered training subset; The training module is used to perform LoRA model recovery training on the pruned student model using the recovered training subset, and output the recovered student model.

8. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the data selection method for pruning large language model recovery based on student model uncertainty signals as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the method for selecting large language model recovery data after pruning based on student model uncertainty signals as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein, when the program instructions are executed by a computer, the computer performs the steps of the method for selecting large language model recovery data after pruning based on student model uncertainty signals as described in any one of claims 1-6.

Images