A two-stage sparse method for fast sparsification of large language models based on small samples

Abstract

The present invention discloses a two-stage sparsification method for rapidly sparsifying a large language model based on a small sample, comprising the following steps: S1, mask initialization: initializing the mask of the language model using a plurality of mask initialization methods based on a plurality of importance indices for measuring parameters; S2, mask reordering: reordering the masks based on a small sample calibration set and a straight-through estimator; S3, mask adjustment: based on the small sample calibration set and the reordered mask matrix of step S2, using the original model as a teacher model and the sparse model as a student model, implementing a block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain a sparse model; while achieving comparable performance, this method consumes less computing power, uses a smaller data set, occupies less GPU memory, and does not affect the inference speed of the sparse model, and has significant practical value and application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model sparsification, and in particular to a two-stage sparsification method for rapidly sparsifying a large language model based on a small sample. Background Art

[0002] Sparsification of large language models is an important research direction in the field of neural network compression. Its goal is to improve model inference efficiency through sparse masking while maintaining language model performance. In recent years, research on large language models based on the Transformer architecture has become a hot topic in deep learning. However, while some large language models offer excellent performance, they also lead to significant computational resource consumption during the inference phase. Therefore, reducing these inference costs has become a pressing issue.

[0003] There are two major challenges in sparsifying large language models. First, there is a lack of effective compression methods for large language models. Many current sparsification algorithms and compression methods are prohibitively expensive, making them unsuitable for large language models. For example, using the Hessian matrix as a measure of parameter importance to guide the sparsification process, while effective for small models, is computationally intensive and expensive, making it difficult to apply to large language models. Second, there is a lack of cost-effective fine-tuning methods for accuracy recovery. When sparsifying small models, fine-tuning all parameters using the entire training set is often used to restore accuracy, but this approach is prohibitively expensive for large language models. Currently, these methods still face challenges in practical application, such as evaluating parameter importance during sparsification and fine-tuning sparse models. Therefore, it is important to propose a new, low-cost, and efficient sparsification method for large language models. Summary of the Invention

[0004] To solve the above problems, the present invention proposes a two-stage sparse method for rapidly sparseening large language models based on small samples.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A two-stage sparse method for rapidly sparseening a large language model based on a small sample size includes the following steps:

[0007] S1. Mask initialization: Based on a variety of importance metrics for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model;

[0008] S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator;

[0009] The specific process of step S2 is:

[0010] S21. In the mask reordering stage, we operate on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block;

[0011] S22. Use a small sample calibration set to freeze all parameters in the large language model and input the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ;

[0012] S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q'. The pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the rest of the masks are set to 1 to achieve mask binarization; where r represents the sparsity rate, Indicates the number of mask values;

[0013] S24, input the sparse activation X' into the pseudo sparse block Q' for forward propagation to obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; and W represents the weight matrix;

[0014] S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is used as the loss L, and the gradient relative to the mask value is obtained by backpropagation to update the mask value to obtain the reordered mask matrix; among them, the straight-through estimator is adopted, the gradient of the M⊙W operation is ignored and the output gradient is directly backpropagated. The calculation formula of the straight-through estimator is:

[0015]

[0016] in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix;

[0017] S3. Mask adjustment: Based on the small sample calibration set and reordering mask matrix in step S2, the original model is used as the teacher model, the sparse model is used as the student model, and the small sample calibration set used in the mask reordering is used as the input of the teacher model and the student model. The small sample block distillation method is adopted to adjust the mask value by minimizing the reconstruction error block by block, optimize the non-zero mask elements of each sparse operator, and realize the block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain the sparse model.

[0018] Preferably, the specific process of step S1 is:

[0019] S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Where ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model;

[0020] S12. In the mask initialization stage, a mask matrix is ​​pre-initialized for each operator to be sparsified, and all elements are set to 1; an importance index of a large language model parameter is used to quantify each parameter in the operator parameter matrix by the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter,

[0021] The formula for calculating the amplitude of the parameter is:

[0022]

[0023] in, Indicates the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight;

[0024] The product of the parameter and the activation value is calculated as:

[0025]

[0026] in, Indicates the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position;

[0027] The product of the Hessian matrices of the parameters is calculated as:

[0028]

[0029] in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op Represents activation, λ represents hyperparameters, I represents the identity matrix, and diag() represents converting the input into a diagonal matrix. It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input;

[0030] S13. Using multiple mask initialization methods, generate a specific value for each position in the mask matrix to be reordered, and obtain an initialized mask matrix.

[0031] Preferably, the mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, Sigmoid initialization, multi-step initialization and linear initialization.

[0032] Preferably, the specific process of the small sample block distillation method in step S3 is:

[0033] S31. In block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and mask matrices of each sparse operator, the mask matrix consisting of {0, 1} elements according to a preset sparsity ratio;

[0034] S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block currently requiring adjustment, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing;

[0035] S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

[0036] After adopting the above technical solution, the present invention has the following beneficial effects: the mask reordering step proposed in the present invention provides a higher quality initial mask matrix, so that the subsequent mask adjustment process is more likely to find a better local optimum or even a global optimum, thereby improving the final performance of the sparse model; using an extremely small calibration set in the mask adjustment stage can reduce data requirements while maintaining model accuracy, which is particularly suitable for resource-constrained scenarios; the final performance of the sparse model is significantly improved through a two-stage mask adjustment scheme, and this performance improvement is independent of the adjustment time, and the effect is better at the same time cost; therefore, while achieving comparable performance, this method consumes less computing power, uses a smaller data set, occupies less GPU memory, and does not affect the inference speed of the sparse model, and has significant practical value and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 It is a schematic diagram of the main steps of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] like Figures 1 to 2 As shown in FIG, a two-stage sparse method for quickly sparsely populating a large language model based on a small sample size includes the following steps:

[0041] S1. Mask initialization: Based on a variety of importance metrics for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model;

[0042] The specific process of step S1 is:

[0043] S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Where ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model;

[0044] S12. In the mask initialization stage, a mask matrix is ​​pre-initialized for each operator to be sparsified, and all elements are set to 1; an importance index of a large language model parameter is used to quantify each parameter in the operator parameter matrix by the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter,

[0045] The formula for calculating the amplitude of the parameter is:

[0046]

[0047] in, Indicates the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight;

[0048] The product of the parameter and the activation value is calculated as:

[0049]

[0050] in, Indicates the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position;

[0051] The product of the Hessian matrices of the parameters is calculated as:

[0052]

[0053] in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op Represents activation, λ represents hyperparameter, i represents the identity matrix, diag() represents converting the input into a diagonal matrix, It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input;

[0054] S13, using multiple mask initialization methods to generate specific values ​​for each position in the mask matrix to be reordered, to obtain an initialized mask matrix;

[0055] The mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, sigmoid initialization, multi-step initialization and linear initialization;

[0056] S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator;

[0057] The specific process of step S2 is:

[0058] S21. In the mask reordering stage, we operate on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block;

[0059] S22. Use a small sample calibration set to freeze all parameters in the large language model and input the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ;

[0060] S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q'. The pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the rest of the masks are set to 1 to achieve mask binarization; where r represents the sparsity rate, Indicates the number of mask values;

[0061] S24, input the sparse activation X' into the pseudo sparse block Q' for forward propagation to obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; and W represents the weight matrix;

[0062] S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is used as the loss L, and the gradient relative to the mask value is obtained by backpropagation to update the mask value to obtain the reordered mask matrix; among them, the straight-through estimator is adopted, the gradient of the W⊙W operation is ignored and the output gradient is directly backpropagated. The calculation formula of the straight-through estimator is:

[0063]

[0064] in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix;

[0065] S3. Mask adjustment: Based on the small-sample calibration set and reordering mask matrix from step S2, the original model is used as the teacher model, the sparse model is used as the student model, and the small-sample calibration set used in the mask reordering is used as the input of the teacher model and the student model. The small-sample block distillation method is used to adjust the mask value by minimizing the reconstruction error block by block, optimizing the non-zero mask elements of each sparse operator, and realizing a block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain the sparse model.

[0066] The specific process of the small sample block distillation method in step S3 is as follows:

[0067] S31. In block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and mask matrices of each sparse operator, the mask matrix consisting of {0, 1} elements according to a preset sparsity ratio;

[0068] S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block currently requiring adjustment, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing;

[0069] S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

[0070] Performance testing:

[0071] The performance of mask reordering, mask adjustment, and the complete two-stage sparse method of the present invention based on fast sparsification of large language models with small samples is evaluated on LLaMA-V1-7B and LLaMA-V2-7B. Table 1 shows the performance of large language models with unstructured sparsity on multiple zero-shot tasks. Among them, w.m_re and w.m_ft respectively represent the introduction of reordering strategy and fine-tuning strategy on the baseline, w.ELO-Mask represents the two-stage mask reordering and fine-tuning strategy of the present invention, and Race, ARC-C, ARC-E, WinoGrande, and StoryCloze respectively represent the corresponding test data.

[0072] Table 1: Performance test results of large language models on multiple zero-shot tasks

[0073]

[0074] As can be seen from Table 1, at a sparsity rate of 50%, after mask reordering, mask adjustment, or the two-stage sparsification method (w.ELO-Mask) based on small sample fast sparsification of large language models adopted in this invention, the sparse model maintains its generalization performance on downstream tasks, and the accuracy is improved in most tasks.

[0075] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A two-stage sparse method for fast sparse large language models based on small samples, characterized by: The following steps are involved: S1. Mask initialization: Based on a variety of importance metrics for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model; S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator; The specific process of step S2 is: S21. In the mask reordering stage, we operate on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block; S22. Use a small sample calibration set to freeze all parameters in the large language model and input the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ; S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q'. The pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the rest of the masks are set to 1 to achieve mask binarization; where r represents the sparsity rate, Indicates the number of mask values; S24, input the sparse activation X' into the pseudo sparse block Q' for forward propagation to obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; and W represents the weight matrix; S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is used as the loss L, and the gradient relative to the mask value is obtained by backpropagation to update the mask value to obtain the reordered mask matrix; among them, the straight-through estimator is adopted, the gradient of the M⊙W operation is ignored and the output gradient is directly backpropagated. The calculation formula of the straight-through estimator is: in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix; S3. Mask adjustment: Based on the small sample calibration set and reordering mask matrix in step S2, the original model is used as the teacher model, the sparse model is used as the student model, and the small sample calibration set used in the mask reordering is used as the input of the teacher model and the student model. The small sample block distillation method is adopted to adjust the mask value by minimizing the reconstruction error block by block, optimize the non-zero mask elements of each sparse operator, and realize the block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain the sparse model.

2. The two-stage sparse method for rapidly sparse large language models based on small samples according to claim 1, characterized in that: The specific process of step S1 is: S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Where ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model; S12. In the mask initialization stage, a mask matrix is ​​pre-initialized for each operator to be sparsified, and all elements are set to 1; an importance index of a large language model parameter is used to quantify each parameter in the operator parameter matrix by the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter, The formula for calculating the amplitude of the parameter is: in, Indicates the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight; The product of the parameter and the activation value is calculated as: in, Indicates the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position; The product of the Hessian matrices of the parameters is calculated as: in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op Represents activation, λ represents hyperparameters, I represents the identity matrix, and diag() represents converting the input into a diagonal matrix. It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input; S13. Using multiple mask initialization methods, generate a specific value for each position in the mask matrix to be reordered, and obtain an initialized mask matrix.

3. The two-stage sparse method for rapidly sparse large language models based on small samples according to claim 2, characterized in that: The mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, Sigmoid initialization, multi-step initialization and linear initialization.

4. The two-stage sparse method for rapidly sparse large language models based on small samples according to claim 1, characterized in that: The specific process of the small sample block distillation method in step S3 is as follows: S31. In block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and mask matrices of each sparse operator, the mask matrix consisting of {0, 1} elements according to a preset sparsity ratio; S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block currently requiring adjustment, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing; S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

Images