Two-stage sparse method based on small sample fast sparse large language model

Abstract

The invention discloses a two-stage sparse method based on a small sample rapid sparse large-scale language model, and the method comprises the following steps: S1, mask initialization: based on a plurality of importance indexes for measuring parameters, employing a plurality of mask initialization methods to initialize a mask of a language model; s2, reordering the masks: reordering the masks based on the small sample calibration set and the straight-through estimator; s3, mask adjustment: based on the small sample calibration set and the reordered mask matrix in the step S2, taking an original model as a teacher model, taking a sparse model as a student model, realizing a block-level self-distillation process of the large language model in a time dimension before and after sparsification, and obtaining the sparse model; the method consumes less computing power, uses a smaller data set, occupies less GPU memory while realizing considerable performance, does not influence the reasoning speed of the sparse model, and has remarkable practical value and application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model sparsification, and in particular to a two-stage sparsification method for quickly sparsifying a large language model based on a small sample. Background Art

[0002] Sparsification of large language models is an important research direction in the field of neural network compression. Its purpose is to improve the efficiency of model reasoning through sparse masks while ensuring the performance of the language model. In recent years, research on large language models based on the Transformer architecture has become a hot topic in the field of deep learning. However, although some large language models have excellent performance, they also bring about the problem of huge consumption of computing resources in the reasoning stage. Therefore, how to reduce these reasoning costs has become an urgent problem to be solved.

[0003] There are two major challenges in sparsifying large language models: first, there is a lack of effective compression methods for large language models. Currently, many sparsification algorithm compression methods are too expensive, making them unsuitable for large language models. For example, taking the Hessian matrix as a measure of parameter importance to guide the sparsification process, although it performs well in small models, has a huge amount of computation and high sparsification cost, making it difficult to apply to large language models; second, there is a lack of cost-effective precision recovery fine-tuning methods. In the sparsification process of small models, the entire training set is usually used for full parameter fine-tuning to restore precision, but this method is too expensive for large language models. At present, these methods still face challenges in practical applications, such as the evaluation of parameter importance in the sparsification process and the fine-tuning of sparse models. Therefore, it is of great significance to propose a new low-cost and efficient sparsification method for large language models. Summary of the invention

[0004] To solve the above problems, the present invention proposes a two-stage sparse method for quickly sparseening a large language model based on small samples.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A two-stage sparse method for quickly sparseening a large language model based on a small sample includes the following steps:

[0007] S1. Mask initialization: Based on a variety of importance indicators for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model;

[0008] S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator;

[0009] The specific process of step S2 is:

[0010] S21. In the mask reordering stage, the operation is performed on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block;

[0011] S22. Use a small sample calibration set to freeze all parameters in the large language model and feed the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ;

[0012] S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q', the pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the remaining masks are set to 1 to achieve mask binarization; where r represents the sparseness rate, Indicates the number of mask values;

[0013] S24, input the sparse activation X' into the pseudo-sparse block Q' for forward propagation, and obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; W represents the weight matrix;

[0014] S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is taken as the loss L, and the gradient relative to the mask value is obtained by back propagation to update the mask value to obtain the reordered mask matrix; wherein, the straight-through estimator is adopted, the gradient of the M⊙W operation is ignored and the output gradient is directly back propagated. The calculation formula of the straight-through estimator is:

[0015]

[0016] in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix;

[0017] S3. Mask adjustment: Based on the small sample calibration set and reordering mask matrix in step S2, the original model is used as the teacher model, the sparse model is used as the student model, and the small sample calibration set used in the mask reordering is used as the input of the teacher model and the student model. The small sample block distillation method is used to adjust the mask value by minimizing the reconstruction error block by block, optimize the non-zero mask elements of each sparse operator, and realize the block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain a sparse model.

[0018] Preferably, the specific process of step S1 is:

[0019] S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Among them, ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model;

[0020] S12, in the mask initialization stage, pre-initialize a mask matrix for each operator to be sparse, and set all elements to 1; use the importance index of the large language model parameter to quantize each parameter in the operator parameter matrix through the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter,

[0021] The amplitude of the parameter is calculated as:

[0022]

[0023] in, represents the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight;

[0024] The product of the parameter and the activation value is calculated as:

[0025]

[0026] in, represents the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position;

[0027] The formula for calculating the product of the Hessian matrices of the parameters is:

[0028]

[0029] in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op represents activation, λ represents hyperparameter, I represents the identity matrix, and diag() represents converting the input into a diagonal matrix. It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input;

[0030] S13. Using a variety of mask initialization methods, generate specific values ​​for each position in the mask matrix to be reordered, and obtain an initialized mask matrix.

[0031] Preferably, the mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, sigmoid initialization, multi-step initialization and linear initialization.

[0032] Preferably, the specific process of the small sample block distillation method in step S3 is:

[0033] S31, in block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and the mask matrix of each sparse operator, the mask matrix is ​​composed of {0,1} elements according to a preset sparsity rate;

[0034] S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block that currently needs to be adjusted, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing;

[0035] S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

[0036] After adopting the above technical scheme, the present invention has the following beneficial effects: the mask reordering step proposed in the present invention provides a higher quality initial mask matrix, so that the subsequent mask adjustment process is more likely to find a better local optimum or even a global optimum, thereby improving the final performance of the sparse model; using an extremely small calibration set in the mask adjustment stage can reduce data requirements while maintaining model accuracy, which is particularly suitable for resource-constrained scenarios; the final performance of the sparse model is significantly improved through a two-stage mask adjustment scheme, and this performance improvement is independent of the adjustment time, and the effect is better at the same time cost; therefore, while achieving comparable performance, the method consumes less computing power, uses a smaller data set, occupies less GPU memory, and does not affect the inference speed of the sparse model, and has significant practical value and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 It is a schematic diagram of the main steps of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] like Figure 1 to Figure 2 As shown, a two-stage sparse method for quickly sparseening a large language model based on a small sample includes the following steps:

[0041] S1. Mask initialization: Based on a variety of importance indicators for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model;

[0042] The specific process of step S1 is:

[0043] S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Among them, ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model;

[0044] S12, in the mask initialization stage, pre-initialize a mask matrix for each operator to be sparse, and set all elements to 1; use the importance index of the large language model parameter to quantize each parameter in the operator parameter matrix through the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter,

[0045] The amplitude of the parameter is calculated as:

[0046]

[0047] in, represents the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight;

[0048] The product of the parameter and the activation value is calculated as:

[0049]

[0050] in, represents the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position;

[0051] The formula for calculating the product of the Hessian matrices of the parameters is:

[0052]

[0053] in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op represents activation, λ represents hyperparameter, i represents the identity matrix, and diag() represents converting the input into a diagonal matrix. It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input;

[0054] S13, using a plurality of mask initialization methods to generate a specific value for each position in the mask matrix to be reordered, to obtain an initialized mask matrix;

[0055] The mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, sigmoid initialization, multi-step initialization and linear initialization;

[0056] S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator;

[0057] The specific process of step S2 is:

[0058] S21. In the mask reordering stage, the operation is performed on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block;

[0059] S22. Use a small sample calibration set to freeze all parameters in the large language model and feed the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ;

[0060] S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q', the pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the remaining masks are set to 1 to achieve mask binarization; where r represents the sparseness rate, Indicates the number of mask values;

[0061] S24, input the sparse activation X' into the pseudo-sparse block Q' for forward propagation, and obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; W represents the weight matrix;

[0062] S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is taken as the loss L, and the gradient relative to the mask value is obtained by back propagation to update the mask value to obtain the reordered mask matrix; wherein, the straight-through estimator is adopted, the gradient of the W⊙W operation is ignored and the output gradient is directly back propagated. The calculation formula of the straight-through estimator is:

[0063]

[0064] in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix;

[0065] S3, mask adjustment: Based on the small sample calibration set and reordering mask matrix in step S2, the original model is used as the teacher model, the sparse model is used as the student model, the small sample calibration set used in the mask reordering is used as the input of the teacher model and the student model, and the small sample block distillation method is adopted to adjust the mask value by minimizing the reconstruction error block by block, optimize the non-zero mask elements of each sparse operator, realize the block-level self-distillation process of the large language model before and after sparseness in the time dimension, and obtain the sparse model;

[0066] The specific process of the small sample block distillation method in step S3 is:

[0067] S31, in block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and the mask matrix of each sparse operator, the mask matrix is ​​composed of {0,1} elements according to a preset sparsity rate;

[0068] S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block that currently needs to be adjusted, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing;

[0069] S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

[0070] Performance Test:

[0071] The performance of mask reordering, mask adjustment, and the complete two-stage sparse method of the present invention based on fast sparsification of large language models with small samples is evaluated on LLaMA-V1-7B and LLaMA-V2-7B. The performance of large language models with unstructured sparsity on multiple zero-shot tasks is shown in Table 1. Among them, w.m_re and w.m_ft respectively represent the introduction of rearrangement strategy and fine-tuning strategy on the baseline, w.ELO-Mask represents the two-stage mask rearrangement and fine-tuning strategy of the present invention, and Race, ARC-C, ARC-E, WinoGrande, and StoryCloze represent the corresponding test data.

[0072] Table 1: Performance test results of large language models on multiple zero-shot tasks

[0073]

[0074] It can be seen from Table 1 that at a sparsity rate of 50%, after mask reordering, mask adjustment or the two-stage sparsification method (w.ELO-Mask) based on small sample fast sparsification of large language models adopted by the present invention, the sparse model maintains its generalization performance on downstream tasks, and the accuracy is improved in most tasks.

[0075] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A two-stage sparse method for fast sparse large language models based on small samples, characterized in that: The following steps are involved: S1. Mask initialization: Based on a variety of importance indicators for measuring parameters, a variety of mask initialization methods are used to initialize the mask of the language model; S2, mask reordering: reorder the masks based on the small sample calibration set and the straight-through estimator; The specific process of step S2 is: S21. In the mask reordering stage, the operation is performed on a block-by-block basis; given the block Q to be sparsified, the sparsity rate r, and the mask matrix M obtained by applying the mask initialization method to each operator op within the block op ,The reordering operation takes the sparse activations of the previous block as the input of the current block; S22. Use a small sample calibration set to freeze all parameters in the large language model and feed the sparse activation X' output by the shallow sparse block into the unsparsed block Q to obtain the dense output activation Y dense ; S23, perform a sparse operation on the block Q to obtain a pseudo-sparse block Q', the pseudo-sparse process includes collecting all mask values ​​from the mask matrices of all operators in the block Sort by value and remove those below the threshold The mask value is set to 0, and the remaining masks are set to 1 to achieve mask binarization; where r represents the sparseness rate, Indicates the number of mask values; S24, input the sparse activation X' into the pseudo-sparse block Q' for forward propagation, and obtain the sparse output activation Y sp ; In the forward process of each training iteration of the pseudo-sparse block Q', all mask values ​​in the mask matrices of all operators in the block are sorted and binarized to generate an intermediate mask matrix containing only {0,1} elements. The intermediate mask is used to calculate M⊙W for forward propagation; where M is the general symbol for the mask matrix; ⊙ represents the Hadamard product between matrices; W represents the weight matrix; S25. Calculate sparse output activation Y sp and dense output activation Y dense The mean square error between them is taken as the loss L, and the gradient relative to the mask value is obtained by back propagation to update the mask value, and the reordered mask matrix is ​​obtained; among them, the straight-through estimator is adopted, the gradient of the M⊙W operation is ignored and the output gradient is directly back propagated. The calculation formula of the straight-through estimator is: in, represents the gradient of the loss function with respect to the network output, represents the gradient of the loss function with respect to the mask matrix, represents the gradient of the activation with respect to the mask matrix; S3. Mask adjustment: Based on the small sample calibration set and reordering mask matrix in step S2, the original model is used as the teacher model, the sparse model is used as the student model, and the small sample calibration set used in the mask reordering is used as the input of the teacher model and the student model. The small sample block distillation method is used to adjust the mask value by minimizing the reconstruction error block by block, optimize the non-zero mask elements of each sparse operator, and realize the block-level self-distillation process of the large language model before and after sparsification in the time dimension to obtain a sparse model.

2. A two-stage sparse method for fast sparse large language models based on small samples as claimed in claim 1, characterized in that: The specific process of step S1 is: S11. Use represents the mask of the parameter at position (i, j) in the op-th operator of the large language model, where M is the general symbol for a mask matrix consisting of elements 0 and 1; indicates that the corresponding element in the parameter matrix should be pruned, Indicates that the corresponding elements should be retained; the sparsification of the operator can be expressed mathematically as Among them, ⊙ represents the Hadamard product between matrices, Represents the parameter at position (i, j) in the op-th operator of the large language model; S12, in the mask initialization stage, pre-initialize a mask matrix for each operator to be sparse, and set all elements to 1; use the importance index of the large language model parameter to quantize each parameter in the operator parameter matrix through the importance index to generate an initial mask that needs to be optimized through subsequent processing; the importance index includes the amplitude of the parameter, the product of the parameter and the activation value, and the Hessian matrix of the parameter, The amplitude of the parameter is calculated as: in, represents the amplitude value of the corresponding position parameter, Represents the 1-norm of the corresponding position weight; The product of the parameter and the activation value is calculated as: in, represents the amplitude value of the corresponding position parameter, Represents the absolute value of the corresponding position weight, represents the 2-norm of the activation at the corresponding position; The formula for calculating the product of the Hessian matrices of the parameters is: in, Indicates the amplitude value of the corresponding position parameter, |W op | 2 represents the 2-norm of the weight, represents the transposed matrix of activations, X op represents activation, λ represents hyperparameter, I represents the identity matrix, and diag() represents converting the input into a diagonal matrix. It represents the 2-norm of the weight at the corresponding position divided by the diagonal element at the corresponding position, reflecting the correlation between the amplitude value of the parameter and the input; S13. Using a variety of mask initialization methods, generate specific values ​​for each position in the mask matrix to be reordered, and obtain an initialized mask matrix.

3. A two-stage sparse method for rapidly sparse large language models based on small samples as claimed in claim 2, characterized in that: The mask initialization method in step S13 includes default initialization, percentage initialization, random initialization, sigmoid initialization, multi-step initialization and linear initialization.

4. A two-stage sparse method for rapidly sparseening a large language model based on a small sample as claimed in claim 1, characterized in that: The specific process of the small sample block distillation method in step S3 is: S31, in block mask adjustment, starting from a sparse large language model, the large language model includes the original dense model and the mask matrix of each sparse operator, the mask matrix is ​​composed of {0,1} elements according to a preset sparsity rate; S32, entering the mask adjustment stage, the mask adjustment stage is performed block by block, for the block that currently needs to be adjusted, back propagation is used to optimize the non-zero elements of the mask matrix in each sparse operator, and the gradient of the zero element is set to zero to achieve freezing; S33. After performing multiple rounds of numerical optimization on the mask, the Hadamard product of the new mask matrix and the parameter matrix corresponding to the original model is calculated to obtain the parameters of the sparse model and obtain the final sparse model.

Images