Low-bit large model quantification method and system based on sensitivity analysis and abnormal value processing

Abstract

The invention provides a low-bit large model quantification method based on sensitivity analysis and abnormal value processing, and the method comprises the steps: dividing a model layer into a high-sensitivity layer, a medium-sensitivity layer and a low-sensitivity layer through employing a hierarchical mixing precision quantification strategy based on sensitivity analysis, and executing different quantification operations; carrying out weight abnormal value suppression and weight layer-by-layer quantization execution by using a quantization method based on two-stage combination; and carrying out activation outlier offline smoothing processing based on equivalent transformation. According to the method, key weights sensitive to quantization errors are identified based on a hierarchical mixed precision quantization strategy of sensitivity analysis, so that a differentiation strategy is adopted in the quantization process, the precision of high-sensitivity weights is reserved, and more aggressive quantization is performed on low-sensitivity weights to minimize model performance loss; the two-stage weight quantization collaboratively solves the problem of abnormal value and error accumulation: the quantization difficulty is migrated from activation to weight offline, the friendliness of activation quantization is improved, outliers do not need to be dynamically detected during operation, and the reasoning speed is prevented from being influenced.

Description

Technical Field

[0001] This invention relates to the field of large model technology, and in particular to a low-bit large model quantization method and system based on sensitivity analysis and outlier processing. Background Technology

[0002] As the scale of large model parameters surges to the tens or hundreds of billions, their enormous storage requirements and computational overhead severely hinder their deployment and application on real-world devices. To overcome this bottleneck, large model compression technology has emerged, with the core objective of significantly reducing model size, memory footprint, and accelerating the inference process while preserving as much of the original model performance as possible. Among numerous compression methods (such as knowledge distillation and model pruning), large model quantization has become a key technology due to its relatively direct implementation and significant effects. It fundamentally reduces storage requirements and computational complexity by converting model weights and activation values ​​from high-precision floating-point numbers (such as FP32) to low-precision representations (such as INT8, FP16, or even lower), making it a crucial breakthrough for achieving lightweight and efficient deployment of large models. However, existing quantization methods have a series of objective drawbacks:

[0003] 1) Existing quantization methods use a uniform number of bits (e.g., 8 bits or 4 bits) for all weights, failing to consider the varying quantization error sensitivities of different weights. High-sensitivity weights (e.g., gate_proj in deep MLP layers) suffer significant performance loss due to low-bit quantization, while low-sensitivity weights (e.g., o_proj in attention mechanisms) retain redundant precision, resulting in low resource utilization. 2) Existing quantization methods lack mechanisms to suppress outliers in weights. Structured outliers exist in the input channel dimension of the weight matrix, causing the scaling factor to be dominated by outliers during per-channel quantization. This long-tailed distribution exacerbates quantization errors, leading to a sharp performance degradation during low-bit quantization. Furthermore, inter-layer output bias is not considered during weight quantization. 3) When processing activation quantization, existing methods encounter channel-level outliers (with amplitudes exceeding normal values ​​by 10 times) in the activation layer (e.g., k_proj activation), making activation quantization significantly more difficult than weight quantization. This necessitates dynamic outlier detection at runtime, reducing inference speed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a low-bit large-model quantization method and system based on sensitivity analysis and outlier processing.

[0005] The technical solution of this invention is: a low-bit large-model quantization method based on sensitivity analysis and outlier processing, comprising the following steps:

[0006] S1) The model layer is divided into high, medium and low sensitivity layers using a hierarchical mixed precision quantization strategy based on sensitivity analysis, and different quantization operations are performed.

[0007] S2) Employ a two-stage joint quantization method to suppress outliers in weights and perform weight quantization layer by layer;

[0008] S3) Perform offline smoothing of activated outliers based on equivalent transformation.

[0009] Preferably, in step S1), the model layer is divided into high, medium, and low sensitivity layers using a hierarchical mixed precision quantization strategy based on sensitivity analysis, and different quantization operations are performed. Specifically, this includes the following steps:

[0010] S11) Select a small calibration input set X, and record the input activation and output of each layer of the model through forward propagation;

[0011] S12), for the weight matrix W of the l-th layer l ∈R m×n Define the quantization sensitivity ΔE∈R m×n The original weighted predicted value W l X and the predicted value of the quantized weight W l ′ The error between X and m, where m and n are the weight matrices W, respectively. l Width and height;

[0012] S13) The sensitivity ΔE is quantized by performing a second-order Taylor expansion of the perturbation of the objective function E(w) at local points, and the gradient term of the convergent model is ignored, resulting in:

[0013]

[0014] In the formula, δ w This is the perturbation value introduced by weight quantization, where H is the Hessian matrix; quant() is the quantization function. It is the unit selection vector where position q is 1 and all other positions are 0; w q Let w be the q-th element of w; w is the weight parameter of the model, i.e., the quantization object; T represents the transpose operation;

[0015] S14) The optimization problem is constructed using the Lagrange multiplier method, namely:

[0016]

[0017] pass The optimal perturbation is obtained:

[0018]

[0019] After substituting into the error formula, the analytical expression for quantization sensitivity is:

[0020]

[0021] In the formula, L represents the Lagrange function; H -1 Denotes the inverse of the Hessian matrix, [H -1 ] qq H represents -1 The value at the intersection of row q and column q; H represents -1 The qth column.

[0022] S15) Based on the sensitivity score, the model layer is divided into high, medium, and low sensitivity layers. The scaling factor s and the quantized weight W are calculated using the following quantization strategy. q ; and through the quantized weight W q Quantization is performed, with different sensitivity levels using different quantization bit depths; where:

[0023]

[0024] In the formula, the clamp function ensures that the value does not exceed the valid range, round indicates that the rounding operation is performed, and b is the number of quantization bits.

[0025] Preferably, in step S1), the number of samples in the calibration input set X is required to cover common input distributions of the model, and the Hessian matrix is ​​calculated through layer-by-layer forward propagation and gradient backtracking, ignoring the gradient term g after model convergence. T δ w Only the second-order terms are retained to simplify the calculation.

[0026] Preferably, in step S1), the threshold for dividing the high, medium, and low sensitivity layers is determined based on the statistical distribution of sensitivity scores: the top 30% of the sensitivity scores are high sensitivity layers, the middle 40% are medium sensitivity layers, and the bottom 30% are low sensitivity layers; the value range of the clamp function is (-2...). b-1 +1,2 b-1 -1) is used to avoid the weights after quantization from exceeding the representation range of low-precision integers.

[0027] Preferably, in step S2), the first stage of the two stages is based on rotation invariance to suppress outliers in the weights; the second stage uses the inverse Hessian matrix to reduce interlayer quantization error.

[0028] Preferably, in step S2), the first stage uses a randomly selected Hadamard matrix that matches the model's hidden dimensions to transform the weights and inter-block activations in the transformer block, for the attention layer W. q W k W v and MLP layer Wgate W up Multiply the weight matrix by Q -1 *diag(α); for W o W down The weight matrix is ​​multiplied by Q, and the linear part of the RMS Orm is fused into adjacent weights to satisfy the following equation, so that the model output remains unchanged:

[0029] RMSNorm(X L ) = RMSNorm(X L Q T (8)Q;

[0030] In the formula, X L is the input activation; Q is the Hadamard matrix.

[0031] Preferably, in step S2), the random Hadamard matrix Q satisfies Q = QQ. T And QQ T =I.

[0032] In the preferred step S2), the second stage solves the reconstruction problem of each layer by performing quantization layer by layer, and for the weight W of the linear layer L... L Input to activate X L The goal is to find the quantization weight matrix that minimizes the squared error relative to the full-precision layer output. The specific expression is:

[0033]

[0034] Specifically, the steps include the following:

[0035] S221) Initialize the quantization output matrix Quantization error matrix B is the size of the block; for the inverse Hessian matrix H -l =(2X L X L T +λI) -1 Perform Cholesky decomposition:

[0036] H -1 ←Cholesky(H -1 ); (twenty one)

[0037] Where λ represents the regularization parameter; I represents the identity matrix;

[0038] S222), Traversing the weight matrix W by block size B L The column;

[0039] For each column j∈[i,i+B-1] in the current block, perform the following:

[0040] Quantitative operation: Q :,j ←quant(W L:,j ); (twenty two)

[0041] Error calculation:

[0042] In the formula, Q :,j For W L The weights of the j-th column after quantization; E :,j-i This represents the quantization error in the (ji)th column within the current block, where i represents the starting index of the block (i = 0, B, 2B, ...); [H -1 ] jj This represents the value at the intersection of the j-th row and j-th column of the inverse Hessian matrix;

[0043] S223), Update the weights within the block;

[0044]

[0045] In the formula, Represents the weight matrix W L The columns from column j to column (i+B) represent the remaining columns within the current block; E :,j-i This represents the quantization error of the (ji)th column within the current block; This represents the elements in the j-th row and j-th column to (i+B) column of the inverse Hessian matrix;

[0046] S224), Update the global weights between blocks:

[0047]

[0048] In the formula, W :,(i+B): This indicates the weighted block following column (i+B); Let represent the row vector following the i-th row and (i+B)-th column of the inverse Hessian matrix.

[0049] Preferably, in step S3), offline smoothing processing of activated outliers based on equivalent transformation is performed, specifically including the following steps:

[0050] S31) Through forward inference with calibration data, calculate the maximum absolute value (max(|X)) of the activation of each linear layer in the Channel dimension. j |) ∝ With weight in C i The maximum absolute value of the dimension, max(|W) j |) 1-∝ Calculate the smoothing factor S for each channel.j ;Right now:

[0051]

[0052] In the formula, α represents the scaling factor; S j X represents the smoothing factor for the j-th channel; j W represents the activation value of the j-th channel; j This represents the weight of the j-th channel;

[0053] S32), by using the channel smoothing factor S j Embed the RMSNorm parameters of the preceding module, and divide the RMSNorm module weights by S. j ;

[0054] In W q W k W v W gate W up Multiply by S in the linear layer weights j This enables the migration of activation outliers to weights while maintaining the equivalence of the linear layer output with the original output, i.e.:

[0055]

[0056] In the formula, Y is the output of the linear layer; S is the channel smoothing factor; X represents the activation value of the linear layer; and W represents the weight of the linear layer. and These represent the activation value and weight after smoothing, respectively.

[0057] The beneficial effects of this invention are as follows:

[0058] 1. The present invention identifies key weights that are sensitive to quantization error based on a hierarchical mixed precision quantization strategy using sensitivity analysis. Thus, a differentiated strategy is adopted in the quantization process to retain the precision of high-sensitivity weights and implement more aggressive quantization on low-sensitivity weights to minimize model performance loss. Differentiated quantization reduces model performance loss by more than 30% in low-bit scenarios.

[0059] 2. The present invention uses a two-stage weight quantization approach to solve the problems of outliers and error accumulation: the Hadamard transformation transforms the long-tailed distribution of weights into a uniform distribution, reducing quantization error by 40%; the inverse Hessian optimization reduces the error accumulation of deep networks by 50%.

[0060] 3. This invention migrates the quantization difficulty from activation to weight offline, improving the friendliness of activation quantization, and eliminating the need for dynamic detection of outliers at runtime, thus avoiding impact on inference speed; inference speed is improved by 15%-20%; it is suitable for real-time inference on the edge; the sensitivity layer uses 2-bit quantization, further compressing the model storage by 4 times compared to 8-bit quantization, reducing memory usage by 75%, and consuming less hardware resources. Attached Figure Description

[0061] Figure 1 This is a schematic flowchart of the method of the present invention;

[0062] Figure 2 This is a weight distribution diagram of q_proj and k_proj in the last layer of LLaMA-7B in this embodiment of the invention;

[0063] Figure 3 This is a heatmap of the weighted sensitivity of q_proj and k_proj in the last layer of an embodiment of the present invention.

[0064] Figure 4 This is a graph showing the quantization error curves for different layer depths and layer types in the model of this invention embodiment;

[0065] Figure 5 This is a model structure diagram for inserting a random Hadamard matrix in an embodiment of the present invention;

[0066] Figure 6 This is a distribution diagram of the weights and absolute activation values ​​before and after smoothing the last layer k_proj in an embodiment of the present invention. Detailed Implementation

[0067] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0068] like Figure 1 As shown, this embodiment provides a low-bit, large-model quantization method based on sensitivity analysis and outlier handling, including the following steps:

[0069] S1) The model layer is divided into high, medium, and low sensitivity layers using a hierarchical mixed-precision quantization strategy based on sensitivity analysis, and different quantization operations are performed accordingly; specifically, the following steps are included:

[0070] S11) Select a small calibration input set X, and record the input activation and output of each layer of the model through forward propagation;

[0071] S12), for the weight matrix W of the l-th layer l ∈R m×n Define the quantization sensitivity ΔE∈R m×n The original weighted predicted value W l X and the predicted value of the quantized weight W l′ The error between X;

[0072] S13) The sensitivity ΔE is quantized by performing a second-order Taylor expansion of the perturbation of the objective function E(w) at local points, and the gradient term of the convergent model is ignored, resulting in:

[0073]

[0074] In the formula, δ w This is the perturbation value introduced by weight quantization, where H is the Hessian matrix; quant() is the quantization function. It is the unit selection vector where position q is 1 and all other positions are 0; w q Let w be the q-th element of w; w is the weight parameter of the model, i.e., the quantization object; T represents the transpose operation; W l ′ Represents the quantized weights; g represents the first derivative of the objective function; O represents higher-order infinitesimal terms;

[0075] S14) The optimization problem is constructed using the Lagrange multiplier method, namely:

[0076]

[0077] pass The optimal perturbation is obtained:

[0078]

[0079] After substituting into the error formula, the analytical expression for quantization sensitivity is:

[0080]

[0081] In the formula, L represents the Lagrange function; H -1 Denotes the inverse of the Hessian matrix, [H -1 ] qq H represents -1 The value at the intersection of row q and column q; H represents -1 The qth column.

[0082] S15) Based on the sensitivity score, the model layer is divided into high, medium, and low sensitivity layers. The scaling factor s and the quantized weight W are calculated using the following quantization strategy. q ; and through the quantized weight W q Quantization is performed, with different sensitivity levels using different quantization bit depths; where:

[0083]

[0084] In the formula, the clamp function ensures that the value does not exceed the valid range, round indicates that the rounding operation is performed, b is the number of quantization bits, b=8 bits for high sensitivity layers, b=4 bits for medium sensitivity layers, and b=2 bits for low sensitivity layers.

[0085] like Figure 2 As shown, the left and right figures represent the weights of q_proj and k_proj, respectively. From the perspective of the output channels, the absolute values ​​of the weights are relatively large in some input channel dimensions. Because the weight quantization uses a per-channel strategy, each output channel is independently assigned a scaling factor, and outliers in the input channel dimensions directly affect the quantization effect.

[0086] Figure 3 The heatmaps show the sensitivity of the q_proj and k_proj weights in the last layer; the left image shows the sensitivity of q_proj, and the right image shows the sensitivity of k_proj. The darker the color in the heatmap, the higher the sensitivity (the larger the ΔE), indicating that the weight at that position is more sensitive to quantization error and requires higher precision. The 2000-3000 range of the q_proj input channel and the 1000-1500 range of the k_proj input channel have the highest sensitivity and require higher bit quantization.

[0087] Figure 4 The distribution of quantization error for different layer types and depths is shown. The difference in layer type is reflected in the fact that the weights gate_proj, up_proj, and down_proj in the MLP layer are more sensitive to quantization, while the sensitivity of the attention output projection weight self_attn.o_proj is lower. The effect of layer depth is reflected in the fact that the quantization error increases significantly from the 16th layer, which is related to the cumulative amplification effect of the weight quantization error in deep networks.

[0088] S2) Employ a two-stage joint quantization method to suppress outliers in weights and perform weight quantization layer by layer;

[0089] The first stage of the two phases uses rotation invariance to suppress outliers in the weights; the second stage uses the inverse Hessian matrix to reduce interlayer quantization error.

[0090] The first stage selects a random Hadamard matrix that matches the hidden dimension of the model to transform the weights and inter-block activations in the transformer block, and merges the linear part of the RMSNorm into the adjacent weight matrix; at the same time, a random Hadamard matrix Q with a size matching the hidden dimension is selected.

[0091] like Figure 5 As shown, for the attention layer W q Wk W v and MLP layer W gate W up Multiply the weight matrix by Q -1 *diag(α); for W o W down The weight matrix is ​​multiplied by Q, and the linear part of the RMS Orm is fused into adjacent weights to satisfy the following equation, so that the model output remains unchanged:

[0092] RMSNorm(X L ) = RMSNorm(X L Q T (8)Q;

[0093] W q ←Q -1 *diag(α)*W q (9)

[0094] W k ←Q -1 *diag(α)*W k (10)

[0095] W v ←Q -1 *diag(α)*W v (11)

[0096] W o ←W o *Q; (12)

[0097] W gate ←Q -1 *diag(α)*W gafe (13)

[0098] W up ←Q -1 *diag(α)*W up (14)

[0099] W down ←W down *Q; (15)

[0100] In the formula, W q W k W v These are the q, k, and v weight matrices of the attention layer, respectively; W gate W up W downLet X represent the weight matrices of gate_proj, up_proj, and down_proj, respectively; diag(α) represents a diagonal matrix formed by the scaling factor α of RMSNorm, which is absorbed into the adjacent weight matrices; X L Activation is the input; Q is the Hadamard matrix; W o It is the output projection weight; Q -1 It is the inverse Hadamard matrix;

[0101] In the standard Transformer architecture, the calculation of the attention output can be expressed as:

[0102]

[0103] In the formula, Y is the final output of the attention layer in the standard Transformer structure; P h It is an attention score matrix; It is the value vector of the h-th head;

[0104] Value vector V h and output projection weight W o Matrix multiplication is susceptible to outliers during quantization, leading to significant low-precision quantization errors; however, the output is kept constant through a first-stage Hadamard transform; that is:

[0105]

[0106]

[0107] In the formula, This represents the projection weight matrix of the value vector of the h-th attention head; express The result after performing the Hadamard transform; Let h be the output projection weight matrix of the h-th attention head; for The result after performing the Hadamard transform; The Hadamard matrix; The final output after performing the Hadamard transform on the attention layer in the Transformer structure remains unchanged from the original output Y.

[0108] Due to the Hadamard matrix satisfy Therefore, the output remains unchanged through the first-level Hadamard transform. This transform distributes outliers across the entire vector space, changing the distribution from long-tailed to uniform, which can be seamlessly integrated into the attention module.

[0109] The second stage addresses the reconstruction problem at each layer by performing quantization layer by layer, specifically the weight W of the linear layer L. LInput to activate X L The goal is to find the quantization weight matrix that minimizes the squared error relative to the full-precision layer output. The specific expression is:

[0110]

[0111] Specifically, the steps include the following:

[0112] S221) Initialize the quantization output matrix Quantization error matrix B is the size of the block; for the inverse Hessian matrix H -1 =(2X L X L T +λI) -1 Perform Cholesky decomposition:

[0113] H -1 ←Cholesky(H -1 ); (twenty one)

[0114] Where λ represents the regularization parameter; I represents the identity matrix;

[0115] S222), Traversing the weight matrix W by block size B L The column;

[0116] For each column j∈[i,i+B-1] in the current block, perform the following:

[0117] Quantitative operation: Q :,j ←quant(W L:,j ); (twenty two)

[0118] Error calculation:

[0119] In the formula, Q :,j For W L The weights of the j-th column after quantization; E :,j-i This represents the quantization error in the (ji)th column within the current block, where i represents the starting index of the block (i = 0, B, 2B, ...); [H -1 ] jj This represents the value at the intersection of the j-th row and j-th column of the inverse Hessian matrix; The weight matrix W L The j-th column;

[0120] S223), Update the weights within the block;

[0121]

[0122] In the formula, Represents the weight matrix W L The columns from column j to column (i+B) represent the remaining columns within the current block; E :,j-i This represents the quantization error of the (ji)th column within the current block; This represents the elements in the j-th row and j-th column to (i+B) column of the inverse Hessian matrix;

[0123] S224), Update the global weights between blocks:

[0124]

[0125] In the formula, W :,(i+B) : indicates the weighted block following column (i+B); Let represent the row vector following the i-th row and (i+B)-th column of the inverse Hessian matrix.

[0126] In the implementation process, the weights are first grouped into multiple blocks. All parameters within a block are quantized column by column. After each parameter is quantized, the other unquantized parameters within that block need to be adjusted appropriately to compensate for the precision loss caused by quantization. Quantization, error calculation, and internal weight updates are performed sequentially on each weight column within the current block. After completing the quantization within the current block, the error matrix of the current block is used to update all subsequent weight blocks.

[0127] S3) Perform offline smoothing of activation outliers based on equivalent transformation; specifically including the following steps:

[0128] S31) Through forward inference with calibration data, calculate the maximum absolute value (max(|X)) of the activation of each linear layer in the Channel dimension. j |) ∝ With weight in C i The maximum absolute value of the dimension, max(|W) j |) 1-∝ Calculate the smoothing factor S for each channel. j ;Right now:

[0129]

[0130] In the formula, α represents the scaling factor; S j X represents the smoothing factor for the j-th channel; j W represents the activation value of the j-th channel; j This represents the weight of the j-th channel;

[0131] S32), by using the channel smoothing factor S j Embed the RMSNorm parameters of the preceding module, and divide the RMSNorm module weights by S.j ;

[0132] In W q W k W v W gate W up Multiply by S in the linear layer weights j This enables the migration of activation outliers to weights while maintaining the equivalence of the linear layer output with the original output, i.e.:

[0133]

[0134] In the formula, Y is the output of the linear layer; S is the channel smoothing factor; X represents the activation value of the linear layer; and W represents the weight of the linear layer. and These represent the activation and weights after smoothing, respectively.

[0135] like Figure 6 As shown, the top figure shows the weight distribution before and after smoothing (uniform amplitude), and the bottom figure shows the activation distribution before and after smoothing (outliers disappear). The smoothing factor reduces the maximum activation value from 12 to 1.2 (a 90% reduction), while the weight amplitude only increases from 0.4 to 4 (a 10-fold increase). Activation outliers are effectively migrated to the weight layer, making the smoothed activation distribution closer to the ideal quantization range, while the weight layer still maintains its smoothing properties.

[0136] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A low-bit, large-model quantization method based on sensitivity analysis and outlier handling, characterized in that, Includes the following steps: S1) The model layer is divided into high, medium and low sensitivity layers using a hierarchical mixed precision quantization strategy based on sensitivity analysis, and different quantization operations are performed. S2) Employ a two-stage joint quantization method to suppress outliers in weights and perform weight quantization layer by layer; S3) Perform offline smoothing of activated outliers based on equivalent transformation.

2. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 1, characterized in that: In step S1), the model layer is divided into high, medium, and low sensitivity layers using a hierarchical mixed precision quantization strategy based on sensitivity analysis, and different quantization operations are performed. Specifically, the steps are as follows: S11) Select a small calibration input set X, and record the input activation and output of each layer of the model through forward propagation; S12), for the weight matrix W of the l-th layer l ∈R m×n Define the quantization sensitivity ΔE∈R m×n The original weighted predicted value W l X and the predicted value of the quantized weight W l The error between 'X' and 'X', where m and n are the weight matrices W, respectively. l Width and height; S13) The sensitivity ΔE is quantized by performing a second-order Taylor expansion of the perturbation of the objective function E(w) at local points, and the gradient term of the convergent model is ignored, resulting in: In the formula, δ w This is the perturbation value introduced by weight quantization, where H is the Hessian matrix; quant() is the quantization function. It is the unit selection vector where position q is 1 and all other positions are 0; w q Let w be the q-th element of w; w is the weight parameter of the model, i.e., the quantization object; T represents the transpose operation; S14) The optimization problem is constructed using the Lagrange multiplier method, namely: pass The optimal perturbation is obtained: After substituting into the error formula, the analytical expression for quantization sensitivity is: In the formula, L represents the Lagrange function; H -1 Denotes the inverse of the Hessian matrix, [H -1 ] qq H represents -1 The value at the intersection of row q and column q; H represents -1 The qth column; S15) Based on the sensitivity score, the model layer is divided into high, medium, and low sensitivity layers. The scaling factor s and the quantized weight W are calculated using the following quantization strategy. q ; and through the quantized weight W q Quantization is performed, with different sensitivity levels using different quantization bit depths; where: In the formula, the clamp function ensures that the value does not exceed the valid range, round indicates that the rounding operation is performed, and b is the number of quantization bits.

3. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 2, characterized in that: In step S1), the number of samples in the calibration input set X is required to cover common input distributions of the model. The Hessian matrix is ​​calculated through layer-by-layer forward propagation and gradient backtracking, ignoring the gradient term g after model convergence. T δ w Only the second-order terms are retained to simplify the calculation.

4. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 2, characterized in that: In step S1), the threshold for dividing the high, medium, and low sensitivity layers is determined based on the statistical distribution of sensitivity scores: the top 30% of the sensitivity scores are high sensitivity layers, the middle 40% are medium sensitivity layers, and the bottom 30% are low sensitivity layers; the value range of the clamp function is (-2...). b-1 +1,2 b-1 -1) is used to avoid the weights after quantization from exceeding the representation range of low-precision integers.

5. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 1, characterized in that: In step S2), the first stage of the two stages is based on rotation invariance to suppress outliers in the weights; the second stage uses the inverse Hessian matrix to reduce interlayer quantization error.

6. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 5, characterized in that: In step S2), the first stage uses a randomly selected Hadamard matrix that matches the model's hidden dimensions to transform the weights and inter-block activations in the transformer block. For the attention layer W... q W k W v and MLP layer W gate W up Multiply the weight matrix by Q -1 *diag(α); for W o W down The weight matrix is ​​multiplied by Q, and the linear part of the RMS Orm is fused into adjacent weights to satisfy the following equation, so that the model output remains unchanged: RMSNorm(X L )=RMSNorm(X L Q T )Q; (8) In the formula, X L is the input activation; Q is the Hadamard matrix.

7. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 5, characterized in that: In step S2), the second stage solves the reconstruction problem of each layer by performing quantization layer by layer. For the weight W of the linear layer L, L Input to activate X L The goal is to find the quantization weight matrix that minimizes the squared error relative to the full-precision layer output. The specific expression is:

8. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 7, characterized in that: In step S2), the second stage involves performing quantization layer by layer, specifically including the following steps: S221) Initialize the quantization output matrix Quantization error matrix B is the size of the block; for the inverse Hessian matrix H -1 =(2X L X L T +λI) -1 Perform Cholesky decomposition: H -1 ←Cholesky(H -1 ); (21) Where λ represents the regularization parameter; I represents the identity matrix; S222), Traversing the weight matrix W by block size B L The column; For each column j∈[i,i+B-1] in the current block, perform the following: Quantitative operation: Q :,j ←quant(W L:,j ); (twenty two) Error calculation: In the formula, Q :,j For W L The weights of the j-th column after quantization; E :,j-i This represents the quantization error in the (ji)th column within the current block, where i represents the starting index of the block (i = 0, B, 2B, ...); [H -1 ] jj This represents the value at the intersection of the j-th row and j-th column of the inverse Hessian matrix; S223), Update the weights within the block; In the formula, W L:,j:(i+N) Represents the weight matrix W L The columns from column j to column (i+B) represent the remaining columns within the current block; E :,j-i This represents the quantization error of the (ji)th column within the current block; This represents the elements in the j-th row and j-th column to (i+B) column of the inverse Hessian matrix; S224), Update the global weights between blocks: In the formula, W :,(i+B): This indicates the weighted block following column (i+B); Let represent the row vector following the i-th row and (i+B)-th column of the inverse Hessian matrix.

9. The low-bit large-model quantization method based on sensitivity analysis and outlier processing according to claim 1, characterized in that: In step S3), offline smoothing of activation outliers based on equivalent transformation is performed, specifically including the following steps: S31) Through forward inference with calibration data, calculate the maximum absolute value (max(|X)) of the activation of each linear layer in the Channel dimension. j |) ∝ With weight in C i The maximum absolute value of the dimension, max(|W) j |) 1-∝ Calculate the smoothing factor S for each channel. j ;Right now: In the formula, α represents the scaling factor; S j X represents the smoothing factor for the j-th channel; j W represents the activation value of the j-th channel; j This represents the weight of the j-th channel; S32), by using the channel smoothing factor S j Embed the RMSNorm parameters of the preceding module, and divide the RMSNorm module weights by S. j ; In W q W k W v W gate W up Multiply by S in the linear layer weights j This enables the migration of activation outliers to weights while maintaining the equivalence of the linear layer output with the original output, i.e.: In the formula, Y is the output of the linear layer; S is the channel smoothing factor; X represents the activation value of the linear layer; and W represents the weight of the linear layer. and These represent the activation value and weight after smoothing, respectively.

Images