A method for reducing layernorm quantization error
By calculating feature quantization parameters and calibrating the weights and biases of LayerNorm, the problem of errors in the quantization process is solved, improving the accuracy and performance of deep learning models, especially in the Transformer architecture for vision and generative models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INGENIC SEMICON CO LTD
- Filing Date
- 2024-12-20
- Publication Date
- 2026-06-23
AI Technical Summary
During the quantization process of deep learning models, the mean and variance of LayerNorm introduce significant errors, affecting model performance, especially in complex tasks and large model networks. Existing methods reduce the loss by increasing the quantization bit width, but this leads to a decrease in computational performance.
By calculating feature quantization parameters, saving the output features of each layer in the neural network, calculating scaling and bias parameters, and calibrating the weights and biases of LayerNorm, quantization errors are reduced.
Without increasing computational cost, it reduces quantization error and improves the accuracy of quantized features. It is suitable for LayerNorm layers in the Transformer architecture, especially in visual and generative models, improving the recognition and generation performance of the models.
Smart Images
Figure CN122263977A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of deep learning technology, and specifically relates to a method for reducing layer norm quantization error. Background Technology
[0002] With the advancement of technology, the complexity of deep learning models is increasing. As model size expands, computational and memory overhead increases significantly, leading to a pressing need for efficient inference. Quantization techniques, which reduce numerical precision (e.g., converting floating-point numbers to integers), can reduce model size and improve inference speed while maintaining performance as much as possible. LayerNorm is a normalization technique used in deep neural networks. It normalizes the output of each neuron in the network, ensuring that the outputs of each layer have a similar distribution. LayerNorm aims to reduce the size differences between different training samples by normalizing all activation values of a single sample, thereby helping to improve training speed and model performance. It is used in deep learning models to stabilize training and improve convergence speed; however, maintaining its normalization effect during quantization remains a challenge.
[0003] In other words, the current problem is that the mean and variance of LayerNorm may introduce significant errors during the quantization process, affecting model performance, especially in complex tasks. The LayerNorm operator is commonly used in large model networks. Since it dynamically calculates the mean and variance of the current feature during its calculation, the loss caused by quantization itself is further amplified in the LayerNorm layer, leading to increased quantization difficulty.
[0004] Furthermore, the layernorm calculation process and formulas involved in this application are described below:
[0005] Mean:
[0006] Standard deviation:
[0007] Final calculation formula Where γ is the weight and β is the bias.
[0008] Currently, the only way to reduce the loss of layernorm quantization is to increase the quantization bit width, but increasing the quantization bit width will also lead to a decrease in model computation performance. Summary of the Invention
[0009] To address the aforementioned problems, the present invention aims to provide a method based on quantization error calibration to reduce quantization errors in layer norms within neural network models. This method can reduce quantization errors within the same quantization bit width, thereby utilizing the lowest possible quantization bit width. Currently, layer norms are a crucial component of the Transformer architecture, widely applied within each sub-layer. A typical Transformer architecture is as follows:
[0010] Input → Self-attention layer → LayerNorm → Feedforward network → LayerNorm → Output
[0011] In visual models such as Vision Transformer (ViT) and other visual models, LayerNorm is applied to the input and output of each layer. ViT uses Transformer to process image data. In some generative or reinforcement learning models, such as GPT-3 and AlphaStar, LayerNorm is also applied to each sub-layer. This method is mainly used in the quantization domain of AI deep learning network models, which can reduce the errors introduced by quantization to the layernorm layer and improve accuracy. Ultimately, small to medium-sized networks that include layernorms for image recognition and text generation can run on devices.
[0012] Specifically, this method is a method for reducing layer norm quantization error, the method comprising the following steps:
[0013] S1: Calculate the feature quantization parameters:
[0014] To quantize the output of layernorm, it is necessary to determine the numerical range of the features and obtain the minimum and maximum values by statistically analyzing the feature values in the dataset.
[0015] Specifically, after normalization is completed following layernorm, a node is added to store batch layernorm outputs. When the model runs through a small batch of data, the range of which is between 0.0 and 1.0, this node stores the output results of this batch of layernorm. Then, the quantization range is determined by the maximum and minimum values in the statistical data.
[0016] After obtaining the maximum and minimum values, assuming quantization to 8 bits, calculate the scaling factor scale and the zero point. These parameters are used for linear transformation to map floating-point numbers to integer values.
[0017] S2: Saves the features of the output of each layer in the neural network:
[0018] Here, the feature results of all operators in the floating-point network are saved for use in subsequent calculations to detect errors.
[0019] Specifically, in the PyTorch framework, the output of each layer can be captured using hook functions. The hook function `def hook(self, module, input, output)` captures the input and output of each layer in the model. Here, `module` is the module of the current layer, usually a subclass of `nn.Module`; `input` is the input data passed to this layer; and `output` is the output data of this layer, usually the result after computation by this layer.
[0020] S3: Calculate scaling and bias parameters:
[0021] This step refers to the feature information of the layer norm in the floating-point network and the quantized feature information to calculate the scaling and bias parameters, and further includes:
[0022] S3.1, retrieve data from layerorm. Feature data;
[0023] This step requires reverse processing of the feature;
[0024] Because the calculation process of the layernorm layer is weight Therefore, we need to find The result of layernorm calculation (feature-biasβ) needs to be divided by weight γ to obtain... Let it be denoted as feature_pre;
[0025] The feature_pre of the layer norm of a float is denoted as f_Lf;
[0026] The quantized layer norm feature_pre is denoted as q_Lf;
[0027] S3.2, Calculate the scaling factor:
[0028] The scaling factor (scales) is calculated using the covariance and variance of f_Lf and q_Lf.
[0029] The following is the Python code implementation for calculating scales:
[0030]
[0031] The scaling factor (scales) can be used to further reduce the errors in quantization and floating-point results;
[0032] S3.3, Calculation error bias:
[0033] S3.3.1 When calculating the error bias, the weight γ needs to be updated. The goal is to integrate the calculated scaling factor sclaes into the weight γ, denoted as new_gamma.
[0034] new_gamma = weight γ × scales, which is equivalent to updating the weights by incorporating the scales that reduce quantization error;
[0035] S3.3.2, Update the bias used for bias correction: f_Lf, q_LF
[0036] f_LF = f_LF × weight γ;
[0037] q_LF = q_LF × new_gamma;
[0038] S3.3.3, use the updated f_LF and q_LF to calculate the bias:
[0039] The bias is calculated by calculating the average error.
[0040] The following is the completed code:
[0041]
[0042] S3.4 After the scales and bias are calculated, the error after quantization can be reduced by combining the weights γ and biasβ fused into the layernorm. This is actually a correction operation.
[0043] The new biasβ is denoted as new_bias;
[0044] new_bias = biasβ + bias;
[0045] This completes the bias calibration of the layer norm for one layer;
[0046] S3.5 At this point, the new weights, i.e., new_gamma and bias, i.e., new_bias, need to be updated in the corresponding layernorm layer of the quantization network. Considering the error that calibration may bring at this time, the weights and biases in the subsequent layernorm are updated in turn.
[0047] In step S1, the scaling factor (scale) and the zero point (zero_point) are defined as follows: for the layernorm output feature, the quantization parameter is defined as quantize_scale, which is the quantization bit width value divided by the feature range; and the zero point is defined as zero_point, which represents the integer value of 0 within the quantization bit width range.
[0048] Step S1 is further implemented using Python code, including:
[0049]
[0050] The implementation of step S2 further includes:
[0051] Import functions and classes from the PyTorch library into the current file: import torch;
[0052] Define a class, class FeatureExtractor:
[0053] def__init__(self, model):
[0054] self.model = model
[0055] self.features = [];
[0056] Register hook:
[0057] for layer in model.children():
[0058] layer.register_forward_hook(self.hook)
[0059] def hook(self,module,input,output):
[0060] self.features.append(output)
[0061] def clear(self):
[0062] self.features = []
[0063] use:
[0064] model = a neural network model;
[0065] extractor=FeatureExtractor(model);
[0066] The main function of this FeatureExtractor class is:
[0067] Feature extraction: It captures the output features of each layer of a neural network model by registering hooks;
[0068] Saving features: The output of each captured layer is added to the self.features list, so that the output of these intermediate layers can be analyzed later;
[0069] Assuming the input is:
[0070] input_data=torch.randn(1,3,224,224);
[0071] Forward propagation:
[0072] output = model(input_data);
[0073] Saved features:
[0074] features=extractor.features;
[0075] The feature information of all nodes in the neural network can be obtained through the hook function. The feature information in the floating-point network is saved for calibration. All quantized feature information of the network is quantized using the quantization parameters calculated in step S1.
[0076] The steps and methods described herein, implemented in Python code for the entire network, include:
[0077]
[0078]
[0079] The beneficial effects of this invention include: This application primarily corrects the error by referencing the error between the floating-point feature and the quantized feature, and integrates the scaling and translation parameters into the weights and biases of the layer norm itself. This reduces quantization error without increasing additional computation. Through this method, the similarity of quantized features can be improved, and the error can be reduced. Attached Figure Description
[0080] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.
[0081] Figure 1 This is a flowchart illustrating the method.
[0082] Figure 2 This is a schematic diagram of the code involved in step S1 of this method.
[0083] Figure 3 This is a code diagram involving step S2 in this method.
[0084] Figure 4 This is a code diagram involving step S3.2 in this method.
[0085] Figure 5 This is a code diagram involving step S3.3 in this method.
[0086] Figure 6 This is a code diagram illustrating the entire network involved in this method. Detailed Implementation
[0087] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.
[0088] like Figure 1 As shown, the present invention relates to a method for reducing layer norm quantization error, comprising the following steps:
[0089] Step S1: Calculate feature quantization parameters
[0090] To quantize the output of Layernorm, the numerical range of the features needs to be determined. This is done by statistically analyzing the feature values in the dataset to obtain the minimum and maximum values. Specifically, we add a node after Layernorm that stores batches of Layernorm output. When the model runs through a small batch of data, this node stores the output results for that batch of Layernorm. Then, we determine the quantization range by analyzing the maximum and minimum values in the statistical data. After obtaining the maximum and minimum values, assuming quantization to 8 bits, we need to calculate the scaling factor and zero point. These parameters are used for linear transformation, mapping floating-point numbers to integer values.
[0091] The common quantization bit width is 8 bits, which is an 8-bit binary integer, containing 256 numbers.
[0092] Here we define the quantization parameter of the layernorm output features as quantize_scale.
[0093] Zero point is defined as zero_point
[0094] Quantize_scale: Quantization bit width value divided by the feature range.
[0095] zero_point: Represents an integer value where the value of 0 is within the quantization bit width range.
[0096] Python code implementation as follows Figure 2 As shown:
[0097] import numpy as np
[0098] #Assuming this is the feature value array of layernorm
[0099] features=np.array([0.1,0.5,1.3,2.2,3.7],dtype=np.float32)
[0100] #Get the minimum and maximum values of the feature
[0101] min_val = np.min(features)
[0102] max_val = np.max(features)
[0103] #Select the quantization bit width (here, we choose an 8-bit integer, which is 256 discrete values).
[0104] num_bits = 8
[0105] quant_levels = 2**num_bits
[0106] #Calculate scaling factor and zero point
[0107] quantize_scale=(quant_levels-1) / (max_val-min_val)
[0108] zero_point=round(-min_val / scale)
[0109] Step S2: Save the features of the output of each layer in the neural network.
[0110] Here, we save the feature results of all operators in the floating-point network for use in subsequent calculations to check errors. Specifically, we implement this using the PyTorch framework. PyTorch is an open-source deep learning framework for machine learning and deep learning. Currently, PyTorch has become the framework with the highest market share in the open-source machine learning field in scientific research, with its share at top AI conferences reaching 80% in 2022. Hook functions can be used to capture the output of each layer. For example... Figure 3 The following is an example of the implementation code:
[0111]
[0112]
[0113] The main function of this FeatureExtractor class is:
[0114] Feature extraction: It captures the output features of each layer of a neural network model by registering hooks;
[0115] Saving features: The output of each captured layer is added to the self.features list, so that the output of these intermediate layers can be analyzed later;
[0116] Assuming the input is:
[0117] input_data=torch.randn(1,3,224,224);
[0118] Forward propagation:
[0119] output = model(input_data);
[0120] Saved features:
[0121] features=extractor.features;
[0122] Hook functions can be used to obtain feature information of all nodes in a neural network.
[0123] 1. Use the feature information in the floating-point network for calibration.
[0124] 2. All quantized feature information of the network will be quantized using the quantization parameters calculated in step S1. Since this is not the focus of this application, it will not be described in detail here.
[0125] Step S3: Calculate scaling and bias parameters
[0126] This step refers to the feature information of the layer norm in the floating-point network and the quantized feature information to calculate the scaling and bias parameters.
[0127] S3.1, the first step is to obtain the data from layerorm. The feature data. This step requires reverse processing of the features;
[0128] Because the calculation process of the layernorm layer is weight Therefore, we need to find The result of layernorm calculation (feature-biasβ) needs to be divided by weight γ to obtain... Let it be denoted as feature_pre;
[0129] Find the feature_pre of the layer norm of a float, denoted as f_Lf.
[0130] The quantized layer norm feature_pre is denoted as q_Lf.
[0131] S3.2, Calculate the scaling factor (scales)
[0132] The scaling factor (scales) is calculated using the covariance and variance of f_Lf and q_Lf.
[0133] like Figure 4 As shown, the following is the Python code implementation for calculating scales:
[0134]
[0135]
[0136] The scaling factor (scales) can further reduce the quantization error in floating-point operations;
[0137] S3.3, Calculation error bias:
[0138] S3.3.1, when calculating the error bias, the weight γ needs to be updated. The goal is to incorporate the calculated scaling factor sclaes into the weight γ, denoted as new_gamma.
[0139] new_gamma = weight γ × scales, which means that the updated weights incorporate scales that reduce quantization error.
[0140] S3.3.2, Update the bias used for bias correction: f_Lf, q_LF;
[0141] f_LF = f_LF × weight γ;
[0142] q_LF = q_LF X new_gamma;
[0143] In S3.3.3, we use the updated f_LF and q_LF to calculate the bias;
[0144] The bias is calculated by calculating the average error, such as Figure 5 As shown, the following is the completed code:
[0145]
[0146] S3.4 After the scales and bias are calculated, we can reduce the error after quantization by combining the weights γ and bias β that are fused into the layer norm. This is actually a correction operation.
[0147] The new biasβ is denoted as new_bias
[0148] new_bias = biasβ + bias
[0149] At this point, we have completed the bias calibration of the layer norm for one layer.
[0150] In step S3.5, the new weights (new_gamma) and biases (new_bias) need to be updated in the corresponding layer normalization layer of the quantization network. Considering the potential errors from calibration at this point, the weights and biases in the subsequent layer normalization layers are updated accordingly. Because this affects subsequent layer normalization layers, further updates to those layers are also necessary.
[0151] The above example is for a single layer. Below is the Python code implementation for the entire network (multi-layered), as shown below. Figure 6 As shown, the following is the core part:
[0152]
[0153]
[0154] In summary, the technical solution of this application is as follows:
[0155] This method can correct the error by referencing the error between the floating-point feature and the quantization feature, and integrate the scaling and translation parameters into the weights and biases of the layer norm itself, reducing the quantization error without increasing the amount of additional computation.
[0156] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for reducing layer norm quantization error, characterized in that, The method includes the following steps: S1: Calculate the feature quantization parameters: To quantize the output of layernorm, it is necessary to determine the numerical range of the features and obtain the minimum and maximum values by statistically analyzing the feature values in the dataset. Specifically, after normalization is completed following layernorm, a node is added to store batch layernorm outputs. When the model runs through a small batch of data, the range of which is between 0.0 and 1.0, this node stores the output results of this batch of layernorm. Then, the quantization range is determined by the maximum and minimum values in the statistical data. After obtaining the maximum and minimum values, assuming quantization to 8 bits, calculate the scaling factor scale and the zero point. These parameters are used for linear transformation to map floating-point numbers to integer values. S2: Saves the features of the output of each layer in the neural network: Here, the feature results of all operators in the floating-point network are saved for use in subsequent calculations to detect errors. Specifically, in the PyTorch framework, the output of each layer can be captured using hook functions. The hook function `def hook(self, module, input, output)` captures the input and output of each layer in the model. Here, `module` is the module of the current layer, usually a subclass of `nn.Module`; `input` is the input data passed to this layer; and `output` is the output data of this layer, usually the result after computation by this layer. S3: Calculate scaling and bias parameters: This step refers to the feature information of the layer norm in the floating-point network and the quantized feature information to calculate the scaling and bias parameters, and further includes: S3.1, retrieve data from layerorm. Feature data; This step requires reverse processing of the feature; Because the calculation process of the layernorm layer is weight Therefore, we need to find The result of layernorm calculation (feature-biasβ) needs to be divided by weight γ to obtain... Let it be denoted as feature_pre; The feature_pre of the layer norm of a float is denoted as f_Lf; The quantized layer norm feature_pre is denoted as q_Lf; S3.2, Calculate the scaling factor: The scaling factor (scales) is calculated using the covariance and variance of f_Lf and q_Lf. The following is the Python code implementation for calculating scales: def compute_scale_layernorm(self,X,Y): X: represents f_Lf,, Y: represents q_Lf, that is, defcompute_scale_layernorm(self,f_Lf,q_Lf); scales = [] Number of channels: num_channels = X.shape[2] for iin range(num_channels): x = X[:,:,i] y = Y[:,:,i] Calculate the mean: x_mean = np.mean(x) y_mean = np.mean(y); Calculate covariance and variance: covariance=np.mean((x-x_mean)*(y-y_mean)) variance=np.mean((x-x_mean)**2); Calculate the scaling factor: s = covariance / variance scales.append(s) return np.array(scales); The scaling factor (scales) can be used to further reduce the errors in quantization and floating-point results; S3.3, Calculation error bias: S3.3.1 When calculating the error bias, the weight γ needs to be updated. The goal is to integrate the calculated scaling factor sclaes into the weight γ, denoted as new_gamma. new_gamma = weight γ × scales, which is equivalent to updating the weights by incorporating the scales that reduce quantization error; S3.3.2, Update the bias used for bias correction: f_Lf, q_LF f_LF = f_LF × weight γ; q_LF = q_LF × new_gamma; S3.3.3, use the updated f_LF and q_LF to calculate the bias: The bias is calculated by calculating the average error. The following is the completed code: S3.4 After the scales and bias are calculated, the error after quantization can be reduced by combining the weights γ and biasβ fused into the layernorm. This is actually a correction operation. The new biasβ is denoted as new_bias; new_bias = biasβ + bias; This completes the bias calibration of the layer norm for one layer; S3.5 At this point, the new weights, i.e., new_gamma and bias, i.e., new_bias, need to be updated in the corresponding layernorm layer of the quantization network. Considering the error that calibration may bring at this time, the weights and biases in the subsequent layernorm are updated in turn.
2. The method for reducing layer norm quantization error according to claim 1, characterized in that, In step S1, the scaling factor (scale) and the zero point (zero_point) are defined as follows: for the layernorm output feature, the quantization parameter is defined as quantize_scale, which is the quantization bit width value divided by the feature range; and the zero point is defined as zero_point, which represents the integer value of 0 within the quantization bit width range.
3. The method for reducing layer norm quantization error according to claim 2, characterized in that, Step S1 is further implemented using Python code, including: Adding a node is represented as `import numpy as np`; Assume this is the eigenvalue array of layernorm: features=np.array([0.1,0.5,1.3,2.2,3.7],dtype=np.float32); Find the minimum and maximum values of a feature: min_val = np.min(features); max_val = np.max(features); Choose the quantization bit width. Here, we choose an 8-bit integer, which is 256 discrete values: that is, the quantization bit width is 8 bits, an 8-bit binary integer, and 256 numbers. num_bits = 8; quant_levels = 2**num_bits; Calculate the scaling factor and zero point: quantize_scale=(quant_levels-1) / (max_val-min_val); zero_point=round(-min_val / scale).
4. The method for reducing layer norm quantization error according to claim 1, characterized in that, The implementation of step S2 further includes: Import functions and classes from the PyTorch library into the current file: import torch; Define a class, class FeatureExtractor: def__init__(self, model): self.model = model self.features = []; Register hook: for layer in model.children(): layer.register_forward_hook(self.hook) def hook(self,module,input,output): self.features.append(output) def clear(self): self.features = [] use: model = a neural network model; extractor=FeatureExtractor(model); The main function of this FeatureExtractor class is: Feature extraction: It captures the output features of each layer of a neural network model by registering hooks; Saving features: The output of each captured layer is added to the self.features list, so that the output of these intermediate layers can be analyzed later; Assuming the input is: input_data=torch.randn(1,3,224,224); Forward propagation: output = model(input_data); Saved features: features=extractor.features; The feature information of all nodes in the neural network can be obtained through the hook function. The feature information in the floating-point network is saved for calibration. All quantized feature information of the network is quantized using the quantization parameters calculated in step S1.
5. The method for reducing layer norm quantization error according to claim 1, characterized in that, The steps and methods described herein, implemented in Python code for the entire network, include: for name,module in self.wrappers.items(): trace_idx = module.index if isinstance(module,LayerNormal): Traverse the entire network model and find the layernorm layer; alpha,beta=self.get_alpha_beta_param(name) if str(trace_idx)in float_outputs: Conditional statement to determine whether the layer is storing floating-point results; float_feature=float_outputs[str(trace_idx)] quant_feature=quant_outputs[str(trace_idx)] div_float_feature=(float_feature-beta) / alpha div_quant_feature=(quant_feature-beta) / alpha scales = self.compute_scale_layernorm(div_quant_feature, div_float_feature), where div_quant_feature represents the quantization of the layernorm calculation. The value of div_float_feature represents the floating-point layernorm calculation. The value of is used to calculate the error scale; div_quant_feature=div_quant_feature*scales bias = self.compute_bias_layernorm(div_quant_feature, div_float_feature), where add_quant_feature represents the quantization of the layernorm calculation. The value is multiplied by the scaling error value, and div_float_feature is still calculated for floating-point layer norm. The value is used to calculate the bias. add_quant_feature=add_quant_feature+bias alpha = alpha * scales beta = beta + bias self.set_alpha_beta_param(name,alpha,beta).