Quantization methods, devices, and electronic equipment for normalization operators in neural network models

Abstract

This application provides a quantization method, apparatus, electronic device, and computer-readable storage medium for a normalization operator in a neural network model, relating to the field of neural networks. The method includes: obtaining a first quantization coefficient based on floating-point output data, quantization bit width, floating-point scaling parameters, and a first shift number; obtaining a fixed-point normalization result based on fixed-point input data, a second shift number, and the quantization bit width; obtaining initial fixed-point output data based on fixed-point scaling parameters, fixed-point translation parameters, and the fixed-point normalization result; quantizing the initial fixed-point output data according to the first quantization coefficient; and shifting the quantization result using the first shift number to obtain the fixed-point output data. This application avoids the errors caused by traditional methods of calculating the mean and variance by eliminating the need to calculate all mean and variance. For partial division operations, a shifting method based on the first and second shift numbers is used, greatly reducing the computational load and improving computational efficiency.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and more specifically, to a quantization method, apparatus, and electronic device for a normalization operator in a neural network model. Background Technology

[0002] Artificial intelligence (AI) technology is being applied more and more widely in people's production and daily lives. Neural network models can process data such as text, video, audio, and images to obtain the desired results. For example, when a user is on a mobile phone, the phone's built-in facial recognition model can process captured facial images to identify the person; another example is that a phone's built-in text recognition model can process a passage on a webpage to determine its source; yet another example is that a phone's built-in audio matching model can process audio to identify voices / songs, and so on.

[0003] Because neural network models are resource-intensive algorithms with high computational costs and memory consumption, the neural network models built into electronic devices are usually quantized versions. Neural network model quantization generally refers to converting high-precision floating-point calculations in a neural network model into fixed-point calculations, resulting in a fixed-point neural network model.

[0004] A normalization operator is an operator that can be used in neural network models to process feature data during the operation of the neural network model. For example, in the face recognition model mentioned above, the normalization operator in the normalization layer can process the feature data associated with the face image input from the previous layer, thereby obtaining new, processed feature data associated with the face image.

[0005] Because the radical calculations in the normalization operator are floating-point calculations, the current method for normalizing fixed-point data in neural networks involves dequantizing the fixed-point data, performing floating-point operations, and then quantizing the result before outputting the fixed-point data. This method is computationally intensive, resulting in high computational costs and slow processing speed. Summary of the Invention

[0006] This application provides a quantization method, apparatus, electronic device, computer-readable storage medium, and computer program product for normalization operators in neural network models, which can solve the above-mentioned problems of the prior art. The technical solution is as follows:

[0007] According to one aspect of the embodiments of this application, a quantization method for a normalization operator in a neural network model is provided, the method comprising:

[0008] Determine the fixed-point input data Q of the normalization operatorinputs Quantization bit width b, floating-point scaling parameter γ, fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Floating-point output data outputs, first shift number N outputs And the second shift number exp;

[0009] Based on the floating-point output data outputs, quantization bit width b, floating-point scaling parameter γ, and first shift number N outputs Obtain the first quantization coefficient M outputs ;

[0010] Based on the fixed-point input data Q inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ;

[0011] Based on the fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Fixed-point normalization result Q norm Obtain initial fixed-point output data; based on the first quantization coefficient M outputs The initial fixed-point output data is quantized, and the quantization result is shifted by the first shift number N. outputs Perform a shift operation to obtain the fixed-point output data Q. outputs ;

[0012] Wherein, the first shift number N outputs The input data, not exceeding the quantization bit width b, is a feature vector representing preset information, and the second shift bit is even and 2. exp The length N of the feature vector that is closest to it is a power of 2.

[0013] Based on the above embodiments, as an optional embodiment, the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N are used. outputs Obtain the first quantization coefficient M outputs ,include:

[0014] The second quantization coefficients outputs are determined based on the first difference between the maximum and minimum values ​​of the floating-point output data outputs and the quantization bit width b. scale ;

[0015] The third quantization coefficient γ is determined based on the second difference between the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b. scale ;

[0016] The fourth quantization coefficient norm is determined based on the length N and the quantization bit width b. scale ;

[0017] According to the second quantization coefficient, outputs scale Third quantization coefficient γ scale and the fourth quantization coefficient norm scale Determine the fifth quantization coefficient;

[0018] The fifth quantization coefficient is shifted by the first shift number N. outputs Perform a shift operation to obtain the first quantization coefficient M. outputs .

[0019] Based on the above embodiments, as an optional embodiment, according to the fixed-point input data Q inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ,include:

[0020] For fixed-point input data Q inputs Take the mean to obtain the fixed-point mean Q. μ ;

[0021] Determine the fixed-point input data Q inputs and fixed-point mean Q μ The degree of difference between them is shifted by the second shift number exp to determine the reciprocal of the square root of the fixed-point variance Q. alpha ;

[0022] Based on the fixed-point input data Q inputs Fixed-point mean Q μ and the reciprocal of the square root of the fixed-point variance Q alpha Determine the initial fixed-point normalization result Q norm_tmp ;

[0023] According to Q alpha The number of bits and the second shifted bit exp are used to normalize the initial fixed-point result Q. norm_tmp Shift to the number of bits of quantization width to determine the fixed-point normalization result Q. norm .

[0024] Based on the above embodiments, as an optional embodiment, the fixed-point scaling parameter Q γ It is determined in the following ways:

[0025] The third quantization coefficient γ is determined based on the third difference between the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b. scale ;

[0026] Based on the floating-point scaling parameter γ and the third quantization coefficient γ scale Determine the fixed-point scaling parameter Q γ .

[0027] Based on the above embodiments, as an optional embodiment, the fixed-point translation parameter Q is determined. β ,include:

[0028] The fourth quantization coefficient norm is determined based on the length N and the quantization bit width b. scale ;

[0029] Based on the quantization coefficient norm scale and quantization coefficient γ scale Determine the sixth quantization coefficient β scale ;

[0030] Determine the floating-point shift parameter β, based on the floating-point shift parameter β and the quantization coefficient β. scale Determine the fixed-point translation parameter Q β .

[0031] Based on the above embodiments, as an optional embodiment, according to the fixed-point input data Q inputs Fixed-point mean Q μ and the reciprocal of the square root of the fixed-point variance Q alpha Determine the initial fixed-point normalization result Q norm_tmp ,include:

[0032] Determine the fixed-point input data Q inputs With fixed-point mean Q μ The fourth difference between them;

[0033] The fourth difference and the reciprocal of the square root of the fixed-point variance Q are... alpha The product of these two terms is used as the initial fixed-point normalization result Q. norm_tmp .

[0034] Based on the above embodiments, as an optional embodiment, the preset information is any one of image, text, audio, and environmental information.

[0035] According to another aspect of the embodiments of this application, a quantization device for a normalization operator in a neural network model is provided, the device comprising:

[0036] The preparation module is used to determine the fixed-point input data Q of the normalization operator. inputs Quantization bit width b, floating-point scaling parameter γ, fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Floating-point output data outputs, first shift number N outputs And the second shift number exp;

[0037] The quantization coefficient determination module is used to determine the quantization coefficients based on the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N. outputsObtain the first quantization coefficient M outputs ;

[0038] The fixed-point normalization module is used to normalize the data based on the fixed-point input data Q. inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ;

[0039] The output module is used to calculate the fixed-point scaling parameter Q. γ , Fixed-point translation parameter Q β Fixed-point normalization result Q norm Obtain initial fixed-point output data; based on the first quantization coefficient M outputs The initial fixed-point output data is quantized, and the quantization result is shifted by the first shift number N. outputs Perform a shift operation to obtain the fixed-point output data Q. outputs ;

[0040] Wherein, the first shift number N outputs The input data, not exceeding the quantization bit width b, is a feature vector representing preset information, and the second shift bit is even and 2. exp The length N of the feature vector that is closest to it is a power of 2.

[0041] According to another aspect of the present application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, the processor executing the computer program to implement the steps of the above method.

[0042] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0043] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0044] The beneficial effects of the technical solutions provided in this application are:

[0045] This application embodiment does not require calculating the mean and variance completely, thus avoiding the errors caused by traditional methods of calculating the mean and variance. Furthermore, for partial division operations, a shifting method based on the first and second shifting bits is adopted, which greatly reduces the amount of computation and improves the computational efficiency. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0047] Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application;

[0048] Figure 2 A flowchart illustrating a quantization method for a normalization operator in a neural network model provided in this application embodiment;

[0049] Figure 3 A schematic diagram of the structure of a quantization device for a normalization operator in a neural network model provided in this application embodiment;

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0052] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0054] First, let's introduce and explain several terms used in this application:

[0055] (1) Symmetric quantization: Quantize the range [X] min X max The floating-point number X is mapped to a range in [Q] through a quantization factor. min Q max The fixed-point number Q within the range is shown in Formula 1:

[0056] Q = X * S (1)

[0057] The quantization parameters include a scaling factor S, which is the smallest scale at which different floating-point numbers X are quantized to the same fixed-point number Q. Understandably, for symmetric quantization, once the scaling factor S in the quantization parameters is determined, the quantized fixed-point number Q can be obtained from the input floating-point number X using the quantization parameters. Understandably, the range of the fixed-point number Q corresponds to the quantization data type (i.e., the data type corresponding to the fixed-point number Q). Quantization data types include: int32, int16, int8, int4, uint32, uint16, uint8, or uint4, etc. For example, if the quantization data type is int8, meaning the fixed-point number Q is of type int8, then [Q...] min Q max Specifically, it is [-128, 127] (i.e., [-2n-1, 2n-1-1], where n = 8). The floating-point number X can be a statistical value obtained by statistically analyzing historical training data. For example, during historical training, the maximum statistical value of X is 10, and the minimum statistical value is -20, then [X... min X max ] can be defined as [-20, 10]. In other embodiments, [X] min X max It can also be a preset experience value.

[0058] For symmetric quantization, formula (2) shows a formula for obtaining the quantization coefficient S for int type quantization, where the range of the floating-point number X is [X... min X max The range of the fixed-point number Q is [Q min Q max ]:

[0059]

[0060] It should be noted that in other embodiments of this application, in addition to calculating the quantization coefficient S by the calculation method shown in formula (2), the quantization coefficient S can also be calculated by other modified formulas, and no restrictions are imposed here.

[0061] (2) Normalization operator

[0062] Neural network models typically involve stacking multiple layers of data. This stacking process alters the data range. This variation can lead to slower gradient descent during training, resulting in slower convergence, decreased accuracy, or inconsistent data ranges.

[0063] To avoid the aforementioned issues, it is typically necessary to normalize the data in a neural network model. When data normalization is required, a normalization layer is usually added to the neural network model, and normalization operators are deployed within this layer to normalize the data that needs to be normalized.

[0064] In this embodiment of the application, the normalization operator can be the InstanceNorm(IN) operator, LayerNorm(LN) operator, GroupNorm(GN) operator, SwitchableNorm(SN) operator, etc.

[0065] The data structure input to the normalization operator can be [Z, C, H, W], where Z is the number of samples (e.g., an image), C is the number of channels, H is the height of the sample, and W is the width of the sample. The InstanceNorm operator normalizes the HW channels of the data. The LayerNorm operator normalizes the CHW channels. The GroupNorm operator groups the C channels and then performs normalization. SwitchableNorm combines BN, LN, and IN normalization methods to normalize the data. Then, the InstanceNorm, LayerNorm, GroupNorm, and SwitchableNorm operators can scale and shift their respective normalization results according to scaling and shifting factors, outputting the scaled and shifted data.

[0066] In some embodiments of this application, the input data (inputs) represents the relevant feature data such as text, audio, and images to be processed by the normalization operator, the normalization result (normalize) represents the normalized data of the relevant features corresponding to text, video, audio, and images after processing by the normalization operator, and the output data (outputs) is the relevant feature result data corresponding to text, video, audio, and images after processing by the normalization operator in the neural network model. The feature result data is the data obtained after scaling and shifting the feature normalized data.

[0067] It is understandable that the technical solution of this application can be applied to any scenario that requires processing data such as text, audio, and images through a neural network model.

[0068] In related technologies, the InstanceNorm operator normalizes the HW channels of the input data; LayerNorm normalizes the CHW channels; GroupNorm groups channels C and then normalizes them; SwitchableNorm combines the normalization methods BN, LN, and IN. The normalized results obtained above are then scaled and shifted according to the scaling parameter γ and the translation parameter β. Since the normalization operator calculates the mean and variance at each forward inference step, the accuracy of the mean and variance calculation must also be considered during the quantization process.

[0069] All related technical solutions require calculating the mean and variance, which introduces errors.

[0070] Edge computing devices typically only support fixed-point operations. When a neural network model is adapted to such a device, it will first be quantized (that is, the floating-point parameters in the model will be mapped and transformed into integer parameters, and the quantization coefficients of each layer activation response will be determined), and then the quantization coefficients will be calculated offline.

[0071] To better understand this solution, we will first introduce an application scenario of the technical solution of this application.

[0072] The quantization method, apparatus, electronic device, computer-readable storage medium, and computer program product for the normalization operator in the neural network model provided in this application are intended to solve the above-mentioned technical problems in the prior art.

[0073] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0074] Figure 1 The illustration shows a scenario where a terminal 100 uses a face recognition model to recognize a captured face image. For example... Figure 1 As shown, terminal 100 is equipped with a face recognition model, which is trained and obtained by server 200 in the floating-point domain. Figure 1 The face recognition model is used as an example of a neural network model. In other embodiments, the neural network model can also be other models, such as a speech recognition model or a text recognition model.

[0075] It should be noted that terminal 100 includes, but is not limited to, one of the following electronic devices: mobile phone, tablet computer, smart screen, wearable device (e.g., watch, bracelet, helmet, headset, etc.), in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. Server 200 can be a single server or a server cluster consisting of multiple servers. Figure 1 The mobile phone is used as an example of terminal 100. Since face recognition models are resource-intensive algorithms, terminal 100 processes face image data using a built-in floating-point face recognition model, resulting in a large computational load and slow processing speed. Therefore, it is generally necessary to quantize the face recognition model to reduce computational load and improve processing speed.

[0076] In this embodiment of the application, the face image data may be obtained by the terminal 100 by taking a picture of the user, or it may be pre-stored, or it may be transmitted to the terminal 100 by other devices, and there are no restrictions on this.

[0077] In some embodiments, the server 200 quantizes the face recognition model trained in the floating-point domain to obtain a quantized face recognition model, and then deploys it to the terminal 100.

[0078] In other implementations, the face recognition model to be trained is deployed on terminal 100, which trains the model in the floating-point domain. The trained model is then quantized to obtain a quantized face recognition model. Understandably, when terminal 100 uses the quantized face recognition model, high-precision floating-point calculations can be converted to fixed-point calculations, thereby reducing the computational load and increasing computational speed.

[0079] In the embodiments of this application, Figure 1 The face recognition model in the model may include a normalization layer and other layers (e.g., input layer, activation layer, output layer, etc.). The normalization layer includes normalization operators, such as the InstanceNorm operator, LayerNorm operator, GroupNorm operator, and SwitchableNorm operator. The normalization operators in the normalization layer are used to normalize the feature data associated with the input face image data (i.e., the input data of the normalization layer). The input data of the normalization layer is the feature data output by the layer preceding the normalization layer (e.g., the activation layer).

[0080] This application employs symmetric quantization, requiring offline calculation of the maximum and minimum values ​​of parameters and activation values ​​for each layer to compute the quantization parameter S. The general formula for calculating the normalized norm is as follows:

[0081]

[0082]

[0083]

[0084] outputs=γ*norm+β (6)

[0085] The input data consists of feature vectors with multiple dimensions, where the feature vector of dimension i is represented as x. i The feature vector has a length of N, δ represents the variance of the floating-point input data, epsilon represents the smallest floating-point number greater than 0 (1e-8), μ represents the mean of the floating-point input data, norm represents the normalization result, γ and β represent the scaling and translation parameters respectively, outputs are the floating-point output data, representing the layer normalization layerNorm result, ignoring epsilon. Combining formulas 3, 4, and 5 yields formula 7:

[0086]

[0087] The following describes the steps of the quantization stage and fixed-point forward computation:

[0088] Assuming the quantization bit width is b, then Q in Formula 2 max =2 b-1 -1,Q min =-2 b-1 ,

[0089] Offline quantization steps:

[0090] Step 1: Calculate the maximum and minimum values ​​of the floating-point output data outputs in the normalization operator, and the maximum and minimum values ​​of the floating-point scaling parameter γ.

[0091] Step 2: According to Formula 7, the maximum and minimum values ​​of the normalized result norm are respectively Substituting into Formula 2, we can find the quantization coefficient of the normalization result. Since the quantization bit width b and the eigenvector length N are known, the quantization coefficient norm of the normalization result in the embodiments of this application is... scale It can be obtained easily.

[0092] Furthermore, in this embodiment, the nearest power of 2 to N is calculated as exp = ceil(log₂N), thus formula 7 can be transformed into:

[0093]

[0094] After this change, the division by N operation in Formula 5 is transformed into a right shift operation. This shift operation replaces the division operation, which greatly improves performance. Considering that the second shift number exp needs to be an even number, we need exp = exp + (exp & 1).

[0095] It should be noted that in fixed-point calculations, formula 8 needs to be multiplied by norm. scale This further simplifies to Formula 9, thus reducing the calculation of multiplying by the quantization coefficient:

[0096]

[0097] It should be noted that in Formula 9 The fixed-point operation result is in the format Q31.

[0098] The second shift value exp obtained in Step 2 needs to be stored for use during online forward operations.

[0099] Step 3: Substitute the statistical value of the floating-point scaling parameter γ into formula (2) to obtain the quantization coefficient γ of the scaling parameter. scale Furthermore, the floating-point scaling parameter γ is updated to the fixed-point scaling parameter Q. γ Q γ =γ*γ scale .

[0100] In order to perform addition and subtraction on the same metric, the quantization coefficient β of the translation parameter... scale =norm scale *γ scale Therefore, the fixed-point translation parameter Q β =β*β scale .

[0101] Step 4: Based on formula (6), we can obtain formulas (10), (11), and (12):

[0102]

[0103]

[0104]

[0105] In the above formula, the first shift number N outputs The quantization coefficients of the output data do not exceed the quantization bit width b.scale The maximum and minimum values ​​of the floating-point output data can be substituted into Formula 2 to obtain the values. Similarly, the quantization coefficients of the normalized result are also obtained. scale The maximum and minimum values ​​of the floating-point normalization result can be substituted into Formula 2 to obtain the result. Therefore, based on the outputs... scale γ scale norm scale The result of the equation (12) can be obtained, and then the first shift number N is used to obtain the coordinates. outputs By performing a shift operation, the quantization coefficient M can be obtained. outputs .

[0106] The fixed-point calculation process in this embodiment is as follows:

[0107] Step 1: Input data Q at a fixed point inputs Take the mean to obtain the fixed-point mean Q. μ Based on fixed-point input data Q inputs and fixed-point mean Q μ Implement formula (13) to obtain the result of Q31:

[0108]

[0109] It should be noted that in formula (13), >>exp means shifting by the second shift number exp.

[0110] Furthermore, regarding Q alpha Shifting by 16 bits yields Q15, expressed by the formula: Q alpha =Q alpha >>16.

[0111] Step 2: Transform Formula 9:

[0112] Q norm_tmp =(Q input -Q μ )*Q alpha (14)

[0113] Among them, Q norm_tmp This represents the initial fixed-point normalization result, which has a bit width Q greater than the quantization bit width b. alpha It occupies a bit width of 15 bits.

[0114] To avoid loss of precision, this 15-bit shift is incorporated into Equation 9. It is important to note that the division here... This is a right shift operation, so the 15 bits that need to be shifted to the right in formula (14) are merged, thus transforming it into... So in the end, all you need to do is put Q. norm_tmp Shifting right by (16 + exp / 2 - b) bits yields the fixed-point normalized result Q. norm .

[0115] Step 3: Calculate the final result according to formula (15), that is, the fixed-point output data Q. outputs :

[0116] Q outputs =(Q γ *Q norm +Q β )*M outputs / (2^N outputs (15)

[0117] This application provides a quantization method for normalization operators in a neural network model, such as... Figure 2 As shown, the method includes:

[0118] S101. Determine the fixed-point input data Q of the normalization operator. inputs Quantization bit width b, floating-point scaling parameter γ, fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Floating-point output data outputs, first shift number N outputs And the second shift number exp;

[0119] S102, based on the floating-point output data outputs, quantization bit width b, floating-point scaling parameter γ, and first shift number N... outputs Obtain the first quantization coefficient M outputs ;

[0120] S103, Based on the fixed-point input data Q inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ;

[0121] S104, Based on the fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Fixed-point normalization result Q norm Obtain initial fixed-point output data; based on the first quantization coefficient M outputs The initial fixed-point output data is quantized, and the quantization result is shifted by the first shift number N. outputs Perform a shift operation to obtain the fixed-point output data Q. outputs ;

[0122] Wherein, the first shift number N outputs The input data, not exceeding the quantization bit width b, is a feature vector representing preset information, and the second shift bit is even and 2.exp The length N of the feature vector that is closest to it is a power of 2.

[0123] The method in this application embodiment does not require calculating the mean and variance completely, thus avoiding the errors caused by traditional methods of calculating the mean and variance. Furthermore, for partial division operations, a shifting method based on the first shift number and the second shift number is adopted, which greatly reduces the amount of computation and improves the computational efficiency.

[0124] Based on the above embodiments, as an optional embodiment, the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N are used. outputs Obtain the first quantization coefficient M outputs ,include:

[0125] S201. Determine the second quantization coefficients outputs based on the first difference between the maximum and minimum values ​​of the floating-point output data outputs and the quantization bit width b. scale .

[0126] Specifically, in this embodiment of the application, the maximum and minimum values ​​of the floating-point output data outputs and the quantization bit width b are substituted into formula (2) to obtain the quantization coefficients of the floating-point output data, that is, the second quantization coefficients outputs. scale .

[0127] S202. Determine the third quantization coefficient γ based on the second difference between the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b. scale .

[0128] Specifically, in this embodiment, the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b are substituted into formula (2) to obtain the quantization coefficient of the scaling parameter, which is the third quantization coefficient γ. scale .

[0129] S203. Determine the fourth quantization coefficient norm based on the length N and the quantization bit width b. scale .

[0130] According to the formula The quantization coefficients of the normalized result can be obtained, which are also known as the fourth quantization coefficients.

[0131] S204, Outputs based on the second quantization coefficient scale Third quantization coefficient γ scale and the fourth quantization coefficient norm scale Determine the fifth quantization coefficient.

[0132] Specifically, the formula for the fifth quantization coefficient can be expressed as: That is, the left side of equation (12).

[0133] S205, the fifth quantization coefficient is shifted by the first shift number N. outputs Perform a shift operation to obtain the first quantization coefficient M. outputs .

[0134] Based on the above embodiments, as an optional embodiment, according to the fixed-point input data Q inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ,include:

[0135] S301, Input data Q at a fixed point inputs Take the mean to obtain the fixed-point mean Q. μ .

[0136] S302, Determine the fixed-point input data Q inputs and fixed-point mean Q μ The degree of difference between them is shifted by the second shift number exp to determine the reciprocal of the square root of the fixed-point variance Q. alpha .

[0137] Specifically, the embodiments of this application can be based on Determine the fixed-point input data Q inputs and fixed-point mean Q μ The degree of difference between the variances is determined, and then the degree of difference is shifted based on the second shift value exp. The square root of the shifted result is then taken back to obtain the reciprocal of the square root of the fixed-point variance Q. alpha .

[0138] S303, Based on the fixed-point input data Q inputs Fixed-point mean Q μ and the reciprocal of the square root of the fixed-point variance Q alpha Determine the initial fixed-point normalization result Q norm_tmp ;

[0139] The embodiments of this application can obtain the initialization normalization result Q according to formula (14). norm_tmp .

[0140] S304, according to Q alpha The number of bits and the second shifted bit exp are used to normalize the initial fixed-point result Q. norm_tmp Shift to the number of bits of quantization width to determine the fixed-point normalization result Q. norm .

[0141] It is important to note that Q norm_tmp This represents the initial fixed-point normalization result, which has a bit width Q greater than the quantization bit width b. alphaIt occupies a bit width of 15 bits.

[0142] To avoid loss of precision, this 15-bit shift is incorporated into Equation 9. It is important to note that the division here... This is a right shift operation, so the 15 bits that need to be shifted to the right in formula (14) are merged, thus transforming it into... So in the end, all you need to do is put Q. norm_tmp Shifting right by (16 + exp / 2 - b) bits yields the fixed-point normalized result Q. norm .

[0143] Based on the above embodiments, as an optional embodiment, the fixed-point scaling parameter Q γ ,include:

[0144] S401. Determine the third quantization coefficient γ based on the third difference between the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b. scale ;

[0145] S402, Based on the floating-point scaling parameter γ and the third quantization coefficient γ scale Determine the fixed-point scaling parameter Q γ .

[0146] Specifically, in this embodiment of the application, the quantization coefficient γ of the scaling parameter can be obtained by substituting the statistical value of the floating-point scaling parameter γ into formula (2). scale Furthermore, the floating-point scaling parameter γ is updated to the fixed-point scaling parameter Q. γ Q γ =γ*γ scale .

[0147] Based on the above embodiments, as an optional embodiment, the fixed-point translation parameter Q is determined. β ,include:

[0148] S501. Determine the quantization coefficient norm of the normalization result based on the length N and the quantization bit width b. scale ;

[0149] Specifically, the embodiments of this application are based on the formula The quantization coefficients obtained from the normalization result, also known as the fourth quantization coefficient, are the norm. scale .

[0150] S502, Based on the quantization coefficient norm scale and quantization coefficient γ scale Determine the quantization coefficient β of the translation parameter. scale ;

[0151] Specifically, according to formula β scale =norm scale *γ scale Obtain the quantization coefficients of the translation parameters, i.e., the sixth quantization coefficient β. scale ;

[0152] S503. Determine the floating-point shift parameter β, based on the floating-point shift parameter β and the quantization coefficient β. scale Determine the fixed-point translation parameter Q β .

[0153] Specifically, according to formula Q β =β*β scale Determine the fixed-point translation parameter Q β .

[0154] Based on the above embodiments, as an optional embodiment, the step of inputting fixed-point data Q... inputs Fixed-point mean Q μ and the reciprocal of the square root of the fixed-point variance Q alpha Determine the initial fixed-point normalization result Q norm_tmp ,include:

[0155] Determine the fixed-point input data Q inputs With fixed-point mean Q μ The fourth difference between them;

[0156] The fourth difference and the reciprocal of the square root of the fixed-point variance Q are... alpha The product of these two terms is used as the initial fixed-point normalization result Q. norm_tmp .

[0157] This application provides a quantization device for a normalization operator in a neural network model, such as... Figure 3 As shown, the quantization device for the normalization operator in this neural network model may include: a preparation module 301, a quantization coefficient determination module 302, a fixed-point normalization module 303, and an output module 304, wherein,

[0158] Preparation module 301 is used to determine the fixed-point input data Q of the normalization operator. inputs Quantization bit width b, floating-point scaling parameter γ, fixed-point scaling parameter Q γ , Fixed-point translation parameter Q β Floating-point output data outputs, first shift number N outputs And the second shift number exp;

[0159] Quantization coefficient determination module 302 is used to determine the quantization coefficients based on the floating-point output data outputs, quantization bit width b, floating-point scaling parameter γ, and first shift number N. outputs Obtain the first quantization coefficient M outputs ;

[0160] Fixed-point normalization module 303 is used to normalize data based on fixed-point input data Q. inputs The second shift value exp and the quantization bit width b are used to obtain the fixed-point normalized result Q. norm ;

[0161] Output module 304 is used to calculate the fixed-point scaling parameter Q. γ , Fixed-point translation parameter Q β Fixed-point normalization result Q norm Obtain initial fixed-point output data; based on the first quantization coefficient M outputs The initial fixed-point output data is quantized, and the quantization result is shifted by the first shift number N. outputs Perform a shift operation to obtain the fixed-point output data Q. outputs ;

[0162] Wherein, the first shift number N outputs The input data, not exceeding the quantization bit width b, is a feature vector representing preset information, and the second shift bit is even and 2. exp The length N of the feature vector that is closest to it is a power of 2.

[0163] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0164] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the quantization method for the normalization operator in a neural network model. Compared with related technologies, this application can achieve the following: it does not require calculating all the mean and variance, thus avoiding the errors caused by traditional methods of calculating the mean and variance. Furthermore, for partial division operations, a shifting method based on the first shift number and the second shift number is adopted, which greatly reduces the amount of computation and improves the computational efficiency.

[0165] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0166] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0167] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0168] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0169] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0170] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0172] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0173] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0174] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A quantization method for normalization operators in a neural network model, characterized in that, include: determining a fixed-point input data Q for the normalization operator inputs , a quantization bit-width b, a floating-point scaling parameter γ, a fixed-point scaling parameter Q γ , a fixed-point translation parameter Q β , a floating-point output data outputs, a first shift number N outputs and a second shift number exp; According to the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N outputs , a first quantized coefficient M outputs is obtained. According to the fixed-point input data Q inputs , the second shift number exp and the quantization bit width b, a fixed-point normalization result Q norm is obtained. According to the fixed-point scaling parameter Q γ , a fixed-point translation parameter Q β , a fixed-point normalization result Q norm , obtain initial fixed-point output data; according to the first quantization coefficient M outputs , quantize the initial fixed-point output data, and shift the quantization result by the first shift number N outputs to obtain fixed-point output data Q outputs ; The first shift number N outputs The input data is a feature vector for representing preset information, the second shift number is an even number and 2 exp The preset information is any one of an image, a text, an audio, and environmental information. The first quantization coefficient M is obtained according to the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N outputs , obtains the first quantization coefficient M outputs , comprising: A second quantization coefficient outputs is determined according to a first difference between a maximum value and a minimum value of the floating-point output data outputs and a quantization bit width b scale ; A third quantization coefficient γ is determined according to a second difference between a maximum value and a minimum value of the floating-point scaling parameter γ and the quantization bit width b scale ; According to the length N and the quantization bit width b, a fourth quantization coefficient norm is determined scale ; determining a fifth quantization coefficient based on a second quantization coefficient outputs scale , a third quantization coefficient γ scale , and a fourth quantization coefficient norm scale . said fifth quantization coefficient is shifted by said first shift number N outputs shifted to obtain said first quantization coefficient M outputs .

2. The method according to claim 1, characterized in that, The fixed-point input data Q inputs , a second shift number exp, and a quantization bit width b, to obtain a fixed-point normalization result Q norm , comprising: Q = Q + Q inputs Q = Q + Q μ Q = Q + Q determining a difference between the fixed-point input data Q inputs and the fixed-point mean Q μ , shifting the difference by the second shift number exp to determine a fixed-point inverse square root of variance Q alpha ; According to the fixed-point input data Q inputs , the fixed-point mean Q μ , and the fixed-point variance square root reciprocal Q alpha , an initial fixed-point normalization result Q norm_tmp is determined. According to Q alpha the number of bits of the initial fixed-point normalization result Q norm_tmp shifted to the number of bits of the quantization bit width, a fixed-point normalization result Q norm is determined.

3. The method according to claim 1, characterized in that, The determining the point scaling parameter Q γ comprises: A third quantization coefficient γ is determined according to a third difference between a maximum value and a minimum value of the floating-point scaling parameter γ and the quantization bit width b scale ; According to the floating-point scaling parameter γ and the third quantization coefficient γ scale , the fixed-point scaling parameter Q γ is determined.

4. The method according to claim 3, characterized in that, The determining the point translation parameter Q β , comprising: According to the length N and the quantization bit width b, a fourth quantization coefficient norm is determined scale ; According to the quantization coefficient norm scale and the quantization coefficient γ scale , a sixth quantization coefficient β scale is determined; Determine the floating-point shift parameter β, based on the floating-point shift parameter β and the quantization coefficient β. scale Determine the fixed-point translation parameter Q β .

5. The method according to claim 2, characterized in that, The initial fixed-point normalization result Q inputs is determined according to the fixed-point input data Q μ , the fixed-point mean Q alpha , and the fixed-point variance square root reciprocal Q norm_tmp , and includes: determining a fourth difference between the fixed-point input data Q inputs and the fixed-point mean Q μ ​ the product of the fourth difference and the fixed-point inverse square root of variance Q alpha as the initial fixed-point normalization result Q norm_tmp .

6. A quantization device for a normalization operator in a neural network model, characterized in that, include: a preparation module for determining a fixed-point input data Q of the normalization operator inputs , a quantization bit-width b, a floating-point scaling parameter γ, a fixed-point scaling parameter Q γ , a fixed-point translation parameter Q β , a floating-point output data outputs, a first shift number N outputs and a second shift number exp; The quantization coefficient determination module is configured to determine first quantization coefficients M according to the floating-point output data outputs, the quantization bit width b, the floating-point scaling parameter γ, and the first shift number N outputs outputs ;​ a fixed-point normalization module configured to obtain a fixed-point normalization result Q inputs from the fixed-point input data Q norm , a second shift number exp, and a quantization bit width b. The output module is used to calculate the fixed-point scaling parameter Q. γ , Fixed-point translation parameter Q β Fixed-point normalization result Q norm Obtain initial fixed-point output data; based on the first quantization coefficient M outputs The initial fixed-point output data is quantized, and the quantization result is shifted by the first shift number N. outputs Perform a shift operation to obtain the fixed-point output data Q. outputs ; Wherein, the first shift number N outputs The input data is a feature vector representing preset information, and the second shift bit is even and 2. The input data does not exceed the quantization bit width b. exp The value is the power of 2 of the length N closest to the feature vector, and the preset information is any one of image, text, audio, and environmental information; The parameters are: floating-point output data outputs, quantization bit width b, floating-point scaling parameter γ, and first shift number N. outputs Obtain the first quantization coefficient M outputs ,include: The second quantization coefficients outputs are determined based on the first difference between the maximum and minimum values ​​of the floating-point output data outputs and the quantization bit width b. scale ; The third quantization coefficient γ is determined based on the second difference between the maximum and minimum values ​​of the floating-point scaling parameter γ and the quantization bit width b. scale ; The fourth quantization coefficient norm is determined based on the length N and the quantization bit width b. scale ; According to the second quantization coefficient, outputs scale Third quantization coefficient γ scale and the fourth quantization coefficient norm scale Determine the fifth quantization coefficient; The fifth quantization coefficient is shifted by the first shift number N. outputs Perform a shift operation to obtain the first quantization coefficient M. outputs .

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images