Model quantification method, equipment, storage medium and device

Abstract

The invention relates to the technical field of computers, and discloses a model quantization method and device, and a storage medium, and the method comprises the steps: decoupling a quantization step size of a multi-layer weight in a to-be-quantized neural network model into a weight inverse quantization step size, and enabling the weight inverse quantization step size to be used for converting an integer weight back to a floating point weight, and performing pseudo quantization on the to-be-quantized neural network model, reasoning a quantization error before and after pseudo quantization of the to-be-quantized neural network model based on the calibration data set, and updating the weight inverse quantization step size according to the quantization error to obtain a quantized neural network model. Deploying the quantized neural network model to a target deployment platform according to the quantization parameters of the quantized neural network model; according to the method, the weighted quantization step size is decoupled into the weighted inverse quantization step size, and the weighted inverse quantization step size can participate in optimization updating, so that the quantization error loss can be reduced, the quantization performance can be improved, and the model quantization precision can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model quantization method, device, storage medium and apparatus. Background Art

[0002] In recent years, deep learning has made great breakthroughs in image classification, target detection, and face recognition, significantly improving the performance of image processing and speech recognition, and has been widely used in video surveillance, smart home, and the Internet of Things. Current neural network models generally require the support of high-performance computer equipment. In order to apply neural network models to low-performance devices, especially in terminal products, it is necessary to reduce the computational complexity of neural network models. Model quantization is a commonly used model compression technology. Reducing the calculation process from floating-point numbers to lower-bit fixed-point operations will greatly accelerate the forward reasoning process.

[0003] However, in the existing model quantization method, the quantization step parameters of the weights remain fixed after initialization and do not participate in optimization and updating, resulting in large quantization errors and low model quantization accuracy. Summary of the invention

[0004] The main purpose of this application is to provide a model quantization method, equipment, storage medium and device, aiming to solve the technical problems of the existing model quantization method, in which the quantization step parameters of the weights remain fixed after initialization and do not participate in optimization and updating, resulting in large quantization errors and low model quantization accuracy.

[0005] To achieve the above object, the present application provides a model quantization method, which includes the following steps:

[0006] Decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into a weight inverse quantization step, wherein the weight inverse quantization step is used to convert integer weights back to floating-point weights;

[0007] Performing pseudo-quantization on the neural network model to be quantized;

[0008] Inferring the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set;

[0009] The weight dequantization step size is updated according to the quantization error to obtain a quantized neural network model;

[0010] The quantized neural network model is deployed to a target deployment platform according to the quantized parameters of the quantized neural network model.

[0011] Optionally, updating the weight dequantization step size according to the quantization error to obtain a quantized neural network model includes:

[0012] The weight inverse quantization step size and weight adaptive rounding parameter are updated according to the quantization error to obtain a quantized neural network model, and the weight adaptive rounding parameter is used to adjust the rounding method of the weight.

[0013] Optionally, updating the weight inverse quantization step size and the weight adaptive rounding parameter according to the quantization error to obtain a quantized neural network model includes:

[0014] Inferring the output error of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set and the quantization error;

[0015] The weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step are updated according to the output error to obtain a quantized neural network model, wherein the activation value quantization step is used to convert the floating-point activation value into an integer activation value.

[0016] Optionally, updating the weight inverse quantization step, the weight adaptive rounding parameter, and the activation value quantization step according to the output error to obtain a quantized neural network model includes:

[0017] The weight inverse quantization step size, the weight adaptive rounding parameter and the activation value quantization step size are updated according to the output error to obtain a pseudo-quantized neural network model;

[0018] The integer weights, weight inverse quantization step size and activation value quantization step size of the true quantized neural network model are determined according to the quantization parameters of the pseudo-quantized neural network model to obtain a quantized neural network model.

[0019] Optionally, decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into weight inverse quantization step includes:

[0020] Decoupling the quantization step of the multi-layer weights in the neural network model to be quantized into a weight quantization step and a weight dequantization step, wherein the weight quantization step is used to convert the floating-point weights into integer weights, and the weight dequantization step is used to convert the integer weights back into floating-point weights;

[0021] The weight quantization step size is set to be non-updatable, and the weight inverse quantization step size is set to be updatable.

[0022] Optionally, the model quantization method further includes:

[0023] Fusing the network layers of the initial neural network model according to the deployment configuration information of the initial neural network model to obtain a fused neural network model;

[0024] Initializing the quantization step lengths of the multi-layer weights in the fused neural network model;

[0025] The activation value quantization step length of the multi-layer output activation values ​​in the fused neural network model is initialized to obtain the neural network model to be quantized.

[0026] Optionally, the inferring the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set includes:

[0027] Perform forward reasoning in the neural network model to be quantized based on the calibration data set to obtain floating-point activation values ​​of multiple layers of output in the neural network model to be quantized;

[0028] Performing forward reasoning in the pseudo-quantized neural network model based on the calibration data set to obtain quantized activation values ​​of multiple layers of output in the pseudo-quantized neural network model;

[0029] The quantization errors of the neural network model to be quantized before and after pseudo-quantization are inferred based on the floating-point activation value and the quantized activation value.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a model quantization device, which includes a memory, a processor, and a model quantization program stored in the memory and executable on the processor, and the model quantization program is configured to implement the model quantization method as described above.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a model quantization program is stored. When the model quantization program is executed by a processor, the model quantization method as described above is implemented.

[0032] In addition, to achieve the above purpose, the present application also proposes a model quantization device, the model quantization device comprising:

[0033] A decoupling module, used for decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into a weight inverse quantization step, wherein the weight inverse quantization step is used to convert the integer weights back to floating-point weights;

[0034] A pseudo-quantization module, used for pseudo-quantizing the neural network model to be quantized;

[0035] An inference module, used for inferring the quantization error of the neural network model to be quantized before and after pseudo-quantization based on a calibration data set;

[0036] A quantization module, used for updating the weight inverse quantization step size according to the quantization error to obtain a quantized neural network model;

[0037] A deployment module is used to deploy the quantized neural network model to a target deployment platform according to the quantized parameters of the quantized neural network model.

[0038] In the present application, it is disclosed that the quantization step of multiple layers of weights in a neural network model to be quantized is decoupled into a weight inverse quantization step, the weight inverse quantization step is used to convert integer weights back into floating-point weights, the neural network model to be quantized is pseudo-quantized, the quantization error of the neural network model to be quantized before and after pseudo-quantization is inferred based on a calibration data set, the weight inverse quantization step is updated according to the quantization error, a quantized neural network model is obtained, and the quantized neural network model is deployed to a target deployment platform according to the quantization parameters of the quantized neural network model; since the present application decouples the quantization step of the weight into a weight inverse quantization step, the weight inverse quantization step can participate in optimization and updating, thereby reducing the quantization error loss, thereby improving the quantization performance and the model quantization accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural diagram of a model quantization device of a hardware operating environment involved in an embodiment of the present application;

[0040] Figure 2 This is a flow chart of the first embodiment of the model quantization method of the present application;

[0041] Figure 3 This is a flow chart of the second embodiment of the model quantization method of the present application;

[0042] Figure 4 This is a flow chart of the third embodiment of the model quantization method of the present application;

[0043] Figure 5 This is a structural block diagram of the first embodiment of the model quantization device of the present application.

[0044] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a model quantization device for a hardware operating environment involved in an embodiment of the present application.

[0047] like Figure 1As shown, the model quantization device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in this application. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable memory (Non-volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0048] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the model quantization device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0049] like Figure 1 As shown, the memory 1005 identified as a computer storage medium may include an operating system, a network communication module, a user interface module, and a model quantization program.

[0050] exist Figure 1 In the model quantization device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the user device; the model quantization device calls the model quantization program stored in the memory 1005 through the processor 1001, and executes the model quantization method provided in the embodiment of the present application.

[0051] Based on the above hardware structure, an embodiment of the model quantization method of the present application is proposed.

[0052] Reference Figure 2 , Figure 2 This is a flow chart of the first embodiment of the model quantization method of the present application, and the first embodiment of the model quantization method of the present application is proposed.

[0053] It should be understood that in recent years, deep learning has made great breakthroughs in image classification, target detection, and face recognition, significantly improving the performance of image processing and speech recognition, and has been widely used in video surveillance, smart home, and the Internet of Things. Current neural network models generally require the support of high-performance computer equipment. In order to apply neural network models to low-performance devices, especially in terminal products, it is necessary to reduce the computational complexity of neural network models. Model quantization is a model compression technology that reduces the calculation process from floating-point numbers to lower-bit fixed-point operations, which greatly accelerates the forward reasoning process.

[0054] In order to solve the problem of excessive decrease in accuracy of low-bit quantization models, model quantization usually has the following methods: 1. Introducing quantization of activation values ​​during weight fine-tuning can effectively improve the performance of the quantized model and make the quantized model flatter in some directions; 2. Comparing the errors of the initial data set and the quantized data set to obtain the quantization error set; then error analysis can be performed based on the quantization error set, and the initial model can be adjusted, thereby effectively improving the accuracy of the quantization model; 3. When the total accuracy loss value is greater than the preset accuracy loss threshold, some group parameters in the N group parameters are quantized M times to make the total accuracy loss value less than or equal to the preset accuracy loss threshold, thereby solving the problem that adaptive correction cannot be achieved when the total quantization error of the entire network model does not meet the requirements. When achieving fast quantization, it can also reduce the quantization error and accuracy loss as much as possible; 4. By comprehensively combining the actual values, expected values, and constraint values ​​of multiple optimization target parameters to obtain the loss function in the optimization process, it is possible to fully consider the conflict relationship between the optimization targets and optimize them uniformly.

[0055] In the above model quantization method, the quantization step size parameter of the weight remains fixed after initialization and does not participate in optimization updates. However, in actual hardware inference, unlike the quantization of activation values, the integer weights in weight quantization can be obtained before deployment. In this way, in pseudo-quantization simulation, the quantization step size for converting floating-point weights to integer weights and the dequantization step size for converting integer weights to floating-point weights can be different values.

[0056] Therefore, in order to overcome the above defects, in this embodiment, the quantization step of the weight is decoupled into the weight inverse quantization step, and the weight inverse quantization step can participate in the optimization update, thereby reducing the quantization error loss, and further improving the quantization performance and the model quantization accuracy.

[0057] In a first embodiment, the model quantization method comprises the following steps:

[0058] Step S10: decoupling the quantization step of the multi-layer weights in the neural network model to be quantized into a weight dequantization step, wherein the weight dequantization step is used to convert the integer weights back to floating-point weights.

[0059] It should be understood that the executor of the method of this embodiment can be a model quantization device with data processing, network communication and program running functions, for example, a terminal device such as a computer, or other electronic devices that can achieve the same or similar functions, and this embodiment does not impose any restrictions on this.

[0060] It should be noted that the neural network model to be quantized can be a neural network model that needs to be quantized, and the multi-layer weights in the neural network model to be quantized can be the weights of each layer in the neural network model to be quantized, or can be part of the weights in the neural network model to be quantized. This embodiment does not limit this.

[0061] For ease of understanding, the following examples are given, but are not intended to limit this solution. In one example, decoupling the quantization step of multiple layers of weights in a neural network model to be quantized into weight inverse quantization step mainly involves the following contents:

[0062] Neural network model: uppercase letters represent matrices, lowercase letters represent vectors, and specifically correspond to weights. W and w represent the matrix and vector of weights respectively. x and y represent the input and output feature maps of the neural network model. A classic forward neural network layer can be expressed as:

[0063]

[0064] Among them, W i,j Indicates the connection of the i th input neuron and the jth th The weight parameters of the output neurons, x i represents the input of the i-th input neuron, b represents the bias parameter of the neural network layer, f(·) represents the activation function, for example, the ReLU activation function, l represents the layer number of the neural network layer, and y j represents the output of the jth output neuron.

[0065] Existing weight quantization and activation value quantization methods: Neural network model quantization includes weight quantization and activation value quantization. The existing weight quantization and activation value quantization methods can be expressed as:

[0066]

[0067]

[0068] in, represents the quantized weight, represents the quantized activation value, w represents the weight vector, x represents the input of the neural network model, and s w and x They represent the quantization step size of the weight and the quantization step size of the activation value, clip(·) represents the truncation function, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, x l and x u represents the upper and lower truncation boundaries of the activation value quantization, and [·] represents the nearest neighbor rounding operation.

[0069] Weight quantization and activation value quantization method of this embodiment: The weight quantization and activation value quantization of this embodiment adopt the following method:

[0070]

[0071]

[0072] in, represents the quantized weight, represents the quantized activation value, w represents the weight vector, x represents the input of the neural network model, and s w and x They represent the quantization step size of the weight and the quantization step size of the activation value, clip(·) represents the truncation function, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, x l and x u represents the upper and lower truncation boundaries of the activation value quantization, [·] represents the nearest neighbor rounding operation, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, which is a learnable adaptive rounding parameter of the weight.

[0073] Different from the existing quantization method, the weight quantization in this embodiment adopts an adaptive rounding method.

[0074] Pseudo-quantization and true quantization: In order to enable the post-quantization method to be executed on the hardware GPU or CPU, the quantization operation needs to be simulated using the FP32 floating point format. This operation is defined as pseudo-quantization. When the actual hardware performs quantization inference, the simulated calculation of the FP32 floating point will be converted into a fixed-point integer calculation. This operation is defined as true quantization. The two operations can be expressed as:

[0075]

[0076] Among them, y represents the output of the neural network model, represents the quantized weight, represents the quantized activation value, x represents the input of the neural network model, and sw and x Respectively represent the quantization step size of the weight and the quantization step size of the activation value, W int Integer data representing weights, which are parameters that can be obtained before actual hardware quantization deployment:

[0077]

[0078] Among them, W int Integer data representing weights, clip(·) represents the cutoff function, w represents the vector of weights, s w represents the quantization step size of the weight, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, and w l and w u Represents the upper and lower truncation boundaries of weight quantization.

[0079] Furthermore, the decoupling of the quantization step of multiple layers of weights in the neural network model to be quantized into weight inverse quantization step includes: decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into weight quantization step and weight inverse quantization step, the weight quantization step is used to convert floating-point weights into integer weights, and the weight inverse quantization step is used to convert integer weights back into floating-point weights; setting the weight quantization step to be non-updatable, and setting the weight inverse quantization step to be updatable.

[0080] It should be understood that in true quantization, the integer weights W int and weight quantization step size s w can be obtained before actual hardware deployment, so they can be two independent parameters. Therefore, in pseudo quantization, the quantization step size s of the weight can be w Decoupled into two independent step sizes, one is the weight quantization step size s that converts floating point weights into integer weights w , one is the dequantization step size s′ that converts integer weights back to floating-point weights w , and set the weight quantization step size to non-updatable, and the weight inverse quantization step size to updatable.

[0081] Step S20: performing pseudo-quantization on the neural network model to be quantized.

[0082] It should be noted that pseudo-quantization is an approximate quantization method used to simulate the quantization process, but the quantized weights and activation values ​​are not used in actual reasoning. In the pseudo-quantization process, the model will perform operations similar to quantization, such as mapping floating-point weights and activation values ​​to integers or fixed-point representations within a fixed range, but floating-point precision is still used in actual calculations. Pseudo-quantization can be used to evaluate the impact of quantization on model performance and assist in optimizing parameter selection during the quantization process.

[0083] Therefore, in this embodiment, after pseudo-quantization of the neural network model to be quantized, the quantization errors of the neural network model to be quantized before and after pseudo-quantization can be inferred based on the calibration data set, and the weight inverse quantization step size can be updated according to the quantization error.

[0084] Step S30: inferring the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set.

[0085] It should be noted that the calibration data set can be a pre-constructed data sample set, which is used to infer the quantization error of the neural network model before and after pseudo-quantization when optimizing the model, and optimize and update the model quantization based on the quantization error to reduce the quantization error loss, improve the quantization performance, and improve the model quantization accuracy.

[0086] It can be understood that the quantization error of the neural network model to be quantized before and after pseudo-quantization inferred based on the calibration data set can be performed by performing forward reasoning in the neural network model to be quantized based on the calibration data set to obtain floating-point activation values ​​of multiple layers of output in the neural network model to be quantized, performing forward reasoning in the pseudo-quantized neural network model based on the calibration data set to obtain quantized activation values ​​of multiple layers of output in the pseudo-quantized neural network model, and inferring the quantization error of the neural network model to be quantized before and after pseudo-quantization based on the floating-point activation values ​​and the quantized activation values.

[0087] Step S40: updating the weight inverse quantization step size according to the quantization error to obtain a quantized neural network model.

[0088] Optionally, in this embodiment, by decoupling the quantization step of the weight, the weight quantization process is transformed into the following:

[0089]

[0090] in, represents the quantized weight, clip(·) represents the truncation function, w represents the vector of weights, s w represents the weight quantization step size, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, s′ w Represents the weight inverse quantization step size.

[0091] Correspondingly, the true quantization process is transformed into the following:

[0092]

[0093]

[0094] Among them, y represents the output of the neural network model, s′ w Represents the weight dequantization step size, s x represents the quantization step size of the activation value, W int Integer data representing weights, x represents the input of the neural network model, clip(·) represents the truncation function, s w represents the weight quantization step size, w represents the weight vector, h(α) represents the weight adaptive rounding parameter, w l and w u Represents the upper and lower truncation boundaries of weight quantization.

[0095] Under the condition of weight quantization step decoupling, the weight quantization step s w The parameters are fixed, and the weight is dequantized with a step size s′ w Participating in optimization updates can reduce quantization error losses, thereby improving quantization performance and improving model quantization accuracy.

[0096] It should be understood that updating the weight dequantization step size according to the quantization error to obtain the quantized neural network model can be based on the quantization error through the gradient back propagation algorithm to propagate the quantization error back to the model's weight dequantization step size, updating the weight dequantization step size, and determining the weight dequantization step size through the mean square error optimization algorithm to obtain the quantized neural network model.

[0097] In a specific implementation, for example, the mean square error optimization algorithm can be expressed as:

[0098]

[0099] Among them, s′ w represents the weight inverse quantization step size, represents the quantized weight, w represents the weight vector, Represents the absolute value of the quantization error. The above formula means that the quantization error is reduced by optimizing and updating the weight inverse quantization step size, and the model is quantized according to the weight inverse quantization step size corresponding to the minimum quantization error to obtain the quantized neural network model.

[0100] Step S50: deploying the quantized neural network model to a target deployment platform according to the quantization parameters of the quantized neural network model.

[0101] It should be understood that in order to apply the neural network model to low-performance devices, especially to end-side products, in this embodiment, after obtaining the quantized neural network model, the quantized neural network model can be deployed to the target deployment platform according to the quantization parameters of the quantized neural network model, wherein the target deployment platform can be a low-performance device, for example, a mobile device (mobile phone, tablet computer), an IoT device, an edge computing device, etc., which is not limited in this embodiment, and the quantization parameters include but are not limited to at least one of the weight inverse quantization step, the weight adaptive rounding parameter, and the activation value quantization step. High-performance devices may refer to servers, high-performance computers, etc.

[0102] In this embodiment, it is disclosed that the quantization step of multiple layers of weights in the neural network model to be quantized is decoupled into the weight inverse quantization step, the weight inverse quantization step is used to convert the integer weights back into floating-point weights, the neural network model to be quantized is pseudo-quantized, the quantization error of the neural network model to be quantized before and after pseudo-quantization is inferred based on a calibration data set, the weight inverse quantization step is updated according to the quantization error, the quantized neural network model is obtained, and the quantized neural network model is deployed to the target deployment platform according to the quantization parameters of the quantized neural network model; because the quantization step of the weight is decoupled into the weight inverse quantization step in this embodiment, the weight inverse quantization step can participate in the optimization update, thereby reducing the quantization error loss, thereby improving the quantization performance and the model quantization accuracy.

[0103] Reference Figure 3 , Figure 3 This is a flow chart of the second embodiment of the model quantification method of this application, based on the above Figure 2 The first embodiment shown proposes a second embodiment of the model quantization method of the present application.

[0104] In the second embodiment, the step S40 includes:

[0105] Step S401: updating the weight inverse quantization step size and weight adaptive rounding parameter according to the quantization error to obtain a quantized neural network model, wherein the weight adaptive rounding parameter is used to adjust the weight rounding method.

[0106] It should be understood that in order to further improve the quantization performance and the model quantization accuracy, in this embodiment, multiple parameters can also be jointly optimized and updated.

[0107] It can be understood that updating the weight dequantization step size and weight adaptive rounding parameters according to the quantization error can be achieved by propagating the quantization error back to the model through the gradient back propagation algorithm according to the quantization error, updating the weight dequantization step size and weight adaptive rounding parameters, determining the weight dequantization step size and weight adaptive rounding parameters through the mean square error optimization algorithm, and obtaining the quantized neural network model.

[0108] In a specific implementation, for example, the mean square error optimization algorithm can be expressed as:

[0109]

[0110] Among them, s′ w represents the weight inverse quantization step size, h(α) represents the weight adaptive rounding parameter, represents the quantized weight, w represents the weight vector, Represents the absolute value of the quantization error. The above formula means that the quantization error is reduced by jointly optimizing and updating the weight inverse quantization step size and the weight adaptive rounding parameter, and the model is quantized according to the weight inverse quantization step size and the weight adaptive rounding parameter corresponding to the minimum quantization error to obtain the quantized neural network model.

[0111] This embodiment optimizes and updates multiple parameters together, thereby further improving the quantization performance and the model quantization accuracy.

[0112] Furthermore, in order to improve the effect of optimization and updating, the step S401 includes:

[0113] The output errors of the neural network model to be quantized before and after pseudo-quantization are inferred based on the calibration data set and the quantization error; the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step are updated according to the output error to obtain the quantized neural network model, and the activation value quantization step is used to convert the floating-point activation value into an integer activation value.

[0114] It should be understood that updating the weight inverse quantization step size and weight adaptive rounding parameter according to the quantization error only considers the quantization error, which has certain limitations. Therefore, in this embodiment, in order to improve the effect of optimization and updating, the output error of the neural network model to be quantized before and after pseudo-quantization can be determined according to the quantization error and the output error, and the weight inverse quantization step size, weight adaptive rounding parameter and activation value quantization step size are updated according to the output error to obtain the quantized neural network model.

[0115] It can be understood that the output error of the neural network model to be quantized before and after pseudo-quantization can be inferred based on the calibration data set and the quantization error by taking the calibration data set as the input of the neural network model, multiplying the input with the floating-point activation value and the quantization activation value respectively, and then calculating the difference between the two to obtain the output error.

[0116] It should be understood that updating the weight dequantization step size, weight adaptive rounding parameters and activation value quantization step size according to the output error can be accomplished by propagating the quantization error back to the model through a gradient back propagation algorithm based on the output error, updating the weight dequantization step size, weight adaptive rounding parameters and activation value quantization step size, determining the weight dequantization step size, weight adaptive rounding parameters and activation value quantization step size through a mean square error optimization algorithm, and obtaining a quantized neural network model.

[0117] In a specific implementation, for example, the mean square error optimization algorithm can be expressed as:

[0118]

[0119] Among them, s′ w represents the weight inverse quantization step size, h(α) represents the weight adaptive rounding parameter, represents the quantized weight, w represents the weight vector, s x represents the quantization step size of the activation value, represents the output error, represents the quantized weight, W represents the weight matrix, and x represents the input of the neural network model. Represents the absolute value of the output error. The above formula means that the output error is reduced by jointly optimizing and updating the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step, and the model is quantized according to the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step corresponding to the minimum output error to obtain the quantized neural network model.

[0120] Further, in order to improve the reliability of model quantization, the weight inverse quantization step, the weight adaptive rounding parameter and the activation value quantization step are updated according to the output error to obtain a quantized neural network model, including:

[0121] The weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step are updated according to the output error to obtain a pseudo-quantized neural network model; the integer weights, weight inverse quantization step and activation value quantization step of the true quantized neural network model are determined according to the quantization parameters of the pseudo-quantized neural network model to obtain a quantized neural network model.

[0122] It should be understood that in order to improve the reliability of model quantization, in this embodiment, the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step can be updated according to the output error to obtain a pseudo-quantized neural network model, and then the integer weights, weight inverse quantization step and activation value quantization step of the true quantized neural network model are determined according to the quantization parameters of the pseudo-quantized neural network model to obtain the quantized neural network model, that is, the quantization parameters in the simulated quantization are updated first, and then the quantization parameters in the actual quantization are updated.

[0123] Reference Figure 4 , Figure 4 The flowchart of the third embodiment of the model quantization method of the present application is shown in FIG. Based on the above embodiments, the third embodiment of the model quantization method of the present application is proposed.

[0124] In the third embodiment, before step S10, the method further includes:

[0125] Step S01: Fusing the network layers of the initial neural network model according to the deployment configuration information of the initial neural network model to obtain a fused neural network model.

[0126] It should be understood that in order to reduce the amount of computation and memory usage of the model, in this embodiment, the network layers of the neural network model may be fused before the model is quantized.

[0127] It should be noted that the deployment configuration information may include information such as the target deployment platform and the deployment method.

[0128] It can be understood that the network layers of the initial neural network model are fused according to the deployment configuration information to obtain the neural network model to be quantized. The quantization nodes and quantization modes are determined according to the deployment configuration information, and the network layer fusion method is determined according to the quantization nodes and the quantization mode, and the network layers of the initial neural network model are fused based on the network layer fusion method to obtain the neural network model to be quantized.

[0129] In a specific implementation, for example, the convolutional layer (fully connected layer) and the BatchNormalization layer in the neural network model are fused into one convolutional layer (fully connected layer).

[0130] Step S02: Initializing the quantization step sizes of the multi-layer weights in the fused neural network model.

[0131] It should be understood that in order to reduce the workload of model quantization, in this embodiment, before quantizing the model, the quantization step size of the multi-layer weights in the fused neural network model can also be initialized.

[0132] In the specific implementation, for each layer of weight parameters in the network model, a floating-point weight set and a quantized weight set are obtained according to different weight cutoff values, a quantization error set is calculated, and an initialized weight quantization step is obtained according to the quantization error set.

[0133] Step S03: Initialize the activation value quantization step of the multi-layer output activation values ​​in the fused neural network model to obtain the neural network model to be quantized.

[0134] It should be understood that in order to reduce the workload of model quantization, in this embodiment, before quantizing the model, the activation value quantization step of the multi-layer output activation values ​​in the fused neural network model can also be initialized.

[0135] In the specific implementation, the calibration data set is used to perform forward inference on the model to obtain the output activation value of each layer in the network model, and the floating-point activation value and the quantized activation value set are obtained according to different activation value cutoff values. The quantization error set is calculated, and the initialized activation value quantization step is obtained according to the quantization error set.

[0136] For ease of understanding, the following examples are given, but are not intended to limit this solution. In one example, the model quantization method includes the following steps:

[0137] A. Obtain the set deployment configuration information, and determine the quantization node and quantization mode according to the deployment configuration information.

[0138] B. Obtain the neural network model to be quantized and the calibration dataset.

[0139] C. Model network layer fusion. Determine the network layer fusion method based on the quantization nodes and quantization mode, and fuse the convolution layer (fully connected layer) and Batch Normalization layer in the model into one convolution layer (fully connected layer).

[0140] D. Initialization of weight quantization parameters. For each layer of weight parameters in the network model, a floating-point weight set and a quantized weight set are obtained according to different weight cutoff values, and a quantization error set is calculated; and based on the quantization error set, the initialized weight quantization step size is obtained.

[0141] E. Initialization of activation value quantization parameters. Use the calibration data set to perform model forward inference to obtain the output activation value of each layer in the network model. According to different activation value cutoff values, the floating point activation value and the quantized activation value set are obtained, and the quantization error set is calculated; and according to the quantization error set, the initial activation value quantization step size is obtained.

[0142] F. Decoupling of weight quantization step. The quantization step of each layer of weights in the network model is decoupled into two independent step sizes: one is the weight quantization step size that converts floating-point weights to integer weights, and the other is the weight dequantization step size that converts integer weights back to floating-point weights. The weight quantization step size parameter is set to be non-updatable, while the weight dequantization step size is updatable.

[0143] G. Jointly optimize quantization parameters. Use the calibration data set to perform forward inference on the floating-point model to obtain the floating-point activation values ​​output by each layer in the network model; use the calibration data set to perform forward inference on the quantized model to obtain the quantized activation values ​​output by each layer in the network model; calculate the quantization error set based on the floating-point activation value and the quantized activation value set, perform gradient backpropagation, and simultaneously update the weight inverse quantization step, weight adaptive rounding parameter, and activation value quantization step to obtain the quantized model.

[0144] H. Fixed-point inference deployment. According to the quantization parameters in the pseudo-quantization model, the integer weights, weight inverse quantization step, and activation value quantization step of the true quantization model are obtained. These quantization parameters deploy the model to the target deployment platform.

[0145] Compared with the model quantized by the prior art, the quantized model of this embodiment has higher quantization accuracy under the same quantization bits, which can ensure the accuracy of the model and does not require quantization training. This embodiment decouples the weight quantization step size so that the weight quantization step size can be optimized and updated together with other quantization parameters, which can obtain a smaller quantization error loss and improve the quantization performance. This embodiment is universal and can be applied to quantization on different tasks and different deployment devices.

[0146] In addition, an embodiment of the present application further proposes a storage medium, on which a model quantization program is stored. When the model quantization program is executed by a processor, the model quantization method as described above is implemented.

[0147] In addition, refer to Figure 5 , an embodiment of the present application also proposes a model quantization device.

[0148] It should be understood that in recent years, deep learning has made great breakthroughs in image classification, target detection, and face recognition, significantly improving the performance of image processing and speech recognition, and has been widely used in video surveillance, smart home, and the Internet of Things. Current neural network models generally require the support of high-performance computer equipment. In order to apply neural network models to low-performance devices, especially in terminal products, it is necessary to reduce the computational complexity of neural network models. Model quantization is a model compression technology that reduces the calculation process from floating-point numbers to lower-bit fixed-point operations, which greatly accelerates the forward reasoning process.

[0149] In order to solve the problem of excessive decrease in accuracy of low-bit quantization models, model quantization can be adopted in the following ways: 1. Introducing quantization of activation values ​​during weight fine-tuning can effectively improve the performance of the quantized model and make the quantized model flatter in some directions; 2. Comparing the errors of the initial data set and the quantized data set to obtain the quantization error set; then error analysis can be performed based on the quantization error set, and the initial model can be adjusted, thereby effectively improving the accuracy of the quantization model; 3. When the total accuracy loss value is greater than the preset accuracy loss threshold, some group parameters in the N groups of parameters are quantized M times so that the total accuracy loss value is less than or equal to the preset accuracy loss threshold, thereby solving the problem that adaptive correction cannot be achieved when the total quantization error of the entire network model does not meet the requirements. When achieving fast quantization, it can also reduce the quantization error and accuracy loss as much as possible; 4. By comprehensively combining the actual values, expected values, and constraint values ​​of multiple optimization target parameters to obtain the loss function in the optimization process, it is possible to fully consider the conflict relationship between the optimization targets and optimize them uniformly.

[0150] In the above model quantization method, the quantization step size parameter of the weight remains fixed after initialization and does not participate in optimization updates. However, in actual hardware inference, unlike the quantization of activation values, the integer weights in weight quantization can be obtained before deployment. In this way, in pseudo-quantization simulation, the quantization step size for converting floating-point weights to integer weights and the dequantization step size for converting integer weights to floating-point weights can be different values.

[0151] Therefore, in order to overcome the above defects, in this embodiment, the quantization step of the weight is decoupled into the weight inverse quantization step, and the weight inverse quantization step can participate in the optimization update, thereby reducing the quantization error loss, and further improving the quantization performance and the model quantization accuracy.

[0152] In this embodiment, the model quantization device includes:

[0153] The decoupling module 10 is used to decouple the quantization step of the multi-layer weights in the neural network model to be quantized into a weight dequantization step, and the weight dequantization step is used to convert the integer weights back to floating-point weights.

[0154] It should be noted that the neural network model to be quantized can be a neural network model that needs to be quantized, and the multi-layer weights in the neural network model to be quantized can be the weights of each layer in the neural network model to be quantized, or can be part of the weights in the neural network model to be quantized. This embodiment does not limit this.

[0155] For ease of understanding, the following examples are given, but are not intended to limit this solution. In one example, decoupling the quantization step of multiple layers of weights in a neural network model to be quantized into weight inverse quantization step mainly involves the following contents:

[0156] Neural network model: uppercase letters represent matrices, lowercase letters represent vectors, and specifically correspond to weights. W and w represent the matrix and vector of weights respectively. x and y represent the input and output feature maps of the neural network model. A classic forward neural network layer can be expressed as:

[0157]

[0158] Among them, W i,j Indicates the connection of the i th input neuron and the jth th The weight parameters of the output neurons, x i represents the input of the i-th input neuron, b represents the bias parameter of the neural network layer, f(·) represents the activation function, for example, the ReLU activation function, l represents the layer number of the neural network layer, and y j represents the output of the jth output neuron.

[0159] Existing weight quantization and activation value quantization methods: Neural network model quantization includes weight quantization and activation value quantization. The existing weight quantization and activation value quantization methods can be expressed as:

[0160]

[0161]

[0162] in, represents the quantized weight, represents the quantized activation value, w represents the weight vector, x represents the input of the neural network model, and s w and x They represent the quantization step size of the weight and the quantization step size of the activation value, clip(·) represents the truncation function, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, x l and x u represents the upper and lower truncation boundaries of the activation value quantization, and [·] represents the nearest neighbor rounding operation.

[0163] Weight quantization and activation value quantization method of this embodiment: The weight quantization and activation value quantization of this embodiment adopt the following method:

[0164]

[0165]

[0166] in, represents the quantized weight, represents the quantized activation value, w represents the weight vector, x represents the input of the neural network model, and s w and x They represent the quantization step size of the weight and the quantization step size of the activation value, clip(·) represents the truncation function, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, x l and x u represents the upper and lower truncation boundaries of the activation value quantization, [·] represents the nearest neighbor rounding operation, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, which is a learnable adaptive rounding parameter of the weight.

[0167] Different from the existing quantization method, the weight quantization in this embodiment adopts an adaptive rounding method.

[0168] Pseudo-quantization and true quantization: In order to enable the post-quantization method to be executed on the hardware GPU or CPU, the quantization operation needs to be simulated using the FP32 floating point format. This operation is defined as pseudo-quantization. When the actual hardware performs quantization inference, the simulated calculation of the FP32 floating point will be converted into a fixed-point integer calculation. This operation is defined as true quantization. The two operations can be expressed as:

[0169]

[0170] Among them, y represents the output of the neural network model, represents the quantized weight, represents the quantized activation value, x represents the input of the neural network model, and s w and x Respectively represent the quantization step size of the weight and the quantization step size of the activation value, W int Integer data representing weights, which are parameters that can be obtained before actual hardware quantization deployment:

[0171]

[0172] Among them, W int Integer data representing weights, clip(·) represents the cutoff function, w represents the vector of weights, s w represents the quantization step size of the weight, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, and w l and w u Represents the upper and lower truncation boundaries of weight quantization.

[0173] Furthermore, the decoupling of the quantization step of multiple layers of weights in the neural network model to be quantized into weight inverse quantization step includes: decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into weight quantization step and weight inverse quantization step, the weight quantization step is used to convert floating-point weights into integer weights, and the weight inverse quantization step is used to convert integer weights back into floating-point weights; setting the weight quantization step to be non-updatable, and setting the weight inverse quantization step to be updatable.

[0174] It should be understood that in true quantization, the integer weights W int and weight quantization step size s w can be obtained before actual hardware deployment, so they can be two independent parameters. Therefore, in pseudo quantization, the quantization step size s of the weight can be w Decoupled into two independent step sizes, one is the weight quantization step size s that converts floating point weights into integer weights w , one is the dequantization step size s′ that converts integer weights back to floating-point weights w , and set the weight quantization step size to non-updatable, and the weight inverse quantization step size to updatable.

[0175] The pseudo-quantization module 20 is used to perform pseudo-quantization on the neural network model to be quantized.

[0176] It should be noted that pseudo-quantization is an approximate quantization method used to simulate the quantization process, but the quantized weights and activation values ​​are not used in actual reasoning. In the pseudo-quantization process, the model will perform operations similar to quantization, such as mapping floating-point weights and activation values ​​to integers or fixed-point representations within a fixed range, but floating-point precision is still used in actual calculations. Pseudo-quantization can be used to evaluate the impact of quantization on model performance and assist in optimizing parameter selection during the quantization process.

[0177] Therefore, in this embodiment, after pseudo-quantization of the neural network model to be quantized, the quantization errors of the neural network model to be quantized before and after pseudo-quantization can be inferred based on the calibration data set, and the weight inverse quantization step size can be updated according to the quantization error.

[0178] The inference module 30 is used to infer the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set.

[0179] It should be noted that the calibration data set can be a pre-constructed data sample set, which is used to infer the quantization error of the neural network model before and after pseudo-quantization when optimizing the model, and optimize and update the model quantization based on the quantization error to reduce the quantization error loss, improve the quantization performance, and improve the model quantization accuracy.

[0180] It can be understood that the quantization error of the neural network model to be quantized before and after pseudo-quantization inferred based on the calibration data set can be performed by performing forward reasoning in the neural network model to be quantized based on the calibration data set to obtain floating-point activation values ​​of multiple layers of output in the neural network model to be quantized, performing forward reasoning in the pseudo-quantized neural network model based on the calibration data set to obtain quantized activation values ​​of multiple layers of output in the pseudo-quantized neural network model, and inferring the quantization error of the neural network model to be quantized before and after pseudo-quantization based on the floating-point activation values ​​and the quantized activation values.

[0181] The quantization module 40 is used to update the weight inverse quantization step size according to the quantization error to obtain a quantized neural network model.

[0182] It can be understood that, in this embodiment, by decoupling the quantization step of the weight, the weight quantization process is transformed into the following:

[0183]

[0184] in, represents the quantized weight, clip(·) represents the truncation function, w represents the vector of weights, s w represents the weight quantization step size, represents the rounding down operation, h(α) represents the weight adaptive rounding parameter, and w l and w u Represents the upper and lower truncation boundaries of weight quantization, s′ w Represents the weight inverse quantization step size.

[0185] Correspondingly, the true quantization process is transformed into the following:

[0186]

[0187]

[0188] Among them, y represents the output of the neural network model, s′ w Represents the weight dequantization step size, s x represents the quantization step size of the activation value, W int Integer data representing weights, x represents the input of the neural network model, clip(·) represents the truncation function, s w represents the weight quantization step size, w represents the weight vector, h(α) represents the weight adaptive rounding parameter, w l and w u Represents the upper and lower truncation boundaries of weight quantization.

[0189] Under the condition of weight quantization step decoupling, the weight quantization step s w The parameters are fixed, and the weight is dequantized with a step size s′ wParticipating in optimization updates can reduce quantization error losses, thereby improving quantization performance and improving model quantization accuracy.

[0190] It should be understood that updating the weight dequantization step size according to the quantization error to obtain the quantized neural network model can be based on the quantization error through the gradient back propagation algorithm to propagate the quantization error back to the model's weight dequantization step size, updating the weight dequantization step size, and determining the weight dequantization step size through the mean square error optimization algorithm to obtain the quantized neural network model.

[0191] In a specific implementation, for example, the mean square error optimization algorithm can be expressed as:

[0192]

[0193] Among them, s′ w represents the weight inverse quantization step size, represents the quantized weight, w represents the weight vector, Represents the absolute value of the quantization error. The above formula means that the quantization error is reduced by optimizing and updating the weight inverse quantization step size, and the model is quantized according to the weight inverse quantization step size corresponding to the minimum quantization error to obtain the quantized neural network model.

[0194] The deployment module 50 is used to deploy the quantized neural network model to a target deployment platform according to the quantized parameters of the quantized neural network model.

[0195] It should be understood that in order to apply the neural network model to low-performance devices, especially to terminal products, in this embodiment, after obtaining the quantized neural network model, the quantized neural network model can be deployed to the target deployment platform according to the quantization parameters of the quantized neural network model, wherein the target deployment platform can be a low-performance device, for example, a mobile device (mobile phone, tablet computer), an Internet of Things device, an edge computing device, etc. This embodiment does not limit this, and the quantization parameters include but are not limited to at least one of the weight inverse quantization step, the weight adaptive rounding parameter, and the activation value quantization step.

[0196] In this embodiment, it is disclosed that the quantization step of multiple layers of weights in the neural network model to be quantized is decoupled into the weight inverse quantization step, the weight inverse quantization step is used to convert the integer weights back into floating-point weights, the neural network model to be quantized is pseudo-quantized, the quantization error of the neural network model to be quantized before and after pseudo-quantization is inferred based on a calibration data set, the weight inverse quantization step is updated according to the quantization error, the quantized neural network model is obtained, and the quantized neural network model is deployed to the target deployment platform according to the quantization parameters of the quantized neural network model; because the quantization step of the weight is decoupled into the weight inverse quantization step in this embodiment, the weight inverse quantization step can participate in the optimization update, thereby reducing the quantization error loss, thereby improving the quantization performance and the model quantization accuracy.

[0197] In one embodiment, the quantization module 40 is also used to update the weight inverse quantization step and weight adaptive rounding parameter according to the quantization error to obtain a quantized neural network model, and the weight adaptive rounding parameter is used to adjust the rounding method of the weight.

[0198] In one embodiment, the quantization module 40 is also used to infer the output error of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set and the quantization error; the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step are updated according to the output error to obtain the quantized neural network model, and the activation value quantization step is used to convert the floating-point activation value into an integer activation value.

[0199] In one embodiment, the quantization module 40 is also used to update the weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step according to the output error to obtain a pseudo-quantized neural network model; determine the integer weights, weight inverse quantization step and activation value quantization step of the true quantized neural network model according to the quantization parameters of the pseudo-quantized neural network model to obtain a quantized neural network model.

[0200] In one embodiment, the decoupling module 10 is also used to decouple the quantization step of the multiple layers of weights in the neural network model to be quantized into a weight quantization step and a weight dequantization step, wherein the weight quantization step is used to convert the floating-point weights into integer weights, and the weight dequantization step is used to convert the integer weights back into floating-point weights; the weight quantization step is set to be non-updatable, and the weight dequantization step is set to be updateable.

[0201] In one embodiment, the model quantization device further includes:

[0202] The preprocessing module is used to fuse the network layers of the initial neural network model according to the deployment configuration information of the initial neural network model to obtain a fused neural network model; initialize the quantization step of the multi-layer weights in the fused neural network model; initialize the activation value quantization step of the multi-layer output activation values ​​in the fused neural network model to obtain a neural network model to be quantized.

[0203] In one embodiment, the inference module 30 is used to perform forward inference in the neural network model to be quantized based on the calibration data set to obtain floating-point activation values ​​of multiple layers of output in the neural network model to be quantized; perform forward inference in the pseudo-quantized neural network model based on the calibration data set to obtain quantized activation values ​​of multiple layers of output in the pseudo-quantized neural network model; and infer the quantization error of the neural network model to be quantized before and after pseudo-quantization based on the floating-point activation values ​​and the quantized activation values.

[0204] Other embodiments or specific implementations of the model quantization device described in this application can refer to the above-mentioned method embodiments and will not be described in detail here.

[0205] It should be noted that in the technical solution of this specification, the operations on the data involved are in compliance with relevant regulations and do not violate public order and good morals. For example, operations on data are performed on the premise of obtaining user authorization. In this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0206] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0207] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0208] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A model quantization method, characterized in that: The model quantization method comprises: Decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into a weight inverse quantization step, wherein the weight inverse quantization step is used to convert integer weights back to floating-point weights; Performing pseudo-quantization on the neural network model to be quantized; Inferring the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set; The weight dequantization step size is updated according to the quantization error to obtain a quantized neural network model; The quantized neural network model is deployed to a target deployment platform according to the quantized parameters of the quantized neural network model.

2. The model quantization method according to claim 1, characterized in that: The step of updating the weight dequantization step size according to the quantization error to obtain a quantized neural network model includes: The weight inverse quantization step size and weight adaptive rounding parameter are updated according to the quantization error to obtain a quantized neural network model, and the weight adaptive rounding parameter is used to adjust the rounding method of the weight.

3. The model quantization method according to claim 2, characterized in that: The step of updating the weight inverse quantization step size and the weight adaptive rounding parameter according to the quantization error to obtain a quantized neural network model includes: Inferring the output error of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set and the quantization error; The weight inverse quantization step, weight adaptive rounding parameter and activation value quantization step are updated according to the output error to obtain a quantized neural network model, wherein the activation value quantization step is used to convert the floating-point activation value into an integer activation value.

4. The model quantization method according to claim 3, characterized in that: The step of updating the weight inverse quantization step, the weight adaptive rounding parameter, and the activation value quantization step according to the output error to obtain a quantized neural network model includes: The weight inverse quantization step size, the weight adaptive rounding parameter and the activation value quantization step size are updated according to the output error to obtain a pseudo-quantized neural network model; The integer weights, weight inverse quantization step size and activation value quantization step size of the true quantized neural network model are determined according to the quantization parameters of the pseudo-quantized neural network model to obtain a quantized neural network model.

5. The model quantization method according to any one of claims 1 to 4, characterized in that: The step of decoupling the quantization step of the multi-layer weights in the neural network model to be quantized into the weight inverse quantization step includes: Decoupling the quantization step of the multi-layer weights in the neural network model to be quantized into a weight quantization step and a weight dequantization step, wherein the weight quantization step is used to convert the floating-point weights into integer weights, and the weight dequantization step is used to convert the integer weights back into floating-point weights; The weight quantization step size is set to be non-updatable, and the weight inverse quantization step size is set to be updatable.

6. The model quantization method according to any one of claims 1 to 4, characterized in that: The model quantization method further includes: Fusing the network layers of the initial neural network model according to the deployment configuration information of the initial neural network model to obtain a fused neural network model; Initializing the quantization step lengths of the multi-layer weights in the fused neural network model; The activation value quantization step length of the multi-layer output activation values ​​in the fused neural network model is initialized to obtain the neural network model to be quantized.

7. The model quantization method according to any one of claims 1 to 4, characterized in that: The inferring the quantization errors of the neural network model to be quantized before and after pseudo-quantization based on the calibration data set includes: Perform forward reasoning in the neural network model to be quantized based on the calibration data set to obtain floating-point activation values ​​of multiple layers of output in the neural network model to be quantized; Performing forward reasoning in the pseudo-quantized neural network model based on the calibration data set to obtain quantized activation values ​​of multiple layers of output in the pseudo-quantized neural network model; The quantization errors of the neural network model to be quantized before and after pseudo-quantization are inferred based on the floating-point activation value and the quantized activation value.

8. A model quantization device, characterized in that: The model quantization device comprises: A decoupling module, used for decoupling the quantization step of multiple layers of weights in the neural network model to be quantized into a weight inverse quantization step, wherein the weight inverse quantization step is used to convert the integer weights back to floating-point weights; A pseudo-quantization module, used for pseudo-quantizing the neural network model to be quantized; An inference module, used for inferring the quantization error of the neural network model to be quantized before and after pseudo-quantization based on a calibration data set; A quantization module, used for updating the weight inverse quantization step size according to the quantization error to obtain a quantized neural network model; A deployment module is used to deploy the quantized neural network model to a target deployment platform according to the quantized parameters of the quantized neural network model.

9. A model quantization device, characterized in that: The model quantization device comprises: a memory, a processor, and a model quantization program stored in the memory and executable on the processor. When the model quantization program is executed by the processor, the model quantization method according to any one of claims 1 to 7 is implemented.

10. A storage medium, characterized in that: The storage medium stores a model quantization program, and when the model quantization program is executed by the processor, the model quantization method according to any one of claims 1 to 7 is implemented.

Images