Training method and device of quantization model, electronic equipment and readable storage medium

Abstract

The application relates to a training method and device of a quantization model, a computer device and a storage medium. The method comprises the following steps: acquiring sample data; performing first-stage training on the quantization model through the sample data, obtaining a floating-point weight range of the quantization model; performing second-stage training on the quantization model trained in the first stage based on the floating-point weight range and the sample data, and obtaining a floating-point activation range of a trained target quantization model in the second-stage training; wherein the floating-point weight range and the floating-point activation range in the target quantization model are used for quantization processing on to-be-quantized data. The method can improve the accuracy of the quantization model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for training a quantization model. Background Technology

[0002] With the development of computer technology, data quantization processing techniques have emerged. For example, by using quantization models to quantize floating-point data, it is possible to convert floating-point data into integer data, thereby reducing the amount of computation. However, traditional quantization models do not have high quantization accuracy. Summary of the Invention

[0003] This application provides a training method, apparatus, electronic device, and computer-readable storage medium for a quantization model, which can improve the quantization accuracy of the quantization model.

[0004] A method for training a quantization model, comprising:

[0005] Obtain sample data;

[0006] The quantization model is trained in the first stage using the sample data to obtain the floating-point weight range of the quantization model.

[0007] Based on the floating-point weight range and the sample data, the quantization model trained in the first stage is trained in the second stage, and the floating-point activation range of the trained target quantization model is obtained in the second stage of training.

[0008] The floating-point weight range and floating-point activation range in the target quantization model are used to perform quantization processing on the data to be quantized.

[0009] A training device for a quantization model, comprising:

[0010] The sample acquisition module is used to acquire sample data;

[0011] The first training module is used to train the quantization model in the first stage using the sample data to obtain the floating-point weight range of the quantization model.

[0012] The second training module is used to train the quantization model trained in the first stage based on the floating-point weight range and the sample data, and to obtain the floating-point activation range of the trained target quantization model in the second stage of training.

[0013] The floating-point weight range and floating-point activation range in the target quantization model are used to perform quantization processing on the data to be quantized.

[0014] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0015] Obtain sample data;

[0016] The quantization model is trained in the first stage using the sample data to obtain the floating-point weight range of the quantization model.

[0017] Based on the floating-point weight range and the sample data, the quantization model trained in the first stage is trained in the second stage, and the floating-point activation range of the trained target quantization model is obtained in the second stage of training.

[0018] The floating-point weight range and floating-point activation range in the target quantization model are used to perform quantization processing on the data to be quantized.

[0019] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0020] Obtain sample data;

[0021] The quantization model is trained in the first stage using the sample data to obtain the floating-point weight range of the quantization model.

[0022] Based on the floating-point weight range and the sample data, the quantization model trained in the first stage is trained in the second stage, and the floating-point activation range of the trained target quantization model is obtained in the second stage of training.

[0023] The floating-point weight range and floating-point activation range in the target quantization model are used to perform quantization processing on the data to be quantized.

[0024] The quantization model training method, apparatus, electronic device, and computer-readable storage medium in this embodiment acquire sample data and perform a first-stage training on the quantization model using the sample data to obtain the floating-point weight range of the quantization model. Based on the floating-point weight range and the sample data, a second-stage training is performed on the quantization model trained in the first stage, and the floating-point activation range of the trained target quantization model is obtained in the second-stage training. Thus, through different stages of training, the processing accuracy of the quantization model can be improved. The floating-point weight range and floating-point activation range of the target quantization model can perform quantization processing on the data to be quantized, converting floating-point data into integer data, reducing the amount of data computation and improving the speed of quantization processing. Furthermore, converting floating-point data into integer data also reduces memory usage.

[0025] A quantization processing method, comprising:

[0026] Obtain the quantization type corresponding to the data to be quantized, and obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type;

[0027] Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the data to be quantized are determined.

[0028] Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the data to be quantized are determined; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data.

[0029] Based on the weight quantization parameter and the activation quantization parameter, the data to be quantized is quantized into target data under the quantization type.

[0030] A quantization processing device, comprising:

[0031] The type acquisition module is used to acquire the quantization type corresponding to the data to be quantized, and to acquire the fixed-point weight range and fixed-point activation range corresponding to the quantization type.

[0032] The weight determination module is used to determine the weight quantization parameters corresponding to the data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model.

[0033] The activation determination module is used to determine the activation quantization parameters corresponding to the data to be quantized based on the fixed-point activation range and the floating-point activation range of the trained quantization model; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data.

[0034] The quantization module is used to quantize the data to be quantized into target data under the quantization type based on the weight quantization parameters and the activation quantization parameters.

[0035] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0036] Obtain the quantization type corresponding to the data to be quantized, and obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type;

[0037] Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the data to be quantized are determined.

[0038] Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the data to be quantized are determined; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data.

[0039] Based on the weight quantization parameter and the activation quantization parameter, the data to be quantized is quantized into target data under the quantization type.

[0040] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0041] Obtain the quantization type corresponding to the data to be quantized, and obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type;

[0042] Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the data to be quantized are determined.

[0043] Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the data to be quantized are determined; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data.

[0044] Based on the weight quantization parameter and the activation quantization parameter, the data to be quantized is quantized into target data under the quantization type.

[0045] The aforementioned quantization processing method, apparatus, electronic device, and computer-readable storage medium acquire the quantization type corresponding to the data to be quantized, and acquire the fixed-point weight range and fixed-point activation range corresponding to the quantization type; based on the fixed-point weight range and the floating-point weight range of the trained quantization model, determine the weight quantization parameters corresponding to the data to be quantized; based on the fixed-point activation range and the floating-point activation range of the trained quantization model, determine the activation quantization parameters corresponding to the data to be quantized; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data; based on the weight quantization parameters and activation quantization parameters, it can accurately quantize the data to be quantized from floating-point type to integer target data under the quantization type.

[0046] When applied to image processing, operations based on integer data can reduce the amount of computation and increase the speed of computation, thereby improving the speed of subsequent image processing. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a diagram illustrating the application environment of a training method for a quantized model in one embodiment.

[0049] Figure 2 This is a flowchart of a training method for a quantization model in one embodiment;

[0050] Figure 3 This is a schematic diagram of a training method for a quantization model in one embodiment;

[0051] Figure 4 This is a schematic diagram of a pseudo-quantized node in one embodiment;

[0052] Figure 5 A flowchart of a training method for a quantization model in another embodiment;

[0053] Figure 6 Here is a flowchart of a quantization processing method in one embodiment;

[0054] Figure 7 This is a structural block diagram of a training device for a quantization model in one embodiment;

[0055] Figure 8 This is a structural block diagram of a quantization processing device in one embodiment;

[0056] Figure 9 This is a diagram of the internal structure of an electronic device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] Figure 1 This is a schematic diagram illustrating the application environment of a quantization model training method in one embodiment. For example... Figure 1As shown, the application environment includes an electronic device 110 and a server 120. In one embodiment, both the electronic device 110 and the server 120 can execute the training method of the quantization model independently, or they can execute it collaboratively. When the electronic device 110 and the server 120 collaboratively execute the training method of the quantization model, the electronic device 110 can acquire sample data and send it to the server 120. The server 120 performs a first-stage training of the quantization model using the sample data to obtain the floating-point weight range of the quantization model. Based on the floating-point weight range and the sample data, the server 120 performs a second-stage training of the quantization model trained in the first stage, and obtains the floating-point activation range of the trained target quantization model in the second-stage training. The floating-point weight range and floating-point activation range in the target quantization model are used for quantization processing of the data to be quantized.

[0059] In this system, electronic device 110 communicates with server 120 via a network. Electronic device 110 may be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. Server 120 may be implemented using a standalone server or a server cluster consisting of multiple servers.

[0060] Figure 2 This is a flowchart of a training method for a quantization model in one embodiment. The training method for the quantization model in this embodiment is implemented on... Figure 1 The description will be based on an example of an electronic device. Figure 2 As shown, the training method for this quantization model includes:

[0061] Step 202: Obtain sample data.

[0062] Sample data refers to floating-point data used to train quantization models, and can be multimedia data. Multimedia data can specifically include image data, text data, audio data, and video data, but is not limited to these. Floating-point data refers to floating-point type data, such as 0.13, 5.789, etc.

[0063] Specifically, the electronic device can acquire sample data from local devices or other devices or networks. For example, the electronic device can acquire multimedia data from local devices or other devices or networks. This multimedia data can be at least one of image data, text data, audio data, and video data, with at least one of these types of data used as sample data.

[0064] In one embodiment, an electronic device can acquire multimedia data from a local device or other device or network, convert the content information of the multimedia data into numerical information, and obtain the numerical information corresponding to the sample multimedia data.

[0065] For example, electronic devices can acquire sample images from local devices or other devices or networks, obtain sample images through photography, or extract video frames as sample images. The image information of the sample images is then converted into data information to obtain the image data of the sample image. Image data refers to the collection of grayscale values ​​of each pixel, represented numerically. This image data serves as sample data and is used to train a quantization model. The trained quantization model is then used to quantize the image data of the image to be processed.

[0066] Similarly, electronic devices can acquire sample text, convert it into corresponding data information, and obtain text data. This text data serves as sample data for training a quantization model, and the trained quantization model is used to quantize the text data to be processed.

[0067] Similarly, electronic devices can acquire sample audio and video and convert them into corresponding data information, obtaining audio data and video data. This audio and video data serves as sample data for training a quantization model. The trained quantization model is then used to quantize the audio and video data to be processed.

[0068] Step 204: Train the quantization model in the first stage using sample data to obtain the floating-point weight range of the quantization model.

[0069] Specifically, the floating-point weight range is the weight range corresponding to floating-point data. The electronic device inputs sample data into the quantization model, which uses the sample data for the first stage of training. During the first stage of training, the weight range of the quantization model is adjusted until the first stage of training stops, at which point the floating-point weight range in the quantization model is obtained.

[0070] Step 206: Based on the floating-point weight range and sample data, perform a second-stage training on the quantization model trained in the first stage, and obtain the floating-point activation range of the trained target quantization model in the second-stage training.

[0071] Among them, the floating-point weight range and floating-point activation range in the target quantization model are used to quantize the data to be quantized.

[0072] Specifically, the floating-point activation range is the activation range corresponding to floating-point data, capable of mapping floating-point data to a specific range. After the first stage of training stops, the floating-point weight range of the quantization model is obtained. In the second stage of training, this floating-point weight range is fixed, meaning it remains unchanged throughout the second stage. Based on the floating-point weight range and sample data, the quantization model trained in the first stage is trained in the second stage. During the second stage, the activation range of the quantization model is adjusted until the second stage of training stops, yielding the floating-point activation range of the quantization model.

[0073] The floating-point weight range and floating-point activation range in this target quantization model are used to quantize the data to be quantized.

[0074] It is understandable that by training a quantization model using image data, the floating-point weight range and floating-point activation range of the image in the target quantization model can be obtained. The floating-point weight range and floating-point activation range of the image are used to quantize the image data to be processed, converting floating-point image data into integer image data. The quantization of text, audio, and video data is similar to that of image data and will not be elaborated upon here.

[0075] The quantization model training method in this embodiment acquires sample data and performs a first-stage training on the quantization model using this sample data to obtain the floating-point weight range of the quantization model. Based on the floating-point weight range and the sample data, a second-stage training is performed on the quantization model trained in the first stage. During the second-stage training, the floating-point activation range of the trained target quantization model is obtained. This allows for higher processing accuracy of the quantization model through different training stages. The floating-point weight range and floating-point activation range of the target quantization model enable quantization processing of the data to be quantized, converting floating-point data into integer data, reducing the amount of data computation and improving the speed of quantization processing. Furthermore, converting floating-point data into integer data also reduces memory usage.

[0076] In one embodiment, the quantization model is trained in the first stage using sample data to obtain the floating-point weight range of the quantization model, including:

[0077] For the first stage of training, the sample data is processed by forward propagation through the quantization model to obtain the first intermediate result; the first loss error of the quantization model is determined by forward propagation, and the initial weight range of the quantization model is statistically determined by forward propagation; during the backpropagation process on the first intermediate result, the initial weight range is adjusted based on the first loss error until the quantization model meets the first stopping condition, thus obtaining the floating-point weight range of the quantization model.

[0078] Forward propagation refers to the process of processing from the input to obtaining the loss error, while backpropagation is the process of continuously adjusting parameters based on the loss error. The initial weight range is the initial weight range corresponding to floating-point data.

[0079] Specifically, the electronic device inputs sample data into the quantization model, using the sample data for the first stage of training the quantization model. In this first stage, the quantization model performs forward propagation processing on the sample data. During forward propagation, convolution, quantization, and dequantization processes are performed on the sample data to obtain the first intermediate result. Furthermore, during forward propagation, the initial weight range of the quantization model is statistically determined through convolution, quantization, and dequantization processes, and the loss error of the quantization model, i.e., the first loss error, is determined.

[0080] During the backpropagation process of the first intermediate result, the initial weight range is adjusted based on the first loss error. Based on the adjusted initial weight range, the quantization model continues to be trained until it meets the first stopping condition, at which point the floating-point weight range of the quantization model is obtained.

[0081] In one embodiment, the first stopping condition may be that the first loss error is less than an error threshold, or that the number of training iterations in the first stage reaches a preset number of iterations, etc.

[0082] In one embodiment, the electronic device can determine the first loss error of the quantization model using a first intermediate result and sample data.

[0083] In one embodiment, the initial activation range of the statistical quantization model is also processed via forward propagation.

[0084] In this embodiment, for the first stage of training, the sample data is processed by forward propagation through the quantization model to obtain the first intermediate result. The first loss error of the quantization model is determined by forward propagation, and the initial weight range of the quantization model is statistically determined by forward propagation. During the backpropagation process of the first intermediate result, the initial weight range is adjusted based on the first loss error so that the weight range of the quantization model reaches the optimal value. The process continues until the quantization model meets the first stopping condition, thereby accurately obtaining the final floating-point weight range of the quantization model through the first stage of training.

[0085] In one embodiment, forward propagation processing of sample data using a quantization model yields a first intermediate result, including:

[0086] The quantization process quantizes the features output by the current convolution operator in the quantization model, obtaining quantized features. The output features of the first convolution operator are obtained by convolution processing the sample data. The quantized features are dequantized to obtain dequantized features. The dequantized features are used as the input of the next convolution operator, and the next convolution operator is used as the current convolution operator. The process returns to the step of quantizing the features output by the current convolution operator and continues until the last dequantization process is performed, resulting in the first intermediate result.

[0087] Specifically, in the first stage of training, the sample data is input into the first convolution operator of the quantization model. This first convolution operator performs convolution processing on the sample data to obtain the output features. The output features are then quantized to obtain quantized features, and finally, the quantized features are dequantized to obtain dequantized features.

[0088] The inverse quantized features are used as input to the second convolution operator. The second convolution operator performs convolution processing on the input inverse quantized features, quantizes the features obtained from the convolution processing, and then performs inverse quantization processing on the quantized features obtained from the quantization processing to obtain the inverse quantized features.

[0089] Understandably, in the first stage of training, the features output by the current convolution operator are quantized to obtain quantized features. These quantized features are then dequantized to obtain dequantized features, which are used as input to the next convolution operator. The next convolution operator is then used as the current convolution operator, and the process returns to the step of quantizing the features output by the current convolution operator, continuing until the final dequantization process, at which point the first intermediate result is obtained.

[0090] In one embodiment, the convolution operator's kernel size is consistent with the width and height of the input features, and it is a matrix multiplication operation between the input features and the kernel. The processing procedure of this convolution operator is as follows:

[0091] out float =Matmul(input) float kernel float )

[0092] output float =output scale (output quant -output zero )

[0093] input float =input scale (input quant -inputzero )

[0094] kernel float =kernel scale (kernel quant -kernel zero )

[0095] Where, output float The input represents the output floating-point data. float Represents the input floating-point data, kernel float This represents the floating-point data corresponding to the convolution kernel, Matmul(input) float kernel float ) represents input float and kernel float Perform matrix multiplication.

[0096] output scale Represents the fixed-point quantization scaling factor, output quant The output represents the fixed-point data. zero This represents the fixed-point quantization offset. scale Indicates the floating-point quantization scaling factor, input quant This represents the fixed-point input data. zero This represents the floating-point quantization offset. kernel scale This represents the quantization scaling factor corresponding to the convolution kernel. quant This represents the fixed-point data corresponding to the convolution kernel. zero This represents the quantization offset corresponding to the convolution kernel.

[0097] The quantization calculation formula can be obtained from the above formula, which yields the fixed-point data output of the convolution operator. quant .

[0098] In this embodiment, the features output by the current convolution operator in the quantization model are quantized to obtain quantized features. The quantized features are then dequantized to obtain dequantized features. This allows the error generated after quantization and dequantization of the data to be determined. The dequantized features are used as the input of the next convolution operator, and the next convolution operator is used as the current convolution operator. The process returns to the step of quantizing the features output by the current convolution operator and continues until the last dequantization is performed, resulting in the first intermediate result. This allows the loss error generated during the processing to be accurately obtained, so as to adjust the weight range of the quantization model.

[0099] In one embodiment, based on the floating-point weight range and sample data, a second stage of training is performed on the quantization model trained in the first stage, and the floating-point activation range of the trained target quantization model is obtained in the second stage of training, including:

[0100] For the second stage of training, the sample data is forward-propagated using the floating-point weight range of the quantization model trained in the first stage to obtain the second intermediate result. The second loss error of the quantization model is determined based on the forward propagation process. During the backpropagation process on the second intermediate result, the initial activation range of the quantization model is adjusted based on the second loss error until the quantization model stops when it meets the second stopping condition, thus obtaining the floating-point activation range of the trained target quantization model.

[0101] The initial activation range is obtained through forward propagation statistics during the first phase of training.

[0102] Specifically, in the first stage of training, the initial activation range of the statistical quantization model is processed through forward propagation. The quantization model trained in the first stage is then trained in the second stage using sample data and floating-point weight ranges to adjust this initial activation range.

[0103] After the first stage of training, the quantization model performs forward propagation on the sample data. During forward propagation, convolution is applied to the sample data based on the floating-point weight range, and quantization and dequantization are then performed on the convolutionally processed data based on the initial activation range to obtain the second intermediate result. Finally, the loss error of the quantization model, i.e., the second loss error, is determined by performing convolution on the sample data based on the floating-point weight range and quantization and dequantization on the convolutionally processed data based on the initial activation range.

[0104] In the backpropagation process, the second intermediate result is used as input, and the process is performed in reverse order of the forward propagation process. During the backpropagation of the second intermediate result, the initial activation range is adjusted based on the second loss error. Based on the adjusted initial activation range, the quantized model continues to be trained until the quantized model meets the second stopping condition, thus obtaining the floating-point activation range of the quantized model.

[0105] In one embodiment, the second stopping condition may be that the second loss error is less than the error threshold, or that the number of training iterations in the second stage reaches a preset number of iterations, etc.

[0106] In one embodiment, the electronic device can determine the second loss error of the quantization model using a second intermediate result and sample data.

[0107] In this embodiment, for the second stage of training, the sample data is forward-propagated using the floating-point weight range in the quantization model after the first stage of training to obtain a second intermediate result. The second loss error of the quantization model is determined based on the forward propagation process. During the backpropagation process on the second intermediate result, the initial activation range of the quantization model is adjusted based on the second loss error until the quantization model meets the second stopping condition. Thus, the target quantization model and the floating-point activation range in the target quantization model can be accurately obtained through the second stage of training.

[0108] like Figure 3 The diagram shown is a schematic of a training method for a quantization model in one embodiment.

[0109] The quantization model used for training includes convolution operators (i.e., dense operators) and pseudo-quantization nodes (i.e., FakeQuant). The quantization model is trained in two phases. In the first phase, the weight range of the quantization model is set as the training parameters, i.e., param. trainable ={dense's weights}. The first stage of training includes forward propagation and backward propagation. In the forward propagation, sample data is input into the first convolution operator for convolution processing to determine the initial weight range corresponding to the first convolution operator. The features output by the first convolution operator are used as the input to the first pseudo-quantized node. The pseudo-quantized node performs quantization and dequantization on the input features to obtain the dequantized features output by the first pseudo-quantized node, and determines the initial activation range corresponding to the first quantized node. The output of the first pseudo-quantized node is used as the input to the next convolution operator, and the above convolution, quantization, and dequantization processes are performed until the dequantized features output by the last pseudo-quantized node are obtained, which is the first intermediate result.

[0110] The first loss error of the quantization model is calculated using the first intermediate result and sample data, and the gradient of the weight range with respect to the loss of the quantization model is calculated based on the first loss error.

[0111] The structure of the pseudo-quantization node is as follows Figure 4 As shown, the pseudo-quantization node includes quantizing the input features and dequantizing the quantized features to obtain dequantized features.

[0112] A pseudo-quantization node can be viewed as a custom operator implementation that quantizes the input data and then dequantizes it to simulate the error introduced by quantization. The quantization process is as follows:

[0113]

[0114] x Q =clamp(0, N)levels -1,x int )

[0115]

[0116] Where x is the input floating-point data, x Q The data is quantized, Δ is the quantization parameter scale, and z is the quantization zero point.

[0117] N levels This refers to the quantization range, such as when quantized to 8 bits, N levels =2 8 =256.

[0118] The dequantization process is as follows:

[0119] x float =(x Q -z)Δ

[0120] x float This is the floating-point data obtained after dequantization, with a quantization error of diff = xx. float .

[0121] In the backpropagation process, the last pseudo-quantization node in the forward propagation process becomes the first quantization node in the backpropagation process, and the last convolution operator becomes the first convolution operator in the backpropagation process. The first intermediate result is used as the input to the first pseudo-quantization node. The first pseudo-quantization node performs dequantization and quantization on the first intermediate result, obtaining the features output by the first pseudo-quantization node. The initial weight range of the first convolution operator is adjusted based on the gradient of the quantization model loss. The output of the first pseudo-quantization node is used as the input to the first convolution operator, and the input features are convolved using the adjusted weight range of the first convolution operator. The output of the first convolution operator is used as the input to the next pseudo-quantization node, and the above dequantization, quantization, and convolution processes are executed sequentially until the features output by the last convolution operator are obtained. This completes the adjustment of the initial weight range of each convolution operator, resulting in the updated quantization model.

[0122] In the backpropagation process, after adjusting the initial weight range of each convolution operator, the loss error of the updated quantization model is calculated based on the input features of the backpropagation process and the output features of the last convolution operator, in order to determine whether the updated quantization model has reached the convergence state, i.e., satisfies the first stopping condition.

[0123] If the convergence state is not reached, the updated quantization model is trained again in the first stage until the quantization model reaches the convergence state, thereby obtaining the floating-point weight range corresponding to each convolution operator in the quantization model.

[0124] The quantized model trained in the first stage undergoes a second stage of training. In this second stage, the floating-point weight range is set as a non-trainable parameter, while the activation range is set as a trainable parameter, i.e., param. trainable ={activations′range}. The second stage of training includes forward propagation and backpropagation. Sample data is input into the first convolution operator, and convolution is performed on the sample data using the floating-point weight range corresponding to the first convolution operator. The features output by the first convolution operator are used as input to the first pseudo-quantized node. The pseudo-quantized node performs quantization and dequantization on the input features, obtaining the dequantized features output by the first pseudo-quantized node. The output of the first pseudo-quantized node is used as input to the next convolution operator, performing the above convolution, quantization, and dequantization processes until the dequantized features output by the last pseudo-quantized node are obtained, which is the second intermediate result.

[0125] The second loss error of the quantization model is calculated using the second intermediate result and sample data, and the gradient of the activation range with respect to the loss of the quantization model is calculated based on the second loss error.

[0126] In the backpropagation process, the last pseudo-quantization node in the forward propagation process becomes the first quantization node in the backpropagation process, and the last convolution operator becomes the first convolution operator in the backpropagation process. The initial activation range of the first convolution operator is adjusted based on the gradient of the quantized model loss. The second intermediate result is used as the input to the first pseudo-quantization node. The second intermediate result is then dequantized and quantized using the initial activation range of the first pseudo-quantization node to obtain the features output by the first pseudo-quantization node. The output of the first pseudo-quantization node is then used as the input to the first convolution operator, and the input features are convolved using the adjusted activation range of the first convolution operator. The output of the first convolution operator is then used as the input to the next pseudo-quantization node, and the above dequantization, quantization, and convolution processes are executed sequentially until the features output by the last convolution operator are obtained. This completes the adjustment of the initial activation range of each convolution operator, resulting in the updated quantized model.

[0127] In the backpropagation process, after adjusting the initial activation range of each convolution operator, the loss error of the updated quantization model is calculated based on the input features of the backpropagation process and the output features of the last convolution operator, in order to determine whether the updated quantization model has reached the convergence state, i.e., satisfies the second stopping condition.

[0128] If the convergence state is not reached, the updated quantization model is trained again in the second stage until the quantization model reaches the convergence state, thereby obtaining the floating-point activation range corresponding to each convolution operator in the quantization model.

[0129] It is understandable that pseudo-quantized nodes are used for training, and the trained target quantized model includes convolution operators but does not include pseudo-quantized nodes.

[0130] like Figure 5 The diagram shown is a flowchart of a training method for a quantization model in one embodiment.

[0131] Step 510: Insert pseudo-quantization nodes into the floating-point model to obtain the quantization model.

[0132] Step 520: Perform the first stage of training on the quantization model. The first stage of training includes steps 521-525:

[0133] Step 521: Obtain sample data and input the sample data into the quantization model.

[0134] Step 522: Perform forward propagation processing based on sample data, and calculate the initial weight range and initial activation range.

[0135] Step 523: Calculate the first loss error of the quantization model through forward propagation processing, and then proceed to step 524.

[0136] Step 524: Based on the first intermediate result and the first loss error obtained from the forward propagation process, perform backpropagation processing, adjust the initial weight range in the backpropagation process, and obtain the updated quantization model.

[0137] Step 525: Determine whether the updated quantization model has reached convergence. Otherwise, return to step 521. If it has, the first stage of training is completed, and proceed to step 530.

[0138] Step 530: After completing the first stage of training, set the floating-point weight range in the quantized model trained in the first stage to a non-trainable parameter, and set the initial activation range to a trainable parameter. Furthermore, the activation range and weight range are not calculated during the forward propagation in the second stage.

[0139] Step 540: Perform the second stage of training on the quantization model trained in the first stage. The second stage of training includes steps 541-545:

[0140] Step 541: Obtain sample data and input the sample data into the quantization model trained in the first stage.

[0141] Step 542: Perform forward propagation processing based on the sample data to obtain the second intermediate result.

[0142] Step 543: Calculate the second loss error of the quantization model through forward propagation processing, and then proceed to step 544.

[0143] Step 544: Based on the second intermediate result and the second loss error obtained from the forward propagation process, perform backpropagation processing, adjust the initial weight activation range in the backpropagation process, and obtain the updated quantization model.

[0144] Step 545: Determine whether the updated quantization model has reached convergence. Otherwise, return to step 541. If it has, end the second stage of training and obtain the trained target quantization model.

[0145] In one embodiment, the method further includes:

[0146] Obtain the preset weight range and preset activation range corresponding to the preset quantization type; determine the quantization parameters of the target quantization model based on the preset weight range, preset activation range, floating-point weight range, and floating-point activation range; the quantization parameters are used to quantize the data to be quantized into the data corresponding to the preset quantization type.

[0147] The preset quantization type refers to the fixed-point type of the floating-point data to be quantized, such as 8 bits, 12 bits, etc. Different preset quantization types have different preset weight ranges and preset activation ranges. The preset weight range is the fixed-point weight range, and the preset activation range is the fixed-point activation range.

[0148] Specifically, the electronic device acquires the preset weight range and preset activation range corresponding to the preset quantization type, and calculates the quantization parameters of the target quantization model based on the preset weight range, preset activation range, floating-point weight range, and floating-point activation range. When it is necessary to quantize the data to be quantized, the data to be quantized is input into the target quantization model, and the target quantization model processes the data to be quantized using the quantization parameters to obtain the data corresponding to the preset quantization type.

[0149] In one embodiment, the data to be quantized is floating-point data, and the preset quantization type is fixed-point quantization. By quantizing the floating-point data through the quantization parameters in the target quantization model, the floating-point data can be quantized into fixed-point data, and the fixed-point data is under a specific quantization type.

[0150] In one embodiment, the electronic device can acquire preset weight ranges and preset activation ranges corresponding to multiple preset quantization types. Based on the preset weight ranges and preset activation ranges corresponding to the same preset quantization type, as well as the floating-point weight ranges and floating-point activation ranges, the quantization parameters corresponding to that same preset quantization type are determined, and thus the quantization parameters are retained in the target quantization model. Following the same processing method, the quantization parameters corresponding to each preset quantization type can be obtained. When it is necessary to quantize the data to be quantized, it is determined which preset quantization type the data to be quantized should be. Then, the data to be quantized is processed using the corresponding quantization parameters in the target quantization model to obtain fixed-point data under the corresponding preset quantization type.

[0151] In this embodiment, a preset weight range and a preset activation range corresponding to a preset quantization type are obtained; the quantization parameters of the target quantization model are determined based on the preset weight range, the preset activation range, the floating-point weight range, and the floating-point activation range. The quantization parameters can accurately quantize the data to be quantized into the data corresponding to the preset quantization type, thereby accurately mapping floating-point data to fixed-point data of a specific type.

[0152] Figure 6 This is a flowchart of a quantization processing method in one embodiment. The quantization processing method in this embodiment is designed to run on... Figure 1 The description will be based on an example from a Chinese electronic device. Figure 6 As shown, the quantization process includes:

[0153] Step 602: Obtain the quantization type corresponding to the data to be quantized, and obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type.

[0154] In this context, "data to be quantized" refers to floating-point data that requires quantization. "Quantization type" refers to the fixed-point type of the floating-point data to be quantized, such as 8-bit or 12-bit. Different quantization types have different fixed-point weight ranges and fixed-point activation ranges.

[0155] Specifically, the electronic device can acquire the data to be quantized, determine the quantization type that the data to be quantized needs to be quantized, and thus obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type.

[0156] Step 604: Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, determine the weight quantization parameters corresponding to the data to be quantized.

[0157] Specifically, the electronic device can determine the weight quantization parameters corresponding to the data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model.

[0158] In one embodiment, the electronic device can determine the maximum and minimum fixed-point weights within the fixed-point weight range, and the maximum and minimum floating-point weights within the floating-point weight range. Based on the maximum and minimum fixed-point weights, the maximum and minimum floating-point weights, the weight quantization parameters corresponding to the data to be quantized are calculated.

[0159] Step 606: Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, determine the activation quantization parameters corresponding to the data to be quantized; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on the sample data.

[0160] Specifically, the electronic device can determine the activation quantization parameters corresponding to the data to be quantized based on the fixed-point activation range and the floating-point activation range of the trained quantization model.

[0161] In one embodiment, the electronic device can determine the maximum and minimum fixed-point activation values ​​within the fixed-point activation range, and the maximum and minimum floating-point activation values ​​within the floating-point activation range. Based on the maximum and minimum fixed-point activation values, the maximum and minimum floating-point activation values, the activation quantization parameters corresponding to the data to be quantized are calculated.

[0162] Step 608: Based on the weighted quantization parameters and the activation quantization parameters, quantize the data to be quantized into target data under the quantization type.

[0163] The target data is integer data under the quantization type.

[0164] Specifically, the electronic device performs quantization processing on the data to be quantized based on the weight quantization parameters and the activation quantization parameters to obtain the target data under the quantization type.

[0165] In one embodiment, the electronic device can acquire the quantization type corresponding to the data to be quantized, and acquire the fixed-point weight range and fixed-point activation range corresponding to the quantization type. The data to be quantized, the fixed-point weight range, and the fixed-point activation range are input into a trained quantization model. The trained quantization model includes a floating-point weight range and a floating-point activation range. Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the trained quantization model determines the weight quantization parameters corresponding to the data to be quantized. Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the training model determines the activation quantization parameters corresponding to the data to be quantized. Using the weight quantization parameters and the activation quantization parameters, the data to be quantized is quantized, and the target data under the quantization type is output.

[0166] In this embodiment, the quantization type corresponding to the data to be quantized is obtained, along with the fixed-point weight range and fixed-point activation range corresponding to the quantization type. Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the data to be quantized are determined. Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the data to be quantized are determined. The trained quantization model is obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data. Based on the weight quantization parameters and activation quantization parameters, the data to be quantized can be accurately quantized from floating-point to integer target data under the quantization type. Performing operations based on integer data reduces the amount of computation and increases the computation speed, thereby improving the subsequent image processing speed.

[0167] In one embodiment, the weight quantization parameters corresponding to the data to be quantized are determined based on the floating-point weight range and the fixed-point weight range, including:

[0168] Based on the maximum and minimum floating-point weights within the floating-point weight range, and the maximum and minimum fixed-point weights within the fixed-point weight range, determine the weight quantization factor corresponding to the data to be quantized; based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight, determine the weight quantization offset corresponding to the data to be quantized.

[0169] Among them, the weight quantization factor refers to the weight scaling factor for quantizing floating-point data into fixed-point data, and the weight quantization offset, also known as the weight quantization zero-point, refers to the weight value corresponding to the zero point of floating-point data in fixed-point data when floating-point data is quantized into fixed-point data.

[0170] Specifically, the weight quantization parameters include the weight quantization factor and the weight quantization offset. The electronic device can determine the maximum and minimum fixed-point weights within the fixed-point weight range, and the maximum and minimum floating-point weights within the floating-point weight range. A first difference between the maximum and minimum floating-point weights, and a second difference between the maximum and minimum fixed-point weights, are calculated. The weight quantization factor corresponding to the data to be quantized is determined based on the first and second differences. Further, the ratio of the first and second differences is used as the weight quantization factor corresponding to the data to be quantized.

[0171] Calculate the ratio between the minimum floating-point weight and the quantization factor of that weight, and determine the weight quantization offset corresponding to the data to be quantized based on the minimum fixed-point weight and this ratio. Further, use the difference between the minimum fixed-point weight and this ratio as the weight quantization offset corresponding to the data to be quantized.

[0172] For example, the weight quantization factor and weight quantization offset can be calculated using the following formulas:

[0173]

[0174] Where scale is the weight quantization factor, r max r represents the maximum floating-point weight. min q is the minimum floating-point weight. max q represents the maximum value of the fixed-point weight. min is the minimum value of the fixed-point weight, and zero is the weight quantization offset.

[0175] In this embodiment, based on the maximum and minimum floating-point weights within the floating-point weight range, and the maximum and minimum fixed-point weights within the fixed-point weight range, the weight quantization factor corresponding to the data to be quantized can be accurately determined. Based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight, the weight quantization offset corresponding to the data to be quantized can be accurately determined.

[0176] In one embodiment, the activation quantization parameters include an activation quantization factor and an activation quantization offset. Determining the activation quantization parameters corresponding to the data to be quantized based on the floating-point activation range and the fixed-point activation range includes: determining the activation quantization factor corresponding to the data to be quantized based on the maximum and minimum floating-point activation values ​​in the floating-point activation range, and the maximum and minimum fixed-point activation values ​​in the fixed-point activation range; and determining the activation quantization offset corresponding to the data to be quantized based on the activation quantization factor, the minimum floating-point activation value, and the minimum fixed-point activation value.

[0177] Among them, the activation quantization factor refers to the activation scaling factor for quantizing floating-point data into fixed-point data, and the activation quantization offset, also known as the activation quantization zero point, refers to the activation value of the zero point of the floating-point data in the fixed-point data when the floating-point data is quantized into fixed-point data.

[0178] Specifically, the activation quantization parameters include the activation quantization factor and the activation quantization offset. The electronic device can determine the maximum and minimum fixed-point activation values ​​within the fixed-point activation range, and the maximum and minimum floating-point activation values ​​within the floating-point activation range. A third difference between the maximum and minimum floating-point activation values, and a fourth difference between the maximum and minimum fixed-point activation values, are calculated. The activation quantization factor corresponding to the data to be quantized is determined based on the third and fourth differences. Further, the ratio of the third and fourth differences is used as the activation quantization factor corresponding to the data to be quantized.

[0179] Calculate the ratio between the minimum floating-point activation value and the activation quantization factor, and determine the activation quantization offset corresponding to the data to be quantized based on the minimum fixed-point activation value and this ratio. Further, use the difference between the minimum fixed-point activation value and this ratio as the activation quantization offset corresponding to the data to be quantized.

[0180] In one embodiment, based on weighted quantization parameters and activation quantization parameters, the data to be quantized is quantized into target data under the quantization type, including:

[0181] The data to be quantized is processed by convolution based on the weighted quantization parameters, and the result of the convolution is activated by activating the quantization parameters to obtain the target data under the quantization type.

[0182] Specifically, the electronic device performs convolution processing on the data to be quantized based on the weight quantization parameters to obtain the convolution result. Then, it performs activation processing on the convolution result based on the activation quantization parameters to obtain the target data under the specified quantization type.

[0183] In this embodiment, the influence of weight quantization parameters and activation quantization parameters on data quantization is considered. Convolution processing is performed on the data to be quantized based on the weight quantization parameters, and activation processing is performed on the result after convolution processing through activation quantization parameters, so that the target data obtained under the quantization type is more accurate.

[0184] In one embodiment, the data to be quantized is multimedia data to be quantized; the trained quantization model is a quantization model obtained by training the floating-point weight range of multimedia in the first stage and the floating-point activation range of multimedia in the second stage based on the sample multimedia data; the floating-point weight range and the floating-point activation range of multimedia in the trained quantization model are used to quantize the multimedia data to be quantized, and the target data is the target multimedia data obtained by quantization.

[0185] Specifically, the electronic device acquires sample multimedia, converts the multimedia information of the sample multimedia into data information, and obtains sample multimedia data. The quantization model is trained in the first stage using the sample multimedia data to obtain the floating-point weight range of the multimedia in the quantization model. Based on the floating-point weight range of the multimedia and the sample multimedia data, the quantization model trained in the first stage is trained in the second stage, and the floating-point activation range of the multimedia in the trained target quantization model is obtained in the second stage of training.

[0186] The electronic device acquires the multimedia to be quantized and its corresponding quantization type. It then inputs these two data points into a target quantization model. The target quantization model extracts the data information of the multimedia to be quantized, obtaining the multimedia data to be quantized. The target quantization model acquires the fixed-point weight range and fixed-point activation range corresponding to the quantization type. Based on the fixed-point weight range and the floating-point weight range of the multimedia, it determines the weight quantization parameters corresponding to the multimedia data to be quantized. Based on the fixed-point activation range and the floating-point activation range of the multimedia quantization model, it determines the activation quantization parameters corresponding to the multimedia data to be quantized. Based on the weight quantization parameters and the activation quantization parameters, the multimedia data to be quantized is quantized into target multimedia data under the specified quantization type.

[0187] In one embodiment, multimedia recognition, multimedia classification, multimedia segmentation, and other processing can be performed based on the target multimedia data, but it is not limited to these.

[0188] It is understandable that the process of determining the weight quantization parameters corresponding to the multimedia data to be quantized based on the fixed-point weight range and the floating-point weight range of the multimedia, and the process of determining the activation quantization parameters corresponding to the multimedia data to be quantized based on the fixed-point activation range and the floating-point activation range of the multimedia quantization model, are similar to the above process of obtaining the quantization type corresponding to the data to be quantized and obtaining the fixed-point weight range and fixed-point activation range corresponding to the quantization type, and the process of determining the weight quantization parameters corresponding to the data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model.

[0189] It is understandable that when the multimedia data to be quantized is image data, the floating-point weight range of multimedia refers to the floating-point weight range of image, the floating-point activation range of multimedia refers to the floating-point activation range of image, and the target multimedia data refers to the target image data. When the multimedia data to be quantized is text data, the floating-point weight range of multimedia refers to the floating-point weight range of text, the floating-point activation range of multimedia refers to the floating-point activation range of text, and the target multimedia data refers to the target text data.

[0190] In this embodiment, the quantization processing method is applied to the quantization processing of multimedia data. The quantization model is trained in two stages using sample multimedia data. The first stage of training determines the floating-point weight range of multimedia in the quantization model. In the second stage of training, the floating-point activation range of multimedia in the quantization model is determined based on the sample multimedia data and the weight range of multimedia. This results in a well-trained target quantization model for multimedia data quantization, thereby improving the quantization accuracy of the target quantization model and making the quantization of multimedia data more accurate.

[0191] By employing a target quantization model to quantize multimedia data, floating-point data such as images, text, audio, and video can be accurately quantized into integer data. Performing calculations based on integer data reduces computational load and increases processing speed, thereby improving the processing speed of images, text, audio, and video.

[0192] In one embodiment, a method for training a quantization model is provided, comprising:

[0193] The electronic device acquires sample data and quantizes the features output by the current convolution operator in the quantization model through convolution processing to obtain quantized features; the output features of the first convolution operator are obtained by convolution processing the sample data.

[0194] Next, the electronic device performs dequantization on the quantized features to obtain dequantized features. The dequantized features are used as the input to the next convolution operator, and the next convolution operator is used as the current convolution operator. The process returns to the step of quantizing the features output by the current convolution operator through convolution and continues until the last dequantization is performed, at which point the first intermediate result is obtained.

[0195] Next, the electronic device determines the first loss error of the quantization model through forward propagation processing, and statistically determines the initial weight range and initial activation range of the quantization model through forward propagation processing.

[0196] Furthermore, during the backpropagation process of the first intermediate result, the electronic device adjusts the initial weight range based on the first loss error until the quantization model meets the first stopping condition, thus obtaining the floating-point weight range of the quantization model.

[0197] Next, for the second stage of training, the electronic device performs forward propagation processing on the sample data using the floating-point weight range in the quantization model trained in the first stage, and obtains the second intermediate result.

[0198] Furthermore, the electronic device determines the second loss error of the quantization model based on forward propagation processing, and adjusts the initial activation range of the quantization model based on the second loss error during the backpropagation processing of the second intermediate result, until the quantization model stops when it meets the second stopping condition, thus obtaining the floating-point activation range of the trained target quantization model.

[0199] The target quantization model is used to quantize the data to be quantized, including:

[0200] Obtain the quantization type corresponding to the data to be quantized, and input the data to be quantized and the corresponding quantization type into the target quantization model.

[0201] The target quantization model obtains the fixed-point weight range and fixed-point activation range corresponding to the quantization type. Based on the maximum and minimum floating-point weights in the floating-point weight range, and the maximum and minimum fixed-point weights in the fixed-point weight range, it determines the weight quantization factor corresponding to the data to be quantized.

[0202] Next, based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight, the weight quantization offset corresponding to the data to be quantized is determined.

[0203] Furthermore, based on the maximum and minimum floating-point activation values ​​within the floating-point activation range, and the maximum and minimum fixed-point activation values ​​within the fixed-point activation range, the activation quantization factor corresponding to the data to be quantized is determined.

[0204] Next, based on the activation quantization factor, the minimum floating-point activation value, and the minimum fixed-point activation value, the activation quantization offset corresponding to the data to be quantized is determined.

[0205] Furthermore, the data to be quantized is convolved based on the weight quantization parameters, and the result of the convolution is activated by activating the quantization parameters to output the target data under the quantization type.

[0206] In this embodiment, the quantization model is trained in the first stage using sample data to obtain the floating-point weight range of the quantization model. After the first stage of training, with the weight range fixed, the activation range parameter is trained again. Based on the floating-point weight range and sample data, the quantization model trained in the first stage is trained in the second stage. In the second stage of training, the floating-point activation range of the trained target quantization model is obtained. Thus, through different stages of training, the quantization accuracy of the quantization model is improved, and the performance of the quantization model is further enhanced.

[0207] The floating-point weight range and floating-point activation range of the target quantization model enable quantization processing of the data to be quantized, converting floating-point data into integer data. This reduces the amount of computation and improves the speed of quantization processing. Furthermore, quantizing floating-point data into integer data also reduces memory usage.

[0208] It should be understood that, although Figures 2-6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 2-6At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0209] Figure 7 This is a structural block diagram of a training device for a quantization model according to one embodiment. Figure 7 As shown, the device includes:

[0210] The sample acquisition module 702 is used to acquire sample data.

[0211] The first training module 704 is used to train the quantization model in the first stage using sample data to obtain the floating-point weight range of the quantization model.

[0212] The second training module 706 is used to train the quantization model trained in the first stage based on the floating-point weight range and sample data, and to obtain the floating-point activation range of the trained target quantization model in the second stage of training; wherein, the floating-point weight range and floating-point activation range in the target quantization model are used to quantize the data to be quantized.

[0213] In this embodiment, sample data is acquired, and the quantization model is trained in the first stage using this sample data to obtain the floating-point weight range of the quantization model. Based on the floating-point weight range and the sample data, the quantization model trained in the first stage is trained in the second stage, and the floating-point activation range of the trained target quantization model is obtained in the second stage. Thus, through different stages of training, the processing accuracy of the quantization model can be improved. The floating-point weight range and floating-point activation range of the target quantization model can be used to quantize the data to be quantized, converting floating-point data into integer data, reducing the amount of data computation and improving the speed of quantization processing. Furthermore, quantizing floating-point data into integer data also reduces memory usage.

[0214] In one embodiment, the first training module 706 is further configured to perform forward propagation processing on the sample data through the quantization model to obtain a first intermediate result; determine the first loss error of the quantization model through forward propagation processing, and statistically determine the initial weight range of the quantization model through forward propagation processing; and adjust the initial weight range based on the first loss error during the backpropagation processing of the first intermediate result until the quantization model meets the first stopping condition, thereby obtaining the floating-point weight range of the quantization model.

[0215] In this embodiment, for the first stage of training, the sample data is processed by forward propagation through the quantization model to obtain the first intermediate result. The first loss error of the quantization model is determined by forward propagation, and the initial weight range of the quantization model is statistically determined by forward propagation. During the backpropagation process of the first intermediate result, the initial weight range is adjusted based on the first loss error so that the weight range of the quantization model reaches the optimal value. The process continues until the quantization model meets the first stopping condition, thereby accurately obtaining the final floating-point weight range of the quantization model through the first stage of training.

[0216] In one embodiment, the first training module 704 is further configured to quantize the features output by the current convolution operator in the quantization model through convolution processing to obtain quantized features; the output features of the first convolution operator are obtained by convolution processing of sample data; the quantized features are dequantized to obtain dequantized features, the dequantized features are used as the input of the next convolution operator, and the next convolution operator is used as the current convolution operator, the step of quantizing the features output by the current convolution operator through convolution processing is returned and execution continues until the last dequantization processing is performed to obtain the first intermediate result.

[0217] In this embodiment, the features output by the current convolution operator in the quantization model are quantized to obtain quantized features. The quantized features are then dequantized to obtain dequantized features. This allows the error generated after quantization and dequantization of the data to be determined. The dequantized features are used as the input of the next convolution operator, and the next convolution operator is used as the current convolution operator. The process returns to the step of quantizing the features output by the current convolution operator and continues until the last dequantization is performed, resulting in the first intermediate result. This allows the loss error generated during the processing to be accurately obtained, so as to adjust the weight range of the quantization model.

[0218] In one embodiment, the second training module 706 is further configured to perform forward propagation processing on the sample data using the floating-point weight range in the quantization model trained in the first stage, thereby obtaining a second intermediate result; determine the second loss error of the quantization model based on the forward propagation processing, and adjust the initial activation range of the quantization model based on the second loss error during the backpropagation processing of the second intermediate result, until the quantization model stops when it meets the second stopping condition, thereby obtaining the floating-point activation range of the trained target quantization model; wherein, the initial activation range is statistically obtained through forward propagation processing in the first stage of training.

[0219] In this embodiment, for the second stage of training, the sample data is forward-propagated using the floating-point weight range in the quantization model after the first stage of training to obtain a second intermediate result. The second loss error of the quantization model is determined based on the forward propagation process. During the backpropagation process on the second intermediate result, the initial activation range of the quantization model is adjusted based on the second loss error until the quantization model meets the second stopping condition. Thus, the target quantization model and the floating-point activation range in the target quantization model can be accurately obtained through the second stage of training.

[0220] In one embodiment, the device further includes: a quantization parameter determination module; the quantization parameter determination module is used to obtain a preset weight range and a preset activation range corresponding to a preset quantization type; determine the quantization parameters of the target quantization model based on the preset weight range, the preset activation range, the floating-point weight range, and the floating-point activation range; the quantization parameters are used to quantize the data to be quantized into data corresponding to the preset quantization type.

[0221] In this embodiment, a preset weight range and a preset activation range corresponding to a preset quantization type are obtained; the quantization parameters of the target quantization model are determined based on the preset weight range, the preset activation range, the floating-point weight range, and the floating-point activation range. The quantization parameters can accurately quantize the data to be quantized into the data corresponding to the preset quantization type, thereby accurately mapping floating-point data to fixed-point data of a specific type.

[0222] Figure 8 This is a structural block diagram of a quantization processing apparatus according to one embodiment. Figure 8 As shown, the device includes:

[0223] The type acquisition module 802 is used to acquire the quantization type corresponding to the data to be quantized, and to acquire the fixed-point weight range and fixed-point activation range corresponding to the quantization type.

[0224] The weight determination module 804 is used to determine the weight quantization parameters corresponding to the data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model.

[0225] The activation determination module 806 is used to determine the activation quantization parameters corresponding to the data to be quantized based on the fixed-point activation range and the floating-point activation range of the trained quantization model; the trained quantization model is a quantization model obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on the sample data.

[0226] The quantization module 808 is used to quantize the data to be quantized into target data under the quantization type based on the weight quantization parameters and the activation quantization parameters.

[0227] In this embodiment, the quantization type corresponding to the data to be quantized is obtained, along with the fixed-point weight range and fixed-point activation range corresponding to the quantization type. Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the data to be quantized are determined. Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the data to be quantized are determined. The trained quantization model is obtained by training the first stage of floating-point weight range and the second stage of floating-point activation range based on sample data. Based on the weight quantization parameters and activation quantization parameters, the data to be quantized can be accurately quantized from floating-point to integer target data under the quantization type. Performing operations based on integer data reduces the amount of computation and increases the computation speed, thereby improving the subsequent image processing speed.

[0228] In one embodiment, the weight determination module 804 is further configured to determine the weight quantization factor corresponding to the data to be quantized based on the maximum and minimum floating-point weights in the floating-point weight range, and the maximum and minimum fixed-point weights in the fixed-point weight range; and to determine the weight quantization offset corresponding to the data to be quantized based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight.

[0229] In this embodiment, based on the maximum and minimum floating-point weights within the floating-point weight range, and the maximum and minimum fixed-point weights within the fixed-point weight range, the weight quantization factor corresponding to the data to be quantized can be accurately determined. Based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight, the weight quantization offset corresponding to the data to be quantized can be accurately determined.

[0230] In one embodiment, the quantization module 808 is further configured to perform convolution processing on the data to be quantized based on the weighted quantization parameters, and to activate the result after convolution processing by activating the quantization parameters to obtain the target data under the quantization type.

[0231] In this embodiment, the influence of weight quantization parameters and activation quantization parameters on data quantization is considered. Convolution processing is performed on the data to be quantized based on the weight quantization parameters, and activation processing is performed on the result after convolution processing through activation quantization parameters, so that the target data obtained under the quantization type is more accurate.

[0232] In one embodiment, the data to be quantized is multimedia data to be quantized; the trained quantization model is a quantization model obtained by training the floating-point weight range of multimedia in the first stage and the floating-point activation range of multimedia in the second stage based on the sample multimedia data; the floating-point weight range and the floating-point activation range of multimedia in the trained quantization model are used to quantize the multimedia data to be quantized, and the target data is the target multimedia data obtained by quantization.

[0233] In this embodiment, the quantization processing method is applied to the quantization processing of multimedia data. The quantization model is trained in two stages using sample multimedia data. The first stage of training determines the floating-point weight range of multimedia in the quantization model. In the second stage of training, the floating-point activation range of multimedia in the quantization model is determined based on the sample multimedia data and the weight range of multimedia. This results in a well-trained target quantization model for multimedia data quantization, thereby improving the quantization accuracy of the target quantization model and making the quantization of multimedia data more accurate.

[0234] By using a target quantization model to quantize multimedia data, floating-point multimedia data can be accurately quantized into integer data. Performing calculations based on integer data reduces computational load and increases processing speed, thereby improving the overall processing speed of multimedia data.

[0235] The division of the modules in the training device and quantization processing device of the above-mentioned quantization model is only for illustrative purposes. In other embodiments, the training device and quantization processing device of the quantization model can be divided into different modules as needed to complete all or part of the functions of the training device and quantization processing device of the above-mentioned quantization model.

[0236] Specific limitations regarding the training and processing devices for the quantization model can be found in the above description of the training and processing methods for the quantization model, and will not be repeated here. Each module in the aforementioned training and processing devices can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0237] Figure 9This is a schematic diagram of the internal structure of an electronic device in one embodiment. The electronic device can be any terminal device such as a mobile phone, tablet computer, laptop computer, desktop computer, PDA (Personal Digital Assistant), POS (Point of Sales), in-vehicle computer, wearable device, etc. The electronic device includes a processor and a memory connected via a system bus. The processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), etc. The memory may include a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The computer programs can be executed by the processor to implement a training method and a quantization processing method for a quantization model provided in the following embodiments. The internal memory provides a cached runtime environment for the operating system computer programs in the non-volatile storage medium.

[0238] The training device and quantization processing device for the quantization model provided in this application embodiment are implemented in the form of computer programs. These computer programs can run on a terminal or server. The program modules constituted by these computer programs can be stored in the memory of an electronic device. When the computer program is executed by a processor, it implements the steps of the methods described in this application embodiment.

[0239] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform steps of a training method or a quantization processing method for a quantization model.

[0240] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute a training method or a quantization processing method for a quantization model.

[0241] Any references to memory, storage, databases, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or flash memory. Volatile memory may include RAM (Random Access Memory), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), Double Data Rate DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory), ESDRAM (Enhanced Synchronous Dynamic Random Access Memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), and DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0242] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A training method for a quantization model, characterized in that, include: Obtain floating-point sample multimedia data; The sample multimedia data includes at least one of image data, text data, audio data, or video data; For the first stage of training, the sample multimedia data is processed by forward propagation using a quantization model to obtain the first intermediate result; The first loss error of the quantization model is determined through the forward propagation process, and the initial weight range and initial activation range of the quantization model are statistically analyzed through the forward propagation process. During the backpropagation process of the first intermediate result, the initial weight range is adjusted based on the first loss error until the quantization model meets the first stopping condition, thereby obtaining the floating-point weight range of multimedia in the quantization model. For the second stage of training, under the premise that the floating-point weight range obtained from the first stage of training is frozen, the sample multimedia data is forward-propagated through the floating-point weight range of the multimedia in the quantization model after the first stage of training to obtain the second intermediate result. The second loss error of the quantization model is determined based on the forward propagation process, and the initial activation range of the quantization model is adjusted based on the second loss error during the backpropagation process of the second intermediate result until the quantization model stops when it meets the second stopping condition, so as to obtain the floating-point activation range of multimedia in the trained target quantization model. The floating-point weight range and floating-point activation range of the multimedia in the target quantization model are used to quantize the floating-point multimedia data to be quantized into integer multimedia data.

2. The method according to claim 1, characterized in that, The step of performing forward propagation processing on the sample multimedia data using a quantization model to obtain a first intermediate result includes: The features output by the current convolution operator in the quantization model are quantized to obtain quantized features; the output features of the first convolution operator are obtained by convolution processing the sample multimedia data. The quantized features are dequantized to obtain dequantized features. The dequantized features are used as the input of the next convolution operator, and the next convolution operator is used as the current convolution operator. The process returns to the step of quantizing the features output by the current convolution operator through convolution and continues until the last dequantization process is performed to obtain the first intermediate result.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the preset weight range and preset activation range corresponding to the preset quantization type; The quantization parameters of the target quantization model are determined based on the preset weight range, the preset activation range, the floating-point weight range, and the floating-point activation range; the quantization parameters are used to quantize the data to be quantized into the data corresponding to the preset quantization type.

4. A quantization processing method, characterized in that, include: Obtain the quantization type corresponding to the multimedia data to be quantized, and obtain the fixed-point weight range and fixed-point activation range corresponding to the quantization type; The multimedia data to be quantized is floating-point data; Based on the fixed-point weight range and the floating-point weight range of the trained quantization model, the weight quantization parameters corresponding to the multimedia data to be quantized are determined. Based on the fixed-point activation range and the floating-point activation range of the trained quantization model, the activation quantization parameters corresponding to the multimedia data to be quantized are determined; the trained quantization model is a quantization model that obtains the floating-point weight range of multimedia in the first stage and the floating-point activation range of multimedia in the second stage based on the sample multimedia data. The first stage of training includes: for the first stage of training, forward propagation processing is performed on the sample multimedia data using a quantization model to obtain a first intermediate result; the first loss error of the quantization model is determined through the forward propagation processing; and the initial weight range and initial activation range of the quantization model are statistically determined through the forward propagation processing; during the backpropagation processing of the first intermediate result, the initial weight range is adjusted based on the first loss error until the quantization model stops when it meets a first stopping condition, thus obtaining the floating-point weight range of the multimedia in the quantization model. The second stage of training includes: for the second stage of training, under the premise that the floating-point weight range obtained in the first stage of training is frozen, forward propagation processing is performed on the sample multimedia data using the floating-point weight range of the multimedia in the quantization model trained in the first stage to obtain a second intermediate result; the second loss error of the quantization model is determined based on the forward propagation processing; and during the backpropagation processing of the second intermediate result, the initial activation range of the quantization model is adjusted based on the second loss error until the quantization model stops when it meets a second stopping condition, thus obtaining the floating-point activation range of the multimedia in the trained target quantization model. Based on the weight quantization parameter and the activation quantization parameter, the multimedia data to be quantized is quantized into target multimedia data under the quantization type; the target multimedia data is integer data.

5. The method according to claim 4, characterized in that, The step of determining the weight quantization parameters corresponding to the multimedia data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model includes: Based on the maximum and minimum floating-point weights within the floating-point weight range, and the maximum and minimum fixed-point weights within the fixed-point weight range, the weight quantization factor corresponding to the multimedia data to be quantized is determined. The weight quantization offset corresponding to the multimedia data to be quantized is determined based on the weight quantization factor, the minimum floating-point weight, and the minimum fixed-point weight.

6. The method according to claim 4, characterized in that, The step of quantizing the multimedia data to be quantized into target multimedia data under the quantization type based on the weight quantization parameter and the activation quantization parameter includes: The multimedia data to be quantized is convolved based on the weight quantization parameters, and the result of the convolution is activated by the activation quantization parameters to obtain the target multimedia data under the quantization type.

7. A training device for a quantization model, characterized in that, include: The sample acquisition module is used to acquire multimedia data from samples. The sample multimedia data includes at least one of image data, text data, audio data, or video data; The first training module is used for training in the first stage. It performs forward propagation processing on the sample multimedia data through a quantization model to obtain a first intermediate result. It determines the first loss error of the quantization model through the forward propagation processing and statistically analyzes the initial weight range and initial activation range of the quantization model through the forward propagation processing. During the backpropagation processing of the first intermediate result, it adjusts the initial weight range based on the first loss error until the quantization model stops when it meets a first stopping condition, thereby obtaining the floating-point weight range of the multimedia in the quantization model. The second training module is used for training the second stage. Under the premise that the floating-point weight range obtained from the first stage training is frozen, the module performs forward propagation processing on the sample multimedia data using the floating-point weight range of the multimedia in the quantization model after the first stage training to obtain a second intermediate result. Based on the forward propagation processing, the module determines the second loss error of the quantization model. During the backpropagation processing of the second intermediate result, the module adjusts the initial activation range of the quantization model based on the second loss error until the quantization model stops when it meets the second stopping condition, thereby obtaining the floating-point activation range of the multimedia in the trained target quantization model. The floating-point weight range and floating-point activation range of the multimedia in the target quantization model are used to quantize the multimedia data to be quantized.

8. A quantization processing device, characterized in that, include: The type acquisition module is used to acquire the quantization type corresponding to the multimedia data to be quantized, and to acquire the fixed-point weight range and fixed-point activation range corresponding to the quantization type. The weight determination module is used to determine the weight quantization parameters corresponding to the multimedia data to be quantized based on the fixed-point weight range and the floating-point weight range of the trained quantization model. The activation determination module is used to determine the activation quantization parameters corresponding to the multimedia data to be quantized based on the fixed-point activation range and the floating-point activation range of the trained quantization model; the trained quantization model is a quantization model that obtains the floating-point weight range of multimedia in the first stage and the floating-point activation range of multimedia in the second stage based on the sample multimedia data. The first stage of training includes: for the first stage of training, forward propagation processing is performed on the sample multimedia data using a quantization model to obtain a first intermediate result; the first loss error of the quantization model is determined through the forward propagation processing; and the initial weight range and initial activation range of the quantization model are statistically determined through the forward propagation processing; during the backpropagation processing of the first intermediate result, the initial weight range is adjusted based on the first loss error until the quantization model stops when it meets a first stopping condition, thus obtaining the floating-point weight range of the multimedia in the quantization model. The second stage of training includes: for the second stage of training, under the premise that the floating-point weight range obtained in the first stage of training is frozen, forward propagation processing is performed on the sample multimedia data using the floating-point weight range of the multimedia in the quantization model trained in the first stage to obtain a second intermediate result; the second loss error of the quantization model is determined based on the forward propagation processing; and during the backpropagation processing of the second intermediate result, the initial activation range of the quantization model is adjusted based on the second loss error until the quantization model stops when it meets a second stopping condition, thus obtaining the floating-point activation range of the multimedia in the trained target quantization model. The quantization module is used to quantize the multimedia data to be quantized into target multimedia data under the quantization type based on the weight quantization parameter and the activation quantization parameter.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

11. A computer program product having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images