Model quantification method, electronic equipment and computer readable medium

Abstract

The application discloses a model quantization method, comprising: determining a floating point value set of a to-be-quantized parameter of a to-be-quantized layer in a to-be-quantized neural network model, the floating point value set of the to-be-quantized parameter comprising a plurality of to-be-quantized parameter floating point values; determining a supplementary floating point value set and an initial floating point value set according to the truncation value of the to-be-quantized parameter; splicing the supplementary floating point value set and the initial floating point value set in a range of an output channel corresponding to a target deployment platform to obtain a target floating point value set; and quantizing the target floating point value set to obtain a target quantized value set. The invention further provides electronic equipment and a computer readable medium.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular, to a model quantization method, an electronic device, and a computer-readable medium. Background Art

[0002] With the great breakthroughs made by deep learning in image classification, object detection and face recognition, the accuracy of image processing and speech recognition technology has been significantly improved. In addition, deep learning has been widely used in video surveillance, smart home, and the Internet of Things.

[0003] Current neural network models generally require the support of high-performance computer equipment. In order to apply neural network models to low-performance devices such as end-side products, it is necessary to reduce the computational complexity of neural network models. Model quantization is a commonly used neural network model compression technology. Through model quantization, the calculation process can be reduced from floating-point numbers to lower-bit fixed-point operations, thereby greatly improving the speed of the forward reasoning process.

[0004] However, a low-bit quantization model will reduce the accuracy of the result. Therefore, how to improve the accuracy of the output result of the quantization model has become a technical problem to be solved in this field. Summary of the invention

[0005] The present application aims to solve one of the technical problems in the related art to a certain extent. To this end, the present application provides a model quantization method, an electronic device and a computer readable medium.

[0006] As a first aspect of the present application, a model quantization method is provided, wherein the model quantization method comprises:

[0007] Determine a set of floating-point values ​​of parameters to be quantized of a layer to be quantized in a neural network model to be quantized, wherein the set of floating-point values ​​of parameters to be quantized includes a plurality of floating-point values ​​of parameters to be quantized;

[0008] Determining a supplementary floating-point value set and an initial floating-point value set according to the truncated value of the parameter to be quantized, wherein the truncated value is related to a target deployment platform, the initial floating-point value set includes a portion of a plurality of the floating-point values ​​of the parameter to be quantized truncated by the truncated value, and the supplementary floating-point value set includes at least one floating-point value of the parameter to be quantized in a portion of the plurality of floating-point values ​​of the parameter to be quantized truncated by the truncated value;

[0009] splicing the supplementary floating-point value set and the initial floating-point value set within the range of the output channel corresponding to the target deployment platform to obtain a target floating-point value set;

[0010] The target floating-point value set is quantized to obtain a target quantized value set.

[0011] Optionally, the supplementary floating-point value set includes parts of the plurality of floating-point values ​​of the to-be-quantized parameters truncated by the truncation value.

[0012] Optionally, the parameter to be quantized includes a parameter related to an activation value, and the target floating-point value set is represented by the following formula (1):

[0013]

[0014] Wherein, l is the layer number of the layer to be quantized in the neural network model;

[0015] j is the serial number of the output neuron, and j is a variable;

[0016] Cout1 is the total number of output channels corresponding to the target deployment platform;

[0017] Cout2 is the total number of output channels corresponding to the equivalent deployment platform corresponding to the truncation value, wherein the number of hardware bits of the equivalent deployment platform is greater than the number of hardware bits of the target deployment platform, Cout1<Cout2≤2*Cout1;

[0018] is a set of initial floating-point values ​​of the floating-point values ​​of the parameters to be quantized of the j-th output neuron in the l-th neural network layer;

[0019] is a set of supplementary floating-point values ​​including the floating-point value of the parameter to be quantized of the j-cout1th output neuron in the lth neural network layer;

[0020] is the target floating-point value set of the lth neural network layer.

[0021] Optionally, the neural network model to be quantized includes a cascaded first convolution layer, an activation function layer, and a second convolution layer, and the parameter related to the activation value includes a bias value. Before determining a set of floating-point values ​​of a to-be-quantized parameter of a to-be-quantized layer in the neural network model to be quantized, the model quantization method further includes:

[0022] The first convolution layer and the activation function layer are merged to obtain a first layer to be quantized, wherein the parameters to be quantized of the first layer to be quantized include weights to be quantized and bias values ​​to be quantized;

[0023] The second convolutional layer is used as a second layer to be quantized, and the parameters to be quantized of the second layer to be quantized include weights to be quantized.

[0024] Optionally, the target floating-point value set of the weight of the first layer to be quantized is expressed by the following formula (2), and the target floating-point value set of the weight of the second layer to be quantized is expressed by the following formula (3):

[0025]

[0026]

[0027] in, is the target floating-point value set for the weights of the lth neural network layer;

[0028] is the target floating-point value set for the weights of the l+1th neural network layer;

[0029] is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l-th neural network layer;

[0030] is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the lth neural network layer;

[0031] is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l+1-th neural network layer;

[0032] is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the l+1th neural network layer.

[0033] Optionally, before quantizing the target floating-point value set, the model quantization method further includes:

[0034] Determining a quantization step size of the parameter to be quantized;

[0035] Wherein, in the step of quantizing the target floating-point value set, the target floating-point value set is quantized according to the quantization step size.

[0036] Optionally, the parameter to be quantized includes an activation value, and the truncation value satisfies the following formula (4) and formula (5):

[0037] x clip =x u •s x (4)

[0038] x u =(2 a -1)·s x (5)

[0039] Among them, xclip is the cutoff value;

[0040] a is the number of hardware bits of the target deployment platform;

[0041] s x is the quantization step size of the activation value.

[0042] Optionally, before determining the supplementary floating-point value set and the initial floating-point value set according to the truncated value of the parameter to be quantized, the model quantization method further includes:

[0043] Determine a quantization node according to a target deployment platform, wherein the quantization node is used to represent a position that needs to be quantized in a neural network model to be quantized;

[0044] Determine the calibration dataset based on the target deployment scenario.

[0045] Optionally, the model quantization method further includes:

[0046] The neural network model is deployed to the target deployment platform according to the target quantization value set of each layer to be quantized.

[0047] As a second aspect of the present application, an electronic device is provided, wherein the electronic device includes:

[0048] one or more processors;

[0049] A memory having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the model quantization method described in the first aspect of the present application.

[0050] As a third aspect of the present application, a computer-readable medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the model quantization method provided in the first aspect of the present application is implemented.

[0051] In the model quantization method provided in the present application, the part of the floating-point value of the parameter to be quantized that should be truncated during the quantization process is spliced ​​into the range of the output channel corresponding to the target deployment platform, and then the target floating-point value set obtained by splicing is quantized. In the obtained target quantization value set, more information of the neural network model before quantization can be retained without increasing the quantization cutoff value. The quantized neural network model obtained by the model quantization method provided in the present application has a relatively high accuracy. At the same time, even if the number of hardware bits of the target deployment platform is not increased, the above-mentioned "quantized neural network model" with higher accuracy can still be run, reducing the hardware cost of the target deployment platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present application is further described below in conjunction with the accompanying drawings:

[0053] Figure 1 It is a flowchart of an implementation method of the model quantization method provided in this application;

[0054] Figure 2 It is a schematic diagram of the principle of the model quantization method provided in this application;

[0055] Figure 3 is a flow chart of another implementation of the model quantization method provided in this application;

[0056] Figure 4 It is a module schematic diagram of an implementation mode of the electronic device provided by the present application;

[0057] Figure 5 It is a module schematic diagram of the computer-readable medium provided in this application.

[0058] Description of Reference Numerals

[0059] 101: Processor 102: Memory

[0060] 103: I / O interface 104: bus DETAILED DESCRIPTION

[0061] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments based on the embodiments are intended to be used to explain the present application and cannot be understood as limiting the present application.

[0062] References to "one embodiment" or "an example" or "an example" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment itself may be included in at least one embodiment disclosed in the present application. The appearance of the phrase "in one embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0063] Since the quantization bits of the hardware processor used to deploy the model are fixed after leaving the factory, if you want to increase the quantization bits, you need to redesign the processor. At the same time, increasing the quantization bits of the processor will also increase the power consumption and cost of the processor.

[0064] In some related technologies, there is a self-truncation operation when the model is quantized, that is, the activation values ​​of the neural network model that exceed the quantization truncation value will be truncated, which will cause a decrease in quantization performance. If you want to retain more activation values ​​while keeping the quantization step size unchanged, you need to increase the quantization truncation value. One way to increase the quantization truncation value is to increase the number of quantization bits.

[0065] For example, the cascade structure of the three network layers [Conv+ReLU+Conv] can be expressed as:

[0066] y l =W l x l +b l

[0067] x l+1 =f(y l )

[0068] y l+1 =W l+1 x l+1 +b l+1

[0069] Where l is the layer number of the layer to be quantized in the neural network model, where 1≤l≤n;

[0070] y is the output feature map of the neural network model;

[0071] x is the input feature map of the neural network model;

[0072] f(·) is the activation function;

[0073] W is the weight of the network layer;

[0074] b is the bias value of the network layer.

[0075] For the output x of the middle layer l+1 , the activation value quantization corresponds to As shown in the following formula (a):

[0076]

[0077] f(·) represents the ReLU activation function, and formula (a) is converted to formula (b):

[0078]

[0079] Where, clip(·) is the cutoff function;

[0080] s x is the quantization step size of the activation value;

[0081] x l The lower cutoff boundary for activation value quantization;

[0082] x u The upper cutoff boundary for activation value quantization;

[0083] [·] is the nearest neighbor rounding operation.

[0084] The number of bits of hardware deployed on the model is a, then x u =(2 a -1)·s x Due to the truncation operation of the truncation function clip(•), the activation value x l+1 China exceeds x clip =x u •s x The value will be truncated to x clip , which will cause the quantization performance to decrease. At this time, the activation value quantization corresponds to If the hardware increases the quantization bits, the quantization step size remains unchanged, and the cutoff value increases, more activation value features can be retained and the quantization performance can be improved. In this case, the activation value quantization corresponds to: When β=2, the hardware corresponds to a+1 bits, and the activation value quantization corresponds to: In other words, increasing the number of hardware bits can retain more activation values.

[0085] In view of this, as a first aspect of an embodiment of the present application, a model quantization method is provided, wherein: Figure 1 As shown, the model quantization method includes:

[0086] In step S110, a set of floating-point values ​​of parameters to be quantized of a layer to be quantized in a neural network model to be quantized is determined, wherein the set of floating-point values ​​of parameters to be quantized includes a plurality of floating-point values ​​of parameters to be quantized;

[0087] In step S120, a supplementary floating-point value set and an initial floating-point value set are determined according to the truncated value of the parameter to be quantized, wherein the truncated value is related to the target deployment platform, the initial floating-point value set includes a plurality of portions of the floating-point values ​​of the parameter to be quantized truncated by the truncated value, and the supplementary floating-point value set includes at least one floating-point value of the parameter to be quantized in the portion of the plurality of floating-point values ​​of the parameter to be quantized truncated by the truncated value;

[0088] In step S130, the supplementary floating-point value set and the initial floating-point value set are spliced ​​within the range of the output channel corresponding to the target deployment platform to obtain a target floating-point value set;

[0089] In step S140, the target floating-point value set is quantized to obtain a target quantized value set.

[0090] In the model quantization method provided in the embodiment of the present application, the part of the floating-point value of the parameter to be quantized that should be truncated during the quantization process is spliced ​​into the range of the output channel corresponding to the target deployment platform, and then the target floating-point value set obtained by splicing is quantized. In the obtained target quantization value set, more information of the neural network model before quantization can be retained without increasing the quantization cutoff value. The quantized neural network model obtained by the model quantization method provided in the embodiment of the present application has a higher accuracy. At the same time, even if the number of hardware bits of the target deployment platform is not increased, a neural network model with higher accuracy can still be run, reducing the hardware cost of the target deployment platform.

[0091] In the embodiment of the present application, the specific type of the parameter to be quantized is not particularly limited. For example, the parameter to be quantized can be a parameter related to the activation value (for example, the parameter to be quantized can be the activation value directly, or it can be a bias value reflecting the activation value). The quantized neural network model obtained by the model quantization method provided in the embodiment of the present application can retain more activation value information.

[0092] Of course, the embodiments of the present application are not limited thereto. The parameter to be quantized may be any model parameter that needs to be quantized during the process of model quantization. For another example, the parameter to be quantized may be the weight of the layer to be quantized.

[0093] Refer to the following Figure 2 (a) to (f) in the figure provide an exemplary description of the model quantization method provided in the embodiments of the present application.

[0094] Figure 2 (a) to (c) show schematic diagrams of quantizing the [Conv+ReLU+Conv] cascade network layer using the model quantization method in the related art.

[0095] Specifically, Figure 2 (a) is a schematic diagram of the output after the convolution operation through the first convolutional layer; Figure 2 (b) is a schematic diagram of the floating-point activation value obtained by using the activation function layer ReLU to perform the ReLU function operation on the first convolutional layer and taking the positive value; Figure 2 (c) in the figure is the quantization result of quantizing the floating-point activation value output by the ReLU layer.

[0096] Since the hardware bit number of the target deployment platform is limited, the cutoff value x is set clip ,pass Figure 2 As can be seen from (c) in the figure, the cutoff value x clip The floating-point value of the parameter on the right (shown by the dashed line) is truncated, leaving only the truncated value x clip The floating point value of the left-hand argument.

[0097] like Figure 2 As shown in (d) in the figure, the output of the first convolutional layer is copied and then negatively translated by x. clip (See Figure 2 The part filled with diagonal lines in (d) can be truncated and then the activation function layer ReLU can be used to perform ReLU function operation on the first convolution layer to make it positive. The floating point value of the activation value (such as Figure 2 As shown in (e) in the figure, when the activation value is quantized, the activation value can be obtained at the same time. clip The set of quantized values ​​corresponding to the truncated part (such as Figure 2 , as shown in (f) in the figure).

[0098] As described above, the supplementary floating-point value set includes at least one floating-point value of the parameter to be quantized in the portion truncated by the truncation value among the plurality of floating-point values ​​of the parameter to be quantized. In the embodiment of the present application, there is no special limitation on the floating-point value of the parameter to be quantized in the supplementary floating-point value set, as long as more information before quantization of the neural network model can be retained.

[0099] As an optional implementation manner, the supplementary floating-point value set includes parts of the plurality of floating-point values ​​of the parameters to be quantized that are truncated by the truncation value.

[0100] exist Figure 2 It can be seen that when the parameter to be quantized is an activation value, when determining the supplementary floating-point value set, it is necessary to copy the floating-point value set of the parameter to be quantized of the layer to be quantized, and then negatively shift the truncation value. After negatively shifting, the activation function is used to perform activation operation to obtain the corresponding supplementary floating-point value set.

[0101] In the embodiment of the present application, when determining the supplementary floating-point value set, an equivalent deployment platform may be set, the number of hardware bits of the equivalent deployment platform is greater than the number of hardware bits of the target deployment platform, and therefore, the number of output channels Cout2 of the equivalent deployment platform is greater than the number of output channels Cout1 of the target deployment platform (specifically, Cout1<Cout2≤2*Cout1). The first Cout1 output channels in the equivalent deployment platform are the same as the Cout1 output channels of the target deployment platform, and therefore, the floating-point values ​​of the parameters to be quantized in the supplementary floating-point value set correspond to the Cout2-Cout1th channel to the Cout2th channel.

[0102] That is, when the parameter to be quantized is a parameter related to the activation value, the target floating-point value set can be expressed by the following formula (1):

[0103]

[0104] Wherein, l is the layer number of the layer to be quantized in the neural network model;

[0105] j is the serial number of the output neuron, and j is a variable;

[0106] Cout1 is the total number of output channels corresponding to the target deployment platform;

[0107] Cout2 is the total number of output channels corresponding to the equivalent deployment platform corresponding to the truncation value, wherein the number of hardware bits of the equivalent deployment platform is greater than the number of hardware bits of the target deployment platform;

[0108] is a set of initial floating-point values ​​of the floating-point values ​​of the parameters to be quantized of the j-th output neuron in the l-th neural network layer;

[0109] is a set of supplementary floating-point values ​​including the floating-point value of the parameter to be quantized of the j-cout1th output neuron in the lth neural network layer;

[0110] is the target floating-point value set of the lth neural network layer.

[0111] As an optional implementation, Cout2=2*Cout1.

[0112] As mentioned above, the parameter related to the activation value can be the activation value itself or the bias value.

[0113] As an optional implementation, the neural network model to be quantized includes a cascaded first convolution layer, an activation function layer, and a second convolution layer (which can be represented by [Conv+ReLU+Conv]). In this case, the parameters related to the activation value include a bias value.

[0114] Accordingly, before determining the set of floating-point values ​​of the to-be-quantized parameters of the to-be-quantized layer in the to-be-quantized neural network model, the model quantization method further includes:

[0115] The first convolution layer and the activation function layer are merged to obtain a first layer to be quantized, wherein the parameters to be quantized of the first layer to be quantized include weights to be quantized and bias values ​​to be quantized;

[0116] The second convolutional layer is used as a second layer to be quantized, and the parameters to be quantized of the second layer to be quantized include weights to be quantized.

[0117] In the neural network model, the letter "b" can be used to represent the bias value. Accordingly, the target floating-point value set of the bias value can be expressed by the following formula:

[0118]

[0119] in, is an initial set of floating-point values ​​including the floating-point value of the bias value parameter of the j-th output neuron in the l-th neural network layer;

[0120] The set of complementary floating-point values ​​of the bias value parameter floating-point value of the j-cout1-th output neuron in the l-th neural network layer;

[0121] is the target floating-point value set of the lth neural network layer.

[0122] In the embodiment of the present application, the target floating-point value set of the weight of the first layer to be quantized can be expressed by the following formula (2), and the target floating-point value set of the weight of the second layer to be quantized can be expressed by the following formula (3):

[0123]

[0124]

[0125] in, is the target floating-point value set for the weights of the lth neural network layer;

[0126] is the target floating-point value set for the weights of the l+1th neural network layer;

[0127] is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l-th neural network layer;

[0128] is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the lth neural network layer;

[0129] is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l+1-th neural network layer;

[0130] is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the l+1th neural network layer.

[0131] In order to quantize the target floating-point value set, before quantizing the target floating-point value set, the model quantization method further includes:

[0132] Determine a quantization step size of the parameter to be quantized.

[0133] Correspondingly, in the step of quantizing the target floating-point value set, the target floating-point value set is quantized according to the quantization step size.

[0134] The quantization step size can also be understood as the quantization resolution, which is used to measure the accuracy of the target quantization value. Generally, the smaller the quantization step size, the higher the accuracy, and the larger the quantization step size, the lower the accuracy.

[0135] When the parameters to be quantized include activation values ​​and weights, it is necessary to determine the quantization step size of the activation values ​​and the quantization step size of the weights respectively.

[0136] As described above, the cutoff value satisfies the following formula (4) and formula (5):

[0137] x clip =x u •s x (4)

[0138] x u =(2 a -1)·s x (5)

[0139] Among them, x clip is the cutoff value;

[0140] a is the number of hardware bits of the target deployment platform;

[0141] s x is the quantization step size of the activation value.

[0142] It should be noted that when calculating activation value related parameters, it is necessary to use a calibration data set for forward reasoning. Therefore, before determining the supplementary floating-point value set and the initial floating-point value set according to the truncated value of the parameter to be quantized, the model quantization method also includes:

[0143] Determine a quantization node according to a target deployment platform, wherein the quantization node is used to represent a position that needs to be quantized in a neural network model to be quantized;

[0144] Calibrate the dataset according to the target deployment scenario.

[0145] When performing model quantization, the first step is to determine the position in the neural network model to be quantized. The step of determining the quantization node can also be called "inserting the quantization node". For example, two quantization nodes, QuantStub and DeQuantSub, can be inserted during the forward training of the neural network model to specify the position that needs to be quantized.

[0146] like Figure 3As shown, after determining the target quantization value set of each layer to be quantized, the model quantization method further includes:

[0147] In step S150, the neural network model is deployed to the target deployment platform according to the target quantization value set of each layer to be quantized.

[0148] In the embodiments of the present application, there is no special limitation on the target deployment platform. Generally, the target deployment platform can be a terminal device (for example, a mobile phone terminal). By using the model quantization method provided in the embodiments of the present application, high quantization bit operations can be achieved when the number of hardware bits of the terminal device is limited.

[0149] The following describes the specific steps of quantizing the [Conv+ReLU+Conv] cascade network layer using the model quantization method provided in the embodiment of the present application:

[0150] Get the set deployment configuration information, and get the quantization node and quantization mode according to the target deployment platform. As an optional implementation, the target deployment platform can be a mobile phone, a tablet computer, a web camera, etc. The quantization mode is used to limit the granularity of quantization, for example, it can be a per-tensor mode or a per-channel mode. When executing the model quantization method, a quantization mode that matches the target deployment platform can be selected according to the specific situation of the target deployment platform.

[0151] Obtain a neural network model to be quantized and a calibration data set. As an optional implementation, the neural network model to be quantized may be a neural network for classifying images, and accordingly, the calibration data set may be a data set including feature data and label data, wherein the feature data includes a plurality of images carrying categories, and the label data also includes a plurality of images carrying categories.

[0152] The functional layer (e.g., convolutional layer or fully connected layer) and batch normalization (BN) layer in the neural network model to be quantized are fused into one functional layer.

[0153] Weight quantization includes: for the weight parameters of each layer to be quantized in the neural network model to be quantized, obtaining a floating-point weight set and a quantized weight set according to different weight cutoff values, calculating a weight quantization error set; and obtaining an initialized weight quantization step size according to the weight quantization error set.

[0154] The activation value quantization includes: using the calibration data set to perform model forward inference, obtaining the output activation value of each layer to be quantized in the network model to be quantized, obtaining a floating point activation value set and a quantized activation value set according to different activation value cutoff values, and calculating an activation value quantization error set; and obtaining an initialized activation value quantization step size and a cutoff value x according to the activation value quantization error set. clip .

[0155] For the [Conv+ReLU+Conv] cascade network layer in the model, for the first convolution layer, c out is the number of output channels. The original weight W l Become the new weight W l ′, if 0≤j≤c out , if c out ≤j≤2c out , The original bias value b l Become the new bias value b l′ , if 0≤j≤c out , if c out ≤j≤2c out ,

[0156] Corresponding to the second convolutional layer, the original weight W l+1 becomes W l+1′ , if 0≤i≤c out , if c out ≤i≤2c out ,

[0157] According to the quantization parameters in the quantization model, the integer weights, weight quantization step size, and activation value quantization step size of the fixed-point inference model are obtained. These quantization parameters deploy the model to the target deployment platform.

[0158] In the embodiments of the present application, there is no special limitation on the specific type of the target deployment platform, as long as it is a device that needs to use a neural network model to process various physical information such as pictures, texts, videos, and voices. As an optional implementation, the target deployment platform can be an end-side device as mentioned above. For example, the target deployment platform can be a mobile phone, a tablet computer, a web camera, etc., and the quantized neural network model can be used to process various physical information such as pictures, texts, videos, and voices mentioned above.

[0159] As a second aspect of the embodiments of the present application, an electronic device is provided, wherein: Figure 4 As shown, the electronic device includes:

[0160] One or more processors 101;

[0161] The memory 102 stores one or more computer programs. When the one or more computer programs are executed by the one or more processors 101, the one or more processors 101 implement the model quantization method provided in the first aspect of the embodiment of the present application.

[0162] The electronic device may further include one or more I / O interfaces 103 connected between the processor 101 and the memory 102 and configured to implement information interaction between the processor 101 and the memory 102 .

[0163] Among them, the processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, and can realize information exchange between the processor and the memory, including but not limited to a data bus (Bus), etc.

[0164] In some embodiments, the processor 101 , the memory 102 , and the I / O interface 103 are connected to each other via a bus 104 , and further connected to other components of the computing device.

[0165] As a third aspect of the embodiment of the present application, Figure 5 As shown, a computer-readable medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the model quantization method described in the first aspect of the embodiment of the present application is implemented.

[0166] It can be understood by a person of ordinary skill in the art that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. Accordingly, the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can implement the method of any of the above-mentioned embodiments when executed. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the embodiments of the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0167] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Those skilled in the art should understand that the present application includes but is not limited to the contents described in the drawings and the above specific embodiments. Any modification that does not deviate from the functional and structural principles of the present application will be included in the scope of the claims.

Claims

1. A model quantization method, characterized in that: The model quantization method comprises: Determine a set of floating-point values ​​of parameters to be quantized of a layer to be quantized in a neural network model to be quantized, wherein the set of floating-point values ​​of parameters to be quantized includes a plurality of floating-point values ​​of parameters to be quantized; Determining a supplementary floating-point value set and an initial floating-point value set according to the truncated value of the parameter to be quantized, wherein the truncated value is related to a target deployment platform, the initial floating-point value set includes a portion of a plurality of the floating-point values ​​of the parameter to be quantized truncated by the truncated value, and the supplementary floating-point value set includes at least one floating-point value of the parameter to be quantized in a portion of the plurality of floating-point values ​​of the parameter to be quantized truncated by the truncated value; splicing the supplementary floating-point value set and the initial floating-point value set within the range of the output channel corresponding to the target deployment platform to obtain a target floating-point value set; The target floating-point value set is quantized to obtain a target quantized value set.

2. The model quantization method according to claim 1, characterized in that: The supplementary floating-point value set includes parts of the plurality of floating-point values ​​of the to-be-quantized parameters truncated by the truncation value.

3. The model quantization method according to claim 2, characterized in that: The parameters to be quantized include parameters related to the activation value, and the target floating-point value set is expressed by the following formula (1): Wherein, l is the layer number of the layer to be quantized in the neural network model; j is the serial number of the output neuron, and j is a variable; Cout1 is the total number of output channels corresponding to the target deployment platform; Cout2 is the total number of output channels corresponding to the equivalent deployment platform corresponding to the truncation value, wherein the number of hardware bits of the equivalent deployment platform is greater than the number of hardware bits of the target deployment platform, Cout1<Cout2≤2*Cout1; is a set of initial floating-point values ​​of the floating-point values ​​of the parameters to be quantized of the j-th output neuron in the l-th neural network layer; is a set of supplementary floating-point values ​​including the floating-point value of the parameter to be quantized of the j-cout1th output neuron in the lth neural network layer; ′ is the target floating point value set of the lth neural network layer.

4. The model quantization method according to claim 3, characterized in that: The neural network model to be quantized includes a cascaded first convolution layer, an activation function layer, and a second convolution layer, and the parameters related to the activation value include a bias value. Before determining the floating-point value set of the to-be-quantized parameter of the to-be-quantized layer in the neural network model to be quantized, the model quantization method further includes: The first convolution layer and the activation function layer are merged to obtain a first layer to be quantized, wherein the parameters to be quantized of the first layer to be quantized include weights to be quantized and bias values ​​to be quantized; The second convolutional layer is used as a second layer to be quantized, and the parameters to be quantized of the second layer to be quantized include weights to be quantized.

5. The model quantization method according to claim 4, characterized in that: The target floating-point value set of the weight of the first layer to be quantized is expressed by the following formula (2), and the target floating-point value set of the weight of the second layer to be quantized is expressed by the following formula (3): in, is the target floating-point value set for the weights of the lth neural network layer; is the target floating-point value set for the weights of the l+1th neural network layer; is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l-th neural network layer; is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the lth neural network layer; is an initial set of floating-point values ​​comprising the floating-point value of the weight of the j-th output neuron in the l+1-th neural network layer; is a set of complementary floating-point values ​​comprising the weights of the j-cout1th output neuron in the l+1th neural network layer.

6. The model quantization method according to any one of claims 1 to 5, characterized in that: Before quantizing the target floating-point value set, the model quantization method further includes: Determining a quantization step size of the parameter to be quantized; Wherein, in the step of quantizing the target floating-point value set, the target floating-point value set is quantized according to the quantization step size.

7. The model quantization method according to claim 6, characterized in that: The parameter to be quantized includes an activation value, and the truncation value satisfies the following formula (4) and formula (5): x clip =x u ·s x (4) x u =(2 a -1)·s x (5) Among them, x clip is the cutoff value; a is the number of hardware bits of the target deployment platform; s x is the quantization step size of the activation value.

8. The model quantization method according to any one of claims 1 to 5, characterized in that: Before determining the supplementary floating-point value set and the initial floating-point value set according to the truncated value of the parameter to be quantized, the model quantization method further includes: Determine a quantization node according to a target deployment platform, wherein the quantization node is used to represent a position that needs to be quantized in a neural network model to be quantized; Determine the calibration dataset based on the target deployment scenario.

9. The model quantization method according to any one of claims 1 to 5, characterized in that: The model quantization method further includes: The neural network model is deployed to the target deployment platform according to the target quantization value set of each layer to be quantized.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory having one or more computer programs stored thereon, wherein when the one or more computer programs are executed by the one or more processors, the one or more processors implement the model quantization method according to any one of claims 1 to 9.

Images