A high-precision quantization method and system for a concat operator-oriented neural network model

Abstract

The application discloses a high-precision quantization method and system for a neural network model facing a concat operator, and a fast equivalent transformation is performed on the model according to the method before quantization, so that the concat model is replaced by an Eltwise model, and the calculation result of the model before quantization is not affected. The quantization of the concat model needs to set each input branch to the same quantization parameter, and when the data ranges of the input branches are quite different, the consistent quantization parameter will bring great precision loss, and when the Eltwise model is quantized, although the quantization parameters of each input branch of the Eltwise model need to be consistent, since the Eltwise model performs an addition operation within a channel, the quantization loss caused by the difference between the channels can be effectively avoided, so that the quantization of the equivalent transformed model can effectively avoid the problem of great precision loss caused by the difference between the channels when the concat model is quantized.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of neural network model quantization, and relates to a high-precision quantization method and system for a concat operator-oriented neural network model. BACKGROUND

[0002] Computing resources are limited on embedded devices such as edge computing and end-side computing. An original neural network model generally adopts floating-point precision for calculation and numerical value storage. In order to enable the neural network model to run on a resource-limited computing platform, improve inference calculation performance, and realize high-speed operation, it is necessary to quantize the floating-point neural network model into an integer-type neural network, thereby greatly reducing the storage space, calculation amount and energy consumption of the neural network model. Quantization is the key to realizing high-speed and low-power operation of the neural network model on a resource-limited computing platform. Since quantization is to convert a high-precision floating-point neural network into a low-bit-width integer neural network for storage and calculation, it inevitably causes a loss of calculation precision. The main technical problem faced is how to minimize the decrease in calculation precision caused by the decrease in numerical value bit width. Concat is a commonly used operator of a neural network model and has the important function of feature fusion. Since the role of Concat is to splice each input branch together, the quantization parameters of each input branch must be consistent. When the data ranges of each input branch are quite different, the consistent quantization parameters will cause a large loss of quantization precision of each input branch, resulting in a serious decrease in the performance of the neural network. Moreover, the Concat operator is widely used in current neural network models, and the problem of large loss of quantization precision of the Concat operator has seriously affected the application of the neural network model on embedded devices such as edge computing and end-side computing. Therefore, it is of important practical value to solve the problem of poor quantization precision of the Concat operator. SUMMARY

[0003] The application aims to solve the problem of large loss of quantization precision of the Concat operator in the neural network model in the prior art and provide a high-precision quantization method and system for a concat operator-oriented neural network model.

[0004] To achieve the above objective, the application adopts the following technical solutions:

[0005] The high-precision quantization method for a concat operator-oriented neural network model provided by the application comprises the following steps:

[0006] The Concat model is equivalently replaced by an Eltwise model;

[0007] The input branches of the Eltwise model are sequentially subjected to quantization, convolution and dequantization operations to obtain an integer-type input of the Eltwise operator.

[0008] The Eltwise operator is operated to obtain the output of the Eltwise model, and high-precision quantization of the neural network model is realized.

[0009] Preferably, the process of equivalently replacing the Concat model into the Eltwise model is as follows:

[0010] The Concat operator in the Concat model is replaced by the Eltwise operator;

[0011] The Convolution in the Concat model is equivalently replaced by multiple Convolution.

[0012] Preferably, each input branch of the Eltwise model is consistent with the output dimension.

[0013] Preferably, the convolution weight of the Eltwise model is obtained by splitting the convolution weight of the Concat model.

[0014] Preferably, the Eltwise operator is dequantized to obtain the output of the Eltwise model.

[0015] Preferably, the measurement method of the precision quantization loss Q is as follows:

[0016]

[0017] Wherein, Q1 is the floating point type result corresponding to the output of the Eltwise model after quantization, and Q2 is the floating point type result compared with the unquantized result.

[0018] Preferably, the Yolo-v3 tiny model is used to verify the precision quantization loss result.

[0019] The present application provides a neural network model high-precision quantization system for concat operator, comprising:

[0020] The model equivalent replacement module is used for equivalently replacing the Concat model into the Eltwise model;

[0021] The input parameter acquisition module is used for sequentially quantizing, convoluting and dequantizing each input branch of the Eltwise model to obtain the integer type input of the Eltwise operator.

[0022] The output parameter acquisition module is used for operating the Eltwise operator to obtain the output of the Eltwise model, and realizing high-precision quantization of the neural network model.

[0023] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the high-precision quantization method for a concat operator-oriented neural network model when executing the computer program.

[0024] A computer-readable storage medium stores a computer program, and the computer program implements the steps of the high-precision quantization method for a concat operator-oriented neural network model when executed by a processor.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] The high-precision quantization method for a concat operator-oriented neural network model proposed by the present application, on the one hand, performs a fast equivalent transformation on the model before quantization according to the method proposed by the present application, and replaces the Concat model with the Eltwise model, which does not affect the calculation result of the model before quantization. On the other hand, the quantization of the Concat model requires setting each input branch to the same quantization parameter, and when the data ranges of each input branch are quite different, a huge precision loss will be caused due to the consistent quantization parameter. However, when the Eltwise model is quantized, although the quantization parameters of each input branch of the Eltwise model also need to be consistent, since the Eltwise model performs an addition operation within the channel, the quantization loss caused by the difference between the channels can be effectively avoided. Therefore, quantizing the equivalent transformed model can effectively avoid the problem of large precision loss caused by the difference between the channels when quantizing the Concat model, and is conducive to the rapid and convenient deployment and implementation of the neural network model on the computing device with limited computing resources.

[0027] The high-precision quantization system for a concat operator-oriented neural network model proposed by the present application realizes high-precision quantization of the neural network model by dividing the system into a model equivalent replacement module, a model input acquisition module and a model output acquisition module. The modular idea makes each module independent of each other, facilitating unified management of each module. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0029] Figure 1 The flowchart of the high-precision quantization method for a concat operator-oriented neural network model of the present application.

[0030] Figure 2 A high-precision quantization method for a concat operator in a neural network model of the application.

[0031] Figure 3 A high-precision quantization system for a concat operator-oriented neural network model of the application. DETAILED DESCRIPTION

[0032] To make the objects, technical solutions and advantages of the embodiments of the application clearer, the following will be combined with the drawings of the embodiments of the application to clearly and completely describe the technical solutions in the embodiments of the application. Obviously, the described embodiments are some but not all of the embodiments of the application. The components of the embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations.

[0033] Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the application without creative labor are within the scope of protection of the application.

[0034] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0035] In the description of the embodiments of the application, it should be noted that if the terms "upper", "lower", "horizontal", "inner" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the application is usually placed, only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation on the application. In addition, the terms "first", "second" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0036] The application will be further described in detail below in combination with the drawings:

[0037] The application proposes a high-precision quantization method for a concat operator-oriented neural network model, as shown in Figure 1 The method comprises the following steps:

[0038] S1, replacing the Concat model into an Eltwise model;

[0039] The process of replacing the Concat model into the Eltwise model is as follows:

[0040] replace the Concat operator in the Concat model with an Eltwise operator;

[0041] replace one Convolution in the Concat model with multiple Convolution.

[0042] The input branches of the Eltwise model are consistent with the output dimensions.

[0043] The convolution weights of the Eltwise model are obtained by splitting the convolution weights of the Concat model.

[0044] S2, sequentially quantize, convolve and dequantize each input branch of the Eltwise model to obtain the integer input of the Eltwise operator;

[0045] S3, perform operation on the Eltwise operator to obtain the output of the Eltwise model, and realize high-precision quantization of the neural network model.

[0046] dequantize the Eltwise operator to obtain the output of the Eltwise model.

[0047] The precision quantization loss Q is measured as follows:

[0048]

[0049] Wherein, Q1 is the floating point result corresponding to the output of the Eltwise model after quantization, and Q2 is the floating point result compared with the unquantized result.

[0050] The Yolo-v3 tiny model is used to verify the precision quantization loss result.

[0051] The application provides a high-precision quantization method for a neural network model facing a concat operator, which specifically includes a structure design method for equivalent replacement of Concat and a weight splitting method.

[0052] Concat is a commonly used operator of a neural network model, and has the important function of feature fusion. The function of Concat is to concatenate each input branch in order, as shown in the following formula:

[0053]

[0054] Wherein, is the input of each branch of the Concat operator, n is the number of branches of the Concat operator input, Y C,H,WThe output of the Concat operator. The size of the output and the feature map of each branch of the input is consistent, and C and H represent the size of the feature map. The number of channels of the output is the sum of the number of channels of each input, that is, W = W1 + W2 + … + Wn n .

[0055] The Eltwise(sum) operator is to add each input corresponding position to obtain the output result, as shown in the following formula:

[0056] Y i,j,k = X(1) i,j,k + … + X(n) i,j,k (2)

[0057] Wherein, X(1), X(2), … X(n) is each branch input of the Eltwise(sum) operator, Y is the output of the Eltwise(sum) operator, and n is the number of branches of the Eltwise(sum) operator input. C, H, W represent the number of channels and the height and width of the feature map respectively, i ∈ [1, C], j ∈ [1, H], k ∈ [1, W]. The dimension of Y is consistent with X(1), X(2), … X(n).

[0058] Since the Concat operator is to splice each input, it does not involve data operation, so the quantization parameters of each branch input and output must be consistent, that is, all inputs and outputs must be quantized by using the same data truncation range, therefore, when the data distribution range of each branch is quite different, it will bring a large loss of quantization precision. Generally speaking, the Concat operator is followed by the Convolution operator, in order to avoid the precision loss caused by quantizing the Concat operator, the application designs a method of replacing the Concat structure with an equivalent Eltwise structure, that is, replacing “Concat + Convolution” with “multiple Convolution + Eltwise”, and gives the splitting method of Convolution weight.

[0059] As Figure 2 shown in the upper left corner, the structure of the original model is “Concat + Convolution”, the input of Concat is X(1) and X(2), and the weight of Convolution is W i,l,m,m . C out , C in , S respectively represent the output channel number, input channel number and convolution kernel size of the convolution, i ∈ [1, C out ], l ∈ [1, C in ], m ∈ [1, S]. When quantizing this structure, as Figure 2As shown in the lower left, it is necessary to comprehensively consider the data distribution of X(1) and X(2), select a unified quantization parameter to quantize X(1) and X(2), then concatenate them, perform a Convolution operation after concatenation, and finally obtain the output through dequantization. This invention replaces the original model's "Concat + Convolution" structure with an equivalent "multiple Convolution + Eltwise" structure, such as... Figure 2 As shown in the upper right corner, after performing Convolution operations on X(1) and X(2) respectively, the output is obtained through Eltwise(sum). The weight of Convolution1 after X(1) is... C in1 Let X(1) represent the number of channels, i∈[1,C]. out ],l1∈[1,C in1 ],m∈[1,S]. The weights of Convolution2 following X(2) are C in2 Let X(2) represent the number of channels, i∈[1,C]. out ],l2∈[1,C in2 ], m∈[1,S]. That is, the original model's Convolution is split into a corresponding number of Convolutions according to the number of input branches, as shown in the following equation:

[0060]

[0061] C in1 C in2 And C in1 The following relationship must be satisfied:

[0062] C in1 +C in2 =C in (4)

[0063] When quantizing the equivalent substitution model, such as Figure 2 As shown in the lower right corner, X(1) and X(2) are first quantized, and then Convolution and dequantization operations are performed respectively. Finally, the output is obtained through Eltwise(sum) and dequantization. Although the inputs of Eltwise also need to have consistent quantization parameters, since it performs the addition operation within the channel, it can effectively avoid the quantization loss caused by the differences between channels, and effectively improve the performance of the neural network.

[0064] High-precision quantization methods for the Concat operator in neural network models, such as Figure 2 As shown, Figure 2The original model structure shown in the upper left is equivalently replaced by the Eltwise structure shown in the upper right, and when the model after the equivalent replacement is quantized, as shown in the lower left Figure 2 As shown in the lower right, the precision loss problem caused by the difference in data distribution of different channels can be avoided by the quantized Concat, and the performance of deploying the neural network model on embedded devices such as edge side and edge computing can be effectively improved.

[0065] Embodiment 1

[0066] Figure 2 In the original model structure in the upper left, it is assumed that two inputs X(1) are X(2) is The convolution kernel size of Convolution is 1x2x2x2, and the weight W is C out = 1, C in = 2, S = 2, pad = 0, stride = 1. The floating point calculation result of the original model without quantization is 170.2187.

[0067] According to the method provided by the application, the model is equivalently replaced by Figure 2 The model in the upper right. The convolution kernel of Convolution is split into the convolution kernels of Convolution1 and Convolution2 according to formula (3), then the convolution kernel size of Convolution1 is 1x1x2x2, and the weight value is C out = 1, C in1 = 1, S = 2, pad = 0, stride = 1; the convolution kernel size of Convolution2 is 1x1x2x2, and the weight value is C out = 1, C in2 = 1, S = 2, pad = 0, stride = 1. The floating point calculation result of the model after equivalent replacement without quantization is 170.2187.

[0068] As can be seen from the above calculation process, according to the method proposed by the application, the model is equivalently transformed, the Concat operator is replaced by the Eltwise operator, and the floating point calculation result of the model after equivalent replacement without quantization is consistent with the original model, which will not affect the calculation result of the model before quantization.

[0069] The precision loss of the quantized model is compared as follows. The original model in the upper left is quantized, as shown in the lower left Figure 2 The input X(1) and X(2) take the same quantization parameters, X(1) is quantized as Figure 2 X(1) is quantized, and the corresponding floating point true value is X(1) is quantized, and the corresponding floating point true value is (2) is quantized as The true floating-point value corresponding to X(2) after quantization is The weight quantization of Convolution is The floating-point true value corresponding to the weight quantization is The original model, after quantization, outputs a floating-point calculation result of 182.9434, compared to the unquantized floating-point calculation result of 170.2187. The quantization precision loss is [missing value].

[0070] right Figure 2 The equivalent substitution model in the upper right corner is quantized, such as... Figure 2 As shown in the lower right corner, inputs X(1) and X(2) are given with different quantization parameters. X(1) is quantized as... The true floating-point value corresponding to X(1) after quantization is X(2) is quantized as The true floating-point value corresponding to X(2) after quantization is The weights of Convolution1 are quantized as follows The true floating-point value corresponding to the weight quantization of Convolution1 is... Convolution2's weight quantization is The true floating-point value corresponding to the weight quantization of Convolution2 is The equivalent replacement model, after quantization, outputs a floating-point calculation result of 170.3589, compared to the unquantized floating-point calculation result of 170.2187. The quantization precision loss is [missing value].

[0071] As can be seen from the above calculation process, the model after equivalent substitution suffers less loss in quantization accuracy. By performing equivalent transformation on the model according to the method proposed in this invention and then deploying it on the edge device, the quantization accuracy of the neural network model can be effectively improved, and the application of the neural network model on resource-constrained edge devices can be promoted.

[0072] Example 2

[0073] The small target detection model has relatively high precision requirement for quantization, and the small target detection model Yolo-v3tiny is used to verify the practicability of the application. The data set is HRSID, which is a data set for ship detection, semantic segmentation and instance segmentation tasks in high-resolution SAR images. The data set contains 5604 SAR images with resolutions of 0.5 m, 1 m and 3 m. The original model and the equivalent replacement model obtained by using the application are quantized to 8-bit integers and deployed on a neural network accelerator, and the calculation speed and calculation accuracy are tested. The test index used is the precision index mAP (Mean Average Precision) and the speed index FPS. The calculation method of mAP is the same as that of the international famous target detection competition PASCAL VOC Challenge. The test results are shown in the following table. For the small target detection model, the high-precision quantization method of the concat operator in the neural network model proposed by the application can improve the detection accuracy by 13.78% without affecting the calculation speed of the embedded end.

[0074] Table 1: Calculation speed and calculation accuracy test results of small target detection application

[0075]

[0076] The high-precision quantization system for the concat operator-oriented neural network model proposed by the application, as shown in Figure 3 , includes a model equivalent replacement module, a model input acquisition module and a model output acquisition module.

[0077] The model equivalent replacement module is used to replace the Concat model into the Eltwise model.

[0078] The input parameter acquisition module is used to sequentially perform quantization, convolution and dequantization operations on each input branch of the Eltwise model to obtain the integer input of the Eltwise operator.

[0079] The output parameter acquisition module is used to operate the Eltwise operator to obtain the output of the Eltwise model, and realize high-precision quantization of the neural network model.

[0080] The terminal device provided by the embodiment of the application includes a processor, a memory and a computer program stored in the memory and executable on the processor. The processor executes the computer program to realize the steps in each method embodiment. Alternatively, the processor executes the computer program to realize the functions of each module / unit in each device embodiment.

[0081] The computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application.

[0082] The terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal device can include, but is not limited to, a processor, a memory.

[0083] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, and the like.

[0084] The memory can be used to store the computer program and / or modules. The processor realizes various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory.

[0085] The modules / units integrated in the terminal device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can realize the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0086] This invention proposes a high-precision quantization method for neural network models using the concat operator. The concat operator is widely used in current neural network models, but its significant quantization accuracy loss is a bottleneck restricting the application of neural network models on embedded devices such as edge computing. This invention proposes a high-precision quantization method for the concat operator in neural network models, effectively replacing the concat operator with an Eltwise operator. Quantizing the equivalent replacement model effectively reduces quantization accuracy loss and promotes the application of neural network models in resource-constrained embedded devices. Firstly, performing a fast equivalent transformation on the model according to the method proposed in this invention before quantization, replacing the concat operator with an Eltwise operator, does not affect the computational results of the model before quantization. On the other hand, quantization of the Concat operator requires setting the same quantization parameters for each input branch. When the data ranges of each input branch differ significantly, the consistent quantization parameters will lead to a huge loss of accuracy. While quantization of the Eltwise operator also requires consistent quantization parameters for each input branch, it effectively avoids the quantization loss caused by the differences between channels because it performs an intra-channel addition operation. Therefore, quantizing the equivalent transformation model and replacing the quantization of the multi-channel concatenated Concat operator with the addition-based Eltwise operator can effectively avoid the problem of large accuracy loss caused by the differences between channels when quantizing Concat, which is conducive to the rapid and convenient deployment of neural network models in embedded devices.

[0087] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for high-precision quantization of a neural network model oriented to a concat operator, characterized in that, The method comprises the following steps: The Concat model is equivalent to the Eltwise model; Each input branch of the Eltwise model is sequentially quantized, convoluted and dequantized to obtain integer inputs of the Eltwise operator; The Eltwise operator is operated to obtain the output of the Eltwise model, and high-precision quantization of the neural network model is realized; The quantized neural network model is used for image processing; The process of equivalent replacement of the Concat model into the Eltwise model is as follows: The Concat operator in the Concat model is replaced by the Eltwise operator; One Convolution in the Concat model is equivalent to multiple Convolutions.

2. The method of claim 1, wherein the method is applied to a concat operator. The input branches of the Eltwise model are consistent with the output dimensions; the convolution weights of the Eltwise model are obtained by splitting the convolution weights of the Concat model.

3. The method of claim 1, wherein the method is applied to a concat operator. precision quantization loss The metric method is as follows: wherein, is the corresponding float type result of the output of the quantized Eltwise model, is the float type result of the contrastive unquantized.

4. The method of claim 3, wherein the method is applied to a concat operator. The Eltwise operator is dequantized to obtain the output of the Eltwise model.

5. A high-precision quantization system for a neural network model oriented to a concat operator, characterized in that, The Yolo-v3 tiny model is used to verify the precision quantization loss result. The method of any one of claims 1-4 comprises: A model equivalent replacement module is configured to equivalent replace the Concat model into the Eltwise model; An input parameter acquisition module is configured to sequentially quantize, convolute and dequantize each input branch of the Eltwise model to obtain integer inputs of the Eltwise operator; 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. An output parameter acquisition module is configured to operate the Eltwise operator to obtain the output of the Eltwise model and realize high-precision quantization of the neural network model.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The processor executes the computer program to realize the steps of the high-precision quantization method of the neural network model facing the concat operator according to any one of claims 1-4. The computer program is executed by the processor to realize the steps of the high-precision quantization method of the neural network model facing the concat operator according to any one of claims 1-4.

Images