Image-based target detection method and device

Abstract

The invention discloses an image-based target detection method and device, and the method comprises the steps: carrying out the fixed-point reasoning of to-be-detected image data through a trained convolution neural network model for image target detection, and obtaining a reasoning result, and converting the reasoning result according to a channel stretching scale which enables the weight of the network layer and the feature data of the sample image output by the network layer to be balanced in a model quantization process. The quantitative model is obtained by inputting sample image data into a pseudo-quantitative model for reasoning, counting sample image feature data output by each network layer and weight data distribution of a convolutional layer, and according to the counted data distribution, obtaining the quantitative model by stretching the scale of the sample image feature data of each channel in each network layer. And balancing the weight data and the sample image feature data. And according to the statistical data, determining a quantization parameter, and performing fixed-point processing on the equalized weight and the sample image feature data by using the quantization parameter. The target detection speed is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular, to an image-based target detection method and device. Background Art

[0002] With the development of artificial intelligence technology, more and more image-based object detection is implemented using convolutional neural network models.

[0003] In order to improve the speed of image-based object detection, it becomes a requirement to increase the inference speed of convolutional neural network models. Summary of the invention

[0004] The present invention provides an image-based target detection method, which is conducive to improving the target detection speed.

[0005] The first aspect of the present application provides an image-based target detection method, the method comprising:

[0006] Using the convolutional neural network model for image target detection that has been trained to convergence and its quantized model, fixed-point inference is performed on the image data to be detected to obtain an inference result, which is a fixed-point inference result.

[0007] According to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model, the inference result is converted and inferred to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model.

[0008] in,

[0009] The quantitative model is obtained as follows:

[0010] The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0011] The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted.

[0012] According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data.

[0013] determining the quantization parameter used for quantization,

[0014] Quantize the equalized weight data and sample image feature data using quantization parameters respectively;

[0015] The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer used for stretching.

[0016] Preferably, the sample image data does not include label information of the sample target;

[0017] The converting the floating-point operator in the floating-point model of the convolutional neural network model into a pseudo-quantization operator includes:

[0018] Convert the floating-point operator itself into the corresponding pseudo-quantization operator,

[0019] Keep the operator parameters of the floating-point operator unchanged and use them as the operator parameters of the pseudo-quantization operator;

[0020] The step of quantizing the equalized weight data and the sample image feature data using quantization parameters respectively includes:

[0021] For each network layer in the pseudo-quantization model,

[0022] The weight data of the network layer is equalized using channel-level quantization, where each convolution kernel shares the same quantization parameter.

[0023] The sample image feature data for equalization of the network layer is quantized in a hierarchical manner, wherein the sample image feature data of all channels share the same quantization parameter.

[0024] Preferably, the step of inputting the sample image data into the pseudo-quantization model for inference, and counting the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer includes:

[0025] For any network layer to be quantized in the pseudo-quantization model,

[0026] After the network layers before the network layer to be quantized have completed quantization,

[0027] Using the pseudo-quantization operator in the pseudo-quantization model, fixed-point reasoning is simulated in each quantized network layer, and floating-point reasoning is simulated in the network layer to be quantized.

[0028] The characteristic data distribution of the sample images output by the network layer to be quantized is counted, and the weight data distribution of the convolutional layer in the network layer to be quantized is counted.

[0029] Preferably, the step of inputting the sample image data into the pseudo-quantization model comprises:

[0030] The sample image data is divided into two parts according to a set ratio to obtain first sample image data and second sample image data.

[0031] Inputting the first sample image data and the second sample image data into the pseudo quantization model in sequence;

[0032] The counting of the distribution of sample image feature data output by the network layer to be quantized includes:

[0033] For each channel sample image feature data output by the network layer to be quantized,

[0034] When the first sample image data is input into the pseudo-quantization model for inference, the minimum-maximum moving average statistical method is used to obtain the maximum and minimum values ​​in the channel sample image feature data, and the interval formed by the minimum and maximum values ​​is evenly divided into at least two sub-intervals.

[0035] When the second sample image data is input into the pseudo-quantization model for inference, the number of the channel sample image feature data falling into each sub-interval is counted and normalized to obtain an approximate probability density function;

[0036] The counting of weight data distribution of the convolutional layer in the network layer to be quantized includes:

[0037] For the weight data of the two stages corresponding to the first sample image data and the second sample image data being input into the pseudo-quantization model for inference, the weight data of any stage is selected and the minimum-maximum value statistical method is used for statistics;

[0038] The simulating fixed-point reasoning at the quantized network layer includes:

[0039] The network parameters in each quantized network layer are quantized and dequantized.

[0040] Preferably, the method of balancing all weight data and sample image feature data by stretching the scale of sample image feature data of each channel in each network layer according to the statistical data distribution includes:

[0041] For any network layer to be quantized in the pseudo-quantization model,

[0042] The sample image feature data of each channel output by the network layer to be quantized is stretched according to the stretching scale of the channel, so that the network layer to be quantized utilizes the properties of the activation layer to fuse the stretching scale of the channel into the weights and biases of the convolutional layer in the network layer to be quantized,

[0043] in,

[0044] The channel stretching scale is determined according to the distribution of sample image feature data counted for the network layer to be quantized.

[0045] Preferably, the determining of a quantization parameter for quantization includes:

[0046] For any network layer to be quantized in the pseudo-quantization model,

[0047] Determine a first cutoff value by minimizing the mean square error between the data before and after quantization, and determine a first quantization parameter for fixing the weight according to the first cutoff value and the first bit width,

[0048] The second cutoff value is determined by minimizing the relative entropy between the data before and after quantization.

[0049] A second quantization parameter for fixing the sample image feature data is determined according to the second truncation value and the second bit width.

[0050] Preferably, the step of quantizing the equalized weight data and the sample image feature data using quantization parameters respectively includes:

[0051] Calculate the ratio of the equalized weight data to the first quantization parameter to obtain a first ratio,

[0052] Rounding the first ratio to an integer,

[0053] According to the minimum value and the maximum value of the first quantization interval, the first ratio after rounding is controlled to be an integer within the first quantization interval; if the first ratio after rounding is less than the minimum value, the first ratio after rounding is set to be equal to the minimum value; if the first ratio after rounding is greater than the maximum value, the first ratio after rounding is set to be equal to the maximum value; if the first ratio after rounding is within the first quantization interval, the first ratio after rounding is kept unchanged;

[0054] Calculating the ratio of the equalized sample image feature data to the second quantization parameter to obtain a second ratio,

[0055] Round the second ratio to an integer,

[0056] According to the minimum value and the maximum value of the second quantization interval, the first ratio after rounding is controlled to be an integer within the quantization interval, if the second ratio after rounding is less than the minimum value, the second ratio after rounding is set to be equal to the minimum value, if the second ratio after rounding is greater than the maximum value, the second ratio after rounding is set to be equal to the maximum value, if the second ratio after rounding is within the second quantization interval, the second ratio after rounding is kept unchanged;

[0057] The first quantization interval is determined according to the first bit width, and the second quantization interval is determined according to the second bit width.

[0058] Preferably, the converting of the inference result according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers during the quantization process of the convolutional neural network model comprises:

[0059] For the sample image feature data of each channel output by each network layer in the quantization model,

[0060] The sample image feature data of the channel is divided by the stretching scale of the channel to obtain the floating-point sample image feature data of the channel.

[0061] A second aspect of the present application provides an image-based target detection device, the device comprising:

[0062] The fixed-point reasoning module is used to use the convolutional neural network model for image target detection that has been trained to convergence and its quantized model to perform fixed-point reasoning on the image data to be detected to obtain the reasoning result, which is the fixed-point reasoning result.

[0063] A conversion module is used to convert the inference result according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model, to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model.

[0064] in,

[0065] The quantitative model is obtained as follows:

[0066] The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0067] The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted.

[0068] According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data.

[0069] determining the quantization parameter used for quantization,

[0070] Quantize the equalized weight data and sample image feature data using quantization parameters respectively;

[0071] The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer used for stretching.

[0072] In a third aspect, the present application provides a quantization device for a convolutional neural network model for image target detection, the device comprising:

[0073] An operator conversion module is used to convert the floating-point operators in the floating-point model of the convolutional neural network model into pseudo-quantization operators, fuse the convolutional layer operators and normalization layer operators included in each network layer into convolutional layer operators, and fuse the activation layer included in each network layer into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0074] The simulation inference module is used to input sample image data into the pseudo-quantization model for inference, and to count the distribution of sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer.

[0075] The equalization module is used to equalize all weight data and sample image feature data by stretching the scale of sample image feature data of each channel in each network layer according to the statistical data distribution, so as to obtain the equalized weight data and sample image feature data.

[0076] The fixed-point module is used to determine the quantization parameters used for quantization, and quantize the equalized weight data and the sample image feature data using the quantization parameters respectively.

[0077] The image-based target detection method provided by the present application performs fixed-point reasoning on the image data to be detected by training a convolutional neural network model for image target detection and its quantization model to convergence, and performs conversion reasoning on the reasoning result by using a channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers during the quantization process of the convolutional neural network model, which is beneficial for both compressing the model data and accelerating the reasoning process of the convolutional neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of an image-based target detection method according to an embodiment of the present application is provided.

[0079] Figure 2 A flow chart of a convolutional neural network model for image target detection and its quantization model trained to convergence is obtained in an embodiment of the present application.

[0080] Figure 3 A schematic diagram of data distribution statistics at each network layer.

[0081] Figure 4A schematic diagram of balancing two consecutive network layers.

[0082] Figure 5 An example of parameter equalization where the outputs of two network layers are input into an additive layer.

[0083] Figure 6 A schematic diagram of an image-based target detection device according to an embodiment of the present application.

[0084] Figure 7 A schematic diagram of a quantization device for quantizing a convolutional neural network model for image target detection according to an embodiment of the present application.

[0085] Figure 8 Another schematic diagram of an image-based target detection device or a quantization device for quantizing a convolutional neural network model for image target detection according to an embodiment of the present application. DETAILED DESCRIPTION

[0086] In order to make the objectives, technical means and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings.

[0087] The applicant has found that the detection efficiency and accuracy of using a convolutional neural network model for image target detection are related to the model parameters themselves, information loss during the quantization process, and computing power. The present application provides an image target detection method that reduces information loss during the quantization process by balancing model-related weights and image feature data, thereby improving detection efficiency without increasing computing power and ensuring detection accuracy.

[0088] See also Figure 1 As shown, Figure 1 This is a flow chart of an image-based target detection method according to an embodiment of the present application. The method includes:

[0089] Step 101, using the convolutional neural network model for image target detection that has been trained to convergence and its quantized model, performs fixed-point reasoning on the image to be detected to obtain a reasoning result, which is a fixed-point reasoning result.

[0090] It should be understood that a model trained to convergence refers to a model that is trained using sample image data and trained until the model converges. For a converged model, even if more sample image data is input, the model's detection performance indicators do not change much, and the model can be considered to have converged.

[0091] As another example, the trained convolutional neural network model refers to a model whose output detection results meet expectations, and it is also a converged model. Step 102, according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers during the quantization process of the convolutional neural network model, the inference result is converted and inferred to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model.

[0092] The quantitative model is obtained in the following manner:

[0093] The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0094] The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted.

[0095] According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data, making them more suitable for model quantization operations.

[0096] determining the quantization parameter used for quantization,

[0097] Quantize the equalized weight data and sample image feature data using quantization parameters respectively;

[0098] The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer. It should be understood that the inference result is a multi-dimensional matrix tensor corresponding to the input image size, indicating whether there is an object at a certain position on the image and the size of the object. After decoding and other post-processing, the information can be used to obtain the object detection information.

[0099] The embodiment of the present application targets the differences in image feature data between channels and transfers the differences to the parameters in the convolutional layer, so that the parameters of the entire network model are more adaptable to quantization, the precision loss of the model parameters is reduced, and the detection efficiency is improved.

[0100] To facilitate understanding of the present application, an existing convolutional neural network model is used as an example for explanation. It should be understood that the present application embodiment is not limited to the model structure of the convolutional neural network model in the embodiment, and can be used for convolutional neural network models of any structure.

[0101] The convolutional neural network model is a deep model built using some specific network layers such as convolution, activation and pooling, including network structure and model parameters, which are used to extract image features or perform linear and logistic regression prediction tasks; in image target detection, each network layer in the convolutional neural network model is used to extract image features. It should be understood that the network structure and model parameters are implemented by the computer in the form of calculation graph operators and operator parameters when running. In this embodiment, each network layer in the convolutional neural network model includes a convolution layer, a normalization layer and an activation layer.

[0102] See also Figure 2 As shown, Figure 2 A flow chart of a convolutional neural network model trained to convergence for image target detection and its quantization model is provided for the embodiment of the present application. The method includes:

[0103] Step 201, converting the floating-point operators in the floating-point model of the convolutional neural network model trained to convergence into pseudo-quantized operators,

[0104] The convergent convolutional neural network model refers to a convolutional neural network model trained to convergence, and the model parameters in the model are floating-point parameters. Image sample data may not have label information, which can effectively reduce the sample data collection cost and decouple it from the model training to convergence process, which is conducive to accelerating the compression and quantization of the model.

[0105] As an example, each floating-point operator in the computation graph of the convergent convolutional neural network model is converted into a pseudo-quantized operator, and the operator parameters of the floating-point operator are copied to the pseudo-quantized operator, so that the pseudo-quantized operator has the same operator parameters as the floating-point operator.

[0106] In addition, different pseudo-quantization operators can be inserted as needed, for example:

[0107] 1) Simulate the floating-point operator reasoning process and collect data during the model forward reasoning;

[0108] 2) Count the weight data in the network layer or the distribution of the sample image feature data output by the network layer;

[0109] 3) Calculate the quantization parameters for fixed-point conversion using statistical data distribution, and convert floating-point data into integer data for fixed-point conversion.

[0110] Step 202, performing a fusion operation on the convolutional neural network model structure trained to convergence, including: convolution layer and normalization layer fusion, and activation layer fusion, wherein the convolution layer and normalization layer fusion is used to fuse the convolution layer operator and the normalization layer operator into the convolution layer operator, and the activation layer fusion is used to fuse the activation layer into the previous operator of the activation layer.

[0111] 1) Fusion of convolutional layer and normalization layer

[0112] The fusion of convolutional layer and normalization layer is to merge the two layers into one convolutional layer for calculation. This is an equivalent replacement of fusing the parameters of the normalization layer into the convolutional layer. There are two cases for the fusion of convolutional layer and normalization layer. One is that the convolutional layer is in front and the normalization layer is in the back, and the other is that the convolutional layer is in the back and the normalization layer is in the front. The fusion methods of the two cases are as follows.

[0113] Assume that the calculation formula of the convolution layer is: X′=WX+b, where W is the weight in the convolution layer, b is the bias in the convolution layer, X is the input image feature data of the convolution layer, and X′ is the output image feature data of the convolution layer.

[0114] The calculation formula of the normalization layer is: Among them, Y is the output image feature data of the normalization layer, X′ is the input image feature data of the normalization layer, and γ, u, σ, ε, and β are all normalization layer parameters.

[0115] When the convolution layer comes first and the normalization layer comes later, we have:

[0116]

[0117] Therefore,

[0118] The weight W′ of the fused convolutional layer is:

[0119] The convolutional layer bias b′ after fusion:

[0120] When the normalization layer is in front and the convolution layer is in the back, we have:

[0121]

[0122] Therefore,

[0123] The fused convolutional layer weight W′:

[0124] The convolutional layer bias b′ after fusion:

[0125] The above fused convolutional layer weight W′ and bias b′ are the operator parameters of the convolutional layer operator.

[0126] 2) Activation layer fusion

[0127] The hardware-friendly nonlinear activation function ReLU will compress the value range of image feature data, for example, image feature values ​​less than 0 are set to 0. If the output of the activation layer is processed separately, for example, an asymmetric quantization method is used, then all image feature values ​​less than the quantization parameter z are 0. For example, the quantization parameter z takes values ​​in the interval [0, 255] under the uint8 quantization configuration, and some values ​​are set to 0. This makes it impossible to fully utilize the entire quantization interval, and the quantization accuracy is low. The activation layer is fused into the previous operator of the activation layer for processing together, so that the activation function in the activation layer is absorbed into the previous operator, and a quantizer is inserted after the activation function. In this way, the pseudo-quantization operator outputs the data range compressed by the ReLU activation function, and the output of the fused activation function will fall into the entire quantization interval, thereby improving the quantization accuracy.

[0128] As an example, when the previous operator of an activation layer is a convolutional layer, the activation layer is absorbed into the convolutional layer.

[0129] After processing each network layer in the convergent convolutional neural network model through the above steps 201 and 202, a pseudo-quantization model can be obtained.

[0130] Step 203, input the sample image data into the pseudo-quantization model, perform inference, and collect the data in the network layer to be quantized for statistics.

[0131] Since the pseudo-quantization operator in the pseudo-quantization model can simulate the floating-point inference process, it can also perform quantization and dequantization operations on the quantized network parameters to add noise and simulate the fixed-point inference process on the hardware. The quantization and dequantization operation is essentially to map the quantized network parameters to the integer domain and round them, then truncate them according to the minimum and maximum values ​​represented by the bits, and finally map them back to floating points for subsequent calculations. It is a simulation of fixed-point inference. The calculation formula for the quantization and dequantization operation is as follows:

[0132]

[0133] Among them, the round() function means rounding the independent variable according to an integer, for example, round(1.23)=1; the clip() function means controlling the elements in the array within a given range, the value less than the lower boundary is taken as the lower boundary, the value greater than the upper boundary is taken as the upper boundary, and the value between the lower boundary and the upper boundary remains unchanged, r is the floating point value of the quantized network parameter, s and z are quantization parameters, q min ,q max are the minimum and maximum values ​​represented by the bit width, r q is the dequantized value.

[0134] As can be seen from the calculation formula, the floating-point number of the quantized network parameter can be mapped to the integer domain and rounded by the round() function, and truncated according to the minimum and maximum values ​​represented by the bit width by the clip() function.

[0135] Since the quantization error will accumulate layer by layer during the fixed-point inference process, this embodiment performs data distribution statistics on a network layer by network layer basis. Figure 3 As shown, Figure 3 This is a schematic diagram of data distribution statistics for each network layer. When performing data distribution statistics on the current network layer, it is necessary to first complete the quantization operation on all network layers before the current network layer, so that the statistical data distribution is more consistent with the actual distribution in the fixed-point reasoning of the quantization model, which can alleviate the problem of inconsistency between simulation and fixed-point reasoning data to a certain extent. The quantization operation involves parameter equalization and quantization parameter calculation, which can be performed according to steps 205 to 206.

[0136] In view of the fact that when performing data distribution statistics on the current network layer, all network layers before the current network layer need to be quantized first, it is better to quantize the pseudo-quantization model layer by layer, that is, after the quantization of the current network layer is completed, the next network layer adjacent to the current network layer is quantized.

[0137] Take the quantization of any network layer to be quantized as an example. For the current network layer to be quantized, all network layers before the current network layer have been quantized.

[0138] The weights in the quantized network layer and the sample image feature data output by it are quantized and dequantized.

[0139] Dividing the sample image feature data into two parts, first sample image data and second sample image data, according to a set ratio;

[0140] The first sample image data and the second sample image data are successively input into the pseudo quantization model.

[0141] When performing data distribution statistics on the current network layer, the statistical data includes: the weight of the network layer, and the sample image feature data output by the network layer.

[0142] The statistics of sample image feature data include two stages:

[0143] A first stage when the first sample image data is input, and a second stage when the second sample image data is input;

[0144] When the first sample image data is sent to the pseudo-quantization model, for the sample image feature values ​​of each channel output by the activation function of the current network layer, the minimum-maximum (min-max) moving average statistical method is used to perform statistics on the sample image feature values ​​of the channel to obtain the maximum and minimum values, and the interval determined by the minimum value to the maximum value is evenly divided into N sub-intervals to obtain the channel sub-intervals.

[0145] When the second sample image data is sent to the pseudo-quantization model, for the sample image feature values ​​of each channel output by the activation function of the current network layer, the number of sample image feature values ​​of the channel falling into the sub-interval of the channel is counted, and then normalized to obtain the approximate probability density function of the sample image feature values ​​of the channel.

[0146] The image feature values ​​of each channel output by the activation layer are separately counted according to the above two-stage method to facilitate parameter equalization operation.

[0147] Although the first sample image data and the second sample image data are successively input into the pseudo-quantization model, since the weight parameters are fixed, the above two parts of sample image data only need to be counted once when they are successively output, and the min-max statistical method can be used for weight statistics.

[0148] Step 204, equalize the network layer parameters to be quantized.

[0149] After obtaining the distribution statistics of weights and sample image feature data, the weights and sample image features are equalized. Equalization means that the scale of the sample image feature data of each channel is stretched so that the entire weight of the convolution layer or the sample image feature data output by the activation layer is at the same level, thereby improving the quantization accuracy when using the hierarchical quantization method. For example, the value range that can be represented by 8-bit unsigned is [0,255]. When using hierarchical quantization, the entire layer uses the same quantization parameter, that is, the same s and the same z. If the data ranges of two channels are very different, assuming that the data range of channel 1 is [0,1] and the data range of channel 2 is [0,255], then the quantization parameters s and z will be solved according to the maximum value 255. In this way, since the value 1 in the value range [0,1] is only 1 / 255 compared to the maximum value 255, there is only 1 bit after quantization, and the information loss is large when channel 1 is quantized. Therefore, the data range of each channel is equalized, and the data ranges of channel 1 and channel 2 are pulled to [0,1]. In this way, at the same scale level, channel 1 can also use 8 bits with less information loss.

[0150] See also Figure 4 As shown, Figure 4A schematic diagram of equalization for two consecutive network layers. Assuming there are two consecutive network layers, the sample image feature data input to the first network layer is x1, and the sample image feature data output by the second network layer is y. The mathematical formula is:

[0151]

[0152] Among them, x2 represents the sample image feature data output by the first convolutional layer, which includes multiple channels, x3 represents the sample image feature data output by the second convolutional layer, which includes multiple channels, w1 and b1 are the weights and biases of the first convolutional layer, w2 and b2 are the weights and biases of the second convolutional layer,

[0153] The sample image feature data x2 output by the first convolutional layer is multiplied by a scale s2 to equalize the sample image feature data output by the first convolutional layer. Then, the formula 1 is equivalent to:

[0154]

[0155] Among them, s2x2 represents the sample image feature data output by the equalized first convolutional layer, and the data includes multiple channels. It should be understood that different channels have different stretching scales, and the stretching scale multiplied by each channel is the reciprocal of the difference between the maximum and minimum values ​​of the channel image feature value. In this embodiment, since the image feature data on all channels are equalized, the image feature value range on each channel is almost the same. In practice, for better processing, the minimum value of all image feature values ​​is set to 0. In this way, the stretching scale is the reciprocal of the maximum value, which is recorded as: 1 / max, where max is the maximum value of the image feature value.

[0156] Given that for any ReLU activation function f, the following properties exist:

[0157] sy=sf(wx+b)=f(s(wx+b))

[0158] That is, the sample image feature data output by the activation function is multiplied by a coefficient s, and the coefficient can be transferred to the activation function. Therefore, during inference, the scale of the output data can be transferred to the weights and biases in the convolution layer, without the need for additional scale transformation calculations. In this way, using the properties of the activation function, the scale s2 is absorbed into the weights and biases in the current first convolution layer, and the second formula is:

[0159]

[0160] The sample image feature data x3 output by the second convolutional layer is multiplied by a scale s3 to equalize the sample image feature data output by the second convolutional layer. Then, Formula 3 is equivalent to:

[0161]

[0162] Also using the properties of the activation function, scale s3 is absorbed into the weights and biases in the current second convolutional layer, then Equation 4 is equivalent to:

[0163]

[0164] Among them, s2x2 represents the sample image feature data output by the equalized first convolution layer, s3x3 represents the sample image feature data output by the equalized second convolution layer, and the data includes multiple channels. Similarly, s2w1, is the weight for equalization.

[0165] It should be understood that different channels have different stretching scales. The stretching scale multiplied by each channel is the difference between the maximum and minimum values ​​of the image feature value of the channel. In this embodiment, since the image feature data on all channels are equalized, the image feature value range on each channel is similar. In practice, for better processing, the minimum value of all image feature values ​​will be set to 0. In this way, the stretching scale is the reciprocal of the maximum value, which is recorded as: 1 / max, where max is the maximum value of the image feature value. For example, the stretching scale of each channel in the activation layer is calculated by the statistical distribution of sample image feature data. The sample image feature data of channel 1 is statistically analyzed, and the maximum value is 255. Then the stretching scale is 1 / 255. All values ​​of the channel are multiplied by this number, so that the sample image feature data of the channel has a value range of [0,1]. In order to ensure the inference equivalence with the floating-point model, the stretching scale is passed on for calculation.

[0166] The sample image feature data after equalization is suitable for quantization processing. s3x3 is an output sample image feature data with a stretching scale. This stretching scale is calculated after the entire model has completed parameter equalization and is known. After completing the fixed-point inference of the quantization model, it is multiplied by That's it.

[0167] What is explained here is that the sample image feature data in the activation layer adopts a hierarchical quantization method, and all sample image feature data share a set of quantization parameters. Since the image feature value ranges of each channel in the activation layer may be different, the scale scaling on the channel can make the image feature value ranges of all channels as close as possible, thereby improving the quantization accuracy of activation. Each channel of the activation layer is obtained by convolving a convolution kernel with the input data, which is a linear calculation. Therefore, the scale of the channel data can be transferred to the convolution kernel. Since the channel-level quantization scheme is used for the weights, each convolution kernel shares a set of quantization parameters, so the scale transfer to the convolution layer has no effect on quantization.

[0168] This embodiment equalizes the operator parameters of each operator in the pseudo-quantization model, that is, the output of each layer is scaled until it is suitable for quantization. For the convolution layer, the scale transformation can be integrated into the weight and bias calculation, while for some operators without operator parameters, such as the addition layer, additional scale transformation of the input is required.

[0169] See also Figure 5 As shown, Figure 5 This is an example of parameter equalization when the outputs of two network layers are input to the addition layer. Assume that the first network layer and the second network layer are both connected to an addition layer, as shown below:

[0170]

[0171] Among them, x represents the sample image feature data input into the first network layer and the sample image feature data input into the second network layer, x1 represents the sample image feature data output by the first network layer, x2 represents the sample image feature data output by the second network layer, and y represents the output sample image feature data of the addition layer.

[0172] After equalizing x1, x2, and x3, the calculation formula is:

[0173]

[0174] Among them, s1x1 represents the sample image feature data output by the equalized first network layer, s2x2 represents the sample image feature data output by the equalized second network layer, and s3x3 represents the sample image feature data output by the equalized addition layer.

[0175] It can be seen from the above formula that the pseudo-quantization operator of the addition layer needs to add parameters s1 and s2 to stretch the input x1 and x2 channel by channel respectively, and absorb the stretched scale output from the network layer respectively, that is, and It is noted that the two absorbed stretching scales can be merged into the quantization parameters during quantized fixed-point inference. The difference is that the quantization parameters of the additive layer are hierarchical before, and then the quantization parameters of the additive layer become channel-by-channel. There is no obvious change in the amount of computation during fixed-point inference.

[0176] The linear operation of scaling in this step can be incorporated into the forward reasoning of the operator without additional computational overhead. For operators with parameters, scaling can be directly incorporated into the parameter linear calculation; for operators without parameters, scaling parameters can be added and incorporated into fixed-point reasoning. This solves the computational problem of additional scaling changes caused by parameter equalization.

[0177] Step 205 , converting the equalized weights and sample image feature data into integers, that is, performing fixed-point conversion.

[0178] In this step, for the current network layer to be quantized, the quantization parameter is calculated according to the distribution characteristics of its statistical data, and then the floating point value is converted into an integer. The calculation formula is part of the quantization and dequantization calculation formula, which is:

[0179]

[0180] Among them, q is the fixed-point value after quantization, that is, the quantized value; s and z are quantization parameters; q min ,q max are the minimum and maximum values ​​of the quantization interval, determined according to the bit width; r is a floating point value.

[0181] The calculation of quantization parameters s and z requires the use of truncation values. The truncation value affects the quantization error. A truncation value that is too small will lose more data information, and a truncation value that is too large will make the floating point interval too large and introduce a larger quantization error. This embodiment uses the minimum mean square error (MSE) method on the weight, that is, by minimizing the mean square error between the data before and after quantization, the optimal truncation value is obtained. The method of minimizing the relative entropy (KL, Kullback-Leible) divergence is used on the activation layer, that is, by minimizing the KL divergence before and after quantization to find the optimal truncation value.

[0182] This embodiment uses a symmetric full-axis quantization method on the weights, and the first quantization parameter calculation formula for weight fixed-point is:

[0183]

[0184] Among them, v c1 is the first truncation value, b1 is the first bit width, and s1 is the first quantization parameter.

[0185] According to the quantization formula, the ratio of the equalized weight data to the first quantization parameter is calculated to obtain a first ratio, the first ratio is rounded to an integer, and according to the minimum value and the maximum value of the first quantization interval, the first ratio after rounding is controlled to be an integer within the first quantization interval. If the first ratio after rounding is less than the minimum value, the first ratio after rounding is set to be equal to the minimum value. If the first ratio after rounding is greater than the maximum value or the minimum value, the first ratio after rounding is set to be equal to the maximum value. If the first ratio after rounding is within the first quantization interval, the first ratio after rounding is kept unchanged; wherein, the first quantization interval is determined according to the counted weights.

[0186] In the activation layer, the characteristic values ​​of the sample image feature data after ReLU are all greater than 0, so [0, v c ] is mapped to the entire quantization interval, and a symmetrical full-axis quantization method is used. The second quantization parameter calculation formula for the fixed-point quantization of the sample image feature data is:

[0187]

[0188] Among them, v c2 is the second truncation value, b2 is the second bit width, and s2 is the second quantization parameter.

[0189] According to the quantization formula, the ratio of the equalized sample image feature data to the second quantization parameter is calculated to obtain the second ratio, the second ratio is rounded to an integer, and according to the minimum and maximum values ​​of the second quantization interval, the first ratio after rounding is controlled to be an integer within the quantization interval; if the second ratio after rounding is less than the minimum value, the second ratio after rounding is set equal to the minimum value; if the second ratio after rounding is greater than the maximum value, the second ratio after rounding is set equal to the maximum value; if the second ratio after rounding is within the second quantization interval, the second ratio after rounding is kept unchanged; wherein the second quantization interval is determined based on the statistically counted sample image feature data.

[0190] Steps 203 to 205 are repeatedly performed until each network layer in the pseudo-quantization model is quantized, thereby obtaining a quantized model.

[0191] It should be understood that the quantization calculation in this step can be calculated by a pseudo quantization operator inserted into the pseudo quantization model.

[0192] This embodiment equalizes the weights and sample image data, which can effectively control the data distribution differences between weights or activation layer channels, help improve quantization accuracy, and reduce task losses caused by model quantization. A channel-level quantization method is used for weights, that is, each convolution kernel shares the same set of quantization parameters; a hierarchical quantization method is used for sample image feature data, that is, all activated data share a set of quantization parameters. This method is easy to implement in hardware, and at the same time, the scale differences between the channels of the activation layer can be transferred to the weights in the convolution layer through parameter equalization. Since the weights use a channel-level quantization method, the weights will not be affected by the scale differences of the transferred channels, so the quantization accuracy of the model can be improved. In this embodiment, the parameters of the entire network are balanced from top to bottom, making it more adaptable to quantization operations, which can reduce the accuracy loss of model quantization to a certain extent; model quantization can effectively achieve compression and reasoning acceleration of deep convolutional neural networks.

[0193] The following describes an image-based object detection method based on fixed-point reasoning of a quantized model.

[0194] Fixed-point reasoning based on the quantized model completes forward reasoning with full integer data streams. Taking the convolution and addition operators as examples, the calculation process of full integer data stream operators is explained.

[0195] Assume that the calculation formula of a convolutional layer with floating-point parameters is:

[0196] r y =r w r x +r b

[0197] Among them, r y Output image feature floating point data for the convolution layer, r w is a floating point weight, r x is the input image feature floating point data, r b is a floating point bias,

[0198] The calculation formula for converting floating point to fixed point is:

[0199] s y (q y -z y )=s w (q w -z w )s x (q x -z x )+s w s x (q b -z b )

[0200] Among them, q y Output image feature fixed-point data for the convolution layer, q w is the fixed-point weight, q x is the input image feature fixed-point data, q b is the fixed-point bias, s y 、z y 、s w 、z w 、s x 、z x 、z b are all quantization parameters, z b It has no effect on the result, let z b =0, then the original formula is equal to:

[0201] s y (q y -z y )=s w s x ((q w -zw )(q x -z x )+q b )

[0202] Arranged as:

[0203]

[0204] According to the calculation formulas for the first quantization parameter and the second quantization parameter in step 206 of this embodiment, z w 、z x 、z v are all 0, so formula 6 is:

[0205]

[0206] in, is a floating point number, and the others are fixed point numbers.

[0207] make

[0208]

[0209] Formula 7 can be approximately equal to

[0210] q y =(M(q w q x +q b ))>>n

[0211] The symbol >> indicates a right shift operation, and n is the number of right shifts. The larger the value, the higher the accuracy, but there will be a large computational overhead. At this point, the integer inference of the convolutional layer is completed.

[0212] For the inference calculation of multi-input operators, the input image feature data needs to be unified to a scale before calculation can be performed. For example, the floating-point calculation formula of the addition layer is:

[0213] r y =r x1 +r x2

[0214] Among them, r y Output image feature floating point data, r x1 is the first input image feature floating point data, r x2 is the second input image feature floating point data,

[0215] The formula converted into fixed-point calculation is:

[0216] s y (q y -z y )=s x1 (qx1 -z x1 )+s x2 (q x2 -z x2 )

[0217] Let r y =q y -z y , r x1 =q x1 -z x1 , r x2 =q x2 -z x2 , then:

[0218] s y q y =s x1 q x1 +s x2 q x2

[0219] The above formula can be rearranged as:

[0220]

[0221] in and are floating point numbers, and the rest are fixed point numbers.

[0222] make

[0223]

[0224]

[0225] Formula 8 is approximately equal to

[0226] q y =(M1q x1 +M2q x2 )>>n

[0227] The symbol >> indicates a right shift operation, and n is the number of bits to be shifted right.

[0228] This completes the fixed-point inference calculation of the addition layer.

[0229] The final output data of the quantization model, i.e. the inference result, is multiplied back by the inverse of the scale stretched by the last network layer to obtain the detection result corresponding to the floating-point model inference.

[0230] It should be understood that the image feature data output by any network layer, i.e., the inference result of the network layer, can be multiplied back by the inverse of the scale stretched by the network layer to obtain the corresponding floating-point inference result of the network layer. Specifically, for the sample image feature data of each channel output by each network layer in the quantization model, the sample image feature data of the channel is divided by the stretched scale of the channel to obtain the floating-point sample image feature data of the channel.

[0231] After parameter equalization is performed in this embodiment, the parameters are more suitable for quantization operation, and the precision loss caused by the whole-layer quantization method is less, so the quantization information loss is less, which is conducive to improving the target detection accuracy.

[0232] See also Figure 6 As shown, Figure 6 A schematic diagram of an image-based target detection device according to an embodiment of the present application.

[0233] The device includes:

[0234] The fixed-point reasoning module is used to use the convolutional neural network model for image target detection that has been trained to convergence and its quantized model to perform fixed-point reasoning on the image data to be detected to obtain the reasoning result, which is the fixed-point reasoning result.

[0235] A conversion module is used to convert the inference result according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model, so as to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model.

[0236] in,

[0237] The quantitative model is obtained as follows:

[0238] The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0239] The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted.

[0240] According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data, making them more suitable for model quantization operations.

[0241] determining the quantization parameter used for quantization,

[0242] Quantize the equalized weight data and sample image feature data using quantization parameters respectively;

[0243] The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer used for stretching.

[0244] See also Figure 7 As shown, Figure 7 A schematic diagram of a quantization device for quantizing a convolutional neural network model for image target detection in an embodiment of the present application. The device includes:

[0245] An operator conversion module is used to convert the floating-point operators in the floating-point model of the convolutional neural network model into pseudo-quantization operators, fuse the convolutional layer operators and normalization layer operators included in each network layer into convolutional layer operators, and fuse the activation layer included in each network layer into the previous operator of the activation layer to obtain a pseudo-quantization model.

[0246] The simulation inference module is used to input sample image data into the pseudo-quantization model for inference, and to count the distribution of sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer.

[0247] The equalization module is used to equalize all weight data and sample image feature data by stretching the scale of sample image feature data of each channel in each network layer according to the statistical data distribution, so as to obtain the equalized weight data and sample image feature data.

[0248] The fixed-point module is used to determine the quantization parameters used for fixed-point according to the statistical data, and to fix the equalized weight data and the sample image feature data respectively using the quantization parameters.

[0249] See also Figure 8 As shown, Figure 8 Another schematic diagram of an image-based target detection device or a quantization device for quantizing a convolutional neural network model for image target detection according to an embodiment of the present application. The device includes a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of the image-based target detection method according to an embodiment of the present application or the steps of obtaining a trained convolutional neural network model for image target detection and its quantization model.

[0250] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0251] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0252] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of an image-based target detection method or the steps of obtaining a trained convolutional neural network model for image target detection and its quantization model.

[0253] As for the apparatus / network-side device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0254] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0255] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image-based target detection method, characterized in that: The method includes: Using the convolutional neural network model for image target detection that has been trained to convergence and its quantized model, fixed-point inference is performed on the image data to be detected to obtain an inference result, which is a fixed-point inference result. According to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model, the inference result is converted and inferred to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model. in, The quantitative model is obtained as follows: The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model. The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted. According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data. determining the quantization parameter used for quantization, Quantize the equalized weight data and sample image feature data using quantization parameters respectively; The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer used for stretching.

2. The target detection method according to claim 1, characterized in that: The sample image data does not include label information of the sample target; The converting the floating-point operator in the floating-point model of the convolutional neural network model into a pseudo-quantization operator includes: Convert the floating-point operator itself into the corresponding pseudo-quantization operator, Keep the operator parameters of the floating-point operator unchanged and use them as the operator parameters of the pseudo-quantization operator; The step of quantizing the equalized weight data and the sample image feature data using quantization parameters respectively includes: For each network layer in the pseudo-quantization model, The weight data of the network layer is equalized using channel-level quantization, where each convolution kernel shares the same quantization parameter. The sample image feature data for equalization of the network layer is quantized in a hierarchical manner, wherein the sample image feature data of all channels share the same quantization parameter.

3. The target detection method according to claim 1, characterized in that: The sample image data is input into the pseudo-quantization model for inference, and statistics are given on the distribution of sample image feature data output by each network layer, and the distribution of weight data of the convolution layer in each network layer, including: For any network layer to be quantized in the pseudo-quantization model, After the network layers before the network layer to be quantized have completed quantization, Using the pseudo-quantization operator in the pseudo-quantization model, fixed-point reasoning is simulated in each quantized network layer, and floating-point reasoning is simulated in the network layer to be quantized. The characteristic data distribution of the sample images output by the network layer to be quantized is counted, and the weight data distribution of the convolutional layer in the network layer to be quantized is counted.

4. The target detection method according to claim 3, characterized in that: The step of inputting the sample image data into the pseudo quantization model comprises: The sample image data is divided into two parts according to a set ratio to obtain first sample image data and second sample image data. Inputting the first sample image data and the second sample image data into the pseudo quantization model in sequence; The counting of the distribution of sample image feature data output by the network layer to be quantized includes: For each channel sample image feature data output by the network layer to be quantized, When the first sample image data is input into the pseudo-quantization model for inference, the minimum-maximum moving average statistical method is used to obtain the maximum and minimum values ​​in the channel sample image feature data, and the interval formed by the minimum and maximum values ​​is evenly divided into at least two sub-intervals. When the second sample image data is input into the pseudo-quantization model for inference, the number of the channel sample image feature data falling into each sub-interval is counted and normalized to obtain an approximate probability density function; The counting of the weight data distribution of the convolutional layer in the network layer to be quantized includes: For the weight data of the two stages corresponding to the first sample image data and the second sample image data being input into the pseudo-quantization model for inference, the weight data of any stage is selected and the minimum-maximum value statistical method is used for statistics; The simulating fixed-point reasoning at the quantized network layer includes: The network parameters in each quantized network layer are quantized and dequantized.

5. The target detection method according to claim 1, characterized in that: The method of balancing all weight data and sample image feature data by stretching the scale of sample image feature data of each channel in each network layer according to the statistical data distribution includes: For any network layer to be quantized in the pseudo-quantization model, The sample image feature data of each channel output by the network layer to be quantized is stretched according to the stretching scale of the channel, so that the network layer to be quantized utilizes the properties of the activation layer to fuse the stretching scale of the channel into the weights and biases of the convolutional layer in the network layer to be quantized, in, The channel stretching scale is determined according to the distribution of sample image feature data counted for the network layer to be quantized.

6. The target detection method according to claim 3, characterized in that: The determining of a quantization parameter for quantization includes: For any network layer to be quantized in the pseudo-quantization model, The first cutoff value is determined by minimizing the mean square error between the data before and after quantization. Determine a first quantization parameter for fixing the weight according to the first cutoff value and the first bit width, The second cutoff value is determined by minimizing the relative entropy between the data before and after quantization. A second quantization parameter for fixing the sample image feature data is determined according to the second truncation value and the second bit width.

7. The target detection method according to claim 6, characterized in that: The step of quantizing the equalized weight data and the sample image feature data using quantization parameters respectively includes: Calculating the ratio of the equalized weight data to the first quantization parameter to obtain a first ratio, Rounding the first ratio to an integer, According to the minimum value and the maximum value of the first quantization interval, the first ratio after rounding is controlled to be an integer within the first quantization interval; if the first ratio after rounding is less than the minimum value, the first ratio after rounding is set to be equal to the minimum value; if the first ratio after rounding is greater than the maximum value, the first ratio after rounding is set to be equal to the maximum value; if the first ratio after rounding is within the first quantization interval, the first ratio after rounding is kept unchanged; Calculating the ratio of the equalized sample image feature data to the second quantization parameter to obtain a second ratio, Round the second ratio to an integer, According to the minimum value and the maximum value of the second quantization interval, the first ratio after rounding is controlled to be an integer within the quantization interval, if the second ratio after rounding is less than the minimum value, the second ratio after rounding is set to be equal to the minimum value, if the second ratio after rounding is greater than the maximum value, the second ratio after rounding is set to be equal to the maximum value, if the second ratio after rounding is within the second quantization interval, the second ratio after rounding is kept unchanged; The first quantization interval is determined according to the first bit width, and the second quantization interval is determined according to the second bit width.

8. The target detection method according to claim 1, characterized in that: The converting of the inference result according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model comprises: For the sample image feature data of each channel output by each network layer in the quantization model, The sample image feature data of the channel is divided by the stretching scale of the channel to obtain the floating-point sample image feature data of the channel.

9. An image-based target detection device, characterized in that: The device includes: The fixed-point reasoning module is used to use the convolutional neural network model for image target detection that has been trained to convergence and its quantized model to perform fixed-point reasoning on the image data to be detected to obtain the reasoning result, which is the fixed-point reasoning result. A conversion module is used to convert the inference result according to the channel stretching scale that equalizes the weights of all network layers in the convolutional neural network model and the sample image feature data output by all network layers in the convolutional neural network model during the quantization process of the convolutional neural network model, to obtain a detection result, which is the inference result corresponding to the floating-point model of the convolutional neural network model. in, The quantitative model is obtained as follows: The floating-point operators in the floating-point model of the convolutional neural network model are converted into pseudo-quantization operators, the convolutional layer operators and normalization layer operators included in each network layer are fused into a convolutional layer operator, and the activation layer included in each network layer is fused into the previous operator of the activation layer to obtain a pseudo-quantization model. The sample image data is input into the pseudo-quantization model for inference, and the distribution of the sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer are counted. According to the statistical data distribution, by stretching the scale of the sample image feature data of each channel in each network layer, all weight data and sample image feature data are balanced to obtain balanced weight data and sample image feature data. determining the quantization parameter used for quantization, Quantize the equalized weight data and sample image feature data using quantization parameters respectively; The channel stretching scale is determined by the scale of the sample image feature data of the channel in each network layer used for stretching.

10. A quantization device for a convolutional neural network model for image target detection, characterized in that: The device includes: An operator conversion module is used to convert the floating-point operators in the floating-point model of the convolutional neural network model into pseudo-quantization operators, fuse the convolutional layer operators and normalization layer operators included in each network layer into convolutional layer operators, and fuse the activation layer included in each network layer into the previous operator of the activation layer to obtain a pseudo-quantization model. The simulation inference module is used to input sample image data into the pseudo-quantization model for inference, and to count the distribution of sample image feature data output by each network layer and the weight data distribution of the convolution layer in each network layer. The equalization module is used to equalize all weight data and sample image feature data by stretching the scale of sample image feature data of each channel in each network layer according to the statistical data distribution, so as to obtain the equalized weight data and sample image feature data. The fixed-point module is used to determine the quantization parameters used for quantization, and quantize the equalized weight data and the sample image feature data using the quantization parameters respectively.

Images