A model compression outlier elimination method

Abstract

The present invention discloses a model compression outlier elimination method, wherein the model is a radar-vision fusion target detection and tracking model containing a Transformer structure. Through the quantization algorithm proposed in the present invention, the distribution of weights can be effectively balanced, and the operators in the model can be further compressed to the representation bit width of int8 / uint8. This can not only reduce the size of the model and reduce the temporary use of memory on the end side, but also, during the model reasoning process, the cache can carry more data, reduce the number of times the model weights are loaded, and achieve the effect of performance acceleration while ensuring the detection accuracy.

Description

Technical Field

[0001] The present invention relates to a method for eliminating multi-peak non-Gaussian distribution or outliers of feature weights brought about by a Transformer structure. Background Art

[0002] Due to the limitations of the Transformer structure compared to traditional CNN or RNN-based models, its self-attention mechanism improves long-distance interaction modeling, performs better in dealing with long-distance dependency problems, and can capture global context information, which is essential for understanding multimodal fusion data. It can establish position correspondences between different modalities, achieve spatial alignment, enhance the feature representation of the target, and make target tracking more stable and accurate. These excellent performances are crucial in complex scenarios. Given the many advantages of the Transformer, in the current increasingly complex autonomous driving scenarios, in order to meet the requirements of stable target detection and high-certainty target tracking in multiple scenarios, the Transformer structure is widely used for target detection and tracking to meet the stability and accuracy of target detection and tracking in complex scenarios.

[0003] When deploying CNN models on the client side, the models are usually compressed to gain benefits from model acceleration on the client side. While using the Transformer structure to gain benefits in model robustness, the Transformer structure also brings about the appearance of multi-peak non-Gaussian distribution or outliers of feature weights. Such multi-peak weight distribution or outliers will cause the mapping range to be enlarged and unevenly distributed. Under the traditional MINMAX quantization mapping, the quantization error is too large, and the quantization accuracy can only be regressed to use float32 precision to meet the accuracy requirements. When the bit width of float32 precision is used to represent numerical values, the number of calculations and loading times of the client side hardware and the pressure on the inter-core bandwidth will increase, which will greatly extend the inference performance of the model and reduce the frame rate of model detection, so the performance benefits are significantly reduced.

[0004] The invention patent application with publication number CN110097187A, published on August 6, 2019, proposed a method based on activation-entropy weight pruning to combine neuron activation value and information entropy to calculate weight score to measure the importance of weight, and prune the model according to the importance of weight. This technical solution is for the multi-peak non-Gaussian distribution of conv weights brought by the Transformer structure or when there are multiple outlier data, the multi-peak distribution of its weight data indicates that each peak may correspond to an important position or feature in the input sequence. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in the current intelligent driving industry, the end-to-end multimodal fusion model builds adaptive functions on the basis of modules based on the Transformer structure. For the application requirements of radar-vision fusion target tracking, how to effectively accelerate the model containing the Transformer structure under the condition of low computing power on the device side to meet the requirements of end-side application accuracy and real-time performance.

[0006] In order to solve the above technical problems, the technical solution of the present invention is to disclose a model compression outlier elimination method, wherein the model is a radar-vision fusion target detection and tracking model containing a Transformer structure, and is characterized in that it comprises the following steps:

[0007] Step 1, obtain multimodal data including image data and point cloud data, and the preprocessed multimodal data is the input data of the model; Step 2, input the input data into the model, and obtain the conv operator with a relatively large quantization error in the low-bit quantization process of the model; Step 3, assume that the input data of the conv operator obtained in step 2 is X, and the weight data is W: if the weight data W of the current conv operator presents a multi-peak non-Gaussian distribution, the current conv operator is configured to perform equalization processing during quantization; if the weight data W of the current conv operator presents a unimodal Gaussian distribution, the current conv operator is configured not to perform equalization processing during quantization; performing equalization processing during quantization includes the following steps: Step 301, mapping the weight data W to the quantization precision bit width, performing equalization processing on the weight data W, and obtaining the weight data W after quantization equalization q Step 302: Map the input data X to the quantization precision bit width, perform equalization processing on the input data X, and obtain the quantized equalized input data X q ; Step 303, weight data W q and input data X q Bring it into the current conv operator, calculate the quantization result of balanced quantization, and then dequantize the quantization result to obtain the dequantized output result, and compare the dequantized output result with the output result under the original precision bit width to verify whether the output result of the current conv operator after balanced quantization has converged in accuracy. If not, configure the current conv operator not to perform quantization during quantization; Step 4, perform model quantization processing according to the configuration of step 3 to generate a quantized model; Step 5, load the same true value data, compare the output results of the quantized model with the model under the original precision bit width, and compare the result accuracy: if the accuracy requirements are met, perform model platform adaptation conversion and load it to the device for use; if the accuracy does not meet the requirements, relocate the cause of the quantization error and the corresponding conv operator for algorithm optimization processing or release it to the original precision bit width for accuracy alignment to meet the application accuracy requirements.

[0008] Preferably, in step 1, during preprocessing, mean subtraction and normalization processing are performed on the image data; and voxelization processing is performed on the point cloud data.

[0009] Preferably, in step 2, the following method is used to determine whether the quantization error of the current conv operator is large:

[0010] After the current conv operator is quantized, the output result of the current quantized conv operator is dequantized, and the similarity between the dequantized result and the calculation result of the current conv operator at the original precision bit width is calculated: if the similarity is greater than the preset threshold, it indicates that the quantization error is acceptable; otherwise, it indicates that the quantization error of the current conv operator is large.

[0011] Preferably, in step 301, the weight data W q The calculation is done using the following formula: , where: W' is the weight data after equalization, W'=W×E, E is the equalization factor, which is calculated based on the statistical characteristics of the weight data W; S w ' is the scaling factor after weight quantization and equalization, S w =S W ×E,S W is the quantization factor; w min and w max are the minimum and maximum values ​​of the quantization range respectively; clip means limiting the data range to [w min ,w max ]; adaround refers to a quantization algorithm that minimizes the quantization error by adjusting the rounding method of the weights.

[0012] Preferably, the equalization factor E is the standard deviation of the weight data W.

[0013] Preferably, the quantization factor S W The calculation is done using the following formula: , where w fp32_max 、w fp32_min They are respectively the maximum and minimum values ​​of the weight data W under the original precision.

[0014] Preferably, in step 301, the input data X q The calculation is done using the following formula: , where X' is the input data after equalization, X'=X / E; S X is the scaling factor.

[0015] Preferably, the scaling factor S X The calculation is done using the following formula: , where: X float32_maxis the maximum value of the optimal data distribution of the input data X under the original precision bit width, X float32_min It is the minimum value of the optimal data distribution of the input data X under the original precision bit width.

[0016] Preferably, step 303 includes the following steps: Step 303-1, obtaining the output result Y of the original precision bit width of the current conv operator f : Y f = conv(X,W); Step 303-2, weight data W q and input data X q Bring it into the current conv operator, calculate the quantization result of the balanced quantization, and then dequantize the quantization result to obtain the dequantized output result Y q : Y q =S X ×S w '×conv(X q ,W q ); Step 303-3, calculate the inverse quantization output result Y q And the output result Y f Error ∆Y e :∆Y e =cosine_sim(Y f -Y q ), where cosine_sim represents cosine similarity; Step 303-4, if the error ∆Y e If the value is less than the preset threshold, it means that the accuracy of the current conv operator after equalization quantization exceeds the error range.

[0017] Preferably, the method further includes verifying and evaluating the accuracy of the quantized model obtained in step 4 on the application platform side to confirm whether it meets the accuracy requirements of the application side.

[0018] The quantization algorithm proposed in the present invention can effectively balance the distribution of weights and further compress the operators in the model to the representation bit width of int8 / uint8. This can not only reduce the size of the model and reduce the temporary memory usage on the end side, but also enable the cache to carry more data during the model reasoning process, reduce the number of times the model weights are loaded, and achieve the effect of performance acceleration while ensuring the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flow chart of the present invention is illustrated. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0021] A model compression outlier elimination method disclosed in an embodiment of the present invention is aimed at the application requirements of radar-vision fusion target detection and tracking. Under the condition of low computing power on the device side, the radar-vision fusion target detection and tracking model (hereinafter referred to as the "model") containing the Transformer structure is effectively accelerated to meet the requirements of the accuracy and real-time performance of the terminal-side application, such as Figure 1 As shown, the specific steps include:

[0022] Step 1: Prepare input data

[0023] The camera obtains image data images, and the 4D millimeter wave radar obtains point cloud data pcds. The two different modal data are used as multimodal inputs of the model.

[0024] The present invention refers to the image data of multiple frames and the point cloud data of the corresponding frame numbers as a calibration data set. Before model compression, sample data covering multiple scenes is selected as much as possible as filling data of the calibration data set to meet the robustness of the model.

[0025] Step 2: Model compression processing, further comprising:

[0026] Step 201: Data preprocessing

[0027] For image data, we perform mean subtraction and normalization. For point cloud data, we perform voxelization. The preprocessed data is then used as the input data of the model.

[0028] Step 202: Balancing algorithm configuration

[0029] The image data after mean subtraction and normalization and the point cloud data after voxelization are input into the model. For the conv operator with large quantization error during the low-bit quantization process of the model, the input data X (i.e., feature map) distribution and weight data W (i.e., weights) distribution of the current conv operator are specifically analyzed, and the current quantization error is eliminated through the equalization algorithm.

[0030] For the first operator of the model, the input of the model is the feature map of the first operator. For subsequent operators, the output data of the previous operator is the input data feature map distribution of the current operator. For the weight data W, after the model is trained, the trained data is solidified into the weights of the conv operator. The currently acquired model already has solidified weights data.

[0031] In an embodiment of the present invention, the following method is used to determine whether the quantization error of the current conv operator is large: after the same conv operator is quantized, the output result of the current quantized conv operator is dequantized, and the dequantized result and the result calculated by the current conv operator under the float32 bit width are used to calculate the cosine familiarity: if the result of the cosine familiarity is greater than or equal to 0.9, it indicates that the quantization error is acceptable; if the result of the cosine familiarity is less than 0.9, it indicates that the quantization error of the current operator is too large.

[0032] The configuration of the balancing algorithm involves a series of data distribution acquisition and analysis, as well as balancing algorithm processing. The following further describes the processing flow module of the balancing algorithm, which specifically includes the following steps:

[0033] Step 2021: Quantization mapping calculation

[0034] First, the weight data W of the conv operator is mapped and calculated, and then the equalized data is brought into the quantization calculation to achieve balanced quantization processing.

[0035] The model weight data currently obtained is expressed numerically with float32 precision, and the data range is [-3.402823e 38 , 3.402823e 38 ], and the data range of int8 precision expression is [-128,127]. On the end device, in order to better increase the amount of data loaded by the device-side register, the data is usually mapped to int8, and the int8 precision data is used for fixed-point calculation to improve the speed of data calculation and pursue the upper limit of performance optimization.

[0036] If the weight data W of the conv operator presents a unimodal Gaussian distribution, the quantization factor is calculated by the following formula:

[0037]

[0038] In the formula, w fp32_max 、w fp32_min They are respectively the maximum and minimum values ​​of the weight data in float32 precision.

[0039] The weight scaling factor S can be obtained through the above calculation formula W Next, we optimize the weights of the multi-peak conv operator in the Transformer structure by calculating the equalization factor and bringing the equalization data into the quantization mapping.

[0040] Assume that the input data of the conv operator is X. Since it is also a float32 precision data type, the input data X is converted into an int8 precision data type. For the input data X, the KLD algorithm is used to calculate the source data distribution P(x) to obtain the optimal data distribution Q(X), and then the scaling factor S of the input data is calculated through Q(X). X , the specific calculation formula is as follows:

[0041] Step 2021-1: Obtain the optimal data distribution Q(X) of the current input data X:

[0042]

[0043] Step 2021-2: Calculate the data range of the optimal data distribution Q(X):

[0044]

[0045]

[0046] Step 2021-3: Calculate the scaling factor S of the input data X X : .

[0047] Step 2022: Calculate the balance factor

[0048] For the multi-peak feature weights shown in the conv operator when the Transformer structure captures multi-target and multi-dimensional characteristics, an equalization factor is calculated for each channel of the input data X of the operator to be quantized. The equalization factor here is used to adjust the representation range of the weight to avoid excessive divergence of the weight range and quantization errors.

[0049] The current equalization factor E is calculated based on the statistical characteristics of the weight data. The configuration of the current equalization factor E can be adapted according to the distribution characteristics of the weight data W of the current conv operator. The embodiment of the present invention uses the standard deviation of the weight data W as the equalization factor to illustrate the calculation process of the algorithm. Furthermore, the equalization factor E=std(W), where std represents the standard deviation.

[0050] Step 2023: Data balancing processing

[0051] After obtaining the equalization factor E, the weight data W of the conv operator needs to be equalized to reduce the quantization error caused by the large peak difference of the numerical distribution of the model operator weight data W. The present invention divides the input data by the equalization factor and multiplies the weight by the equalization factor to balance the input or weight of the model.

[0052] The input data of the current conv operator is X and weight W, and the weight after equalization is W', then:

[0053] W'=W×E

[0054] The balanced data is X', then:

[0055] X'=X / E.

[0056] Step 2024: Quantization equalization processing

[0057] For the quantitative equalization of the weight data W, it is necessary to bring the equalized data into the quantitative calculation. For the equalized weight data W, when its data distribution is relatively balanced and presents a single-peaked Gaussian distribution, the data is then quantized. Here, the equalization factor and the quantization factor are merged. The merged quantitative equalization factor S w =S W ×E.

[0058] The calculation method of weight data W in quantization is:

[0059]

[0060] Among them, S w ' is the scaling factor after weight quantization and equalization, W q is the weight data after balanced quantization, w min and w max are the minimum and maximum values ​​of the quantization range, respectively. Clip means limiting the data range to [w min ,w max ], adaround represents a quantization algorithm that minimizes the quantization error by adjusting the rounding method of the weights.

[0061] For the input data X, quantization equalization processing is also required. Here, the equalized input data X' is brought into the quantization calculation, and then:

[0062]

[0063] Where, X q The input data is equalized and quantized.

[0064] The weight data and input data after balanced quantization are brought into the conv operator, the result of balanced quantization is calculated, and then dequantized back to obtain float32 precision data. The output result under float32 precision is compared with the result after balanced quantization to verify whether the output result of the conv operator after balanced quantization has converged in accuracy.

[0065] Calculation of the conv operator under float32 precision and the output result Y f :

[0066] Y f = conv(X,W)

[0067] The same conv operator after equalization quantization and the calculation formula for dequantizing its result, the output result is Y q :

[0068] Y q = S X ×S w '×conv(X q ,W q )

[0069] Calculate the error ∆Y between the conv operator after equalization and the conv operator under float32 precision e In the embodiment of the present invention, the cosine similarity is used to calculate:

[0070] ∆Y e =cosine_sim(Y f -Y q )

[0071] Where cosine_sim represents cosine similarity.

[0072] In the embodiment of the present invention, the error ∆Y e The value is greater than 0.9 to judge that the accuracy of the current conv operator after balanced quantization is within the error range: e If the value is less than 0.9, it means that the accuracy of the current conv operator after balanced quantization exceeds the error range. If the accuracy exceeds the error range, it means that the current conv operator has failed in balanced quantization, and the current conv operator needs to be configured as non-quantized mode to perform inference under float32 accuracy.

[0073] Step 203: Balance operator configuration

[0074] In the entire structure of the radar-vision fusion target detection and tracking model, not all conv operators have a multi-peak non-Gaussian distribution. The backbone of the current radar-vision fusion target detection and tracking model is resnet50, in which the conv operator weights are mostly unimodal and Gaussian. The operator balancing process can be shielded during quantization, and for operators with multi-peak distribution, balancing needs to be configured during quantization.

[0075] Step 204: Model quantization and equalization

[0076] Load the model file, input data, and quantization configuration file, perform model quantization and balancing processing, and generate the corresponding balanced and quantized model after weight balancing and quantization is completed.

[0077] Here, we need to compare the results of the quantized model with the model under float32 precision. Load the same GT (true value) data and compare the result accuracy: if the accuracy requirements are met, perform model platform adaptation conversion and load it to the device side (i.e., the application platform side) for use; if the accuracy does not meet the requirements, it is necessary to re-locate the cause of the quantization error and the corresponding conv operator for algorithm optimization or release it to float32 precision for precision alignment to meet the application accuracy requirements.

[0078] Step 3: Application platform side

[0079] For the quantized model, its accuracy also needs to be verified and evaluated on the application platform side to confirm whether it meets the accuracy requirements of the application side and avoid problems such as operator result anomalies and adaptation anomalies on the device side. When aligning the results on the application platform side, there are usually the following steps:

[0080] Step 301: Data preprocessing platform acceleration

[0081] Step 3011: For the preprocessing operation of the model, use an ASIC (Application-Specific Integrated Circuit) specific accelerator to perform specified processing. The current end-side device usually has a built-in preprocessing accelerator to perform data pre-acceleration processing.

[0082] Step 3012: The end-side device obtains the image data images of the camera and the point cloud data of the millimeter-wave radar, inputs them into the data preprocessing platform accelerator, and configures the preprocessing method to obtain the preprocessed data.

[0083] Step 302: Run resource allocation

[0084] When the end-side device needs to load multiple models, it is necessary to divide the resources of the model accelerator. Here, the device model acceleration resources are generally preliminarily divided according to the proportion of the model's performance time consumption in the total time consumption at the application layer to balance the device load.

[0085] When there is only one model to be loaded on the terminal device, all the model accelerator resources of the current device can be allocated to the current model.

[0086] Step 303: Model loading

[0087] After allocating device resources, load the model data and load the model structure, weights and other data to the corresponding accelerator on the device side.

[0088] Step 304: Model execution

[0089] The accelerator loaded with the model data can obtain the corresponding model output result data by calling the corresponding accelerator resources to execute the model's inference execution processing.

[0090] Step 4: Accuracy verification and evaluation

[0091] After obtaining the running results on the device side, the accuracy of the device side results can be compared with GT (true value). Here, the accuracy comparison is to load the quantized model and the model under non-quantized float32 accuracy respectively, and compare the error with the model output results under float32 accuracy to evaluate whether the quantization accuracy error meets the requirements. If the accuracy requirements are not met, it is necessary to locate the operator abnormal points on the device side and obtain the log information of the operator abnormality to confirm the adaptability of the operator after balanced quantization on the platform, or the abnormal result of the operator itself, and feedback to the device manufacturer to fix the problem.

Claims

1. A model compression outlier elimination method, wherein the model is a radar-vision fusion target detection and tracking model containing a Transformer structure, characterized in that: The following steps are involved: Step 1: obtain multimodal data including image data and point cloud data, and the preprocessed multimodal data is the input data of the model; Step 2: input the input data into the model, and obtain the conv operator with a relatively large quantization error that appears in the low-bit quantization process of the model, wherein the following method is used to determine whether the quantization error of the current conv operator is large: after the current conv operator is quantized, the output result of the current quantized conv operator is dequantized, and the similarity between the dequantized result and the calculation result of the current conv operator at the original precision bit width is calculated: if the similarity is greater than the preset threshold, it indicates that the quantization error is acceptable; otherwise, it indicates that the quantization error is acceptable. It is clear that the quantization error of the current conv operator is large; step 3, assume that the input data of the conv operator obtained in step 2 is X, and the weight data is W: if the weight data W of the current conv operator presents a multi-peak non-Gaussian distribution, the current conv operator is configured to perform equalization processing during quantization; if the weight data W of the current conv operator presents a unimodal Gaussian distribution, the current conv operator is configured not to perform equalization processing during quantization; performing equalization processing during quantization includes the following steps: step 301, mapping the weight data W to the quantization precision bit width, performing equalization processing on the weight data W, and obtaining the weight data W after quantization equalization q Step 302: Map the input data X to the quantization precision bit width, perform equalization processing on the input data X, and obtain the quantized equalized input data X q ; Step 303, weight data W q and input data X q Bring it into the current conv operator, calculate the quantization result of balanced quantization, and then dequantize the quantization result to obtain the dequantized output result, and compare the dequantized output result with the output result under the original precision bit width to verify whether the output result of the current conv operator after balanced quantization has converged in accuracy. If not, configure the current conv operator not to perform quantization during quantization; Step 4, perform model quantization processing according to the configuration of step 3 to generate a quantized model; Step 5, load the same true value data, compare the output results of the quantized model with the model under the original precision bit width, and compare the result accuracy: if the accuracy requirements are met, perform model platform adaptation conversion and load it to the device for use; if the accuracy does not meet the requirements, relocate the cause of the quantization error and the corresponding conv operator for algorithm optimization processing or release it to the original precision bit width for accuracy alignment to meet the application accuracy requirements.

2. A model compression outlier elimination method as claimed in claim 1, characterized in that: In step 1, during preprocessing, the image data is subjected to mean subtraction and normalization processing; and the point cloud data is subjected to voxelization processing.

3. A model compression outlier elimination method as claimed in claim 1, characterized in that: In step 301, the weight data W q The calculation is done using the following formula: , where: W' is the weight data after equalization, W'=W×E, E is the equalization factor, which is calculated based on the statistical characteristics of the weight data W; S w ' is the scaling factor after weight quantization and equalization, S w =S W ×E,S W is the quantitative factor; w min and w max are the minimum and maximum values ​​of the quantization range respectively; clip means limiting the data range to [w min ,w max ]; adaround refers to a quantization algorithm that minimizes the quantization error by adjusting the rounding method of the weights.

4. A model compression outlier elimination method as claimed in claim 3, characterized in that: The equalization factor E is the standard deviation of the weight data W.

5. A model compression outlier elimination method as claimed in claim 3, characterized in that: The quantization factor S W The following formula is used for calculation: , where w fp32_max 、w fp32_min They are respectively the maximum and minimum values ​​of the weight data W under the original precision.

6. A model compression outlier elimination method as claimed in claim 4, characterized in that: In step 301, the input data X q The calculation is done using the following formula: , where X' is the input data after equalization, X'=X / E; S X is the scaling factor.

7. A model compression outlier elimination method as claimed in claim 6, characterized in that: The scaling factor S X The calculation is done using the following formula: , where: X float32_max is the maximum value of the optimal data distribution of the input data X under the original precision bit width, X float32_min It is the minimum value of the optimal data distribution of the input data X under the original precision bit width.

8. A model compression outlier elimination method according to claim 7, characterized in that: The step 303 includes the following steps: Step 303-1, obtaining the output result Y of the original precision bit width of the current conv operator f : Y f = conv(X,W); Step 303-2, weight data W q and input data X q Bring it into the current conv operator, calculate the quantization result of the balanced quantization, and then dequantize the quantization result to obtain the dequantized output result Y q : Y q = S X ×S w '×conv(X q ,W q ); Step 303-3, calculate the inverse quantization output result Y q And the output result Y f Error ∆Y e :∆Y e =cosine_sim(Y f -Y q ), where cosine_sim represents cosine similarity; Step 303-4, if the error ∆Y e If the value is less than the preset threshold, it indicates that the accuracy of the current conv operator after equalization quantization exceeds the error range.

9. A model compression outlier elimination method as claimed in claim 1, characterized in that: It also includes verifying and evaluating the accuracy of the quantized model obtained in step 4 on the application platform side to confirm whether it meets the accuracy requirements of the application side.

Images