Model quantization method and apparatus, computer device, and storage medium

Abstract

The application relates to a model quantization method and device, computer equipment and a storage medium. The method comprises the following steps: determining a target quantization strategy for each original network layer in an original network model from at least two groups of candidate quantization strategies according to the corresponding quantization error of each original network layer in the original network model under the at least two groups of candidate quantization strategies; determining a network layer to be quantized in the original network model according to the corresponding quantization error of each original network layer under the target quantization strategy; and performing quantization processing on the original network model according to the target quantization strategy of the network layer to be quantized, so as to obtain a target quantization model. That is, the target quantization strategy is first selected for each original network layer in the original network model, and then the network layer to be quantized in the original network model is selected. Compared with the fixed-bit-based quantization processing on each original network layer in the original network model, the balance between the accuracy and the calculation resource occupation of the target quantization model is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model quantization method, apparatus, computer device, and storage medium. Background Technology

[0002] In recent years, quantization processing of neural network models has received widespread attention.

[0003] Currently, the quantization method for high-precision floating-point neural network models (i.e., high-precision original network models) involves quantizing all weights and activation values ​​of the original network model to a fixed number of bits, resulting in a low-precision integer model (the target quantized model). If the fixed number of bits is high, while this ensures the accuracy of the quantized target model, it leads to high memory usage and computational complexity. Conversely, if the fixed number of bits is low, while this ensures low memory usage and computational complexity, it results in low accuracy.

[0004] It is evident that in existing technologies, quantizing the original network model based on fixed bits is difficult to achieve a good balance between accuracy and computational resource consumption, and improvements are urgently needed. Summary of the Invention

[0005] Therefore, it is necessary to provide a model quantization method, apparatus, computer equipment, and storage medium that can balance the relationship between accuracy and computational resource consumption to address the above-mentioned technical problems.

[0006] Firstly, this application provides a model quantization method, which includes:

[0007] Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies.

[0008] Based on the quantization error of each original network layer under the target quantization strategy, determine the network layers to be quantized in the original network model;

[0009] Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0010] In one embodiment, based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies, including:

[0011] Based on each original network layer in the original network model, at least two sets of candidate quantization strategies are used for quantization to obtain at least two quantized network layers corresponding to each original network layer.

[0012] Determine the quantization error between each original network layer and each quantized network layer of that original network layer;

[0013] The candidate quantization strategy corresponding to the minimum quantization error of each original network layer is used as the target quantization strategy for each original network layer.

[0014] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of that original network layer includes:

[0015] Inference is performed on each original network layer based on the sample input data to obtain the first inference result;

[0016] Based on the sample input data, inference is performed on each quantized network layer of each original network layer to obtain the second inference result;

[0017] Based on the first and second inference results of each original network layer, the quantization error between each original network layer and each quantized network layer of that original network layer is determined.

[0018] In one embodiment, the quantization error between each original network layer and each quantized network layer of that original network layer is determined based on the first inference result and the second inference result of each original network layer, including:

[0019] The second inference result of each quantized network layer in each original network layer is dequantized to obtain the third inference result of each original network layer.

[0020] Determine the similarity between the first and third inference results of each original network layer;

[0021] Based on similarity, the quantization error between each original network layer and each quantized network layer of that original network layer is determined.

[0022] In one embodiment, determining the network layers to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy includes:

[0023] Candidate quantization layers are determined from each original network layer of the original network model;

[0024] Determine whether the candidate quantization model meets the quantization index requirements; wherein, the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer.

[0025] If the requirements are not met, the candidate quantization layers are adjusted based on the quantization error of each original network layer under the target quantization strategy. The candidate quantization model is then re-evaluated based on the adjusted candidate quantization layers to determine whether it meets the quantization index requirements, until the quantization index requirements are met.

[0026] In one embodiment, determining candidate quantization layers from the original network layers of the original network model includes:

[0027] All original network layers in the original network model are used as candidate quantization layers;

[0028] Accordingly, based on the quantization error of each original network layer under the target quantization strategy, the candidate quantization layers are adjusted, including:

[0029] Based on the quantization error of each original network layer under the target quantization strategy, the original network layer with the largest quantization error is removed from the candidate quantization layers.

[0030] In one embodiment, the original network model is an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0031] Secondly, this application also provides a model quantization device. The device includes:

[0032] The first determining module is used to determine a target quantization strategy for each original network layer from at least two sets of candidate quantization strategies based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies.

[0033] The second determining module is used to determine the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy.

[0034] The quantization module is used to quantize the original network model according to the target quantization strategy of the network layer to be quantized, so as to obtain the target quantized model.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0036] Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies.

[0037] Based on the quantization error of each original network layer under the target quantization strategy, determine the network layers to be quantized in the original network model;

[0038] Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0039] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0040] Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies.

[0041] Based on the quantization error of each original network layer under the target quantization strategy, determine the network layers to be quantized in the original network model;

[0042] Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0043] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0044] Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies.

[0045] Based on the quantization error of each original network layer under the target quantization strategy, determine the network layers to be quantized in the original network model;

[0046] Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0047] The aforementioned model quantization method, apparatus, computer equipment, and storage medium determine a target quantization strategy for each original network layer in the original network model based on the quantization error corresponding to each of at least two sets of candidate quantization strategies. Then, based on the quantization error corresponding to each original network layer under the target quantization strategy, the network layers to be quantized in the original network model are determined. Finally, the original network model is quantized according to the target quantization strategy of the network layers to be quantized, resulting in the target quantized model. The scheme of this application first filters for target quantization strategies for each original network layer in the original network model, and then filters for the network layers to be quantized in the original network model. Compared with the existing technology that quantizes each original network layer in the original network model based on a fixed-bit method, this greatly improves the balance between accuracy and computational resource consumption in the target quantization model. Attached Figure Description

[0048] Figure 1 This embodiment provides an application environment diagram for a model quantization method.

[0049] Figure 2 This is a flowchart illustrating the first model quantization method provided in this embodiment;

[0050] Figure 3 This embodiment provides a flowchart illustrating the process of determining the target quantization strategy for each original network layer.

[0051] Figure 4 This embodiment provides a flowchart for determining the network layer to be quantized in the original network model.

[0052] Figure 5 This is a flowchart illustrating the second model quantization method provided in this embodiment;

[0053] Figure 6 This is a structural block diagram of the first model quantization device provided in this embodiment;

[0054] Figure 7 This is a structural block diagram of the second model quantization device provided in this embodiment;

[0055] Figure 8 This is a structural block diagram of the third model quantization device provided in this embodiment;

[0056] Figure 9 This embodiment provides an internal structural diagram of a computer device. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] This application provides a computer device, which may be a terminal, and its internal structure diagram may be as follows: Figure 1 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wireless-Fidelity (WIFI), mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a model quantization method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0059] In one embodiment, such as Figure 2 As shown, a model quantization method is provided, which includes the following steps:

[0060] S201. Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, determine the target quantization strategy for each original network layer from at least two sets of candidate quantization strategies.

[0061] Optionally, the original network model in this embodiment can be any high-precision floating-point type neural network model including at least two original network layers. For example, the original network model can be an object detection model, an image recognition model, and an image classification model.

[0062] A quantization strategy refers to the strategy used to quantize each original network layer in the original network model. Optionally, optional quantization methods and algorithms can be arbitrarily combined to obtain at least two sets of candidate quantization strategies, where different candidate quantization strategies contain different optional quantization methods and / or optional quantization algorithms. Optional quantization methods include symmetric quantization, asymmetric quantization, layer-by-layer quantization, and channel-by-channel quantization. Optional quantization algorithms include the maximum-minimum method, the moving average maximum-minimum method, and relative entropy (Kullback-Leibler Divergence, KL).

[0063] Optionally, quantization error refers to the error between the inference result of any original network layer on a set of data and the inference result of the original network layer on the same set of data after quantization. In practical applications, the smaller the quantization error, the smaller the difference between the inference result of the original network layer on the set of data and the inference result of the original network layer on the same set of data after quantization. In other words, the higher the similarity between the inference result of the original network layer on the set of data and the inference result of the original network layer on the same set of data after quantization, and vice versa.

[0064] The target quantization strategy refers to the optimal quantization strategy selected for each original network layer. For example, it could be the candidate quantization strategy corresponding to the minimum quantization error.

[0065] In this embodiment, for each original network layer in the original network model, each candidate quantization strategy is traversed to determine each quantization network layer that quantizes each original network layer under each candidate quantization strategy. Then, based on the same set of data, the quantization error between the inference result of an original network layer on the set of data and the inference result of each quantization network layer on the set of data is determined. The quantization errors corresponding to each quantization network layer are sorted, and the candidate quantization strategy corresponding to the smallest quantization error is determined as the target quantization strategy for each original network layer.

[0066] S202. Based on the quantization error of each original network layer under the target quantization strategy, determine the network layers to be quantized in the original network model.

[0067] Among them, the network layer to be quantized is the original network layer in the original network model that needs to be quantized.

[0068] In this embodiment, for each original network layer in the original network model, the quantization error corresponding to each quantized network layer under the target quantization strategy is determined, and the quantization errors corresponding to each quantized network layer are sorted in ascending order to obtain the quantization error arrangement order. Based on this arrangement order, the network layer to be quantized in the original network model is determined.

[0069] In one optional implementation, a preset error threshold is set, and based on the order of quantization errors, each original network layer whose quantization error is less than the preset error threshold is taken as a network layer to be quantized. It should be noted that the error threshold can be determined according to the accuracy or speed requirements of the quantized original network model, and the method of determining the error threshold is not limited here.

[0070] In another optional implementation, a maximum permissible error value after quantization of the original network model is preset. Based on the ascending order of quantization errors corresponding to each quantized network layer, the quantization errors are sequentially accumulated starting from the first quantization error until the total accumulated value exceeds the maximum permissible error value. All original network layers with error values ​​not exceeding the maximum permissible error value are then considered as the network layers to be quantized. It should be noted that the maximum permissible error value can be determined based on requirements such as accuracy or speed after quantization of the original network model; the method for determining the maximum permissible error value is not limited here.

[0071] Optionally, to further ensure the quantization effect of the original network model, after determining the network layer to be quantized in the above manner, it is also possible to further check whether the accuracy or speed of the original network model obtained after quantizing the network layer to be quantized based on the above quantization strategy meets the standards. If not, it is necessary to further adjust the network layer to be quantized in the original network model based on the quantization error.

[0072] S203. Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0073] After identifying the network layers to be quantized in the original network model, for each original network layer, it is first determined whether the original network layer is the network layer to be quantized. If the original network layer is determined to be the network layer to be quantized, the original network model is quantized based on the target quantization strategy corresponding to each network layer to be quantized, and the target quantized model is obtained.

[0074] The model quantization method provided in this application determines a target quantization strategy for each original network layer from at least two sets of candidate quantization strategies based on the quantization error of each original network layer under the target quantization strategy. Then, based on the quantization error of each original network layer under the target quantization strategy, the network layers to be quantized in the original network model are determined. Finally, the original network model is quantized according to the target quantization strategy of the network layers to be quantized to obtain the target quantized model. Clearly, this application's scheme first screens for target quantization strategies for each original network layer in the original network model, and then screens for the network layers to be quantized in the original network model. Compared to existing technologies that quantize each original network layer based on a fixed-bit method, this greatly improves the balance between accuracy and computational resource consumption in the target quantized model.

[0075] Optionally, in this embodiment, the original network model can be an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0076] The system comprises three layers: a point cloud feature extraction layer, a point cloud feature fusion layer, and a target detection layer. The point cloud feature extraction layer extracts target features from the point cloud data. The target detection layer detects the target features from the point cloud data fused by the point cloud feature fusion layer.

[0077] In this embodiment, assuming the original network model is an object detection model, the object detection model can be quantized by quantizing the three original network layers: the point cloud feature extraction layer, the point cloud feature fusion layer, and the object detection layer.

[0078] To understand the target quantization strategy for each original network layer, this embodiment provides an optional method for determining the target quantization strategy for each original network layer, such as... Figure 3 As shown, this method includes the following steps:

[0079] S301. Based on each original network layer in the original network model, quantize using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer.

[0080] Specifically, for each original network layer, candidate quantization strategies are iterated sequentially to obtain the corresponding quantization network layer under each quantization strategy. In other words, for each original network layer, multiple quantization network layers corresponding to multiple quantization strategies can be obtained.

[0081] Specifically, at least two sets of candidate quantization strategies are obtained by arbitrarily combining the optional quantization methods and algorithms. For each original network layer in the original network model, at least two sets of candidate quantization strategies are used to quantize the point cloud feature extraction layer, resulting in at least two quantized network layers. Assuming the original network layer is a point cloud feature extraction layer, arbitrarily combining the optional quantization methods and algorithms yields 10 sets of candidate quantization strategies. These 10 sets of candidate quantization strategies can be used to quantize the point cloud feature extraction layer, resulting in 10 quantized network layers corresponding to the point cloud feature extraction layer.

[0082] S302. Determine the quantization error between each original network layer and each quantized network layer of that original network layer.

[0083] Specifically, this step can be implemented through the following sub-steps to determine the quantization error between each original network layer and each quantized network layer of that original network layer:

[0084] Sub-step 1: Perform inference on each original network layer based on the sample input data to obtain the first inference result.

[0085] In this embodiment, the sample input data can be the data used for data inference by each original network layer in any original network model. Optionally, the sample input data in each original network layer of the original network model can be the same or different.

[0086] Specifically, a single-operator model can be constructed for each original network layer in the original network model. Sample input data is input into the single-operator model, which then performs inference on the input sample input data to obtain a first inference result. Since the original network layers are not quantized, this first inference result is represented in floating-point form. Optionally, the first inference result can be the activation values ​​output by the original network layers.

[0087] For example, when the original network model is an object detection model, the sample input data for the point cloud feature extraction layer is the original point cloud data. The point cloud feature extraction layer infers from the original point cloud data to obtain its inference result. The sample input data for the point cloud feature fusion layer is point cloud feature data, which is the inference result of the point cloud feature extraction layer. The point cloud feature fusion layer infers from the inference result of the point cloud feature extraction layer to obtain its own inference result. The sample input data for the object detection layer is point cloud fused feature data, which can be the inference result of the point cloud feature fusion layer. The object detection layer infers from the inference result of the point cloud feature fusion layer to obtain its own inference result.

[0088] Sub-step two: Based on the sample input data, perform inference on each quantized network layer of each original network layer to obtain the second inference result.

[0089] In this embodiment, each quantized network layer corresponds to a second inference result for each original network layer. The second inference result can be the activation value output by the quantized network layer. Since the quantized network layer is obtained by quantizing the original network layer, the second inference result is represented in integer form.

[0090] For each original network layer, multiple quantized network layers corresponding to multiple quantization strategies can be obtained. The sample input data is input into each corresponding quantized network layer (quantized single operator model) of each original network layer. The quantized network layer performs inference on the input sample input data and obtains the second inference result.

[0091] It should be noted that, to ensure the accuracy of the quantization error between the inference of each original network layer on the input sample data and the inference of the corresponding quantized network layer on the input sample data, the sample input data for each quantized network layer of each original network layer in the original network model is the same as the sample input data for each original network layer in the original network model. In other words, the sample input data for inference performed by each original network layer is the same as the sample input data for inference performed by the corresponding quantized network layers.

[0092] Sub-step 3: Based on the first and second inference results of each original network layer, determine the quantization error between each original network layer and each quantized network layer of that original network layer.

[0093] Optionally, in this embodiment, the difference between the first inference result of each original network layer and the second inference result corresponding to each quantized network layer of the original network layer can be directly calculated, and the difference can be used as the quantization error between each original network layer and each quantized network layer of the original network layer.

[0094] In this embodiment, a similarity algorithm can also be used to determine the similarity between the first inference result of each original network layer and the second inference result of the corresponding quantized network layer. Based on this similarity, the quantization error between each original network layer and each quantized network layer of that original network layer can be determined. It should be noted that similarity and quantization error are inversely proportional; the greater the similarity, the smaller the quantization error.

[0095] To accurately determine the quantization error between each original network layer and each quantized network layer of that original network layer, the specific implementation process of sub-step three above includes:

[0096] The second inference result of each quantized network layer in each original network layer is dequantized to obtain the third inference result of each original network layer.

[0097] Determine the similarity between the first and third inference results of each original network layer;

[0098] Based on similarity, the quantization error between each original network layer and each quantized network layer of that original network layer is determined.

[0099] Specifically, since the first inference result of each original network layer is represented in floating-point form, and the second inference result of each quantized network layer of each original network layer is represented in integer form, in order to accurately determine the quantization error between the inference of each original network layer on the input sample data and the inference of each quantized network layer on the input sample data, the second inference results corresponding to each quantized network layer of each original network layer can be dequantized to obtain at least two third inference results for each original network layer. These third inference results are represented in floating-point form, and optionally, they can be the activation values ​​output by each quantized network layer.

[0100] Specifically, formula (1) is used to calculate the similarity between the first inference result of each original network layer and the third inference result of each quantized network layer corresponding to that original network layer.

[0101]

[0102] Here, sqrt() is the function that calculates the square root, x i For each original network layer, after quantization, the third inference result of each quantized network layer is y. i This represents the first inference result for each original network layer, where N is the number of dimensions and S is the similarity value.

[0103] After obtaining the similarity between the first inference result corresponding to each original network layer and the third inference result of each quantized network layer corresponding to the original network layer, the quantization error between each original network layer and each quantized network layer of the original network layer is determined based on the similarity.

[0104] S303. The candidate quantization strategy corresponding to the minimum quantization error of each original network layer is taken as the target quantization strategy of each original network layer.

[0105] After determining the quantization error between each original network layer and each quantized network layer of that original network layer, the candidate quantization strategy of the quantized network layer corresponding to the minimum quantization error of each original network layer is taken as the target quantization strategy of each original network layer.

[0106] The original network layer described above is a point cloud feature extraction layer. Ten candidate quantization strategies can be used to quantize this point cloud feature extraction layer to obtain ten quantized network layers.

[0107] Suppose that the point cloud feature extraction layer infers from the input sample data A, obtaining a first inference result A1. The first set of candidate quantization strategies quantizes this point cloud feature extraction layer to obtain a first quantization network layer. The first quantization network layer infers from the input sample data A, obtaining a second inference result B1. The second inference result B1 is then dequantized to obtain a third inference result C1. The similarity between the first inference result A1 and the third inference result C1 is calculated, assuming a similarity of 90%. The second set of candidate quantization strategies is then applied to quantize this point cloud feature extraction layer to obtain a second quantization network layer. The second quantization network layer infers from the input sample data B, obtaining a second inference result B2. The second inference result B2 is then dequantized to obtain a third inference result C2. The similarity between the first inference result A1 and the third inference result C2 is calculated, assuming a similarity of 95%. Each quantization network layer is executed sequentially to infer the sample input data X, obtaining a second inference result Xn. The second inference result Xn is then dequantized to obtain a third inference result Cn. The similarity between the first inference result A1 and the third inference result Cn is calculated, until the 10th similarity is obtained, where n is a positive integer from 1 to 10. Assuming the 10 candidate quantization strategies correspond to similarities of 90%, 95%, 70%, 60%, 50%, 80%, 90%, 95%, 88%, and 97% respectively, the candidate quantization strategy corresponding to the maximum similarity value of 97% can be used as the target quantization strategy for the point cloud feature extraction layer.

[0108] In this embodiment, a first inference result is obtained by inferring from the sample input data by each original network layer, and a second inference result is obtained by inferring from the sample input data by each quantized network layer. Based on the first and second inference results, the quantization error between each original network layer and its respective quantized network layers is determined. To more accurately determine the quantization error between each quantized network layer, the second inference result of each quantized network layer of each original network layer can be dequantized to obtain a third inference result for each original network layer. Based on the similarity between the first and third inference results, the quantization error between each original network layer and its respective quantized network layers is accurately determined. The candidate quantization strategy corresponding to the minimum quantization error is then used as the target quantization strategy for each original network layer, thus accurately determining the target quantization strategy for each original network layer.

[0109] To further ensure the accuracy of determining the network layer to be quantized from the original network model, this embodiment provides an optional method for determining the network layer to be quantized in the original network model. Specifically, such as... Figure 4 As shown.

[0110] S401. Determine candidate quantization layers from each original network layer of the original network model.

[0111] Optionally, all original network layers in the original network model can be used as candidate quantization layers. Alternatively, based on the maximum permissible error value, each original network layer in the original network model whose quantization error is no greater than the maximum permissible error value can be used as a candidate quantization layer. The maximum permissible error value is determined based on requirements such as accuracy or speed after quantization of the original network model; the method for determining the maximum permissible error value is not limited here. Alternatively, only the original network layer with the smallest quantization error in the original network model can be used as a candidate quantization layer.

[0112] Assuming the original network consists of 5 layers, the quantization error between the first layer and its corresponding quantized layer is 10%. For the second layer, the quantization error is 50%. Similarly, the quantization error between the third layer and its corresponding quantized layer is 20%, the fourth layer is 5%, and the fifth layer is 30%.

[0113] In this embodiment, all five network layers can be used as candidate quantization layers. Alternatively, assuming a maximum allowable error of 20%, the original network layers with an error not exceeding 20%, namely the first, third, and fourth network layers, can be used as candidate quantization layers. Alternatively, only the fourth network layer corresponding to the minimum quantization error of 5% can be used as a candidate quantization layer.

[0114] S402. Determine whether the candidate quantization model meets the quantization index requirements. If it does not meet the requirements, proceed to S403; otherwise, proceed to S404.

[0115] Among them, the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer.

[0116] Optionally, the quantification metrics may include metrics for inference accuracy and / or inference speed. That is, quantification metrics may include inference accuracy, inference speed, or both.

[0117] In this embodiment, after determining candidate quantization layers from each original network layer, each candidate quantization layer in the original network model can be quantized based on the target quantization strategy corresponding to each candidate quantization layer to obtain a candidate quantization model. The candidate quantization model is then tested for inference accuracy and / or inference speed based on test data; and based on the test results, it is determined whether the candidate quantization model meets the quantization index requirements for inference accuracy and / or inference speed.

[0118] If the candidate quantization model does not meet the quantization index requirements for inference accuracy and / or inference speed, the candidate quantization layer needs to be adjusted, and S403 is executed in this case.

[0119] If the candidate quantization model meets the quantization index requirements for inference accuracy and / or inference speed, then there is no need to adjust the candidate quantization layer, and S404 is executed.

[0120] S403. Based on the quantization error of each original network layer under the target quantization strategy, adjust the candidate quantization layers, and re-evaluate whether the candidate quantization model meets the quantization index requirements based on the adjusted candidate quantization layers, until the quantization index requirements are met.

[0121] Optionally, if S401 treats all original network layers in the original network model as candidate quantization layers, then the original network layer with the largest quantization error can be removed from the candidate quantization layers based on the quantization error of each original network layer under the target quantization strategy.

[0122] Specifically, if all original network layers in the original network model are considered as candidate quantization layers, or if each original network layer in the original network model whose quantization error is no greater than the maximum allowable error value is considered as a candidate quantization layer, the quantization errors of each candidate quantization layer can be sorted from smallest to largest, and the original network layer with the largest quantization error can be removed from the candidate quantization layers. This is because the original network layer with the largest quantization error has the lowest similarity between the inference result of the quantized network layer and the inference result of the original network layer on the sample input data after quantization, which greatly affects the inference accuracy and / or inference speed of the candidate quantization model. Therefore, when it is determined that the inference accuracy and / or inference speed of the candidate quantization model does not meet the quantization index requirements for inference accuracy, the original network layer with the largest quantization error needs to be removed from the candidate quantization layers, and the operation of S402 should be executed again based on the adjusted candidate quantization layers until the quantization index requirements are met.

[0123] Assuming the quantization errors of the five network layers in the original network layer are 10%, 50%, 20%, 5%, and 30%, respectively, all five network layers are used as candidate quantization layers. After quantization based on the target quantization strategy corresponding to each of the five network layers, candidate quantization models are obtained. It is then determined whether the candidate quantization models meet the quantization index requirements. If a candidate quantization model does not meet the quantization index requirements, the second network layer corresponding to the maximum quantization error of 50% can be removed from the candidate quantization layers. Based on the first, third, fourth, and fifth network layers, the candidate quantization models are re-evaluated to see if they meet the quantization index requirements. This process of adjusting the candidate quantization layers and re-evaluating whether the candidate quantization models meet the quantization index requirements is repeated until the quantization index requirements are met.

[0124] Optionally, if S401 selects the original network layer with the smallest quantization error in the original network model as the candidate quantization layer, then the original network layer with the smallest quantization error among the other original network layers besides the candidate quantization layer can be added to the candidate quantization network layer, and the candidate quantization model can be re-evaluated based on the adjusted candidate quantization layer to determine whether it meets the quantization index requirements.

[0125] Specifically, after sorting the quantization errors of each candidate quantization layer from smallest to largest, the original network layer with the smallest quantization error is selected as the candidate quantization layer. This is because the original network layer with the smallest quantization error, after quantization, exhibits the highest similarity between the inference result of the quantized network layer and the inference result of the original network layer on the sample input data. This results in higher inference accuracy for the candidate quantization model, potentially far exceeding the quantization metric. Therefore, the original network layer with the smallest quantization error from the remaining original network layers can be added to the candidate quantization network layer list. The S402 operation is then executed based on the adjusted candidate quantization layer until the quantization metric requirements are met.

[0126] Assuming the quantization errors of the five original network layers are 10%, 50%, 20%, 5%, and 30%, respectively, the fourth network layer with the smallest quantization error of 5% is first selected as a candidate quantization layer. This fourth network layer is then quantized according to the target quantization strategy. The resulting candidate quantization model is then evaluated to determine if it meets the quantization metric requirements. If it does, the first network layer with the smallest quantization error of 10% (excluding the fourth layer) is added to the candidate quantization network layer. The adjusted first and fourth network layers are then used to re-evaluate whether the candidate quantization model meets the quantization metric requirements. This process of adjusting the candidate quantization layers and evaluating the candidate quantization model continues until the quantization metric requirements are met.

[0127] S404. Use the candidate quantization layer as the network layer to be quantized in the original network model.

[0128] It should be noted that, in this embodiment, if the candidate quantization model is determined to meet the quantization index requirements in S402, the candidate quantization model can be directly used as the target quantization model, and the target quantization model and the target quantization strategy corresponding to each original network layer can be output.

[0129] In this embodiment, each candidate quantization layer can be adjusted. That is, when the quantization error between the inference result of each quantized network layer on the sample input data and the inference result of the original network layer on the sample input data before quantization is large, the quantization of the original network layer can be canceled, thereby avoiding problems such as low accuracy or slow calculation speed caused by quantization. Therefore, this application can well balance the relationship between accuracy and computational resource consumption during the quantization process.

[0130] Based on the above embodiments, this embodiment provides an optional implementation of the model quantization method, such as... Figure 5 As shown, the method includes the following steps:

[0131] S501. Based on the sample input data, perform inference on each original network layer to obtain the first inference result.

[0132] S502. Based on each original network layer in the original network model, quantize using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer.

[0133] S503. Based on the sample input data, inference is performed on each quantized network layer of each original network layer to obtain the second inference result.

[0134] S504. Perform dequantization on the second inference result of each quantized network layer of each original network layer to obtain the third inference result of each original network layer.

[0135] S505. Determine the similarity between the first inference result and the third inference result of each original network layer.

[0136] S506. Based on the similarity, determine the quantization error between each original network layer and each quantized network layer of that original network layer.

[0137] S507. The candidate quantization strategy corresponding to the minimum quantization error of each original network layer is taken as the target quantization strategy of each original network layer.

[0138] S508. All original network layers in the original network model are used as candidate quantization layers.

[0139] S509. The original network model is quantized based on the target quantization strategy of the candidate quantization layer to obtain the candidate quantization model.

[0140] S510. Determine whether the candidate quantization model meets the quantization index requirements; if not, execute S511; if yes, execute S512.

[0141] S511. Based on the quantization error of each original network layer under the target quantization strategy, remove the original network layer with the largest quantization error from the candidate quantization layers, and return to execute the operation of S509 based on the removed candidate quantization layers.

[0142] S512. The candidate quantization layer is used as the network layer to be quantized in the original network model.

[0143] S513. Based on the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

[0144] It should be noted that, in this embodiment, if the candidate quantization model is determined to meet the quantization index requirements in S510, the candidate quantization model can be directly used as the target quantization model, and the target quantization model and the target quantization strategy corresponding to each original network layer can be output.

[0145] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0146] Based on the same inventive concept, this application also provides a model quantization apparatus for implementing the model quantization method described above. The solution provided by this apparatus is similar to the implementation scheme and beneficial effects described in the above method. Therefore, the specific limitations of one or more embodiments of a model quantization apparatus provided below can be found in the limitations of the model quantization method described above, and will not be repeated here.

[0147] In one embodiment, such as Figure 6 As shown, a model quantization device is provided, comprising:

[0148] The first determining module 601 is used to determine a target quantization strategy for each original network layer from at least two candidate quantization strategies based on the quantization error corresponding to each original network layer in the original network model under at least two candidate quantization strategies.

[0149] The second determining module 602 is used to determine the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy.

[0150] The quantization module 603 is used to quantize the original network model according to the target quantization strategy of the network layer to be quantized, so as to obtain the target quantized model.

[0151] In one embodiment, such as Figure 7 As shown, above Figure 6 The first determining module 601 in the middle also includes:

[0152] The quantization unit 6011 is used to quantize each original network layer in the original network model using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer.

[0153] The first determining unit 6012 is used to determine the quantization error between each original network layer and each quantized network layer of the original network layer.

[0154] The second determining unit 6013 is used to take the candidate quantization strategy corresponding to the minimum quantization error of each original network layer as the target quantization strategy of each original network layer.

[0155] In one embodiment, the first determining unit 6012 includes:

[0156] The first inference subunit is used to perform inference on each original network layer based on the sample input data to obtain the first inference result;

[0157] The second inference subunit is used to infer each quantized network layer of each original network layer based on the sample input data to obtain the second inference result.

[0158] The first determining subunit is used to determine the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer.

[0159] In one embodiment, the first determining subunit is specifically used for:

[0160] The second inference result of each quantized network layer in each original network layer is dequantized to obtain the third inference result of each original network layer.

[0161] Determine the similarity between the first and third inference results of each original network layer;

[0162] Based on similarity, the quantization error between each original network layer and each quantized network layer of that original network layer is determined.

[0163] In one embodiment, such as Figure 8 As shown above, in the above Figure 6-7 Based on any one of them, the second determining module 602,

[0164] The third determining unit 6021 is used to determine candidate quantization layers from each original network layer of the original network model;

[0165] The judgment unit 6022 is used to determine whether the candidate quantization model meets the quantization index requirements; wherein, the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer.

[0166] The adjustment unit 6023 is used to adjust the candidate quantization layer according to the quantization error corresponding to each original network layer under the target quantization strategy when the quantization index requirements are not met, and return to the operation of the execution judgment unit 6022 based on the adjusted candidate quantization layer until the quantization index requirements are met.

[0167] In one embodiment, the third determining unit 6021 is specifically used to: take all the original network layers in the original network model as candidate quantization layers;

[0168] Accordingly, the adjustment judgment unit 6023 is specifically used to: remove the original network layer with the largest quantization error from the candidate quantization layer based on the quantization error corresponding to each original network layer under the target quantization strategy.

[0169] In one embodiment, the original network model is an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0170] Each module in the aforementioned model quantization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0171] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a model quantization method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0172] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: determining a target quantization strategy for each original network layer from at least two sets of candidate quantization strategies based on the quantization error corresponding to each original network layer under at least two sets of candidate quantization strategies; determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy; and quantizing the original network model according to the target quantization strategy of the network layer to be quantized to obtain a target quantized model.

[0174] In one embodiment, based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies. This includes: quantizing each original network layer in the original network model using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer; determining the quantization error between each original network layer and each quantized network layer of the original network layer; and using the candidate quantization strategy corresponding to the minimum quantization error of each original network layer as the target quantization strategy for each original network layer.

[0175] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer includes: performing inference on each original network layer based on sample input data to obtain a first inference result; performing inference on each quantized network layer of the original network layer based on sample input data to obtain a second inference result; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer.

[0176] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer includes: performing dequantization processing on the second inference result of each quantized network layer of the original network layer to obtain the third inference result of each original network layer; determining the similarity between the first inference result and the third inference result of each original network layer; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the similarity.

[0177] In one embodiment, determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy includes: determining candidate quantization layers from each original network layer of the original network model; determining whether the candidate quantization model meets the quantization index requirements; wherein the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer; if it does not meet the requirements, the candidate quantization layer is adjusted according to the quantization error corresponding to each original network layer under the target quantization strategy, and the candidate quantization model is re-determined based on the adjusted candidate quantization layer until the quantization index requirements are met.

[0178] In one embodiment, determining candidate quantization layers from each original network layer of the original network model includes: taking all original network layers in the original network model as candidate quantization layers; correspondingly, adjusting the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy includes: removing the original network layer with the largest quantization error from the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy.

[0179] In one embodiment, the original network model is an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0180] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon. When executed by a processor, the computer program performs the following steps: determining a target quantization strategy for each original network layer from at least two sets of candidate quantization strategies based on the quantization error corresponding to each original network layer under at least two sets of candidate quantization strategies; determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy; and quantizing the original network model according to the target quantization strategy of the network layer to be quantized to obtain a target quantized model.

[0181] In one embodiment, based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies. This includes: quantizing each original network layer in the original network model using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer; determining the quantization error between each original network layer and each quantized network layer of the original network layer; and using the candidate quantization strategy corresponding to the minimum quantization error of each original network layer as the target quantization strategy for each original network layer.

[0182] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer includes: performing inference on each original network layer based on sample input data to obtain a first inference result; performing inference on each quantized network layer of the original network layer based on sample input data to obtain a second inference result; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer.

[0183] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer includes: performing dequantization processing on the second inference result of each quantized network layer of the original network layer to obtain the third inference result of each original network layer; determining the similarity between the first inference result and the third inference result of each original network layer; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the similarity.

[0184] In one embodiment, determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy includes: determining candidate quantization layers from each original network layer of the original network model; determining whether the candidate quantization model meets the quantization index requirements; wherein the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer; if it does not meet the requirements, the candidate quantization layer is adjusted according to the quantization error corresponding to each original network layer under the target quantization strategy, and the candidate quantization model is re-determined based on the adjusted candidate quantization layer until the quantization index requirements are met.

[0185] In one embodiment, determining candidate quantization layers from each original network layer of the original network model includes: taking all original network layers in the original network model as candidate quantization layers; correspondingly, adjusting the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy includes: removing the original network layer with the largest quantization error from the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy.

[0186] In one embodiment, the original network model is an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0187] This application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps: determining a target quantization strategy for each original network layer from at least two sets of candidate quantization strategies based on the quantization error corresponding to each original network layer under at least two sets of candidate quantization strategies; determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy; and quantizing the original network model according to the target quantization strategy of the network layer to be quantized to obtain a target quantized model.

[0188] In one embodiment, based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from at least two sets of candidate quantization strategies. This includes: quantizing each original network layer in the original network model using at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer; determining the quantization error between each original network layer and each quantized network layer of the original network layer; and using the candidate quantization strategy corresponding to the minimum quantization error of each original network layer as the target quantization strategy for each original network layer.

[0189] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer includes: performing inference on each original network layer based on sample input data to obtain a first inference result; performing inference on each quantized network layer of the original network layer based on sample input data to obtain a second inference result; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer.

[0190] In one embodiment, determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the first inference result and the second inference result of each original network layer includes: performing dequantization processing on the second inference result of each quantized network layer of the original network layer to obtain the third inference result of each original network layer; determining the similarity between the first inference result and the third inference result of each original network layer; and determining the quantization error between each original network layer and each quantized network layer of the original network layer based on the similarity.

[0191] In one embodiment, determining the network layer to be quantized in the original network model based on the quantization error corresponding to each original network layer under the target quantization strategy includes: determining candidate quantization layers from each original network layer of the original network model; determining whether the candidate quantization model meets the quantization index requirements; wherein the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer; if it does not meet the requirements, the candidate quantization layer is adjusted according to the quantization error corresponding to each original network layer under the target quantization strategy, and the candidate quantization model is re-determined based on the adjusted candidate quantization layer until the quantization index requirements are met.

[0192] In one embodiment, determining candidate quantization layers from each original network layer of the original network model includes: taking all original network layers in the original network model as candidate quantization layers; correspondingly, adjusting the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy includes: removing the original network layer with the largest quantization error from the candidate quantization layers according to the quantization error corresponding to each original network layer under the target quantization strategy.

[0193] In one embodiment, the original network model is an object detection model, and the original network layers in the object detection model include: a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer.

[0194] It should be noted that all data involved in this application (including but not limited to data used for analysis, stored data, and displayed data) are data authorized by the user or fully authorized by all parties.

[0195] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0196] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model quantization method, characterized in that, The method includes: Based on the quantization error of each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from the at least two sets of candidate quantization strategies; wherein, the original network model is an object detection model, and the original network layers in the object detection model include a point cloud feature extraction layer, a point cloud feature fusion layer, and an object detection layer; different candidate quantization strategies include different quantization methods and / or quantization algorithms; the quantization methods include symmetric quantization, asymmetric quantization, layer-by-layer quantization, or channel-by-channel quantization; the quantization algorithms include the maximum-minimum method, the moving average maximum-minimum method, or relative entropy; Candidate quantization layers are determined from each original network layer of the original network model; Determine whether the candidate quantization model meets the quantization index requirements; wherein, the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer; If not, based on the quantization error of each original network layer under the target quantization strategy, the original network layer with the largest quantization error is removed from the candidate quantization layer, and the candidate quantization model is re-evaluated based on the removed candidate quantization layer until the quantization index requirement is met. The candidate quantization layer that meets the quantization index requirements is determined as the network layer to be quantized in the original network model. According to the target quantization strategy of the network layer to be quantized, the original network model is quantized to obtain the target quantized model.

2. The method according to claim 1, characterized in that, Based on the quantization error corresponding to each original network layer in the original network model under at least two sets of candidate quantization strategies, a target quantization strategy is determined for each original network layer from the at least two sets of candidate quantization strategies, including: Based on each original network layer in the original network model, quantization is performed using the at least two sets of candidate quantization strategies to obtain at least two quantized network layers corresponding to each original network layer. Determine the quantization error between each original network layer and each quantized network layer of that original network layer; The candidate quantization strategy corresponding to the minimum quantization error of each original network layer is taken as the target quantization strategy of each original network layer.

3. The method according to claim 2, characterized in that, Determining the quantization error between each original network layer and each quantized network layer of that original network layer includes: Inference is performed on each original network layer based on the sample input data to obtain a first inference result; Based on the sample input data, inference is performed on each quantized network layer of each original network layer to obtain a second inference result; Based on the first inference result and the second inference result of each original network layer, the quantization error between each original network layer and each quantized network layer of the original network layer is determined.

4. The method according to claim 3, characterized in that, Based on the first inference result and the second inference result of each original network layer, determine the quantization error between each original network layer and each quantized network layer of that original network layer, including: The second inference result of each quantized network layer of each original network layer is dequantized to obtain the third inference result of each original network layer. Determine the similarity between the first inference result and the third inference result of each original network layer; Based on the similarity, the quantization error between each original network layer and each quantized network layer of that original network layer is determined.

5. The method according to claim 3, characterized in that, The inference process performed on each original network layer based on sample input data to obtain a first inference result includes: Construct a single operator model for each of the original network layers; The sample input data is input into the single operator model corresponding to each original network layer to obtain the first inference result output by the single operator model on the sample input data.

6. The method according to any one of claims 1-5, characterized in that, The step of determining candidate quantization layers from each original network layer of the original network model includes: All original network layers in the original network model, or each original network layer in the original network model whose corresponding quantization error is not greater than the maximum allowable error value, are identified as candidate quantization layers.

7. A model quantization device, characterized in that, The device includes: The first determining module is used to determine a target quantization strategy for each original network layer in the original network model based on the quantization error corresponding to each original network layer under at least two sets of candidate quantization strategies. The original network model includes a target detection model, and the original network layers in the target detection model include a point cloud feature extraction layer, a point cloud feature fusion layer, and a target detection layer. Different candidate quantization strategies include different quantization methods and / or quantization algorithms. The quantization methods include symmetric quantization, asymmetric quantization, layer-by-layer quantization, or channel-by-channel quantization. The quantization algorithms include the maximum-minimum method, the moving average maximum-minimum method, or relative entropy. The second determining module is used to determine candidate quantization layers from each original network layer of the original network model; determine whether the candidate quantization model meets the quantization index requirements; wherein the candidate quantization model is obtained by quantizing the original network model based on the target quantization strategy of the candidate quantization layer; if it does not meet the requirements, according to the quantization error corresponding to each original network layer under the target quantization strategy, the original network layer with the largest quantization error is removed from the candidate quantization layers, and the candidate quantization model is re-determined based on the removed candidate quantization layers until the quantization index requirements are met; the candidate quantization layer that meets the quantization index requirements is determined as the network layer to be quantized in the original network model; The quantization module is used to quantize the original network model according to the target quantization strategy of the network layer to be quantized, so as to obtain the target quantized model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images