Network model dynamic quantization method and apparatus, terminal device, and readable storage medium

Abstract

The present application relates to the technical field of data processing. Disclosed are a network model dynamic quantization method and apparatus, a terminal device, and a readable storage medium. In the method, for all model parameters of a network model trained on the basis of point cloud data, a network search method is used to iteratively find a quantization scale factor that makes quantization errors of the network model smaller, so as to finally obtain a quantized network model. The method performs dynamic quantization on the basis of grid search, which can effectively improve the performance loss of models, which use point cloud data as input, after quantization, thereby facilitating the deployment of complex network models on devices with limited resources.

Description

Network model dynamic quantization method and device, terminal equipment and readable storage medium

[0001] The present application claims priority to the Chinese patent application No. 202411098734.8, filed on August 9, 2024 in the China Patent Office, and entitled "Network model dynamic quantization method and device, terminal equipment and readable storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of model quantization, in particular to a network model dynamic quantization method, device, terminal equipment and readable storage medium. BACKGROUND

[0003] In order to deploy neural network models on resource-constrained edge devices, model quantization technology is an effective solution. Model quantization is a way to compress network parameters, which can replace the originally floating-point represented values such as activation feature maps and weight parameters with fixed-point (integer) representation, and then quantize the fixed-point data back to floating-point data in the calculation process to obtain the result.

[0004] At present, the input of most neural network models is RGB (color) image data, and the most research on the quantization of such neural networks is also the most. However, for radar point cloud data, due to the difference in statistical distribution characteristics from RGB image data, when the quantization method that performs well in the model with RGB image data as input is directly migrated to the model with radar point cloud data as input, a large performance loss is easily caused, which brings new challenges to the deployment of detectors based on 3D laser radar on edge devices. TECHNICAL PROBLEM

[0005] Therefore, the embodiments of the present application provide a network model dynamic quantization method, device, terminal equipment and readable storage medium, which can effectively improve the performance loss of the quantized model. TECHNICAL SOLUTION

[0006] In a first aspect, the embodiments of the present application provide a network model dynamic quantization method, comprising:

[0007] obtaining all model parameters of a network model trained based on point cloud data and a quantization precision;

[0008] determining a parameter quantization range based on the quantization precision, and determining an initial parameter range based on all the model parameters;

[0009] determining a quantization truncation interval for the current search round based on the current parameter range and a preset grid search number, calculating a candidate quantization scaling factor for the current search round according to the quantization precision and the quantization truncation interval, and calculating quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range, and updating the initial parameter range or the parameter range of the previous search round using the quantization truncation interval of the current search round when the quantization error is less than a quantization error threshold;

[0010] determining a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each search round, and determining a quantized network model based on the target quantization scaling factor.

[0011] In some embodiments, the network model dynamic quantization method further comprises:

[0012] updating the current quantization error threshold using the quantization error of the current search round when the quantization error of the current search round is less than the quantization error threshold.

[0013] In some embodiments, the calculating of the quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range comprises:

[0014] calculating quantized values of each of the model parameters under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization values of each of the model parameters;

[0015] calculating the quantization errors of all the model parameters in the current search round according to the pre-quantization values and the quantized values of each of the model parameters.

[0016] In some embodiments, the calculating of the quantized values of each of the model parameters under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization values of each of the model parameters comprises:

[0017] clipping a ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor based on the parameter quantization range to obtain a clipping error;

[0018] calculating the quantized value of the corresponding model parameter under the candidate quantization scaling factor based on the clipping error and a zero-point offset during quantization;

[0019] When a symmetric quantization method is used, the zero-point offset is zero; and when an asymmetric quantization method is used, the zero-point offset is a non-zero value.

[0020] In some embodiments, the calculation formula of the quantized value is:

[0021] In the formula, x and represent the pre-quantization value and the post-quantization value of the corresponding model parameter; s represents the candidate quantization scaling factor, q min and q max represent the minimum value and the maximum value of the parameter quantization range; z is the zero-point offset, and the function clamp() represents clipping values that exceed the parameter quantization range.

[0022] In some embodiments, the determination of the quantization truncation interval in the current search based on the current parameter range and the set grid search number includes:

[0023] calculating the ratio of the maximum value in the current parameter range to the grid search number, and multiplying the ratio by the search sequence number to obtain the quantization truncation value in the current search;

[0024] selecting the positive and negative values of the quantization truncation value as two boundary values of the quantization truncation interval to obtain the quantization truncation interval in the current search.

[0025] In some embodiments, the calculation formula of the candidate quantization scaling factor is: s=(x max -x min ) / (2 b -1);

[0026] In the formula, s represents the candidate quantization scaling factor, x min and x max represent the lower boundary value and the upper boundary value of the quantization truncation interval, and b is the quantization precision.

[0027] In a second aspect, the embodiments of the present application provide a network model dynamic quantization device, which includes:

[0028] a data acquisition module configured to acquire all model parameters of a network model trained based on point cloud data and quantization precision;

[0029] a range determination module configured to determine a parameter quantization range based on the quantization precision and determine an initial parameter range based on all the model parameters;

[0030] the network search module is configured to, in each round of search, determine a quantization truncation interval in the current round of search based on the current parameter range and a set number of grid searches, calculate a candidate quantization scaling factor in the current round of search according to the quantization precision and the quantization truncation interval, and calculate quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range, and when the quantization errors are less than a quantization error threshold, update the initial parameter range or the parameter range in the previous round of search using the quantization truncation interval in the current round of search.

[0031] The model quantization module is configured to determine a target quantization scaling factor that minimizes the quantization errors according to the quantization errors obtained in each round of search, and determine a quantized network model based on the target quantization scaling factor.

[0032] In a third aspect, an embodiment of the present application provides a terminal device, which comprises a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the network model dynamic quantization method.

[0033] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed on a processor, implements the network model dynamic quantization method. Advantages

[0034] Embodiments of the present application have the following advantages:

[0035] The network model dynamic quantization method proposed in the embodiments of the present application quantizes a network model trained based on point cloud data dynamically by using a network search method based on a post-training quantization principle. In each round of search, a quantization truncation interval in the current round of search is determined based on the current parameter range and a number of grid searches, which is then used to calculate a candidate quantization scaling factor in the current round of search. When the quantization errors meet a threshold condition, the current parameter range is updated to serve as a basis for the next round of search. In this way, a quantization scaling factor with minimum model precision loss is found through iterative calculation, so as to obtain a required quantized model. The quantized model is conducive to the deployment of a complex network model on a terminal device with limited resources. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. Other related drawings can also be obtained by those skilled in the art without creative labor.

[0037] Fig. 1 shows a schematic diagram of point cloud data;

[0038] Fig. 2 shows another schematic diagram of point cloud data containing a vehicle target;

[0039] Fig. 3 shows a schematic diagram of a terminal device according to an embodiment of the application;

[0040] Fig. 4 shows a flowchart of a method for dynamically quantizing a network model according to an embodiment of the application;

[0041] Fig. 5 shows a flowchart of a method for calculating quantization error according to an embodiment of the application;

[0042] Fig. 6 shows a schematic diagram of a device for dynamically quantizing a network model according to an embodiment of the application. Embodiments of the application

[0043] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments of the application.

[0044] The components of the embodiments of the application generally described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the application, all other embodiments obtained by a person skilled in the art without creative work fall within the scope of protection of the application.

[0045] Hereinafter, the terms "include", "have", and their conjugates used in various embodiments of the application are only intended to denote a certain characteristic, number, step, operation, element, component, or combination of the foregoing, and should not be construed as excluding the presence or addition of one or more other characteristics, numbers, steps, operations, elements, components, or combinations of the foregoing. In addition, the terms "first", "second", "third", and the like are only used to distinguish descriptions, and should not be understood as indicating or implying relative importance.

[0046] Unless specifically defined, all other technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present application belong. The terminology used herein (e.g., the terminology used in the description of the figures) is for the purpose of describing particular embodiments only and is not intended to be limiting of various embodiments of the present application. Unless otherwise defined, all terms of art used herein have the same meaning as commonly understood by one of ordinary skill in the art in the field of various embodiments of the present application. The terminology used herein, such as the terminology used in the description of the figures, will be interpreted as having a meaning that is consistent with the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined in various embodiments of the present application.

[0047] Some embodiments of the present application are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments can be combined with each other in the case of no conflict.

[0048] For an RGB image, each pixel in the RGB image represents information. In a point cloud image, for example, FIG. 1 shows a point cloud image collected by a laser radar. As can be understood, the gray part (non-zero pixel) where the laser radar scans is information, and the white background (zero pixel) is no information. As can be seen from the figure, unlike the information-intensive RGB image, the information density of the point cloud data is sparse, and the non-zero pixel only occupies a very limited part of the entire scene, that is, has the characteristics of irregular distribution. Not only that, but the foreground instance in the point cloud is imbalanced with the large redundant background area, for example, as shown in FIG. 2, the box represents the target (here is a vehicle) detected by the radar. For a vehicle of 4m x 2m, it only occupies 40 x 20 pixels in the input 1504 x 1504 BEV feature map, in other words, the size of the box occupied by the target to be detected is very small compared to the overall point cloud image. In addition, for the point cloud data obtained by the 3D radar, the operation range is large, for example, the point coordinates after voxelization are located in a 3D space of 1504 x 1504 x 40 with a voxel size of 0.1m, which is more susceptible to quantization. In other words, the greater the quantization error, the greater the impact on the operation result.

[0049] Due to the characteristics of sparsity and irregular distribution of point cloud data, performance loss phenomenon is prone to occur when quantizing the neural network model. Therefore, based on the principle of post-training quantization (PTQ, Post-Training Quantization), a network model dynamic quantization method based on network search is proposed to effectively reduce the quantization error when converting the full-precision model to the low-precision model, thereby improving the performance loss of the quantized model with point cloud data as input and improving the overall performance of the model.

[0050] Please refer to FIG. 3, which is a structural schematic diagram of a terminal device according to an embodiment of the present application. It can be understood that the terminal device is limited in hardware resources and can be an edge device, including but not limited to a vehicle, a robot, etc. Illustratively, the terminal device 10 can include a memory 11, a processor 12 and a perception unit 13, wherein the memory 11 and the perception unit 13 are connected to the processor 12, the perception unit 13 can be used to perceive current environment information of the terminal device 10, the memory 11 stores a computer program, and the processor 12 is used to execute the computer program to implement a network model dynamic quantization method according to an embodiment of the present application to improve performance loss of a quantized model.

[0051] The memory 11 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electric erasable programmable read only memory (EEPROM), etc. The memory 11 is used to store a computer program, and the processor 12 can execute the computer program correspondingly after receiving an execution instruction.

[0052] The processor 12 can be an integrated circuit chip with a signal processing capability. The processor 12 can be a general purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, a discrete gate or transistor logic device, a discrete hardware component, at least one of them. The general purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0053] The perception unit 13 is configured to deliver the perceived external environment information and the like to the control system of the terminal device 10 for corresponding processing by the control system. For example, the perception unit 13 can include a laser radar and the like, which is not limited in type and can be used to obtain external person / object information and the like of the terminal device 10 during walking, thereby obtaining target distance information and the like, but is not limited to this information. In addition, the perception unit 13 can also include a visual camera and the like, which can be used to obtain RGB image information and the like of the terminal device 10 during walking. It can be understood that the specific configuration of the perception unit 13 is determined according to the specific use scenario of the terminal device 10, which is not strictly limited here.

[0054] Based on the above structure of the terminal device 10, the network model dynamic quantization method will be described below in combination with some specific embodiments.

[0055] FIG. 4 shows a flowchart of the network model dynamic quantization method according to an embodiment of the present application. The network model dynamic quantization method includes the following steps:

[0056] S110, obtaining all model parameters of the network model trained based on point cloud data and quantization precision.

[0057] It can be understood that the present application calibrates the model quantization process by using a small amount of additional data or not using additional data on the trained network model, in other words, based on the post-training quantization (PTQ) technology to perform dynamic quantization. The post dynamic quantization can directly convert the parameters of each layer in the network through the quantization formula without using a calibration data set. In the embodiments of the present application, the model parameters are weight parameters.

[0058] Model quantization is the process of quantizing full-precision floating-point model parameters to low-precision integer data format. The quantization precision refers to the integer bit width of quantization, for example, the quantization precision of int4 is 4 bits, and the quantization precision of int8 is 8 bits.

[0059] Due to the sparse characteristics of the point cloud data, the corresponding quantization range difference is large, and the difference part often contains a large amount of geometric information, in other words, the point cloud data often has abnormal data points, if directly quantized, the number of effective quantization grid points will be reduced, resulting in serious precision loss, therefore, the application will use a lightweight grid search technology to eliminate the influence of abnormal data points. Therefore, the application also needs to pre-set the hyperparameter value range to be set in the grid search, which is denoted as the grid search number T. It can be understood that the original model parameter data is evenly divided into T groups, and the quantization scaling factor corresponding to the minimum quantization error is found by loop traversal.

[0060] In S120, the parameter quantization range is determined based on the quantization precision, and the initial parameter range is determined based on all model parameters.

[0061] The parameter quantization range is usually determined by the bit width, including the minimum value and the maximum value of the parameter quantization range. For example, for int8 integer type, and the data type is symmetrical positive and negative, q min =-2^8 / 2=-128, q max =2^8 / 2-1=127, that is, the parameter quantization range is (-128, 127). It can be understood that the parameter quantization range can be determined according to the actual integer type.

[0062] For the network model with point cloud data as input, the distribution range of the model parameters is wide, but the maximum value and the minimum value of the model weight do not necessarily occupy the full range, therefore, the application embodiment proposes to use the actual effective parameter range for quantization, which can reduce the precision loss in the quantization process.

[0063] Demonstratively, for the first search, the maximum absolute value (denoted as max(|x|) in all model parameters is selected, and the positive and negative values of the maximum absolute value are respectively taken as the maximum value (denoted as x max =max(|x|) and the minimum value (denoted as x min =-max(|x|) of the initial parameter range. It can be understood that the quantization grid is different each time the search is performed, therefore, the parameter range is dynamically updated each time the search is performed.

[0064] In S130, the quantization truncation interval of the current search is determined based on the current parameter range and the set grid search number, the candidate quantization scaling factor of the current search is calculated according to the quantization precision and the quantization truncation interval, and the quantization error of all model parameters under the candidate quantization scaling factor is calculated based on the parameter quantization range, when the quantization error is less than the quantization error threshold, the initial parameter range or the parameter range of the last search is updated using the quantization truncation interval of the current search.

[0065] The embodiment is based on the current parameter range and the set grid search times to dynamically determine the quantization threshold interval in each search round, and then update the parameter range by using the quantization threshold interval.

[0066] Exemplarily, in an embodiment, the ratio of the maximum value in the current parameter range and the grid search times can be calculated, and the ratio is multiplied by the search serial number to obtain the quantization threshold value in the current search round. Further, the positive and negative values of the quantization threshold value are selected as the two boundary values of the quantization threshold interval to obtain the quantization threshold interval in the current search round. It can be understood that the threshold value represents that the data outside the current parameter range is all normalized to the boundary value.

[0067] For example, if described by an expression, the calculation formula of the quantization threshold value is: threhold=range / T*i,

[0068] In the formula, threhold is the quantization threshold value in the i-th search round, range is the maximum value in the parameter range in the i-th search round, i represents the search serial number, for example, in the first search round, i=1, and in the second search round, i=2. Further, according to x min =-threshold, x max =threshold, to obtain the lower boundary value x min and the upper boundary value x max of the quantization threshold interval.

[0069] For example, assuming that all model parameters are distributed in the interval 1-1000, and the model parameters are mainly concentrated in the interval 200-800, the application avoids the influence of the difference value by using the grid search algorithm. Specifically, assuming that the grid search times is 10, and the parameter quantization range is (-128, 127), then for each quantization grid, there is a corresponding quantization threshold interval, and the threshold value can be calculated by the above formula.

[0070] It can be understood that in the conversion formula (i.e., quantization formula) between floating-point numbers and fixed-point (integer) numbers, two coefficients need to be used for conversion, which are the quantization scaling factor s (i.e., scale) and the quantization zero-point offset z (i.e., zero).

[0071] It is worth noting that if the symmetric quantization method is used, the zero-point offset z is zero; on the contrary, if the asymmetric quantization method is used, the zero-point offset will be a non-zero value. In the embodiment, the symmetric quantization method will be used to describe the dynamic quantization method.

[0072] In the grid search process, the quantization scaling factor calculated in each round of search is recorded as a candidate quantization scaling factor, so as to distinguish the target quantization scaling factor selected from the candidate quantization scaling factors and making the quantization error minimum.

[0073] In an embodiment, the quantization scaling factor is calculated by using the boundary values of the quantization truncation interval in each round of search. For example, if described by an expression, the calculation formula of the candidate quantization scaling factor in each round of search is: s = (x max -x min ) / (2 b -1);

[0074] In the expression, s represents the candidate quantization scaling factor in the current round of search, x miin and x max represent the lower boundary value and the upper boundary value of the quantization truncation interval in the current round of search, and b is the quantization precision.

[0075] After the candidate quantization scaling factor is calculated, the error of each model parameter before and after quantization is further calculated.

[0076] In an embodiment, as shown in FIG. 5, the quantization error of all model parameters under the candidate quantization scaling factor is calculated, including the following steps:

[0077] S210, based on the parameter quantization range and the pre-quantization value of each model parameter, the post-quantization value of each model parameter under the candidate quantization scaling factor is calculated.

[0078] For example, according to the parameter quantization range, the ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor is clipped to obtain a clipping error. Further, based on the clipping error and the zero offset during quantization, the post-quantization value of the corresponding model parameter under the candidate quantization scaling factor is calculated. It can be understood that the pre-quantization value of the model parameter is determined after the network model training is completed, and the post-quantization value can be calculated by using the quantization formula. According to different quantization formulas, it can be divided into linear quantization and nonlinear quantization. Here, linear quantization is mainly taken as an example for illustration.

[0079] For example, if described by an expression, the calculation formula of the post-quantization value of a single model parameter is:

[0080] In the expression, x and represent the pre-quantization value and the post-quantization value of the corresponding model parameter; s represents the candidate quantization scaling factor, q min and q maxmin and max represent the minimum and maximum values of the parameter quantization range; z is a zero offset; and the function clamp() represents clipping values that exceed the parameter quantization range.

[0081] S220, according to the pre-quantization value and the post-quantization value of each model parameter, calculating the quantization error of all model parameters in the current round of search.

[0082] Exemplarily, the quantization error of each model parameter is obtained by subtracting the pre-quantization value from the post-quantization value, and then the sum of the quantization errors of all model parameters is calculated.

[0083] As an optional solution, the method further comprises:

[0084] When the quantization error in each round of search is less than the quantization error threshold, the current quantization error threshold is updated by using the quantization error in each round of search. Otherwise, if the quantization error is not less than the quantization error threshold, no update is made. The initial value of the quantization error threshold can be set according to actual experience value. Then, in the subsequent search process, the quantization error threshold can be updated by assignment, thereby accelerating the search speed.

[0085] S140, according to the quantization error obtained in each round of search, determining a target quantization scaling factor that minimizes the quantization error, to determine the quantized network model based on the target quantization scaling factor.

[0086] A plurality of quantization errors can be calculated through multiple rounds of search. To find the optimal quantization scaling factor with the minimum precision loss, a target optimal function can be constructed to achieve this. For example, a two-norm function or the like can be used to construct a score for evaluating the quantization error under the current quantization scaling factor. It can be understood that the above-mentioned target optimal function is not limited to being constructed by using the two-norm, as long as it can be used to evaluate the quantization precision loss corresponding to each search.

[0087] In an embodiment, the two-norm is used to obtain the minimum value of the quantization error, i.e.:

[0088] In the formula, X and represent the pre-quantization value and the post-quantization value of all model parameters; S t represents the candidate quantization scaling factor in the tthround of search.

[0089] After determining an optimal quantization scaling factor, the quantized values of the model parameters can be calculated using the quantization formula to obtain a quantized network model. Then, the quantized network model is deployed on a terminal device such as a terminal device equipped with a laser radar, so that after the laser radar collects point cloud data, the deployed model can be used to perform point cloud data processing through dequantization. It can be understood that by using the above dynamic quantization method to quantize the model, the performance loss of the point cloud data after model quantization can be effectively improved, and the overall performance of the model can be improved.

[0090] The embodiment of the present application provides a network model dynamic quantization method. The network model trained based on point cloud data is dynamically quantized by using a network search method, that is, quantization is performed based on a post-training quantization principle. In each search process, the quantization truncation interval in each search is determined based on the current parameter range, and then the candidate quantization scaling factor in each search is calculated. When the quantization error meets the threshold condition, the current parameter range is updated, that is, the parameter range is dynamically adjusted, so that the effective parameters with a large proportion are selected from the model parameters as the basis for the next search. Thus, the quantization scaling factor with the minimum model precision loss is found through iterative calculation. The method reduces the rounding error when converting a full-precision model to a low-precision model, and improves the performance of the quantized model.

[0091] FIG. 6 shows a structural schematic diagram of a network model dynamic quantization device 100 according to an embodiment of the present application. The network model dynamic quantization device 100 includes, for example:

[0092] The data acquisition module 110 is configured to acquire all model parameters of a network model trained based on point cloud data and quantization precision.

[0093] The range determination module 120 is configured to determine a parameter quantization range based on the quantization precision, and determine an initial parameter range based on all the model parameters.

[0094] The network search module 130 is configured to, in each search, determine a quantization truncation interval in the current search based on the current parameter range and a set grid search number, calculate a candidate quantization scaling factor in the current search according to the quantization precision and the quantization truncation interval, and calculate quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range. When the quantization error is less than a quantization error threshold, the initial parameter range or the parameter range in the previous search is updated using the quantization truncation interval in the current search.

[0095] The model quantization module 140 is configured to determine a target quantization scaling factor that minimizes the quantization error obtained in each round of search, and determine a quantized network model based on the target quantization scaling factor.

[0096] In some optional solutions, the network model dynamic quantization apparatus 100 further includes:

[0097] The error threshold updating module is configured to update the current quantization error threshold by using the quantization error obtained in the current round of search when the quantization error obtained by the network search module 130 in the current round of search is less than the quantization error threshold.

[0098] As an optional implementation, the network search module 130 includes a truncation interval unit, a scaling factor calculation unit, a quantization error calculation unit, and a parameter range updating unit. The truncation interval unit is configured to determine a quantization truncation interval in the current round of search based on the current parameter range and a set number of grid searches. The scaling factor calculation unit is configured to calculate a candidate quantization scaling factor in the current round of search according to the quantization precision and the quantization truncation interval. The quantization error calculation unit is configured to calculate quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range. The parameter range updating unit is configured to update the initial parameter range or the parameter range in the previous round of search by using the quantization truncation interval in the current round of search when the quantization error is less than a quantization error threshold.

[0099] Further, the quantization error calculation unit includes a quantized value calculation subunit and an error calculation subunit. The quantized value calculation subunit is configured to calculate quantized values of each of the model parameters under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization value of each of the model parameters. The error calculation subunit is configured to calculate the quantization error of all the model parameters in the current round of search according to the pre-quantization value and the quantized value of each of the model parameters.

[0100] Further, the quantized value calculation subunit is specifically configured to clip a ratio of the pre-quantization value of a corresponding model parameter to the candidate quantization scaling factor to obtain a clipping error according to the parameter quantization range, and then calculate the quantized value of the corresponding model parameter under the candidate quantization scaling factor based on the clipping error and a zero-point offset during quantization. When a symmetric quantization method is used, the zero-point offset is zero. When an asymmetric quantization method is used, the zero-point offset is a non-zero value.

[0101] As an optional implementation, the truncation interval unit is specifically configured to calculate a ratio of a maximum value in the current parameter range to the grid search number, multiply the ratio by the search sequence number to obtain a quantization truncation value in the current search, and then select positive and negative values of the quantization truncation value as two boundary values of a quantization truncation interval to obtain the quantization truncation interval in the current search.

[0102] It can be understood that the device of the embodiment corresponds to the network model dynamic quantization method of the above-described embodiment, and the optional items in the above-described embodiment are also applicable to the embodiment, and thus are not repeatedly described herein.

[0103] The application further provides a computer readable storage medium for storing the computer program used in the terminal device. For example, the computer readable storage medium can include, but is not limited to, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0104] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other means. The apparatus embodiment described above is only schematic. For example, the flowchart and block diagram in the drawings show the possible implementation architecture, function and operation of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that, in alternative implementation, the functions noted in the block can occur in different order from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the function involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0105] In addition, each functional module or unit in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0106] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0107] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be encompassed in the protection scope of the present application.

Claims

1. A method for dynamic quantization of network models, characterized in that, The method comprises the following steps: acquiring all model parameters of a network model trained based on point cloud data and quantization precision; determining a parameter quantization range based on the quantization precision and determining an initial parameter range based on all the model parameters; in each search round, determining a quantization truncation interval in the current search round based on the current parameter range and a set grid search number, calculating a candidate quantization scaling factor in the current search round according to the quantization precision and the quantization truncation interval, and calculating quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range, and when the quantization errors are less than a quantization error threshold, updating the initial parameter range or a parameter range in the previous search round by using the quantization truncation interval in the current search round; determining a target quantization scaling factor that minimizes the quantization errors according to the quantization errors obtained in each search round, and determining a quantized network model based on the target quantization scaling factor.

2. The method of claim 1, wherein, The method further comprises the following steps: when the quantization errors in the current search round are less than the quantization error threshold, updating the current quantization error threshold by using the quantization errors in the current search round.

3. The method of claim 1, wherein, The step of calculating the quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range comprises the following steps: calculating quantized values of each model parameter under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization values of each model parameter; calculating the quantization errors of all the model parameters in the current search round according to the pre-quantization values and the quantized values of each model parameter.

4. The method of claim 3, wherein, The step of calculating the quantized values of each model parameter under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization values of each model parameter comprises the following steps: clipping a ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor according to the parameter quantization range to obtain a clipping error; calculating the quantized value of the corresponding model parameter under the candidate quantization scaling factor based on the clipping error and a zero-point offset during quantization; wherein, when a symmetric quantization method is used, the zero-point offset is zero; and when an asymmetric quantization method is used, the zero-point offset is a non-zero value.

5. The method of claim 4, wherein, The calculation formula of the quantized value is: In the formula, x and quantized value of a corresponding model parameter; s represents the candidate quantization scale factor, q min and q max represent the minimum and maximum values of the parameter quantization range; z is the zero-point offset, and the function clamp() represents clipping values that are outside the parameter quantization range.

6. The method of claim 1, wherein, The step of determining the quantization truncation interval in the current search round based on the current parameter range and the set grid search number comprises the following steps: calculating a ratio of a maximum value in the current parameter range to the grid search number, and multiplying the ratio by a search serial number to obtain a quantization truncation value in the current search round; selecting positive and negative values of the quantization truncation value as two boundary values of a quantization truncation interval to obtain the quantization truncation interval in the current search round.

7. The method of claim 6, wherein, The calculation formula of the candidate quantization scaling factor is: s = (x max - x min ) / (2 b - 1); where s represents the candidate quantization scale factor, x min and x max denote the lower and upper boundary values of the quantization truncation interval, and b is the quantization precision.

8. A network model dynamic quantization apparatus, comprising: The method comprises the following steps: a data acquisition module is configured to acquire all model parameters of a network model trained based on point cloud data and quantization precision; a range determination module is configured to determine a parameter quantization range based on the quantization precision and determine an initial parameter range based on all the model parameters; The network search module is configured to, in each round of search, determine a quantization truncation interval for the current round of search based on the current parameter range and a set grid search number, calculate a candidate quantization scaling factor for the current round of search according to the quantization precision and the quantization truncation interval, and calculate quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range; when the quantization error is less than a quantization error threshold, update the initial parameter range or the parameter range of the previous round of search by using the quantization truncation interval of the current round of search. The model quantization module is configured to determine a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each round of search, and determine a quantized network model based on the target quantization scaling factor.

9. A terminal device, comprising: The terminal device comprises a processor and a memory, the memory stores a computer program, and the processor is configured to execute the computer program to implement the network model dynamic quantization method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is executed on the processor to implement the network model dynamic quantization method in any one of claims 1-7.

Images