Network model dynamic quantification method and device, terminal equipment and readable storage medium
Through dynamic quantization method, the neural network model trained by point cloud data is quantified, which solves the performance loss problem when RGB image data is migrated to radar point cloud data, and realizes efficient deployment on edge devices.
Patent Information
- Application Number
- CN202411098734.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-06
AI Technical Summary
Existing neural network models are prone to performance losses when migrating RGB image data to radar point cloud data, resulting in challenges in the deployment of 3D lidar-based detectors on edge devices.
A dynamic quantization method for network model is proposed. By obtaining model parameters trained based on point cloud data, the network search method is used to dynamically quantize, and the quantization truncation interval is determined based on the quantization accuracy and the number of grid searches, the candidate quantization scaling factor is calculated, and the candidate quantization scaling ratio is calculated. When the quantization error meets the threshold condition, the parameter range is updated, and iteratively finds the scaling factor with the minimum quantization error.
It effectively reduces the quantization error when converting the full-precision model to a low-precision model, improves the performance of the quantized model, and improves the deployment capability on resource-limited terminal devices.
Smart Images

Figure CN119089951A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of model quantization technology, and in particular to a network model dynamic quantization method, apparatus, terminal device and readable storage medium. Background Art
[0002] Model quantization technology is an effective solution for deploying neural network models on resource-constrained edge devices. Model quantization is a way to compress network parameters by reducing the values of activation feature maps and weight parameters that were originally represented by floating points and replacing them with fixed-point (integer) values. During the calculation process, the fixed-point data is dequantized back to floating-point data to obtain the result.
[0003] At present, the input of most neural network models is RGB (color) image data, and the quantization research on this type of neural network is the most. However, for radar point cloud data, due to its statistical distribution characteristics are different from RGB image data, when the quantization method that performs well in the model with RGB image data as input is directly migrated to the model with radar point cloud data as input, it is easy to have a large performance loss, which brings new challenges to the deployment of 3D lidar-based detectors on edge devices. Summary of the invention
[0004] In view of this, the embodiments of the present application provide a method, apparatus, terminal device and readable storage medium for dynamic quantization of a network model, which can effectively improve the performance loss of the model after quantization.
[0005] In a first aspect, an embodiment of the present application provides a network model dynamic quantization method, including:
[0006] Obtain all model parameters and quantization accuracy of the network model trained based on point cloud data;
[0007] Determining a parameter quantization range based on the quantization accuracy, and determining an initial parameter range based on all the model parameters;
[0008] When performing each round of search, the quantization truncation interval of the current round of search is determined based on the current parameter range and the set number of grid searches, the candidate quantization scaling factor of the current round of search is calculated according to the quantization accuracy and the quantization truncation interval, and the quantization error of all the model parameters under the candidate quantization scaling factor is calculated based on the parameter quantization range, and when the quantization error is less than the quantization error threshold, the initial parameter range or the parameter range of the previous round of search is updated using the quantization truncation interval of the current round of search;
[0009] According to the quantization error obtained in each round of search, a target quantization scaling factor that minimizes the quantization error is determined, so as to determine a quantized network model based on the target quantization scaling factor.
[0010] In some embodiments, the network model dynamic quantization method further includes:
[0011] When the quantization error in this round of search is less than the quantization error threshold, the current quantization error threshold is updated by using the quantization error in this round of search.
[0012] In some embodiments, the calculating, based on the parameter quantization range, the quantization errors of all the model parameters under the candidate quantization scaling factor comprises:
[0013] Calculating a post-quantization value of each of the model parameters under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization value of each of the model parameters;
[0014] According to the pre-quantization value and the post-quantization value of each model parameter, the quantization errors of all the model parameters in this round of search are calculated.
[0015] In some embodiments, the calculating, based on the parameter quantization range and the pre-quantization value of each of the model parameters, the post-quantization value of each of the model parameters under the candidate quantization scaling factor comprises:
[0016] According to the parameter quantization range, clipping the ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor to obtain a clipping error;
[0017] Calculating the quantized value of the corresponding model parameter under the candidate quantization scaling factor based on the clipping error and the zero offset during quantization;
[0018] When the symmetric quantization method is adopted, the zero point offset is zero; when the asymmetric quantization method is adopted, the zero point offset is a non-zero value.
[0019] In some embodiments, the calculation formula of the quantized value is:
[0020]
[0021] In the formula, x and represents the pre-quantization value and post-quantization value of the corresponding model parameter; s represents the candidate quantization scaling factor, q min and q maxrepresents the minimum and maximum values of the parameter quantization range; z is the zero point offset, and the function clamp() represents clipping the values that exceed the parameter quantization range.
[0022] In some embodiments, determining the quantization cutoff interval in the current round of search based on the current parameter range and the set number of grid searches includes:
[0023] Calculate the ratio of the maximum value in the current parameter range to the number of grid searches, and multiply the ratio by the current search sequence number to obtain a quantization cutoff value for the current search;
[0024] The positive and negative values of the quantization cutoff value are selected as two boundary values of the quantization cutoff interval respectively, so as to obtain the quantization cutoff interval in this search.
[0025] In some embodiments, the candidate quantization scaling factor is calculated as:
[0026] s(xmax-xmin) / (2b-1);
[0028] Where s represents the candidate quantization scaling factor, xmi n and xma x represents the lower boundary value and the upper boundary value of the quantization truncation interval, and b is the quantization accuracy.
[0029] In a second aspect, an embodiment of the present application provides a network model dynamic quantization device, including:
[0030] The data acquisition module is used to obtain all model parameters and quantization accuracy of the network model trained based on point cloud data;
[0031] A range determination module, configured to determine a parameter quantization range based on the quantization precision, and to determine an initial parameter range based on all the model parameters;
[0032] A network search module, for determining the quantization cutoff interval in the current search based on the current parameter range and the set number of grid searches during each round of search, calculating the candidate quantization scaling factor in the current search based on the quantization accuracy and the quantization cutoff interval, and calculating the quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range, and when the quantization error is less than the quantization error threshold, using the quantization cutoff interval in the current search to update the initial parameter range or the parameter range in the previous search;
[0033] The model quantization module is used to determine a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each round of search, so as to determine a quantized network model based on the target quantization scaling factor.
[0034] In a third aspect, an embodiment of the present application provides a terminal device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the network model dynamic quantization method.
[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed on a processor, implements the network model dynamic quantization method.
[0036] The embodiments of the present application have the following beneficial effects:
[0037] The network model dynamic quantization method proposed in the embodiment of the present application is based on the principle of post-training quantization and dynamically quantizes the network model trained based on point cloud data by adopting a network search method, wherein, in each round of search, the quantization cutoff interval for each round of search is determined based on the current parameter range and the number of grid searches, and then used to calculate the candidate quantization scaling factor for each round of search, and when the quantization error meets the threshold condition, the current parameter range is updated to serve as the basis for the next round of search, thereby finding the quantization scaling factor with the smallest loss of model accuracy through iterative calculation to obtain a quantized model that meets the requirements, and the quantized model is conducive to the deployment of complex network models on terminal devices with limited resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0039] Figure 1 A schematic diagram of point cloud data is shown;
[0040] Figure 2 Another schematic diagram showing point cloud data including a vehicle target;
[0041] Figure 3 A schematic diagram of the structure of a terminal device according to an embodiment of the present application is shown;
[0042] Figure 4 A flow chart of a network model dynamic quantization method according to an embodiment of the present application is shown;
[0043] Figure 5 A flowchart of quantization error calculation according to an embodiment of the present application is shown;
[0044] Figure 6 A structural schematic diagram of a network model dynamic quantization device according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.
[0046] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0047] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.
[0048] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.
[0049] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0050] For RGB images, each pixel in the RGB image represents information. In point cloud images, for example, Figure 1The figure shows a point cloud image collected by the LiDAR. It can be understood that the gray part (non-zero pixels) scanned by the LiDAR is informative, while the white background (zero pixels) is informationless. It can be seen from the figure that, unlike the information-dense RGB image, the information density of the point cloud data is sparse, and its non-zero pixels only occupy a very limited part of the entire scene, that is, it has the characteristic of irregular distribution. In addition, there is an imbalance between the foreground instances and the large redundant background areas in the point cloud, for example, Figure 2 As shown in the figure, the box represents the target detected by the radar (here is a vehicle). For example, a 4m×2m vehicle only occupies 40×20 pixels in the input 1504×1504BEV feature map. In other words, the size of the box occupied by the target to be detected is very small compared to the overall point cloud image. In addition, for the point cloud data obtained by the 3D radar, its calculation range is very large. For example, the voxelized point coordinates are located in a 1504×1504×40 3D space with a voxel size of 0.1m, which is more susceptible to quantization. In other words, the larger the quantization error, the greater the impact on the calculation result.
[0051] Due to the sparsity and irregular distribution of point cloud data, performance loss is prone to occur when quantizing neural network models. To this end, this application proposes a network model dynamic quantization method based on network search based on the post-training quantization (PTQ) principle to effectively reduce the quantization error when converting the full-precision model to the low-precision model, thereby improving the performance loss of the quantized model with point cloud data as input and improving the overall performance of the model.
[0052] Please refer to Figure 3 , which is a schematic diagram of the structure of a terminal device proposed in an embodiment of the present application. It can be understood that the terminal device has limited hardware resources and can be some edge devices, including but not limited to vehicles, robots, etc. Exemplarily, the terminal device 10 may include a memory 11, a processor 12 and a perception unit 13, wherein the memory 11 and the perception unit 13 are both connected to the processor 12, the perception unit 13 can be used to perceive the current environment information of the terminal device 10, the memory 11 stores a computer program, and the processor 12 is used to execute the computer program to implement the network model dynamic quantization method of the embodiment of the present application to improve the performance loss of the quantized model.
[0053] The memory 11 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 11 is used to store a computer program, and the processor 12 may execute the computer program accordingly after receiving an execution instruction.
[0054] The processor 12 may be an integrated circuit chip with signal processing capability. The processor 12 may be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or at least one of other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc., which may implement or execute the disclosed methods, steps, and logic block diagrams in the embodiments of the present application.
[0055] The perception unit 13 is used to transmit the perceived external environment information to the control system of the terminal device 10 for corresponding processing by the control system. For example, the perception unit 13 may include a laser radar, etc., wherein the type of the laser radar is not limited, and it can be used to obtain external human / object information of the terminal device 10 during walking, thereby obtaining information such as the target distance, but not limited to this information. In addition, the perception unit 13 may also include a visual shooting device, etc., wherein the visual shooting device can be used to obtain RGB image information of the terminal device 10 during walking. It can be understood that the setting of the perception unit 13 is determined according to the specific usage scenario of the terminal device 10, and is not strictly limited here.
[0056] Based on the terminal device 10 of the above structure, the network model dynamic quantization method is described below in conjunction with some specific embodiments.
[0057] Figure 4A flow chart of a network model dynamic quantization method according to an embodiment of the present application is shown. Exemplarily, the network model dynamic quantization method includes the following steps:
[0058] S110, obtaining all model parameters and quantization accuracy of the network model trained based on the point cloud data.
[0059] It can be understood that the present application calibrates the model quantization process on the trained network model using a small amount or no additional data. In other words, dynamic quantization is performed based on the post-training quantization (PTQ) technology. Among them, post-training dynamic quantization (PostDynamic Quantization) can directly convert the parameters of each layer in the network through a quantization formula without using a calibration data set. In the embodiment of the present application, the above-mentioned model parameters are weight parameters.
[0060] Model quantization is the process of quantizing full-precision floating-point model parameters into low-precision integer data formats. The quantization precision refers to the quantized integer bit width, for example, the quantization precision of the integer int4 is 4 bits, and the quantization precision of int8 is 8 bits.
[0061] Due to the inherent sparseness of point cloud data, the corresponding quantization ranges vary greatly, and the difference often contains a lot of geometric information. In other words, point cloud data often has abnormal data points. If it is directly quantized, it will reduce the number of effective quantization grid points, resulting in serious loss of accuracy. Therefore, this application will use lightweight grid search technology to eliminate the influence of abnormal data points. Therefore, this application also needs to pre-set the range of hyperparameter values that need to be set in the grid search, which is recorded as the number of grid searches T. It can be understood that the original model parameter data is evenly divided into T groups, and the quantization scaling factor corresponding to the minimum quantization error is found through loop traversal.
[0062] S120, determining a parameter quantization range based on the quantization accuracy, and determining an initial parameter range based on all model parameters.
[0063] The parameter quantization range is usually determined by the bit width, including the minimum and maximum values of the parameter quantization range. For example, for an int8 integer with positive and negative symmetric data types, q min =-2^8 / 2 = -128, q max =2^8 / 2-1=127, that is, the parameter quantization range is (-128, 127). It can be understood that the parameter quantization range can be specifically determined according to the actual integer type.
[0064] For a network model that uses point cloud data as input, the distribution range of its model parameters is wide, but the maximum and minimum values of the model weights do not necessarily occupy the entire range. Therefore, the embodiment of the present application proposes to use the actual effective parameter range for quantization, which can reduce the loss of accuracy in the quantization process.
[0065] For example, for the first search, the maximum absolute value of all model parameters (referred to as max(|x|)) is selected, and the positive and negative values of the maximum absolute value are used as the maximum value of the initial parameter range (referred to as x). max = max(|x|) and the minimum value (denoted as x min = -max(|x|) It can be understood that the quantization grid is different each time a search is performed, so the parameter range is dynamically updated each time a search is performed.
[0066] S130, when executing each round of search, the quantization truncation interval of this round of search is determined based on the current parameter range and the set number of grid searches, the candidate quantization scaling factors of this round of search are calculated according to the quantization accuracy and the quantization truncation interval, and the quantization errors of all model parameters under the candidate quantization scaling factors are calculated based on the parameter quantization range. When the quantization error is less than the quantization error threshold, the quantization truncation interval of this round of search is used to update the initial parameter range or the parameter range of the previous round of search.
[0067] This embodiment dynamically determines the quantization cutoff interval in each round of search based on the current parameter range and the set number of grid searches, and then uses the quantization cutoff interval to update the parameter range.
[0068] Exemplarily, in one implementation, the ratio of the maximum value in the current parameter range to the number of grid searches can be calculated, and the ratio can be multiplied by the current search sequence number to obtain the quantization cutoff value for this round of search. Further, the positive and negative values of the quantization cutoff value are selected as two boundary values of the quantization cutoff interval, respectively, to obtain the quantization cutoff interval for this search. It can be understood that the cutoff value represents normalizing all data outside the current parameter range to the boundary value.
[0069] For example, if described by an expression, the calculation formula for the quantitative cutoff value is:
[0070] threshold = range / T*i,
[0071] In the formula, threshold is the quantization cutoff value in the i-th round of search, range is the maximum value in the parameter range in the i-th round of search, and i represents the search sequence number. For example, in the first round of search, i=1, and in the second round of search, i=2. Then, according to x min = -threshold, x max=threshold, to obtain the lower boundary value x of the quantization cutoff interval min and the upper boundary value x max .
[0072] For example, assuming that all model parameters are distributed in the interval 1-1000, and the model parameters are mainly concentrated in the interval 200-800, the present application uses a grid search algorithm to avoid the influence of difference values. Specifically, assuming that the number of grid searches is 10 and the parameter quantization range is (-128, 127), then each quantization grid corresponds to a quantization truncation interval, and the truncation value can be calculated by the above formula.
[0073] It can be understood that in the conversion formula between floating-point numbers and fixed-point (integer) numbers (i.e., quantization formula), two coefficients are needed for conversion, namely, the quantization scaling factor s (i.e., scale), and the quantization zero offset z (i.e., zero).
[0074] It is worth noting that if the symmetric quantization method is used, the zero point offset z is zero; conversely, if the asymmetric quantization method is used, the zero point offset will be a non-zero value. In this embodiment, the symmetric quantization method will be used to illustrate the dynamic quantization method.
[0075] During the grid search process, the quantization scaling factor calculated in each round of search is recorded as a candidate quantization scaling factor, so as to be distinguished from the target quantization scaling factor that minimizes the quantization error and is finally selected from the candidate quantization scaling factors.
[0076] In one embodiment, the quantization scaling factor is calculated using the boundary value of the quantization cutoff interval in each round of search. For example, if described by an expression, the calculation formula of the candidate quantization scaling factor in each round of search is:
[0077] s(xmax-xmin) / (2b-1);
[0079] Where s represents the candidate quantization scaling factor in this round of search, xmi n and xma x It represents the lower and upper boundary values of the quantization cutoff interval in this round of search, and b is the quantization accuracy.
[0080] After calculating the candidate quantization scaling factors, the error of each model parameter before and after quantization is further calculated.
[0081] In one embodiment, Figure 5 As shown, calculating the quantization errors of all model parameters under the candidate quantization scaling factors includes the following steps:
[0082] S210 , calculating a post-quantization value of each model parameter under a candidate quantization scaling factor based on a parameter quantization range and a pre-quantization value of each model parameter.
[0083] Exemplarily, the ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor can be trimmed according to the parameter quantization range to obtain a trimming error. Then, based on the trimming error and the zero point offset during quantization, the post-quantization value of the corresponding model parameter under the candidate quantization scaling factor is calculated. It can be understood that the pre-quantization value of the model parameter is determined after the network model training is completed, while the post-quantization value can be calculated using the quantization formula. Among them, according to the different quantization formulas, it can be divided into linear quantization and nonlinear quantization. Here, linear quantization is mainly used as an example for explanation.
[0084] For example, if described by an expression, the calculation formula for the quantized value of a single model parameter is:
[0085]
[0086] In the formula, x and represents the pre-quantization value and post-quantization value of the corresponding model parameter; s represents the candidate quantization scaling factor, q min and q max Represents the minimum and maximum values of the parameter quantization range; z is the zero point offset, and the function clamp() represents clipping of values that exceed the parameter quantization range.
[0087] S220, calculating the quantization errors of all model parameters in this round of search according to the pre-quantization value and the post-quantization value of each model parameter.
[0088] Exemplarily, the quantization error of each model parameter is obtained by subtracting the value before quantization and the value after quantization of each model parameter, and then the sum of the quantization errors of all model parameters is calculated.
[0089] As an optional solution, the method further includes:
[0090] When the quantization error in each round of search is less than the quantization error threshold, the current quantization error threshold is updated by using the quantization error in each round of search. Otherwise, if the quantization error is not less than the quantization error threshold, no update is performed. The initial value of the quantization error threshold can be set according to an actual experience value. Then, in the subsequent search process, the quantization error threshold can be updated by assignment, thereby accelerating the search speed.
[0091] S140, determining a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each round of search, and determining a quantized network model based on the target quantization scaling factor.
[0092] After multiple rounds of searching, multiple quantization errors can be calculated. To find the optimal quantization scaling factor with the smallest precision loss, a target optimal function can be constructed. For example, a two-norm function can be used to construct a score for evaluating the quantization error under the current quantization scaling factor. It can be understood that the above-mentioned target optimal function is not limited to being constructed using the two-norm, as long as it can be used to evaluate the quantization precision loss corresponding to each search.
[0093] In one implementation, the second norm is used to obtain the minimum value of the quantization error, that is,
[0094]
[0095] In the formula, X and Represents the pre-quantization and post-quantization values of all model parameters; S t Represents the candidate quantization scaling factor in the t-th round of search.
[0096] After determining an optimal quantization scaling factor, the quantization formula can be used to calculate the quantized values of each model parameter to obtain a quantized network model. Then, the quantized network model is deployed on a terminal device such as a laser radar, so that after the laser radar collects the point cloud data, the deployed model can be used to perform point cloud CNC processing after dequantization. It can be understood that by using the above-mentioned dynamic quantization method for model quantization, the performance loss of point cloud data after model quantization can be effectively improved, thereby improving the overall performance of the model.
[0097] The embodiment of the present application proposes a method for dynamic quantization of a network model, which dynamically quantizes the network model trained based on point cloud data by using a network search method, that is, quantization is performed based on the principle of post-training quantization; and in each round of search, the quantization cutoff interval for each round of search is determined based on the current parameter range, and then the candidate quantization scaling factor for each round of search is calculated. When the quantization error meets the threshold condition, the current parameter range is updated, that is, the parameter range is dynamically adjusted to select a relatively large number of effective parameters from the model parameters as the basis for the next round of search, thereby finding the quantization scaling factor with the smallest loss of model accuracy through iterative calculation. This method reduces the rounding error when converting the full-precision model to the low-precision model, and improves the performance of the quantized model.
[0098] Figure 6A schematic diagram of the structure of a network model dynamic quantization device 100 according to an embodiment of the present application is shown. Exemplarily, the network model dynamic quantization device 100 includes:
[0099] The data acquisition module 110 is used to obtain all model parameters and quantization accuracy of the network model trained based on the point cloud data.
[0100] The range determination module 120 is used to determine a parameter quantization range based on the quantization precision, and to determine an initial parameter range based on all the model parameters.
[0101] The network search module 130 is used to determine the quantization cutoff interval of the current search based on the current parameter range and the set number of grid searches when performing each round of search, calculate the candidate quantization scaling factor of the current search according to the quantization accuracy and the quantization cutoff interval, and calculate the quantization error of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range. When the quantization error is less than the quantization error threshold, the quantization cutoff interval of the current search is used to update the initial parameter range or the parameter range of the previous search.
[0102] The model quantization module 140 is used to determine a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each round of search, so as to determine a quantized network model based on the target quantization scaling factor.
[0103] In some optional solutions, the network model dynamic quantization device 100 further includes:
[0104] The error threshold updating module is used to update the current quantization error threshold by using the quantization error in the current search when the quantization error in the current search calculated by the network search module 130 is less than the quantization error threshold.
[0105] As an optional implementation, the network search module 130 includes a truncation interval unit, a scaling factor calculation unit, a quantization error calculation unit and a parameter range update unit, wherein the truncation interval unit is used to determine the quantization truncation interval in the current search based on the current parameter range and the set number of grid searches; the scaling factor calculation unit is used to calculate the candidate quantization scaling factor in the current search according to the quantization accuracy and the quantization truncation interval; the quantization error calculation unit is used to calculate the quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range; the parameter range update unit is used to update the initial parameter range or the parameter range in the previous search using the quantization truncation interval in the current search when the quantization error is less than the quantization error threshold.
[0106] Furthermore, the quantization error calculation unit includes a quantization value calculation subunit and an error calculation subunit, wherein the quantization value calculation subunit is used to calculate the post-quantization value of each model parameter under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization value of each model parameter; the error calculation subunit is used to calculate the quantization error of all the model parameters in this round of search based on the pre-quantization value and the post-quantization value of each model parameter.
[0107] Furthermore, the quantization value calculation subunit is specifically used to, according to the parameter quantization range, clip the ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor to obtain a clipping error; and then, based on the clipping error and the zero point offset during quantization, calculate the post-quantization value of the corresponding model parameter under the candidate quantization scaling factor. When the symmetric quantization method is adopted, the zero point offset is zero; when the asymmetric quantization method is adopted, the zero point offset is a non-zero value.
[0108] As an optional implementation, the truncation interval unit is specifically used to calculate the ratio of the maximum value in the current parameter range to the number of grid searches, multiply the ratio by the current search sequence number to obtain the quantization truncation value for this round of search; then, select the positive and negative values of the quantization truncation value as the two boundary values of the quantization truncation interval, respectively, to obtain the quantization truncation interval for this search.
[0109] It can be understood that the device of this embodiment corresponds to the network model dynamic quantization method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.
[0110] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned terminal device. For example, the computer-readable storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0111] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0112] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0113] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.
[0114] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.
Claims
1. A dynamic quantization method for network models, It is characterized in that include: Obtain all model parameters and quantization accuracy of the network model trained based on point cloud data; Determining a parameter quantization range based on the quantization accuracy, and determining an initial parameter range based on all the model parameters; When performing each round of search, the quantization truncation interval of the current round of search is determined based on the current parameter range and the set number of grid searches, the candidate quantization scaling factor of the current round of search is calculated according to the quantization accuracy and the quantization truncation interval, and the quantization error of all the model parameters under the candidate quantization scaling factor is calculated based on the parameter quantization range, and when the quantization error is less than the quantization error threshold, the initial parameter range or the parameter range of the previous round of search is updated using the quantization truncation interval of the current round of search; According to the quantization error obtained in each round of search, a target quantization scaling factor that minimizes the quantization error is determined, so as to determine a quantized network model based on the target quantization scaling factor.
2. The network model dynamic quantization method according to claim 1, It is characterized in that Also includes: When the quantization error in this round of search is less than the quantization error threshold, the current quantization error threshold is updated by using the quantization error in this round of search.
3. The network model dynamic quantization method according to claim 1, It is characterized in that The calculating, based on the parameter quantization range, the quantization errors of all the model parameters under the candidate quantization scaling factor comprises: Calculating a post-quantization value of each of the model parameters under the candidate quantization scaling factor based on the parameter quantization range and the pre-quantization value of each of the model parameters; According to the pre-quantization value and the post-quantization value of each model parameter, the quantization errors of all the model parameters in this round of search are calculated.
4. The network model dynamic quantization method according to claim 3, It is characterized in that The calculating, based on the parameter quantization range and the pre-quantization value of each of the model parameters, the post-quantization value of each of the model parameters under the candidate quantization scaling factor comprises: According to the parameter quantization range, clipping the ratio of the pre-quantization value of the corresponding model parameter to the candidate quantization scaling factor to obtain a clipping error; Calculating the quantized value of the corresponding model parameter under the candidate quantization scaling factor based on the clipping error and the zero offset during quantization; When the symmetric quantization method is adopted, the zero point offset is zero; when the asymmetric quantization method is adopted, the zero point offset is a non-zero value.
5. The network model dynamic quantization method according to claim 4, It is characterized in that The calculation formula for the quantized value is: In the formula, x and represents the pre-quantization value and the post-quantization value of the corresponding model parameter; s represents the candidate quantization scaling factor, q min and q max represents the minimum and maximum values of the parameter quantization range; z is the zero point offset, and the function clamp() represents clipping the values that exceed the parameter quantization range.
6. The network model dynamic quantization method according to claim 1, It is characterized in that The determining of the quantization cutoff interval in the current round of search based on the current parameter range and the set number of grid searches includes: Calculate the ratio of the maximum value in the current parameter range to the number of grid searches, and multiply the ratio by the current search sequence number to obtain a quantization cutoff value for the current search; The positive and negative values of the quantization cutoff value are selected as two boundary values of the quantization cutoff interval respectively, so as to obtain the quantization cutoff interval in this search.
7. The network model dynamic quantization method according to claim 6, It is characterized in that The calculation formula of the candidate quantization scaling factor is: s=(x max -x min ) / (2 b -1);; Where s represents the candidate quantization scaling factor, x min and x max represents the lower boundary value and the upper boundary value of the quantization truncation interval, and b is the quantization accuracy.
8. A network model dynamic quantization device, It is characterized in that include: The data acquisition module is used to obtain all model parameters and quantization accuracy of the network model trained based on point cloud data; A range determination module, configured to determine a parameter quantization range based on the quantization precision, and to determine an initial parameter range based on all the model parameters; A network search module, for determining the quantization cutoff interval in the current search based on the current parameter range and the set number of grid searches during each round of search, calculating the candidate quantization scaling factor in the current search based on the quantization accuracy and the quantization cutoff interval, and calculating the quantization errors of all the model parameters under the candidate quantization scaling factor based on the parameter quantization range, and when the quantization error is less than the quantization error threshold, using the quantization cutoff interval in the current search to update the initial parameter range or the parameter range in the previous search; The model quantization module is used to determine a target quantization scaling factor that minimizes the quantization error according to the quantization error obtained in each round of search, so as to determine a quantized network model based on the target quantization scaling factor.
9. A terminal device, It is characterized in that The terminal device includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the computer program to implement the network model dynamic quantization method according to any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that It stores a computer program, which, when executed on a processor, implements the network model dynamic quantization method according to any one of claims 1-7.
Citation Information
Patent Citations
Model compression method and device, electronic equipment and storage medium
CN117973481A
Network model quantization method and electronic apparatus
US20220036162A1
Cited By
Model quantification method and device, storage medium, product and terminal equipment
CN121835777A
Network model dynamic quantization method and apparatus, terminal device, and readable storage medium
WO2026031454A1
Method and apparatus for quantizing floating-point numbers, and electronic device and storage medium
WO2026200578A1
