An online quantization method, electronic device, and readable storage medium for 3D point cloud object detection models.

Abstract

This disclosure relates to an online quantization method for a 3D point cloud object detection model, comprising: converting the sparse floating-point feature values ​​into a first sparse low-ratio specific number of points based on the distribution zeros and scaling factors of the calculated sparse floating-point feature values; converting the sparse floating-point weights into a second sparse low-ratio specific number of points based on the distribution zeros and scaling factors of the calculated sparse floating-point weights; obtaining the convolution output value of a 3D sparse convolution module of the 3D point cloud object detection model by convolution based on the first sparse low-ratio specific number of points and the second sparse low-ratio specific number of points; dequantizing the convolution output value of the 3D sparse convolution module to output data in point cloud data format; and updating the sparse floating-point weights of each layer of the 3D sparse convolution module based on the gradient between the output data and the input point cloud data.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, specifically to the field of neural grid model quantization, and more specifically, to an online quantization method for point cloud object detection models. Background Technology

[0002] In advanced autonomous driving perception solutions, 3D target detection algorithms based on LiDAR are crucial for accurately perceiving surrounding obstacles. Therefore, how to leverage high-performance LiDAR algorithms with limited resources has become a core issue. Summary of the Invention

[0003] Currently feasible methods include designing lightweight 3D point cloud models, accelerating the underlying computing units of the point cloud as much as possible, optimizing memory access, and compressing the model. These methods have been proven in neural network theory research to reduce the computational load of the model, accelerate model inference, and reduce the model's consumption of hardware resources. At present, model compression mainly falls into two categories. One is model pruning, which involves analyzing the impact of inter-layer parameters on model performance and pruning redundant parameters to reduce the computational load and achieve model compression. The other is model quantization, which uses low-bit integers to represent model parameters, significantly reducing the space occupied by the model and improving the inference speed, thus achieving model compression. From an algorithmic perspective, point cloud data consists of sparse reflections formed on the surface of an object by discrete points scanned by LiDAR. This results in many blank, data-free areas in a 3D point cloud scene. Since the image is a dense image composed of continuous pixels, using normal 3D convolution would waste significant resources calculating these invalid areas. Therefore, researchers have designed 3D Sparse Convolution (3D Conv) modules to extract sparse 3D point cloud features. Unlike normal 2D Convolution (2D Conv) modules, these sparse 3D features do not require continuous sliding convolution kernels for feature calculation. Instead, the possible locations of the output are determined based on the effective input feature values, forming an input-output data mapping table. Discrete convolution kernels are then calculated based on this table to obtain the final sparse feature map. Due to the unique nature of 3D Sparse Convolution modules, there are few quantization methods specifically designed for 3D point cloud algorithms.

[0004] Furthermore, although some research has proposed offline quantization methods, the most commonly used quantization algorithm in the actual industrial deployment of 2D image algorithms is still the 8-bit post-training quantization algorithm. However, for 3D data such as point clouds with uncertain data distribution, the fixed-point quantization model suffers a significant performance loss compared to the floating-point model. Although there are optimization methods for post-training quantization, such as symmetric and asymmetric methods, truncation / boundary value fitting, the disruption of the original data distribution caused by quantization is still unavoidable, and therefore generally will inevitably lead to a loss of accuracy.

[0005] Therefore, this invention aims to provide a relatively mature online quantization method for point cloud models based on 3D coefficient convolution, namely Quantization Aware Training (QAT). This method takes into account the loss caused by quantization operations during training, so that the trained parameters naturally adapt to the fixed point distribution, thereby reducing the accuracy loss caused by quantization operations and providing a more effective method for point cloud model quantization and compression.

[0006] A brief overview of this disclosure is given below to provide a basic understanding of some aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0007] According to a first aspect of this disclosure, a quantization method for a 3D point cloud object detection model is provided, comprising:

[0008] S1. Statistically analyze the overall distribution of floating-point feature values ​​of the input point cloud data during training and calculate the mapping table between the original point cloud data and the sparse point cloud data;

[0009] S2. Calculate the distribution zeros and scaling factors of the sparse floating-point feature values ​​of the 3D sparse convolution module of the 3D point cloud target detection model, as well as the distribution zeros and scaling factors of the sparse floating-point weights, and update the calculated distribution zeros and scaling factors in real time.

[0010] S3. Based on the calculated distribution zeros and scaling factor of the sparse floating-point feature values, the sparse floating-point feature values ​​are converted into a first sparse low-ratio specific number of points, and based on the calculated distribution zeros and scaling factor of the sparse floating-point weights, the sparse floating-point weights are converted into a second sparse low-ratio specific number of points.

[0011] S4. Based on the first sparse low ratio specific number of points and the second sparse low ratio specific number of points, the convolution output value of the 3D sparse convolution module of the 3D point cloud target detection model is obtained;

[0012] S5. Dequantize the convolution output value of the 3D sparse convolution module to output data in point cloud data format;

[0013] S6. Update the sparse floating-point weights of each layer of the 3D sparse convolution module based on the gradient between the output data and the input point cloud data; and

[0014] S7. Repeat steps S2-S6 based on the updated sparse floating-point weights until the difference between the feature distribution of the first sparse low-ratio specific number of points and the second sparse low-ratio specific number of points obtained in the current training and the feature distribution obtained in the previous training is within a predetermined range.

[0015] According to a second aspect of this disclosure, an electronic device is provided, comprising: one or more processors and one or more memories having executable instructions stored thereon, the executable instructions, when executed by the one or more processors, causing the one or more processors to perform the method according to a first aspect of this disclosure.

[0016] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided having executable instructions stored thereon, the executable instructions causing the one or more processors to perform the method according to a first aspect of this disclosure when executed by one or more processors.

[0017] Other features and advantages of the invention will become clearer from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0019] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0020] Figure 1 This is a schematic diagram illustrating a quantization method for a 3D point cloud target detection model according to an exemplary embodiment of the present disclosure;

[0021] Figure 2 This is a block diagram illustrating the configuration of a quantization method for a 3D point cloud object detection model according to exemplary embodiments of the present disclosure; and

[0022] Figure 3 This is a block diagram illustrating an application example of a quantization method for a 3D point cloud object detection model according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0023] The present invention will be described in the following order:

[0024] 1. Example

[0025] 2. Application Examples

[0026] 3. Conclusion

[0027] In the following description, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in this specification and the drawings, the same reference numerals are sometimes used across different drawings to denote the same parts or parts having the same function, thus omitting repeated descriptions. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0028] For ease of understanding, the positions, dimensions, and extents of the structures shown in the accompanying drawings and other references may not represent actual positions, dimensions, and extents. Therefore, the disclosed invention is not limited to the positions, dimensions, and extents disclosed in the accompanying drawings and other references.

[0029] 1. Example

[0030] The following will combine Figure 1 and Figure 2 The present disclosure provides a detailed description of a quantization method for a 3D point cloud object detection model according to exemplary embodiments. Those skilled in the art will understand that the invention is not limited to the steps shown in the figures, but can be implemented using the quantization method for a 3D point cloud object detection model.

[0031] Figure 1 This is a schematic diagram illustrating a quantization method 100 for a 3D point cloud target detection model according to an exemplary embodiment of the present disclosure.

[0032] In step 101, the overall distribution of floating-point feature values ​​of the input point cloud data during training is statistically analyzed, and a mapping table between the original point cloud data and the sparse point cloud data is calculated.

[0033] In some embodiments of this disclosure, when the original floating-point data is of type float32, the point cloud is first divided into grids within a certain area. For example, if the sensing area is 100m in front and back, 50m in the left and right, and 4m in the top and bottom, and each grid is 0.05cm*0.05cm*0.1cm in size, then there are a total of 2000*1000*40 grids. Most of the invalid grids do not contain point clouds. The point clouds existing in each valid grid are averaged and normalized to generate N*4 float32 voxel floating-point features. The average value and floating-point range are normalized. N refers to the number of valid voxels, 4 refers to the floating-point attributes xyz and intensity of the point cloud, and the N*3 spatial index position of each network is recorded. 3 refers to the integer index position of the grid voxel.

[0034] Furthermore, as mentioned above, since point cloud voxels are sparse, there are mostly blank areas. The 3D point cloud target detection model according to the present invention includes a 3D sparse convolution module and a 2D convolution (2D conv) module (see detailed description for more information). Figure 2The difference between the 3D sparse convolution module and the 3D convolution module is that the 3D sparse convolution module only calculates and outputs 3D sparse point cloud features on non-empty input voxels. Based on the position of the input features, the size of the convolution kernel, the stride of the convolution kernel, and other attribute information, the correspondence between the input data and the position of the output sparse features can be directly calculated, i.e., the input-output mapping table, so as to facilitate the acceleration of convolution calculation by using the mapping table during calculation.

[0035] In some embodiments according to this disclosure, sparse point cloud data is extracted using a 3D sparse convolution module, thereby improving the computational efficiency of the 3D point cloud target detection model.

[0036] Next, proceed to step 102. In step 102, calculate the distribution zeros and scaling factors of the sparse floating-point feature values ​​of the 3D sparse convolution module of the 3D point cloud target detection model, as well as the distribution zeros and scaling factors of the sparse floating-point weights, and update the calculated distribution zeros and scaling factors in real time.

[0037] In some embodiments according to this disclosure, a distribution histogram of sparse floating-point feature values ​​and sparse floating-point weights is collected; the histogram is truncated according to different truncation lengths to obtain truncated features; a low-ratio specific number of points of the truncated features is created and supplemented to the sample length before truncation to obtain quantized features; the relative entropy of the probability distribution between the truncated features and quantized features under different truncation lengths is iteratively calculated; and the distribution zero point z and scaling factor s are calculated by the following formulas (1) and (2):

[0038] s=(F max -F min ) / (Q max -Q min )……(1)

[0039]

[0040] in,

[0041] F max The maximum value of the truncation feature represents the minimum relative entropy.

[0042] F min The minimum value of the truncation feature represents the minimum value of the relative entropy.

[0043] Q max This represents the maximum value of a specific number of points, and

[0044] Q min This represents the minimum value of a specific point.

[0045] It should be noted that, in order not to obscure the calculation methods of the above-mentioned distribution zeros and compression factors, this invention does not distinguish between sparse floating-point eigenvalues ​​and sparse floating-point weights. However, the above calculation methods simultaneously satisfy the calculation of the distribution zeros and compression factors of sparse floating-point eigenvalues ​​and sparse floating-point weights.

[0046] Furthermore, in some embodiments according to this disclosure, the low specific point number may include at least one of the quantization types such as int8, int4, and int2.

[0047] Next, proceed to step 103. In step 103, based on the calculated distribution zeros and scaling factor of the sparse floating-point feature values, the sparse floating-point feature values ​​are converted into a first sparse low-ratio specific number of points, and based on the calculated distribution zeros and scaling factor of the sparse floating-point weights, the sparse floating-point weights are converted into a second sparse low-ratio specific number of points.

[0048] In some implementations, the sparse floating-point eigenvalues ​​and sparse floating-point weights are converted based on the following formula (3):

[0049]

[0050] in,

[0051] x f The floating-point number representing the current layer of the 3D sparse convolution module includes at least one of the floating-point number of sparse floating-point feature values ​​and the floating-point number of sparse floating-point weights.

[0052] s represents the scaling factor of the current layer of the 3D sparse convolution module.

[0053] z represents the distribution zero point of the current layer of the 3D sparse convolution module.

[0054] b represents the number of bits. For example, when the lower-order specific point number is int8 quantization mode, b = 8.

[0055] x int It represents at least one of the first sparse ratio specific number of points and the second sparse ratio specific number of points.

[0056] round indicates rounding to the nearest whole number, and...

[0057] "Clip" means to cut off.

[0058] In some embodiments according to this disclosure, a conversion is performed from the original float32 point cloud type to a low-ratio specific number of points of int8 quantization type. In int8 quantization type mode, both the first and second sparse low-ratio specific number of points are of int8 quantization type. That is, the first and second sparse low-ratio specific number of points are of the same quantization type.

[0059] Next, proceed to step 105. In step 105, the convolution output value of the 3D sparse convolution module is dequantized to output data in point cloud data format.

[0060] In some embodiments according to this disclosure, the convolution output value of the 3D sparse convolution module is dequantized to floating-point output data in point cloud data format based on the following formula (4):

[0061] y = dequant(x int′ )+b′……(4)

[0062] in,

[0063] y represents the dequantized output data of the convolution output value of the 3D sparse convolution module.

[0064] dequant represents the inverse quantization operation.

[0065] x int’ This represents the convolution output value of the 3D sparse convolution module, and

[0066] b' indicates sparse floating-point bias.

[0067] Next, we proceed to step 106. In step 106, the sparse floating-point weights of each layer of the 3D sparse convolutional module are updated based on the gradient between the output data and the input point cloud data.

[0068] In some embodiments of this disclosure, the sparse floating-point weights of each layer of the 3D sparse convolution module are updated according to gradient descent.

[0069] Next, proceed to step 107. In step 107, repeat steps 102-106 above based on the updated sparse floating-point weights until the difference between the feature distribution of the first sparse low-ratio specific number of points and the second sparse low-ratio specific number of points obtained by the current training and the feature distribution obtained by the previous training is within a predetermined range.

[0070] In other words, after each update of the sparse floating-point weights, steps S102-S106 are iteratively executed based on the updated sparse floating-point weights until the feature distribution difference of a specific number of sparse points is within an acceptable predetermined range. It should be noted that the acceptable predetermined range described in this paper is the generally accepted error range in the field of 3D point cloud object detection quantization.

[0071] Next, proceed to step 108. In step 108, the output data of the trained 3D sparse convolutional module is mapped to 2D dense floating-point feature values ​​and input into the 2D convolutional module of the 3D point cloud object detection model.

[0072] Next, proceed to step 109. In step 109, calculate the distribution zeros and scaling factors of the dense floating-point feature values ​​of each layer of the 2D convolutional module, as well as the distribution zeros and scaling factors of the dense floating-point weights.

[0073] Next, proceed to step 110. In step 110, based on the calculated distribution zeros and scaling factors of the dense floating-point feature values, the dense floating-point feature values ​​are converted into a first dense low-ratio specific number of points, and the dense floating-point weights are converted into a second dense low-ratio specific number of points.

[0074] As described above, in some embodiments according to this disclosure, the dense low-ratio specific point number feature is a dense int8 fixed point number.

[0075] Next, we proceed to step 111. In step 111, the convolution output value of the 2D convolution module is obtained based on the first dense low-ratio specific number of points and the second dense low-ratio specific number of points convolution.

[0076] Next, proceed to step 112. In step 112, the convolution output values ​​of the 2D convolution module are converted into output data in point cloud data format.

[0077] In some implementations, the convolution output values ​​of the 2D convolution module are converted into output data in point cloud data format based on the following formula (5):

[0078] x f′ =s′×(x int″ -z′)……(5)

[0079] in,

[0080] s' represents the scaling factor of the current output layer of the 2D convolutional module.

[0081] x int″ This represents the 2D dense low-ratio specific point count feature.

[0082] z′ represents the zero point of the current layer of the 2D convolutional module, and

[0083] x f′ A floating-point number representing the target data format.

[0084] In some implementations, the output data in the point cloud data format is further post-processed (e.g., anchor calculation and NMS non-maximum suppression) and finally outputs the 3D bounding box information of the obstacle target.

[0085] The quantization method for a 3D point cloud object detection model according to an exemplary embodiment of this disclosure introduces quantization simulation nodes during network training, packaging the original convolutional layers as quantization nodes, thereby introducing quantization errors into the network training. In the quantized fixed-point inference process, the quantization operations involved mainly include two parts: one is the quantization of the neural network layer weights, and the other is the quantization of the activation values ​​of each layer's output. Therefore, the main approach of online quantization is to embed quantization / dequantization operations for weights and activation in each layer during the forward process. Dequantization is used during the training phase, indicating that the actual data used in training is still in floating-point form, but the quantization error of each layer has been introduced into the network training process, thereby minimizing the quantization error through gradient descent.

[0086] In the quantization method according to this disclosure, during online quantization, the weights of the floating-point point cloud model are updated simultaneously to make the point cloud model more accurate, while the quantization parameter values ​​of the floating-point model are updated to convert it to a low-precision point model, so as to reduce the accuracy loss after quantization. Here, the quantization parameters include the distribution zeros and scaling factors of the floating-point feature values, as well as the channel-wise distribution zeros and scaling factors of the weights of each convolutional layer.

[0087] Figure 2 A block diagram illustrating the configuration of a quantization method for a 3D point cloud object detection model according to exemplary embodiments of the present disclosure is shown. For a more complete and comprehensive understanding of the invention, the following will describe... Figure 2 The following example illustrates a specific example of a quantization method for a 3D point cloud object detection model according to exemplary embodiments of the present disclosure. Please note that this example is not intended to limit the scope of the invention. For example, the invention is not limited to... Figure 2 It is not a specific configuration shown, but is applicable to all 3D point cloud target detection models with the same requirements or design considerations.

[0088] like Figure 2 As shown, the 3D point cloud object detection model 201 includes a 3D sparse convolution module 2011 and a 2D convolution module 2012. After quantization training of the 3D sparse convolution module 2011 of the 3D point cloud object detection model 201, the output data of the 3D sparse convolution module 2011 is fed to the 2D convolution module 2012 for quantization training, finally obtaining the quantized trained 3D point cloud object detection model 202, which includes a 3D sparse convolution quantization module 2021 and a 2D convolution quantization module 2022.

[0089] Still as Figure 2As shown, after completing the online quantization of the 3D point cloud target detection model 201, i.e., obtaining the 3D sparse convolutional quantization module 2021 and the 2D convolutional quantization module 2022, the preprocessed input data can be quantized using the updated quantization parameters: after passing through the 3D sparse convolutional quantization module 2021 and the 2D convolutional quantization module 2022, the int8 network prediction information is output, dequantized to the float32 floating-point result for post-processing (e.g., anchor calculation and NMS non-maximum suppression), and finally the 3D bounding box information of the float32 obstacle target is output.

[0090] To avoid unnecessary obfuscation of this invention, further details will be omitted. Figure 2 The configuration shown is described in detail above. Figure 1 The quantization method described above for 3D point cloud object detection models is fully applicable to Figure 2 The configuration shown.

[0091] Taking the SECOND model as an example, after online quantization, the memory space occupied by the SECOND model is reduced to about 25%, the inference speed of the int8 model is improved by 60%, and compared with the float32 model, the model is based on lossless and the accuracy is maintained at the level of the floating-point model.

[0092] 2. Application Examples

[0093] Figure 3 This is a block diagram illustrating an example of a schematic configuration of a vehicle 300 to which the present disclosure may be applied. The vehicle 300 includes a processor 301, a memory 302, a storage device 303, an external connection interface 304, a camera device 305, a sensor 306, a microphone 307, an input device 308, a display device 309, a speaker 310, a wireless communication interface 311, and a bus 312.

[0094] Processor 301 may be, for example, a CPU or a system-on-a-chip (SoC), and controls the application layer and other functions of vehicle 300. Memory 302 includes RAM and ROM, and stores data and programs executed by processor 301. Storage device 303 may include storage media such as semiconductor memory and hard disk. External connection interface 304 is an interface for connecting external devices (such as memory cards and Universal Serial Bus (USB) devices) to vehicle 300.

[0095] Camera device 305 includes an image sensor (such as a charge-coupled device (CCD) and complementary metal-oxide-semiconductor (CMOS)) and generates captured images. Sensor 306 may include a set of sensors, such as a measurement sensor, a gyroscope sensor, a geomagnetic sensor, and an accelerometer. Microphone 307 converts sound input to vehicle 300 into an audio signal. Input device 308 includes, for example, a touch sensor, keypad, keyboard, button, or switch configured to detect touches on the screen of display device 309 and receives operations or information input from the user. Display device 309 includes a screen (such as a liquid crystal display (LCD) and an organic light-emitting diode (OLED) display) and displays the output image of vehicle 300. Speaker 310 converts the audio signal output from vehicle 300 into sound.

[0096] The wireless communication interface 311 supports any communication scheme and performs wireless communication.

[0097] Bus 312 connects processor 301, memory 302, storage device 303, external connection interface 304, camera device 305, sensor 306, microphone 307, input device 308, display device 309, speaker 310 and wireless communication interface 311 to each other.

[0098] exist Figure 3 Of the 300 vehicles shown, reference Figure 1 The described method can be implemented by processor 301 and memory 302.

[0099] Furthermore, it should be noted that although this document describes the online quantization method for 3D point cloud models according to this disclosure using vehicles as an application example, this does not constitute a limitation on the quantization method according to this disclosure. In other words, the online quantization method for 3D point cloud models according to this disclosure can be used in any similar communication and image processing scenarios.

[0100] 3. Conclusion

[0101] In an exemplary embodiment according to this disclosure, during online quantization, the weights of the floating-point point cloud model are updated simultaneously to make the point cloud model more accurate, while the quantization parameter values ​​for converting the floating-point model to a lower-ratio specific point model are also updated, resulting in lower accuracy loss after quantization. Based on the online quantization model inference process of this disclosure, memory bandwidth and storage space are reduced compared to the original floating-point model, while simultaneously improving system throughput and inference speed, maintaining the accuracy requirements of the floating-point model, and with an accuracy loss not exceeding one percentage point.

[0102] It should be noted that the terms "front," "back," "top," "bottom," "above," and "below," if present in the specification and claims, are used for descriptive purposes and do not necessarily describe unchanging relative positions. It should be understood that such terms are interchangeable where appropriate, so that embodiments of this disclosure described herein can, for example, operate on orientations different from those shown or otherwise described herein.

[0103] As used herein, the term "exemplary" means "serving as an example, instance, or illustration," and not as a "model" to be precisely copied. Any implementation described herein by example is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, this disclosure is not limited to any theory expressed or implied as given in the foregoing technical field, background, summary of invention, or detailed description.

[0104] As used herein, the term "basic" means any minor variation that may be caused by design or manufacturing defects, device or component tolerances, environmental influences and / or other factors. The term "basic" also allows for differences from the perfect or ideal situation due to parasitic effects, noise, and other practical considerations that may exist in actual implementation.

[0105] Additionally, terms such as “first,” “second,” etc., may be used in this document for reference purposes only and are not intended to be limiting. For example, unless the context clearly indicates otherwise, the words “first,” “second,” and other such numerical terms relating to structures or elements do not imply order or sequence.

[0106] It should also be understood that when the term “including / contains” is used herein, it indicates the presence of the indicated feature, whole, step, operation, unit and / or component, but does not preclude the presence or addition of one or more other features, wholes, steps, operations, units and / or components and / or combinations thereof.

[0107] In this disclosure, the term “provide” is used broadly to cover all ways of obtaining an object, and therefore “provide an object” includes, but is not limited to, “purchasing,” “preparing / manufacturing,” “arranging / setting up,” “installing / assembling,” and / or “ordering” an object.

[0108] Those skilled in the art will recognize that the boundaries between the above operations are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed among additional operations, and operations may be performed with at least partial overlap in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be changed in various other embodiments. However, other modifications, variations, and substitutions are equally possible. Therefore, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

[0109] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0110] This invention can be a system, method, and / or computer program product. A computer program product may include a computer and a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0111] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0112] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples above are merely illustrative and not intended to limit the scope of this disclosure. The various embodiments disclosed herein can be combined in any way without departing from the spirit and scope of this disclosure. Those skilled in the art should also understand that many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. An online quantization method for a 3D point cloud object detection model, comprising: S1. Statistically analyze the overall distribution of floating-point feature values ​​of the input point cloud data during training and calculate the mapping table between the original point cloud data and the sparse point cloud data; S2. Calculate the distribution zeros and scaling factors of the sparse floating-point feature values ​​of the 3D sparse convolution module of the 3D point cloud target detection model, as well as the distribution zeros and scaling factors of the sparse floating-point weights, and update the calculated distribution zeros and scaling factors in real time. S3. Based on the calculated distribution zeros and scaling factor of the sparse floating-point feature values, the sparse floating-point feature values ​​are converted into a first sparse low-ratio specific number of points, and based on the calculated distribution zeros and scaling factor of the sparse floating-point weights, the sparse floating-point weights are converted into a second sparse low-ratio specific number of points. S4. Based on the first sparse low ratio specific number of points and the second sparse low ratio specific number of points, the convolution output value of the 3D sparse convolution module of the 3D point cloud target detection model is obtained; S5. Dequantize the convolution output value of the 3D sparse convolution module to output data in point cloud data format; S6. Update the sparse floating-point weights of each layer of the 3D sparse convolution module based on the gradient between the output data and the input point cloud data; as well as S7. Repeat steps S2-S6 based on the updated sparse floating-point weights until the difference between the feature distribution of the first sparse low-ratio specific number of points and the second sparse low-ratio specific number of points obtained in the current training and the feature distribution obtained in the previous training is within a predetermined range.

2. The method according to claim 1, characterized in that, Also includes: S8. Map the output data of the trained 3D sparse convolution module to 2D dense floating-point feature values ​​and input them into the 2D convolution module of the 3D point cloud target detection model; S9. Calculate the distribution zeros and scaling factors of the dense floating-point feature values ​​of each layer of the 2D convolution module, as well as the distribution zeros and scaling factors of the dense floating-point weights; S10. Based on the calculated distribution zeros and scaling factors of the dense floating-point feature values, the dense floating-point feature values ​​are converted into a first dense low-ratio specific number of points, and the dense floating-point weights are converted into a second dense low-ratio specific number of points; S11. Obtain the convolution output value of the 2D convolution module based on the first dense low-ratio specific number of points and the second dense low-ratio specific number of points; as well as S12. Convert the convolution output value of the 2D convolution module into output data in point cloud data format.

3. The method according to claim 1, characterized in that, In step S2, Collect histograms of the distribution of sparse floating-point eigenvalues ​​and sparse floating-point weights; Cut off the data according to different cutoff lengths in the distribution histogram to obtain the cutoff features; The truncated feature is created with a low ratio of specific points and padded to the original sample length to obtain the quantized feature; Iteratively calculate the relative entropy of the probability distributions between truncated and quantized features for different truncation lengths; and The distribution zero point z and scaling factor s are calculated using the following formulas (1) and (2): S=(F max -F min ) / (Q max -Q min )……(1) in, F max The maximum value of the truncation feature represents the minimum relative entropy. F min The minimum value of the truncation feature represents the minimum value of the relative entropy. Q max This represents the maximum value of a specific number of points, and Q min This represents the minimum value of a specific point.

4. The method according to claim 1, characterized in that, In step S3, the sparse floating-point eigenvalues ​​and sparse floating-point weights are converted based on the following formula (3): in, x f The floating-point number representing the current layer of the 3D sparse convolution module includes at least one of the floating-point number of sparse floating-point feature values ​​and the floating-point number of sparse floating-point weights. s represents the scaling factor of the current layer of the 3D sparse convolution module. z represents the distribution zero point of the current layer of the 3D sparse convolution module. b represents the number of bits. x int It represents at least one of the first sparse ratio specific number of points and the second sparse ratio specific number of points. round indicates rounding to the nearest whole number, and... "Clip" means to cut off.

5. The method according to claim 1, characterized in that, In step S5, the convolution output value of the 3D sparse convolution module is dequantized to output data in point cloud data format based on the following formula (4): y=dequant(x int′ )+b′……(4) in, y represents the dequantized output data of the convolution output value of the 3D sparse convolution module. dequant represents the inverse quantization operation. x int’ This represents the convolution output value of the 3D sparse convolution module, and b' indicates sparse floating-point bias.

6. The method according to claim 2, characterized in that, In step S12, the 2D dense low-ratio specific point count feature is converted into the target data format based on the following formula (5): x f′ =s′×(x int″ -Z′)……(5) in, s' represents the scaling factor of the current output layer of the 2D convolutional module. x int” This represents the convolution output value of the 2D convolution module. z′ represents the distribution zero point of the current layer of the 2D convolution module, and x f′ This represents the output data in point cloud data format.

7. The method according to claim 1, characterized in that, Also includes: In step S6, the sparse floating-point weights of each layer of the 3D sparse convolution module are updated according to the gradient descent method.

8. The method according to claim 1, characterized in that The sparse low-ratio specific point count includes at least one of the quantization types int8, int4, and int2.

9. An electronic device, comprising: One or more processors, and One or more memories having executable instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of claims 1-8.

10. A non-transitory computer-readable storage medium having executable instructions stored thereon, the executable instructions, when executed by one or more processors, causing the one or more processors to perform the method as described in any one of claims 1-8.

Images