Point cloud target detection network model quantification method and system and readable storage medium

By quantitatively optimizing the input data and nodes of the point cloud target detection network model, the computing resources and bandwidth problems of the point cloud target detection network model on edge computing devices are solved, and the computing resources are reduced while ensuring accuracy. and the effect of running bandwidth enables real-time fast computing on edge devices.

CN118942061APending Publication Date: 2024-11-12HESAI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310540930.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the field of autonomous driving, due to its complex structure and high computing requirements, it is difficult to achieve real-time rapid computing on edge computing devices with limited cost, and the prior art is difficult to reduce computing resources while ensuring accuracy and Running bandwidth.

Method used

By quantifying the input data of the point cloud target detection network model, including segmented quantization and feature mapping of floating-point data, adjusting the quantized bit count and proportional coefficients, quantizing the input data and weights of the convolutional layer and fully connected layer Data, combining the quantitative strategies of feature extraction and object detection network models, optimize network structure and training strategies.

Benefits of technology

It effectively compresses the computing resources and operating bandwidth of the point cloud target detection network model, improves the accuracy and speed of the algorithm, and enables it to realize real-time and fast network computing on edge computing devices, maintaining the high accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942061A_ABST
    Figure CN118942061A_ABST
Patent Text Reader

Abstract

The invention provides a point cloud target detection network model quantification method and system and a readable storage medium, and the method comprises the steps: obtaining a trained point cloud target detection network model; quantizing point cloud data of the input end of the trained point cloud target detection network model; and quantifying the trained point cloud target detection network model. According to the scheme, while the accuracy of the point cloud target detection network model is guaranteed, the computing resources and the operation bandwidth occupied by the network model are reduced, so that real-time and rapid network computing can be realized on the edge computing equipment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification relate to the field of neural network technology, and in particular to a point cloud target detection network model quantization method, system and readable storage medium. Background Art

[0002] In recent years, due to the rapid development of deep learning network models, deep learning network models have gradually been deployed on many mobile electronic devices such as smartphones, smart watches, smart homes, and self-driving cars.

[0003] However, due to the limitations of cost and hardware resources, the computing resources and computing power of deep learning network models on mobile electronic devices are often unable to meet the needs of actual applications.

[0004] For example, in the point cloud target detection task in the field of autonomous driving, since the point cloud data belongs to the three-dimensional spatial structure, the structure of the point cloud target detection network model is more complex than the network model structure with images as input, and more computing resources are required. However, due to the limitation of vehicle costs, it is often not possible to install more expensive high-computing power computing modules on the vehicle, and only cost-effective edge computing devices can be used.

[0005] Therefore, how to ensure the accuracy of the point cloud target detection network model while reducing the computing resources and operating bandwidth occupied by the network model so that it can achieve real-time and fast network computing on edge computing devices needs to be solved by technical personnel in this field.

[0006] The contents of the background technology section are merely the technologies known to the public and do not necessarily represent the existing technologies in the field. Summary of the invention

[0007] In response to the above technical problems, the embodiments of this specification provide a point cloud target detection network model quantization method, system and readable storage medium, which can reduce the computing resources and operating bandwidth occupied by the network model while ensuring the accuracy of the point cloud target detection network model, so that it can realize real-time and fast network computing on edge computing devices.

[0008] First, the embodiment of this specification provides a point cloud target detection network model quantization method, including:

[0009] Obtain the trained point cloud target detection network model;

[0010] quantizing the point cloud data at the input end of the trained point cloud object detection network model;

[0011] Quantizing the trained point cloud object detection network model.

[0012] Optionally, the acquiring a trained point cloud target detection network model includes:

[0013] Determining the network structure of the point cloud object detection network model;

[0014] Training the point cloud target detection network model under the precision of floating point cloud data;

[0015] Obtain the trained point cloud object detection network model.

[0016] Optionally, the point cloud data at the input end of the trained point cloud object detection network model includes floating point cloud data and / or integer point cloud data.

[0017] Optionally, quantizing the point cloud data at the input end of the trained point cloud object detection network model includes:

[0018] The floating-point point cloud data is segmented and quantized to a target quantization range.

[0019] Optionally, the step of quantizing the floating point cloud data into segments to a target quantization range includes:

[0020] Segmenting the floating point cloud data according to the characteristics of the floating point cloud data;

[0021] Based on the characteristics of each segmented data, each segmented data is mapped to a preset range, and each segmented data mapped to the preset range is quantized to a corresponding quantization range.

[0022] Optionally, the feature of the floating point cloud data includes any one of the following:

[0023] Data distribution density;

[0024] Data confidence;

[0025] Data accuracy;

[0026] Data region of interest.

[0027] Optionally, mapping each segmented data to a preset range based on the characteristics of each segmented data, and quantizing each segmented data mapped to the preset range to a corresponding quantization range includes:

[0028] Based on the characteristics of each segmented data, a linear mapping function of each segmented data is determined respectively;

[0029] Based on the linear mapping function and each segmented data, each segmented data is mapped to a preset range;

[0030] determining the number of quantization bits;

[0031] Determine a quantization scale coefficient based on the preset range and the number of quantization bits;

[0032] Based on the quantization scale coefficient, each segmented data mapped to the preset range is quantized to a corresponding quantization range.

[0033] Optionally, mapping each segmented data to a preset range based on the linear mapping function and each segmented data includes:

[0034] Mapping part of the segmented data in each segmented data to the maximum value of a preset range;

[0035] and / or

[0036] Part of the segmented data in each segmented data is mapped to the minimum value of the preset range.

[0037] Optionally, quantizing the point cloud data at the input end of the trained point cloud object detection network model includes:

[0038] A translation operation is performed on the integer point cloud data to translate it to a target range.

[0039] Optionally, the trained point cloud target detection network model includes: a feature extraction network model, suitable for extracting features from point cloud data; and a target detection network model coupled to the feature extraction network model, suitable for performing target detection based on the extracted feature data;

[0040] The quantizing the trained point cloud target detection network model includes at least one of the following:

[0041] quantizing the feature extraction network model;

[0042] Quantize the object detection network model.

[0043] Optionally, the quantizing the trained point cloud object detection network model includes:

[0044] Quantize the quantization nodes in the trained point cloud object detection network model, wherein the quantization nodes include convolutional layers and fully connected layers.

[0045] Optionally, the quantizing the quantized nodes in the trained point cloud object detection network model includes:

[0046] Quantize the input data and weight data of each quantization node.

[0047] Optionally, the quantizing the feature extraction network model includes:

[0048] Quantifying the feature extraction network model after training to obtain a first quantized feature extraction network model;

[0049] Determine whether the accuracy of the first quantitative feature extraction network model meets the preset first accuracy requirement. When it is determined that the preset first accuracy requirement is not met, adjust the quantization scale coefficient of the quantization after training until the accuracy meets the preset first accuracy requirement.

[0050] Optionally, after determining that the accuracy of the first quantitative feature extraction network model meets a preset first accuracy requirement, the method further includes:

[0051] Performing quantization training on the first quantitative feature extraction network model to obtain a second quantitative feature extraction network model;

[0052] Determine whether the accuracy of the second quantitative feature extraction network model meets a preset second accuracy requirement. When it is determined that the preset second accuracy requirement is not met, adjust the training strategy of the quantization training until the accuracy meets the preset second accuracy requirement. When it is determined that the preset second accuracy requirement is met, lock the weight of the second quantitative feature extraction network model.

[0053] Optionally, the quantizing the target detection network model includes:

[0054] Quantizing the target detection network model after training to obtain a first quantized target detection network model;

[0055] Determine whether the accuracy of the first quantized target detection network model meets a preset third accuracy requirement. When it is determined that the preset third accuracy requirement is not met, adjust the quantization scale coefficient of the post-training quantization until the accuracy meets the preset third accuracy requirement.

[0056] Optionally, after determining that the accuracy of the first quantized object detection network model meets a preset third accuracy requirement, the method further includes:

[0057] Performing quantization training on the first quantized object detection network model to obtain a second quantized object detection network model;

[0058] Determine whether the accuracy of the second quantized target detection network model meets a preset fourth accuracy requirement; when it is determined that the preset fourth accuracy requirement is not met, adjust the training strategy of the quantized training until the accuracy meets the preset fourth accuracy requirement; when it is determined that the preset fourth accuracy requirement is met, lock the weight of the second quantized target detection network model.

[0059] Optionally, the adjusting the training strategy of the quantization training includes:

[0060] Adjust the hyperparameters for quantized training.

[0061] Optionally, adjusting the hyper parameters of the quantization training includes:

[0062] Adjust at least one of the learning rate, loss function, and learning strategy of quantization training.

[0063] The embodiment of this specification also provides a point cloud target detection network model quantization system, including:

[0064] A model acquisition unit, adapted to acquire a trained point cloud target detection network model;

[0065] A point cloud quantization unit, adapted to quantize the point cloud data at the input end of the trained point cloud object detection network model;

[0066] The model quantization unit is adapted to quantize the trained point cloud target detection network model.

[0067] An embodiment of the present specification also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is run, the steps of the point cloud object detection network model quantization method described in any of the aforementioned embodiments are executed.

[0068] The point cloud target detection network model quantization method provided in the embodiment of this specification is adopted. By quantizing the point cloud data at the input end of the trained point cloud target detection network model and quantizing the trained point cloud target detection network model, on the one hand, the quantization of the trained point cloud target detection network model can effectively compress the trained point cloud target detection network model. On the other hand, considering the difference between point cloud data and traditional image data, quantizing the input point cloud data can further speed up the network operation speed and reduce the computing power requirements. Therefore, the point cloud target detection network model quantization method can reduce the computing resources and operating bandwidth occupied by the network model while ensuring the accuracy of the point cloud target detection network model, so that it can realize real-time and fast network computing on edge computing devices.

[0069] Furthermore, after determining the network structure of the point cloud target detection network model, since the point cloud target detection network model is trained under the precision of floating-point point cloud data, the precision loss of point cloud data can be avoided. Therefore, the obtained trained point cloud target detection network model has no precision loss and the model has a high accuracy, thereby ensuring the accuracy of the quantized trained point cloud target detection network model.

[0070] Furthermore, by performing segmented quantization on the floating-point point cloud data in the point cloud data at the input end of the trained point cloud target detection network model, the precision loss of the floating-point point cloud data during the quantization process can be reduced, thereby improving the accuracy of the quantized trained point cloud target detection network model.

[0071] Furthermore, by segmenting the floating-point point cloud data according to the characteristics of the floating-point point cloud data, and then mapping each segmented data to a preset range based on the characteristics of each segmented data, and then quantizing each segmented data mapped to the preset range to a corresponding quantization range, the quantization result can be made more consistent with the distribution characteristics of the floating-point point cloud distance data. In other words, the integer point cloud data obtained after quantization can retain as much information carried by the original floating-point point cloud distance data as possible, thereby reducing the accuracy loss of the floating-point point cloud data in the quantization process, and thus can improve the accuracy of the quantized trained point cloud target detection network model.

[0072] Furthermore, by determining the linear mapping function of each segmented data based on the characteristics of each segmented data, and then mapping each segmented data to a preset range based on the linear mapping function and each segmented data, and by determining the number of quantization bits, and then determining the quantization scale coefficient based on the preset range and the number of quantization bits, each segmented data mapped to the preset range can be quantized to the corresponding quantization range based on the quantization scale coefficient. Since different linear mapping functions are determined according to the original distribution of the point cloud data, and each segmented data is mapped to the preset range, the distribution of the quantized point cloud data can be adjusted, so that the precision loss of the floating point cloud data in the quantization process can be further reduced, so the accuracy of the trained point cloud target detection network model after quantization can be improved.

[0073] Furthermore, by mapping part of the segmented data to the maximum or minimum value of a preset range, a wider mapping range can be provided for other segmented data, so that more original floating-point point cloud data can retain as much information as possible, thereby further reducing the accuracy loss of the floating-point point cloud data during the quantization process, and thus further improving the accuracy of the quantized trained point cloud target detection network model.

[0074] Furthermore, since the integer point cloud data contained in the point cloud data at the input end of the trained point cloud target detection network model itself occupies a small storage space and requires fewer computing resources, it is only necessary to perform a translation operation on it and translate it to the target quantization range to adapt the quantization of the point cloud target detection network model.

[0075] Furthermore, the trained point cloud target detection network model may include a feature extraction network model and a target detection network model. Since the feature extraction network model is suitable for feature extraction of point cloud data, and the feature extraction network model is suitable for target detection based on the extracted feature data, by quantizing the feature extraction network model and the target detection network model respectively, the quantization strategy can be adaptively adjusted according to different training objectives of the network model, thereby further improving the accuracy of the quantized trained point cloud target detection network model.

[0076] Furthermore, the quantized nodes in the trained point cloud target detection network model may include convolutional layers and fully connected layers. Since the convolutional layer is an important component of the point cloud target detection network model, the convolution process involves a large number of calculations and is time-consuming. Therefore, in order to reduce the computing resources and operating bandwidth occupied by the network model, quantizing the convolutional layer has a significant effect; and, since each neuron in the fully connected layer has parameters connected to all neurons in the previous layer, there will be a large number of parameters, resulting in a series of problems such as excessive storage space, large amount of calculation, and slow operating efficiency. Therefore, quantizing the fully connected layer can further reduce the computing resources and operating bandwidth occupied by the network model.

[0077] Furthermore, by simultaneously quantizing the input data and weight data of each quantization node, the computational complexity of the point cloud target detection network model can be effectively reduced, thereby further reducing the computing resources and operating bandwidth occupied by the network model, enabling it to realize real-time and fast network computing on edge computing devices.

[0078] Furthermore, by quantizing the feature extraction network model after training, obtaining a first quantized feature extraction network model, and then adjusting the proportional coefficient of the quantization after training until the accuracy of the first quantized feature extraction network model meets the preset first accuracy requirement, the accuracy of the quantized feature extraction network model can be effectively guaranteed. In addition, on the one hand, since the post-training quantization method directly performs a quantization operation on the trained feature extraction network model, the quantization process is simple and easy to implement; on the other hand, since the coupling degree between the post-training quantization method and the model to be quantized is low, it has strong universality.

[0079] Furthermore, by performing quantization training on the first quantitative feature extraction network model to obtain a second quantitative feature extraction network model, and then adjusting the training strategy of the quantization training until the accuracy of the second quantitative feature extraction network model meets the preset second accuracy requirement, the compatibility of the quantized point cloud data and the feature extraction network model can be improved, thereby further ensuring the accuracy of the feature extraction network model.

[0080] Furthermore, by quantizing the target detection network model after training, obtaining a first quantized target detection network model, and then adjusting the proportional coefficient of the post-training quantization until the accuracy of the first quantized target detection network model meets the preset third accuracy requirement, the accuracy of the quantized target detection network model can be effectively guaranteed. In addition, on the one hand, since the post-training quantization method directly performs quantization operations on the trained target detection network model, the quantization process is simple and easy to implement; on the other hand, since the coupling degree between the post-training quantization method and the model to be quantized is low, it has strong universality.

[0081] Furthermore, by performing quantization training on the first quantized target detection network model to obtain a second quantized target detection network model, and then adjusting the training strategy of the quantization training until the accuracy of the second quantized feature extraction network model meets the preset fourth accuracy requirement, the compatibility of the quantized point cloud data with the target detection network model can be improved, thereby further ensuring the accuracy of the target detection network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0083] Figure 1 A schematic diagram of the steps of a point cloud target detection network model quantization method in an embodiment of this specification is shown;

[0084] Figure 2 A schematic diagram of the steps of obtaining a trained point cloud target detection network model in an embodiment of this specification is shown;

[0085] Figure 3 A schematic diagram of a step of quantizing the point cloud data at the input end of the trained point cloud target detection network model in an embodiment of this specification is shown;

[0086] Figure 4 A schematic diagram of the steps of a quantitative feature extraction network model in an embodiment of this specification is shown;

[0087] Figure 5 A schematic diagram of the steps of a quantized target detection network model in an embodiment of this specification is shown;

[0088] Figure 6 A flowchart of a point cloud target detection network model quantization method in a specific application in an embodiment of this specification is shown;

[0089] Figure 7 A schematic diagram showing the working principle of a symmetric quantizer in an embodiment of this specification is shown;

[0090] Figure 8 A schematic diagram of a topological structure of a quantitative model framework in an embodiment of this specification is shown;

[0091] Fig. 9 shows a data statistical histogram in a linear quantizer in an embodiment of this specification;

[0092] Fig.10 A schematic diagram showing a comparison of detection accuracy of a point cloud target detection network model under an unquantized floating-point precision in an embodiment of this specification and a point cloud target detection network model under a quantized integer precision in an embodiment of this specification is shown;

[0093] Fig.11a A schematic diagram of the detection results of a point cloud target detection network model based on an existing quantization scheme is shown;

[0094] Fig.11b A schematic diagram showing the detection results of the point cloud target detection network model under quantized integer precision in one embodiment of this specification is shown;

[0095] Fig.12 A structural schematic diagram of a point cloud target detection network model quantization system in an embodiment of this specification is shown. DETAILED DESCRIPTION

[0096] In recent years, due to the rapid development of deep learning network models, deep learning network models have gradually been deployed on many mobile electronic devices such as smartphones, smart watches, smart homes, and self-driving cars.

[0097] As described in the background technology, due to the limitations of cost and hardware resources, the computing resources and computing power of deep learning network models on mobile electronic devices are often difficult to meet actual application requirements.

[0098] For example, in the point cloud target detection task in the field of autonomous driving, since the point cloud data belongs to the three-dimensional spatial structure, the structure of the point cloud target detection network model is more complex than the network model structure with images as input, and more computing resources are required. However, due to the limitation of vehicle cost, it is often not possible to carry more expensive high-power computing modules on the vehicle, and only cost-effective edge computing devices can be used. Therefore, it is necessary to reduce the computing resources and operating bandwidth occupied by the network model while ensuring the accuracy of the point cloud target detection network model, so that it can realize real-time and fast network computing on edge computing devices such as vehicle terminals.

[0099] In response to the above problems, an embodiment of the present specification provides a point cloud target detection network model quantization method, which can reduce the computing resources and operating bandwidth occupied by the network model while ensuring the accuracy of the point cloud target detection network model, so that it can realize real-time and fast network computing on edge computing devices.

[0100] In order to enable those skilled in the art to better understand and implement the embodiments of this specification, the concepts, schemes, principles and advantages of the embodiments of this specification are described in detail below with reference to the drawings and through specific application examples.

[0101] First, the embodiment of this specification provides a point cloud target detection network model quantization method, referring to Figure 1 The schematic diagram of the steps of a point cloud target detection network model quantization method is shown. In some embodiments of this specification, the following steps can be used to quantize the point cloud target detection network model:

[0102] Step A: obtain the trained point cloud target detection network model.

[0103] It should be noted that the type of point cloud target detection network model applicable to the embodiments of this specification is not limited here, as long as the network model is suitable for point cloud target detection after training.

[0104] Step B, quantizing the point cloud data at the input end of the trained point cloud target detection network model.

[0105] In a specific implementation, different types of data included in the point cloud data may be quantized into data with fixed-point accuracy.

[0106] Step C, quantizing the trained point cloud object detection network model.

[0107] In a specific implementation, the trained point cloud target detection network model can be quantized to a point cloud target detection network model that performs forward calculations under fixed-point accuracy, thereby accelerating the speed of forward calculations of the point cloud target detection network model.

[0108] By adopting the above-mentioned embodiment, by quantizing the point cloud data at the input end of the trained point cloud target detection network model and quantizing the trained point cloud target detection network model, on the one hand, the quantization of the trained point cloud target detection network model can effectively compress the trained point cloud target detection network model, and on the other hand, considering the difference between point cloud data and traditional image data, quantizing the input point cloud data can further speed up the network operation speed and reduce the computing power requirements. Therefore, the point cloud target detection network model quantization method can ensure the accuracy of the point cloud target detection network model while reducing the computing resources and operating bandwidth occupied by the network model, so that it can realize real-time and fast network computing on the edge computing device.

[0109] In some embodiments of this specification, for step A, refer to Figure 2 A schematic diagram of steps for obtaining a trained point cloud target detection network model is shown. Specifically, the trained point cloud target detection network model can be obtained by the following steps:

[0110] Step A1, determining the network structure of the point cloud target detection network model.

[0111] In specific implementations, the point cloud target detection network model usually needs to implement three stages of work, including point cloud feature extraction, point cloud projection, and target detection. Since point cloud feature extraction and target detection correspond to different training objectives, different network structures need to be designed for the point cloud feature extraction stage and the target detection stage respectively.

[0112] Step A2: training the point cloud target detection network model under the floating point cloud data accuracy.

[0113] Specifically, by training the point cloud target detection network model under the precision of floating point cloud data, the precision loss of point cloud data and network model can be avoided.

[0114] Step A3, obtaining the trained point cloud target detection network model.

[0115] By adopting the above-mentioned embodiment, after determining the network structure of the point cloud target detection network model, since the point cloud target detection network model is trained under the precision of floating-point point cloud data, the precision loss of point cloud data can be avoided. Therefore, the obtained trained point cloud target detection network model has no precision loss and the accuracy of the model is relatively high, thereby ensuring the accuracy of the quantized trained point cloud target detection network model.

[0116] It should be noted that the above embodiments are only exemplary descriptions. In specific implementations, it is not limited to using the above method to obtain the trained point cloud target detection network model.

[0117] In a specific implementation, the point cloud data at the input end of the trained point cloud target detection network model may include floating-point point cloud data and integer point cloud data. For example, the data representing distance information contained in the point cloud data is usually floating-point point cloud data, while the data representing reflection intensity information is usually integer point cloud data.

[0118] It is understandable that the embodiments of this specification do not impose any restrictions on the data type of the point cloud data at the input end of the trained point cloud target detection network model.

[0119] In some implementations of this specification, reference is made to Figure 3 A schematic diagram of the steps of quantizing the point cloud data at the input end of the trained point cloud target detection network model is shown, and for step B, it may specifically include:

[0120] Step B1, quantizing the floating point cloud data into segments to a target quantization range.

[0121] By adopting the above embodiment, by segmentally quantizing the floating-point point cloud data in the point cloud data at the input end of the trained point cloud target detection network model, the accuracy loss of the floating-point point cloud data during the quantization process can be reduced, thereby improving the accuracy of the quantized trained point cloud target detection network model.

[0122] In a specific implementation, the floating point cloud data may be segmented and quantized to a target quantization range by the following steps:

[0123] Step B11, segmenting the floating point cloud data according to the characteristics of the floating point cloud data.

[0124] In a specific implementation, data distribution density, data confidence, data accuracy, data area of ​​interest, etc. can all be used as features of floating-point point cloud data.

[0125] Step B12, based on the characteristics of each segmented data, each segmented data is mapped to a preset range, and each segmented data mapped to the preset range is quantized to a corresponding quantization range.

[0126] In a specific implementation, each segmented data can be mapped to a preset range based on data distribution density, data confidence, data accuracy, data area of ​​interest, etc., and then each segmented data mapped to the preset range can be quantized to a corresponding quantization range.

[0127] By adopting the above embodiment, the quantization result can be made more consistent with the distribution characteristics of the floating-point point cloud distance data. In other words, the integer point cloud data obtained after quantization can retain as much information carried by the original floating-point point cloud distance data as possible, thereby reducing the accuracy loss of the floating-point point cloud data during the quantization process, and thus improving the accuracy of the trained point cloud target detection network model after quantization.

[0128] As a specific example, for example, quantization is performed based on the distribution density of each segmented data. Specifically, segmented data with a larger distribution density can be mapped to a wider range, and segmented data with a smaller distribution density can be mapped to a narrower range.

[0129] As another specific example, when the distribution density of some segmented data is much smaller than the distribution density of other segmented data, the part of the segmented data can also be mapped to the boundary value of the preset range, specifically, mapped to the maximum value or minimum value of the preset range.

[0130] By adopting the above embodiment, by mapping part of the segmented data to the boundary value of the preset range, a wider mapping range can be provided for other segmented data, so that more original floating-point point cloud data can retain as much information as possible, thereby further reducing the accuracy loss of the floating-point point cloud data in the quantization process, and thus further improving the accuracy of the quantized trained point cloud target detection network model.

[0131] In some embodiments of the present specification, for step B12, the following steps may be specifically used to quantize each segmented data to a corresponding quantization range:

[0132] Step B121, based on the characteristics of each segmented data, respectively determine the linear mapping function of each segmented data.

[0133] In a specific implementation, different linear mapping functions may be determined according to the original distribution of the point cloud data.

[0134] Step B122: Based on the linear mapping function and each segmented data, each segmented data is mapped to a preset range.

[0135] Step B123, determine the number of quantization bits.

[0136] Specifically, the lower the data accuracy, the smaller the storage space occupied and the less computing resources required. Experiments show that the point cloud target detection network model has a certain tolerance for parameter accuracy, and it is unnecessary to use the full-precision data format for point cloud target detection. Therefore, the floating-point precision point cloud data can be quantized to fixed-point data format.

[0137] As a specific example, point cloud data with floating point precision may be quantized to 8-bit integer precision.

[0138] It is understandable that the embodiments of this specification do not impose any specific limitation on the number of quantization bits. In other embodiments, point cloud data with floating-point precision may also be quantized to 4-bit integer precision.

[0139] Step B124, determining a quantization scale coefficient based on the preset range and the number of quantization bits.

[0140] Step B125, based on the quantization scale coefficient, quantize each segmented data mapped to the preset range to a corresponding quantization range.

[0141] By adopting the above embodiment, firstly, based on the characteristics of each segmented data, the linear mapping function of each segmented data is determined respectively, and then based on the linear mapping function and each segmented data, each segmented data is mapped to a preset range respectively, and by determining the number of quantization bits, the quantization scale coefficient is determined based on the preset range and the number of quantization bits, and each segmented data mapped to the preset range can be quantized to the corresponding quantization range based on the quantization scale coefficient. Since different linear mapping functions are determined according to the original distribution of the point cloud data, each segmented data is mapped to the preset range, and the distribution of the quantized point cloud data can be adjusted, so that the precision loss of the floating point cloud data in the quantization process can be further reduced, so the accuracy of the trained point cloud target detection network model after quantization can be improved.

[0142] In other embodiments of this specification, continue to refer to Figure 3 , for step B, it may specifically include:

[0143] Step B2, performing translation operation on the integer point cloud data to translate to the target range.

[0144] Using the above embodiment, since the storage space occupied by the integer point cloud data itself is relatively small and the computing resources required are also relatively small, it is only necessary to perform a translation operation on it and translate it to the target quantization range to adapt the quantization of the point cloud target detection network model.

[0145] In a specific implementation, the trained point cloud target detection network model may specifically include a feature extraction network model and a target detection network model, wherein:

[0146] The feature extraction network model is suitable for extracting features from point cloud data;

[0147] The target detection network model is coupled to the feature extraction network model and is suitable for performing target detection based on the extracted feature data.

[0148] In some embodiments of the present specification, only the feature extraction network model may be quantized.

[0149] In other embodiments of the present specification, only the target detection network model may be quantized.

[0150] In some other embodiments of the present specification, the feature extraction network model and the target detection network model may be quantized simultaneously.

[0151] In the specific implementation, since the feature extraction network model is suitable for feature extraction of point cloud data, and the feature extraction network model is suitable for target detection based on the extracted feature data, by quantizing the feature extraction network model and the target detection network model respectively, the quantization strategy can be adaptively adjusted according to the different training objectives of the network model, thereby further improving the accuracy of the quantized trained point cloud target detection network model.

[0152] Although the embodiments of this specification do not limit the specific structure of the point cloud target detection network model used, in specific implementations, the point cloud target detection network model can generally use a convolutional neural network for feature extraction, so the point cloud target detection network model generally includes a convolutional layer and a fully connected layer. Accordingly, for the point cloud target detection network model containing the above structure, it can be specifically quantized to quantize the quantized nodes in the trained point cloud target detection network model, wherein the quantized nodes include a convolutional layer and a fully connected layer.

[0153] Among them, the convolution layer is an important part of the point cloud target detection network model. The convolution process involves a large number of calculations and is time-consuming. Therefore, in order to reduce the computing resources and operating bandwidth occupied by the network model, quantizing the convolution layer has a significant effect.

[0154] A fully connected layer means that each neuron in the current layer has parameters connected to all neurons in the previous layer. Since each neuron in the two layers is connected to each other, there will be a large number of parameters, resulting in a series of problems such as excessive storage space, large amount of calculation, and low operating efficiency. Therefore, quantizing the fully connected layer can further reduce the computing resources and operating bandwidth occupied by the network model.

[0155] It can be understood that the embodiments of this specification do not impose specific restrictions on quantization nodes, and the above embodiments are only preferred examples.

[0156] In a specific implementation, the quantization nodes in the trained point cloud target detection network model are quantized, and specifically the input data and weight data of each quantization node can be quantized, wherein the input data corresponds to the point cloud data input to each quantization node, and the weight data corresponds to the weight parameters of each quantization node.

[0157] By adopting the above embodiment, by simultaneously quantizing the input data and weight data of each quantization node, the computational complexity of the point cloud target detection network model can be effectively reduced, thereby further reducing the computing resources and operating bandwidth occupied by the network model, enabling it to realize real-time and fast network computing on edge computing devices.

[0158] In some embodiments of this specification, step C may specifically include:

[0159] Step C1, quantizing the feature extraction network model.

[0160] In the specific implementation, refer to Figure 4 The schematic diagram of a step of quantizing a feature extraction network model is shown. Specifically, the feature extraction network model can be quantized using the following steps:

[0161] Step C11, quantizing the feature extraction network model after training to obtain a first quantized feature extraction network model.

[0162] Specifically, post-training quantization refers to directly performing a quantization operation on the trained feature extraction network model without training the model again, that is, the weight-end data obtained in the first quantized feature extraction network model is a quantized fixed-point value.

[0163] In some embodiments of the present specification, a symmetric quantizer may be added to the trained feature extraction network model to directly perform a quantization operation on the feature extraction network model. Specifically, a linear quantization operation and a linear inverse quantization operation of the symmetric quantizer may be added to both the input and weight ends of the convolutional layer and the fully connected layer of the trained feature extraction network model.

[0164] Step C12, determining whether the accuracy of the first quantitative feature extraction network model meets the preset first accuracy requirement. When it is determined that the accuracy does not meet the preset first accuracy requirement, adjusting the quantization scale coefficient of the quantization after training until the accuracy meets the preset first accuracy requirement.

[0165] Specifically, the data distribution information of each quantization node of the first quantitative feature extraction network model can be obtained, and then based on the distribution information, the proportional coefficient of the post-training quantization can be adjusted until the accuracy of the first quantitative feature extraction network model meets the preset first accuracy requirement.

[0166] By adopting the above embodiment, the feature extraction network model is quantized after training to obtain a first quantized feature extraction network model, and then the proportional coefficient of the quantization after training is adjusted until the accuracy of the first quantized feature extraction network model meets the preset first accuracy requirement, which can effectively ensure the accuracy of the quantized feature extraction network model. In addition, on the one hand, since the post-training quantization method directly performs a quantization operation on the trained feature extraction network model, the quantization process is simple and easy to implement; on the other hand, since the coupling degree between the post-training quantization method and the model to be quantized is low, it has strong universality.

[0167] In a specific implementation, after determining that the accuracy of the first quantitative feature extraction network model meets a preset first accuracy requirement, the method may further include:

[0168] Step C13: Perform quantization training on the first quantitative feature extraction network model to obtain a second quantitative feature extraction network model.

[0169] Specifically, quantization training refers to quantizing the model during its training process. By adjusting the weight distribution of the first quantitative feature extraction network model through quantization training, the impact of quantization disturbance can be reduced.

[0170] In actual implementation, quantized training usually faces the problem of gradient vanishing during the back-propagation process of the model, which can be solved by using a direct estimator.

[0171] Step C14, determining whether the accuracy of the second quantitative feature extraction network model meets the preset second accuracy requirement. When it is determined that the accuracy does not meet the preset second accuracy requirement, adjusting the training strategy of the quantization training until the accuracy meets the preset second accuracy requirement. When it is determined that the accuracy meets the preset second accuracy requirement, locking the weight of the second quantitative feature extraction network model.

[0172] In a specific implementation, the training strategy for adjusting the quantization training may specifically be adjusting the hyperparameters of the quantization training.

[0173] In some embodiments of the present specification, the accuracy of the second quantitative feature extraction network model can be adjusted by adjusting the learning rate of the quantitative training.

[0174] In other embodiments of the present specification, the accuracy of the second quantitative feature extraction network model may also be adjusted by adjusting the loss function of the quantization training.

[0175] In some other embodiments of the present specification, the accuracy of the second quantitative feature extraction network model can also be adjusted by adjusting the learning strategy of the quantitative training.

[0176] It is understandable that the present embodiment does not limit the specific implementation of adjusting the training strategy of the quantitative training, as long as the accuracy of the second quantitative feature extraction network model can be adjusted. For example, two or more hyperparameters can also be adjusted at the same time.

[0177] By adopting the above embodiment, the second quantitative feature extraction network model is obtained by performing quantization training on the first quantitative feature extraction network model, and then the training strategy of the quantization training is adjusted until the accuracy of the second quantitative feature extraction network model meets the preset second accuracy requirement. This can improve the compatibility of the quantized point cloud data with the feature extraction network model, thereby further ensuring the accuracy of the feature extraction network model.

[0178] In other embodiments of this specification, step C may specifically include:

[0179] Step C2, quantizing the target detection network model.

[0180] In the specific implementation, refer to Figure 5 The schematic diagram of a step of quantizing a target detection network model is shown in FIG. 1 . Specifically, the target detection network model can be quantized using the following steps:

[0181] Step C21, quantizing the target detection network model after training to obtain a first quantized target detection network model.

[0182] Specifically, reference may be made to the relevant description of step C11 in the aforementioned embodiment, which will not be repeated here.

[0183] Step C22, determine whether the accuracy of the first quantized target detection network model meets the preset third accuracy requirement. When it is determined that the preset third accuracy requirement is not met, adjust the quantization scale coefficient of the quantization after training until the accuracy meets the preset third accuracy requirement.

[0184] Specifically, reference may be made to the relevant description of step C12 in the aforementioned embodiment, which will not be repeated here.

[0185] By adopting the above embodiment, the target detection network model is quantized after training to obtain a first quantized target detection network model, and then the proportional coefficient of the quantization after training is adjusted until the accuracy of the first quantized target detection network model meets the preset third accuracy requirement, which can effectively ensure the accuracy of the quantized target detection network model. In addition, on the one hand, since the post-training quantization method directly performs a quantization operation on the trained target detection network model, the quantization process is simple and easy to implement; on the other hand, since the coupling degree between the post-training quantization method and the model to be quantized is low, it has strong universality.

[0186] In a specific implementation, after determining that the accuracy of the first quantized object detection network model meets a preset third accuracy requirement, the method may further include:

[0187] Step C23, performing quantization training on the first quantized object detection network model to obtain a second quantized object detection network model.

[0188] Specifically, reference may be made to the relevant description of step C13 in the aforementioned embodiment, which will not be repeated here.

[0189] Step C24, determine whether the accuracy of the second quantized target detection network model meets the preset fourth accuracy requirement. When it is determined that the preset fourth accuracy requirement is not met, adjust the training strategy of the quantized training until the accuracy meets the preset fourth accuracy requirement. When it is determined that the preset fourth accuracy requirement is met, lock the weight of the second quantized target detection network model.

[0190] Specifically, reference may be made to the relevant description of step C14 in the aforementioned embodiment, which will not be repeated here.

[0191] By adopting the above embodiment, the second quantized target detection network model is obtained by performing quantization training on the first quantized target detection network model, and then the training strategy of the quantized training is adjusted until the accuracy of the second quantized feature extraction network model meets the preset fourth accuracy requirement. The compatibility of the quantized point cloud data and the target detection network model can be improved, thereby further ensuring the accuracy of the target detection network model.

[0192] In order to enable those skilled in the art to better understand and implement the point cloud target detection network model quantization method in the embodiments of this specification, a specific example is used below to illustrate.

[0193] Reference Figure 6 The flowchart of the point cloud target detection network model quantization method in a specific application in the embodiment of this specification is shown. Specifically, the point cloud target detection network model quantization can be performed according to the following process:

[0194] S01, determine the network structure of the point cloud target detection network model.

[0195] In a specific implementation, the point cloud target detection network model may include a feature extraction network model suitable for extracting features from point cloud data and a target detection network model suitable for performing target detection based on the extracted feature data.

[0196] S02, training the point cloud target detection network model under the precision of floating point cloud data.

[0197] S03: Obtain the trained point cloud target detection network model.

[0198] S04, obtaining point cloud data at the input end of the trained point cloud target detection network model.

[0199] In a specific implementation, the point cloud data may include floating point cloud data representing distance information and integer point cloud data representing reflection intensity information.

[0200] S05: Segment the floating point cloud data according to the characteristics of the floating point cloud data.

[0201] In a specific implementation, the floating point cloud data representing distance information may be segmented according to distribution density.

[0202] Specifically, due to the strong short-range ranging capability of the laser radar, the density of the point cloud data collected at a close distance is often greater than the density of the point cloud data at a longer distance. Therefore, for the floating-point point cloud data representing distance information, segmentation can be performed with the goal of compensating for the accuracy of the point cloud data at a close distance by sacrificing the accuracy of the point cloud data at a longer distance.

[0203] S06, determining the linear mapping function of each segmented data based on the characteristics of each segmented data.

[0204] In a specific implementation, different linear mapping functions may be determined according to the original distribution of the point cloud data.

[0205] As a specific example, for example, according to the original distribution characteristics of the point cloud data, the segment nodes m and n are determined, and the original point cloud data distribution interval [0, k] is divided into three segments: [0, m), [m, n), and [n, k], and the linear mapping functions f1, f2, and f3 of each segment data are determined respectively.

[0206] S07, based on the linear mapping function and each segmented data, mapping each segmented data to a preset range respectively.

[0207] In a specific implementation, according to the linear mapping function of each segmented data determined in step S06, the data segment can be mapped to a preset range, which can be specifically expressed as follows:

[0208]

[0209] Among them, x represents the original point cloud data, m and n are segment nodes, [-1,1] is the preset range, and the linear mapping functions f1, f2 and f3 satisfy:

[0210] f1(0)=-1;

[0211] f1(m)=f2(m)=0;

[0212] f2(n)=f3(x)=1.

[0213] S08, determining the number of quantization bits, and determining a quantization scale coefficient based on the preset range and the number of quantization bits.

[0214] In a specific implementation, the number of quantization bits may be 8, that is, the target quantization range is [-127, 127].

[0215] In the specific implementation, a symmetric quantizer can be used to quantize the point cloud data. Figure 7 The schematic diagram of the working principle of a symmetric quantizer shown in FIG. 1 can determine the quantization scale coefficient according to the following formula:

[0216]

[0217] Wherein, b is the number of quantization bits, and α is the preset range in step S07. In this example, α=1.

[0218] S09: quantize each segmented data mapped to a preset range to a corresponding quantization range based on the quantization scale coefficient.

[0219] In a specific implementation, the point cloud data can be quantified according to the following formula:

[0220]

[0221] Among them, quantize() represents quantization operation, clip() represents clipping operation, and round() represents rounding operation.

[0222] In some implementations of this specification, data may be quantized outside the target quantization range. Therefore, in order to prevent the above situation from happening, the data may be clipped. The clipping process may be specifically expressed as follows:

[0223]

[0224] S10, performing a translation operation on the integer point cloud data to translate it to a target range.

[0225] Specifically, the integer point cloud data is translated to a target quantization range of [-127, 127].

[0226] S11, quantizing the feature extraction network model after training to obtain a first quantized feature extraction network model.

[0227] Specifically, refer to Figure 8A schematic diagram of a quantization model framework topology structure is shown, in which linear quantization operations and linear inverse quantization operations of a symmetric quantizer can be added to the input and weight ends of the convolutional layer and the fully connected layer of the trained feature extraction network model.

[0228] Among them, the linear quantization operation is shown in formula (3):

[0229] The linear dequantization operation is as follows:

[0230]

[0231] Then the convolution operation in the feature extraction network model can be simplified after quantization and dequantization as follows:

[0232]

[0233] Among them, s W is the quantization ratio coefficient of the weight end, s x is the quantization scale factor at the input, W q is the quantized weight matrix W, x q is the input data after quantization of the input data x, and Represent the weight matrix and input data after linear dequantization respectively.

[0234] S12, determining whether the accuracy of the first quantitative feature extraction network model meets a preset first accuracy requirement. When it is determined that the preset first accuracy requirement is not met, executing step S13; when it is determined that the preset first accuracy requirement is met, executing step S14.

[0235] S13, adjusting the quantization scale coefficient of the quantization after training until the accuracy meets the preset first accuracy requirement.

[0236] In a specific implementation, the data distribution information of each quantization node of the first quantitative feature extraction network model can be obtained. First, the maximum value in the data is selected as the quantization ratio coefficient for quantization after training, so that all the data to be quantized can be mapped to a preset range, and then all the data to be quantized can be quantized to the target quantization range without losing edge data.

[0237] However, taking distance data as input data as an example, according to the laser radar characteristics, the larger the value of the data to be quantized, the sparser the data distribution. Mapping all the data to be quantized to the preset range will affect the quantization accuracy of the data with high data distribution density. Therefore, when the accuracy of the first quantitative feature extraction network model does not meet the preset first accuracy requirement, the value of the quantization scale coefficient of the quantization after training can be adjusted.

[0238] As a specific example, refer to Fig. 9 A data statistical histogram in a linear quantizer is shown, wherein the horizontal axis represents the numerical value of the data in the linear quantizer, and the vertical axis represents the number of times the numerical value appears. Fig. 9 As shown, the numerical range of the data in the quantizer is [0, 12.05]. First, in order not to lose edge data, all the values ​​in [0, 12.05] can be mapped to the preset range, that is, the quantization scale coefficient s of the quantizer is determined to be 12.05. Fig. 9 It can be clearly seen that the data distribution density of the interval near the maximum value 12.05 is obviously much lower than the data division density of other intervals. If the value of this interval is retained, the quantization accuracy of other intervals with larger data division density will be affected, thereby affecting the accuracy of the entire first quantitative feature extraction network model. Therefore, the quantizer scale coefficient, that is, the percentage of quantized data, can be adjusted so that the accuracy of the first quantitative feature extraction network model meets the preset first accuracy requirement. For example, data with a value range of [0, 4.88] accounts for 99.999% of all data. The quantizer scale coefficient s = 4.88 can be selected, and only data with a value range of [0, 4.88] is quantized. Data greater than 4.88 only accounts for 0.001% of all data. This part of the data is uniformly quantized to the maximum value. Therefore, by sacrificing the accuracy of part of the data accounting for 0.001%, the quantization accuracy of most of the data accounting for 99.999% can be improved. According to the accuracy of the first quantitative feature extraction network model, the quantizer proportional coefficient value can be further adjusted according to the percentage of quantized data. For example, only data with a value range of [0,3.64] is quantized, accounting for 99.99% of all data, or data with a value range of [0,1.88] is quantized, accounting for 99.9% of all data. As the maximum value of the quantized data continues to decrease and the total amount of quantized data decreases, the quantization accuracy of most data will become higher and higher, and the accuracy of the first quantitative feature extraction network model will also become higher and higher, until the accuracy of the first quantitative feature extraction network model reaches the preset first accuracy requirement.

[0239] S14, performing quantization training on the first quantitative feature extraction network model to obtain a second quantitative feature extraction network model.

[0240] In a specific implementation, the front-end model of the projection process, i.e., the first quantitative feature extraction network model, is quantitatively trained with the goal of extracting richer point cloud features.

[0241] Since the rounding function is essentially a step function and its reciprocal is an impulse function, it is impossible to perform gradient backpropagation in the quantization training of the first quantization feature extraction network model. For this, a straight-through estimator can be used to replace the reciprocal of the step function in the process of gradient backpropagation so that gradient backpropagation can be performed. The straight-through estimator is used to calculate the derivative process of the symmetric quantizer, as shown in the following formula:

[0242]

[0243] S15, determining whether the accuracy of the second quantitative feature extraction network model meets a preset second accuracy requirement. When it is determined that the preset second accuracy requirement is not met, executing step S16; when it is determined that the preset second accuracy requirement is met, executing step S17.

[0244] S16, adjusting the training strategy of the quantization training until the accuracy meets the preset second accuracy requirement.

[0245] In a specific implementation, the accuracy of the second quantitative feature extraction network model can be adjusted by adjusting the learning rate, loss function or learning strategy of the quantitative training.

[0246] S17, locking the weight of the second quantitative feature extraction network model.

[0247] S18, quantizing the target detection network model after training to obtain a first quantized target detection network model.

[0248] In a specific implementation, the target detection network model may be quantized after training with reference to step S11, which will not be described in detail here.

[0249] S19, determining whether the accuracy of the first quantized target detection network model meets a preset third accuracy requirement. When it is determined that the preset third accuracy requirement is not met, executing step S20; when it is determined that the preset third accuracy requirement is met, executing step S21.

[0250] S20, adjusting the quantization scale coefficient of the trained quantization until the accuracy meets the preset third accuracy requirement.

[0251] In a specific implementation, the quantization scale coefficient of the post-training quantization may be adjusted with reference to step S13, which will not be described in detail here.

[0252] S21, performing quantization training on the first quantized target detection network model to obtain a second quantized target detection network model.

[0253] In a specific implementation, the model behind the projection process, i.e., the first quantitative target detection network model, is quantitatively trained with the training goal of being able to more accurately detect objects in the application scene.

[0254] In a specific implementation, step S14 can be referred to for performing gradient back propagation in the quantization training of the first quantized target detection network model, which will not be described in detail here.

[0255] S22, determine whether the accuracy of the second quantized target detection network model meets the preset fourth accuracy requirement. When it is determined that the preset fourth accuracy requirement is not met, execute step S23; when it is determined that the preset fourth accuracy requirement is met, execute step S24.

[0256] S23, adjusting the training strategy of the quantization training until the accuracy meets the preset fourth accuracy requirement.

[0257] In a specific implementation, the training strategy of the quantization training may be adjusted with reference to step S16, which will not be described in detail here.

[0258] S24, locking the weight of the second quantized object detection network model.

[0259] S25, deploy the quantized point cloud target detection network model on the vehicle-mounted edge computing device.

[0260] In the specific implementation, refer to Fig.10 A schematic diagram comparing the detection accuracy of a point cloud target detection network model under an unquantized floating-point precision and a point cloud target detection network model under a quantized integer precision in an embodiment of this specification is shown in FIG. Fig.10 As shown in the figure, for various common targets, such as trucks, pedestrians, bicycles, and cars, the point cloud target detection network model under quantized integer precision has a smaller accuracy loss than the point cloud target detection network model under floating-point precision without quantization. However, since quantization is used to compress the network scale and reduce the computing resources and operating bandwidth occupied by the network model, it is possible to achieve fast network computing on the on-board edge computing device while ensuring the accuracy of the point cloud target detection task.

[0261] Reference Fig.11a The detection results of a point cloud target detection network model based on an existing quantization scheme are shown in Figure 2. Fig.11b The schematic diagram of the detection results of the point cloud target detection network model under the quantized integer precision in one embodiment of this specification is shown, wherein the existing quantization scheme adopts the traditional image-based neural network method, does not perform segmented quantization on the input data, and only uses the post-training quantization method for the target detection network model. Fig.11a As shown, the rectangular box represents the identified target. The existing scheme identifies 3 targets, such as Fig.11bAs shown, when the input point cloud data is the same, the point cloud target detection network model described in the embodiment of this specification recognizes 6 targets. Obviously, the point cloud target detection network model described in the embodiment of this specification can recognize more targets from the same point cloud data and achieve higher network accuracy.

[0262] For ease of implementation, the present specification also provides a point cloud target detection network model quantization system, referring to Fig.12 The schematic diagram of the structure of a point cloud target detection network model quantization system is shown in FIG. The point cloud target detection network model quantization system T includes:

[0263] A model acquisition unit T1, adapted to acquire a trained point cloud target detection network model;

[0264] A point cloud quantization unit T2, adapted to quantize the point cloud data at the input end of the trained point cloud object detection network model;

[0265] The model quantization unit T3 is suitable for quantizing the trained point cloud target detection network model.

[0266] By adopting the above-mentioned quantization system T, the trained point cloud target detection network model is acquired through the model acquisition unit T1, and then the point cloud data at the input end of the trained point cloud target detection network model is quantized through the point cloud quantization unit T2, and the trained point cloud target detection network model is quantized through the model quantization unit T3. On the one hand, the quantization of the trained point cloud target detection network model can effectively compress the trained point cloud target detection network model. On the other hand, considering the difference between point cloud data and traditional image data, quantizing the input point cloud data can further speed up the network operation speed and reduce the computing power requirements. Therefore, the quantization system T can ensure the accuracy of the point cloud target detection network model while reducing the computing resources and operating bandwidth occupied by the network model, so that it can realize real-time and fast network calculation on the edge computing device.

[0267] In a specific implementation, the above-mentioned point cloud target detection network model quantization system can be implemented using the point cloud target detection network model quantization method described in any of the aforementioned embodiments. The specific implementation method can be referred to the aforementioned embodiments and will not be described in detail here.

[0268] An embodiment of the present specification also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is run, the steps of the point cloud target detection network model quantization method described in any of the aforementioned embodiments are executed. The specific steps can be found in the aforementioned embodiments and will not be repeated here.

[0269] In a specific implementation, the computer-readable storage medium may be any appropriate readable storage medium such as an optical disk, a mechanical hard disk, a solid-state hard disk, etc.

[0270] It should be noted that the terms "first", "second", "third", "fourth", etc. in the embodiments of this specification are only used to distinguish the descriptions of different precisions and are not used to impose any limitations on their specific values.

[0271] Although the embodiments of this specification are disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, so the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A point cloud target detection network model quantization method, characterized in that: include: Obtain the trained point cloud target detection network model; quantizing the point cloud data at the input end of the trained point cloud object detection network model; Quantizing the trained point cloud object detection network model.

2. The method according to claim 1, characterized in that: The step of obtaining a trained point cloud target detection network model comprises: Determining the network structure of the point cloud object detection network model; Training the point cloud target detection network model under the precision of floating point cloud data; Obtain the trained point cloud object detection network model.

3. The method according to claim 1, characterized in that The point cloud data at the input end of the trained point cloud target detection network model includes floating point cloud data and / or integer point cloud data.

4. The method according to claim 3, characterized in that The quantizing the point cloud data at the input end of the trained point cloud target detection network model includes: The floating-point point cloud data is segmented and quantized to a target quantization range.

5. The method according to claim 4, characterized in that The step of quantizing the floating point cloud data into segments to a target quantization range includes: Segmenting the floating point cloud data according to the characteristics of the floating point cloud data; Based on the characteristics of each segmented data, each segmented data is mapped to a preset range, and each segmented data mapped to the preset range is quantized to a corresponding quantization range.

6. The method according to claim 5, characterized in that The characteristics of the floating point cloud data include any one of the following: Data distribution density; Data confidence; Data accuracy; Data region of interest.

7. The method according to claim 6, characterized in that The step of mapping each segmented data to a preset range based on the characteristics of each segmented data, and quantizing each segmented data mapped to the preset range to a corresponding quantization range includes: Based on the characteristics of each segmented data, a linear mapping function of each segmented data is determined respectively; Based on the linear mapping function and each segmented data, each segmented data is mapped to a preset range; determining the number of quantization bits; Determine a quantization scale coefficient based on the preset range and the number of quantization bits; Based on the quantization scale coefficient, each segmented data mapped to the preset range is quantized to a corresponding quantization range.

8. The method according to claim 7, characterized in that The step of mapping each segmented data to a preset range based on the linear mapping function and each segmented data comprises: Mapping part of the segmented data in each segmented data to the maximum value of a preset range; and / or Part of the segmented data in each segmented data is mapped to the minimum value of the preset range.

9. The method according to claim 3, characterized in that: The quantizing the point cloud data at the input end of the trained point cloud target detection network model includes: A translation operation is performed on the integer point cloud data to translate it to a target range.

10. The method according to claim 1, characterized in that The trained point cloud target detection network model includes: a feature extraction network model, suitable for extracting features from point cloud data; and a target detection network model coupled to the feature extraction network model, suitable for performing target detection based on the extracted feature data; The quantizing the trained point cloud target detection network model includes at least one of the following: quantizing the feature extraction network model; Quantize the object detection network model.

11. The method according to claim 10, characterized in that The quantizing the trained point cloud target detection network model comprises: Quantize the quantization nodes in the trained point cloud object detection network model, wherein the quantization nodes include convolutional layers and fully connected layers.

12. The method according to claim 11, characterized in that The quantizing of the quantized nodes in the trained point cloud target detection network model includes: Quantize the input data and weight data of each quantization node.

13. The method according to claim 10, characterized in that The quantization of the feature extraction network model includes: Quantifying the feature extraction network model after training to obtain a first quantized feature extraction network model; Determine whether the accuracy of the first quantitative feature extraction network model meets the preset first accuracy requirement. When it is determined that the preset first accuracy requirement is not met, adjust the quantization scale coefficient of the quantization after training until the accuracy meets the preset first accuracy requirement.

14. The method according to claim 13, characterized in that After determining that the accuracy of the first quantitative feature extraction network model meets a preset first accuracy requirement, the method further includes: Performing quantization training on the first quantitative feature extraction network model to obtain a second quantitative feature extraction network model; Determine whether the accuracy of the second quantitative feature extraction network model meets a preset second accuracy requirement. When it is determined that the preset second accuracy requirement is not met, adjust the training strategy of the quantization training until the accuracy meets the preset second accuracy requirement. When it is determined that the preset second accuracy requirement is met, lock the weight of the second quantitative feature extraction network model.

15. The method according to claim 10, characterized in that The quantizing the target detection network model includes: Quantizing the target detection network model after training to obtain a first quantized target detection network model; Determine whether the accuracy of the first quantized target detection network model meets a preset third accuracy requirement. When it is determined that the preset third accuracy requirement is not met, adjust the quantization scale coefficient of the post-training quantization until the accuracy meets the preset third accuracy requirement.

16. The method according to claim 15, characterized in that After determining that the accuracy of the first quantized object detection network model meets a preset third accuracy requirement, the method further includes: Performing quantization training on the first quantized object detection network model to obtain a second quantized object detection network model; Determine whether the accuracy of the second quantized target detection network model meets a preset fourth accuracy requirement; when it is determined that the preset fourth accuracy requirement is not met, adjust the training strategy of the quantized training until the accuracy meets the preset fourth accuracy requirement; when it is determined that the preset fourth accuracy requirement is met, lock the weight of the second quantized target detection network model.

17. The method according to claim 14 or 16, characterized in that The training strategy for adjusting the quantization training includes: Adjust the hyperparameters for quantized training.

18. The method according to claim 17, characterized in that The step of adjusting the hyper parameters of the quantization training includes: Adjust at least one of the learning rate, loss function, and learning strategy of quantization training.

19. A point cloud target detection network model quantization system, characterized in that: include: A model acquisition unit, adapted to acquire a trained point cloud target detection network model; A point cloud quantization unit, adapted to quantize the point cloud data at the input end of the trained point cloud object detection network model; The model quantization unit is adapted to quantize the trained point cloud target detection network model.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is running, the steps of the point cloud target detection network model quantization method as described in any one of claims 1 to 18 are executed.