Three-dimensional point cloud data processing method and device based on multi-scale feature fusion

Abstract

The invention discloses a three-dimensional point cloud data processing method and device based on multi-scale feature fusion, and the method comprises the following steps: obtaining original point cloud data collected by a laser radar, and carrying out the preprocessing of the original point cloud data; and based on the preprocessed three-dimensional point cloud data, dividing a plurality of voxel grids with preset sizes in a three-dimensional space. And traversing all voxel grids, and dividing a low-density voxel grid and a high-density grid. And merging the low-density voxel grids to adjacent high-density grids by adopting a preset merging algorithm, and marking the low-density voxel grids as effective voxel grids. And inputting the effective voxel grids into a preset recognition model, and outputting adaptive anchor frames of the plurality of target objects. According to the invention, the voxel division and combination strategy is optimized, so that the defects of an existing three-dimensional target detection method in processing complex scenes and target scale differences are effectively overcome, the detection precision of small targets and large targets is improved, and the adaptability and robustness of a target detection model are enhanced.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method and apparatus for processing three-dimensional point cloud data based on multi-scale feature fusion. Background Technology

[0002] With the rapid development of autonomous driving and robotics technologies, 3D point cloud data, as an important form of environmental information representation acquired by sensors such as LiDAR, has become a key data source for environmental perception and target detection. 3D point clouds provide accurate spatial location and geometric information and are widely used in autonomous driving, robot localization and navigation, and object recognition. However, existing 3D target detection methods still have significant shortcomings in practical applications.

[0003] Current 3D object detection methods typically rely on static receptive fields and fixed voxel partitioning strategies. This approach has significant limitations when handling scenes with large differences in object scale. For example, for targets such as pedestrians, cyclists, and vehicles, whose scales vary considerably, fixed voxel partitioning cannot capture the feature representation of all targets. Furthermore, small targets have low resolution after voxelization, resulting in sparse point clouds and a tendency to lose geometric features; furthermore, using excessively small voxel partitioning for large targets may lead to the loss of local details.

[0004] In summary, existing 3D target detection methods suffer from insufficient adaptability and low detection accuracy. Summary of the Invention

[0005] To address the technical problems of insufficient adaptability and low accuracy in existing 3D target detection methods, the present invention aims to provide a 3D point cloud data processing method and apparatus based on multi-scale feature fusion. This method effectively solves the shortcomings of existing 3D target detection methods in handling complex scenes and target scale differences, improves the detection accuracy of both small and large targets, enhances the adaptability and robustness of the target detection model, and has significant practical application value.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] In a first aspect, the present invention provides a method for processing three-dimensional point cloud data based on multi-scale feature fusion, the method comprising the following steps:

[0008] Acquire raw point cloud data collected by lidar and preprocess the raw point cloud data;

[0009] Based on the preprocessed 3D point cloud data, multiple voxel meshes of preset sizes are divided in 3D space;

[0010] Traverse all voxel meshes and divide them into low-density voxel meshes and high-density meshes;

[0011] A preset merging algorithm is used to merge low-density voxel meshes into adjacent high-density meshes and mark them as effective voxel meshes;

[0012] Input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

[0013] In one possible implementation, all voxel meshes are traversed, and low-density voxel meshes and high-density meshes are divided, specifically including the following steps:

[0014] Traverse all voxel grids and count the number of points in each voxel grid;

[0015] If the number of point clouds in a voxel mesh is lower than a preset threshold, it is marked as a low-density mesh.

[0016] If the number of point clouds in a voxel mesh exceeds a preset threshold, it is marked as a high-density mesh.

[0017] In one possible implementation, the preset merging algorithm is an eight-neighbor search strategy; wherein, if all neighbors of the current low-density grid are low-density, the low-density grid is skipped directly and marked as an invalid voxel grid.

[0018] In one possible implementation, the preset recognition model includes:

[0019] The feature encoding module is used to extract and encode features from the effective voxel grid to generate an initial voxel feature map;

[0020] A progressive downsampling module has multiple cascaded 3D sparse convolutional layers. The progressive downsampling module is used to extract features from the initial voxel feature map and generate an output feature map.

[0021] The multi-scale feature fusion layer is used to fuse the output feature maps of different 3D sparse convolutional layers to output a multi-scale feature map.

[0022] An attention mechanism layer with a Transformer decoder is used to compute attention weights for multi-scale feature maps.

[0023] The anchor box generation module generates adaptive anchor boxes for several target objects based on multi-scale feature maps.

[0024] In one possible implementation, the preset recognition model performs the following steps:

[0025] For the effective voxel grid, local features of each point within the grid are extracted, including center coordinate offset, mean and standard deviation of reflection intensity;

[0026] The local features are mapped to a 64-dimensional feature space using a multilayer perceptron (MLP) to generate an initial voxel feature map.

[0027] The initial voxel feature map is input into multiple cascaded 3D sparse convolutional layers for feature extraction, and each 3D sparse convolutional layer outputs a corresponding output feature map; wherein, the stride of the multiple cascaded 3D sparse convolutional layers increases progressively, and the stride of the first 3D sparse convolutional layer is 1.

[0028] The output feature maps of at least two different levels of 3D sparse convolutional layers are fused through a feature pyramid network to output a fused multi-scale feature map.

[0029] The multi-scale feature maps are input into the Transformer decoder, and the attention weights of the multi-scale feature maps are calculated using the KNN algorithm.

[0030] The multi-scale feature map is identified, and adaptive anchor boxes for several target objects are generated. An anchor box offset regression algorithm is used to make the adaptive anchor boxes close to the center of the target objects.

[0031] In one possible implementation, the preset recognition model includes:

[0032] The target scale-aware decision module is used to statistically analyze the feature energy of the output feature map of the second-level 3D sparse convolutional layer in order to determine the downsampling strategy of the progressive downsampling module.

[0033] In one possible implementation, the 3D point cloud data processing method further includes the following optimization step for a preset recognition model:

[0034] Load the pre-trained preset recognition model and use it as the teacher network of the teacher-student model;

[0035] The original point cloud data is input into the teacher network, and the output feature map of the third-level 3D sparse convolutional layer of the teacher network is used as a supervision signal.

[0036] The output feature map of the third-level 3D sparse convolutional layer of the student network is minimized with the corresponding output feature map of the teacher network using L2 distance, and the feature distillation loss is backpropagated to the student network through a gradient inversion layer (GRL); wherein, the student network uses the INT8 quantized version.

[0037] The weights of the convolutional layers of the student network are symmetrically quantized, with the quantization range set to [-α, α], where α is the maximum value of the absolute value of the weights; and the gradient of the quantization function is approximated during backpropagation using a pass-through estimator (STE).

[0038] During the inference process of the student network, the dynamic range of the input activation values ​​of each 3D sparse convolutional layer is statistically analyzed, and the quantization parameters are dynamically adjusted based on the statistical results.

[0039] The 3D sparse convolutional layer, batch normalization, and ReLU activation operation of the student model are fused into a single CBR operator.

[0040] Secondly, the present invention provides a three-dimensional point cloud data processing device, the three-dimensional point cloud data processing device comprising:

[0041] The acquisition module is used to acquire the raw point cloud data collected by the lidar and to preprocess the raw point cloud data.

[0042] The voxelization module is used to divide multiple voxel meshes of preset sizes in three-dimensional space based on preprocessed three-dimensional point cloud data.

[0043] The partitioning module is used to traverse all voxel meshes and partition them into low-density voxel meshes and high-density meshes.

[0044] The merging module is used to merge low-density voxel meshes into adjacent high-density meshes using a preset merging algorithm and mark them as valid voxel meshes.

[0045] The recognition module is used to input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

[0046] Thirdly, an electronic device, the electronic device comprising:

[0047] Memory, used to store computer programs;

[0048] A processor for executing the computer program to implement the three-dimensional point cloud data processing method as in any possible implementation of the first aspect.

[0049] Fourthly, the present invention also provides a computer-readable storage medium storing computer program code, which, when executed by a processing circuit, implements the three-dimensional point cloud data processing method as described in any possible implementation of the first aspect.

[0050] Compared with the prior art, the present invention has at least the following beneficial effects:

[0051] This invention provides a method for processing 3D point cloud data based on multi-scale feature fusion. The method includes the following steps: acquiring raw point cloud data collected by a lidar radar; preprocessing the raw point cloud data; dividing the 3D point cloud data into multiple voxel grids of preset sizes in 3D space based on the preprocessed data; traversing all voxel grids to divide them into low-density and high-density grids; merging low-density voxel grids into adjacent high-density grids using a preset merging algorithm and marking them as valid voxel grids; inputting the valid voxel grids into a preset recognition model and outputting adaptive anchor boxes for several target objects.

[0052] This invention provides a 3D point cloud data processing method based on multi-scale feature fusion. By optimizing the voxel mesh partitioning and merging strategy, it addresses the shortcomings of existing 3D target detection methods in handling target scale differences and point cloud sparsity. This method can effectively improve the detection accuracy of small targets, preserve the local details of large targets, and enhance the adaptability and robustness of the detection model, thereby improving the accuracy and real-time performance of target detection. It has broad application prospects, especially in the fields of autonomous driving and robotics. Attached Figure Description

[0053] Figure 1 A flowchart illustrating a three-dimensional point cloud data processing method provided by the present invention;

[0054] Figure 2 A model structure diagram of a preset recognition model provided by the present invention;

[0055] Figure 3 This is a schematic diagram of the processing flow of a preset recognition model provided by the present invention;

[0056] Figure 4 A schematic diagram of the optimization steps of a preset recognition model provided by the present invention;

[0057] Figure 5 This is a schematic diagram of the structure of a three-dimensional point cloud data processing device provided by the present invention.

[0058] Figure 6 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0059] To facilitate understanding of the present invention, the technical solutions and advantages of the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Any mechanisms or methods not elaborated in this invention can be referred to in the prior art. The specific structures and features of the present invention are illustrated below by way of example and should not be construed as limiting the present invention in any way. Furthermore, any of the technical features mentioned below (including implicit or disclosed features), as well as any technical features directly shown or implied in the figures, can be arbitrarily combined or deleted among these technical features to form more other embodiments that may not be directly or indirectly mentioned in this invention. The accompanying drawings show preferred embodiments of the present invention. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention.

[0060] This application provides a method and apparatus for processing 3D point cloud data based on multi-scale feature fusion, which can be applied to 3D environment perception scenarios. For example, the method can be applied to automobiles, autonomous vehicles, and autonomous cleaning vehicles. The 3D point cloud data processing method based on multi-scale feature fusion provided in this application enhances the adaptability and robustness of the detection model, improves the accuracy and real-time performance of target detection, and has broad application prospects, especially in the fields of autonomous driving and robotics.

[0061] To better understand the embodiments of this application, the system architecture to which the embodiments of this application can be applied is described below.

[0062] The system architecture applicable to the embodiments of this application may include a 3D point cloud data processing device and a sensing module. The sensing module may include a 3D point sensing module. For example, the 3D point sensing module may include, but is not limited to, laser sensors (such as LiDAR or LiDAR scanners), millimeter-wave radar, etc. For specific descriptions of laser sensors, millimeter-wave radar, etc., please refer to the explanations and descriptions in conventional technology; this application will not elaborate further.

[0063] In this application, the 3D point cloud data processing device is used to measure the surrounding environment through a perception module to obtain measurement data of the surrounding environment. For example, the measurement data may be a 3D point cloud of the surrounding environment (e.g., a 3D point cloud of a target scene). Furthermore, the 3D point cloud data processing device is also used to identify targets in the surrounding environment based on the aforementioned measurement data. For example, the targets may be vehicles, pedestrians, guardrails, bollards, bushes, lampposts, vehicle access barriers / stop barriers, overpass culverts, or buildings, etc.

[0064] For example, when applied to automobiles, autonomous vehicles, or autonomous cleaning vehicles, 3D point cloud data processing devices can sense the surrounding environment during the movement of these vehicles via 3D point perception modules. Then, by perceiving the surrounding environment, target recognition is achieved, thereby effectively avoiding obstacles and increasing the comfort and safety of autonomous driving.

[0065] In some implementations, the sensing module can communicate with the 3D point cloud data processing device via wireless or wired networks.

[0066] The three-dimensional point cloud data processing method provided in this application can be applied to the aforementioned three-dimensional point cloud data processing device. The following provides a detailed description of the three-dimensional point cloud data processing method provided in the embodiments of this application.

[0067] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a three-dimensional point cloud data processing method provided in an embodiment of this application. The execution entity of this method can be a perception system, a fused perception system, or a planning / control system integrating the above systems, such as an assisted driving or autonomous driving system. Alternatively, the execution entity can also be software or hardware (such as a three-dimensional point cloud data processing system connected or integrated with a corresponding perception module via wireless or wired connection). The following different execution steps can be implemented centrally or in a distributed manner.

[0068] like Figure 1 As shown in the embodiments of this application, a three-dimensional point cloud data processing method includes the following steps 101-105:

[0069] 101. Acquire the raw point cloud data collected by the lidar and preprocess the raw point cloud data.

[0070] In this embodiment, point cloud data in three-dimensional space is acquired through a sensing module (e.g., LiDAR). The point cloud data generally consists of the spatial coordinates (X, Y, Z) of each point and additional information (such as reflection intensity, color, etc.). LiDAR scanning can acquire high-precision three-dimensional data of the surrounding environment.

[0071] In one implementation, the acquired raw point cloud data undergoes denoising processing to filter out outliers or noisy data, ensuring more accurate subsequent processing. For example, a neighborhood density-based method can be used to remove individual outliers, or the data can be resampled using a voxel grid method to remove overly sparse or irrelevant data. This step effectively improves the quality of the point cloud data, providing accurate data input for subsequent analysis.

[0072] In another implementation, preprocessing includes denoising by filtering out outliers using statistical neighborhood point density; for example, a single point is isolated within a 1-meter radius. The point cloud coordinates are then normalized to the [0,1] interval to eliminate the influence of differences in installation locations between different lidars. This process provides a uniform scale for subsequent spatial division.

[0073] 102. Based on the preprocessed 3D point cloud data, divide the data into multiple voxel meshes of preset sizes in 3D space.

[0074] In this application, this step involves dividing the preprocessed point cloud data into multiple uniformly sized voxel grids in three-dimensional space. A voxel grid divides three-dimensional space into uniform grid units, each unit called a voxel, similar to a "pixel" in three-dimensional space. For example, different sizes of voxel grids are preset according to task requirements. The grid size is adjusted based on factors such as data density, resolution requirements, and the size of the target object.

[0075] Understandably, larger voxel sizes facilitate the processing of large-scale data, while smaller voxel sizes provide higher resolution, suitable for detecting small targets. Voxelization not only converts continuous 3D point cloud data into discrete mesh data but also reduces computational complexity, facilitating subsequent processing and analysis. In this way, the processing efficiency and accuracy of subsequent steps can be effectively improved.

[0076] In one implementation, based on the preprocessed point cloud distribution, a 1m×1m×1m voxel grid is divided in the XYZ three-dimensional space. Each grid is used as an independent computing unit. The mapping relationship between the grid index and the internal point index is stored through a hash table to achieve fast lookup with O(1) time complexity. At the same time, the number of points in each grid is recorded for subsequent density evaluation.

[0077] For example, to achieve fast lookup of point cloud information within each voxel grid, this implementation uses a hash table to store the mapping relationship between grid indices and internal point indices. A hash table is a data structure that uses a hash function for fast data storage and retrieval, providing fast lookup in constant time (O(1)). For each voxel grid, the hash table records the unique index of the grid (such as the hash value generated by the coordinate position of the voxel) and the index list of point cloud data within that grid. Through this mapping method, all point data within any voxel grid can be quickly obtained, avoiding the inefficient operation of traversing all point clouds. The hash table has a time complexity of O(1), thus enabling voxel grid lookup in constant time, greatly improving query efficiency and overall data processing speed.

[0078] 103. Traverse all voxel meshes and divide them into low-density voxel meshes and high-density meshes.

[0079] In this embodiment, for each voxel grid, the density of the voxel grid is determined by counting the number of point clouds contained in each grid. Generally, areas with higher point cloud density usually correspond to denser object areas, while low-density areas may be empty or objectless areas.

[0080] For example, based on a set threshold, the voxel mesh is divided into low-density and high-density meshes. Low-density meshes typically refer to sparser areas of the point cloud data, while high-density meshes represent denser areas, usually the regions of target objects. The purpose of this step is to effectively partition the point cloud data, reduce wasted computational resources, and prepare for subsequent voxel mesh merging and recognition model input. This step helps identify target objects in complex scenes while eliminating unimportant or redundant data.

[0081] In practice, all voxel meshes are traversed, and low-density voxel meshes and high-density meshes are divided. The specific steps include the following:

[0082] Traverse all voxel grids and count the number of points in each voxel grid;

[0083] If the number of point clouds in a voxel mesh is lower than a preset threshold, it is marked as a low-density mesh.

[0084] If the number of point clouds in a voxel mesh exceeds a preset threshold, it is marked as a high-density mesh.

[0085] 104. Use a preset merging algorithm to merge low-density voxel grids into adjacent high-density grids and mark them as valid voxel grids.

[0086] In this application, the goal of this step is to eliminate holes and redundant regions in the point cloud data, retaining only useful high-density regions. This merging process effectively avoids data sparsity issues in subsequent processing, improving the accuracy of target recognition.

[0087] This process employs a neighborhood search strategy: for each low-density grid, it determines whether its neighboring high-density grids can contain the low-density point cloud data; if so, they are merged. The merged high-density grids are marked as effective voxel grids and used as input data for the subsequent target recognition model. These effective grids contain useful 3D information, which helps improve the performance of the target recognition model.

[0088] For example, the preset merging algorithm is an eight-neighbor search strategy; wherein, if all neighbors of the current low-density grid are also low-density, the low-density grid is skipped and marked as an invalid voxel grid. By using the eight-neighbor search strategy to merge low-density grids into adjacent high-density grids (prioritizing merging into the neighbor with the most points), this process can reduce invalid computation by 30%-50%.

[0089] 105. Input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

[0090] In some implementations, data labeled as effective voxel grids is input into a pre-defined target recognition model. The recognition model may employ a deep neural network (such as a 3D convolutional neural network) to further process the effective voxel grids. Features are extracted through convolution operations, and targets are classified and located. In the target detection task, anchor boxes are used to initially determine the location, scale, and other information of potential targets. By fusing and processing multi-scale features, several adaptive anchor boxes are generated to represent possible target locations. The size and shape of these anchor boxes are adaptively adjusted according to different target types and sizes.

[0091] It is understood that this invention preprocesses, voxelizes, and rationally divides and merges low-density and high-density meshes in a step-by-step manner for point cloud data acquired by LiDAR, and finally generates adaptive anchor boxes for target objects through a recognition model. This method can effectively improve the detection accuracy of small targets, preserve the local details of large targets, and enhance the adaptability and robustness of the detection model, thereby improving the accuracy and real-time performance of target detection. It has broad application prospects, especially in the fields of autonomous driving and robotics.

[0092] In summary, the core innovation of this invention lies in proposing an efficient and accurate 3D point cloud data processing method through precise point cloud data preprocessing, voxel mesh generation and optimization, density region division and merging, and adaptive anchor box generation. The innovation lies in its efficient query optimization method, targeted density distribution processing strategy, and adaptive adjustment of the target recognition model, which significantly improves the accuracy and efficiency of point cloud data processing.

[0093] Please refer to Figure 2 , Figure 2 This is a model structure diagram of a preset recognition model provided in an embodiment of this application. Specifically, the preset recognition model includes a feature encoding module, a progressive downsampling module, a multi-scale feature fusion layer, an attention mechanism layer, and an anchor box generation module.

[0094] In specific implementation, the feature encoding module is used to extract and encode features from the effective voxel grid to generate an initial voxel feature map.

[0095] In practice, the progressive downsampling module has multiple cascaded 3D sparse convolutional layers. The progressive downsampling module is used to extract features from the initial voxel feature map and generate an output feature map.

[0096] In practice, the multi-scale feature fusion layer is used to fuse the output feature maps of different 3D sparse convolutional layers to output multi-scale feature maps.

[0097] In a specific implementation, the attention mechanism layer has a Transformer decoder, which is used to calculate the attention weights of the multi-scale feature map.

[0098] In practice, the anchor box generation module generates adaptive anchor boxes for several target objects based on multi-scale feature maps.

[0099] The innovation of this pre-defined recognition model lies in its ingenious combination of multiple techniques, including feature encoding, progressive downsampling, multi-scale feature fusion, attention mechanisms, and adaptive anchor box generation. These techniques not only improve the model's accuracy and efficiency but also employ targeted strategies to extract and optimize features for the complexity and high dimensionality of 3D point cloud data. This results in greater robustness and flexibility for handling object detection tasks in highly complex environments.

[0100] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the processing flow of a preset recognition model provided in an embodiment of this application. Specifically, the preset recognition model under the above model structure performs the following steps:

[0101] 301. For the effective voxel grid, extract the local features of each point in the grid, the local features including the center coordinate offset, the mean and standard deviation of the reflection intensity.

[0102] In this step, local features provide rich semantic information for subsequent feature mapping and model learning. By extracting local features from point cloud data, the model can better understand the structure of each voxel mesh and the surface characteristics of the object, laying the foundation for target recognition.

[0103] Center coordinate offset: This value represents the coordinate offset of a point relative to the center of the grid, and is used to capture the relative position of the point within the voxel grid.

[0104] Mean and standard deviation of reflection intensity: Reflection intensity represents the laser reflection intensity of each point in the point cloud data. The intensity distribution characteristics of the point cloud data are measured by the mean and standard deviation, which is very important for distinguishing the surface features of different objects.

[0105] 302. The local features are mapped to a 64-dimensional feature space using a multilayer perceptron (MLP) to generate an initial voxel feature map.

[0106] In this embodiment, after mapping local features to a 64-dimensional feature space, an initial voxel feature map (M×64, where M is the number of effective grid cells) can be generated. This feature map can provide richer feature representations for subsequent deep feature extraction and object detection.

[0107] Understandably, in steps 301 and 302, local features of each point in the retained effective voxel grid are extracted, including the center coordinate offset (relative to the grid center), the mean and standard deviation of the reflection intensity. These handcrafted features are then mapped to a 64-dimensional feature space using a multilayer perceptron (MLP) to generate an initial voxel feature map (M×64, where M is the number of effective grids).

[0108] 303. Input the initial voxel feature map into multiple cascaded 3D sparse convolutional layers for feature extraction, and each 3D sparse convolutional layer outputs a corresponding output feature map; wherein, the stride of the multiple cascaded 3D sparse convolutional layers increases progressively, and the stride of the first 3D sparse convolutional layer is 1.

[0109] In some implementations, the first-level 3D sparse convolutional layer and the second-level 3D sparse convolutional layer are used as examples for illustration:

[0110] In step 303, the progressive downsampling module inputs the initial voxel feature map into the first-level 3D sparse convolutional layer. The first-level 3D sparse convolutional layer uses a 3×3×3 convolutional kernel with a stride=1 for feature extraction. This first-level 3D sparse convolutional layer retains the feature calculations of all voxels, ensuring that detailed information of small targets (such as pedestrians) is not lost. In some preferred embodiments, the first-level 3D sparse convolutional layer fuses input and output features through residual connections to alleviate the gradient vanishing problem.

[0111] Subsequently, the output feature map of the first-stage 3D sparse convolutional layer is fed into the second-stage 3D sparse convolutional layer. The second-stage 3D sparse convolutional layer uses a convolutional kernel with a stride of 2 for downsampling, which expands the receptive field to a range of 2 meters. In some preferred implementations, the second-stage 3D sparse convolutional layer dynamically adjusts the weights of each feature channel through a channel attention mechanism (e.g., assigning higher weights to height features to distinguish vehicles from the ground), halving the size of the output feature map (M / 2×128).

[0112] For the third, fourth, ... Nth level 3D sparse convolutional layers, the same principle applies, with the step size increasing progressively.

[0113] 304. The output feature maps of at least two different levels of 3D sparse convolutional layers are fused through a feature pyramid network to output a fused multi-scale feature map.

[0114] In some specific implementations, the output feature maps of the second-level and fourth-level 3D sparse convolutional layers can be fused using a Feature Pyramid Network (FPN). For example, the output feature map of the fourth-level 3D sparse convolutional layer is upsampled by a factor of 2 and then element-wise added to the output feature map of the second-level 3D sparse convolutional layer. The resulting multi-scale feature map (M / 2×256) simultaneously contains detailed information about small targets and semantic context for large targets, improving the detector's robustness to targets with varying scales.

[0115] In this field, feature pyramid networks fuse low-level and high-level features through top-down upsampling and element-wise addition. This fusion method effectively improves the model's adaptability to changes in target scale and enhances the robustness of target detection. The fused feature map contains details of small targets as well as the semantic context of large targets.

[0116] 305. Input the multi-scale feature map into the Transformer decoder and calculate the attention weights of the multi-scale feature map using the KNN algorithm.

[0117] In some specific implementations, the fused multi-scale feature map is input into the Transformer decoder. First, a query vector is generated for each spatial location. Then, the KNN algorithm (K=16) is used to search for the key vector and value vector in the neighborhood of each query point. Only the attention weight between the query point and its neighboring points is calculated to avoid the O(N²) computational complexity caused by global attention.

[0118] Understandably, the fused multi-scale feature map is input into the Transformer decoder. This decoder is used to calculate the attention weights at each spatial location, enhancing the model's focus on key regions. To avoid the high computational complexity of global attention calculation, this implementation uses the KNN algorithm (K=16) to calculate the attention weights only for each query point and its neighborhood. In this way, the O(N²) computational complexity of traditional global attention is avoided, significantly improving computational efficiency.

[0119] 306. Identify the multi-scale feature map, generate adaptive anchor boxes for several target objects, and use an anchor box offset regression algorithm to make the adaptive anchor boxes close to the center of the target objects.

[0120] In some specific implementations, adaptive anchor boxes are generated based on the statistical distribution of target center points using feature maps (e.g., fitting the spatial distribution of pedestrian heads and vehicle bottoms using a Gaussian mixture model). The aspect ratio of these anchor boxes is strongly correlated with the target category (e.g., a 2:1 aspect ratio for pedestrian anchor boxes and a 1:2 aspect ratio for vehicle anchor boxes). Simultaneously, anchor box offset regression (Δx, Δy) is used to make the anchor boxes closer to the true target center, reducing the imbalance between positive and negative samples. The generation of adaptive anchor boxes and offset regression enable the model to locate targets more accurately, especially when the target shape varies significantly, thus improving the accuracy of target detection.

[0121] This invention employs an original pre-defined recognition model, which significantly improves the accuracy and efficiency of target detection in complex environments using 3D point cloud data through a series of steps, such as local feature extraction and mapping, progressive downsampling and sparse convolution, multi-scale feature fusion, an attention mechanism optimized by the Transformer decoder, and adaptive anchor box generation. These innovations not only enhance the model's adaptability, especially in situations with large target scale variations and complex environments, but also effectively solve the problems of computational complexity and imprecise feature extraction in traditional methods. Through these unique technical designs, this solution exhibits significant advantages in target detection accuracy, real-time performance, and robustness, making it suitable for various 3D point cloud data processing scenarios.

[0122] This implementation provides an efficient method for processing 3D point cloud data. By progressively extracting local features, utilizing multiple sparse convolutional layers for feature extraction and downsampling, multi-scale feature fusion, attention computation in the Transformer decoder, and adaptive anchor box generation and offset regression, it effectively improves the accuracy and robustness of target detection. This method is particularly suitable for fields such as autonomous driving and environmental monitoring, capable of processing complex 3D point cloud data and accurately identifying target objects.

[0123] Furthermore, this invention achieves breakthroughs in detection, accuracy, and model size through three core technologies: dynamic sparse computation, progressive feature extraction, and hardware-friendly design. Compared to existing technologies, this solution achieves an optimal balance in speed, accuracy, and cost, demonstrating significant social value (improving autonomous driving safety), economic value (reducing hardware and energy costs), and technological value (promoting the lightweight development of 3D detection algorithms). Experimental data and real-world scenario verification fully demonstrate its technological maturity and commercial potential.

[0124] In a preferred embodiment, the preset recognition model includes a target scale perception decision module, which is used to statistically analyze the feature energy of the output feature map of the second-level 3D sparse convolutional layer to determine the downsampling strategy of the progressive downsampling module.

[0125] For example, based on the output feature map of the second-level 3D sparse convolutional layer, the target scale distribution is predicted by statistically analyzing the feature energy (L2 norm) of each spatial location. If the feature energy of a certain region is concentrated in the lower layer (such as the output of stage 2), it is determined to be a small target region (such as a pedestrian), and a lighter downsampling strategy (Stage4 Stride=4) is subsequently adopted; if the energy is concentrated in the higher layer, it is determined to be a large target region (such as a truck), and a deeper downsampling strategy (Stage4 Stride=8) is adopted.

[0126] In this embodiment, the main task of the target scale perception decision module is to statistically determine the target scale information based on the input feature map and dynamically adjust the downsampling strategy of the subsequent progressive downsampling module based on this information. By analyzing feature energy, this module can determine the subsequent downsampling step size according to the scale characteristics of different targets, thereby optimizing the accuracy and efficiency of target detection. Based on the judgment result, the downsampling step size is dynamically selected: for small target regions, a lightweight downsampling strategy is used to preserve more details; for large target regions, a larger step size is used for downsampling to reduce computational complexity.

[0127] Two example target region identification and downsampling strategies:

[0128] I. Small target area (pedestrians):

[0129] Suppose that in a portion of the road surface, the feature energy exhibits a relatively low L2 norm (e.g., the square of the sum of all 64-dimensional features is small). This implies that the area is likely occupied by a pedestrian. The processing steps of the target scale-aware decision module are as follows:

[0130] Energy calculation: By calculating the L2 norm of this region in the feature map, the feature energy value is found to be low.

[0131] Small target identification: Since the feature energy is concentrated in the low layer, the module identifies this area as a small target area.

[0132] Downsampling strategy: Select a stride of 4 for Stage 4 to ensure that details of small targets are not lost.

[0133] II. Large target area (trucks):

[0134] Suppose that in another region ahead, the feature energy is concentrated at higher levels (e.g., a larger L2 norm), which typically indicates that the target object in that region is large, such as a truck. The processing steps of the target scale-aware decision module are as follows:

[0135] Energy calculation: The region has high characteristic energy, and the L2 norm shows a strong characteristic signal.

[0136] Large target identification: Since the feature energy is concentrated in the upper layer, the module determines that the area is a large target area.

[0137] Downsampling strategy: The stride of Stage 4 is set to 8 to improve computational efficiency and maintain sufficient semantic information to detect large targets.

[0138] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the optimization steps of a preset recognition model provided in an embodiment of this application. Specifically, the 3D point cloud data processing method further includes the following optimization steps for the preset recognition model:

[0139] 401. Load the pre-trained preset recognition model and use it as the teacher network of the teacher-student network.

[0140] The purpose of this application is to use a teacher network as a guide, with the student network learning its output features. In this way, the student network can quickly learn similar feature representations without requiring a large amount of data, thereby achieving similar performance to the teacher network.

[0141] 402. Input the original point cloud data into the teacher network, and use the output feature map of the third-level 3D sparse convolutional layer of the teacher network as a supervision signal.

[0142] In some implementations, in steps 401-402, a pre-trained full version of the DARF model (FP32 precision) is loaded as the teacher network. The same batch of point cloud data is input into the teacher network, and the output feature map (M / 4×512) of its third-level 3D sparse convolutional layer is extracted as a supervision signal. This feature map contains rich semantic information (such as vehicle outlines and pedestrian limb structures) and can be used to guide the student network (lightweight version) to learn similar feature representations.

[0143] 403. Minimize the L2 distance between the output feature map of the third-level 3D sparse convolutional layer of the student network and the corresponding output feature map of the teacher network, and backpropagate the feature distillation loss to the student network through a gradient inversion layer (GRL); wherein the student network adopts the INT8 quantization version.

[0144] In one specific implementation, the output feature map of the third-level 3D sparse convolutional layer of the student network (INT8 quantized version) is minimized with the corresponding feature map of the teacher model using L2 distance. The feature distillation loss is backpropagated to the student network through a gradient inversion layer (GRL). At the same time, KL divergence loss and Smooth-L1 loss are applied to the output of the classification head and regression head of the student network to ensure that its decision boundary is consistent with that of the teacher network.

[0145] The student network uses an INT8 quantized version to reduce storage space and computational resource consumption, and its accuracy is improved through feature distillation. To ensure the output distribution of the student network is consistent with that of the teacher network, KL divergence loss is employed. Smooth-L1 loss is used to further refine the model's regression output, ensuring that the predicted anchor boxes match the true targets.

[0146] 404. Symmetrically quantize the weights of the convolutional layers of the student network, with the quantization range set to [-α, α], where α is the maximum value of the absolute value of the weights; and approximate the gradient of the quantization function during backpropagation using the pass-through estimator (STE).

[0147] In one specific implementation, the weights of the convolutional layers of the student network are symmetrically quantized (range [-α, α], where α is the maximum value of the absolute value of the weights). A pass-through estimator (STE) is used to approximate the gradient of the quantization function during backpropagation (i.e., the gradient is directly passed to the floating-point weights, bypassing the quantization operation). The weights of the self-attention layers of the Transformer retain FP32 precision to avoid the accumulation of quantization errors.

[0148] Understandably, during backpropagation, a pass-through estimator (STE) is used to approximate the gradient of the quantization function. This means that the gradient of the quantization operation bypasses the actual quantization process, allowing for smooth backpropagation and avoiding the gradient vanishing problem caused by the quantization operation.

[0149] 405. During the inference process of the student network, the dynamic range of the input activation values ​​of each 3D sparse convolutional layer is statistically analyzed, and the quantization parameters are dynamically adjusted based on the statistical results.

[0150] Understandably, during model inference, the dynamic range of the input activation values ​​of each layer (such as the maximum and minimum values ​​of the ReLU output values) is statistically analyzed, and the quantization parameters (zero point and scaling factor) are dynamically adjusted based on the statistical results. For example, a wider quantization range ([0,8.0]) is used for the activation values ​​of the vehicle detection branch to preserve the details of large targets, while a narrower range ([0,4.0]) is used for the pedestrian detection branch to improve quantization accuracy.

[0151] 406. The 3D sparse convolutional layer, batch normalization, and ReLU activation operation of the student model are merged into a single CBR operator.

[0152] In practice, the continuous convolution (Conv), batch normalization (BN), and ReLU activation operations in the student model are fused into a single CBR operator, and the scaling and offset parameters of the BN layer are merged into the convolution weights. Reducing memory access frequency, experiments show that CBR fusion can reduce single-layer inference time by 15%-20%.

[0153] In one implementation, the convolutional layer (Conv), batch normalization layer (BN), and ReLU activation operation are merged into a single operator. During the merging process, the scaling and offset parameters of the BN layer are incorporated into the convolutional weights, calculated as follows:

[0154] , ;

[0155] in, and α and β are the fused convolution weights and offset parameters, respectively; γ and β are the batch normalization scaling and offset parameters; and μ and σ are the mean and variance.

[0156] This invention innovatively combines multiple techniques, including a teacher-student network framework, feature distillation, convolutional layer quantization, dynamic quantization parameter adjustment, and operation fusion. By reducing the computational resource requirements of the student network while maintaining its performance, it significantly improves the performance of lightweight object detection models on complex 3D point cloud data. Specifically, through feature distillation, the student network can learn high-quality feature representations similar to those of the teacher network; through quantization and dynamic adjustment of quantization parameters, computational resource consumption is effectively reduced; and through operation fusion, inference efficiency is improved. These innovative methods not only improve the model's accuracy and efficiency but also reduce storage space and computational resource consumption, making this method widely applicable in practical applications such as autonomous driving and robot vision.

[0157] In a preferred implementation, the optimized model is converted to the TensorRT engine format, further enhancing inference performance by leveraging TensorRT's layer fusion and kernel auto-tuning capabilities. In this way, sparse convolutions and Transformer attention operations can be efficiently mapped to the GPU's TensorCore units, using WMMA (Warp Matrix Multiply-Accumulate) instructions to achieve 16-bit floating-point mixed-precision computation.

[0158] This invention significantly improves inference throughput, reduces inference time, and enhances real-time processing capabilities through hardware acceleration libraries and Tensor Core optimizations. Using mixed-precision computing (FP16) reduces memory bandwidth and storage requirements while accelerating the inference process.

[0159] In another implementation, for edge devices such as Jetson AGX Xavier, the optimized model is converted to ONNX format and deployed using TensorRT's INT8 quantization engine to adapt to edge devices (such as Jetson AGX Xavier). Simultaneously, dynamic input resolution adjustment and the DLA (Deep Learning Accelerator) hardware accelerator are combined to balance accuracy and latency, ensuring efficient inference.

[0160] This invention combines quantization and hardware accelerators to enable the model to run efficiently on resource-constrained edge devices, ensuring both inference speed and accuracy. Dynamically adjusting the input resolution balances accuracy and inference latency based on device load, guaranteeing system responsiveness.

[0161] In a preferred embodiment, after model deployment, the model's inference latency (ms / frame), accuracy (mAP), and resource utilization (GPU / CPU utilization) are continuously monitored, and a feedback mechanism is established. If a performance degradation is detected (e.g., mAP drops by more than 5%), an optimization process is automatically triggered. This invention, through a monitoring system and an adaptive optimization process, ensures that the model always maintains high efficiency and high accuracy, avoiding performance degradation caused by environmental changes.

[0162] For example, the optimization process is automatically triggered. Specifically, if the latency exceeds the limit, the model is further compressed (e.g., the number of Transformer layers is reduced). If the accuracy decreases, the model is rolled back to the previous version and retrained, forming a closed-loop continuous optimization system.

[0163] The core objective of this invention is to address the shortcomings of existing 3D target detection technologies in terms of multi-scale target adaptability, small target detection accuracy, balance between computational efficiency and accuracy, and multimodal feature fusion. By innovating a dynamic receptive field mechanism, a local-global collaborative attention module, and an efficient dual-domain coupling architecture, this invention achieves high-precision and high-efficiency real-time 3D target detection, meeting the stringent requirements of complex scenarios such as autonomous driving and robot navigation.

[0164] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a three-dimensional point cloud data processing device provided in an embodiment of this application. In a specific implementation, the three-dimensional point cloud data processing device includes:

[0165] The acquisition module is used to acquire the raw point cloud data collected by the lidar and to preprocess the raw point cloud data.

[0166] The voxelization module is used to divide multiple voxel meshes of preset sizes in three-dimensional space based on preprocessed three-dimensional point cloud data.

[0167] The partitioning module is used to traverse all voxel meshes and partition them into low-density voxel meshes and high-density meshes.

[0168] The merging module is used to merge low-density voxel meshes into adjacent high-density meshes using a preset merging algorithm and mark them as valid voxel meshes.

[0169] The recognition module is used to input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

[0170] In a preferred embodiment, the 3D point cloud data processing device also dynamically adjusts the inference batch size based on the density of the input point cloud to optimize GPU computational efficiency and memory management. Low-density point clouds (e.g., fewer than 5000 points) are processed in larger batches (e.g., 8 frames), while high-density point clouds (e.g., more than 20000 points) are processed in smaller batches (e.g., 4 frames). This strategy effectively balances GPU computational power and memory consumption.

[0171] Understandably, using a strategy of dynamically adjusting batch sizes can fully utilize the GPU's computing power, reduce idle time, and improve the efficiency of GPU resource utilization. Dynamically adjusting batch sizes helps optimize video memory usage and avoid memory overflow issues caused by excessively large batches.

[0172] In another preferred implementation, the 3D point cloud data processing device receives updated model parameters from the cloud via OTA (Over-the-Air) technology and performs incremental quantization-aware training (Incremental QAT) on the edge device. This process only updates the weights of key layers in the model (such as the Transformer attention layer) to adapt to new scenarios (such as environmental changes like rainy days) while retaining the original quantization parameters to avoid accuracy fluctuations.

[0173] This invention utilizes OTA (Over-The-Air) technology, enabling the model to receive the latest updates in real time, ensuring continuous operation of the system in complex and changing environments. Incremental quantization training avoids accuracy fluctuations during the update process, guaranteeing that the updated model performs consistently with the original model in new scenarios.

[0174] This invention further enhances the performance, flexibility, and adaptability of 3D point cloud data processing models through techniques such as dynamic batch processing, hardware acceleration, edge device adaptation, online update mechanisms, and performance monitoring feedback loops. These optimizations enable the model to run efficiently on edge devices, ensuring stability and efficiency under various environmental and device conditions.

[0175] This application also provides an electronic device, including at least one processor, a memory, and a communication interface, wherein the processor is used to execute... Figure 1 , Figure 3-4 The method described.

[0176] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0177] like Figure 6 As shown, the electronic device 600 includes at least one processor 601, a memory 603, and a communication interface 602. The processor 601, memory 603, and communication interface 602 are communicatively connected, or they can communicate via wireless transmission or other means. The communication interface 602 is used to receive 3D point cloud data sent by a sensing module (e.g., LiDAR); the memory 603 stores computer instructions, and the processor 601 executes these computer instructions to perform the 3D point cloud data processing method based on multi-scale feature fusion as described in the aforementioned method embodiments.

[0178] It should be understood that in the embodiments of this application, the processor 601 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0179] The memory 603 may include read-only memory and random access memory, and provides instructions and data to the processor 601. The memory 603 may also include non-volatile random access memory.

[0180] The memory 603 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0181] It should be understood that the electronic device 600 according to the embodiments of this application can perform the implementation of the embodiments of this application. Figure 1 , Figure 3-4 The method shown is described in detail above, and will not be repeated here for the sake of brevity.

[0182] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform any of the above-mentioned three-dimensional point cloud data processing methods based on multi-scale feature fusion.

[0183] This application also provides a computer program or computer program product, which includes instructions that, when executed, cause a computer to perform any of the above-described three-dimensional point cloud data processing methods based on multi-scale feature fusion.

[0184] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0185] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, processor-executed software modules, or a combination of both. The software modules can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.

[0186] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for processing 3D point cloud data based on multi-scale feature fusion, characterized in that, The three-dimensional point cloud data processing method includes the following steps: Acquire raw point cloud data collected by lidar and preprocess the raw point cloud data; Based on the preprocessed 3D point cloud data, multiple voxel meshes of preset sizes are divided in 3D space; Traverse all voxel meshes and divide them into low-density voxel meshes and high-density meshes; A preset merging algorithm is used to merge low-density voxel meshes into adjacent high-density meshes and mark them as effective voxel meshes; Input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

2. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 1, characterized in that, Traversing all voxel meshes, dividing them into low-density and high-density voxel meshes, specifically including the following steps: Traverse all voxel grids and count the number of points in each voxel grid; If the number of point clouds in a voxel mesh is lower than a preset threshold, it is marked as a low-density mesh. If the number of point clouds in a voxel mesh exceeds a preset threshold, it is marked as a high-density mesh.

3. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 1, characterized in that: The preset merging algorithm is an eight-neighbor search strategy; wherein, if all neighbors of the current low-density grid are low-density, the low-density grid is skipped directly and marked as an invalid voxel grid.

4. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 1, characterized in that, The preset recognition model includes: The feature encoding module is used to extract and encode features from the effective voxel grid to generate an initial voxel feature map; A progressive downsampling module has multiple cascaded 3D sparse convolutional layers. The progressive downsampling module is used to extract features from the initial voxel feature map and generate an output feature map. The multi-scale feature fusion layer is used to fuse the output feature maps of different 3D sparse convolutional layers to output a multi-scale feature map. An attention mechanism layer with a Transformer decoder is used to compute attention weights for multi-scale feature maps. The anchor box generation module generates adaptive anchor boxes for several target objects based on multi-scale feature maps.

5. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 4, characterized in that, The preset recognition model performs the following steps: For the effective voxel grid, local features of each point within the grid are extracted, including center coordinate offset, mean and standard deviation of reflection intensity; The local features are mapped to a 64-dimensional feature space using a multilayer perceptron (MLP) to generate an initial voxel feature map. The initial voxel feature map is input into multiple cascaded 3D sparse convolutional layers for feature extraction, and each 3D sparse convolutional layer outputs a corresponding output feature map; wherein, the stride of the multiple cascaded 3D sparse convolutional layers increases progressively, and the stride of the first 3D sparse convolutional layer is 1. The output feature maps of at least two different levels of 3D sparse convolutional layers are fused through a feature pyramid network to output a fused multi-scale feature map. The multi-scale feature maps are input into the Transformer decoder, and the attention weights of the multi-scale feature maps are calculated using the KNN algorithm. The multi-scale feature map is identified, and adaptive anchor boxes for several target objects are generated. An anchor box offset regression algorithm is used to make the adaptive anchor boxes close to the center of the target objects.

6. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 4, characterized in that, The preset recognition model includes: The target scale-aware decision module is used to statistically analyze the feature energy of the output feature map of the second-level 3D sparse convolutional layer in order to determine the downsampling strategy of the progressive downsampling module.

7. The three-dimensional point cloud data processing method based on multi-scale feature fusion as described in claim 4, characterized in that, The 3D point cloud data processing method also includes the following optimization steps for the preset recognition model: Load the pre-trained preset recognition model and use it as the teacher network of the teacher-student model; The original point cloud data is input into the teacher network, and the output feature map of the third-level 3D sparse convolutional layer of the teacher network is used as a supervision signal. The output feature map of the third-level 3D sparse convolutional layer of the student network is minimized with the corresponding output feature map of the teacher network using L2 distance, and the feature distillation loss is backpropagated to the student network through a gradient inversion layer (GRL); wherein, the student network uses the INT8 quantized version. The weights of the convolutional layers of the student network are symmetrically quantized, with the quantization range set to [-α, α], where α is the maximum value of the absolute value of the weights; and the gradient of the quantization function is approximated during backpropagation using a pass-through estimator (STE). During the inference process of the student network, the dynamic range of the input activation values ​​of each 3D sparse convolutional layer is statistically analyzed, and the quantization parameters are dynamically adjusted based on the statistical results. The 3D sparse convolutional layer, batch normalization, and ReLU activation operation of the student model are fused into a single CBR operator.

8. A three-dimensional point cloud data processing device, characterized in that, The three-dimensional point cloud data processing device includes: The acquisition module is used to acquire the raw point cloud data collected by the lidar and to preprocess the raw point cloud data. The voxelization module is used to divide multiple voxel meshes of preset sizes in three-dimensional space based on preprocessed three-dimensional point cloud data. The partitioning module is used to traverse all voxel meshes and partition them into low-density voxel meshes and high-density meshes. The merging module is used to merge low-density voxel meshes into adjacent high-density meshes using a preset merging algorithm and mark them as valid voxel meshes. The recognition module is used to input the effective voxel mesh into the preset recognition model and output adaptive anchor boxes for several target objects.

9. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the three-dimensional point cloud data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code, which, when executed by the processing circuit, implements the three-dimensional point cloud data processing method as described in any one of claims 1 to 7.

Images