Regional adaptive dynamic laser radar point cloud compression method and system guided by correlation sorting, electronic equipment and medium

Abstract

The invention discloses a correlation sorting guided regional adaptive dynamic laser radar point cloud compression method and system, electronic equipment and a medium. In order to solve the problem that an existing point cloud compression method does not utilize fine-grained semantic priority to realize adaptive bit distribution, an adaptive correlation learning module utilizing semantic context and uncertainty information is provided to identify important areas, and correlation scores of different areas are given. According to the method, a multi-scale adaptive feature extraction module is guided by using the obtained correlation score, and the local details of the point cloud are obtained in an efficient manner by focusing on a key area in the encoding and decoding process. And meanwhile, the richness and robustness of the motion characteristics are enhanced by combining an adaptive multi-modal motion estimation module, so that more accurate and reliable motion estimation is realized in a complex three-dimensional scene.

Description

Technical Field

[0001] This invention belongs to the field of point cloud compression technology, specifically relating to a region adaptive dynamic lidar point cloud compression method, system, electronic device, and medium guided by correlation ranking. Background Technology

[0002] LiDAR point cloud compression is widely used to address the need for efficient management of massive and complex 3D data. However, the original data is often enormous, reaching GB or even TB levels, with significant spatiotemporal redundancy in consecutive frames. This poses serious challenges to real-time processing, transmission, and storage, thus requiring robust and efficient compression methods.

[0003] Traditional LiDAR point cloud compression (LPCC) methods, such as VPCC and GPCC, primarily rely on 2D projection or native 3D techniques for predictive encoding. These traditional frameworks, based on hand-designed feature designs and heuristic matching, limit their flexibility and encoding efficiency across different scenarios. For example, depth image-based methods convert 3D points into 2D images to simplify encoding, but often sacrifice important geometric details due to introduced distortion; octree-based solutions hierarchically partition the 3D space to utilize multi-scale structures, but suffer from the overhead of constructing and traversing the full octree; autoencoder-based models utilize the learned latent space and refined loss functions, but are still limited by their dependence on global statistics, which restricts their ability to capture local geometric details and semantic diversity within a scene.

[0004] Furthermore, existing techniques also employ semantically guided compression by classifying point clouds into categories such as foreground and background and applying category-specific or selective encoding. While this marks a shift beyond purely geometric encoding, most rely on strict category divisions and treat all points within each category as equally important. This simplistic assumption ignores the significant differences in object relevance within so-called "important" foregrounds. Moreover, because these methods often assume the semantic or feature information used to guide compression is accurate and complete due to the model's limited knowledge of dynamic objects and their changing appearance under different environmental conditions, they implicitly ignore the inherent uncertainties in dynamic scenes, leading to suboptimal bit allocation and limiting compression efficiency.

[0005] In summary, existing semantic ensemble techniques remain fundamental and fragmented. Many methods employ coarse binary segmentation, failing to distinguish key targets such as vehicles and pedestrians, or rely on multi-model ensembles, increasing training complexity and runtime overhead. Neither of these strategies achieves unified fine-grained semantic prioritization for adaptive bit allocation. Furthermore, these frameworks implicitly assume that the input data and features are complete and reliable, thus ignoring model uncertainties when dealing with dynamic or partially observed inputs. Secondly, existing feature extraction paradigms lack true receptive field adaptability. While many networks employ multi-scale layers to process information at various scales, the receptive field of each layer is determined by a hand-crafted, static kernel size. This rigid structure hinders the dynamic adjustment of the information aggregation range to cope with real-time changes in scene content, thereby reducing the efficiency of spatiotemporal redundancy removal. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, the present invention aims to provide a region-adaptive dynamic LiDAR point cloud compression method, system, electronic device, and medium guided by relevance ranking. It adopts a new compression framework that integrates fine-grained semantic awareness, uncertainty quantization, and adaptive context-sensitive feature extraction tailored for complex dynamic 3D scenes.

[0007] To achieve the above objectives, a first aspect of the present invention provides a region-adaptive dynamic lidar point cloud compression method guided by correlation ranking, comprising the following steps:

[0008] Step 1: Input two consecutive frames of LiDAR data and use the adaptive correlation learning module to obtain the correlation ranking score.

[0009] Step 2: Input the features and correlation ranking scores of two consecutive LiDAR point clouds, and use the multi-scale adaptive feature extraction module to extract the latent feature representations of the two point clouds respectively.

[0010] Step 3: The adaptive multimodal motion estimation module analyzes the latent feature representation and calculates the motion vector; it performs motion compression and compensation on the motion vector and the latent representation of the previous frame to generate the prediction frame for the current frame.

[0011] Step 4: Subtract the true latent representation of the current frame from the predicted frame to obtain the residual, and then compress it through residual compression and entropy coding.

[0012] Step 5: Add the decoded residual tensor to the predicted frame, and restore the point cloud of the current frame through the point cloud reconstruction module.

[0013] A second aspect of the present invention provides a region-adaptive dynamic lidar point cloud compression system guided by correlation ranking, comprising:

[0014] An adaptive correlation learning module is used to input two consecutive LiDAR frames and obtain correlation ranking scores.

[0015] The multi-scale adaptive feature extraction module is used to extract the latent feature representations of two consecutive LiDAR point cloud frames from the input features and correlation ranking scores.

[0016] An adaptive multimodal motion estimation module is used to analyze the latent feature representation and calculate the motion vector;

[0017] The motion compensation and compression module is used to perform motion compression and compensation on the motion vector and the latent representation of the previous frame to generate the prediction frame of the current frame.

[0018] The residual compression and entropy coding module is used to compress the residual obtained by subtracting the predicted frame from the true latent representation of the current frame;

[0019] The point cloud reconstruction module is used to add the predicted frame and the residual tensor and input them to obtain the reconstructed current frame.

[0020] A third aspect of the present invention provides an electronic device for region-adaptive dynamic lidar point cloud compression guided by correlation ranking, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-described point cloud compression method when executing the program.

[0021] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program for performing the point cloud compression method described above.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) In view of the problem that existing point cloud compression methods do not utilize fine-grained semantic priority to achieve adaptive bit allocation, this invention proposes an adaptive relevance learning module that uses semantic context and uncertainty information to identify important regions and give relevance scores for different regions.

[0024] (2) In view of the lack of true receptive field adaptability in the existing feature extraction paradigm, the present invention uses the obtained correlation score to guide the multi-scale adaptive feature extraction module to focus on key regions during the encoding and decoding process, so as to obtain local details of point cloud in an efficient manner.

[0025] (3) To address the problem of low efficiency in removing spatiotemporal redundancy, this invention proposes an adaptive multimodal motion estimation module to enhance the richness and robustness of motion features, thereby achieving more accurate and reliable motion estimation in complex 3D scenes. Attached Figure Description

[0026] Figure 1 This is a flowchart of a region adaptive dynamic lidar point cloud compression method guided by correlation ranking, according to an embodiment of this application.

[0027] Figure 2 This is a diagram illustrating the architecture of the multi-scale adaptive feature extraction module in an embodiment of this application.

[0028] Figure 3 This is a structural diagram of the adaptive multimodal motion estimation module in an embodiment of this application;

[0029] Figure 4 This is a comparison analysis of the D1-PSNR of this test case and existing models. Detailed Implementation

[0030] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] like Figure 1 As shown, this application provides a region-adaptive dynamic lidar point cloud compression method guided by relevance ranking, specifically including the following steps:

[0032] Step 1: Adaptive Relevance Learning

[0033] Input two consecutive frames of lidar The correlation ranking score is obtained by using the adaptive correlation learning module.

[0034] It is worth noting that in point clouds of autonomous driving scenarios, the perceived value of different regions is extremely uneven. For example, pedestrians, which are crucial for safety decisions, deserve far more attention than large areas of background road surface. To replace the inefficient homogenized processing paradigm, this embodiment designs an adaptive relevance learning module to obtain a relevance score. By quantifying this difference in importance, the model's computational and learning resources are dynamically focused on the most critical regions.

[0035] Specifically, the adaptive correlation learning module works as follows:

[0036] First, a sparse U-Net backbone network is used for each input voxel. Extract a feature vector rich in contextual information Based on this, a unified scoring function is used to integrate two orthogonal and complementary information channels (i.e., the intrinsic saliency of points and the cognitive uncertainty of the model) to obtain a normalized relevance score:

[0037]

[0038] in, The relevance score. As weight.

[0039] Furthermore, the numerator on the right side of the above formula consists of two key terms:

[0040] 1) Based on feature saliency This section is responsible for evaluating the intrinsic value of the points. The feature vector... First, through a learnable nonlinear embedding function (Implemented by MLP) undergo a transformation, and then it is combined with a learnable attention vector. Perform an inner product. During training, this sub-network learns to identify feature signatures highly correlated with key semantic categories (such as pedestrians and vehicles) and assigns them high base scores. Here... It represents the feature vector of any point in the dataset.

[0041] 2) Uncertainty based on information theory This item predicts entropy. This quantifies the uncertainty of the model's understanding of a point; higher entropy usually indicates a more complex and difficult-to-distinguish scene in that region (such as inter-class boundaries), thus containing higher information gain. Introducing this allows the model to balance the utilization of known key features and the exploration of unknown and ambiguous regions when evaluating its value.

[0042] Step 2: Multi-scale adaptive feature extraction:

[0043] Input two consecutive frames of lidar point cloud features Based on the correlation ranking score, a multi-scale adaptive feature extraction module is used to extract the latent feature representations of the two point clouds. .

[0044] Furthermore, the multi-scale adaptive feature extraction module, by inputting two consecutive frames of LiDAR data and their correlation ranking scores, outputs features. Its structure is as follows Figure 2 As shown. This module utilizes dynamically generated convolution kernels and relevance ranking scores by introducing multi-scale convolution. This approach enables adaptive capture of geometric features at different scales and prioritized encoding of key information. This not only enhances the diversity and richness of feature extraction but also optimizes bit rate allocation through saliency ranking, significantly improving compression efficiency and reconstruction quality, and addressing the shortcomings of traditional methods in multi-scale feature capture and importance assessment.

[0045] Furthermore, the multi-scale adaptive feature extraction module operates as follows:

[0046] First, using the input mask tensor, the index of the nearest neighbor of each point is quickly found using KDTree. And obtain its corresponding relevance score. A weight generation network (kernel_generator) is designed using two linear layers and the ReLU function. This network dynamically generates convolutional kernels based on input features, and the attention weights are obtained after a softmax operation. Further Multiply by the significance score to obtain the final adaptive weight. The intermediate feature representation is obtained by weighted aggregation of neighborhood features. The detailed formula is as follows:

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053] in, This represents the index of the currently processed query point, ranging from 1 to N. This represents the index in the set of reference points, ranging from 1 to M. Indicates the current point Three-dimensional geometric coordinates Indicates the first reference point in the set. The coordinates of the points are all derived from the mask tensor.

[0054] Then The input is fed into multi-scale convolutional units and passed through different sparse convolutional layers. The aggregated features are processed at multiple scales to extract feature representations at different scales. A single convolutional layer is used to weight and merge the features at different scales to obtain the merged features. The features are further optimized through an attention layer to highlight important features, suppress redundant information, and learnable weight parameters are obtained using softmax. Ultimately, and Multiplication outputs high-quality features that integrate multi-scale information and relevance ranking. This ensures that subsequent compression and reconstruction modules can efficiently process and restore point cloud data. The detailed formula is as follows:

[0055]

[0056]

[0057]

[0058] Step 3: Adaptive multimodal motion estimation:

[0059] like Figure 3 As shown, this embodiment employs an adaptive multimodal motion estimation module for motion estimation between consecutive point cloud frames. It takes the latent feature representations of two consecutive point cloud frames as input and outputs a motion vector. Compared to existing methods, the local attention mechanism unit significantly enhances the expressive power of point cloud features, enabling the model to capture local geometric structures and dynamic changes more accurately. Secondly, the motion convolution unit employs a dynamic convolution strategy, adaptively adjusting the convolution kernel based on input features, thereby improving the flexibility and diversity of multi-scale feature extraction.

[0060] In the specific implementation process, the latent feature representation of the two point clouds is first performed. Merge the features to generate a fused feature map. The feature representation is obtained after two layers of sparse convolution. The input is fed into the local attention mechanism unit, which consists of three sparse convolutions that map the latent feature representation of the input to the query (Q), key (K), and value (V) respectively.

[0061]

[0062]

[0063]

[0064] Then, the attention weights of each point are precisely calculated using a self-attention mechanism within the local neighborhood. These weights are subsequently applied to the weighted aggregation of neighboring features to generate an enhanced feature representation, significantly improving the expressive power of local geometric structures. Next, the proposed dynamic convolutional unit is used to perform dynamic convolution operations on the self-attention-processed features, further extracting and integrating multi-scale geometric information, thereby enhancing feature diversity and fine-grained representation. Finally, multi-scale motion fusion is used to obtain the motion vector. .

[0065] Step 4: Motion compensation and compression:

[0066] The motion vector obtained from the input adaptive multimodal motion estimation module and potential representation The motion compensation and compression module processes the data to generate the predicted frame for the current frame. .

[0067] Step 5: Residual compression and entropy coding:

[0068] The residual is obtained by subtracting the true latent representation of the current frame from the predicted frame. It is compressed using residual compression and entropy coding.

[0069] Step 6: Point Cloud Reconstruction

[0070] Predicted frames and residual tensor Add them together, and The input is fed into the point cloud reconstruction module to obtain the reconstructed current frame. The point cloud reconstruction module consists of three units: multi-scale adaptive transformation, two downsampling units, and coordinate offset unit.

[0071] This application also provides a region-adaptive dynamic lidar point cloud compression system guided by correlation ranking, including:

[0072] The adaptive relevance learning module is used to obtain a relevance ranking score from two consecutive LiDAR frames. It guides attention resources to key regions by learning the semantic context and uncertainty of key regions.

[0073] A multi-scale adaptive feature extraction module is used to extract latent feature representations from two consecutive LiDAR point cloud frames, taking as input two consecutive frames of point cloud features and relevance ranking scores. This module dynamically adjusts the feature extraction process, utilizing pre-computed point-by-point relevance scores that capture semantic importance and uncertainty, and combining these scores with local geometric information through a learnable kernel generative network.

[0074] An adaptive multimodal motion estimation module is used to analyze the latent feature representation and calculate motion vectors. It enhances motion estimation for dynamic LiDAR point cloud compression by using process content awareness and sensitivity to semantic relevance across different scene regions. This module, combined with the aforementioned multi-scale adaptive feature extraction module, preserves details through relevant guided feature aggregation while achieving robust motion alignment in dynamic scenes.

[0075] The motion compensation and compression module is used to perform motion compensation and compression on the motion vector and the potential representation of the previous frame to generate the prediction frame of the current frame.

[0076] The residual compression and entropy coding module is used to compress the residual obtained by subtracting the predicted frame from the true latent representation of the current frame.

[0077] The point cloud reconstruction module is used to add the predicted frame and the residual tensor and input them to obtain the reconstructed current frame.

[0078] This application discloses an electronic device for region-adaptive dynamic lidar point cloud compression guided by correlation ranking. It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned point cloud compression method. The memory may include main memory, such as high-speed random access memory, or it may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus. This internal bus can be an industry-standard architecture bus, a peripheral component interconnection standard bus, an extended industry-standard architecture bus, etc., and the bus can be divided into an address bus, a data bus, a control bus, etc. The memory stores the program; specifically, the program may include program code, which includes computer operation instructions. The memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0079] This application further discloses a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the point cloud compression method described above. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0080] To verify the effectiveness of the method designed in this application, this test case conducts compression experiments on dynamic LiDAR point clouds on the nuScenes and Ford datasets. nuScenes is a large, publicly available multimodal dataset released for autonomous driving research, including: one 32-line LiDAR, six cameras, five RADARs, GPS, and IMU. This dataset contains 1000 driving scenes, each 20 seconds long, providing rich 3D object (23 categories) annotations and detailed high-precision map information.

[0081] The Ford dataset is a large dataset released for autonomous driving research, focusing on long-term localization and SLAM. Its sensor suite mainly includes: 4 32-line LiDARs, 7 cameras, and 1 GPS / IMU integrated navigation system. The dataset provides raw data and high-precision localization ground truth from multiple vehicles, and is mainly used to test the robustness of SLAM and localization algorithms under seasonal environmental changes, but does not include 3D object annotations.

[0082] The nuScenes dataset used in this embodiment can be downloaded from https: / / www.nuscenes.org / download, and the Ford dataset can be downloaded from https: / / avdata.ford.com / downloads / default.aspx.

[0083] Meanwhile, the results of this application's method were compared with those of rpcc, SparsePCGC, OctAttention, GPCC, etc. on the nuScenes and Ford datasets, and the results were evaluated using the D1-PSNR metric, which is widely used in point cloud compression evaluation.

[0084] The obtained rate-distortion (RD) curve is shown below. Figure 4 As shown. Among them, Figure 4 The left-middle plot evaluates the performance of five point cloud compression algorithms on the Ford dataset, while the right-middle plot evaluates their performance on the Nuscenes dataset. The horizontal axis, BPP (Bits Per Point), represents the compression ratio, i.e., the number of bits required to compress each point; a lower value indicates more efficient compression. The vertical axis, D1 PSNR (dB), represents the reconstruction quality; a higher value indicates less distortion and better quality. It is clear that the curve for our method consistently lies to the upper left of all other curves, indicating that our method achieves the highest D1 PSNR reconstruction quality at any given BPP compression ratio; or, in other words, requires only the lowest BPP bitrate for any given D1 PSNR quality requirement. This result strongly demonstrates that the performance of our invention is significantly superior to all comparative methods.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A region-adaptive dynamic lidar point cloud compression method guided by correlation ranking, characterized in that, Includes the following steps: Step 1: Input two consecutive LiDAR frames and use the adaptive correlation learning module to obtain the correlation ranking score; Step 2: Input the features and correlation ranking scores of two consecutive LiDAR point clouds, and use the multi-scale adaptive feature extraction module to extract the latent feature representations of the two point clouds respectively. Step 3: The adaptive multimodal motion estimation module analyzes the latent feature representation and calculates the motion vector; it performs motion compression and compensation on the motion vector and the latent representation of the previous frame to generate the prediction frame for the current frame; Step 4: Subtract the true latent representation of the current frame from the predicted frame to obtain the residual, and then compress it using residual compression and entropy coding; Step 5: Add the decoded residual tensor to the predicted frame, and restore the point cloud of the current frame through the point cloud reconstruction module.

2. The method according to claim 1, characterized in that, The adaptive correlation learning module operates as follows: A sparse U-Net backbone network is used to extract a feature vector rich in contextual information for each input voxel; Based on the feature vector, a normalized relevance score is obtained by combining the intrinsic feature significance of the points with the cognitive uncertainty of the model through a scoring function.

3. The method according to claim 2, characterized in that, The intrinsic feature saliency is calculated as follows: the feature vector is transformed through a learnable nonlinear embedding function, and then the inner product is performed with the learnable attention vector.

4. The method according to claim 2 or 3, characterized in that, The uncertainty of this cognition is quantified by predicting entropy.

5. The method according to claim 1, characterized in that, The multi-scale adaptive feature extraction module operates as follows: Find the index of the nearest neighbor for each point and obtain its corresponding relevance score; A weighted generation network dynamically generates convolutional kernels based on input features. Attention weights are obtained through a softmax operation, multiplied by the saliency score to obtain adaptive weights, and then intermediate feature representations are obtained by weighted aggregation of neighborhood features. ; Will Input is fed into multi-scale convolutional units to extract feature representations at different scales; a single convolutional layer is used to weight and merge the features at different scales to obtain the merged features. After further optimizing the features through a self-attention layer, learnable weight parameters are obtained using softmax. ; Will and Multiplication outputs features that integrate multi-scale information and relevance ranking. .

6. The method according to claim 1, characterized in that, Step 3 includes: Step 3.1: Input the latent feature representations of two consecutive point clouds into the adaptive multimodal motion estimation module, and use a local attention mechanism combined with dynamic convolution operation to obtain the motion vector; Step 3.2: The motion compensation and compression module performs motion compression and compensation on the motion vector and the latent representation of the previous frame to generate the prediction frame for the current frame.

7. The method according to claim 6, characterized in that, The adaptive multimodal motion estimation module operates as follows: The latent feature representations of the two point cloud frames are merged to generate a fused feature map, which is then processed through two layers of sparse convolution to obtain the feature representation. ;Will The input is fed into the local attention mechanism unit, which performs querying, key-value mapping on the latent feature representation of the input; The attention weights of each point are calculated through a self-attention mechanism in the local neighborhood, and the obtained attention weights are applied to the weighted aggregation of the neighbor features to generate an enhanced feature representation. The dynamic convolutional unit performs dynamic convolution operations on the enhanced feature representation to further extract and integrate multi-scale geometric information; Finally, multi-scale motion fusion is used to obtain motion vectors.

8. A region-adaptive dynamic lidar point cloud compression system guided by correlation ranking, characterized in that, include: An adaptive correlation learning module is used to input two consecutive LiDAR frames and obtain correlation ranking scores. The multi-scale adaptive feature extraction module is used to extract the latent feature representations of two consecutive LiDAR point cloud frames from the input features and correlation ranking scores. An adaptive multimodal motion estimation module is used to analyze the latent feature representation and calculate the motion vector; The motion compensation and compression module is used to perform motion compression and compensation on the motion vector and the latent representation of the previous frame to generate the prediction frame of the current frame. The residual compression and entropy coding module is used to compress the residual obtained by subtracting the predicted frame from the true latent representation of the current frame; The point cloud reconstruction module is used to add the predicted frame and the residual tensor and input them to obtain the reconstructed current frame.

9. An electronic device for region-adaptive dynamic lidar point cloud compression guided by correlation ranking, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements a correlation-ordered region-adaptive dynamic lidar point cloud compression method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the region adaptive dynamic lidar point cloud compression method guided by correlation ranking as described in any one of claims 1-7.

Images