Vehicle point cloud three-dimensional reconstruction system for under-forest blocking scene

Abstract

This invention belongs to the field of lidar echo signal processing and 3D geometric reconstruction technology of roadside physical targets. It discloses a 3D reconstruction system for vehicle point clouds in forest-covered scenes, solving the problem that existing roadside point cloud processing systems cannot meet the needs of roadside intelligent sensing engineering. The system includes a point cloud preprocessing module, a semantic segmentation module, a point cloud completion module, and a graphical interactive control module. The original 3D point cloud acquired by lidar undergoes dual collaborative denoising and ground adaptive pose normalization in the point cloud preprocessing module, outputting a clean and aligned point cloud. The clean and aligned point cloud is then subjected to point-by-point binary classification by the semantic segmentation module, outputting a vehicle target point cloud and a high-dimensional semantic feature tensor. This is then fed into the point cloud completion module, where cross-module feature fusion, symmetric constraint training, and spatial feature coupling generate decoding, outputting a complete 3D vehicle point cloud. This invention significantly improves the reconstruction accuracy of vehicles in extremely incomplete scenes.

Description

Technical Field

[0001] This invention belongs to the field of lidar echo signal processing and three-dimensional geometric reconstruction technology of roadside physical targets, and particularly relates to a three-dimensional reconstruction system for vehicle point clouds in a forest occlusion scene. Background Technology

[0002] In urban road scenarios, LiDAR is one of the core sensors for acquiring 3D point cloud data of the environment. In road environments with high planting density in roadside green belts, roadside trees, shrubs and other vegetation severely occlude the point cloud data of moving or parked vehicles, resulting in incomplete and fragmented vehicle point cloud data collected by LiDAR, which seriously affects the accuracy of subsequent 3D modeling, target detection and autonomous driving perception.

[0003] Traditional processing workflows deploy outlier noise filtering, semantic segmentation, and shape completion as independent modules, resulting in a progressive accumulation of output errors at each stage and a lack of end-to-end integrated processing capabilities for forest occlusion scenarios. In scenarios with severe vegetation occlusion, the number of vehicle target point clouds is far less than that of vegetation background point clouds. Traditional equal-weight loss functions cause the model to overly favor background class prediction, leading to low target recall. In existing workflows, after the semantic segmentation network outputs classification labels, the rich semantic geometric features accumulated during its encoding and decoding process are discarded. The subsequent completion network can only extract features from the original point coordinates, significantly limiting completion accuracy. Vehicles are typical symmetrical objects, but existing completion methods only use point cloud reconstruction accuracy as the training objective without introducing symmetry constraints. In scenarios with severe unilateral occlusion, the completed results generated have inconsistent left and right structures. Traditional decoding methods based on 2D mesh folding have limited ability to capture the 3D shape of fine local structures such as vehicle rearview mirrors and window corners. The coordinate system of the point cloud collected by LiDAR is inconsistent with the coordinate system of the general completion network training dataset. The roadside slope further causes the vehicle point cloud pose to be skewed, affecting the spatial consistency of the completion results.

[0004] In summary, existing roadside point cloud processing systems suffer from fragmented modules, resulting in issues such as wasted semantic features, lack of vehicle symmetry priors, insufficient ability to restore fine structures, coordinate system differences, and slope interference in forest-covered scenarios. These issues lead to insufficient accuracy in vehicle 3D reconstruction and make it difficult to meet the engineering requirements of roadside intelligent perception. Summary of the Invention

[0005] The purpose of this invention is to provide a 3D reconstruction system for vehicle point clouds in forest-shaded scenes, which effectively solves the problem that existing roadside point cloud processing systems cannot meet the needs of roadside intelligent sensing engineering.

[0006] To address the aforementioned technical problems, the present invention employs the following technical solution: a 3D reconstruction system for vehicle point clouds in forest-covered scenes, comprising a point cloud preprocessing module, a semantic segmentation module, a point cloud completion module, and a graphical interactive control module. The original 3D point cloud acquired by LiDAR undergoes dual collaborative denoising and ground-adaptive pose normalization in the point cloud preprocessing module, outputting a clean and aligned point cloud. The clean and aligned point cloud is then subjected to point-by-point binary classification by the semantic segmentation module, outputting a vehicle target point cloud and a high-dimensional semantic feature tensor. The vehicle target point cloud and the high-dimensional semantic feature tensor are then fed into the point cloud completion module, where cross-module feature fusion, symmetric constraint training, and spatial feature coupling are used to generate decoding, outputting a complete 3D vehicle point cloud.

[0007] The graphical interactive control module adopts a multi-threaded architecture that separates the main thread and the worker sub-thread. The processing tasks are executed asynchronously in the worker sub-thread, and the log information is asynchronously transmitted to the main thread to update the interface through the signal-slot mechanism, ensuring that the operation interface remains responsive during deep learning inference.

[0008] Furthermore, the point cloud preprocessing module includes a statistical outlier filtering unit, a radius outlier filtering unit, and a ground normal vector calculation unit. The statistical outlier filtering unit targets the sparse discrete noise generated by multipath reflections from roadside lidar. For each point, it calculates the mean distance between the point and several nearest neighbors, using a linear combination of the global mean and standard deviation as a statistical threshold. Points exceeding the threshold are identified as outlier noise points and removed. The radius outlier filtering unit targets isolated, small scattered points generated by vegetation leaf reflections. Using a fixed search radius as a constraint, it identifies and removes points with fewer than a preset lower limit of neighboring points as isolated noise points. The ground normal vector calculation unit uses the RANSAC algorithm to fit the local ground point set under the target vehicle after the double-filtered point cloud and extracts the local ground normal vector. ; calculation will Rotation matrix required to rotate to the positive direction of the global vertical axis To achieve adaptive pose normalization based on a local ground reference; and to save the rotation matrix. After the point cloud is completed, apply a function to the output point cloud. The transpose completes the inverse transformation and restoration.

[0009] Furthermore, the semantic segmentation module uses a 3D point cloud segmentation network based on a sparse convolutional UNet backbone network as a feature extractor. It completes initialization by dynamically constructing the model structure and loading pre-trained parameters. The sparse convolutional UNet backbone network adopts an encoding and decoding architecture. In the encoding stage, it downsamples layer by layer to extract multi-scale 3D point cloud features. In the decoding stage, it upsamples layer by layer to restore the spatial resolution of the point cloud. Skip connections preserve multi-scale contextual information and finally output the binary classification probability distribution of each point.

[0010] Furthermore, while performing gradient-free forward inference, the semantic segmentation module extracts the output of the deepest layer of the sparse convolutional UNet backbone network encoder as a high-dimensional semantic feature tensor. ; It includes a cross-module semantic-geometric feature fusion unit that comprehensively expresses the local geometric structure and global semantic context of point clouds and passes them to the point cloud completion module through the inter-module interface.

[0011] Furthermore, the semantic segmentation module is trained using a weighted combination of cross-entropy loss with class weights and Lowaz loss.

[0012] Furthermore, the point cloud completion module includes an adaptive point cloud sampling unit, a coordinate system bidirectional transformation unit, a cross-module semantic-geometric feature coupling unit, a spatial feature coupling generation unit, and a bilateral symmetry constraint loss unit.

[0013] The adaptive point cloud sampling unit unifies any number of input point clouds to the fixed number of input points required by the point cloud completion module. When the number of input points exceeds the fixed number, random sampling is performed without repetition. When the number of input points is less than the fixed number, the point cloud is repeatedly supplemented in a loop. The adaptive point cloud sampling unit translates the sampled point cloud to the centroid origin and scales it to a unit sphere, saving the centroid and scaling parameters.

[0014] Before inference, the coordinate system bidirectional transformation unit converts the Z-axis upward lidar coordinate system to the Y-axis upward coordinate system used by the point cloud completion module training dataset. After inference, it first performs inverse normalization on the completed output point cloud to restore it to its original scale and position based on the centroid and scaling parameters saved by the adaptive point cloud sampling unit, and then performs inverse coordinate transformation and superimposes the rotation matrix saved by the point cloud preprocessing module. The transpose of the output point cloud will completely restore it to the original coordinate system of the lidar.

[0015] Furthermore, the encoder of the point cloud completion module performs PointNet-style feature extraction on the input fixed-number sampled point cloud and outputs a global geometric feature vector of a preset dimension. The cross-module semantic-geometric feature fusion unit receives high-dimensional semantic feature tensors from the semantic segmentation module. The feature dimension alignment transformation layer will Project to In the same dimensional space, the aligned features are then linearly fused together. and Element-wise addition yields the enhanced global feature vector. .

[0016] Furthermore, the spatial feature coupling generation unit adopts a two-stage decoding architecture consisting of a coarse reconstruction stage and a fine reconstruction stage. In the coarse reconstruction stage: As a condition, sparse coarse-reconstructed point clouds are generated directly through regression using a fully connected network. In the detailed reconstruction phase: with The local neighborhood features of each point are used as the query, with The linear projection is used as keys and values. The attention weights of each local region to the global semantic information are calculated through a multi-head cross-modal self-attention mechanism, generating a 3D position offset vector for each local region. The offset vectors of each point are superimposed onto the spatial grid of the locally unfolded points of the coarsely reconstructed point cloud to output a dense and complete point cloud. .

[0017] Furthermore, the bilateral symmetry constraint loss unit explicitly encodes the vehicle's symmetry geometric prior into the optimization objective, including the following steps: S1, determining the symmetry reference surface based on the target geometric centerline or a preset geometric prior; S2, processing the dense complete point cloud generated by the spatial feature coupling generation unit. Perform a mirror transformation along the symmetry reference plane to obtain a mirror point cloud. S3, Calculation and The chamfer distance between them is used as the symmetry constraint loss. Total training loss function :

[0018] ;

[0019] in, and These represent the chamfer distance loss in the coarse reconstruction stage and the fine reconstruction stage, respectively. and All are weighted hyperparameters. Control the strength of symmetry constraints.

[0020] Compared with the prior art, the beneficial technical effects of the present invention are: (1) The present invention breaks through the limitation of segmentation and completion being isolated from each other in traditional point cloud processing. By using a cross-module semantic-geometric feature fusion unit, the semantic perception features extracted in the segmentation stage are transmitted to the point cloud completion module encoder, eliminating the waste of cross-module information and significantly improving the reconstruction accuracy of vehicle outline and details in extremely incomplete scenes.

[0021] (2) This invention addresses the engineering pain point of roadside lidar point clouds being affected by terrain slope. By using RANSAC ground normal vector calculation and bidirectional rotation transformation, it eliminates the spatial consistency interference of physical environment fluctuations on subsequent processing, ensuring the attitude accuracy and scene fit of the reconstructed model.

[0022] (3) This invention addresses the unique noise patterns and extreme class imbalance problems in forest shading scenarios by effectively removing vegetation noise through SOR and ROR cascaded dual filtering, providing high-quality input for subsequent processing; and significantly improves vehicle target recall rate in extreme shading scenarios through an imbalance loss weighting strategy.

[0023] (4) The present invention encodes the vehicle symmetric geometry prior into the training target through bilateral symmetry constraint loss, and replaces the two-dimensional mesh folding decoding with cross-modal self-attention of spatial feature coupling generation unit, thus achieving high-quality three-dimensional reconstruction with coordinated left and right structures and fine local integrity in a single-sided severe occlusion scene.

[0024] (5) This invention enables the general deep learning model to adapt to roadside lidar scenarios without retraining by linking bidirectional coordinate system transformation with ground alignment inverse transformation; the adaptive sampling strategy further supports input point clouds of arbitrary density, significantly improving the engineering versatility and robustness of the system. Attached Figure Description

[0025] Figure 1 This is a system architecture diagram of a vehicle point cloud 3D reconstruction system for forest shading scenarios according to the present invention.

[0026] Figure 2 This is the original point cloud effect before dual collaborative denoising in Example 1.

[0027] Figure 3 This is the point cloud effect diagram after statistical outlier filtering in Example 1.

[0028] Figure 4 This is a point cloud effect diagram after double filtering in Example 1.

[0029] Figure 5 This is a full-scene segmentation visualization result output by the semantic segmentation module in Example 1.

[0030] Figure 6 This is the vehicle target point cloud output by the semantic segmentation module in Example 1.

[0031] Figure 7 This is a schematic diagram of cross-module semantic-geometric feature fusion.

[0032] Figure 8 This is the result image after point cloud completion by the point cloud completion module in Example 1. Detailed Implementation

[0033] Example 1: This example provides a 3D reconstruction system for vehicle point clouds in a forest shading scene, such as... Figure 1As shown, the system consists of a point cloud preprocessing module, a semantic segmentation module, a point cloud completion module, and a graphical interactive control module. The original 3D point cloud acquired by the LiDAR is processed by the point cloud preprocessing module to perform dual collaborative denoising and ground adaptive pose normalization, outputting a clean and aligned point cloud. The clean and aligned point cloud is then subjected to point-by-point binary classification by the semantic segmentation module, outputting a vehicle target point cloud and a high-dimensional semantic feature tensor. The vehicle target point cloud and the high-dimensional semantic feature tensor are then fed into the point cloud completion module, where cross-module feature fusion, symmetric constraint training, and spatial feature coupling are used to generate decoding, outputting a complete 3D vehicle point cloud (i.e., a dense and complete point cloud of 16384 points).

[0034] (I) Point cloud preprocessing module.

[0035] The point cloud preprocessing module is responsible for transforming the raw LiDAR point cloud into a high-quality, pose-normalized, clean, aligned point cloud, providing stable input for subsequent semantic segmentation. The point cloud preprocessing module includes a statistical outlier filtering unit, a radius outlier filtering unit, and a ground normal vector calculation unit.

[0036] (1) Dual collaborative noise reduction.

[0037] The dual collaborative denoising process is performed in two steps. The first step is Statistical Outlier Filtering (SOR): For sparse, discrete noise generated by multipath reflections from roadside lidar, the SOR unit calculates the mean distance between each point and its nearest neighbors. A linear combination of the global mean and standard deviation is used as a statistical threshold, and points exceeding this threshold are identified as outliers and removed. The second step is Radius Outlier Filtering (ROR): For isolated, small scattered points generated by vegetation leaf reflections, a fixed search radius is used as a constraint. Points with fewer than a preset lower limit of neighboring points are identified as isolated noise points and removed.

[0038] like Figure 2 , Figure 3 and Figure 4 As shown, SOR and ROR are operated in series. The former removes systematic sparse noise, while the latter removes isolated sporadic noise. The dual filtering effect is significantly better than any single filtering method.

[0039] (2) Adaptive pose normalization based on local ground normal vector.

[0040] To address the issue of roadside slope causing vehicle point cloud pose skew and affecting the accuracy of subsequent coordinate system transformation and completion, this module introduces a RANSAC-based adaptive ground normal vector alignment mechanism. The specific steps are as follows: The ground normal vector calculation unit uses the RANSAC algorithm to fit the local ground point set under the target vehicle in the double-filtered point cloud, extracting the local ground normal vectors. ; calculation will Rotation matrix required to rotate to the positive direction of the global vertical axis The rotation transformation is applied to the entire point cloud to achieve adaptive pose normalization based on a local ground reference; the rotation matrix is ​​then saved. After the point cloud is completed, apply a function to the output point cloud. The transpose completes the inverse transformation and restoration, ensuring that the final output point cloud is precisely aligned with the original coordinate system of the lidar, and the reconstructed vehicle model fits the real geographical scene.

[0041] (ii) Semantic segmentation module.

[0042] The semantic segmentation module performs point-by-point binary classification of vehicle targets in the roadside point cloud and simultaneously outputs a high-dimensional semantic feature tensor for use by the point cloud completion module.

[0043] (1) Network structure.

[0044] The semantic segmentation module uses a 3D point cloud segmentation network based on a sparse convolutional UNet backbone as its feature extractor. Initialization is achieved by dynamically constructing the model structure and loading pre-trained parameters. The sparse convolutional UNet backbone employs an encoding-decoding architecture. In the encoding stage, it downsamples layer by layer to extract multi-scale 3D point cloud features; in the decoding stage, it upsamples layer by layer to restore the spatial resolution of the point cloud. Skip connections preserve multi-scale contextual information, ultimately outputting the binary classification probability distribution for each point. Input processing: Each frame of point cloud is voxelized with a fixed grid size, generating integer grid coordinates for the sparse convolution index.

[0045] (2) Extraction of high-dimensional semantic features.

[0046] While performing gradient-free forward inference, the semantic segmentation module extracts the output of the deepest layer of the sparse convolutional UNet backbone encoder as a high-dimensional semantic feature tensor. . It includes a comprehensive expression of the local geometric structure and global semantic context of the point cloud, which is passed to the point cloud completion module through the inter-module interface. This cross-module semantic-geometric feature fusion unit avoids the waste of rich feature information extracted in the segmentation stage between modules.

[0047] (3) Imbalanced sample adaptive loss function.

[0048] To address the extreme class imbalance problem in forest occlusion scenarios where the number of vehicle target points is far less than that of vegetation background points, the semantic segmentation module employs a weighted combination of cross-entropy loss with class weights and Lovász loss during training. Specifically, the vehicle target class loss has a significantly higher weight than the background class loss (the weight ratio can be set to target class:background class = 50:1), thus imposing a greater penalty on misclassification of the target class during training. The Lovász loss compensates for the insufficient optimization of the target class by directly optimizing the upper bound of the Intersection over Union (IoU) evaluation metric under extreme imbalance. The two losses work synergistically to significantly improve the recall rate of incomplete vehicle targets in extremely occluded environments.

[0049] (4) Output the segmentation results.

[0050] The semantic segmentation module outputs two types of results: ① Full-scene segmentation visualization files (such as...) Figure 5 As shown), background points (blue) and vehicle target points (red) are marked with different colors, and integer classification labels are recorded; ② Independent point cloud files containing only vehicle target points (such as... Figure 6 (as shown), along with the high-dimensional semantic feature tensor Together, they serve as inputs to the point cloud completion module.

[0051] (III) Point cloud completion module.

[0052] The point cloud completion module integrates an adaptive point cloud sampling unit, a bidirectional coordinate system transformation unit, a cross-module semantic-geometric feature coupling unit, a spatial feature coupling generation unit, and a bilateral symmetry constraint loss unit to reconstruct incomplete vehicle point clouds into dense and complete 3D vehicle models. Figure 6 Perform completion on the vehicle target point cloud shown, and obtain the following: Figure 8 The dense, complete point cloud shown.

[0053] (1) Engineering coordinate system adaptation and adaptive sampling.

[0054] The adaptive point cloud sampling unit unifies any number of input point clouds to the fixed number of input points required by the point cloud completion module (random sampling without duplication when the number exceeds the fixed number of input points, and cyclical supplementation when the number is less than the fixed number of input points). This allows the system to adapt to input point clouds of different devices and different degrees of occlusion without needing to be retrained for a specific LiDAR model, significantly improving the system's engineering versatility. The adaptive point cloud sampling unit translates the sampled point cloud to the centroid origin and scales it to a unit sphere, saving the centroid and scaling parameters for subsequent inverse normalization.

[0055] To address the coordinate system differences between LiDAR physical data and general deep learning frameworks, a bidirectional coordinate system transformation unit converts the Z-axis LiDAR coordinate system to the Y-axis coordinate system used in the point cloud completion module's training dataset before inference, eliminating the impact of coordinate system inconsistency on completion accuracy. After inference, the completed output point cloud is first inversely normalized to its original scale and position based on the centroid and scaling parameters stored by the adaptive point cloud sampling unit. Then, an inverse coordinate transformation is performed, and the rotation matrix stored by the point cloud preprocessing module is superimposed. The transpose of the output point cloud will completely restore it to the original coordinate system of the lidar.

[0056] (2) Cross-module semantic-geometric feature fusion.

[0057] like Figure 7 As shown, the encoder of the point cloud completion module performs PointNet-style (PointNet encoder) feature extraction on the input vehicle target point cloud (2048×3) and outputs a 1024-dimensional global geometric feature vector. The cross-module semantic-geometric feature fusion unit receives a high-dimensional semantic feature tensor from the semantic segmentation module. The feature dimension alignment transformation layer will Project to The same 1024-dimensional space is then used to align the features by linear fusion. and Linear fusion of feature spaces (element-by-element addition) yields a 1024-dimensional enhanced global feature vector. .

[0058] It includes semantic perception information extracted during the semantic segmentation stage and geometric reconstruction information from the encoder of the point cloud completion module, replacing the original... As the sole conditional input to the decoder, cross-module semantic-geometric feature fusion breaks through the limitations of the isolation between segmentation and completion in the traditional process. It makes full use of the fine-grained semantic information already available in the segmentation stage to assist in shape restoration, significantly improving the reconstruction accuracy of vehicle contours and details in extremely incomplete scenes, while eliminating feature waste across modules.

[0059] (3) Spatial feature coupling generation.

[0060] The spatial feature coupling generation unit adopts a two-stage decoding architecture consisting of a coarse reconstruction stage and a fine reconstruction stage. In the coarse reconstruction stage: As a condition, sparse coarse-reconstructed point clouds are generated directly through regression using a fully connected network. In the detailed reconstruction phase: with The local neighborhood features of each point are used as the query, with The linear projection of the points is used as keys and values. The attention weights of each local region to the global semantic information are calculated through a multi-head cross-modal self-attention mechanism, generating a 3D position offset vector for each local region. The offset vectors of each point are superimposed onto the spatial grid of the locally unfolded points of the coarsely reconstructed point cloud, outputting 16384 dense and complete point clouds. Compared to traditional decoding methods based on 2D mesh folding, the spatial feature coupling generation unit models the long-range geometric relationship between the missing parts and the existing parts through global-local attention interaction, which has a stronger ability to capture the 3D shape of fine local structures such as vehicle rearview mirrors and window corners.

[0061] (4) Bilateral symmetry constraint loss.

[0062] Vehicles are typical symmetrical objects. To address the issue of inconsistencies in the left-right structure of the completion result caused by severe occlusion on one side, the point cloud completion module introduces a bilateral symmetry constraint loss unit during the training phase, explicitly encoding the vehicle's symmetrical geometric priors into the optimization objective.

[0063] The steps include: S1. Determine the symmetry reference plane based on the target geometric center line or a preset geometric prior (which can be determined by training data annotation or point cloud principal component analysis).

[0064] S2. Dense, complete point clouds generated by spatial feature coupling generation units. Perform a mirror transformation along the symmetry reference plane to obtain a mirror point cloud. .

[0065] S3, Calculation and The chamfer distance between them is used as the symmetry constraint loss. .

[0066] Total training loss function :

[0067] ;

[0068] in, and These represent the chamfer distance loss in the coarse reconstruction stage and the fine reconstruction stage, respectively. and All are weighted hyperparameters. The strength of the symmetry constraint is controlled. This constraint, without changing the network structure during the inference stage, guides the network to generate a complete vehicle body reconstruction result with coordinated left and right structures in a unilateral occlusion scenario, based on the training objective.

[0069] (iv) Graphical interactive control module.

[0070] The graphical interactive control module is built on the PyQt5 framework. The interface adopts a left-right split design: the left side has three groups of function control areas, which correspond to the three processing stages of point cloud preprocessing, semantic segmentation and point cloud completion, respectively. Each group includes a data loading button, a parameter file selection button, an execution button and a result saving button; the right side has three independent real-time log panels, which display the processing progress, point count statistics and time consumption of each stage.

[0071] The graphical interactive control module adopts a multi-threaded architecture that separates the main thread and the worker child thread (QThread). The processing tasks are executed asynchronously in the worker child thread, and the log information is asynchronously passed to the main thread to update the interface through the signal-slot mechanism (pyqtSignal), ensuring that the operation interface remains responsive and does not block during deep learning inference.

[0072] To verify the technical effectiveness of the vehicle point cloud 3D reconstruction system provided in this embodiment, roadside lidar point cloud data collected in urban road forest occlusion scenarios were used as the test object. Under two test conditions with vehicle occlusion rates of 30%-50% and 50%-70%, the vehicle point cloud 3D reconstruction system of this embodiment was compared with a baseline completion model without semantic-geometric feature coupling mechanism and an ablation model without bilateral symmetry constraint loss.

[0073] Experimental results show that: (1) In extreme environments with occlusion rates of 50%-70%, the semantic-geometric feature coupling mechanism proposed in this embodiment outperforms the baseline completion model without fused semantic features. and The chamfer distance between the two is reduced by about 15%-20%, the vehicle target segmentation recall rate is improved by more than 12%, and the local reconstruction accuracy of key body features (such as rearview mirrors and window corners) is significantly improved.

[0074] (2) After introducing bilateral symmetry constraint loss in this embodiment, the coordination of the left and right structures of the vehicle body is significantly improved in the scenario of severe occlusion on one side, and the symmetry deviation index is reduced by about 25%-30% compared with the ablation model without constraint.

[0075] In addition, the other modules of the vehicle point cloud 3D reconstruction system in this embodiment were ablation verified, and the results were as follows: (3) Compared with single statistical outlier filtering, the noise residual rate of the dual collaborative denoising in this embodiment is reduced by about 40%; the adaptive pose normalization based on ground normal vector in this embodiment reduces the alignment error between the completed output point cloud and the real geographic coordinate system to the sub-degree level.

[0076] (4) Compared with the traditional decoder based on two-dimensional mesh folding, the spatial feature coupling generation unit of this embodiment improves the F1-Score index of fine local structures such as rearview mirrors and window corners by about 10%-15%.

[0077] The above results fully demonstrate that the vehicle point cloud 3D reconstruction system of this embodiment is significantly superior to the existing technical solutions in terms of vehicle 3D reconstruction accuracy, structural integrity, and engineering adaptability in forest occlusion scenarios.

[0078] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A 3D reconstruction system for vehicle point clouds in forest-shaded scenes, characterized in that, It includes a point cloud preprocessing module, a semantic segmentation module, a point cloud completion module, and a graphical interactive control module; The raw 3D point cloud acquired by the lidar is processed by the point cloud preprocessing module to perform dual collaborative denoising and ground adaptive pose normalization, resulting in a clean and aligned point cloud. The clean and aligned point cloud is then processed by the semantic segmentation module to perform point-by-point binary classification, resulting in a vehicle target point cloud and a high-dimensional semantic feature tensor. The vehicle target point cloud and the high-dimensional semantic feature tensor are then fed into the point cloud completion module, where cross-module feature fusion, symmetric constraint training, and spatial feature coupling are used to generate decoding, resulting in a complete 3D vehicle point cloud. The graphical interactive control module adopts a multi-threaded architecture that separates the main thread and the Worker sub-thread. The processing tasks are executed asynchronously in the Worker sub-thread, and the log information is asynchronously transmitted to the main thread to update the interface through the signal-slot mechanism, ensuring that the operation interface remains responsive during deep learning inference. The point cloud preprocessing module includes a statistical outlier filtering unit, a radius outlier filtering unit, and a ground normal vector calculation unit. The statistical outlier filtering unit targets the sparse discrete noise generated by multipath reflection from roadside lidar. For each point, it calculates the mean distance between the point and several nearest neighbor points. Using a linear combination of the global mean and standard deviation as a statistical threshold, points exceeding the threshold are identified as outlier noise points and removed. The radius outlier filtering unit targets isolated small scattered points generated by reflection from vegetation leaves. With a fixed search radius as a constraint, it identifies and removes points whose number of points in the neighborhood is lower than a preset lower limit as isolated noise points. The ground normal vector calculation unit uses the RANSAC algorithm to fit the local ground point set under the target vehicle after double filtering of the point cloud, and extracts the local ground normal vector. ; calculation will Rotation matrix required to rotate to the positive direction of the global vertical axis To achieve adaptive pose normalization based on a local ground reference; and to save the rotation matrix. After the point cloud is completed, apply a function to the output point cloud. The transpose completes the inverse transformation and restoration.

2. The under-forest occluded scene vehicle point cloud 3D reconstruction system according to claim 1, characterized in that, The semantic segmentation module uses a 3D point cloud segmentation network based on a sparse convolutional UNet backbone network as a feature extractor, and completes initialization by dynamically constructing the model structure and loading pre-trained parameters. The sparse convolutional UNet backbone network adopts an encoding and decoding architecture. In the encoding stage, it downsamples layer by layer to extract multi-scale 3D point cloud features, and in the decoding stage, it upsamples layer by layer to restore the spatial resolution of the point cloud. Skip connections preserve multi-scale contextual information and finally output the binary classification probability distribution of each point.

3. The vehicle point cloud three-dimensional reconstruction system for under-forest shelter scene according to claim 2, characterized in that, The semantic segmentation module extracts the output of the deepest layer of a sparse convolution UNet backbone network encoder as a high-dimensional semantic feature tensor while performing gradient-free forward inference ; It includes a cross-module semantic-geometric feature fusion unit that comprehensively expresses the local geometric structure and global semantic context of point clouds and passes them to the point cloud completion module through the inter-module interface.

4. The vehicle point cloud three-dimensional reconstruction system of the under-forest shelter scene according to claim 3, characterized in that, The semantic segmentation module is trained using a weighted combination of cross-entropy loss with class weights and Lowaz loss.

5. The under-forest occluded scene vehicle point cloud 3D reconstruction system according to claim 3, characterized in that, The point cloud completion module includes an adaptive point cloud sampling unit, a coordinate system bidirectional transformation unit, a cross-module semantic-geometric feature coupling unit, a spatial feature coupling generation unit, and a bilateral symmetry constraint loss unit; The adaptive point cloud sampling unit unifies any number of input point clouds to the fixed number of input points required by the point cloud completion module. When the number of input points exceeds the fixed number, random sampling without repetition is performed. When the number of input points is less than the fixed number, cyclical and repeated supplementation is performed. The adaptive point cloud sampling unit translates the sampled point cloud to the centroid origin and scales it to a unit sphere, saving the centroid and scaling parameters. Before inference, the coordinate system bidirectional transformation unit converts the Z-axis upward lidar coordinate system to the Y-axis upward coordinate system used by the point cloud completion module training dataset. After inference, it first performs inverse normalization on the completed output point cloud to restore it to its original scale and position based on the centroid and scaling parameters saved by the adaptive point cloud sampling unit, and then performs inverse coordinate transformation and superimposes the rotation matrix saved by the point cloud preprocessing module. The transpose of the output point cloud will completely restore it to the original coordinate system of the lidar.

6. The under-forest occluded scene vehicle point cloud 3D reconstruction system according to claim 5, characterized in that, The encoder of the point cloud completion module performs PointNet style feature extraction on an input fixed point number sampling point cloud, and outputs a global geometric feature vector of a preset dimension ; The cross-module semantic-geometric feature fusion unit receives high-dimensional semantic feature tensors from the semantic segmentation module. The feature dimension alignment transformation layer will Project to In the same dimensional space, the aligned features are then linearly fused together. and Element-wise addition yields the enhanced global feature vector. .

7. The vehicle point cloud three-dimensional reconstruction system of an under-forest sheltering scene according to claim 6, characterized in that, The spatial feature coupling generation unit adopts a two-stage decoding architecture consisting of a coarse reconstruction stage and a fine reconstruction stage. In the coarse reconstruction stage: generate sparse coarse reconstruction point cloud by direct regression through fully connected network with the condition ; In the detailed reconstruction phase: with The local neighborhood features of each point are used as the query, with The linear projection is used as keys and values. The attention weights of each local region to the global semantic information are calculated through a multi-head cross-modal self-attention mechanism, generating a 3D position offset vector for each local region. The offset vectors of each point are superimposed onto the spatial grid of the locally unfolded points of the coarsely reconstructed point cloud to output a dense and complete point cloud. .

8. The vehicle point cloud three-dimensional reconstruction system of an under-forest sheltering scene according to claim 7, characterized in that, The bilateral symmetry constraint loss unit explicitly encodes the vehicle's symmetric geometry prior into the optimization objective, including the following steps: S1. Determine the symmetry reference plane based on the target geometric centerline or a preset geometric prior; S2, the dense complete point cloud generated by the spatial feature coupling generation unit performing a mirror transformation along the symmetry reference surface to obtain a mirror point cloud ; S3, compute with a chamfer distance between the chamfer distance as a symmetry constraint loss ; Total training loss function : ； in, and These represent the chamfer distance loss in the coarse reconstruction stage and the fine reconstruction stage, respectively. and All are weighted hyperparameters. Control the strength of symmetry constraints.

Images