Laser radar point cloud data semantic segmentation method for severe weather

Abstract

The invention discloses a laser radar point cloud data semantic segmentation method for severe weather, and the method employs a double-branch design, a space geometric structure branch is responsible for capturing a space geometric shape and obtaining the features of the space geometric structure, a reflection intensity branch focuses on the reflection intensity of a laser radar signal, and obtaining the features of the reflection intensity. And feature fusion is carried out under different scales through a robust multi-stage feature cooperation module, so that the influence of severe weather on the segmentation performance is reduced. Performing voxelization processing on the spatial geometric structure features, and extracting the spatial geometric structure features by using a sparse three-dimensional convolutional network; reflection intensity information is converted into a two-dimensional image through spherical projection and is extracted through depth separable convolution. The fused space geometric mechanism features and reflection intensity features are further optimized through a local and global feature fusion mechanism, and finally an accurate semantic segmentation result is generated through a three-dimensional voxel decoding module. The method can be generalized to the target domain after source domain training, effectively deals with domain offset between standard conditions and severe conditions, and shows excellent performance under various severe weather conditions.

Description

Technical Field

[0001] The present invention belongs to the technical field of semantic segmentation of point cloud data, and more specifically, relates to a semantic segmentation method of laser radar point cloud data for severe weather. Background Art

[0002] Current research on LiDAR point cloud data is usually conducted under idealized conditions: standard LiDAR point cloud datasets are used for training and testing. These LiDAR point cloud datasets usually exclude various interference factors that may be encountered in the real world, such as severe weather conditions. The idealized LiDAR point cloud data semantic segmentation system obtained in this way performs well under standard conditions, but is often difficult to generalize under severe weather conditions. The reason is that the spatial geometric structure features and reflection intensity features will be affected by the degradation caused by weather, especially the poor robustness and adaptability of severe weather conditions such as fog, rain, and snow. The common unified feature processing method cannot effectively deal with the domain offset between standard conditions and severe conditions. Therefore, there is an urgent need for a domain generalization LiDAR point cloud data semantic segmentation method that can effectively separate and fuse geometric structure features and reflection intensity features, while reducing their mutual degradation effects under severe weather conditions. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a semantic segmentation method for lidar point cloud data in severe weather, which can be generalized to the target domain after training in the source domain and can still efficiently complete the semantic segmentation task of lidar point cloud data under severe weather conditions.

[0004] To achieve the above-mentioned object of the invention, the semantic segmentation method of laser radar point cloud data for severe weather of the present invention is characterized by comprising the following steps:

[0005] (1) Spatial geometric structure feature extraction

[0006] First, the reflection intensity information is excluded from the acquired LiDAR point cloud data to obtain point cloud data containing only geometric coordinates, namely, geometric point cloud, and the geometric point cloud is discretized into a regular voxel grid. Then, the voxel grid is processed using an encoder network composed of multiple cascaded sparse 3D convolutional blocks to obtain spatial resolution and geometric context information of different scales layer by layer. Finally, the spatial resolution and geometric context information of different scales are aggregated to obtain spatial geometric structure features in the form of sparse representation.

[0007] (2) Reflection intensity feature extraction

[0008] Firstly, the acquired LiDAR point cloud data is converted into a two-dimensional view image using spherical projection, where the reflection intensity information of each point is used as the input feature. Secondly, the reflection intensity information is processed using an encoder network composed of two-dimensional depth-separable convolutional blocks, and the reflection intensity features are composed of features of different scales layer by layer.

[0009] (3) Robust multi-level feature collaboration

[0010] 3.1) Calculate the complementary information constraint loss L for enhancing feature complementarity and robustness cic :

[0011] L cic =KL(p(m geo ∣f geo )||r(m geo ))+KL(p(m ref ∣f ref )‖r(m ref ))-KL(p(m geo ∣f geo )‖p(m ref ∣f ref ))-KL(p(m ref ∣f ref )‖p(m geo ∣f geo ))

[0012] Among them, KL(·‖·) is the maximum inter-class divergence, f geo is the feature vector extracted from the spatial geometric structure features, f ref is the feature vector extracted from the reflection intensity feature, m geo Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo By adding additive noise The generated spatial geometry structure enhances the features, m ref Based on the feature vector f ref The calculated mean μ ref and standard deviation σ ref , by adding additive noise The resulting reflection intensity enhancement feature:

[0013] m geo =μ geo +ε·σ geo

[0014] m ref =μ ref +ε·σ ref

[0015] r(m geo ) is used to generate spatial geometric structure to enhance the feature m geo The formula is converted to a standard Gaussian prior distribution with additive noise, r(m ref ) is used to generate the reflection intensity enhancement feature m ref The formula is converted to a standard Gaussian prior distribution with additive noise, p(m geo ∣f geo ) is the conditional distribution of the geometric structure characteristics of the metric space, p(m ref ∣f ref ) is the conditional distribution of the reflection intensity feature;

[0016] 3.2) Local level feature fusion

[0017] Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo and based on the eigenvector f ref The calculated mean μ ref and standard deviation σ ref Perform dynamic fusion to obtain the local fusion feature F local :

[0018] F local =α·μ geo +(1-α)·μ ref

[0019] Among them, the fusion weight α is based on the standard deviation σ geo and standard deviation σ ref Dynamic calculation:

[0020]

[0021] in, and The standard deviation σ geo and standard deviation σ ref The mean in the channel dimension;

[0022] 3.3) Global level feature fusion

[0023] First, a set of learnable global query values ​​Q is introduced to enhance the spatial geometric structure feature m geo As the cross attention mechanism key value K, value V and global query value Q are aggregated to obtain a temporary global feature value, and then the temporary global feature value is used as the cross attention mechanism key value K, value V and reflection intensity enhancement feature m ref Further aggregation obtains the global fusion feature F global ;

[0024] 3.4) Splice into joint representation F

[0025] The local fusion feature F local And the global fusion feature F global The joint representation F is formed by splicing;

[0026] (4) Optimize the network

[0027] The joint representation F is sent to the 3D voxel decoding module to obtain the semantic segmentation result. The network is optimized by combining the cross entropy loss and the complementary information constraint loss. The loss L of the optimized network is:

[0028] L=L CE +βL cic

[0029] Among them, L CE is the cross entropy loss obtained by the semantic segmentation result obtained by the 3D voxel decoding module and the label calculation, and β is the combination coefficient;

[0030] (5) Semantic segmentation of lidar point cloud data

[0031] According to steps (1) to (3), the joint representation F is obtained and sent to the 3D voxel decoding module to obtain the semantic segmentation result.

[0032] The object of the present invention is achieved in this way.

[0033] The semantic segmentation method of LiDAR point cloud data for severe weather in the present invention adopts a dual-branch design. The spatial geometry branch is responsible for capturing the spatial geometry and obtaining the spatial geometry features. The reflection intensity branch focuses on the reflection intensity of the LiDAR signal and obtains the reflection intensity features. The robust multi-level feature collaboration module then performs feature fusion at different scales to reduce the impact of severe weather on segmentation performance. The spatial geometry features are processed by voxelization and extracted using a sparse three-dimensional convolutional network; the reflection intensity information is converted into a two-dimensional image through spherical projection and extracted through deep separable convolution. The fused spatial geometry features and reflection intensity features are further optimized through local and global feature fusion mechanisms, and finally accurate semantic segmentation results are generated through a three-dimensional voxel decoding module. The present invention can generalize to the target domain after training in the source domain, effectively deal with the domain shift between standard conditions and severe conditions, and show excellent performance under various severe weather conditions.

[0034] The present invention adopts a dual-branch design to independently encode geometric and reflective information, avoiding interference between them and improving the effectiveness of features; through a multi-level feature collaboration mechanism, feature fusion is performed at both the local and global levels, significantly improving the robustness and complementarity of fused features; the introduced information constraint mechanism reduces the redundancy between geometric and reflective features. Through this innovative design, the present invention can better learn from source domain data and effectively migrate to the target domain, thereby maintaining excellent performance under different weather conditions and solving the problem of LiDAR semantic segmentation under adverse weather conditions.

[0035] Compared with the prior art, the present invention significantly improves the performance of the model in adverse weather conditions through innovative design when processing lidar data. Many existing methods process geometric structure information and reflection intensity information in a unified manner, often ignoring their degradation characteristics under different weather conditions, resulting in insufficient robustness and generalization of the model in bad weather. In contrast, the present invention, through a dual-branch design, a multi-level feature collaboration mechanism, and an information constraint strategy, demonstrates stronger robustness, higher generalization ability, and more accurate semantic segmentation performance when processing lidar data under adverse weather conditions, providing a more reliable and effective solution for lidar semantic segmentation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flowchart of a specific implementation method of the semantic segmentation method of laser radar point cloud data for severe weather of the present invention;

[0037] Figure 2 It is an overall network structure diagram of the semantic segmentation method of laser radar point cloud data for severe weather of the present invention;

[0038] Figure 3 is a visualization of the intensity of reflections under different weather conditions;

[0039] Figure 4 This is a comparison chart of segmentation effects. DETAILED DESCRIPTION

[0040] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0041] The present invention processes the spatial geometric structure features and reflection intensity features in three-dimensional point cloud data separately to solve the problem of LiDAR semantic segmentation under severe weather conditions. The spatial geometric structure branch focuses on capturing the geometric form of spatial points, and the reflection intensity branch focuses on the return intensity of the LiDAR signal. At the same time, effective fusion is achieved through a robust multi-level feature collaboration module to reduce the impact of severe weather on segmentation performance. Among them, after the spatial geometric information is voxelized through the point cloud, a sparse three-dimensional convolutional network is used to extract the spatial geometric structure features; the reflection intensity information branch converts the spherical projection into a two-dimensional image, and uses a deeply separable two-dimensional convolutional network to extract the reflection features. The extracted spatial geometric structure and reflection features are efficiently fused through a robust multi-level feature collaboration module, so that they can be generalized to the target domain after training in the source domain, and can still efficiently complete the LiDAR semantic segmentation task under severe weather conditions.

[0042] Figure 1 It is a flow chart of a specific implementation method of the semantic segmentation method of lidar point cloud data for severe weather of the present invention.

[0043] In this embodiment, if Figure 1 As shown, the semantic segmentation method of laser radar point cloud data for severe weather of the present invention comprises the following steps:

[0044] Step S1: Spatial geometric structure feature extraction

[0045] Firstly, the reflection intensity information is excluded from the acquired lidar point cloud data to obtain point cloud data containing only geometric coordinates, namely the geometric point cloud, and the geometric point cloud is discretized into a regular voxel grid. Then, the voxel grid is processed using an encoder network composed of multiple cascaded sparse three-dimensional convolutional blocks to obtain spatial resolution and geometric context information of different scales layer by layer. Finally, the spatial resolution and geometric context information of different scales are aggregated to obtain the spatial geometric structure features in the form of sparse representation.

[0046] In this embodiment, if Figure 2 As shown, the spatial geometric structure feature extraction is completed by the geometric feature extraction module. The encoder network composed of the sparse three-dimensional convolutional blocks is MinkNet18 or MinkNet32.

[0047] Step S2: Reflection intensity feature extraction

[0048] Firstly, spherical projection is used to convert the acquired lidar point cloud data into a two-dimensional view image, in which only the reflection intensity information of each point is used as the input feature. Secondly, an encoder network composed of two-dimensional depthwise separable convolutional blocks is used to process the reflection intensity information, and features of different scales are obtained layer by layer to form reflection intensity features.

[0049] In this embodiment, if Figure 2 As shown, the reflection intensity feature extraction is completed by the reflection feature extraction module. The encoder network composed of the two-dimensional depth separable convolutional blocks is MobileNetV2.

[0050] Step S3: Robust multi-level feature collaboration

[0051] The robust multi-level feature collaboration includes a complementary information constraint loss for calculating enhanced feature complementarity and robustness, a local level fusion combining geometric and reflective information based on feature reliability dynamic weighting, and a global level fusion of reflective features through a two-stage cross-attention mechanism. Figure 2 As shown, this step is completed in the robust multi-level feature collaboration, specifically:

[0052] Step S3.1: Calculate the complementary information constraint loss L for enhancing feature complementarity and robustness cic :

[0053] L cic =KL(p(m geo ∣f geo )||r(m geo ))+KL(p(m ref ∣f ref )||r(m ref ))-KL(p(m geo ∣f geo )||p(m ref ∣f ref ))-KL(p(m ref ∣f ref )||p(m geo ∣f geo ))

[0054] Among them, KL(·|·) is the maximum inter-class divergence, f geo is the feature vector extracted from the spatial geometric structure features, f ref is the feature vector extracted from the reflection intensity feature, m geo Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo By adding additive noise The generated spatial geometry structure enhances the features, m ref Based on the feature vector f ref The calculated mean μ ref and standard deviation σ ref , by adding additive noise The resulting reflection intensity enhancement feature:

[0055] m geo=μ geo +ε·σ geo

[0056] m ref =μ ref +ε·σ ref

[0057] r(m geo ) is used to generate spatial geometric structure to enhance the feature m geo The formula is converted to a standard Gaussian prior distribution with additive noise, r(m ref ) is used to generate the reflection intensity enhancement feature m ref The formula is converted to a standard Gaussian prior distribution with additive noise, p(m geo ∣f geo ) is the conditional distribution of the geometric structure characteristics of the metric space, p(m ref ∣f ref ) is the conditional distribution of the reflection intensity feature.

[0058] In the present invention, the first term KL(p(m geo ∣f geo )‖r(m geo )) The conditional distribution of the metric geometric features p(m geo ∣f geo ) and the standard Gaussian prior distribution r(m geo )(converted to a Gaussian distribution with additive noise through the above reflection intensity enhancement feature formula). Minimizing this term means letting the spatial geometric structure enhance the feature m geo Close to the standard Gaussian prior distribution with additive noise r(m geo ), suppressing overfitting while improving the generalization ability of features, while reducing the spatial geometric structure to enhance the feature m geo The feature vector f extracted from the spatial geometric structure features geo conditional dependency.

[0059] The second term KL(p(m ref ∣f ref )||r(m ref ))Similar to the first term, it is used to measure the conditional distribution p(m) of the reflection intensity feature ref ∣f ref ) and the standard Gaussian prior distribution r(m ref ), minimizing this term means letting the reflection strength enhance feature m ref Close to the standard Gaussian prior distribution with additive noise r(m ref ), while suppressing overfitting and improving the generalization ability of features, while reducing the reflection intensity and enhancing the feature m ref The feature vector f extracted from the reflection intensity featuregeo conditional dependency.

[0060] The negative KL divergence in the third and fourth terms is used to increase p(m geo ∣f geo ) and p(m ref ∣f ref ) and reduce the similarity between the two. Minimizing the negative value of this binomial (i.e. maximizing the KL divergence) means encouraging the distributions between them to be as different as possible, thereby reducing information redundancy and improving the complementarity of features.

[0061] The Kullback-Leibler (KL) divergence, or the maximum inter-class divergence, is a measure of the difference between two probability distributions P and Q and is defined as follows:

[0062]

[0063] KL divergence is asymmetric, that is, usually KL(P||Q)≠KL(Q||P), that is, it has a directional dependence on the deviation between the two distributions. Due to the asymmetry of KL divergence, using KL divergence in two directions at the same time can separate m more evenly. geo With m ref , ensuring their complementarity.

[0064] Step S3.2: Local level feature fusion

[0065] Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo and based on the eigenvector f ref The calculated mean μ ref and standard deviation σ ref Perform dynamic fusion to obtain the local fusion feature F local :

[0066] F local =α·μ geo +(1-α)·μ ref

[0067] Among them, the fusion weight α is based on the standard deviation σ geo and standard deviation σ ref Dynamic calculation:

[0068]

[0069] in, and The standard deviation σ geo and standard deviation σ ref The mean over the channel dimension.

[0070] Step S3.3: Global level feature fusion

[0071] First, a set of learnable global query values ​​Q is introduced to enhance the spatial geometric structure feature m geo As the cross attention mechanism key value K, value V and global query value Q are aggregated to obtain a temporary global feature value, and then the temporary global feature value is used as the cross attention mechanism key value K, value V and reflection intensity enhancement feature m ref Further aggregation obtains the global fusion feature F global :

[0072] F global =CA(M geo ,CA(Q,M ref ))

[0073] Among them, CA(·,·) represents the cross attention operation, and the latter item in the brackets of CA(·,·) is aggregated with the former item as the query value Q as the cross attention mechanism key K and value V.

[0074] Step S3.4: Concatenate into a joint representation F

[0075] The local fusion feature F local And the global fusion feature F global The concatenated joint representation F.

[0076] Step S4: Optimize the network

[0077] In this embodiment, if Figure 2 As shown, the joint representation F is sent to the 3D voxel decoding module to obtain the semantic segmentation result. The network is optimized by combining the cross entropy loss and the complementary information constraint loss. The loss L of the optimized network is:

[0078] L=L CE +βL cic

[0079] Among them, L CE is the cross entropy loss obtained by the semantic segmentation result obtained by the 3D voxel decoding module and the label calculation, and β is the combination coefficient.

[0080] In this embodiment, the 3D voxel decoding module is composed of a MinkNet32 network corresponding to the sparse 3D convolutional network structure and a linear layer in series, and the output result is a 19-dimensional vector, where the index corresponding to the maximum value is the final result of the network prediction.

[0081] The optimization network is to update the network weights. In this embodiment, the public data set SemanticKITTI is selected. It contains 64 lidar sequences and annotates more than 19 semantic categories. According to the official protocol, we use sequences 00-07 and 09-10 as training sets and sequence 08 as validation sets. We apply standard data enhancement, including random dropout, rotation, flipping, scaling, etc. We use the momentum SGD optimizer with a momentum of 0.9, a weight decay of 0.0001, and a batch size of 6, which is suitable for SemanticKITTI. The learning rate is initially set to 0.24 and is adjusted using the OneCycleLR strategy for a total of 50 cycles.

[0082] Step S5: Semantic segmentation of lidar point cloud data

[0083] According to steps S1 to S3, the joint representation F is obtained and sent to the 3D voxel decoding module to obtain the semantic segmentation result.

[0084] Figure 2 It is an overall network structure diagram of the semantic segmentation method of lidar point cloud data for severe weather according to the present invention.

[0085] In this embodiment, the present invention is Figure 2 The overall network structure shown is implemented.

[0086] The present invention is based on a dual-branch network architecture, and realizes the semantic segmentation of lidar point cloud data by constructing a lidar segmentation network including a geometric feature extraction module, a reflection feature extraction module, a robust multi-level feature collaboration module and a three-dimensional voxel decoding module in the offline stage. After the collected lidar signal is voxelized through the point cloud, the geometric structure features are extracted by the geometric feature extraction module; the reflection intensity information is converted into a two-dimensional image through spherical projection, and the reflection intensity features are extracted by the reflection feature extraction module. The extracted spatial geometric structure features and reflection intensity features are efficiently fused through a robust multi-level feature collaboration module to perform local fusion and global fusion. These two fusion methods work together to process feature information at different scales respectively, which improves the performance of the model in complex environments, especially under adverse weather conditions. Among them, local fusion focuses on the local area of ​​the feature, and dynamically fuses the geometric and reflection features by calculating the standard deviation weight; global fusion uses the cross-attention mechanism to extract useful information from the global context and effectively fuse it into the geometric features. The fused local and global features are spliced, and the semantic segmentation results are generated through the 3D voxel decoding module. Finally, the network is optimized by combining the cross entropy loss and complementary information constraint loss, so that it can be generalized to the target domain after training in the source domain, and can still efficiently complete the lidar semantic segmentation task under severe weather conditions.

[0087] LiDAR semantic segmentation network test performance:

[0088] The public datasets SemanticSTF and SemanticKITTI-C are used. SemanticSTF is a severe weather lidar segmentation dataset that contains point-by-point annotations of 21 semantic categories. The dataset includes four common severe weather conditions: dense fog, light fog, snow, and rain. Figure 3 As shown. We follow the official protocol and use its validation set (containing 19 semantic categories) for testing. SemanticKITTI-C is a corrupted LiDAR segmentation dataset. Eight types of corruptions are imposed on the validation set of SemanticKITTI, divided into three severity levels. The experiment uses mean intersection over union (mIoU) as the evaluation metric. The generalization performance of the present invention on the SemanticSTF dataset is significantly improved. Compared with the widely adopted baseline MinkNet, the present invention achieves a significant improvement of +18.1 in mIoU. In addition, the present invention exceeds the state-of-the-art method RDA in mIoU by +3.0. The present invention performs best in all tested adverse weather conditions on the SemanticKITTI-C dataset, reflecting its strong generalization ability from good weather to adverse conditions. The mIoU is improved by +2.1 in dense fog, +2.6 in light fog, +0.5 in rainy days, and +1.0 in snowy days, compared with the state-of-the-art methods. In comparison with the MinkNet baseline, our method achieves significant improvements of +8.6mIoU in dense fog, +14.1mIoU in light fog, +9.7mIoU in rainy days, and +12.7mIoU in snowy days. In addition to performing well in adverse weather conditions, our method also achieves excellent results on most semantic categories, such as Figure 4 As shown in Figure 2. We achieve more than +10mIoU improvement in 13 categories compared to the baseline. Notably, our method achieves more than +5 improvement in mIoU over the state-of-the-art methods in categories such as trucks, other vehicles, motorcyclists, roads, parking, and terrain. Meanwhile, the previous state-of-the-art methods rely on reinforcement learning for data augmentation, and training requires four A6000 GPUs. In contrast, our method achieves superior performance with a simpler framework, does not require such complex augmentation techniques, and can be efficiently trained on a single RTX4090 GPU.

[0089] In summary, the proposed method surpasses previous methods and establishes a new state-of-the-art performance benchmark.

[0090] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A semantic segmentation method for lidar point cloud data in severe weather, characterized in that: The following steps are involved: (1) Spatial geometric structure feature extraction First, the reflection intensity information is excluded from the acquired LiDAR point cloud data to obtain point cloud data containing only geometric coordinates, namely, geometric point cloud, and the geometric point cloud is discretized into a regular voxel grid. Then, the voxel grid is processed using an encoder network composed of multiple cascaded sparse 3D convolutional blocks to obtain spatial resolution and geometric context information of different scales layer by layer. Finally, the spatial resolution and geometric context information of different scales are aggregated to obtain spatial geometric structure features in the form of sparse representation. (2) Reflection intensity feature extraction Firstly, the acquired LiDAR point cloud data is converted into a two-dimensional view image using spherical projection, where the reflection intensity information of each point is used as the input feature. Secondly, the reflection intensity information is processed using an encoder network composed of two-dimensional depth-separable convolutional blocks, and the reflection intensity features are composed of features of different scales layer by layer. (3) Robust multi-level feature collaboration 3.1) Calculate the complementary information constraint loss L for enhancing feature complementarity and robustness cic : L cic =KL(p(m geo ∣f geo )||r(m geo )) +KL(p(m ref ∣f ref )||r(m ref )) -KL(p(m geo ∣f geo )||p(m ref ∣f ref )) -KL(p(m ref ∣f ref )||p(m geo ∣f geo )) Among them, KL(·||·) is the maximum inter-class divergence, f geo is the feature vector extracted from the spatial geometric structure features, f ref is the feature vector extracted from the reflection intensity feature, m geo Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo By adding additive noise The generated spatial geometry structure enhances the features, m ref Based on the feature vector f ref The calculated mean μ ref and standard deviation σ ref , by adding additive noise The resulting reflection intensity enhancement feature: m geo =μ geo +e·s geo m ref =μ ref +e·s ref r(m geo ) is used to generate spatial geometric structure to enhance the feature m geo The formula is converted to a standard Gaussian prior distribution with additive noise, r(m ref ) is used to generate the reflection intensity enhancement feature m ref The formula is converted to a standard Gaussian prior distribution with additive noise, p(m geo ∣f geo ) is the conditional distribution of the geometric structure characteristics of the metric space, p(m ref ∣f ref ) is the conditional distribution of the reflection intensity feature; 3.2) Local level feature fusion Based on the feature vector f geo The calculated mean μ geo and standard deviation σ geo and based on the eigenvector f ref The calculated mean μ ref and standard deviation σ ref Perform dynamic fusion to obtain the local fusion feature F local : F local =a·m geo +(1-a)·m ref Among them, the fusion weight α is based on the standard deviation σ geo and standard deviation σ ref Dynamic calculation: in, and The standard deviation σ geo and standard deviation σ ref The mean in the channel dimension; 3.3) Global level feature fusion First, a set of learnable global query values ​​Q is introduced to enhance the spatial geometric structure feature m geo As the cross attention mechanism key value K, value V and global query value Q are aggregated to obtain a temporary global feature value, and then the temporary global feature value is used as the cross attention mechanism key value K, value V and reflection intensity enhancement feature m ref Further aggregation obtains the global fusion feature F global ; 3.4) Splice into joint representation F The local fusion feature F local And the global fusion feature F global The joint representation F is formed by splicing; (4) Optimize the network The joint representation F is sent to the 3D voxel decoding module to obtain the semantic segmentation result. The network is optimized by combining the cross entropy loss and the complementary information constraint loss. The loss L of the optimized network is: L=L CE +βL cic Among them, L CE is the cross entropy loss obtained by the semantic segmentation result obtained by the 3D voxel decoding module and the label calculation, and β is the combination coefficient; (5) Semantic segmentation of lidar point cloud data According to steps (1) to (3), the joint representation F is obtained and sent to the 3D voxel decoding module to obtain the semantic segmentation result.

2. The method for semantic segmentation of laser radar point cloud data for severe weather according to claim 1, characterized in that: The 3D voxel decoding module is composed of a MinkNet32 network and a linear layer connected in series, which is opposite to the sparse 3D convolutional network structure. The output result is a 19-dimensional vector, in which the index corresponding to the maximum value is the final result of the network prediction.

Images