A degradation reliability modeling method for severe weather laser radar point cloud semantic segmentation

Abstract

The application discloses a degradation reliability modeling method for severe weather laser radar point cloud semantic segmentation, which comprises the following steps: firstly, a point-level observation degradation prior is constructed according to the distance of a point to a sensor, echo intensity and local density, and degradation enhancement is performed on a source domain point cloud in a training stage; then, a point-level degradation reliability encoder is used to estimate the degradation probability and degradation embedding of each point, and the degradation embedding is injected into input point features in a residual manner; in a coding high layer, a degradation perception modulation module is used to generate a scaling factor and a bias factor according to degradation information, and the semantic features are adaptively modulated; in a decoding end, a reliability perception feature correction module is used to combine the degradation probability, semantic context and local residual context to perform conservative residual correction on a low reliability area; and finally, the semantic class prediction of each point is output. The application can improve the cross-domain generalization capability and segmentation stability of a laser radar point cloud semantic segmentation model under rain, fog and snow severe weather without introducing additional sensor modalities and target domain labels.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing technology for lidar point clouds, specifically to a degradation reliability modeling method and apparatus for semantic segmentation of lidar point clouds in adverse weather conditions. Background Technology

[0002] LiDAR can directly acquire information such as the distance, geometry, and echo intensity of targets and scenes in three-dimensional space, and is widely used in autonomous driving, unmanned equipment, intelligent robots, road inspection, and complex environment perception. LiDAR point cloud semantic segmentation aims to assign semantic category labels such as roads, vehicles, buildings, vegetation, and pedestrians to each point in the point cloud, which is an important foundation for three-dimensional environment understanding and autonomous decision-making.

[0003] In recent years, point cloud semantic segmentation methods based on sparse convolution, point cloud attention models, and state-space models have achieved good segmentation results on normal weather and standard datasets. These methods typically extract local geometric features and contextual semantic features through an encoder-decoder network, and then output the semantic category of each point through a classification head. In scenarios with stable weather conditions and good point cloud observation quality, these methods can meet the needs of conventional 3D perception tasks.

[0004] However, in real-world open environments, lidar often faces adverse weather conditions such as rain, fog, and snow. Severe weather can cause point cloud observations to exhibit issues such as missing points, attenuated echo intensity, decreased local density, ranging disturbances, and scattering noise, leading to significant differences in observed distribution between point clouds in normal and adverse weather conditions. In such cases, semantic segmentation models trained on normal weather point clouds are prone to problems such as category confusion, blurred boundaries, missed detection of small targets, and structural misjudgments, impacting subsequent scene understanding, target recognition, path planning, and safety decisions.

[0005] To address the robustness issue of point cloud semantic segmentation under severe weather conditions, existing methods typically employ techniques such as data augmentation, domain generalization, unsupervised domain adaptation, or test-time adaptation. Data augmentation methods expand training samples by randomly dropping points, adding noise perturbations, or geometric transformations; domain generalization methods attempt to learn more stable point cloud representations during the source domain training phase; unsupervised domain adaptation methods utilize unlabeled samples from the target domain to reduce inter-domain differences; and test-time adaptation methods adjust the model based on test samples during the deployment phase. While these methods can mitigate cross-weather distribution shifts to some extent, most still treat the impact of severe weather as an overall domain shift or random perturbation, lacking explicit modeling of the degradation degree and reliability of individual point observations.

[0006] Based on lidar point cloud observations, point cloud degradation caused by severe weather is correlated with observable factors such as the distance from the point to the sensor, the echo intensity of the point, and the point cloud density in the local area where the point is located. For example, distant points are more likely to exhibit sparseness and incomplete structure, weak echo points usually have lower observation reliability, and local density decreases often reflect missing points, occlusion, or insufficient sampling. Existing point cloud semantic segmentation networks typically lack continuous utilization of this type of point-level degradation information during input feature encoding, high-level semantic modeling, and decoding recovery, making it difficult to distinguish between relatively reliable regions and low-reliability regions heavily affected by degradation within the network.

[0007] Therefore, it is necessary to provide a degradation reliability modeling method for lidar point cloud semantic segmentation in severe weather, so that the model can use observable information of point cloud such as distance, echo intensity and local density to construct a point-level degradation reliability representation, and use this representation for point cloud degradation enhancement, input feature encoding, high-level semantic modulation and decoding feature correction in the training stage, thereby improving the adaptability and segmentation stability of lidar point cloud semantic segmentation model under severe weather conditions. Summary of the Invention

[0008] This invention aims to overcome the problems of existing technologies, such as the lack of point-level observation degradation reliability modeling, difficulty in distinguishing reliable and low-reliability regions, and insufficient cross-weather segmentation stability in lidar point cloud semantic segmentation models for severe weather. It provides a degradation reliability modeling method and apparatus for lidar point cloud semantic segmentation in severe weather.

[0009] This invention utilizes directly obtainable or computable distance, echo intensity, and local density from lidar point clouds as proxy variables for observation degradation, constructs point cloud degradation enhancement samples during the training phase, and proposes a degradation reliability modeling model (WDAM) for semantic segmentation of lidar point clouds in severe weather. This model estimates point-level degradation probability, point-level reliability, and degradation embedding at the input side; modulates semantic features based on degradation information at the high-level encoding stage; and performs feature correction on low-reliability regions by combining semantic context at the decoding stage, thereby outputting point-by-point semantic category prediction results for lidar point clouds in severe weather.

[0010] The first aspect of this invention relates to a degraded reliability modeling method for semantic segmentation of lidar point clouds in severe weather, comprising the following steps: Step 1: Construct a LiDAR point cloud semantic segmentation dataset using point cloud data collected by LiDAR, divide the dataset into a training set, a validation set, and a test set, and generate point cloud degradation enhancement samples for the training stage based on the source domain LiDAR point cloud data in the training set. Step 2: Construct a Degraded Reliability Modeling (WDAM) for semantic segmentation of lidar point clouds in severe weather. The WDAM model has the following structure: starting from the input, it sequentially connects an input feature processing module, a point-level degraded reliability encoding module, a hierarchical encoding-decoding point cloud semantic segmentation backbone network, and a semantic segmentation output module. In the hierarchical encoding-decoding point cloud semantic segmentation backbone network, the encoding stage consists of a shallow local geometry encoding module, a mid-level point cloud context modeling module, and a high-level sequence modeling module connected sequentially. The decoding stage consists of an upsampling module and a skip connection fusion module corresponding to the encoding stage. The point-level degraded reliability encoding module is also connected to a high-level degraded sensing modulation module and a decoding-end reliability sensing feature correction module. Step 3: Train and validate the WDAM model using the training and validation sets. Optimize the model parameters based on the semantic labels of the source domain point cloud and the semantic category prediction results output by the WDAM model to obtain the trained WDAM model weight file. Step 4: Input the severe weather lidar point cloud data from the test set into the trained WDAM model for testing, and output the semantic category prediction results for each lidar point.

[0011] The steps for constructing the lidar point cloud semantic segmentation dataset in step 1 are as follows: Step 1-1: Read source domain lidar point cloud data and severe weather lidar point cloud data frame by frame, and unify the format of point cloud coordinates, echo intensity and point-level semantic labels; Steps 1-2: Divide the source domain lidar point cloud data into training and validation sets, and use the lidar point cloud data in severe weather as the test set; Steps 1-3: Calculate the distance, echo intensity, and local density for each point in the training set, and generate the prior degradation probability based on the observable degradation correlations of positive distance correlation, negative echo intensity correlation, and negative local density correlation. Steps 1-4: Perform at least one degradation enhancement operation, including point dropping, coordinate perturbation, and echo intensity attenuation, on the source domain lidar point cloud in the training set according to the prior degradation probability, to obtain the training stage point cloud degradation enhancement sample corresponding to the source domain point-level semantic label.

[0012] In the WDAM model described in step 2, the input feature processing module receives the three-dimensional coordinates and echo intensity information of the LiDAR point cloud and generates input point features; the point-level degradation reliability encoding module generates point-level degradation probability, point-level reliability, and degradation embedding based on the distance, echo intensity, and local density of the input point cloud; the hierarchical encoding-decoding point cloud semantic segmentation backbone network encodes and decodes the point features after fusing degradation reliability information; the high-level degradation-aware modulation module modulates the high-level semantic features of the encoding stage based on the point-level degradation probability and degradation embedding; the decoding-end reliability-aware feature correction module corrects the low-reliability region features of the decoding end based on the point-level degradation probability, degradation embedding, decoding features, and high-level semantic context; and the semantic segmentation output module outputs the semantic category prediction result for each LiDAR point.

[0013] Furthermore, the structure of the WDAM model described in step 2 also includes: the point-level degradation reliability coding module consists of a degradation descriptor construction branch, a degradation embedding branch, a learnable correction branch, and a feature residual injection branch; the shallow local geometry coding module, the mid-level point cloud context modeling module, and the high-level sequence modeling module are sequentially connected to form the coding stage, and the high-level sequence modeling module includes a state space model module, an attention module, or a combination thereof; the upsampling module and the skip connection fusion module are connected to form the decoding stage, and are connected to point features at the same or adjacent levels as the coding stage; the high-level degradation-aware modulation module is connected to the point-level degradation reliability coding module and the high-level sequence modeling module; the decoding-end reliability-aware feature correction module is connected to the point-level degradation reliability coding module, the last upsampled feature of the decoding stage, the upsampled high-level semantic context, and the semantic segmentation output module.

[0014] Furthermore, the process of constructing the WDAM model also includes: setting the input and output signal dimensions and connection relationships of each module in the WDAM model according to the feature dimension of the input point cloud, the point cloud serialization method, the number of layers in the encoding and decoding stages, the number of feature channels at each layer, the embedding dimension of the point-level degradation reliability encoding module, the input and output dimensions of the high-level degradation sensing modulation module, the input and output dimensions of the decoding end reliability sensing feature correction module, and the number of semantic categories.

[0015] Step 2 specifically includes: Step 2-1: Input the point cloud degradation enhancement samples from the training phase into the input feature processing module to obtain the input point features; Step 2-2: Input the input point features into the point-level degradation reliability encoding module to obtain the point-level degradation probability, point-level reliability and degradation embedding, and inject the degradation embedding residual into the input point features; Steps 2-3: Input the point features after fusing the degradation reliability information into the encoding stage, and sequentially pass through the shallow local geometry encoding module, the middle point cloud context modeling module, and the high-level sequence modeling module to obtain encoded features at different levels; Steps 2-4: Aggregate the point-level degradation probabilities and degradation embeddings into the high-level point features of the encoding stage, and modulate the high-level semantic features through the high-level degradation-aware modulation module; Steps 2-5: Input the modulated high-level semantic features into the decoding stage, and recover the high-resolution point features through the upsampling module and the skip connection fusion module; Steps 2-6: Input the final upsampled features, upsampled high-level semantic context, local residual context, point-level degradation probability, and degradation embedding from the decoding stage into the reliability-aware feature correction module at the decoding end to obtain the corrected point features; Steps 2-7: Input the corrected point features into the semantic segmentation output module to obtain the semantic category prediction result for each LiDAR point.

[0016] Furthermore, in step 2, the core functional modules of the WDAM model include a point-level degradation reliability coding module, a high-level degradation sensing modulation module, and a decoding-end reliability sensing feature correction module. In the point-level degradation reliability coding module, for the first... Construct a degenerate descriptor using points: (1) in Represents the normalized distance. Indicates the normalized echo intensity. This represents the normalized local density. Based on the observable degradation correlations—positive correlation of distance, negative correlation of echo intensity, and negative correlation of local density—an observational degradation prior term is constructed: (2) in , and These are the weights corresponding to distance, echo intensity, and local density, respectively. The correction term is obtained using a learnable correction branch. And calculate the point-level degradation probability: (3) in Represents a nonlinear mapping function. To correct for the strength coefficient, point-level reliability is defined as: (4) The degenerate embedding branch generates degenerate embeddings based on point-level degenerate descriptors. The feature residual injection branch injects the degenerate embeddings into the input point features after weighting them according to the point-level degeneracy probability. Let the first... The original point features of each point are: Then the point features after fusing the degradation reliability information are: (5) in Injecting strength into the residual, This represents the feature mapping layer.

[0017] Furthermore, the high-level degradation sensing modulation module receives the point-level degradation probability. and degenerate embedding Based on the correspondence between low-level and high-level points during downsampling, the point-level degradation probability and degradation embedding are aggregated into the high-level point features. For the th The high-level points are aggregated to obtain the high-level degradation probability. and high-level degradation embedding Generate scaling factor and bias factor and high-level semantic features Modulation: (6) in, This represents the modulated high-level semantic features. Indicates the modulation intensity coefficient. This indicates channel-by-channel multiplication.

[0018] Furthermore, the decoding end reliability-aware feature correction module receives the current decoding features, upsampled high-level semantic context, local residual context, point-level degradation probability, and degradation embedding. Let the current decoding features be... Upsampling high-level semantic context is Then the local residual context is: (7) Under the constraints of degradation degree gating, semantic consistency gating, and confidence gating, a correction vector is generated, and the weak residual correction is expressed as: (8) in To correct the intensity, For reliability gating determined by both the degree of degradation and semantic consistency, For confidence gating, This is the correction vector.

[0019] Step 3 specifically includes: Step 3-1: Divide the dataset into training set, validation set and test set according to the proportion, set the hyperparameters of training model, batch, initial learning rate and training rounds, and input the point cloud degradation enhancement samples and their corresponding source domain point-level semantic labels into the training process. Step 3-2: Use the training set and validation set to periodically train and validate the WDAM model. Input the samples into the WDAM model in batches and calculate the semantic category prediction results output by the WDAM model according to steps 2-1 to 2-7. Step 3-3: Calculate the semantic segmentation loss based on the semantic category prediction results and source domain point-level semantic labels, and perform backpropagation and iterative training on the WDAM model; Steps 3-4: During the training process, the WDAM model is validated using a validation set, and the trained WDAM model weight file is obtained based on the validation results. After training is completed, the trained model parameters are saved to the weight file.

[0020] Specifically, step 4 includes: when using the WDAM model for testing, loading the trained WDAM model weight file into the WDAM model, then inputting the severe weather lidar point cloud of the test set frame by frame into the WDAM model, calculating the semantic category prediction result of each lidar point according to steps 2-1 to 2-7, and outputting the semantic segmentation result of the test set point cloud.

[0021] This invention proposes a semantic segmentation method for lidar point clouds in adverse weather conditions based on point cloud observation degradation reliability modeling. The method includes: step 1, constructing a lidar point cloud semantic segmentation dataset and generating point cloud degradation enhancement samples for the training phase; step 2, constructing a WDAM model and performing point-level degradation reliability encoding, high-level degradation sensing modulation, and decoding-end reliability sensing feature correction; step 3, training and validating the WDAM model using training and validation sets; and step 4, testing the semantic segmentation performance of the trained WDAM model using a test set.

[0022] A second aspect of the present invention relates to a degradation reliability modeling apparatus for semantic segmentation of lidar point clouds in severe weather, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather of the present invention.

[0023] The working principle of this invention is as follows: This invention utilizes observable information such as distance, echo intensity, and local density in point clouds to estimate point-level degradation probability and reliability, and incorporates this degradation reliability information throughout the data augmentation, input feature encoding, high-level semantic modulation, and feature correction processes at the decoding end. By explicitly introducing point-level degradation reliability representation at the input stage, performing degradation-aware modulation on semantic features at the high-level encoding stage, and performing controlled residual correction on low-reliability regions at the decoding end, the model can adjust feature representation and semantic prediction according to the observation reliability of different points or regions.

[0024] The present invention has the following advantages: it can establish a point-level degradation reliability representation using the observable information of the lidar point cloud itself without introducing additional sensor modes and target domain labels; it can improve the model's adaptability to observation degradation such as point missing, intensity attenuation and local sparsity through point cloud degradation enhancement during the training phase; and it can reduce the impact of low-reliability regions on semantic segmentation results and improve the stability of lidar point cloud semantic segmentation under adverse weather conditions by using high-level degradation sensing modulation and decoding end reliability sensing feature correction. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the steps of the method of the present invention.

[0026] Figure 2 This is a schematic diagram illustrating the process of constructing a lidar point cloud semantic segmentation dataset and enhancing point cloud degradation during the training phase of the present invention.

[0027] Figure 3 This is a flowchart of the method of the present invention.

[0028] Figure 4 This is a schematic diagram of the point-level degradation reliability coding module of the method of the present invention.

[0029] Figure 5 This is a schematic diagram of the structure of the high-level degradation sensing modulation module of the method of the present invention.

[0030] Figure 6 This is a schematic diagram of the structure of the reliability perception feature correction module at the decoding end of the method of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will be further described below with reference to the accompanying drawings. The embodiments listed in this specification are only used to illustrate the implementation of the present invention and are not intended to limit the scope of protection of the present invention.

[0032] Example 1

[0033] like Figure 1 and Figure 3As shown, this embodiment relates to a degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather, including the following steps: Step 1: Construct a LiDAR point cloud semantic segmentation dataset using point cloud data collected by LiDAR, divide the dataset into a training set, a validation set, and a test set, and generate point cloud degradation enhancement samples for the training stage based on the source domain LiDAR point cloud data in the training set. Step 2: Construct a Degraded Reliability Modeling (WDAM) for semantic segmentation of lidar point clouds in severe weather. The WDAM model predicts the semantic category of each point from the three-dimensional coordinates and echo intensity information of the lidar point cloud. Step 3: Use the training set and validation set to periodically train and validate the WDAM model, and obtain the weight file of the trained WDAM model; Step 4: Input the severe weather lidar point cloud data from the test set into the trained WDAM model, and test the semantic segmentation performance of the WDAM model using different point cloud frames from the test set.

[0034] Among them, such as Figure 2 As shown, step 1 specifically includes: Step 1-1: Read source domain lidar point cloud data and severe weather lidar point cloud data frame by frame. Each point must include at least three-dimensional coordinates x, y, z and echo intensity I; unify the format of point cloud coordinates, echo intensity and point-level semantic labels from different data sources. Steps 1-2: Divide the source domain lidar point cloud data into training and validation sets, and use the lidar point cloud data in severe weather as the test set; Steps 1-3: Calculate the distance to the origin of the lidar coordinates for each point in the training set, read its echo intensity, and count the number of points in the bold voxel or neighborhood of the point as the local density; standardize the distance, echo intensity, and local density in a single frame point cloud, and generate a priori degradation probability based on the observable degradation correlations of positive correlation of distance, negative correlation of echo intensity, and negative correlation of local density. Steps 1-4: Perform at least one degradation enhancement operation, including point dropping, coordinate perturbation, and echo intensity attenuation, on the source domain lidar point cloud in the training set according to the prior degradation probability, to obtain the training stage point cloud degradation enhancement sample corresponding to the source domain point-level semantic label.

[0035] Among them, such as Figure 3 As shown, step 2 specifically includes: Step 2-1: Combine the three-dimensional coordinates and echo intensity of the input lidar point cloud to form point features, and perform input feature processing on the point cloud to obtain a point cloud feature sequence that can be input into the WDAM model; Step 2-2: As Figure 4As shown, the point-level degradation reliability coding module generates point-level degradation probability, point-level reliability, and degradation embedding based on the distance, echo intensity, and local density of the input point cloud. The degradation embedding is then weighted according to the point-level degradation probability and injected into the input point features. Steps 2-3: The encoding stage of the hierarchical encoding-decoding point cloud semantic segmentation backbone network is used to extract hierarchical features from the point features after fusing degraded reliability information. The encoding stage is composed of a shallow local geometric encoding module, a mid-level point cloud context modeling module, and a high-level sequence modeling module connected in sequence. Steps 2-4: As Figure 5 As shown, a high-level degradation-aware modulation module is set after the high-level sequence modeling module in the encoding stage. Based on the aggregated point-level degradation probability and degradation embedding, scaling factors and bias factors are generated to perform degradation-aware modulation on the high-level semantic features. Steps 2-5: Utilize the decoding stage of the hierarchical encoding-decoding backbone network to upsample and fuse high-level semantic features through skip connections, thereby recovering point-by-point semantic features; Steps 2-6: (e.g.) Figure 6 As shown, a reliability-aware feature correction module is set at the end of the decoding stage. Based on the current decoding features, upsampled high-level semantic context, local residual context, point-level degradation probability and degradation embedding, controlled residual feature correction is performed on the low reliability region. Steps 2-7: Use the semantic segmentation output module to output the semantic category prediction result for each LiDAR point based on the corrected point-by-point semantic features.

[0036] Step 3 specifically includes: Step 3-1: Set the hyperparameters of the training model and input the point cloud degradation enhancement samples and their corresponding source domain point-level semantic labels into the training process; Step 3-2: Use the training set and validation set to periodically train and validate the WDAM model. Input the samples into the WDAM model in batches and calculate the semantic category prediction results output by the WDAM model according to steps 2-1 to 2-7. Step 3-3: Calculate the semantic segmentation loss based on the semantic category prediction results and source domain point-level semantic labels, and perform backpropagation and iterative training on the WDAM model; Steps 3-4: Validate the WDAM model using a validation set during training, and obtain the weight file of the trained WDAM model based on the validation results.

[0037] Step 4 specifically includes: Step 4-1: Load the trained WDAM model weight file into the WDAM model; Step 4-2: Input the severe weather lidar point cloud from the test set into the WDAM model frame by frame, and calculate the semantic category prediction result for each lidar point according to steps 2-1 to 2-7; Step 4-3: Output the semantic segmentation results of the test set point clouds, and test the semantic segmentation performance of the WDAM model using different point cloud frames in the test set.

[0038] Example 2

[0039] This embodiment provides a specific network parameter setting. The source domain training set uses labeled LiDAR point clouds collected under normal weather conditions, while the target test set uses LiDAR point clouds under adverse weather conditions, including rain, fog, and snow. The input features for each point include three-dimensional coordinates x, y, z and echo intensity I, with 4 input channels and 19 semantic categories.

[0040] Step 1 includes: Step 1-1: Read source domain lidar point cloud data collected under normal weather conditions and target test point cloud data collected under severe weather conditions. Each point in each frame of the point cloud includes at least three-dimensional coordinates x, y, z and echo intensity I, and the point-level semantic labels are uniformly mapped to 19 training categories. Steps 1-2: Divide the source domain lidar point cloud data into training set and validation set, and use lidar point cloud data collected under rain, fog and snow weather conditions as test set; Steps 1-3: Compile the distance, echo intensity, and local density of the source domain lidar points in the training set from the sensor coordinate origin. The local density grid size is set to 0.5, and the distance weight, echo intensity weight, and local density weight are set to 1.0, 0.5, and 0.8, respectively. Steps 1-4: Perform point cloud degradation enhancement during the training phase on the source domain lidar point cloud in the training set. The degradation enhancement probability is set to 0.35, the basic point drop ratio is set to 0.005, the point drop scaling ratio is set to 0.04, the maximum point drop ratio is set to 0.10, the minimum retention ratio is set to 0.90, the standard deviation of coordinate perturbation is set to 0.006, and the echo intensity attenuation ratio is set to 0.02.

[0041] Step 2 includes: Step 2-1: Combine the three-dimensional coordinates and echo intensity of the input lidar point cloud to form point features, and perform serialization and sparsification processing on the point cloud to obtain a point cloud feature sequence that can be input into the WDAM model. Step 2-2: Use the feature embedding module to map the input point features to a 32-dimensional feature space to obtain the initial point features; Steps 2-3: Using the point-level degradation reliability coding module, generate point-level degradation probability, point-level reliability, and degradation embedding based on the distance, echo intensity, and local density of each point, and inject the degradation embedding into the initial point features in a residual manner; Steps 2-4: Use an encoder to perform five-level encoding processing on the point cloud features. The number of output channels for the five encoding stages are 32, 64, 128, 256 and 512, respectively, and the module depths are 2, 2, 2, 6 and 2, respectively. Steps 2-5: After the fourth and fifth encoding stages, a high-level degradation-aware modulation module is set up to scale and bias the high-level semantic features based on the aggregated degradation probability and degradation embedding. Steps 2-6: The high-level semantic features are fused using a decoder through four levels of upsampling and skip connections. The number of output channels for the four decoding stages are 256, 128, 64 and 64, respectively. Steps 2-7: After the last decoding stage, set up a reliability-aware feature correction module at the decoding end to perform residual feature correction on low reliability regions based on point-level degradation probability, degradation embedding, upsampled high-level semantic context, and local residual context. Steps 2-8: The semantic segmentation output module is used to map the corrected 64-dimensional point features into 19 semantic prediction results. In this embodiment, the parameters of the degradation reliability related module layer are shown in Table 1, and the parameters of the WDAM model backbone network layer are shown in Table 2.

[0042] Table 1

[0043] Table 2

[0044] The number of hidden channels in the point-level degradation reliability coding module is set to 32, and the degradation embedding dimension is set to 16. The high-level degradation-aware modulation module is set after the fourth and fifth coding stages, with a modulation intensity of 0.1. The decoder-side reliability-aware feature correction module is set after the last decoding stage, with the number of hidden channels set to 32, the correction intensity η set to 0.02, the degradation threshold set to 0.65, the initial bias of the confidence branch set to -2.0, and semantic consistency gating and gradient stopping operations enabled.

[0045] Step 3 includes: Step 3-1: Set the training batch size to 6, the training epochs to 50, the loss function to be one or a combination of cross-entropy loss and Lovasz loss, and the optimizer to be an adaptive moment estimation optimizer or a stochastic gradient descent optimizer. Step 3-2: Use the training set and validation set to periodically train and validate the WDAM model. Input the point cloud degradation enhancement samples and their source domain point-level semantic labels into the WDAM model in batches during the training phase, and calculate the semantic category prediction results for each point according to steps 2-1 to 2-8. Step 3-3: Calculate the semantic segmentation loss based on the semantic category prediction results and source domain point-level semantic labels, and perform backpropagation and iterative training on the WDAM model; use the validation set to validate the WDAM model during training, and save the WDAM model weight file after training is completed.

[0046] Step 4 includes: Step 4-1: Load the WDAM model weight file obtained in Step 3, and input the severe weather lidar point cloud from the test set into the WDAM model frame by frame; Step 4-2: Calculate the semantic category prediction result for each point according to steps 2-1 to 2-8, and output the semantic segmentation result of the test set point cloud; Step 4-3: Compare the semantic category prediction results with the point-level semantic labels of the test set, and calculate at least one of the following evaluation metrics: average intersection-union ratio, average category accuracy, and overall accuracy.

[0047] Experimental results show that this invention can improve the semantic segmentation generalization ability of the model from normal weather to severe weather. Under cross-weather test settings, compared with the basic model without a degradation reliability modeling mechanism, this invention achieves improvements in metrics such as average intersection-over-union ratio (AUC), average class accuracy, and overall accuracy in the target domain; it also exhibits relatively stable segmentation performance on subsets of severe weather such as dense fog, light fog, rain, and snow. Qualitative results indicate that this invention helps reduce misclassification in distant regions, small target regions, boundary regions, and locally sparse regions, making the prediction results closer to the true semantic labels.

[0048] Example 3

[0049] This embodiment relates to a degradation reliability modeling device for semantic segmentation of lidar point clouds in severe weather, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in Embodiment 1.

[0050] Example 4

[0051] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements the degradation reliability modeling method for semantic segmentation of lidar point clouds in adverse weather conditions as described in Embodiment 1.

[0052] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A degraded reliability modeling method for semantic segmentation of lidar point clouds in severe weather, characterized in that, Includes the following steps: Step 1: Construct a LiDAR point cloud semantic segmentation dataset using point cloud data collected by LiDAR, divide the dataset into a training set, a validation set, and a test set, and generate point cloud degradation enhancement samples for the training stage based on the source domain LiDAR point cloud data in the training set. Step 2: Construct a Degraded Reliability Modeling (WDAM) for semantic segmentation of lidar point clouds in severe weather. The WDAM model is structured as follows: starting from the input, it sequentially connects an input feature processing module, a point-level degraded reliability encoding module, a hierarchical encoding-decoding point cloud semantic segmentation backbone network, and a semantic segmentation output module. In the hierarchical encoding-decoding point cloud semantic segmentation backbone network, the encoding stage consists of a shallow local geometry encoding module, a mid-level point cloud context modeling module, and a high-level sequence modeling module connected sequentially. The decoding stage consists of an upsampling module and a skip connection fusion module corresponding to the encoding stage. The point-level degraded reliability encoding module is also connected to a high-level degraded sensing modulation module and a decoding-end reliability sensing feature correction module. The high-level degraded sensing modulation module is connected to the output features of the high-level sequence modeling module in the encoding stage. The decoding-end reliability sensing feature correction module is connected to the final upsampled features, the upsampled high-level semantic context, and the semantic segmentation output module in the decoding stage. Step 3: Train and validate the WDAM model using the training and validation sets. Optimize the model parameters based on the semantic labels of the source domain point cloud and the semantic category prediction results output by the WDAM model to obtain the trained WDAM model weight file. Step 4: Input the severe weather lidar point cloud data from the test set into the trained WDAM model for testing, and output the semantic category prediction results for each lidar point.

2. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 1, characterized in that, Step 1 specifically includes: Step 1-1: Read source domain lidar point cloud data and severe weather lidar point cloud data frame by frame, and unify the format of point cloud coordinates, echo intensity and point-level semantic labels; Steps 1-2: Divide the source domain lidar point cloud data into training and validation sets, and use the lidar point cloud data in severe weather as the test set; Steps 1-3: Calculate the distance, echo intensity, and local density for each point in the training set, and generate the prior degradation probability based on the observable degradation correlations of positive distance correlation, negative echo intensity correlation, and negative local density correlation. Steps 1-4: Perform at least one degradation enhancement operation, including point dropping, coordinate perturbation, and echo intensity attenuation, on the source domain lidar point cloud in the training set according to the prior degradation probability, to obtain the training stage point cloud degradation enhancement sample corresponding to the source domain point-level semantic label.

3. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 1, characterized in that, In the WDAM model described in step 2, the input feature processing module receives the three-dimensional coordinates and echo intensity information of the LiDAR point cloud and generates input point features; the point-level degradation reliability encoding module generates point-level degradation probability, point-level reliability, and degradation embedding based on the distance, echo intensity, and local density of the input point cloud; the hierarchical encoding-decoding point cloud semantic segmentation backbone network encodes and decodes the point features after fusing degradation reliability information; the high-level degradation-aware modulation module modulates the high-level semantic features of the encoding stage based on the point-level degradation probability and degradation embedding; the decoding-end reliability-aware feature correction module corrects the low-reliability region features of the decoding end based on the point-level degradation probability, degradation embedding, decoding features, and high-level semantic context; and the semantic segmentation output module outputs the semantic category prediction result for each LiDAR point.

4. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 3, characterized in that, The structure of the WDAM model described in step 2 further includes: the point-level degradation reliability coding module consists of a degradation descriptor construction branch, a degradation embedding branch, a learnable correction branch, and a feature residual injection branch; the shallow local geometry coding module, the mid-level point cloud context modeling module, and the high-level sequence modeling module are sequentially connected to form the coding stage, and the high-level sequence modeling module includes a state space model module, an attention module, or a combination thereof; the upsampling module and the skip connection fusion module are connected to form the decoding stage, and are connected to point features at the same or adjacent levels as the coding stage; the high-level degradation-aware modulation module is connected to the point-level degradation reliability coding module and the high-level sequence modeling module; the decoding-end reliability-aware feature correction module is connected to the point-level degradation reliability coding module, the last upsampled feature of the decoding stage, the upsampled high-level semantic context, and the semantic segmentation output module.

5. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 4, characterized in that, The process of constructing the WDAM model also includes: setting the input and output signal dimensions and connection relationships of each module in the WDAM model according to the feature dimension of the input point cloud, the point cloud serialization method, the number of layers in the encoding and decoding stages, the number of feature channels in each layer, the embedding dimension of the point-level degradation reliability coding module, the input and output dimensions of the high-level degradation sensing modulation module, the input and output dimensions of the decoding end reliability sensing feature correction module, and the number of semantic categories.

6. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 5, characterized in that, In step 2, the core functional modules of the WDAM model include a point-level degradation reliability coding module, a high-level degradation sensing modulation module, and a decoding-end reliability sensing feature correction module. The point-level degradation reliability coding module includes a degradation descriptor construction branch, a degradation embedding branch, a learnable correction branch, and a feature residual injection branch. The degradation descriptor construction branch is used to construct a point-level degradation descriptor based on distance, echo intensity, and local density. The degradation embedding branch is used to generate a degradation embedding based on the point-level degradation descriptor. The learnable correction branch is used to correct the degradation prior of point cloud observations and generate point-level degradation probability and point-level reliability. The feature residual injection branch is used to weight the degradation embedding according to the point-level degradation probability and inject it into the input point features to obtain the point features after fusing degradation reliability information. The high-level degradation-aware modulation module receives point-level degradation probabilities and degradation embeddings, aggregates them into high-level point features based on the correspondence between low-level and high-level points during downsampling, and generates feature scaling factors and feature bias factors to modulate the high-level semantic features. The decoding-end reliability-aware feature correction module receives point-level degradation probabilities, degradation embeddings, current decoding features, upsampled high-level semantic context, and local residual context, and generates reliability gating and correction vectors to perform residual correction on the decoding features of low-reliability regions.

7. A degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 6, characterized in that, Step 2 specifically includes: Step 2-1: Input the point cloud degradation enhancement samples from the training phase into the input feature processing module to obtain the input point features; Step 2-2: Input the input point features into the point-level degradation reliability encoding module to obtain the point-level degradation probability, point-level reliability and degradation embedding, and inject the degradation embedding residual into the input point features; Steps 2-3: Input the point features after fusing the degradation reliability information into the encoding stage, and sequentially pass through the shallow local geometry encoding module, the middle point cloud context modeling module, and the high-level sequence modeling module to obtain encoded features at different levels; Steps 2-4: Aggregate the point-level degradation probabilities and degradation embeddings into the high-level point features of the encoding stage, and modulate the high-level semantic features through the high-level degradation-aware modulation module; Steps 2-5: Input the modulated high-level semantic features into the decoding stage, and recover the high-resolution point features through the upsampling module and the skip connection fusion module; Steps 2-6: Input the final upsampled features, upsampled high-level semantic context, local residual context, point-level degradation probability, and degradation embedding from the decoding stage into the reliability-aware feature correction module at the decoding end to obtain the corrected point features; Steps 2-7: Input the corrected point features into the semantic segmentation output module to obtain the semantic category prediction result for each LiDAR point.

8. The degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 1, characterized in that, Step 3 specifically includes: Step 3-1: Divide the dataset into training set, validation set and test set according to the proportion, set the hyperparameters of training model, batch, initial learning rate and training rounds, and input the point cloud degradation enhancement samples and their corresponding source domain point-level semantic labels into the training process. Step 3-2: Use the training set and validation set to periodically train and validate the WDAM model. Input the samples into the WDAM model in batches and calculate the semantic category prediction results output by the WDAM model according to steps 2-1 to 2-7. Step 3-3: Calculate the semantic segmentation loss based on the semantic category prediction results and source domain point-level semantic labels, and perform backpropagation and iterative training on the WDAM model; Steps 3-4: During the training process, the WDAM model is validated using a validation set, and the trained WDAM model weight file is obtained based on the validation results. After training is completed, the trained model parameters are saved to the weight file.

9. A degradation reliability modeling method for semantic segmentation of lidar point clouds in severe weather as described in claim 1, characterized in that, Step 4 specifically includes: inputting the severe weather lidar point cloud from the test set into the trained WDAM model, calculating the semantic category prediction result for each lidar point according to steps 2-1 to 2-7, and outputting the semantic segmentation result of the test set point cloud.

10. A degradation reliability modeling device for semantic segmentation of lidar point clouds in severe weather, characterized in that, The system includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the degradation reliability modeling method for semantic segmentation of lidar point clouds in adverse weather conditions as described in any one of claims 1 to 9.

Images