Laser radar semantic segmentation method based on point-by-point distance coding and point-level segmentation head

Abstract

The invention relates to the technical field of laser radar three-dimensional point cloud semantic segmentation, in particular to a laser radar semantic segmentation method based on point-by-point distance coding and a point-level segmentation head, and the method comprises the steps: S1, obtaining original three-dimensional point cloud data; s2, projecting each original point to a two-dimensional distance image; aggregating the original points projected to the same pixel position; recording a mapping relation; s3, processing the original points to form enhanced point features; generating an attention weight vector G; generating a modulated point feature = G; s4, filling to corresponding pixel positions to form a two-dimensional feature map, inputting the two-dimensional feature map into a feature extraction network, and extracting multi-level two-dimensional pixel features; s5, mapping the two-dimensional pixel features to the original points, and performing interactive fusion with the corresponding original points to obtain multi-level enhanced three-dimensional point features; and S6, fusing the enhanced three-dimensional point features. The method can make full use of the point-by-point depth information and spatial distribution characteristics of the point cloud while maintaining the advantage of efficient processing of the distance image.

Description

A semantic segmentation method for LiDAR based on point-by-point distance coding and point-level segmentation heads. Technical Field

[0001] This invention relates to the field of semantic segmentation technology for 3D point clouds of lidar, and in particular to a semantic segmentation method for lidar based on point-by-point distance encoding and point-level segmentation head. Background Technology

[0002] With the rapid development of technologies such as autonomous driving, intelligent robots, and intelligent transportation, LiDAR, as a sensor capable of acquiring high-precision three-dimensional spatial information, is widely used in environmental perception and scene understanding. LiDAR obtains three-dimensional point cloud data containing information such as spatial coordinates and reflection intensity by emitting laser beams and receiving reflected signals. Semantic segmentation of this point cloud data—that is, assigning a corresponding semantic category (such as vehicle, pedestrian, building, ground, etc.) to each point—is a crucial foundation for achieving environmental understanding, target recognition, and decision control.

[0003] However, 3D point cloud data exhibits typical characteristics of sparsity, disorder, and unstructuredness, making it difficult for traditional convolutional neural networks based on regular grids to process directly. To address this, existing technologies have proposed various point cloud semantic segmentation methods, such as: 1) methods based on raw points (e.g., the PointNet series) directly model the point set, preserving the geometric integrity of the point cloud, but are limited by their weak ability to model local structures and the problem of computational complexity increasing linearly with the number of points; 2) voxel-based methods (e.g., VoxelNet) divide the space into regular voxels, facilitating 3D convolution processing, but suffer from severe spatial redundancy, especially in long-distance sparse regions where a large number of empty voxels lead to wasted computational resources; 3) multimodal fusion methods combine information from sensors such as vision and millimeter-wave radar, improving semantic understanding capabilities, but require solving complex problems such as cross-modal registration, time synchronization, and feature alignment, increasing the difficulty of system implementation.

[0004] In contrast, range image-based methods project 3D point clouds onto a 2D plane to form range images, enabling efficient extraction of spatial features using mature 2D convolutional neural networks. These methods exhibit significant advantages in computational efficiency and model scalability. Typically, these methods use spherical coordinate projection to map each point to a 2D pixel location and aggregate information from multiple points at corresponding locations. However, because multiple 3D points may map to the same pixel unit during projection, forming a "many-to-one" mapping relationship, traditional methods generally employ nearest neighbor interpolation or maximum / average value aggregation strategies to determine pixel values. While this strategy ensures projection efficiency, it easily loses crucial spatial structural information, especially in complex scenes with distant sparse regions or densely intersecting obstacles. Uneven point cloud distribution leads to blurred important semantic boundaries, thus affecting segmentation accuracy and stability.

[0005] Furthermore, existing methods do not fully utilize depth information in the feature modeling stage. Point clouds in different spatial regions not only differ significantly in density, but their semantic attributes also exhibit obvious spatial dependencies. For example, dense areas in the foreground often contain dynamic targets (such as vehicles and pedestrians), while sparse areas in the distance are mostly static backgrounds (such as building outlines). However, most current methods only use depth as a single channel input, lacking effective encoding of its spatial distribution patterns, making it difficult to capture the semantic change trends of point clouds at different distance levels.

[0006] More importantly, in the multi-level feature fusion stage, most existing methods adopt fixed weighting or simple concatenation strategies, which cannot dynamically adjust the importance of features at each level according to the specific scenario. For example, while high-level semantic features have strong discriminative power, they are insufficient in expressing edge details; while low-level geometric features retain rich local structures, their generalization ability is limited. If the contributions of the two cannot be adaptively coordinated, feature redundancy or information loss can easily occur, limiting the robustness and generalization performance of the model in complex urban environments.

[0007] Therefore, how to fully utilize the point-by-point depth information and spatial distribution characteristics of point clouds to construct enhanced point feature representations while maintaining the advantages of efficient processing of distance images has become an urgent problem to be solved. Summary of the Invention

[0008] To address the aforementioned shortcomings of existing technologies, the present invention aims to provide a LiDAR semantic segmentation method based on point-by-point distance coding and point-level segmentation heads. This method can maintain the advantages of efficient processing of distance images while fully utilizing the point-by-point depth information and spatial distribution characteristics of point clouds to construct enhanced point feature representations, and significantly improve the semantic segmentation accuracy and stability in long-distance sparse regions and complex structural regions.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0010] The semantic segmentation method for LiDAR based on point-by-point distance coding and point-level segmentation head includes the following steps:

[0011] S1. Acquire the raw 3D point cloud data collected by the lidar. The raw 3D point cloud data contains multiple raw points, each with raw features. ;

[0012] S2. According to the spherical coordinate projection formula, project each original point to the corresponding pixel position of the two-dimensional distance image; for multiple original points projected to the same pixel position, based on their original features... The original point is aggregated at the pixel location to generate the two-dimensional distance image, and the mapping relationship between each original point and the pixel it is projected to is recorded.

[0013] S3. For each original point, calculate the Euclidean distance from that point to the origin of the lidar coordinate system. Then, normalization is performed to obtain normalized deep features. The pixel position information projected onto this point in S2 is used as the projection feature. Then normalize the depth features of this point. Projection features and original features Cascaded splicing is performed to form enhanced point features. ;

[0014] Utilizing the enhanced point features As a condition, adaptively generate the enhanced point features. A dimensionally consistent attention weight vector G is then applied to the original features. Channel-level modulation generates modulated point features. = G, This represents element-wise multiplication;

[0015] S4. Modulate the point features of each original point generated in S3. The mapping relationship recorded in S2 is filled into the corresponding pixel positions in the two-dimensional distance image to form a two-dimensional feature map; and the two-dimensional feature map is input into the feature extraction network to extract multi-level two-dimensional pixel features.

[0016] S5. Based on the mapping relationship recorded in S2, map the two-dimensional pixel features of different levels obtained in S4 to their corresponding original points, and combine them with the corresponding modulated point features formed in S3. Interactive fusion is performed to obtain multi-level enhanced 3D point features;

[0017] S6. The enhanced 3D point features obtained in S5 are fused, including: introducing learnable weight coefficients to the features at each level and performing adaptive fusion by weighted summation; and predicting the semantic category of each original point point by point based on the fused features.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1) Explicitly introducing point-by-point distance information enhances the ability to model spatial distribution differences. Existing point cloud semantic segmentation methods based on LiDAR mostly rely on implicit learning using the spatial coordinates or local neighborhood features of the point cloud, making it difficult to effectively characterize the density unevenness and semantic differences that point clouds exhibit with distance. For example, in distant regions, point clouds are sparse and noisy, while in nearby regions they are dense and contain rich details; traditional methods often cannot distinguish the semantic impact of these differences. In step S3 of this invention, the Euclidean distance from each original point to the origin of the LiDAR coordinates is calculated. Then, normalization is performed to obtain normalized deep features. At the same time, combined with projection position information With original features Cascaded splicing is performed to form enhanced point features. This design enables the network to explicitly perceive the spatial distance relationships of each point, thereby more accurately distinguishing target attributes at different distance levels. Compared to traditional methods that only use depth as a channel input or ignore distance variations, this strategy significantly improves the model's ability to represent features of near and far targets, sparse regions, and complex boundary regions, and enhances its robustness in recognizing targets in low-density areas.

[0020] 2) Construct an adaptive attention mechanism to achieve dynamic modulation of the original features. In step S3, enhance point features are utilized. As a condition, an attention weight vector G with the same dimension as the original feature is adaptively generated, and the original feature is processed by element-wise multiplication. Channel weighting is performed to generate modulated point features. = G. This mechanism differs from traditional static or global feature extraction methods. Instead, it dynamically adjusts the importance of feature channels based on information such as the depth and projection position of each point. For example, for sparse points in the distance, the network can automatically enhance the weights of their reflection intensity or geometric features to compensate for information attenuation caused by distance; while for dense points in the vicinity, it may suppress redundant channels and highlight key semantic cues. This process achieves fine-grained control over point cloud features, avoiding the problems of single feature responses and lack of specificity in traditional methods, and significantly improving the discriminativeness and adaptability of feature representation.

[0021] 3) It enables adaptive weighted fusion of multi-level features, enhancing the collaborative representation of features. Existing methods often employ fixed-weight fusion or simple concatenation to address the differences in receptive field, resolution, and semantic abstraction levels of output features from different network layers. This leads to an imbalance in contributions between high-level semantic features and low-level geometric features, easily causing feature redundancy or information loss. In step S6, this invention introduces learnable weight coefficients to adaptively weight and fuse enhanced 3D point features from multiple layers. This mechanism allows the network to automatically adjust the contribution ratio of each layer's features in the final prediction based on the specific characteristics of the input data (such as scene complexity and target distribution), avoiding the uneven information utilization problem caused by traditional fixed fusion strategies. Compared to direct concatenation or average fusion, this method more effectively preserves the advantages of features at each layer, achieving more refined collaborative feature representation and significantly improving the model's generalization ability and discrimination accuracy in complex environments.

[0022] 4) Improving modeling efficiency and accuracy without increasing structural complexity. Many current high-precision point cloud segmentation methods rely on complex attention mechanisms, large-scale neighborhood searches, or deep neural network structures. While these improve segmentation performance, they incur high computational costs, making it difficult to meet real-time requirements. This invention explicitly introduces point-by-point distance encoding and an adaptive weighted fusion mechanism, achieving effective utilization of point cloud spatial structure information without introducing additional complex modules (such as non-local attention, Transformer structures, or large-scale convolutions). Especially noteworthy is the enhanced point features constructed in S3. As a conditionally generated attention weight vector G, the original features are processed with channel weighting, which enhances feature representation while maintaining a lightweight computational flow. Therefore, this method maintains the advantage of efficient processing of distance images while avoiding dependence on complex structures, achieving a good balance between segmentation accuracy and computational efficiency.

[0023] In summary, this method can fully utilize the point-by-point depth information and spatial distribution characteristics of point clouds while maintaining the advantages of efficient processing of distance images, constructing enhanced point feature representations, and significantly improving the semantic segmentation accuracy and stability in distant sparse regions and complex structural regions.

[0024] Preferably, in S3, a depth-sensing gating mechanism is used, based on enhanced point features. The attention weight vector G is generated as a conditional input.

[0025] This setup, through a depth-sensing gating mechanism, enables a channel mechanism that dynamically adjusts based on spatial distance information. This effectively enhances the model's ability to represent point clouds at different distances, while maintaining structural simplicity and improving feature utilization efficiency and semantic segmentation performance.

[0026] Preferably, the process of generating the attention weight vector G includes: converting the enhanced point features... The input is fed into a perceptron structure consisting of two fully connected layers and an activation function to generate an attention weight vector:

[0027] ;

[0028] In the formula, , Here are the learnable parameters for the i-th layer, and the superscript C represents the original features. The number of channels, where H in the upper right corner indicates the dimension of the hidden layer; This is the activation function.

[0029] This setup achieves efficient and learnable attention modeling for enhanced point features through a lightweight perceptron structure, ensuring computational efficiency while enhancing the flexibility and expressiveness of feature modulation, thus providing more discriminative point feature representations for subsequent semantic segmentation tasks.

[0030] Preferably, in S6, the adaptive fusion adopts a bottom-up residual fusion path, progressively fusing multi-level enhanced 3D point features in hierarchical order, wherein the fused features of the i-th layer... Determined by the following formula:

[0031] ;

[0032] In the formula, This indicates that dimensionality processing is performed using a multilayer perceptron. Represents the fusion features of the (i-1)th layer; This represents the enhanced 3D point features of the i-th layer; These are the learnable weight coefficients.

[0033] This setup achieves: 1) a progressive fusion of hierarchical features, enhancing the collaborative expression of low-level details and high-level semantics. Existing methods often employ direct concatenation or fixed-weighted summation in multi-level feature fusion, which can easily lead to high-level semantic features dominating the fusion result, while low-level geometric details are ignored, or information redundancy occurs. This scheme utilizes a bottom-up residual fusion path to integrate the enhanced features of each layer. With learnable weights Add to the previous layer's fusion result The transformation output forms a progressive fusion process. This structure not only preserves the spatial details of the underlying features but also gradually introduces high-level semantic information, achieving a smooth transition from local structure to global semantics. Compared to one-time global fusion, this strategy is more conducive to maintaining the hierarchical consistency of features and improving the model's ability to simultaneously characterize the boundaries and internal structures of targets in complex scenes.

[0034] 2) Introducing learnable weight coefficients to achieve data-driven dynamic feature contribution adjustment. Traditional fusion methods often use fixed weights or simple averaging, which are difficult to adapt to the differences in the importance of features at different levels under different input scenarios. This solution introduces learnable weight coefficients. This enables the network to automatically adjust the contribution ratio of each layer of augmented features in the final prediction based on the distribution characteristics of the input point cloud (such as density, distance, and semantic complexity).

[0035] Preferably, the learnable weight coefficients The activation function is restricted to the range [0, 1] to ensure its numerical stability and controllability.

[0036] This setup, by imposing reasonable numerical constraints on the learnable weight coefficients, improves the stability, controllability, and interpretability of the multi-level feature fusion process, providing solid technical support for building an efficient and reliable point cloud semantic segmentation model.

[0037] Preferably, in S5, the interactive fusion is a multi-stage bidirectional fusion, including: performing two-dimensional to three-dimensional feature mapping and fusion at multiple levels respectively, and aggregating the fused three-dimensional point features back to the corresponding pixels to update the two-dimensional representation.

[0038] This setup enables: 1) bidirectional information flow between 2D and 3D features, enhancing the collaborative modeling capability of spatial structure and semantic representation. Traditional methods typically only project 3D point clouds onto 2D images for feature extraction, then inversely map the 2D features back to 3D points, forming a unidirectional information transmission path, which easily leads to information loss or asymmetry. This scheme uses multi-stage bidirectional fusion, simultaneously utilizing 2D feature maps (extracted by the projected network) and 3D point features (generated from modulated original points) at each level: on the one hand, it maps high-level 2D semantic information back to 3D points, improving the semantic richness of point-level features; on the other hand, it aggregates the fused 3D enhanced features back to 2D pixels, updating the 2D representation to include more accurate spatial structure information. This bidirectional feedback mechanism achieves dynamic alignment between 2D semantic understanding and 3D geometric perception, significantly improving the model's ability to capture details such as target boundaries and occlusion relationships in complex scenes.

[0039] 2) Enhance feature consistency and context awareness to mitigate semantic degradation caused by projection. Due to the "many-to-one" mapping problem when projecting 3D point clouds onto 2D images, unidirectional feature propagation can easily lead to key point information being ignored or buried. This solution repeatedly performs 2D-to-3D mapping and fusion at multiple levels, while simultaneously updating the 2D representation. This ensures that each pixel continuously receives the latest 3D information from its corresponding original point, gradually correcting semantic biases caused by projection aggregation. This iterative interactive fusion effectively alleviates the semantic degradation problem caused by projection distortion in traditional methods, enhancing the model's feature consistency and context awareness in sparse regions, dense structures, and complex topological environments.

[0040] Preferably, in S3, the normalization process is performed in the following manner:

[0041] The calculated Euclidean distance The clamp function limits the distance to a preset range. The data is then linearly normalized to obtain normalized deep features. .

[0042] This setup, through reasonable distance truncation and linear normalization design, significantly improves the robustness and learnability of deep features, providing a stable and reliable spatial information foundation for the subsequent construction of enhanced point features.

[0043] Preferably, the preset distance interval for The unit is meters.

[0044] In typical applications such as autonomous driving and intelligent transportation, the effective perception range of LiDAR is usually concentrated between 3 and 80 meters. Areas smaller than 3 meters are mostly occupied by the vehicle itself or near-field obstacles and are often covered by other sensors (such as ultrasonic or millimeter-wave radar); while areas larger than 80 meters suffer from sparse point clouds and weak reflected signals, making semantic information difficult to extract effectively. This solution limits the distance range to... This approach avoids processing invalid or low-quality data while ensuring the model focuses on the most semantically valuable perceptual regions. This targeted design enables the network to learn spatial-semantic relationships in key scenarios more efficiently, improving the model's practicality and robustness in real-world road environments.

[0045] Preferably, in S4, the feature extraction network is a convolutional neural network with a residual structure.

[0046] This setup, by introducing a convolutional neural network with residual structures, improves the stability and expressive power of the feature extraction process, providing a high-quality, multi-level visual representation foundation for subsequent 3D point cloud semantic segmentation.

[0047] Preferably, in S2, for multiple original points projected to the same pixel location, the aggregation operation is to aggregate the original features of the multiple original points falling into the same pixel. Take the average, maximum, or weighted sum.

[0048] This setup, through the rational design of a multi-point aggregation strategy, improves the information fidelity and expressive power of two-dimensional distance images, laying a solid foundation for subsequent feature extraction and semantic segmentation tasks. Attached Figure Description

[0049] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0050] Figure 1 is a flowchart of this method;

[0051] Figure 2 is a general network block diagram in Example 1;

[0052] Figure 3 is a flowchart of S3 in Example 1;

[0053] Figure 4 is a qualitative analysis diagram of the SemanticKITTI validation set in Example 2. Detailed Implementation

[0054] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0056] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not indicate that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0057] Example 1

[0058] As shown in Figure 1, this invention provides a semantic segmentation method for LiDAR based on point-to-point distance encoding and point-level segmentation head.

[0059] In practice, each step in S2-S6 can be implemented through the corresponding functional modules, and the overall network block diagram is shown in Figure 2.

[0060] This method includes the following steps:

[0061] S1. Acquire the raw 3D point cloud data collected by the lidar. The raw 3D point cloud data contains multiple raw points, each with raw features. ;

[0062] S2. According to the spherical coordinate projection formula, project each original point to the corresponding pixel position of the two-dimensional distance image; for multiple original points projected to the same pixel position, based on their original features... The original point is aggregated at the pixel location to generate the two-dimensional distance image, and the mapping relationship between each original point and the pixel it is projected to is recorded.

[0063] In practice, for multiple original points projected to the same pixel location, the aggregation operation involves combining the original features of the multiple original points falling into the same pixel. The average, maximum, or weighted sum can be used. By rationally designing a multi-point aggregation strategy, the information fidelity and expressive power of the two-dimensional distance image are improved, laying a solid foundation for subsequent feature extraction and semantic segmentation tasks.

[0064] Due to the sparsity and disorder of LiDAR point clouds, direct feature modeling on the original point cloud often results in high computational complexity. Therefore, this invention performs projection processing on the point cloud data. Based on spherical coordinate projection relationships, the 3D point cloud data is projected onto a 2D distance image, so that each pixel position in the 2D distance image corresponds to a cubic region in 3D space. Multiple point clouds belonging to the same cubic region are then aggregated, thereby achieving point cloud structure partitioning based on cubic units. This provides a structural foundation for subsequent joint modeling of 2D and 3D features.

[0065] S3. As shown in Figure 3, step S3 addresses the problem that in existing methods, distance information is mostly implicitly involved in feature learning, making it difficult to effectively characterize the density differences of point clouds as distance changes. This invention constructs a point-by-point distance encoding mechanism. This method mainly includes two core stages: deep feature generation and multimodal feature fusion. In specific implementation, these can be achieved through a preprocessing module and an encoder module, respectively.

[0066] The deep feature generation stage includes: for each original point, calculating the Euclidean distance from that point to the origin of the LiDAR coordinate system. Then, normalization is performed to obtain normalized deep features. The pixel position information projected onto this point in S2 is used as the projection feature. Then normalize the depth features of this point. Projection features and original features Cascaded splicing is performed to form enhanced point features. : .

[0067] In specific implementation, the normalization process is carried out in the following manner: the calculated Euclidean distance is... The clamp function limits the distance to a preset range. The data is then linearly normalized to obtain normalized deep features. .

[0068] Wherein, the preset distance interval for The unit is meters. In typical applications such as autonomous driving and intelligent transportation, the effective perception range of LiDAR is usually concentrated between 3 meters and 80 meters. Areas smaller than 3 meters mostly belong to the vehicle itself or near-field obstacles and are often covered by other sensors (such as ultrasonic and millimeter-wave radar); while areas larger than 80 meters are difficult to effectively extract semantic information due to sparse point clouds and weak reflected signals. This solution limits the distance range to within... This approach avoids processing invalid or low-quality data while ensuring the model focuses on the most semantically valuable perceptual regions. This targeted design enables the network to learn spatial-semantic relationships in key scenarios more efficiently, improving the model's practicality and robustness in real-world road environments.

[0069] The multimodal feature fusion stage includes: utilizing the enhanced point features As a condition, adaptively generate the enhanced point features. A dimensionally consistent attention weight vector G is then applied to the original features. Channel-level modulation generates modulated point features. = G, This indicates element-wise multiplication.

[0070] In practice, a deep perception gating mechanism is used, based on enhanced point features. The attention weight vector G is generated as a conditional input. The generation process of the attention weight vector G includes: using the augmentation point features... The input is fed into a perceptron structure consisting of two fully connected layers and an activation function to generate an attention weight vector:

[0071] ;

[0072] In the formula, , Here are the learnable parameters for the i-th layer, and the superscript C represents the original features. The number of channels, where H in the upper right corner indicates the dimension of the hidden layer; This is the activation function.

[0073] In this way, an efficient and learnable attention model for enhanced point features is achieved through a lightweight perceptron structure, which not only ensures computational efficiency but also enhances the flexibility and expressiveness of feature modulation, providing a more discriminative point feature representation for subsequent semantic segmentation tasks.

[0074] Enhanced features and gating weights are scaled using attention gating, achieving a point-level feature modulation mechanism based on depth information. This effectively improves the model's expressive power and semantic discrimination ability in long-distance sparse regions. Simultaneously, this feature vector is input into a multilayer perceptron to enhance the model's ability to express features at the current depth scale. Furthermore, through feature aggregation and compression operations, effective integration and dimensionality reduction of voxel-level features are achieved, thereby improving computational efficiency and enhancing cross-scale fusion capabilities.

[0075] Introducing this mechanism enables explicit modeling of the spatial distance context of point cloud data, thereby providing richer geometric feature representations for semantic segmentation networks and significantly improving the accuracy and robustness of scene semantic segmentation, especially when distinguishing between distant sparse objects and nearby dense objects.

[0076] S4. Modulate the point features of each original point generated in S3. The mapping relationship recorded in S2 is filled into the corresponding pixel positions in the two-dimensional distance image to form a two-dimensional feature map; and the two-dimensional feature map is input into the feature extraction network to extract multi-level two-dimensional pixel features.

[0077] In practice, the feature extraction network is a convolutional neural network with a residual structure. The purpose of using a residual structure is to alleviate the gradient vanishing problem while maintaining network depth, thereby stabilizing the feature learning process and obtaining multi-scale feature representations with different receptive fields and semantic levels.

[0078] S5. Based on the mapping relationship recorded in S2, map the two-dimensional pixel features of different levels obtained in S4 to their corresponding original points, and combine them with the corresponding modulated point features formed in S3. Interactive fusion is performed to obtain multi-level enhanced 3D point features.

[0079] In specific implementation, the interactive fusion is a multi-stage bidirectional fusion, including: performing two-dimensional to three-dimensional feature mapping and fusion at multiple levels, and aggregating the fused three-dimensional point features back to the corresponding pixels to update the two-dimensional representation.

[0080] Traditional methods typically only project 3D point clouds onto 2D images for feature extraction, then inversely map the 2D features back to 3D points, forming a unidirectional information transmission path that is prone to information loss or asymmetry. This approach employs multi-stage bidirectional fusion, simultaneously utilizing 2D feature maps (extracted by the projected network) and 3D point features (generated from modulated original points) at each level: on one hand, mapping high-level 2D semantic information back to 3D points enhances the semantic richness of point-level features; on the other hand, aggregating the fused 3D enhanced features back to 2D pixels updates the 2D representation, making it contain more accurate spatial structure information. This bidirectional feedback mechanism achieves dynamic alignment between 2D semantic understanding and 3D geometric perception, significantly improving the model's ability to capture details such as target boundaries and occlusion relationships in complex scenes.

[0081] Furthermore, due to the "many-to-one" mapping problem when projecting 3D point clouds onto 2D images, unidirectional feature propagation can easily lead to the neglect or submersion of key point information. This scheme addresses this by repeatedly performing 2D-to-3D mapping and fusion at multiple levels, while simultaneously updating the 2D representation. This ensures that each pixel continuously receives the latest 3D information from its corresponding original point, thereby gradually correcting semantic biases caused by projection aggregation. This iterative interactive fusion effectively alleviates the semantic degradation problem caused by projection distortion in traditional methods, enhancing the model's feature consistency and context awareness in sparse regions, dense structures, and complex topological environments.

[0082] S6. The enhanced 3D point features obtained in S5 are fused, including: introducing learnable weight coefficients to the features at each level and performing adaptive fusion by weighted summation; and predicting the semantic category of each original point point by point based on the fused features.

[0083] In practice, as shown in Figure 2, to achieve effective fusion of multi-level point features, the two-dimensional feature information is first sampled into three-dimensional point features. Then, a multilayer perceptron processes the input point features layer by layer, ensuring that the point features at each level maintain dimensionality. Next, point features from different depth levels are weighted and fused to form a richer intermediate representation. To enhance the flexibility of the fusion, a learnable weight coefficient is introduced for each layer of features. This weight coefficient is constrained to the range of 0 to 1 by an activation function, thus ensuring its numerical stability and controllability. In practice, it can be assumed that features from all layers have equal importance to the final representation, and the initial value of the weight coefficient is set to 0.5. During training, the model autonomously learns the contribution of features from different levels to the final prediction. This design can adaptively adjust the importance of features at different depths, dynamically controlling the influence of each layer's features in the final fused feature.

[0084] In practice, the adaptive fusion adopts a bottom-up residual fusion path, progressively fusing multi-level enhanced 3D point features in hierarchical order, where the fused features of the i-th layer... Determined by the following formula:

[0085] ;

[0086] In the formula, This indicates that dimensionality processing is performed using a multilayer perceptron. Represents the fusion features of the (i-1)th layer; This represents the enhanced 3D point features of the i-th layer; These are the learnable weight coefficients.

[0087] Wherein, the learnable weight coefficients By restricting the activation function to the range [0, 1], its numerical stability and controllability are ensured. By imposing reasonable numerical constraints on the learnable weight coefficients, the stability, controllability, and interpretability of the multi-level feature fusion process are improved, providing solid technical support for building an efficient and reliable point cloud semantic segmentation model.

[0088] Existing methods often employ direct concatenation or fixed-weighted summation when fusing multi-level features, which can easily lead to high-level semantic features dominating the fusion result, while low-level geometric details are ignored or information redundancy occurs. This scheme utilizes a bottom-up residual fusion path to integrate the enhanced features of each layer. With learnable weights Add to the previous layer's fusion result The transformation output forms a progressive fusion process. This structure not only preserves the spatial details of low-level features but also gradually introduces high-level semantic information, achieving a smooth transition from local structure to global semantics. Compared to one-time global fusion, this strategy is more conducive to maintaining the hierarchical consistency of features and improving the model's ability to simultaneously characterize the boundaries and internal structures of targets in complex scenes. Furthermore, traditional fusion methods often use fixed weights or simple averaging, which are difficult to adapt to the differences in the importance of features at different levels under different input scenarios. This solution introduces learnable weight coefficients... This enables the network to automatically adjust the contribution ratio of each layer of augmented features in the final prediction based on the distribution characteristics of the input point cloud (such as density, distance, and semantic complexity).

[0089] Existing point cloud semantic segmentation methods based on LiDAR mostly rely on implicit learning of the spatial coordinates or local neighborhood features of the point cloud, which makes it difficult to effectively characterize the density unevenness and semantic differences of the point cloud as it changes with distance. For example, in distant regions, the point cloud is sparse and noisy, while in nearby regions it is dense and contains rich details. Traditional methods often cannot distinguish the semantic impact of these differences. In step S3 of this invention, the Euclidean distance from each original point to the origin of the LiDAR coordinate system is calculated. Then, normalization is performed to obtain normalized deep features. At the same time, combined with projection position information With original features Cascaded splicing is performed to form enhanced point features. This design enables the network to explicitly perceive the spatial distance relationships of each point, thereby more accurately distinguishing target attributes at different distance levels. Compared to traditional methods that only use depth as a channel input or ignore distance variations, this strategy significantly improves the model's ability to represent features of near and far targets, sparse regions, and complex boundary regions, enhancing its robustness in recognizing targets in low-density areas. Furthermore, in step S3, enhanced point features are utilized... As a condition, an attention weight vector G with the same dimension as the original feature is adaptively generated, and the original feature is processed by element-wise multiplication. Channel weighting is performed to generate modulated point features. = G. This mechanism differs from traditional static or global feature extraction methods. Instead, it dynamically adjusts the importance of feature channels based on information such as the depth and projection position of each point. For example, for sparse points in the distance, the network can automatically enhance the weights of their reflection intensity or geometric features to compensate for information attenuation caused by distance; while for dense points in the vicinity, it may suppress redundant channels and highlight key semantic cues. This process achieves fine-grained control over point cloud features, avoiding the problems of single feature responses and lack of specificity in traditional methods, and significantly improving the discriminativeness and adaptability of feature representation.

[0090] In addition, existing methods often employ fixed-weight fusion or simple concatenation to address the differences in receptive field range, resolution, and semantic abstraction levels of output features from different network layers. This leads to an imbalance in contributions between high-level semantic features and low-level geometric features, easily causing feature redundancy or information loss. In step S6, this invention introduces learnable weight coefficients to adaptively weight and fuse enhanced 3D point features from multiple layers. This mechanism allows the network to automatically adjust the contribution ratio of each layer's features in the final prediction based on the specific characteristics of the input data (such as scene complexity and target distribution), avoiding the uneven information utilization problem caused by traditional fixed fusion strategies. Compared to direct concatenation or average fusion, this method more effectively preserves the advantages of features from each layer, achieves more refined feature co-expression, and significantly improves the model's generalization ability and discrimination accuracy in complex environments.

[0091] Many current high-precision point cloud segmentation methods rely on complex attention mechanisms, large-scale neighborhood searches, or deep neural network structures. While these improve segmentation performance, they also incur high computational costs, making it difficult to meet real-time requirements. This invention, by explicitly introducing point-by-point distance encoding and an adaptive weighted fusion mechanism, achieves effective utilization of point cloud spatial structure information without introducing additional complex modules (such as non-local attention, Transformer structures, or large-scale convolutions). Especially noteworthy is the enhanced point features constructed in S3. As a conditionally generated attention weight vector G, the original features are processed with channel weighting, which enhances feature representation while maintaining a lightweight computational flow. Therefore, this method maintains the advantage of efficient processing of distance images while avoiding dependence on complex structures, achieving a good balance between segmentation accuracy and computational efficiency.

[0092] This method can fully utilize the point-by-point depth information and spatial distribution characteristics of point clouds while maintaining the advantages of efficient processing of distance images, constructing enhanced point feature representations, and significantly improving the semantic segmentation accuracy and stability in distant sparse regions and complex structural regions.

[0093] Example 2

[0094] To better illustrate the effectiveness of this method, the following experiment was conducted.

[0095] To verify the effectiveness of the technical solution disclosed in this invention, experiments were conducted using the SemanticKITTI dataset, and the results were analyzed to verify the effectiveness of the algorithm. SemanticKITTI is a dataset built on the well-known KITTI odometry benchmark. It contains over 43,000 LiDAR scans acquired by the Velodyne HDL-64E sensor, with an average of 104,452 points per scan, providing dense point-by-point annotations for 19 challenging semantic categories. Sequences 00-07 and 09-10 were used for training, and sequence 08 served as the validation set.

[0096] In the experiments, Mean Intersection over Union (MIoU), number of parameters, and inference speed (Frames per second, FPS) were used as metrics for accuracy and model complexity. These three evaluation metrics are the main standard metrics in the current field of point cloud semantic segmentation.

[0097] The experiments were conducted on a computer equipped with two NVIDIA RTX 3090 GPUs. During training, a batch size of 4 was used, and the network was trained with an initial learning rate of 8e-4, dynamically adjusted by the OneCycle scheduler. For optimization, AdamW was used as the default optimizer, with weight decay set to 1e-2. For SemanticKITTI, the network was trained for 50 epochs, and the projected 2D pseudo-image shape was set to 64×512.

[0098] Table 1 compares the mIoU (%) results with other distance-image-based LiDAR segmentation methods on the SemanticKITTI test set. The best results are marked in bold, and the second-best results are marked in underline.

[0099]

[0100] Table 1 shows a comparison between the present invention and representative models on the SemanticKITTI test set (sequences 11-21). The present invention's method achieves state-of-the-art performance in mIoU scores. Although the present invention's method does not maintain a leading position in every category, it consistently maintains above-average scores in most categories. Specifically, the present invention's method performs well in 12 categories and poorly in 8 categories. Among these, the highest IoU scores were achieved in categories such as cars, bicycles, trains, parking lots, and buildings. This indicates that the present invention's method has a significant advantage in handling categories with clear structures, large distance variations, and ambiguous boundaries and complex geometries.

[0101] This invention achieves significant improvements in categories such as bicycles, trains, and buses. These categories typically share common characteristics such as large spatial span, elongated or blurred boundaries, and sparse point clouds, which places higher demands on the model's local feature modeling and distance perception. The point-level segmentation head proposed in this invention enhances geometric discrimination capabilities at a point-by-point scale, while the point-by-point distance encoding mechanism provides continuous depth information to strengthen distance perception. The combination of these two effectively mitigates the geometric information loss caused by projection and sparse sampling, thereby improving the segmentation performance of the aforementioned categories. Although the overall segmentation accuracy of this invention is slightly lower than Range Former, it has more significant advantages in terms of parameter count and inference speed. Overall, this invention achieves a good balance between accuracy and efficiency, demonstrating strong potential for practical applications.

[0102] To more intuitively demonstrate the performance improvement of the proposed method compared to the baseline model, a qualitative comparison was performed on the SemanticKITTI validation set by visualizing the prediction results. As shown in Figure 4, the prediction results and error point prediction results for four scenarios are displayed intuitively, and the proposed method is compared and analyzed with FRNet. Figure 4 shows the qualitative analysis of the SemanticKITTI validation set, where: (a) the true value; (b) the prediction result of the proposed method; (c) the prediction result of FRNet; (d) the error point prediction result of the proposed method; and (e) the error point prediction result of FRNet. Experimental results show that the proposed method has a significant advantage in reducing error prediction points, and the prediction results are closer to the true value. Especially in categories with blurred boundaries, sparse point clouds, and large spatial spans, such as fences and buses, the proposed method exhibits higher prediction accuracy. This qualitative analysis further verifies the effectiveness and advancement of the proposed model in structural modeling and semantic representation.

[0103] To quantitatively evaluate the effectiveness of different components, the method of this invention underwent a series of ablation experiments on the SemanticKITTI validation set. For a fair comparison, FRNet was chosen as the baseline method because of its similarity to the method of this invention in network structure. Based on this, the proposed point-level segmentation head module and point-to-point distance encoding mechanism were gradually introduced to verify their respective contributions.

[0104] Table 2 Evaluation of ablation studies on the SemanticKITTI validation set

[0105]

[0106] As shown in the second row of Table 2, the baseline method FRNet achieved 65.1% mIoU and 73.5% mAcc. Then, the contribution of the point-level segmentation head module was evaluated. As shown in the third row of Table 2, after adding this module, the model's mIoU and mAcc significantly improved to 66.0% and 73.8%, respectively. This result indicates that the point-level segmentation head module can effectively guide the model to utilize feature information from different levels more specifically, thereby improving point-by-point prediction capability. Finally, as shown in the fourth row of Table 2, the overall performance of the model was further improved after introducing the point-by-point distance encoding mechanism. Compared with the baseline method, the final model's mIoU and mAcc improved by 2.1% and 0.7%, respectively. In summary, the ablation experiments clearly demonstrate that the proposed point-level segmentation head module and the point-by-point distance encoding mechanism are key components of the model. The gradual addition of each component brought significant performance improvements, ultimately proving the effectiveness and rationality of the method presented in this invention.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A semantic segmentation method for lidar based on point-to-point distance coding and point-level segmentation head, characterized in that, Includes the following steps: S1. Acquire the raw 3D point cloud data collected by the lidar. The raw 3D point cloud data contains multiple raw points, each with raw features. S2. According to the spherical coordinate projection formula, project each original point to the corresponding pixel position of the two-dimensional distance image; for multiple original points projected to the same pixel position, based on their original features... The points are aggregated at the pixel location to generate the two-dimensional distance image, and the mapping relationship between each original point and the pixel it is projected onto is recorded; S3, for each original point, the Euclidean distance from the point to the origin of the lidar coordinate system is calculated. Then, normalization is performed to obtain normalized deep features. The pixel position information projected onto this point in S2 is used as the projection feature. Then normalize the depth features of this point. Projection features and original features Cascaded splicing is performed to form enhanced point features. ; Utilizing the enhanced point features As a condition, adaptively generate the enhanced point features. A dimensionally consistent attention weight vector G is then applied to the original features. Channel-level modulation generates modulated point features. = G， S4 represents element-wise multiplication; S3 modulates the point features of each original point generated in S3. S2 fills the corresponding pixel positions in the two-dimensional distance image according to the mapping relationship recorded in S2, forming a two-dimensional feature map; and inputs the two-dimensional feature map into the feature extraction network to extract multi-level two-dimensional pixel features; S5, based on the mapping relationship recorded in S2, maps the two-dimensional pixel features of different levels obtained in S4 to their corresponding original points, and combines them with the corresponding modulated point features formed in S3. Perform interactive fusion to obtain multi-level enhanced 3D point features; S6, fuse the multi-level enhanced 3D point features obtained in S5, including: introducing learnable weight coefficients for each level of features and performing adaptive fusion by weighted summation; and predicting the semantic category of each original point point by point based on the fused features.

2. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S3, a deep perception gating mechanism is used, based on enhanced point features. The attention weight vector G is generated as a conditional input.

3. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 2, characterized in that: The process of generating the attention weight vector G includes: [the process of] generating the enhancement point features... The input is fed into a perceptron structure consisting of two fully connected layers and an activation function to generate an attention weight vector: In the formula, 、 For learnable parameters, the C in the superscript represents the original feature. The number of channels, where H in the upper right corner indicates the dimension of the hidden layer; This is the activation function.

4. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S6, the adaptive fusion adopts a bottom-up residual fusion path, progressively fusing multi-level enhanced 3D point features in hierarchical order, where the fused features of the i-th layer... Determined by the following formula: In the formula, This indicates that dimensionality processing is performed using a multilayer perceptron. Represents the fusion features of the (i-1)th layer; This represents the enhanced 3D point features of the i-th layer; Let be the learnable weight coefficients of the i-th layer.

5. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 4, characterized in that: The learnable weight coefficients The activation function is restricted to the range [0, 1] to ensure its numerical stability and controllability.

6. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S5, the interactive fusion is a multi-stage bidirectional fusion, which includes: performing two-dimensional to three-dimensional feature mapping and fusion at multiple levels, and aggregating the fused three-dimensional point features back to the corresponding pixels to update the two-dimensional representation.

7. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S3, the normalization process is performed as follows: the calculated Euclidean distance is... The clamp function limits the distance to a preset range. The data is then linearly normalized to obtain normalized deep features. 。 8. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 7, characterized in that: The preset distance range for The unit is meters.

9. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S4, the feature extraction network is a convolutional neural network with a residual structure.

10. The lidar semantic segmentation method based on point-to-point distance encoding and point-level segmentation head according to claim 1, characterized in that: In S2, for multiple original points projected to the same pixel location, the aggregation operation is to aggregate the original features of the multiple original points falling into the same pixel. Take the average, maximum, or weighted sum.

Images