Point cloud segmentation method based on hierarchical adaptive local and global context fusion

Abstract

The application discloses a hierarchical adaptive local and global context fusion point cloud segmentation method, solves the problems that the existing point-based point cloud semantic segmentation method performs poorly when processing objects or structures with significant scale differences, and that in local feature learning, the discrimination is insufficient, the receptive field is limited and it is difficult to associate long-distance structures, and global context modeling is difficult to apply to large-scale point clouds due to high computational complexity. The hierarchical adaptive local and global context fusion point cloud segmentation method is proposed, multi-head feature extraction, multi-head inter-neighbor context aggregation and adaptive sparse global fusion are used to realize local geometric expression and global structure perception, thereby significantly improving the accuracy and robustness of point cloud semantic segmentation in complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing technology. Specifically, this invention relates to a point cloud segmentation method based on hierarchical adaptive local and global context fusion. Background Technology

[0002] Point cloud semantic segmentation aims to assign semantic labels to each point in a 3D point cloud, and is a key technology for scene understanding in autonomous driving, remote sensing monitoring, and other applications. However, the sparsity and irregularity of point cloud data, as well as the significant differences in the scale of objects in the scene, pose challenges to accurate segmentation.

[0003] With the introduction of deep learning methods, point-based methods have become popular because they can directly process raw point clouds without complex preprocessing or post-processing steps. However, such methods still face the following limitations: traditional single-scale local feature extraction methods, such as KNN with a fixed K value or sphere lookup with a fixed radius, have limited receptive fields and are difficult to adapt to structures with significant scale differences, and biases in the distribution of local features lead to insufficient discriminative power.

[0004] Meanwhile, KNN and ball query exhibit randomness in neighborhood construction, easily generating overlapping regions, but lack effective integration of multi-scale overlapping information, limiting the full extraction of local context. While the introduction of attention mechanisms can improve large-scene segmentation performance, the computational complexity of its self-attention mechanism increases quadratically with the number of points, restricting its application on large-scale point clouds.

[0005] At the same time, while focusing on modeling relative positional relationships to reduce computational burden, the effective capture of global contextual information is often overlooked. Summary of the Invention

[0006] This invention provides a point cloud segmentation method based on hierarchical adaptive local and global context fusion to solve the technical problems existing in the background art.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a hierarchical adaptive local and global context fusion point cloud segmentation method, specifically including the following steps:

[0008] Step S1: Point cloud data input and feature initialization; The original point cloud with unified data format and feature dimensions is used as input, represented as... The form of dimension; in which, The number of points is 3, where 3 corresponds to the three-dimensional coordinates of each point. , Indicates the feature dimensions other than coordinates;

[0009] Step S2: Local geometric feature enhancement and normalization;

[0010] Step S3: Multi-scale neighborhood context aggregation; this is achieved by deploying four independent processing heads in parallel, each with a different number of neighbors. To construct multi-scale receptive fields and gradually expand the perception range of local features;

[0011] At the same time, only the geometric properties of the local neighborhood are considered to be close to the center point to minimize interference; fine-grained neighborhood information is aggregated to a coarse-grained level through scattering operations, and then local global context information is propagated back to the downsampling point through indexing operations to achieve cross-level context awareness.

[0012] Step S4: Adaptive sparse global fusion. Representative global key points are adaptively selected through adaptive sparse global fusion, and a position-enhanced self-attention mechanism is introduced to significantly reduce computational complexity while efficiently aggregating long-range context information.

[0013] Step S5: Feature Decoding and Semantic Prediction. The decoder receives the deepest output features from the encoder and gradually restores them to the original number of points through four nearest neighbor interpolation operations.

[0014] Preferably, step S1 specifically includes the following steps:

[0015] Step S10: First, input the raw point cloud into the encoder network, and then pass it through a multilayer perceptron. The features of each point are mapped to 32 dimensions, which serve as the initial representation for subsequent hierarchical feature extraction;

[0016] Step S11: Original features of the center point obtained by sampling from the farthest point with location features ;

[0017] Step S12: Utilize Divide the local region to generate a local region for each center point The area has Composed of points, the semantic features of its neighbor set are represented as: Location features are represented as .

[0018] Preferably, step S2 specifically includes the following steps:

[0019] Step S20: Based on the center point and Neighbor set The local relative coordinates are calculated as position information, and the formula is expressed as:

[0020] (1)

[0021] in: The three-dimensional coordinate of the center point. Indicates the center point The The three-dimensional coordinates of the neighboring points This provides local relative position coordinate information, used to encode geometric positional relationships within a local neighborhood. This indicates the number of neighboring points selected when constructing the local neighborhood.

[0022] Step S21: Considering the sparsity and irregular geometric structure of the local region, directly learn the semantic features of the local region. It will reduce accuracy and stability;

[0023] To enhance the discriminative power and robustness of semantic features, multidimensional semantic centroids of local regions are further extracted. The calculation formula is as follows:

[0024] (2)

[0025] in: It is a semantic feature of a local region. It is the multidimensional semantic center of gravity of a local region;

[0026] Calculate the sample variance The formula is as follows:

[0027] (3)

[0028] in: This represents the total number of current input points. This indicates the number of neighboring points selected when constructing the local neighborhood. The channel dimension representing the feature;

[0029] Obtain the adjusted semantic features , means as follows:

[0030] (4)

[0031] in: These are learnable parameters. It is a numerically stable term;

[0032] Step S22: Transfer local relative position coordinate information With adjusted semantic features The fusion process yields an enhanced local feature representation. This process can be represented as:

[0033] (5)

[0034] in: express , This indicates element-wise addition.

[0035] Preferably, step S3 specifically includes the following steps:

[0036] Step S30: First, multiple independent processing heads, each with a different neighborhood size, are used to capture multi-scale local features to group the resulting neighborhood points. For example;

[0037] In the obtained local region, if the first The point is the center point. Local feature extraction is achieved by aggregating features of the neighbors of the central point; its features The following was obtained:

[0038] (6)

[0039] in: , For max pooling, This yields the aggregated local features;

[0040] Step S31: To consider broader neighbor relationships, the center point of a neighbor , It is also a neighbor point of other neighbors. , ; Utilizing this overlapping phenomenon to fuse different situations Features of each point;

[0041] Therefore, when the first When the nth point is a neighboring point, it will include the nth point. The neighborhood set of a point is defined as: , That is, all points are both the center point and neighboring points of other local regions, and then they are reshaped into... This facilitates subsequent aggregation;

[0042] Step S32: Find All containing the first The original index of the neighborhood set of each point , ;

[0043] Since the corresponding features of the same point differ in different local regions, a scattering operation is performed based on the original index to obtain the maximum feature corresponding to each original point. This refers to extended contextual features, as shown in the following formula:

[0044] (7)

[0045] in: For all neighbor characteristics, Original point index; this feature is obtained by aggregating all neighboring points in the current layer;

[0046] Step S33: Now, only the extended context features of the center point (yellow point) in the current layer are needed, therefore the corresponding index obtained when sampling from the farthest point is used. , The formula for obtaining the center point of the current layer is as follows:

[0047] (8)

[0048] in: Extended context features for the current layer center point;

[0049] Aggregated local features and extended context features The fusion process is as follows:

[0050] (9)

[0051] in: For the fused local context features, For cascading operations;

[0052] Step S34: To ensure that each head takes into account local features of different scales, the search range of each head's neighbors is set to twice the search range of the previous head's neighbors; finally, a comprehensive feature representation that integrates multi-scale local context information is obtained. The formula is as follows:

[0053] (10)

[0054] In this way, each point can not only perceive its own local neighborhood, but also the contextual information when it is a neighboring point of other regions in the entire point cloud.

[0055] Preferably, step S4 specifically includes the following steps:

[0056] Step S40: For each point in the input point cloud Features First, use a linear layer. and Activation function Will Mapped to importance score In Chinese, it is represented as follows:

[0057] (11)

[0058] Score This is used to evaluate the contribution of each point to the global context; then, the point with the highest score is selected. Each point is used as a global keypoint, and its corresponding feature is obtained based on the keypoint index. and absolute position coordinates in space ;

[0059] Step S41: Subsequently, the key point features are... The input to the self-attention module is processed through linear layers. Mapped to query vector Key vector Sum value vector ;

[0060] (12)

[0061] in: This represents a linear mapping, which maps the query vector. and the transposed key vector Matrix multiplication yields the feature similarity matrix. The calculation formula is as follows:

[0062] (14)

[0063] Step S42: To enhance the model's ability to perceive geometric structures, utilize the original coordinates of key points. go through Encoding as positional encoding The calculation formula is as follows:

[0064] (15)

[0065] Step S43: Use additive fusion to combine feature similarity and location information, i.e., from feature similarity... Add position encoding Obtain attention weights This encourages the model to focus on point pairs that are spatially different but have similar features, thus better capturing global dependencies. The calculation formula is as follows:

[0066] (15)

[0067] in, The aggregated keypoint features are obtained. The calculation formula is as follows:

[0068] (16)

[0069] Step S44: Enhance the keypoint features Based on the spatial nearest neighbor principle, the propagation is applied to every point in the original point cloud to generate features containing global scene information. ; Combine the global features with the local features Perform adaptive fusion to obtain a more discriminative hybrid representation. The formula is as follows:

[0070] (17)

[0071] in: yes function, These are learnable parameters used to dynamically adjust the fusion ratio of local and global features;

[0072] Step S45: Finally, generate a comprehensive feature that integrates local details and global context. This enables each point to simultaneously perceive the local details of its neighboring region and the cross-regional context when it is a distant neighbor point in the entire point cloud.

[0073] Preferably, step S5 specifically includes the following steps:

[0074] Step S50: The interpolation adopts the inverse distance weighting method, and the features are reconstructed based on the three nearest neighbors of each target point; after each interpolation, the interpolated features of the current layer are concatenated with the features of the corresponding layer of the encoder through skip connections, thereby fusing multi-level semantic and geometric information;

[0075] Step S51: The cascaded features are all refined by a multilayer perceptron. The number of input and output channels of each layer of the decoder are as follows: 512 → 256, 256 → 128, 128 → 64, 64 → 32.

[0076] Step S52: The segmentation head maps the 32-dimensional features of each point to the semantic category space through a linear layer, and assigns corresponding semantic labels to all points.

[0077] Preferably, the encoder network comprises four layers; in each layer, the number of feature channels gradually increases, specifically configured as follows:

[0078] The first layer has 32-dimensional input and 64-dimensional output.

[0079] The second layer has 64-dimensional input and 128-dimensional output.

[0080] The third layer has 128 dimensions for input and 256 dimensions for output.

[0081] The fourth layer has 256 input dimensions and 512 output dimensions.

[0082] Meanwhile, the number of input points is gradually reduced layer by layer through sampling from the farthest point, successively decreasing to 1 / 4, 1 / 16, 1 / 64 and 1 / 256 of the original number of points N, forming a hierarchical feature abstraction structure.

[0083] The beneficial effects of adopting the above technical solutions are:

[0084] 1. The hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention, compared with the existing semantic segmentation methods, alleviates the local feature distribution deviation and enhances the geometric feature discrimination power and stability by introducing relative position encoding and semantic centroid standardization.

[0085] 2. The hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention, compared with the existing semantic segmentation methods, integrates the context of overlapping neighborhoods through parallel multi-scale perception and cross-layer information propagation, alleviates feature interference between distant structures, and thus improves the perception capability of complex scenes.

[0086] 3. The hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention, compared with the existing semantic segmentation methods, is based on importance scoring to select key points and combined with position-enhanced attention mechanism to efficiently model long-range dependencies in point clouds with low computational complexity, and realize the effective propagation of global context information in point clouds. Attached Figure Description

[0087] Figure 1 It is a network structure diagram;

[0088] Figure 2 It is a flowchart of hierarchical adaptive local and global fusion;

[0089] Figure 3 This is a diagram of a multi-head feature extraction scheme;

[0090] Figure 4 This is a diagram of a multi-head neighbor context aggregation scheme;

[0091] Figure 5 This is a diagram of an adaptive sparse global fusion scheme;

[0092] Figure 6 This is Visual Example 1;

[0093] Figure 7 This is Visual Example 2; Detailed Implementation

[0094] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, in order to help those skilled in the art to have a more complete, accurate and in-depth understanding of the concept and technical solutions of the present invention, and to facilitate its implementation.

[0095] like Figures 1 to 7 As shown, the hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention aims to solve the problems of existing point cloud semantic segmentation methods, such as poor performance when dealing with objects or structures with significant scale differences, insufficient discriminative power in local feature learning, limited receptive field making it difficult to associate with distant structures, and high computational complexity of global context modeling making it difficult to apply to large-scale point clouds.

[0096] Specifically, such as Figures 1 to 7 As shown, the hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention achieves local geometric representation and global structure perception through multi-head feature extraction, multi-head neighbor context aggregation and adaptive sparse global fusion, thereby significantly improving the accuracy and robustness of point cloud semantic segmentation in complex scenes.

[0097] The specific working method is described below using specific embodiments:

[0098] Example 1:

[0099] The hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention is illustrated in the following architecture diagram: Figure 1 As shown, it includes an encoder and a decoder.

[0100] First, initial feature mapping is performed on the input raw 3D point cloud, and then the feature mapping is performed through farthest point sampling and... Divide the local neighborhood.

[0101] Then, for each local neighborhood, its relative position code is calculated, and the semantic features within the neighborhood are standardized and adjusted based on the centroid distribution. The two are then fused to enhance the discriminative power and robustness of the local geometric features.

[0102] Then, by using cross-level scattering and aggregation operations, the contextual information of the overlapping neighborhoods when they are neighbors is integrated, and local features at various scales are aggregated to form a multi-scale fused local feature representation.

[0103] Then, a set of global key points is adaptively selected based on feature importance scores. A position-enhanced attention mechanism is introduced on these key points to efficiently model long-range dependencies in the point cloud with low computational complexity, and the obtained global context information is propagated back to all points.

[0104] Subsequently, through learnable adaptive weights, the enhanced local features and global features are dynamically fused to form a unified and robust point feature representation.

[0105] The aforementioned feature extraction and fusion process is performed in multiple stacked layers within the encoder, gradually abstracting high-dimensional semantic features. In the decoding stage, the number of points is gradually recovered through interpolation and upsampling, and features from the corresponding levels of the encoder are combined using skip connections for fusion to supplement spatial details.

[0106] Finally, the classifier outputs the corresponding semantic category label for each point, completing the end-to-end prediction from point cloud input to semantic segmentation result.

[0107] Example 2:

[0108] Based on Example 1, the hierarchical adaptive local and global context fusion point cloud segmentation method of the present invention is illustrated in the flowchart below. Figure 2 As shown, its innovation mainly lies in the encoder part. The method mainly includes point cloud data input and feature initialization, local geometric feature enhancement and standardization, multi-scale neighborhood context aggregation, adaptive sparse global fusion, and feature decoding and semantic prediction.

[0109] Step S1: Point cloud data input and feature initialization; The original point cloud with unified data format and feature dimensions is used as input, represented as... The form of dimension; in which, The number of points is 3, where 3 corresponds to the three-dimensional coordinates of each point. , Indicates the feature dimensions other than coordinates;

[0110] The specific steps of step S1 are as follows:

[0111] Step S10: First, input the raw point cloud into the encoder network, and then pass it through a multilayer perceptron. The features of each point are mapped to 32 dimensions, which serve as the initial representation for subsequent hierarchical feature extraction;

[0112] Step S11: Original features of the center point obtained by sampling from the farthest point with location features ;

[0113] Step S12: Utilize Divide the local region to generate a local region for each center point The area has Composed of points, the semantic features of its neighbor set are represented as: Location features are represented as .

[0114] Step S2: Local geometric feature enhancement and normalization; such as Figure 3 As shown,

[0115] The specific steps of step S2 are as follows:

[0116] Step S20: Based on the center point and Neighbor set The local relative coordinates are calculated as position information, and the formula is expressed as:

[0117] (1)

[0118] in: The three-dimensional coordinate of the center point. Indicates the center point The The three-dimensional coordinates of the neighboring points This provides local relative position coordinate information, used to encode geometric positional relationships within a local neighborhood. This indicates the number of neighboring points selected when constructing the local neighborhood.

[0119] Step S21: Considering the sparsity and irregular geometric structure of the local region, directly learn the semantic features of the local region. It will reduce accuracy and stability;

[0120] To enhance the discriminative power and robustness of semantic features, multidimensional semantic centroids of local regions are further extracted. The calculation formula is as follows:

[0121] (2)

[0122] in: It is a semantic feature of a local region. It is the multidimensional semantic center of gravity of a local region;

[0123] Calculate the sample variance The formula is as follows:

[0124] (3)

[0125] in: This represents the total number of current input points. This indicates the number of neighboring points selected when constructing the local neighborhood. The channel dimension representing the feature;

[0126] Obtain the adjusted semantic features , means as follows:

[0127] (4)

[0128] in: These are learnable parameters. It is a numerically stable term;

[0129] Step S22: Transfer local relative position coordinate information With adjusted semantic features The fusion process yields an enhanced local feature representation. This process can be represented as:

[0130] (5)

[0131] in: express , This indicates element-wise addition.

[0132] Step S3: Multi-scale neighborhood context aggregation; such as Figure 4 As shown, due to Due to the randomness of ball queries, the constructed neighbors often produce overlapping parts. While expanding the receptive field, this also merges distant local geometric structures, leading to mutual interference of geometric contexts. To address this, the present invention deploys four independent processing heads in parallel, each using a different number of neighbors. To construct multi-scale receptive fields and gradually expand the perception range of local features;

[0133] At the same time, only the geometric properties of the local neighborhood are considered to be close to the center point to minimize interference; fine-grained neighborhood information is aggregated to a coarse-grained level through scattering operations, and then local global context information is propagated back to the downsampling point through indexing operations to achieve cross-level context awareness.

[0134] The specific steps of step S3 are as follows:

[0135] Step S30: First, multiple independent processing heads, each with a different neighborhood size, are used to capture multi-scale local features to group the resulting neighborhood points. For example;

[0136] In the obtained local region, if the first The point is the center point. Local feature extraction is achieved by aggregating features of the neighbors of the central point; its features The following was obtained:

[0137] (6)

[0138] in: , For max pooling, This yields the aggregated local features;

[0139] Step S31: To consider broader neighbor relationships, the center point of a neighbor , It is also a neighbor point of other neighbors. , ; Utilizing this overlapping phenomenon to fuse different situations Features of each point;

[0140] Therefore, when the first When the nth point is a neighboring point, it will include the nth point. The neighborhood set of a point is defined as: , That is, all points are both the center point and neighboring points of other local regions, and then they are reshaped into... This facilitates subsequent aggregation;

[0141] Step S32: Find All containing the first The original index of the neighborhood set of each point , ;

[0142] Since the corresponding features of the same point differ in different local regions, a scattering operation is performed based on the original index to obtain the maximum feature corresponding to each original point. This refers to extended contextual features, as shown in the following formula:

[0143] (7)

[0144] in: For all neighbor characteristics, Original point index; this feature is obtained by aggregating all neighboring points in the current layer;

[0145] Step S33: Now, only the extended context features of the center point (yellow point) in the current layer are needed, therefore the corresponding index obtained when sampling from the farthest point is used. , The formula for obtaining the center point of the current layer is as follows:

[0146] (8)

[0147] in: Extended context features for the current layer center point;

[0148] Aggregated local features and extended context features The fusion process is as follows:

[0149] (9)

[0150] in: For the fused local context features, For cascading operations;

[0151] Step S34: To ensure that each head takes into account local features of different scales, the search range of each head's neighbors is set to twice the search range of the previous head's neighbors; finally, a comprehensive feature representation that integrates multi-scale local context information is obtained. The formula is as follows:

[0152] (10)

[0153] In this way, each point can not only perceive its own local neighborhood, but also the contextual information when it is a neighboring point of other regions in the entire point cloud.

[0154] Step S4: Adaptive sparse global fusion, such as Figure 5 As shown, while the aforementioned feature extraction and aggregation within local neighborhoods are effective, their receptive field remains limited, making it difficult to model the direct dependencies between points and distant structures. Therefore, this invention employs adaptive sparse global fusion to adaptively select representative global key points and introduces a position-enhanced self-attention mechanism. This significantly reduces computational complexity while efficiently aggregating long-range contextual information.

[0155] The specific steps of step S4 are as follows:

[0156] Step S40: For each point in the input point cloud Features First, use a linear layer. and Activation function Will Mapped to importance score In Chinese, it is represented as follows:

[0157] (11)

[0158] Score This is used to evaluate the contribution of each point to the global context; then, the point with the highest score is selected. Each point is used as a global keypoint, and its corresponding feature is obtained based on the keypoint index. and absolute position coordinates in space ;

[0159] Step S41: Subsequently, the key point features are... The input to the self-attention module is processed through linear layers. Mapped to query vector Key vector Sum value vector ;

[0160] (12)

[0161] in: This represents a linear mapping, which maps the query vector. and the transposed key vector Matrix multiplication yields the feature similarity matrix. The calculation formula is as follows:

[0162] (14)

[0163] Step S42: To enhance the model's ability to perceive geometric structures, utilize the original coordinates of key points. go through Encoding as positional encoding The calculation formula is as follows:

[0164] (15)

[0165] Step S43: Use additive fusion to combine feature similarity and location information, i.e., from feature similarity... Add position encoding Obtain attention weights This encourages the model to focus on point pairs that are spatially different but have similar features, thus better capturing global dependencies. The calculation formula is as follows:

[0166] (15)

[0167] in, The aggregated keypoint features are obtained. The calculation formula is as follows:

[0168] (16)

[0169] Step S44: Enhance the keypoint features Based on the spatial nearest neighbor principle, the propagation is applied to every point in the original point cloud to generate features containing global scene information. ; Combine the global features with the local features Perform adaptive fusion to obtain a more discriminative hybrid representation. The formula is as follows:

[0170] (17)

[0171] in: yes function, These are learnable parameters used to dynamically adjust the fusion ratio of local and global features;

[0172] Step S45: Finally, generate a comprehensive feature that integrates local details and global context. This enables each point to simultaneously perceive the local details of its neighboring region and the cross-regional context when it is a distant neighbor point in the entire point cloud.

[0173] Step S5: Feature Decoding and Semantic Prediction. The decoder receives the deepest output features from the encoder and gradually restores them to the original number of points through four nearest neighbor interpolation operations.

[0174] The specific steps of step S5 are as follows:

[0175] Step S50: The interpolation adopts the inverse distance weighting method, and the features are reconstructed based on the three nearest neighbors of each target point; after each interpolation, the interpolated features of the current layer are concatenated with the features of the corresponding layer of the encoder through skip connections, thereby fusing multi-level semantic and geometric information;

[0176] Step S51: The cascaded features are all refined by a multilayer perceptron. The number of input and output channels of each layer of the decoder are as follows: 512 → 256, 256 → 128, 128 → 64, 64 → 32.

[0177] Step S52: The segmentation head maps the 32-dimensional features of each point to the semantic category space through a linear layer, and assigns corresponding semantic labels to all points.

[0178] Example 3:

[0179] Based on Example 2, the encoder network comprises four layers; in each layer, the number of feature channels gradually increases, specifically configured as follows:

[0180] The first layer has 32-dimensional input and 64-dimensional output.

[0181] The second layer has 64-dimensional input and 128-dimensional output.

[0182] The third layer has 128 dimensions for input and 256 dimensions for output.

[0183] The fourth layer has 256 input dimensions and 512 output dimensions.

[0184] Meanwhile, the number of input points is gradually reduced layer by layer through sampling from the farthest point, successively decreasing to 1 / 4, 1 / 16, 1 / 64 and 1 / 256 of the original number of points N, forming a hierarchical feature abstraction structure.

[0185] Example 4:

[0186] Building upon Example 1, during the visualization phase, the input point cloud retains its 3D coordinates and color information. Both the ground truth labels and the model prediction results are converted into colored point clouds according to a pre-defined category color mapping table in the dataset, where each semantic category corresponds to a specific color. By comparing the ground truth colored and the predicted colored point clouds, the segmentation accuracy and boundary preservation capability of the model in different semantic regions can be intuitively evaluated.

[0187] The training and testing of the hierarchical adaptive local and global context fusion point cloud segmentation method of this invention were both completed on a computing platform equipped with an NVIDIA GeForce RTX 4090 graphics processor (24 GB video memory) and a 13th generation Intel® Core™ i9-13900KF processor.

[0188] The semantic segmentation results of two indoor scenes based on the S3DIS dataset are as follows: Figure 6 As shown in the figure, the segmentation results at the object boundaries are complete and accurate, effectively distinguishing their shapes. This invention, through hierarchical fusion of local and global features, effectively enhances the model's ability to perceive complex structures and preserve details, achieving precise semantic segmentation.

[0189] The semantic segmentation results for two indoor scenes based on the ScanNetV2 dataset are as follows: Figure 7 As shown, the segmentation result is closer to the baseline, accurately segmenting the entire door outline, while also segmenting two cabinets that are far apart. This is because the present invention effectively integrates global information, enabling the correct identification of distant objects or occluded parts, avoiding misjudgments caused by the limited receptive field of local methods. This intuitively demonstrates that our method has good segmentation results in complex scenes.

[0190] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A point cloud segmentation method based on hierarchical adaptive local and global context fusion, characterized in that: Specifically, the following steps are included: Step S1: Point cloud data input and feature initialization; The original point cloud with unified data format and feature dimensions is used as input, represented as... The form of dimension; in which, The number of points is 3, where 3 corresponds to the three-dimensional coordinates of each point. , Indicates the feature dimensions other than coordinates; Step S2: Local geometric feature enhancement and normalization; Step S3: Multi-scale neighborhood context aggregation; this is achieved by deploying four independent processing heads in parallel, each with a different number of neighbors. To construct multi-scale receptive fields and gradually expand the perception range of local features; At the same time, only the geometric properties of the local neighborhood are considered to be close to the center point, so as to minimize interference. The fine-grained neighborhood information is aggregated to the coarse-grained level through scattering operation, and the local global context information is propagated back to the downsampling point through indexing operation, so as to achieve cross-level context awareness. Step S4: Adaptive sparse global fusion. Representative global key points are adaptively selected through adaptive sparse global fusion, and a position-enhanced self-attention mechanism is introduced to significantly reduce computational complexity while efficiently aggregating long-range context information. Step S5: Feature Decoding and Semantic Prediction. The decoder receives the deepest output features from the encoder and gradually restores them to the original number of points through four nearest neighbor interpolation operations.

2. The point cloud segmentation method based on hierarchical adaptive local and global context fusion as described in claim 1, characterized in that: The specific steps of step S1 are as follows: Step S10: First, input the raw point cloud into the encoder network, and then pass it through a multilayer perceptron. The features of each point are mapped to 32 dimensions, which serve as the initial representation for subsequent hierarchical feature extraction; Step S11: Original features of the center point obtained by sampling from the farthest point with location features ; Step S12: Utilize Divide the local region to generate a local region for each center point The area has Composed of points, the semantic features of its neighbor set are represented as: Location features are represented as .

3. The point cloud segmentation method based on hierarchical adaptive local and global context fusion according to claim 1, characterized in that: The specific steps of step S2 are as follows: Step S20: Based on the center point and Neighbor set The local relative coordinates are calculated as position information, and the formula is expressed as: （1） in: The three-dimensional coordinate of the center point is represented by... Indicates the center point The The three-dimensional coordinates of a neighboring point This provides local relative position coordinate information, used to encode geometric positional relationships within a local neighborhood. This indicates the number of neighboring points selected when constructing the local neighborhood; Step S21: Considering the sparsity and irregular geometric structure of the local region, directly learn the semantic features of the local region. It will reduce accuracy and stability; To enhance the discriminative power and robustness of semantic features, multidimensional semantic centroids of local regions are further extracted. The calculation formula is as follows: （2） in: It is a semantic feature of a local region. It is the multidimensional semantic center of gravity of a local region; Calculate the sample variance The formula is as follows: （3） in: This represents the total number of current input points. This indicates the number of neighboring points selected when constructing the local neighborhood. The channel dimension representing the feature; Obtain the adjusted semantic features , means as follows: （4） in: These are learnable parameters. It is a numerically stable term; Step S22: Transfer local relative position coordinate information With adjusted semantic features The fusion process yields an enhanced local feature representation. This process can be represented as: （5） in: express , This indicates element-wise addition.

4. The point cloud segmentation method based on hierarchical adaptive local and global context fusion according to claim 3, characterized in that: The specific steps of step S3 are as follows: Step S30: First, multiple independent processing heads, each with a different neighborhood size, are used to capture multi-scale local features to group the resulting neighborhood points. For example; In the obtained local region, if the first The point is the center point. Local feature extraction is achieved by aggregating features of the neighbors of the center point; Its characteristics The following was obtained: （6） in: , For max pooling, This yields the aggregated local features; Step S31: To consider broader neighbor relationships, the center point of a neighbor , It is also a neighbor point of other neighbors. , ; Utilizing this overlapping phenomenon to fuse different situations Features of each point; Therefore, when the first When the nth point is a neighboring point, it will include the nth point. The neighborhood set of a point is defined as: , That is, all points are both the center point and neighboring points of other local regions, and then they are reshaped into... This facilitates subsequent aggregation; Step S32: Find All containing the first The original index of the neighborhood set of each point , ; Since the corresponding features of the same point differ in different local regions, a scattering operation is performed based on the original index to obtain the maximum feature corresponding to each original point. This refers to extended contextual features, as shown in the following formula: （7） in: For all neighbor characteristics, Original point index; this feature is obtained by aggregating all neighboring points in the current layer; Step S33: Now, only the extended context features of the center point (yellow point) in the current layer are needed, therefore the corresponding index obtained when sampling from the farthest point is used. , The formula for obtaining the center point of the current layer is as follows: （8） in: Extended context features for the current layer center point; Aggregated local features and extended context features The fusion process is as follows: （9） in: For the fused local context features, For cascading operations; Step S34: To ensure that each head takes into account local features of different scales, the search range of each head's neighbors is set to twice the search range of the previous head's neighbors; finally, a comprehensive feature representation that integrates multi-scale local context information is obtained. The formula is as follows: （10） In this way, each point can not only perceive its own local neighborhood, but also the contextual information when it is a neighboring point of other regions in the entire point cloud.

5. The point cloud segmentation method based on hierarchical adaptive local and global context fusion according to claim 2, characterized in that: The specific steps of step S4 are as follows: Step S40: For each point in the input point cloud Features First, use a linear layer. and Activation function Will Mapped to importance score In Chinese, it is represented as follows: （11） Score This is used to evaluate the contribution of each point to the global context; then, the point with the highest score is selected. Each point is used as a global keypoint, and its corresponding feature is obtained based on the keypoint index. and absolute position coordinates in space ; Step S41: Subsequently, the key point features are... The input to the self-attention module is processed through linear layers. Mapped to query vector Key vector Sum value vector ; （12） in: This represents a linear mapping, which maps the query vector. and the transposed key vector Matrix multiplication yields the feature similarity matrix. The calculation formula is as follows: （14） Step S42: To enhance the model's ability to perceive geometric structures, utilize the original coordinates of key points. go through Encoding as positional encoding The calculation formula is as follows: （15） Step S43: Use additive fusion to combine feature similarity and location information, i.e., from feature similarity... Add position encoding Obtain attention weights This encourages the model to focus on point pairs that are spatially different but have similar features, thus better capturing global dependencies. The calculation formula is as follows: （15） in, The aggregated keypoint features are obtained. The calculation formula is as follows: （16） Step S44: Enhance the keypoint features Based on the spatial nearest neighbor principle, the propagation is applied to every point in the original point cloud to generate features containing global scene information. ; Combine the global features with the local features Perform adaptive fusion to obtain a more discriminative hybrid representation. The formula is as follows: （17） in: yes function, These are learnable parameters used to dynamically adjust the fusion ratio of local and global features; Step S45: Finally, generate a comprehensive feature that integrates local details and global context. This enables each point to simultaneously perceive the local details of its neighboring region and the cross-regional context when it is a distant neighbor point in the entire point cloud.

6. The point cloud segmentation method based on hierarchical adaptive local and global context fusion according to claim 1, characterized in that: The specific steps of step S5 are as follows: Step S50: The interpolation adopts the inverse distance weighting method, and the features are reconstructed based on the three nearest neighbors of each target point; after each interpolation, the interpolated features of the current layer are concatenated with the features of the corresponding layer of the encoder through skip connections, thereby fusing multi-level semantic and geometric information; Step S51: The cascaded features are all refined by a multilayer perceptron. The number of input and output channels of each layer of the decoder are as follows: 512 → 256, 256 → 128, 128 → 64, 64 → 32. Step S52: The segmentation head maps the 32-dimensional features of each point to the semantic category space through a linear layer, and assigns corresponding semantic labels to all points.

7. The point cloud segmentation method based on hierarchical adaptive local and global context fusion according to claim 1, characterized in that: The encoder network comprises four layers; in each layer, the number of feature channels gradually increases, specifically configured as follows: The first layer has 32-dimensional input and 64-dimensional output. The second layer has 64-dimensional input and 128-dimensional output. The third layer has 128 dimensions for input and 256 dimensions for output. The fourth layer has 256 input dimensions and 512 output dimensions. Meanwhile, the number of input points is gradually reduced layer by layer through sampling from the farthest point, successively decreasing to 1 / 4, 1 / 16, 1 / 64 and 1 / 256 of the original number of points N, forming a hierarchical feature abstraction structure.

Images