Point cloud region segmentation method based on high-spectrum enhancement

Abstract

This invention relates to the fields of computer vision and 3D point cloud processing technology, and addresses the problem of low-pass spectral bias in existing Mamba-based state-space models when processing serialized point clouds. This low-pass bias leads to the suppression of high-frequency geometric structures and the dilution of deep semantic discriminative ability. The proposed method maintains geometric fidelity and semantic consistency through the synergistic effect of Gaussian eigenvalue decomposition (GSC) and semantic eigenvalue recalibration (SCR). GSC injects high-frequency components guided by local Laplacian analysis to restore structural sensitivity and offset low-pass bias. SCR corrects semantic shifts through frequency-aware channel recalibration and provides two approaches: accurate Laplacian eigenvalue decomposition and linearly complex Chebyshev polynomial approximation. This invention achieves a good accuracy-efficiency tradeoff, demonstrating the importance of addressing spectral bias and semantic dilution in SSM-based point cloud models.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, 3D point cloud processing and deep learning technology, and specifically to a point cloud part segmentation method based on hyperspectral enhancement (SM3D). Background Technology

[0002] 3D point clouds serve as a fundamental data representation for numerous applications, including autonomous driving, robotics, and virtual reality / augmented reality (VR / AR) systems. While their fine-grained geometry enables precise perception, their disordered and irregular nature poses a significant challenge to efficient sequence modeling. In recent years, Mamba, with its linear computational complexity and powerful long sequence modeling capabilities, has become a strong alternative to the Transformer architecture in 3D perception. However, applying Mamba to 3D point clouds typically requires flattening the 3D topology into a 1D token sequence. Therefore, existing Mamba-based methods primarily focus on optimizing serialization strategies to preserve spatial locality within the sequence.

[0003] However, focusing solely on serialization overlooks a more fundamental problem: Mamba-based State Space Models (SSMs) inherently suffer from spectral low-pass bias. Due to their recursive formulas, SSMs essentially function as low-pass filters, systematically attenuating high-frequency components during state transitions. This effect is particularly detrimental in serialized point clouds, as serialization inevitably disperses local geometric topology, causing fine-grained structural details to appear as high-frequency spectral fluctuations in the generated 1D sequence. The low-pass nature of SSMs suppresses this crucial information, resulting in structural ambiguity. More importantly, this phenomenon stems from the inherent spectral characteristics of SSMs, and therefore cannot be completely resolved simply by improving serialization strategies. This spectral bias further induces semantic dilution during deep propagation. Ultimately, as high-frequency geometric components are suppressed, feature representations gradually lose their discriminative power, ultimately affecting the overall representation performance.

[0004] Therefore, there is an urgent need to propose a novel solution that can simultaneously maintain geometric fidelity and semantic consistency. Summary of the Invention

[0005] This invention provides a point cloud region segmentation method based on high-spectrum enhancement to solve the problems of spectral bias and semantic dilution caused by the state space model in point cloud serialization modeling.

[0006] A point cloud region segmentation method based on hyperspectral enhancement is implemented by the following steps:

[0007] Step 1: Serialize the input point cloud to generate an embedding sequence;

[0008] Step 2: Input the embedded sequence into the Geometric Spectrum Compensator (GSC) and the Semantic Consistency Optimizer (SCR) for feature enhancement and calibration;

[0009] For each point cloud block, high-frequency components guided by GSC injection are used, and spectral compensation features are obtained through local graph construction and normalization, high-frequency extraction based on graph Laplacian, and geometric modulation.

[0010] The spectral compensation features are processed using SCR, and parallel spatial context paths are introduced to calibrate the input features to obtain the final calibrated features.

[0011] Step 3: Output the segmentation result from the final calibration features through the task decoder.

[0012] The beneficial effects of this invention are:

[0013] 1. This invention identifies and analyzes the inherent spectral low-pass bias of SSMs when applied to serialized point clouds, revealing its role in suppressing high-frequency geometric information and causing semantic dilution.

[0014] 2. This invention designs a geometric spectrum compensator (GSC), which clarifies the high-frequency response guided by the injection graph through local Laplacian analysis, and effectively alleviates the spectral bias in Mamba-based point cloud models without introducing excessive computational overhead.

[0015] 3. This invention designs a semantic consistency optimizer (SCR) that maintains semantic consistency across different frequency bands during deep propagation and supports efficient instantiation under different spectral formulas, achieving a flexible trade-off between accuracy and efficiency.

[0016] 4. Experimental verification of the method of the present invention was carried out on the ShapeNetPart dataset. The SM3D method of the present invention achieved state-of-the-art performance and achieved a good accuracy-efficiency trade-off, which proved the importance of solving the problems of spectral bias and semantic dilution in SSM-based point cloud models. Attached Figure Description

[0017] Figure 1 This is an architecture diagram of the point cloud region segmentation method based on high-spectrum enhancement described in this invention.

[0018] Figure 2 This is a schematic diagram of the coded block structure;

[0019] Figure 3 This is a schematic diagram of the GSC structure;

[0020] Figure 4 (a) is a schematic diagram of the SCR structure, and (b) is a schematic diagram of the SMB structure.

[0021] Figure 5 This is a schematic diagram of the SCR-C structure;

[0022] Figure 6 The first row shows the qualitative results of SM3D part segmentation on ShapeNetPart. The second row shows the actual labeled values, and the third row shows the effect of segmentation (SM3D-L) achieved by SCR-L in this invention.

[0023] Figure 7 This is a spectral evolution analysis diagram. Detailed Implementation

[0024] Specific Implementation Method 1: Combination Figures 1 to 5 This embodiment describes a point cloud segmentation method based on high-spectrum enhancement (SM3D), such as... Figure 1 As shown, the specific steps include:

[0025] Step S1: Serialize the input point cloud to generate point cloud blocks and embed them into a sequence; the specific implementation process is as follows:

[0026] Step S11: Sampling using the farthest point sampling (FPS) method individual centers And use K-nearest neighbors (KNN) to group K local neighborhoods to form patches;

[0027] Step S12: Project the patch using a lightweight PointNet to generate token embedding features. ,in Indicates the embedding dimension;

[0028] Step S13: Add learnable category tokens and patch location codes to , forming the initial sequence ;

[0029] Step S2: The embedded sequence is input into a 12-layer encoder for feature enhancement and correction. This encoder integrates a geometric spectral compensator (GSC) and a semantic consistency optimizer (SCR). The SCR integrates a spectral Mamba block (SMB), such as... Figures 2 to 5 As shown; the specific implementation process is as follows:

[0030] Step S21: For each point cloud block, high-frequency components guided by the GSC injection map are first used to obtain spectral compensation features. The specific process is as follows: Figure 3 As shown:

[0031] Step S211, Local Graph Construction and Standardization: For each center point Retrieve its KNN to form the neighborhood ,in express In Euclidean distance The feature representation of the nearest neighbor, center-neighborhood pair is as follows: To ensure robustness to rigid transformations and scale changes, the standard deviation is used to measure the neighborhood. Features of center-neighborhood pairs Standardization is performed to obtain standardized neighborhood features. Features of center-neighborhood pairs Then, the features of the normalized center-neighborhood pairs are initially coupled through a learnable affine transformation. and token embedding features Generate extended features :

[0032] (1)

[0033] in, These are learnable scaling and offset parameters.

[0034] Step S212, High-frequency extraction based on graph Laplacian: To explicitly capture high-frequency components, high-frequency extraction is performed based on the center point. Construct an undirected weighted graph Vertex set , Depend on of An edge is established between all pairs of nodes within a local patch, consisting of the nearest neighbors. Represents the set of all undirected edges, specifically referring to... In a local patch consisting of nearest neighbors, edges connect all nodes pairwise. (This is for quantization.) Given the geometric topology, calculate the Gaussian adjacency matrix. :

[0035] (2)

[0036] (3)

[0037] in, Represents vertices and The weights of the edges between them reflect their geometric similarity. This represents the Euclidean distance between vertices. It is a local adaptive scaling parameter.

[0038] Normalized local graph Laplacian matrix It is given by the following formula:

[0039] (4)

[0040] in, It is a degree matrix, and its diagonal elements are , It is an identity matrix.

[0041] In spectrum theory, Matrix operations act as a high-pass filter, quantizing the differences between a node and its neighborhood. This matrix operation is then applied to extended features. To extract high-frequency response Not all high-frequency signals are beneficial; therefore, a learnable frequency selection gate is introduced to select information-rich spectral bands.

[0042] (5)

[0043] in It is the Sigmoid activation function. This represents the refined high-frequency residuals that are crucial for structural differentiation, corresponding to the characteristic components that are most easily suppressed under inherent low-pass bias.

[0044] Step 213, Geometric Modulation: In sequential point cloud modeling, topological proximity is often disrupted. To ensure respect for the original 3D structure, geometric information is used to modulate the spectral reconstruction process. Specifically, high-frequency cues are preferentially acquired from spatially nearby neighborhoods. Geometrically perceptual modulation weights are calculated using normalized distance. :

[0045] (6)

[0046] in It is a local geometric weight scaling parameter. To represent a numerically stable term and prevent the denominator from being 0, the value is [value to be filled in]. .

[0047] These weights suppress contributions from distant points. Then, the modulated high-frequency components are injected back into the feature stream via residual connections.

[0048] (7)

[0049] in, It is a spectral compensation feature. It is a learnable injection strength parameter. Geometric sensing modulation weights Extended weights along the channel dimension.

[0050] Step S22: Enhanced spectral compensation features via GSC The channel semantics are calibrated by a spectrum Mamba block (SMB) with integrated SCR to obtain the final calibration features;

[0051] In this implementation, the core objective of SCR is to correct feature shifts in deep networks. This implementation provides two different instantiation methods: Precise Spectral Anchoring Based on Laplacian Feature Basis (SCR-L) and Scalable Spectral Anchoring Based on Chebyshev Polynomials (SCR-C), to accommodate different trade-offs between accuracy and scalability.

[0052] Precise Spectral Anchoring Based on Laplacian Eigenvalues ​​(SCR-L): For scenarios with a moderate number of point clouds and critical accuracy, a precise Graph Fourier Transform (GFT) is employed. For example... Figure 4 As shown, let Indicates based on central set A KNN graph is constructed, where each node corresponds to a patch token. This is based on pairwise distance. Adjacency matrix The element is defined as ,in It is the nearest neighbor distance. It is Kronek The function. This yields the normalized Laplace matrix. ,in yes The degree matrix.

[0053] Perform Laplacian eigenvalue decomposition on the global graph (i.e., the point cloud obtained after processing the original input point cloud only using the farthest point sampling (FPS) method), that is: To obtain the graph Fourier matrix U, and then project the spatial features into the spectral domain using GFT:

[0054] (8)

[0055] In the formula, The input feature is the spectral compensation feature. , The spectral characteristics projected onto the spectral domain;

[0056] To enhance consistency, a spectrum-gated network (SGNet) is designed to recalibrate the channel directional response. Features are then used to... Grouped by channel dimension And learn frequency-aware modulation weights as follows :

[0057] (9)

[0058] in , It is a learnable linear projection. This is the Sigmoid activation function.

[0059] This gating selectively emphasizes spectral components that consistently contribute to semantic recognition. Finally, the spectral signal is projected back into the spatial domain using inverse graphical Fourier transform (IGFT) to form an attention map independent of sequence order.

[0060] (10)

[0061] This instantiation provides an accurate implementation of global spectral anchoring by operating directly on the Laplacian feature basis, which is particularly suitable for medium-sized point clouds where accurate frequency domain interpretation is required.

[0062] Scalable Spectral Anchoring Based on Chebyshev Polynomials (SCR-C): For large-scale point clouds, SCR-C utilizes linear complexity spectral filters based on Chebyshev polynomials to avoid computationally expensive eigenvalue decomposition. For example... Figure 5 As shown, based on the central set Build with KNN graph of each neighborhood To ensure numerical stability and well-defined spectral characteristics, random walk normalization is applied to the adjacency matrix. Let... Represents the weighted adjacency matrix. If the degree matrix is ​​given, then the normalized transformation matrix is ​​defined as follows: Therefore, the Laplace operator for random walk graphs is expressed as follows: .

[0063] In implementation, Items in Efficient computation is achieved through message passing on a sparse neighborhood list, avoiding the explicit construction of dense matrices. Then, the feature map is centered to suppress zero-frequency components, resulting in... Mean is calculated as the mean. The k-th order Chebyshev polynomial. Calculated recursively:

[0064] (11)

[0065] (12)

[0066] in It is a rescaled Laplacian matrix used for spectral approximation. This represents the upper bound of the eigenvalues.

[0067] In this embodiment, three Chebyshev terms are used to explicitly capture multi-scale semantics. Separate the signal into low ,middle and high frequency band:

[0068] (13)

[0069] This decomposition will improve the global structural smoothness. With sharp geometric changes The frequency bands are separated, and each band is then processed by SGNet to generate the respective frequency bands. , , Corresponding frequency band-specific modulation mask The final global attention graph for:

[0070] (14)

[0071] in, This represents the learnable band importance weights, and .

[0072] This instantiation, which approximates the spectrum using Chebyshev polynomials and has linear complexity, retains frequency selectivity while avoiding explicit eigenvalue decomposition, making it suitable for large-scale point cloud understanding.

[0073] In this implementation, although the spectral path provides global semantic anchoring, local spatial continuity remains crucial for fine-grained details. Therefore, parallel spatial context paths are introduced to directly operate on the input feature X, such as... Figure 4 As shown, the input features are first grouped according to the channel dimension. Then, depthwise convolution is used to aggregate information from neighboring tokens, and finally, a channel shuffle operation is applied to enhance information exchange between groups:

[0074] (15)

[0075] This approach ensures that the optimized features maintain local geometric continuity, complementing the global semantics captured in the spectral domain.

[0076] Final calibration features Obtained by anchoring the modulation spatial content using global spectral attention:

[0077] (16)

[0078] in Obtained from SCR-L Obtained from SCR-C, this invention adaptively selects different SCR paths based on the different point cloud scales.

[0079] This interaction ensures that local features are validated through the global geometric spectrum, enhancing semantic consistency across channels. SCR effectively corrects the feature distribution before SSM recursively updates, thereby reversing semantic dilution during deep propagation.

[0080] Finally, the features calibrated by SCR are integrated into SMB, and the original causal convolutions are replaced with depthwise convolutions to better preserve spatial fidelity. To alleviate the limitations of unidirectional state propagation, a bidirectional SSM is adopted, combining F-SSM and C-SSM, enabling the model to capture semantic dependencies from both directions.

[0081] Step S3: Output the segmentation result through the task decoder.

[0082] Step S31: Extract multi-level features from layers 1, 4, 8, and 12 of the encoder, align the spatial resolution through interpolation, and then refine the features using a simplified DGCNN (S-DGCNN). The features from layers 1, 4, 8, and 12 naturally carry information from different levels. Layer 1 contains shallow features, including local geometric details of the point cloud, corresponding to high spatial resolution, but with weak semantic information. Layers 4 and 8 balance local details with preliminary semantics, have moderate spatial resolution, understand the global structure, and capture the semantic prototype at the "component level." Layer 12 contains deep features containing global semantics, has the lowest spatial resolution, but the strongest semantic discriminative power. The fusion process is as follows:

[0083] First fusion and feature refinement:

[0084] (17)

[0085] In the formula: This is the feature obtained by stitching together the calibration features of layer 12 and layer 8. and These are the calibration features output by the 12th and 8th layer encoders, respectively; To be The first fusion feature is refined using a simplified DGCNN.

[0086] Second fusion and feature refinement:

[0087] (18)

[0088] Third fusion and feature refinement:

[0089] (19)

[0090] Step S32: Finally, the features from the three fusions are... The final segmentation result is obtained by decoding and dimensionality reduction using an MLP (Multilayer Perceptron).

[0091] (20)

[0092] Specific Implementation Method Two: Combination Figure 6 and Figure 7 As shown, this embodiment is an example of the point cloud segmentation method based on high-spectrum enhancement described in Specific Embodiment 1:

[0093] To verify the effectiveness of the proposed point cloud segmentation method based on high-spectrum enhancement, this invention implements SM3D on a single NVIDIA TITAN RTX 24GB GPU within the PyTorch framework. The hidden dimension D of the backbone network is 384. The input point cloud is tokenized into N=128 patches using FPS, with each patch containing K=32 neighborhoods grouped by KNN. In GSC and SCR-C, the spectrogram size is set to... and The AdamW optimizer was used, with an initial learning rate of... The weights decayed to 0.05. The learning rate followed a cosine decay schedule and underwent a linear warm-up of 10 epochs. The model was trained for 300 epochs with a batch size of 32. Cross-entropy loss was used for supervision. Following established protocol, self-supervised pre-training was performed on the ShapeNetCore dataset before fine-tuning.

[0094] The performance of fine-grained part segmentation was evaluated on the ShapeNetPart dataset, which contains approximately 16K shapes from 16 object classes with 50 part labels. Following standard protocols, 2,048 points were uniformly sampled for each shape without using normal vectors. Quantitative results are shown in Table 1, and qualitative visualizations are as follows: Figure 6 As shown.

[0095] Table 1

[0096]

[0097] ShapeNetPart is a highly challenging benchmark, and recent methods have only achieved incremental improvements. Nevertheless, SM3D-L still achieves an instance-level mean intersection-union ratio (Ins.mIoU) of 86.5%, outperforming PointMamba and Mamba3D by 0.5% and 0.9%, respectively. Both SM3D variants consistently outperform Transformer-based methods such as Point-MAE and PointGPT-S. Notably, the performance gap between SM3D-L and SM3D-C is small (class-level mean intersection-union ratio mIoU). C The difference is 0.1%, and the instance-level average crossover and union ratio (mIoU) is... I(Difference of 0.2%). This indicates that for dense segmentation tasks, polynomial spectral approximation is sufficient to capture the required local-global consistency, providing a good accuracy-efficiency tradeoff. Qualitatively, the method of this invention performs excellently in depicting the boundaries of complex structures (such as aircraft engines and motorcycle wheels), such as... Figure 6 As shown, this clarity is directly attributable to the GSC module, which preserves high-frequency edge information. It should be noted that the segmentation results for the car hood and motorcycle seat show slight deviations from the ground truth (GT), indicating that the model still has room for further improvement.

[0098] like Figure 7 As shown, the high-frequency energy of the baseline model (without GSC and SCR) decreased from 15.2% to 12.2%. This confirms the low-pass bias of standard SSMs. In contrast, SM3D-L and SM3D-C reversed this trend, exhibiting a gradual increase in high-frequency energy. This phenomenon validates that GSC effectively preserves and enhances fine-grained geometric details.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A point cloud region segmentation method based on high-spectrum enhancement, characterized by: This method is implemented by the following steps: Step 1: Serialize the input point cloud to generate an embedding sequence; Step 2: Input the embedded sequence into the Geometric Spectrum Compensator (GSC) and the Semantic Consistency Optimizer (SCR) for feature enhancement and calibration; For each point cloud block, high-frequency components guided by GSC injection are used, and spectral compensation features are obtained through local graph construction and normalization, high-frequency extraction based on graph Laplacian, and geometric modulation. The spectral compensation features are processed using SCR, and parallel spatial context paths are introduced to calibrate the input features to obtain the final calibrated features. Step 3: Output the segmentation result from the final calibration features through the task decoder.

2. The point cloud segmentation method based on high-frequency enhancement according to claim 1, characterized in that: The specific process of step one is as follows: Step 11: Sampling using the FPS sampling method Each center point And K local neighborhoods are grouped into K nearest neighbors to form patches; Steps 1 and 2: Project the patch using a lightweight PointNet to generate token embedding features; Step 13: Add learnable category tokens and patch position encodings to the token embedding features to form an initial embedding sequence.

3. The point cloud segmentation method based on high-spectrum enhancement according to claim 2, characterized in that: In step two, the process of local graph construction and standardization is as follows: For each center point Retrieve its KNN to form the domain ,in, for In Euclidean distance The characteristics of the nearest neighbor, center-neighborhood pair are: Using standard deviation to analyze the neighborhood Features of center-neighborhood pairs Standardize to obtain standardized neighborhood features Features of center-neighborhood pairs ; Features of the initial coupling center-neighborhood pairs are obtained through learnable affine transformations. and token embedding features Generate extended features : ； In the formula, These are learnable scaling and offset parameters.

4. The point cloud segmentation method based on high-spectrum enhancement according to claim 3, characterized in that: The process of high-frequency extraction based on graph Laplacian is as follows: Based on the center point Construct an undirected weighted graph ,in, The set of all undirected edges, the vertex set , From the center point of It consists of the nearest neighbors. It is a Gaussian adjacency matrix; Applying the normalized local graph Laplacian matrix to the extended features To extract high-frequency responses, a learnable frequency selection gate is introduced to select information-rich spectral bands and obtain high-frequency residual features, expressed as follows: ； In the formula, It is the Sigmoid activation function. High-frequency residual characteristics; = , The normalized local graph Laplacian matrix, It is a high-frequency feature.

5. The point cloud segmentation method based on high-spectrum enhancement according to claim 4, characterized in that: Calculate the Gaussian adjacency matrix It can be expressed as follows: ； ； In the formula, As vertex and The weight of the edges between them. Let be the Euclidean distance between vertices. For local adaptive scaling parameters; Normalized local graph Laplacian matrix It is given by the following formula: ； In the formula, For degree matrix, It is an identity matrix.

6. The point cloud segmentation method based on high-spectrum enhancement according to claim 5, characterized in that: The geometric modulation process is as follows: Calculate the geometric sensing modulation weights using the normalized distance. : ； In the formula, For local geometric weight scaling parameters, It is a numerically stable term; These are the standardized neighborhood features; The modulated high-frequency components are injected back into the feature stream via residual connection to obtain spectral compensation features; expressed as follows: ； In the formula, For spectral compensation features, For learnable injection strength parameters, Geometric sensing modulation weights Extended weights along the channel dimension.

7. The point cloud segmentation method based on high-spectrum enhancement according to claim 6, characterized in that: The SCR integrates a spectrum Mamba block to perform channel semantic calibration on the spectrum compensation features and obtain calibration features. The specific process is as follows: The spectral compensation features are modulated using global spectral attention anchoring, which includes precise spectral anchoring based on Laplace eigenbases (SCR-L) and scalable spectral anchoring based on Chebyshev polynomials (SCR-C). Parallel spatial context paths are then used to operate on the spectral compensation features to obtain the final calibration features. It can be expressed as follows: ； In the formula, The attention map obtained through SCR-L, This is the global attention map obtained through SCR-C. This refers to the inter-group information exchange characteristics obtained through parallel spatial context paths.

8. The point cloud segmentation method based on high-spectrum enhancement according to claim 7, characterized in that: Attention map obtained through SCR-L The process is as follows: Performing Laplacian eigenvalue decomposition on the global graph yields the graph Fourier matrix. And the spatial features are projected into the spectral domain using GFT: ； In the formula, The input feature is the spectral compensation feature. , The spectral characteristics projected onto the spectral domain; Design a spectrum-gated network SGNet to recalibrate channel directional responses by grouping the channel dimensions and applying modulation weights based on the learning frequency. : ； In the formula , For learnable linear projections, For activation functions, LN For layer normalization, Use the Sigmoid activation function; The spectral signal is projected back into the spatial domain using the inverse graphical Fourier transform (IGFT), forming an attention map independent of the sequence order. 。 9. The point cloud segmentation method based on high-spectrum enhancement according to claim 7, characterized in that: Global attention map obtained through SCR-C The process is as follows: Using three Chebyshev terms Separate the signal into low frequency bands Mid-frequency band and high frequency band ; ； Each band is then processed by SGNet to generate the modulation mask for the corresponding frequency band, resulting in the final global attention map. for: ； In the formula, For learnable band importance weights, Corresponding to low frequency bands Mid-frequency band and high frequency band The weight.

10. The point cloud segmentation method based on high-spectrum enhancement according to claim 7, characterized in that: We introduce parallel spatial context paths to directly operate on the input feature X. First, we group the data by channel dimension, then we use depthwise convolution to aggregate the information of adjacent tokens, and finally we apply channel shuffling to enhance the information exchange between groups, thus obtaining the inter-group information exchange features. .

Images