A three-dimensional point cloud multi-task processing method and system based on a Mamba model

Abstract

The present application relates to a kind of three-dimensional point cloud multitask processing method and system based on Mamba model, obtain original point cloud, pre-processing is carried out, and high-dimensional feature of fusion local geometry and spatial position information is obtained simultaneously;High-dimensional feature is input into the Mamba model of task adaptive parameterization, and high-dimensional implicit hypersurface for representing the local geometric structure of point cloud is generated;Feature screening is carried out based on high-dimensional implicit hypersurface, and core feature representation is extracted, and the task result corresponding to one or more query points is calculated;Method implementation system is based on the present application.The stability and separability of point cloud local feature representation are significantly improved, the long-range dependence modeling capability is enhanced while maintaining linear complexity, especially suitable for large-scale point cloud processing and high real-time requirement application scene;The consistency and robustness of normal vector, average curvature and feature point confidence output result are improved.

Description

Technical Field

[0001] This invention relates to the technical field of image or video recognition or understanding, and in particular to a multi-task processing method and system for 3D point clouds based on the Mamba model in the fields of 3D point cloud processing and deep learning. Background Technology

[0002] Point clouds are collections of discrete points in three-dimensional space, typically generated by lidar, structured light, etc. Point clouds are acquired through methods such as camera scanning and multi-view reconstruction, and are widely used in scenarios such as robot navigation, autonomous driving, industrial inspection, digital twins, and 3D reconstruction. In these applications, the local geometric properties of point clouds, such as normal vectors and curvature, are important foundations for downstream tasks such as surface reconstruction, point cloud registration, scene segmentation, and key point extraction. Feature point confidence can effectively remove noise points, weak texture points, and structurally unstable points, which is the core support for improving the robustness of subsequent tasks throughout the entire process.

[0003] Existing methods for estimating normal vectors and curvature largely rely on traditional geometric algorithms such as local PCA and least-squares plane / surface fitting. These methods only perform statistical fitting based on explicit coordinate information of local point sets, failing to learn higher-order geometric priors and scene generalization rules from massive datasets. In practical engineering scenarios, point clouds commonly suffer from sparse sampling, viewpoint occlusion, outlier noise, and uneven sampling density. This leads to instability in estimation results, loss of fine geometric details, and insufficient generalization ability in these algorithms. Furthermore, the reliability of the estimation results cannot be quantitatively evaluated, making them unsuitable for complex engineering applications.

[0004] With the development of 3D deep learning technology, data-driven methods such as PointNet++ and DGCNN can learn local and global features of point clouds end-to-end, significantly improving their anti-interference capabilities compared to traditional geometric algorithms. However, the effective receptive field and feature aggregation capabilities of these methods are strictly limited by the network structure, making it impossible to simultaneously consider the fine geometric details of local neighborhoods and long-range structural dependencies. This still presents a core problem of insufficient feature representation capabilities in geometric attribute estimation tasks for low-quality point clouds. Self-attention architectures, represented by Transformer, have achieved breakthroughs in point cloud processing performance through their global dependency modeling capabilities. However, the computational and memory overhead of their self-attention mechanism increases quadratically with the length of the input sequence, resulting in extremely high computational loads in large-scale point cloud processing, which is not conducive to real-time deployment at edge devices. Furthermore, these methods are susceptible to noise and outliers, making it difficult to specifically focus on key information of local geometric structures and failing to balance the effectiveness of feature representation with computational efficiency. Structured state-space models (SSMs), represented by Mamba, achieve computational complexity that is linearly related to sequence length through selective scanning mechanisms and hardware-aware parallel algorithms. While maintaining or even surpassing the long-range dependency modeling capabilities of Transformers, they significantly reduce computational and memory overhead, providing a new technical path for efficient modeling of large-scale point cloud sequences.

[0005] However, existing research on Mamba in point clouds largely focuses on single-task scenarios such as classification and segmentation, leaving a significant technological gap in multi-task joint learning for local geometric attributes of point clouds. Firstly, existing methods cannot adaptively adapt model parameters and feature representations to the different characteristics of tasks such as normal vector estimation, curvature prediction, and confidence calculation. When multiple tasks share a backbone network, negative transfer between tasks easily occurs, leading to a significant performance degradation in some tasks. Secondly, they fail to deeply integrate Mamba sequence modeling with local implicit geometric representations, making it impossible to accurately characterize local geometric structures through high-dimensional implicit hypersurfaces. In sparse and noisy point cloud processing, it is difficult to guarantee the consistency and robustness of multi-task prediction results. Summary of the Invention

[0006] This invention solves the problems existing in the prior art and provides a method and system for multi-task processing of three-dimensional point clouds based on the Mamba model.

[0007] The technical solution adopted in this invention is a multi-task processing method for three-dimensional point clouds based on the Mamba model, which acquires the original point cloud, performs preprocessing, and simultaneously obtains high-dimensional features that fuse local geometric and spatial location information.

[0008] The high-dimensional features are input into the task-adaptive parameterized Mamba model to generate a high-dimensional implicit hypersurface for characterizing the local geometry of the point cloud.

[0009] Feature filtering is performed based on the high-dimensional implicit hypersurface to extract core feature representations, and task results corresponding to one or more query points are calculated.

[0010] Preferably, after constructing neighborhood blocks centered on each query point in the original point cloud and performing standardization processing, local feature parallel encoding processing is performed on each neighborhood block based on a preset point cloud local feature encoding strategy to obtain the corresponding high-dimensional features.

[0011] Preferably, the point cloud local feature encoding strategy includes:

[0012] Geometric feature encoding is performed on the standardized neighborhood blocks to obtain local geometric features;

[0013] Spatial location encoding is performed on the standardized neighborhood blocks to obtain local spatial location features;

[0014] By fusing local geometric features and local spatial location features, high-dimensional features are obtained.

[0015] Preferably, the relative coordinates of each point within a neighborhood block are nonlinearly mapped using a multilayer perceptron to obtain initial local geometric features; the initial local geometric features are then weighted and aggregated and fused across scales using an attention mechanism to obtain context-aware optimized local geometric features.

[0016] Preferably, the neighborhood block is downsampled, and a preset number of nearest neighbors to the query point are retained; based on a learnable encoding function, the relative coordinates of each point in the downsampled neighborhood block are calculated to obtain local spatial location features.

[0017] Preferably, the high-dimensional features are input into a hybrid expert pool to obtain the judgment results of several experts;

[0018] High-dimensional features are input into one or more task routes for processing to obtain the corresponding task expert weights, mixture weights, and noise weights.

[0019] The judgment results of all the experts are weighted and fused based on the task expert weights, and combined with learnable prior parameters and feature linear projection based on hybrid weights to generate task-adaptive model parameters.

[0020] By utilizing task-adaptive model parameters, high-dimensional features and their inverse features are processed through a parallel state-space model and fused to obtain a high-dimensional implicit hypersurface.

[0021] Preferably, noise based on noise weights is injected into the task expert weights, and TopK sparsity processing is performed to retain the top K experts with the highest weight values.

[0022] Preferably, the high-dimensional implicit hypersurface is weighted by a multilayer perceptron, and the weights are multiplied by the high-dimensional implicit hypersurface to obtain the core feature representation.

[0023] Preferably, the task includes obtaining the normal vector estimate, the average curvature, and the feature point confidence; and inputting the core feature representations into the normal vector decoding head, the curvature decoding head, and the confidence decoding head, respectively.

[0024] The normal vector decoding head outputs a three-dimensional vector and performs normalization processing to obtain the estimated value of the normal vector;

[0025] The curvature decoding head outputs a scalar to obtain the average curvature;

[0026] The confidence decoding head outputs a scalar between 0 and 1 to obtain the confidence of the feature point.

[0027] A multi-task processing system for 3D point clouds based on the Mamba model includes a processor and a memory. The memory stores computer program instructions. When the processor executes the computer program instructions, it implements the multi-task processing method for 3D point clouds based on the Mamba model.

[0028] This invention relates to a multi-task processing method and system for 3D point clouds based on the Mamba model. The method involves acquiring the original point cloud, preprocessing it, and simultaneously obtaining high-dimensional features that integrate local geometry and spatial location information. These high-dimensional features are then input into a task-adaptive parameterized Mamba model to generate a high-dimensional implicit hypersurface representing the local geometric structure of the point cloud. Feature filtering is performed based on the high-dimensional implicit hypersurface to extract core feature representations, and task results corresponding to one or more query points are calculated. The system is implemented based on this method.

[0029] The beneficial effects of this invention are as follows:

[0030] (1) By using neighborhood block standardization and parallel coding strategies, local geometric information and spatial location information can be captured simultaneously, significantly improving the stability and separability of point cloud local feature representation;

[0031] (2) Introducing the Mamba structured state space model for sequence modeling enhances the ability to model long-range dependencies while maintaining linear complexity, which is especially suitable for large-scale point cloud processing and application scenarios with high real-time requirements.

[0032] (3) Adopting a task adaptive parameterization mechanism based on expert hybrid model, effective feature representations and model parameters are dynamically allocated for different geometric attribute estimation tasks, thereby improving the multi-task collaboration capability and effectively suppressing negative transfer between tasks;

[0033] (4) The point weight estimation and core feature extraction method based on high-dimensional implicit hypersurface can effectively suppress noise and outlier interference, thereby improving the consistency and robustness of the output results of normal vector, average curvature and feature point confidence. Attached Figure Description

[0034] Figure 1 This is a flowchart of the method of the present invention;

[0035] Figure 2 This is a schematic diagram of point cloud local feature encoding in this invention;

[0036] Figure 3 This is a schematic diagram of the Mamba model structure for task adaptive parameterization in this invention;

[0037] Figure 4 This is a schematic diagram of the expert hybrid model structure for constructing parameters B and C in this invention;

[0038] Figure 5 This is a diagram showing the result of the parallel output of normal vectors by the multi-task decoding head in this invention;

[0039] Figure 6 This is a diagram showing the result of the parallel output curvature of the multi-task decoding head in this invention;

[0040] Figure 7 This is a graph showing the results of the parallel output confidence scores of the multi-task decoding head in this invention. Detailed Implementation

[0041] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0042] This invention relates to a multi-task processing method for 3D point clouds based on the Mamba model. The core idea is to construct stable local high-dimensional features of point clouds by combining neighborhood standardization and dual-path parallel coding. The discrete point cloud sequence is transformed into a geometric representation of a continuous high-dimensional implicit hypersurface by using the task-adaptive parameterized Mamba model. Then, the core effective information is purified by adaptive point weighting, providing high-quality feature support for subsequent multi-task parallel decoding.

[0043] Among them, neighborhood standardization and dual-path parallel feature encoding construct a unified local neighborhood for each query point that is not affected by the initial pose, and simultaneously fuse local geometric and spatial location information to eliminate scale differences and pose interference, ensuring the consistency and stability of feature input.

[0044] Task-adaptive parameterized Mamba modeling can dynamically generate core model parameters for different task characteristics, and leverage the linear complexity of Mamba to achieve efficient long-range dependency modeling, while solving the negative transfer problem when multiple tasks share a backbone.

[0045] High-dimensional implicit hypersurface representation transforms discrete point cloud sequence features into continuous geometric representations, fully characterizing the geometric topology of the local neighborhood and providing a unified geometric basis for subsequent feature selection.

[0046] The implicit hypersurface-based point weight adaptive screening performs point-by-point adaptive recalibration of features, effectively suppressing noise and outlier interference, focusing on local core geometric information, and obtaining core features with high robustness and high discriminativeness.

[0047] like Figure 1 As shown, the method includes the following steps:

[0048] (1) Obtain the original point cloud, perform preprocessing, and simultaneously obtain high-dimensional features that integrate local geometry and spatial location information;

[0049] (2) Input the high-dimensional features into the task-adaptive parameterized Mamba model to generate a high-dimensional implicit hypersurface for characterizing the local geometry of the point cloud;

[0050] (3) Based on the high-dimensional implicit hypersurface, feature filtering is performed to extract core feature representations and calculate the task results corresponding to one or more query points.

[0051] The method is described below with reference to specific embodiments. In this embodiment, the tasks include obtaining the normal vector estimate, the average curvature, and the confidence of the feature points.

[0052] (1) Obtain the original point cloud, perform preprocessing, and simultaneously obtain high-dimensional features that integrate local geometry and spatial location information;

[0053] Based on each query point in the original point cloud, a neighborhood block is constructed and standardized. Then, based on a preset point cloud local feature encoding strategy, the local features of each neighborhood block are encoded in parallel to obtain the corresponding high-dimensional features.

[0054] In this invention, the KNN nearest neighbor algorithm is used to construct a structure containing query points. The neighboring blocks of each point And perform standardization processing on it, including normalization and PCA alignment;

[0055] like Figure 2 As shown, the point cloud local feature encoding strategy includes:

[0056] (1-1) Perform geometric feature encoding on the standardized neighborhood blocks to obtain local geometric features;

[0057] Using a multilayer perceptron to analyze neighboring blocks The relative coordinates of each point within the area are nonlinearly mapped to obtain the initial local geometric features. The initial local geometric features are weighted and aggregated using an attention mechanism, and then fused across hierarchical scales to achieve large-scale geometric feature extraction and hierarchical cross-scale fusion, resulting in context-aware optimized local geometric features. .

[0058] Specifically, ,in, For the nonlinear mapping function of the multilayer perceptron, discrete point cloud coordinate information is mapped to a high-dimensional feature space, initially capturing the basic geometric topological relationships within the neighborhood block, and providing basic input for subsequent attention weighting and feature fusion;

[0059] Subsequently, through the convolution module and convolutional modules From initial local geometric features Obtain two feature matrices and The transformation formula is as follows:

[0060]

[0061]

[0062] Calculate the global attention score ,

[0063]

[0064] in, For feature encoding dimension, Given the number of points within the neighboring block, q is first determined by... Function along point dimension Normalize, then adjust the encoding dimensions. The average value is calculated to obtain the final attention score. This operation, through a data-driven approach, assigns differentiated weights to the features of different points in the neighborhood, providing a quantitative basis for the subsequent weighted aggregation of global features.

[0065] Based on attention score AND-valued characteristic matrix Calculate the weighted global feature vector ,

[0066]

[0067] in, , This represents the matrix transpose operation. This is a learnable scaling factor, initially set to 0.1. This represents the multilayer perceptron (MLP) mapping module; this operation deeply fuses the global attention weights with the feature matrix, aggregating them to obtain a global feature vector that can characterize the overall geometric properties of the neighborhood, while balancing the feature response intensity through learnable parameters.

[0068] The following formula is used to fuse global and local features.

[0069]

[0070] in, , Represents the convolutional module. For initial local geometric features The local geometric features of the downsampled slices are then concatenated into the global feature vector. and local geometric features of slices Then, a convolution operation is performed; patch-level (pattern-level) fusion is performed on the fused features to obtain the final fused features. It contains M point features; this operation completes the deep fusion of global geometric features and local detailed features, and achieves effective feature aggregation through hierarchical downsampling. The output local geometric features not only retain fine geometric details, but also have global representation capabilities, and are completely matched with the dimensions of subsequent spatial location features.

[0071] (1-2) Perform spatial location encoding on the standardized neighborhood blocks to obtain local spatial location features;

[0072] For neighboring blocks Downsampling is performed, retaining a preset number of nearest neighbors to the query point; based on a learnable encoding function, the relative coordinates of each point within the downsampled neighborhood block are calculated to obtain local spatial location features. .

[0073] Specifically, during the initial construction of the neighborhood block, the KNN nearest neighbor algorithm was used to select the query point. To match subsequent feature dimensions and focus on the most relevant local spatial information, only the nearest neighbor points to the query point are retained. These points form smaller neighborhood blocks. This ensures that the output spatial location features are fully aligned with the aforementioned local geometric features in terms of the number of points, laying the dimensional foundation for subsequent dual-path feature splicing and fusion, while focusing on the local region most closely related to the query point in space.

[0074] Given any query point within the neighborhood block after downsampling and its corresponding Nearest neighbor set Learning-based relative coordinate position encoding function Calculate the location coding features.

[0075]

[0076] in, Indicates nearest neighbor points Relative to query point The encoding function is located at a relative position within a local frame. and All are multilayer perceptron (MLP) layers. Parameter optimization is completed through end-to-end training, eliminating the dependence on fixed rules for manual encoding. They can adaptively capture the relative positional relationships in the local space of point clouds, making them more suitable for the irregular and discrete distribution characteristics of point clouds.

[0077] It should be noted that the acquisition of local geometric features and local spatial location features is performed in parallel.

[0078] (1-3) Integrating local geometric features With local spatial location features High-dimensional features are obtained. .

[0079] Specifically, the features from the two parallel outputs are concatenated along the channel dimension to obtain a concatenated feature matrix. The concatenated feature matrix is ​​input into the residual convolution fusion module to complete feature alignment, cross-fusion, and serialization adaptation, ultimately obtaining the high-dimensional feature T. The overall mapping formula is as follows:

[0080]

[0081] in, It is a composite residual convolution fusion module, used for splicing inputs. Residual MLP stack Output convolution module Composition, and for each point Perform max pooling operation on each local neighborhood. Since each initial neighborhood block is identified by the query point The high-dimensional feature T is constructed using nearest neighbors, therefore the generated high-dimensional feature T will be based on the query point. The proximity ordering is natural and implies spatial context, which is highly consistent with the sequential modeling paradigm of the subsequent Mamba model.

[0082] For any preprocessed neighborhood block of a query point, this invention employs a decoupled coding architecture with parallel processing of geometric and spatial features. The two core feature classes are extracted independently, optimized hierarchically, and then deeply fused to generate high-dimensional features that simultaneously characterize local fine geometry and spatial distribution patterns. This achieves decoupling and complementary fusion of geometric structure and spatial location, breaking away from the dependence of traditional single-path coding on a single feature dimension. This allows the algorithm to be equally adaptable to point cloud data from various sources, such as LiDAR, structured light, and multi-view reconstruction, significantly broadening its applicability. Regardless of the sampling density, sparsity, or noise interference of the neighborhood block, this parallel coding architecture can output stable and comprehensive local structural representations while balancing local detail resolution and long-range correlation modeling potential. This greatly reduces the learning difficulty of the subsequent Mamba sequence modeling network, allowing it to focus on deep dependency mining of local geometric structures rather than the completion and correction of basic features, thus significantly improving the robustness and accuracy of multi-task estimation. This contrasts sharply with the shortcomings of traditional single-path coding methods, which struggle to simultaneously consider geometric details and spatial information, are susceptible to noise interference in feature representation, and have insufficient generalization ability.

[0083] (2) Input the high-dimensional feature T into the task-adaptive parameterized Mamba model to generate a high-dimensional implicit hypersurface for characterizing the local geometry of the point cloud;

[0084] To be precise, the high-dimensional feature T is input as follows: Figure 3 The task-adaptive parameterized Mamba model shown first obtains task-specific parameters through an expert hybrid model, and then inputs them into the state-space model for prediction.

[0085] (2-1) Input the high-dimensional feature T into the MOE hybrid expert pool In the process, several experts' judgments were obtained. ;

[0086] Specifically, in multi-task learning, such as Figure 4 As shown, the expert hybrid model consists of two main parts: an expert group containing n experts. And a task routing network in which each task route contains n weights. The number of weights here corresponds to the number of experts, which represents the degree of relevance of each expert to the handling of a certain task.

[0087] For each input high-dimensional feature T, each expert will provide a judgment result. These results, after being weighted by task routing, generate preliminary output features, providing a basis for subsequent parameter construction. This operation captures feature information from different perspectives through parallel discrimination by multiple experts, providing rich discrimination criteria for task-adaptive feature selection and improving the model's adaptability to different local geometric structures.

[0088] (2-2) Input the high-dimensional features into one or more task routes for processing to obtain the corresponding task expert weights. Mixed weights and noise weight ;

[0089] Task expert weights determine the importance of each expert to the current task and are used to fuse the MoE results; hybrid weights are used to determine the weights when combining linear projections; and noise weights are used to add Gaussian noise during training to improve routing robustness.

[0090] To achieve adaptive task routing and fully capture local and global features to generate accurate routing weights, high-dimensional features T are input into the parallel-configured task routing, processing them in parallel from both the point and feature dimensions. Task routing is divided into two parts:

[0091] Firstly, by using linear projection 1 in the point dimension, the point channel is reduced to one-quarter of its original size, resulting in... Then, normalization, nonlinear activation, and max pooling are performed to obtain... Then, it is reshaped using linear projection 3. ,get ;

[0092] Secondly, in order to fully capture global features, global pooling is first performed to obtain... Features of shape Then, it is transformed by normalization, nonlinear activation, and linear projection 2. ,get ;

[0093] Merge the features of the two parts ( )for And through linear projection 4 ( The nonlinear layer Act is processed to obtain... Then obtain it through the Softmax function. ,in Weight for task experts Mixed weights and noise weight The sum, the formula is as follows:

[0094]

[0095]

[0096] This operation, through parallel processing of point and feature dimensions, focuses on both local point features and global distribution features, providing comprehensive and accurate weighting criteria for expert routing and enabling task-adaptive feature allocation.

[0097] (2-3) The judgment results of all the experts are weighted and fused based on the task expert weights, and the learnable prior parameters and the feature linear projection based on the mixed weights are combined to generate task-adaptive model parameters.

[0098] Noise based on noise weights is injected into the weights of task experts, and TopK sparsity processing is performed to retain the top K experts with the highest weight values.

[0099] Specifically, the task expert weights With noise weights The constructed Gaussian noise points are summed, and after nonlinear transformation, they are sorted in descending order. The top K values ​​are taken and the rest are set to zero to obtain the result. ;

[0100] During the training phase, Gaussian noise constructed from noisy weights is injected into the task expert weights to break the determinism of expert selection and avoid the problem of expert load imbalance. Then, the weight vector after injecting noise is sorted in descending order of value, and only the top K weight values ​​are retained, while the remaining weights are reset to zero to complete the sparsity processing. Softmax normalization is performed on the retained weights to obtain the route activation result.

[0101]

[0102] Let be the sparsification function, where This is a hyperparameter representing the maximum number of elements to retain; it divides the vector by... All elements except the maximum value are set to zero. These are noise samples sampled from a standard normal distribution.

[0103] Furthermore, to achieve task-adaptive fusion of multi-expert judgment results while retaining the unique feature representation capabilities of a single task, task routing activation results are used as the basis. Compared with expert judgment results The weighted aggregation of multiple expert outputs is completed to obtain the MoE fusion features. ,

[0104]

[0105] To enhance feature adaptability for single tasks, this invention retains a task-specific linear projection layer, which incorporates high-dimensional input features. Linear projection features are obtained through hybrid weight mapping. Then, the MoE fusion features and the linear projection features are added element-wise to obtain the intermediate output of the task adaptive parameters, as shown in the following formula:

[0106]

[0107]

[0108] The above operations not only make full use of the multi-dimensional feature representation capabilities of multi-expert networks, but also preserve the task-specific feature mapping space through linear projection, thus achieving a balance between common features of multiple tasks and specific characteristics of a single task.

[0109] Finally, to embed the geometric priors of different tasks into the core parameters of the model and further improve the adaptability of the parameters to downstream tasks, independent learnable prior parameters are configured for each task. These parameters are then fused with the intermediate outputs to obtain the final result of the task-adaptive parameters. Specifically, task-specific geometric priors are integrated into the state-space model in the form of parameter priors. Independent learnable prior parameters are configured for the three tasks of normal vector estimation, curvature prediction, and confidence calculation. These parameters are used to perform task-specific corrections on the core parameters B and C of the state-space model. These prior parameters are spatial invariants, and after dimensional expansion, they are kept consistent with the spatial dimension of the task features. They are denoted as task prior features. The prior features of the task are added element-wise to the intermediate output to obtain the final result used to generate parameters B and C. The calculation formula is as follows: .

[0110] (2-4) Using task-adaptive model parameters, high-dimensional features and their inverse features are processed through a parallel state-space model and fused to obtain a high-dimensional implicit hypersurface.

[0111] Specifically, the high-dimensional feature T is reversed along the dimension of the point sequence to obtain the reversed high-dimensional feature. ; Targeting the original high-dimensional features With reverse high-dimensional features Two independent state-space models (SSMs) are constructed to capture the geometric dependencies of point sequences in different directions, thereby achieving a comprehensive representation of the local geometry of the neighborhood.

[0112] The parameters B and C obtained through the expert hybrid model, and the default parameters A and D of Mamba, will be used. Input two parallel state-space models (SSMs), add their outputs element-wise along the channel dimension, and finally obtain a high-dimensional implicit hypersurface that can fully characterize the local geometric topology of the neighborhood block. ;

[0113] In this invention, addressing the pain points of multi-task joint learning, such as negative transfer between tasks, the difficulty of adapting fixed-parameter models to the characteristics of tasks with different geometric attributes, and the high computational complexity of Transformer-type models making them difficult to implement, this invention adopts a task-adaptive parameterized Mamba model. Through a hybrid expert pool of MoE and a noisy TopK routing mechanism, expert resources and core model parameters are dynamically allocated to different tasks. This leverages the linear computational complexity advantage of the Mamba model to achieve efficient modeling of long-range dependencies in point cloud sequences, while effectively suppressing negative transfer between multiple tasks through task-specific learnable priors and adaptive parameter generation mechanisms. The resulting high-dimensional implicit hypersurface realizes the transformation of discrete point cloud sequences into continuous geometric representations, accurately depicting the local geometric structure of the neighborhood, and providing a high-quality geometric representation foundation for subsequent point weight selection and multi-task parallel decoding.

[0114] (3) Based on the high-dimensional implicit hypersurface, feature filtering is performed to extract core feature representations and calculate the task results corresponding to one or more query points;

[0115] The high-dimensional implicit hypersurface is analyzed using a multilayer perceptron. Perform point weighting Calculate the point weights With high-dimensional implicit hypersurfaces Multiply to obtain the core feature representation. , expressed as,

[0116]

[0117]

[0118] in It is a multilayer perceptron (MLP) layer. For a small constant, The function performs max pooling on dimension i.

[0119] The task includes obtaining normal vector estimates, average curvature, and feature point confidence scores; and representing the core features. Input the normal vector decoding head, curvature decoding head, and confidence decoding head respectively;

[0120] The normal vector decoding head outputs a three-dimensional vector and performs normalization processing to obtain the estimated value of the normal vector;

[0121] The curvature decoding head outputs a scalar to obtain the average curvature;

[0122] The confidence decoding head outputs a scalar between 0 and 1 to obtain the confidence of the feature point.

[0123] like Figure 5 , Figure 5(a) in the image is the original 3D mesh image. Figure 5 (b) in the image represents the source point cloud extracted from the original 3D mesh image. Figure 5 In the diagram, (c) represents the predicted normal vector obtained after the architecture is applied. Figure 5 (d) in the figure is a magnified view of the local area, which shows the deviation between the predicted normal vector and the actual normal vector at each query point.

[0124] like Figure 6 As shown, this is the prediction result obtained from the average curvature task. The symbol represents the concavity or convexity of the curvature. The larger the absolute value, the more severe the curvature at that point.

[0125] like Figure 7 The figure shows the prediction results obtained from the feature point confidence task. The closer the confidence value is to 1, the more likely the point is to be a real, stable, and repeatable feature point (such as corner points, edge points, etc.).

[0126] The present invention also relates to a multi-task processing system for 3D point clouds based on the Mamba model, comprising a processor and a memory, wherein the memory stores computer program instructions, and when the processor executes the computer program instructions, it implements the multi-task processing method for 3D point clouds based on the Mamba model.

[0127] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0131] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0132] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-task processing method for 3D point clouds based on the Mamba model, characterized in that: The original point cloud is acquired, preprocessed, and high-dimensional features that fuse local geometry and spatial location information are obtained simultaneously. The high-dimensional features are input into a task-adaptive parameterized Mamba model and a hybrid expert pool to obtain the discrimination results of several experts. The high-dimensional features are then processed by one or more task routes to obtain corresponding task expert weights, hybrid weights, and noise weights. Based on the task expert weights, the discrimination results of all the experts are weighted and fused, and combined with learnable prior parameters and linear projection of features based on hybrid weights to generate task-adaptive model parameters. Using these task-adaptive model parameters, the high-dimensional features and their inverse features are processed through a parallel state-space model, fusing them to obtain a high-dimensional implicit hypersurface used to characterize the local geometry of the point cloud. Feature filtering is performed based on the high-dimensional implicit hypersurface to extract core feature representations, and task results corresponding to one or more query points are calculated.

2. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 1, characterized in that: Based on each query point in the original point cloud, a neighborhood block is constructed and standardized. Then, based on a preset point cloud local feature encoding strategy, the local features of each neighborhood block are encoded in parallel to obtain the corresponding high-dimensional features.

3. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 2, characterized in that: Point cloud local feature encoding strategies include: Geometric feature encoding is performed on the standardized neighborhood blocks to obtain local geometric features; Spatial location encoding is performed on the standardized neighborhood blocks to obtain local spatial location features; By fusing local geometric features and local spatial location features, high-dimensional features are obtained.

4. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 3, characterized in that: The relative coordinates of each point within a neighborhood block are nonlinearly mapped using a multilayer perceptron to obtain initial local geometric features. These initial local geometric features are then weighted, aggregated, and fused across scales using an attention mechanism to obtain context-aware optimized local geometric features.

5. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 3, characterized in that: The neighborhood block is downsampled, and a preset number of nearest neighbors to the query point are retained. Based on a learnable encoding function, the relative coordinates of each point in the downsampled neighborhood block are calculated to obtain local spatial location features.

6. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 1, characterized in that: Noise based on noise weights is injected into the weights of task experts, and TopK sparsity processing is performed to retain the top K experts with the highest weight values.

7. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 1, characterized in that: The high-dimensional implicit hypersurface is weighted by a multilayer perceptron, and the weights are multiplied by the high-dimensional implicit hypersurface to obtain the core feature representation.

8. The method for multi-task processing of 3D point clouds based on the Mamba model according to claim 1, characterized in that: The task includes obtaining the normal vector estimate, average curvature, and feature point confidence; and inputting the core feature representations into the normal vector decoding head, curvature decoding head, and confidence decoding head, respectively. The normal vector decoding head outputs a three-dimensional vector and performs normalization processing to obtain the estimated value of the normal vector; The curvature decoding head outputs a scalar to obtain the average curvature; The confidence decoding head outputs a scalar between 0 and 1 to obtain the confidence of the feature point.

9. A multi-task processing system for 3D point clouds based on the Mamba model, characterized in that: It includes a processor and a memory, wherein the memory stores computer program instructions, and when the processor executes the computer program instructions, it implements the 3D point cloud multi-task processing method based on the Mamba model as described in any one of claims 1 to 8.

Images