Point cloud completion method based on space-time joint network
By fusing local and global features through a spatiotemporal joint network, the problem of insufficient global consistency and generalization ability of traditional point cloud completion methods is solved, achieving higher point cloud completion accuracy and robustness, and adapting to point cloud completion tasks on different datasets.
Patent Information
- Application Number
- CN202511448877.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-11
- Publication Date
- 2026-02-13
AI Technical Summary
Traditional point cloud completion methods are insufficient in terms of global consistency and generalization ability, making it difficult to adapt to point cloud completion tasks in different datasets and environments. Furthermore, deep learning-based methods are deficient in terms of interaction between local structure and global features, resulting in low accuracy and robustness of the completion results.
A spatiotemporal joint network-based approach is adopted. By fusing local angle encoding, local coordinate encoding and local feature information, an adaptive local feature information module and a Mamba global feature information module are constructed. Combined with a two-branch adaptive Mamba network and a neighborhood cross Transformer, a key point set is generated and optimized through a loss function, and finally a complete point cloud is output.
It improves the accuracy and robustness of point cloud completion, can adapt to different datasets, achieves better versatility and accuracy, and enhances the effect of point cloud completion.
Smart Images
Figure CN121526894A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically a point cloud completion method based on spatiotemporal joint networks. Background Technology
[0002] With the development of 3D vision technology, point cloud data has become an indispensable and important data form in many fields such as autonomous driving, outdoor environment perception and virtual reality. However, due to the influence of factors such as sensor resolution, viewing angle limitations and occlusion, the acquired point cloud data usually has missing and incomplete problems, which makes the analysis and application of point cloud data face many challenges.
[0003] Point cloud completion, a crucial task in 3D vision, aims to infer missing parts from incomplete point cloud data to obtain a more complete and accurate 3D model. However, the sparsity and irregularity of point cloud data increase the complexity of the completion task. Traditional point cloud completion methods typically rely on manually designed rules or optimized models, utilizing only local information for completion. This lack of understanding of the global structure leads to significant deficiencies in the global consistency of the completion results, making it difficult to generalize to different datasets and application scenarios. Furthermore, traditional methods have limited generalization ability when faced with point cloud data from different sources, of varying quality, and with different densities, failing to effectively adapt to point cloud completion tasks in diverse environments.
[0004] To overcome the problems of traditional point cloud completion methods, deep learning-based point cloud completion methods have emerged in recent years. These methods learn more accurate geometric information from partial point cloud data through deep neural networks, achieving more refined completion results. For example, methods based on generation, point, convolution, and Transformer learn local and global geometric structure information to predict missing parts, thus completing the point cloud. Despite the progress made in deep learning-based point cloud completion methods, some problems remain: First, their generalization ability across different data sources (such as LiDAR, multi-view cameras, etc.) is limited, making it difficult to adapt to the characteristics of different datasets; second, they are insufficient in capturing the effective interaction between local structure and global features of point clouds, resulting in low accuracy and robustness of the completion results. Summary of the Invention
[0005] The purpose of this invention is to provide a point cloud completion method based on spatiotemporal joint networks, which can complete incomplete point cloud data caused by limitations of acquisition equipment or environment, adapt to different datasets, has good versatility, and improves the accuracy and robustness of point cloud completion.
[0006] This invention is achieved through the following technical solution: A point cloud completion method based on spatiotemporal joint networks includes the following steps: Step 1: Integrate local angle coding, local coordinate coding, and local feature information to obtain multi-composite position coding features; Step 2: Construct a two-branch adaptive Mamba network including an adaptive local feature information module and a Mamba global feature information module to achieve multi-level modeling of point cloud features. Specifically, the adaptive local feature information module utilizes the multi-composite location encoding features of points within the neighborhood, and through weighted aggregation and an adaptive weight matrix... Learning local features and calculating adaptive local features; the Mamba global feature information module uses... Normalized multi-composite location coding features are used as input, and are processed by depthwise separable convolution, forward state space model and backward state space model to obtain forward concatenation features and backward concatenation features. These features are then fused by a multilayer perceptron to generate global features. The adaptive local features and global features are then concatenated and input into the multilayer perceptron to obtain the output features of the branch adaptive Mamba network. Step 3: The seed point generator first uses the output features of the two-branch adaptive Mamba network. Select a subset of high-response points from the original point cloud as global feature points. P A subset of points are selected as sampling points. Then, a perturbation factor is added to the sampling points to generate phase perturbation points. Finally, the global feature points, sampling points, and phase perturbation points are combined to obtain a set of key points. Step 4: Alternately use the dual-branch adaptive Mamba network and the neighborhood cross-transformer as a spatiotemporal joint network to output the complete point cloud; Step 5: Design an incomplete point cloud input. Output complete point cloud The loss function between them is expressed as: In the formula: Ground Truth points represent the complete point cloud; It is a constant, 0 < 1; This represents the chamfer distance function.
[0007] Furthermore, the specific process of step 1 is as follows: Step 1.1: Define a point cloud with incomplete input. , is represented as: In the formula: N Representative point cloud Points; Step 1.2, Local Angle Encoding: Define Points The set of neighborhood points is Modeling is performed on the polar angle and azimuth angle information between a point in the point cloud and its neighboring points. By calculating the angle between the neighboring points and the center point, an angle feature vector, i.e., local angle encoding, is generated, represented as: In the formula: and Representing points respectively With neighboring points The polar coordinate angles and azimuths between them ; Step 1.3, Local coordinate encoding: for points Its neighboring points The relative positions between them define the coordinate difference. With point The splicing of points is used to characterize the spatial geometric structure of point clouds within a local neighborhood, and points are set. The local coordinate encoding is then defined as: Step 1.4, Local Feature Information: Given each point The initial feature vector is ,Will Features of its neighboring points are aggregated using a function After processing, they collectively form a local feature description, i.e., local feature information, represented as: In the formula: For feature aggregation functions; Step 1.5: Fusion and Generation of Multi-Compound Location Codes: The local angle code, local coordinate code, and local feature information are concatenated and input into a multilayer perceptron to obtain multi-compound location code features with enhanced expressive power. , is represented as: In the formula: This represents a multi-layered sensing mechanism.
[0008] Furthermore, the specific process of step 2 is as follows: Step 2.1: The adaptive local feature information module utilizes the multi-composite location encoding features of points within the neighborhood, and through weighted aggregation and adaptive weight matrix... Learn local features and calculate adaptive local features, represented as: In the formula: Represents the global characteristic function; Represents the activation function; The weight matrix represents adaptive learning; and Representing points respectively The dimension; Step 2.2, Mamba Global Feature Information Module: Input is Normalized multi-composite positional encoding features are processed by depthwise separable convolution (DWConv), forward state space model (FSSM), and backward state space model (BSSM) to obtain forward concatenated features. and backward splicing features Global features are generated through fusion via a multilayer perceptron. The specific process is as follows: Step 2.2.1: Input features Normalization, expressed as: In the formula: Represents root mean square standardization; The features before input to the State-Space Model (SSM); Represents normalization; Step 2.2.2: Calculate the forward state space model FSSM and the backward state space model BSSM using depthwise separable convolution (DWConv) and the state space model SSM, respectively: In the formula: This represents depthwise separable convolution; Represents the activation function; Represents the forward state space model; Represents the backward state-space model; Represents root mean square standardization; Step 2.2.3: Construct forward splicing features and backward splicing features , is represented as: In the formula: Represents root mean square standardization; Indicates normalization; Represents the activation function; Indicates splicing; Step 2.2.4: Fusion via multilayer perceptron and The global features are obtained and represented as: In the formula: This represents a multi-layered sensing mechanism; Indicates splicing; Step 2.3, Dual-branch feature fusion: Integrating adaptive local features and global features The concatenated data is fed into a multilayer perceptron for local-local and local-global combined learning. The final output features of the dual-branch adaptive Mamba network are represented as follows: In the formula: This represents a multi-layered sensing mechanism; Indicates splicing.
[0009] Furthermore, the specific process of step 3 is as follows: Step 3.1, Global Feature Sampling: Extracting output features from the dual-branch adaptive Mamba network Select a subset of high-response points as global feature points. , is represented as: In the formula: Indicates the selection before K One high-response point; This represents a multi-layered sensing mechanism; Step 3.2, Input Point Cloud Sampling: A random sampling strategy is adopted to sample the original input point cloud. P Selecting a subset of points to form sampling points , is represented as: Step 3.3, Phase Perturbation Enhancement: To enhance the robustness of the model, additional phase perturbation points were added to the key points. To form an enhanced set of key points, phase perturbation points By sampling points The result obtained by adding a perturbation factor is expressed as: In the formula: Indicates the disturbance factor; This indicates that the mean is 0 and the variance is 0. The normal distribution; Step 3.4: Key point set generation: Integrating global feature points Sampling points and phase perturbation point The key point set is obtained, represented as: .
[0010] Furthermore, the specific process of step 4 is as follows: Step 4.1, Local-Global Interaction: [This step involves] key points... Input a bi-branch adaptive Mamba network to extract local and global interaction features. , is represented as: Step 4.2, Neighborhood Cross Transformer Fusion: In the neighborhood cross Transformer, the attention weights between keypoints are calculated. , is represented as: In the formula: This represents the normalized exponential function; Point , neighborhood points; , , and Both represent multi-layer sensing mechanisms; Then, by combining the feature difference and the position difference, multi-dimensional feature interaction across neighborhoods is achieved, resulting in the fused features, represented as: In the formula: Represents the dot product; This represents a multi-layered sensing mechanism; express Neighborhood characteristics; Point With point The coordinate difference; Step 4.3, Point Cloud Reconstruction: Based on alternating iterative local-global interaction and Transformer fusion, new point cloud features are continuously generated to gradually complete the point cloud. P The missing regions are then output as complete point clouds. , is represented as: In the formula: M Representative point cloud Points; This represents a multi-layered perception mechanism.
[0011] Furthermore, the chamfer distance in step 5 is expressed as: In the formula: Point cloud P The point in the middle; Point cloud The point in the middle; Point cloud P The number of midpoints; Point cloud The number of midpoints.
[0012] The present invention has the following beneficial technical effects: First, multi-composite location encoding, through local location encoding, local angle encoding, and local feature integration, enables each point to more comprehensively express its local structure and spatial relationships, improving the perception ability of local structures at different angles and scales. Second, the bi-branch adaptive Mamba network can fully integrate local-local and local-global point cloud feature information. Third, the spatiotemporal joint network combines the bi-branch adaptive Mamba network with the neighborhood cross-transformer, realizing effective interaction between local and global features, making the spatial and temporal consistency of the completion results and improving the accuracy of point cloud completion. Therefore, this invention can complete incomplete point cloud data caused by limitations of acquisition equipment or environment, and adapts to different datasets, exhibiting good versatility and improving the robustness and accuracy of point cloud completion. Attached Figure Description
[0013] Figure 1 This forms the overall framework of the present invention; Figure 2 This is the multi-composite position coding framework of the present invention; Figure 3 This invention provides a dual-branch adaptive Mamba network framework. Figure 4 This is the visualization result of point cloud completion in the PCN dataset according to the present invention; Figure 5 This is the visualization result of point cloud completion in the Completion3D dataset of this invention; Figure 6 This is the visualization result of point cloud completion on the ShapeNet-55 dataset according to the present invention; Figure 7 This is the visualization result of point cloud completion in the MVP dataset of this invention; Figure 8 This is the visualization result of point cloud completion in the KITTI dataset according to the present invention. Detailed Implementation
[0014] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.
[0015] like Figure 1 As shown, a point cloud completion method based on a spatiotemporal joint network includes the following steps: Step 1, Multi-composite positional encoding: such as Figure 2 As shown, this includes local coordinate encoding. Local angle encoding and local feature information It enhances the expressive power of point clouds by capturing their local geometric information and spatial angular relationships. The specific process is as follows: Step 1.1: Define a point cloud with incomplete input. , is represented as: In the formula: N represents the point cloud. Points; Step 1.2, Local Angle Encoding: Define Points The set of neighborhood points is Modeling is performed on the polar angle and azimuth angle information between a point in the point cloud and its neighboring points. By calculating the angle between the neighboring points and the center point, an angle feature vector, i.e., local angle encoding, is generated, represented as: In the formula: and Representing points respectively With neighboring points The polar coordinate angles and azimuths between them ; Step 1.3, Local coordinate encoding: for points Its neighboring points The relative positions between them define the coordinate difference. With point The splicing of points is used to characterize the spatial geometric structure of point clouds within a local neighborhood, and points are set. The local coordinate encoding is then defined as: Step 1.4, Local Feature Information: Given each point The initial feature vector is This provides additional contextual information by capturing the feature vector of each point. Features of its neighboring points are aggregated using a function After processing, they collectively form a local feature description, i.e., local feature information, represented as: In the formula: For feature aggregation functions; Step 1.5: Fusion and Generation of Multi-Compound Location Codes: The local angle code, local coordinate code, and local feature information are concatenated and input into a multilayer perceptron to obtain multi-compound location code features with enhanced expressive power. , is represented as: In the formula: This represents a multi-layered sensing mechanism; Step 2, construct as follows Figure 3 The dual-branch adaptive Mamba network shown includes an adaptive local feature information module and a Mamba global feature information module. Through local-local and local-global learning, it achieves multi-level modeling of point cloud features. The specific process is as follows: Step 2.1, Adaptive Local Feature Information Module: Utilizing the multi-composite location encoding features of points within the neighborhood, through weighted aggregation and adaptive weight matrix... Learn local features and calculate adaptive local features, represented as: In the formula: Represents the global characteristic function; Represents the activation function; The weight matrix represents adaptive learning; and Representing points respectively The dimension; Step 2.2, Mamba Global Feature Information Module: Input is Normalized multi-composite positional encoding features are processed by depthwise separable convolution (DWConv), forward state space model (FSSM), and backward state space model (BSSM) to obtain forward concatenated features. and backward splicing features Global features are generated through fusion via a multilayer perceptron. The specific process is as follows: Step 2.2.1: Input features Normalization, expressed as: In the formula: Represents root mean square standardization; The features before input to the State-Space Model (SSM); Represents normalization; Step 2.2.2: Calculate the forward state space model FSSM and the backward state space model BSSM using depthwise separable convolution (DWConv) and the state space model SSM, respectively: In the formula: This represents depthwise separable convolution; Represents the activation function; Represents the forward state space model; Represents the backward state-space model; Represents root mean square standardization; Step 2.2.3: Construct forward splicing features and backward splicing features , is represented as: In the formula: Represents root mean square standardization; Indicates normalization; Represents the activation function; Indicates splicing; Step 2.2.4: Fusion via multilayer perceptron and The global features are obtained and represented as: In the formula: This represents a multi-layered sensing mechanism; Indicates splicing; Step 2.3, Dual-branch feature fusion: Integrating adaptive local features and global features The concatenated data is fed into a multilayer perceptron for local-local and local-global combined learning. The final output features of the dual-branch adaptive Mamba network are represented as follows: Step 3, Seed Point Generator: This step aims to generate a set of high-quality initial keypoints to represent the main features, structures, or key regions of the point cloud surface. These keypoints reduce the search space required by the point cloud completion algorithm, thereby better understanding the contextual information of missing regions. By guiding subsequent point cloud completion algorithms, they help to complete local and global point clouds. The process of generating seed points is as follows: Step 3.1, Global Feature Sampling: Extracting output features from the dual-branch adaptive Mamba network Select a subset of high-response points as global feature points. , is represented as: In the formula: Indicates the selection before K One high-response point; This represents a multi-layered sensing mechanism; Step 3.2, Input Point Cloud Sampling: A random sampling strategy is adopted to sample the original input point cloud. P Selecting a subset of points to form sampling points , is represented as: Step 3.3, Phase Perturbation Enhancement: To enhance the robustness of the model, additional phase perturbation points are added to the key points. To form an enhanced set of key points, phase perturbation points By sampling points The result obtained by adding a perturbation factor is expressed as: In the formula: Indicates the disturbance factor; This indicates that the mean is 0 and the variance is 0. The normal distribution; Step 3.4: Key point set generation: Integrating global feature points Sampling points and phase perturbation point The key point set is obtained, represented as: Step 4, Spatiotemporal Joint Network: Alternately use a dual-branch adaptive Mamba network and a neighborhood cross-transformer to output a complete point cloud. The specific process is as follows: Step 4.1, Local-Global Interaction: [This step involves] key points... Input a bi-branch adaptive Mamba network to extract local and global interaction features. , is represented as: Step 4.2, Neighborhood Cross Transformer Fusion: In the neighborhood cross Transformer, the attention weights between keypoints are calculated. , is represented as: In the formula: This represents the normalized exponential function; Point , neighborhood points; , , and Both represent multi-layer sensing mechanisms; Then, by combining the feature difference and the position difference, multi-dimensional feature interaction across neighborhoods is achieved, resulting in the fused features, represented as: In the formula: Represents the dot product; This represents a multi-layered sensing mechanism; express Neighborhood characteristics; Point With point The coordinate difference; Step 4.3, Point Cloud Reconstruction: Based on alternating iterative local-global interaction and Transformer fusion, new point cloud features are continuously generated to gradually complete the point cloud. P The missing regions are then output as complete point clouds. , is represented as: In the formula: M Representative point cloud Points; Represents a multi-layered sensing mechanism; Step 5, Loss Function Design: For incomplete point cloud input Output complete point cloud The Chamfer Distance (CD) function is used as the loss function to measure the average distance between spatial nearest neighbors, where: The chamfer distance is expressed as: Then, dotted clouds And point cloud The loss function is expressed as: In the formula: Representative point cloud And point cloud The loss function between Ground Truth points represent the complete point cloud; It is a constant, 0 < 1; Point cloud The point in the middle; Point cloud The point in the middle; Point cloud The number of midpoints; Point cloud The number of midpoints.
[0016] To verify the superior performance of the point cloud completion method based on spatiotemporal joint network proposed in this embodiment, point cloud completion was performed on the CAD synthetic dataset PCN, Completion3D and ShapeNet-55, the real camera multi-view MVP dataset, and the LiDAR acquisition dataset KITTI, respectively. Experiment 1: Verify the performance of the point cloud completion method proposed in this embodiment on the PCN dataset. The PCN dataset contains 8 classes and a total of 30,974 shapes. The number of samples used for training, validation, and testing are 28,974, 800, and 1,200, respectively. The visualization results of the point cloud completion method based on spatiotemporal joint networks proposed in this embodiment are as follows: Figure 4 As shown, Figure 4 The input is an incomplete point cloud, and the output is the completed point cloud. It can be seen that after completion, the overall shape of the point cloud is more complete, the missing geometric structure is effectively restored, and the details are also smoother and more accurate.
[0017] Experiment 2: Verify the performance of the point cloud completion method proposed in this embodiment on the Completion3D dataset. The Completion3D dataset contains 8 classes and a total of 30,958 shapes. Unlike the PCN dataset, this dataset includes both partial and complete shapes. 28,974 shapes were used for training, 800 for validation, and 1,184 for testing. The visualization results of the point cloud completion method based on a spatiotemporal joint network proposed in this embodiment are shown below. Figure 5 As shown, Figure 5 The input part is an incomplete point cloud, and the output part is the completed point cloud. It can be seen that the input point cloud has significant missing parts, while the output result can restore the complete geometric structure. For example, for categories such as airplanes and chairs, the completion result not only fills in the missing parts, but also has a high degree of consistency with the real structure in terms of overall appearance. This shows that the point cloud completion method proposed in this embodiment has good generalization ability and robustness.
[0018] Experiment 3: Verify the performance of the point cloud completion method proposed in this embodiment on the ShapeNet-55 dataset. The ShapeNet-55 dataset contains 55 categories and a total of 52,470 shapes. 41,952 shapes were used for training, and 10,518 were used for testing. The visualization results of the point cloud completion method based on a spatiotemporal joint network proposed in this embodiment are as follows: Figure 6 As shown, the point cloud completion method proposed in this embodiment can adapt to more categories and complex structures, and maintains high completion quality when facing diverse data. Even if the input point cloud has serious defects, it can still reconstruct a relatively reasonable and complete three-dimensional shape.
[0019] Experiment 4: Verify the performance of the point cloud completion method proposed in this embodiment on the MVP dataset. The MVP dataset is a collection of over 100,000 multi-view partial point cloud data points, containing 16 shape classes, including both partial and complete shapes. The visualization results of the point cloud completion method based on a spatiotemporal joint network proposed in this embodiment are shown below. Figure 7As shown, the point cloud completion method proposed in this embodiment successfully completed large missing regions in experiments with complex categories (such as ships and sofas), demonstrating its effectiveness in completion tasks.
[0020] Experiment 5: Verify the performance of the point cloud completion method proposed in this embodiment on the KITTI dataset. The KITTI dataset, collected using a laser scanner, includes various road scenes such as city streets, highways, and rural roads, as well as real-world outdoor scene data of pedestrians, vehicles, and buildings. The visualization results of the point cloud completion method based on a spatiotemporal joint network proposed in this embodiment are shown below. Figure 8 As shown, even if the input point cloud is severely sparse and missing due to scanning limitations, it can still recover the overall outline and geometry of the vehicle. This demonstrates that the point cloud completion method proposed in this embodiment still has good adaptability and practicality in real sparse scenarios.
[0021] The visualization results from Experiments 1 through 5 demonstrate that the point cloud completion method based on a spatiotemporal joint network proposed in this embodiment exhibits excellent versatility and superior completion performance across point cloud data generated from different datasets and sensors. This is achieved by combining multi-composite position coding, a two-branch adaptive Mamba network, and a spatiotemporal joint network. Specifically, multi-composite position coding captures the local structure and angular relationships of the point cloud, improving the model's ability to perceive local features; the two-branch adaptive Mamba network effectively learns and fuses local and global features of the point cloud through the combination of adaptive local feature information modules and global feature information modules; and the spatiotemporal joint network, during the decoding stage, achieves interaction between local and global point cloud information through the alternating use of neighborhood cross-transformers and the two-branch adaptive Mamba network, thereby improving the accuracy of point cloud completion.
Claims
1. A point cloud completion method based on a spatiotemporal joint network, characterized in that, Includes the following steps: Step 1: Integrate local angle coding, local coordinate coding, and local feature information to obtain multi-composite position coding features; Step 2: Construct a two-branch adaptive Mamba network including an adaptive local feature information module and a Mamba global feature information module to achieve multi-level modeling of point cloud features. Specifically, the adaptive local feature information module utilizes the multi-composite location encoding features of points within the neighborhood, and through weighted aggregation and an adaptive weight matrix... Learning local features and calculating adaptive local features; the Mamba global feature information module uses... Normalized multi-composite location coding features are used as input, and are processed by depthwise separable convolution, forward state space model and backward state space model to obtain forward concatenation features and backward concatenation features. These features are then fused by a multilayer perceptron to generate global features. The adaptive local features and global features are then concatenated and input into the multilayer perceptron to obtain the output features of the dual-branch adaptive Mamba network. Step 3: The seed point generator first uses the output features of the two-branch adaptive Mamba network. Select a subset of high-response points from the original point cloud as global feature points. P A subset of points are selected as sampling points. Then, a perturbation factor is added to the sampling points to generate phase perturbation points. Finally, the global feature points, sampling points, and phase perturbation points are combined to obtain a set of key points. Step 4: Alternately use the bi-branch adaptive Mamba network and the neighborhood cross-transformer as a spatiotemporal joint network to output the complete point cloud. ; Step 5: Design an incomplete point cloud input. Output complete point cloud The loss function between them is expressed as: In the formula: Ground Truth points represent the complete point cloud; It is a constant, 0 < 1; This represents the chamfer distance function.
2. The point cloud completion method based on spatiotemporal joint network according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Define a point cloud with incomplete input. , is represented as: In the formula: N Representative point cloud Points; Step 1.2, Local Angle Encoding: Define Points The set of neighborhood points is Modeling is performed on the polar angle and azimuth angle information between a point in the point cloud and its neighboring points. By calculating the angle between the neighboring points and the center point, an angle feature vector, i.e., local angle encoding, is generated, represented as: In the formula: and Representing points respectively With neighboring points The polar coordinate angles and azimuths between them ; Step 1.3, Local coordinate encoding: for points Its neighboring points The relative positions between them define the coordinate difference. With point The splicing of points is used to characterize the spatial geometric structure of point clouds within a local neighborhood, and points are set. The local coordinate encoding is then defined as: Step 1.4, Local Feature Information: Given each point The initial feature vector is ,Will Features of its neighboring points are aggregated using a function After processing, they collectively form a local feature description, i.e., local feature information, represented as: In the formula: For feature aggregation functions; Step 1.5: Fusion and Generation of Multi-Compound Location Codes: The local angle code, local coordinate code, and local feature information are concatenated and input into a multilayer perceptron to obtain multi-compound location code features with enhanced expressive power. , is represented as: In the formula: This represents a multi-layered sensing mechanism.
3. The point cloud completion method based on spatiotemporal joint network according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: The adaptive local feature information module utilizes the multi-composite location encoding features of points within the neighborhood, and through weighted aggregation and adaptive weight matrix... Learn local features and calculate adaptive local features, represented as: In the formula: Represents the global characteristic function; Represents the activation function; The weight matrix represents adaptive learning; and Representing points respectively The dimension; Step 2.2, Mamba Global Feature Information Module: Input is Normalized multi-composite positional encoding features are processed by depthwise separable convolution (DWConv), forward state space model (FSSM), and backward state space model (BSSM) to obtain forward concatenated features. and backward splicing features Global features are generated through fusion via a multilayer perceptron. The specific process is as follows: Step 2.2.1: Input features Normalization, expressed as: In the formula: Represents root mean square standardization; The features before input to the State-Space Model (SSM); Represents normalization; Step 2.2.2: Calculate the forward state space model FSSM and the backward state space model BSSM using depthwise separable convolution (DWConv) and the state space model SSM, respectively: In the formula: This represents depthwise separable convolution; Represents the activation function; Represents the forward state space model; Represents the backward state-space model; Represents root mean square standardization; Step 2.2.3: Construct forward splicing features and backward splicing features , is represented as: In the formula: Represents root mean square standardization; Indicates normalization; Represents the activation function; Indicates splicing; Step 2.2.4: Fusion via multilayer perceptron and The global features are obtained and represented as: In the formula: This represents a multi-layered sensing mechanism; Indicates splicing; Step 2.3, Dual-branch feature fusion: Integrating adaptive local features and global features The concatenated data is fed into a multilayer perceptron for local-local and local-global combined learning. The final output features of the dual-branch adaptive Mamba network are represented as follows: In the formula: This represents a multi-layered sensing mechanism; Indicates splicing.
4. The point cloud completion method based on spatiotemporal joint network according to claim 1, characterized in that, The specific process of step 3 is as follows: Step 3.1, Global Feature Sampling: Extracting output features from the dual-branch adaptive Mamba network Select a subset of high-response points as global feature points. , is represented as: In the formula: Indicates the selection before K One high-response point; This represents a multi-layered sensing mechanism; Step 3.2, Input Point Cloud Sampling: A random sampling strategy is adopted to sample the original input point cloud. P Selecting a subset of points to form sampling points , is represented as: Step 3.3, Phase Perturbation Enhancement: To enhance the robustness of the model, additional phase perturbation points are added to the key points. To form an enhanced set of key points, phase perturbation points By sampling points The result obtained by adding a perturbation factor is expressed as: In the formula: Indicates the disturbance factor; This indicates that the mean is 0 and the variance is 0. The normal distribution; Step 3.4: Key point set generation: Integrating global feature points Sampling points and phase perturbation point The key point set is obtained, represented as: 。 5. The point cloud completion method based on spatiotemporal joint network according to claim 1, characterized in that, The specific process of step 4 is as follows: Step 4.1, Local-Global Interaction: [This step involves] key points... Input a bi-branch adaptive Mamba network to extract local and global interaction features. , is represented as: Step 4.2, Neighborhood Cross Transformer Fusion: In the neighborhood cross Transformer, the attention weights between keypoints are calculated. , is represented as: In the formula: This represents the normalized exponential function; Point , neighborhood points; , , and Both represent multi-layer sensing mechanisms; Then, by combining the feature difference and the position difference, multi-dimensional feature interaction across neighborhoods is achieved, resulting in the fused features, represented as: In the formula: Represents the dot product; This represents a multi-layered sensing mechanism; express Neighborhood characteristics; Point With point The coordinate difference; Step 4.3, Point Cloud Reconstruction: Based on alternating iterative local-global interaction and Transformer fusion, new point cloud features are continuously generated to gradually complete the point cloud. P The missing regions are then output as complete point clouds. , is represented as: In the formula: M Representative point cloud Points; This represents a multi-layered perception mechanism.
6. The point cloud completion method based on spatiotemporal joint network according to claim 1, characterized in that, The chamfer distance in step 5 is expressed as: In the formula: Point cloud P The point in the middle; Point cloud The point in the middle; Point cloud P The number of midpoints; Point cloud The number of midpoints.