A geometric-semantic joint embedding point cloud noise filtering method and system

Abstract

This invention discloses a point cloud noise removal method and system based on geometric-semantic joint embedding, belonging to the fields of deep learning and 3D data processing. The method first constructs multiple initial features, including coordinates, normal vectors, and curvature, for each point in the point cloud and maps them to initial embeddings. Next, it performs nearest-neighbor aggregation based on geometric distance in 3D physical space to obtain distance embeddings. Then, it calculates low-order attention scores based on geometric attributes in 3D physical space and performs weighted aggregation of distance embeddings to represent geometric consistency. Next, it calculates high-order attention scores based on semantic similarity in high-dimensional embedding space and performs weighted aggregation of distance embeddings to represent semantic consistency. Finally, it fuses the low-order and high-order attention embeddings to predict the existence probability of each point in the point cloud and remove noise points. This invention, through a dual-space, dual-order attention aggregation mechanism, jointly models geometric consistency and semantic context, significantly improving the accuracy and detail preservation capability of point cloud noise removal in complex scenes.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and 3D data processing technology, and in particular to a point cloud noise filtering method and system using geometric-semantic joint embedding. Background Technology

[0002] 3D point cloud data, as an important form of spatial perception, has been widely used in fields such as autonomous driving, robot navigation, augmented reality, and industrial inspection. However, due to limitations in sensor accuracy, environmental interference, or poor acquisition conditions, raw point clouds often contain a large number of noisy points (such as outliers, floating points, and density anomalies), severely impacting the performance of subsequent high-level tasks such as 3D reconstruction and target recognition. Therefore, point cloud noise filtering technology has become one of the research hotspots in the field of 3D vision.

[0003] Existing point cloud denoising methods are mainly divided into traditional geometric filtering methods and deep learning-based denoising methods. Traditional methods (such as filtering based on local geometric features and graph signal processing) typically rely on low-order geometric information such as normal vectors and curvature, setting thresholds to remove outliers. However, they struggle to distinguish between real details and noise in structurally complex regions, exhibiting poor robustness. In recent years, deep learning-based denoising methods have made significant progress, with the Transformer architecture employing attention mechanisms attracting widespread attention due to its powerful modeling capabilities for global and local relationships in point clouds.

[0004] For example, Chinese patent application CN118397286A (published on July 26, 2024) discloses a "nested attention 3D point cloud denoising network within and between branches". This scheme extracts multi-scale features using a three-branch graph attention module in the feature embedding layer, then enhances and fuses these features through intra-branch and inter-branch attention. Finally, the denoising layer predicts the offset of each noise point relative to the clean point cloud surface and adjusts the point cloud position using regression to complete denoising. This method utilizes a nested attention mechanism to improve the performance of point cloud denoising to a certain extent.

[0005] However, the aforementioned existing technologies still have the following shortcomings in practical applications: First, they model the denoising task as a regression problem, reconstructing point clouds by predicting the offset of points from the real surface. This approach is prone to over-smoothing when dealing with complex geometric regions such as edges and fine structures, leading to the accidental deletion of effective details or structural distortion. Second, the initial features of this method rely only on the implicit features after linear transformation of the original coordinates, lacking explicit encoding of local geometric structures (such as normal vectors and curvature), which limits the model's ability to perceive complex geometric shapes. Third, their attention mechanisms are all calculated in a unified feature space, failing to distinguish between low-level geometric consistency (such as surface smoothness and continuity) and high-level semantic context (such as component attribution), resulting in a lack of hierarchical constraints in the model's discrimination between noise and structural details, especially in geometrically significant regions such as edges and corners, where it is difficult to maintain both geometric fidelity and semantic consistency simultaneously.

[0006] Therefore, how to design a point cloud noise filtering method that can accurately distinguish between valid points and noise points in complex noise environments, while avoiding over-smoothing and accidental deletion of details, is a technical problem that urgently needs to be solved in the current field of technology. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a point cloud noise filtering method and system with geometric-semantic joint embedding. This method uses dual-space dual-order attention aggregation to extract geometric consistency and semantic consistency features respectively, so as to solve the problem that point cloud noise and effective details are difficult to distinguish in complex scenes.

[0008] To achieve the above objectives, the present invention provides the following technical solution: The point cloud noise filtering method based on geometric-semantic joint embedding provided by this invention includes the following steps: Step 1, Initial Feature Construction and Embedding Generation: Obtain multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; Step 2, nearest neighbor aggregation: In the three-dimensional physical space, based on the geometric distance between each point and its nearest neighbor, the distance aggregation weight is calculated, and the initial embedding representation of the nearest neighbor is weighted and aggregated to generate the distance embedding; Step 3, Low-order neighborhood attention aggregation: In the three-dimensional physical space, based on the geometric properties between each point and its nearest neighbor, a low-order attention score is calculated, and the distance embedding is weighted and aggregated according to the low-order attention score to generate a low-order attention embedding to represent geometric consistency. Step 4, High-order Neighborhood Attention Aggregation: In the high-dimensional embedding space, based on the semantic feature similarity between each point and its neighboring points, a high-order attention score is calculated, and the distance embedding is weighted and aggregated according to the high-order attention score to generate a high-order attention embedding to represent semantic consistency. Step 5, Point Existence Probability Prediction: The low-order attention embedding and the high-order attention embedding are fused, and the existence probability of each point in the point cloud is predicted by the decoder. Noise points are then filtered out based on the existence probability.

[0009] Furthermore, the multiple geometric properties mentioned in step 1 include the three-dimensional coordinates of the point, the normal vector, and the curvature.

[0010] Furthermore, the normal vector is obtained in the following way: For any point in the point cloud, find its k nearest neighbors in the three-dimensional physical space neighborhood and calculate the covariance matrix of the neighbors. Perform eigenvalue decomposition on the covariance matrix and take the eigenvector corresponding to the smallest eigenvalue as the normal vector; The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is the curvature at that point. The calculation formula is: ; λ1 ≤ λ2 ≤ λ3; in, Represents the smallest eigenvalue; Indicates intermediate eigenvalues; Represents the largest eigenvalue; Furthermore, the distance aggregation weight described in step 2 The calculation formula is: , , in, Indicates distance Recent One point; Indicates the current point; Represents point clouds; This means finding the largest value from a set of data. One element; Represents the relationship between two vectors distance; This indicates normalization.

[0011] Furthermore, the calculation method of the low-order attention score in step 3 is as follows: In the three-dimensional physical space, the distance embedding is mapped to query, key and value respectively through a trainable matrix, the low-order attention weight is calculated based on the similarity between query and key, and the distance embedding is weighted and aggregated using the weight.

[0012] Furthermore, the calculation method of the higher-order attention score in step 4 is as follows: In the high-dimensional embedding space, the distance embedding is mapped to query, key and value respectively through a trainable matrix. Based on the similarity between query and key, the neighboring points of the top k largest attention scores are selected, the higher-order attention weight is calculated, and the distance embedding is weighted and aggregated using the weight.

[0013] Furthermore, the fusion method described in step 5 involves concatenating the low-order attention embedding with the high-order attention embedding and then inputting the result into a multilayer perceptron decoder to output the existence probability of each point.

[0014] Furthermore, it also includes a model training step: employing supervised learning, using the cross-entropy loss function to calculate the prediction loss, and backpropagating the loss to update the trainable parameters in the model; the formula for calculating the cross-entropy loss function is: , in, Indicates the model loss; Represents point clouds; This represents the model's predicted probability of the true point; This represents the model's predicted probability for noise points; For real points in a point cloud, These are noise points in the point cloud.

[0015] The point cloud noise filtering system based on geometric-semantic joint embedding provided by this invention includes: The initial feature construction and embedding generation module is configured to acquire multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; The nearest neighbor aggregation module is configured to calculate the distance aggregation weight based on the geometric distance between each point and its nearest neighbor in the three-dimensional physical space, and to perform weighted aggregation on the initial embedding representation of the nearest neighbor to generate a distance embedding. The low-order neighborhood attention aggregation module is configured to calculate a low-order attention score in a three-dimensional physical space based on the geometric attributes between each point and its nearest neighbor, and to perform weighted aggregation on the distance embedding based on the low-order attention score to generate a low-order attention embedding to characterize geometric consistency. The high-order neighborhood attention aggregation module is configured to calculate a high-order attention score in a high-dimensional embedding space based on the semantic feature similarity between each point and its neighboring points, and to perform weighted aggregation on the distance embedding based on the high-order attention score to generate a high-order attention embedding to represent semantic consistency. The point existence probability prediction module is configured to fuse the low-order attention embedding and the high-order attention embedding, predict the existence probability of each point in the point cloud through the decoder, and filter out noise points based on the existence probability.

[0016] Furthermore, the low-order neighborhood attention aggregation module and the high-order neighborhood attention aggregation module share the same distance embedding output by the nearest neighbor aggregation module as input.

[0017] The beneficial effects of this invention are as follows: This invention provides a point cloud noise filtering method and system based on geometric-semantic joint embedding, belonging to the fields of deep learning and 3D data processing. The method first constructs multiple initial features, including coordinates, normal vectors, and curvature, for each point in the point cloud and maps them to initial embeddings. Next, it performs nearest-neighbor aggregation based on geometric distance in 3D physical space to obtain distance embeddings. Then, it calculates low-order attention scores based on geometric attributes in 3D physical space and performs weighted aggregation of distance embeddings to represent geometric consistency. Next, it calculates high-order attention scores based on semantic similarity in high-dimensional embedding space and performs weighted aggregation of distance embeddings to represent semantic consistency. Finally, it fuses the low-order and high-order attention embeddings to predict the existence probability of each point in the point cloud and remove noise points. This invention, through a dual-space, dual-order attention aggregation mechanism, jointly models geometric consistency and semantic context, significantly improving the accuracy and detail preservation capability of point cloud noise filtering in complex scenes. Compared with existing technologies, it has the following significant advantages: First, by explicitly constructing multiple geometric attributes, including three-dimensional coordinates, normal vectors, and curvature, as initial features for each point in the point cloud and mapping them to a high-dimensional embedding space, the organic integration of low-level geometric information and high-level semantic representation is achieved. Compared with existing technologies that rely solely on single coordinates or implicit feature extraction, this invention fully utilizes the local geometric shape information of points, significantly enriching the feature expression capability of each point. This provides a more accurate and complete geometric semantic foundation for subsequent neighborhood aggregation and attention calculation, thereby enabling clearer differentiation between valid and noise points in complex noisy environments.

[0018] Second, by introducing a neighborhood aggregation method based on nearest Euclidean distance, the geometric distance weights between each neighboring point and the center point are calculated, and the initial embedding is weighted and aggregated to generate distance embeddings. This invention effectively extracts the local structural information of the point cloud. This step preserves the spatial proximity relationship between points, providing intermediate feature representations with spatial awareness for subsequent low-order and high-order attention aggregation, and avoiding information confusion caused by direct aggregation.

[0019] Third, by calculating low-order neighborhood attention scores based on geometric attributes such as point position and normal in three-dimensional physical space, and by weighted aggregation of distance embeddings to generate low-order attention embeddings, this invention achieves adaptive enhancement of local neighborhood geometric consistency. This mechanism can dynamically allocate aggregation weights according to the actual geometric differences between points, strengthening the perception of underlying geometric structures such as surface smoothness and edge continuity. Thus, while effectively filtering out noise, it can accurately preserve the geometric integrity of sharp edges and fine structures, avoiding the loss of details caused by over-smoothing.

[0020] Fourth, by calculating high-order neighborhood attention scores based on deep semantic feature similarity of points in a high-dimensional embedding space, and weighting and aggregating distance embeddings to generate high-order attention embeddings, this invention overcomes the limitations of local Euclidean neighborhoods. It can aggregate points that are physically dispersed but semantically belong to the same object or component based on semantic relevance. This mechanism effectively captures the global semantic context, enabling the model to retain structures consistent with high-level semantic logic when filtering noise, significantly reducing the risk of misclassifying points belonging to valid objects as noise.

[0021] In summary, this invention employs four interconnected techniques: enriching initial features with multiple geometric information, extracting local structure through nearest Euclidean distance neighborhood aggregation, preserving geometric consistency through low-order attention aggregation, and capturing semantic context through high-order attention aggregation. These techniques collectively construct a dual-space, dual-order joint embedding denoising framework. Compared to existing technologies, this invention significantly improves the accuracy and robustness of point cloud denoising. While effectively filtering out various types of noise, it better preserves the original details and semantic integrity of the point cloud, providing high-quality point cloud data input for subsequent high-level tasks such as 3D reconstruction, object recognition, and semantic segmentation.

[0022] The above and other objects, advantages, and features of the present invention will be more fully set forth and demonstrated through the following detailed description of specific embodiments in conjunction with the accompanying drawings. Those skilled in the art, upon referring to the following detailed description and the accompanying drawings, will be able to better understand and realize the above advantages of the present invention. Other objects, features, and advantages of the present invention will become clearer after being described in detail in the detailed description section in conjunction with the accompanying drawings. Attached Figure Description

[0023] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following drawings are provided for illustration.

[0024] Figure 1 A method and system for point cloud noise filtering based on geometric-semantic joint embedding; Figure 2 Initial feature construction and initial embedding generation; Figure 3 For low-order neighborhood attention aggregation; Figure 4 This is for high-order neighborhood attention aggregation. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0026] Example 1

[0027] The point cloud noise filtering method based on geometric-semantic joint embedding provided in this embodiment includes the following steps: Step 1, Initial Feature Construction and Embedding Generation: Obtain multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; Step 2, nearest neighbor aggregation: In the three-dimensional physical space, based on the geometric distance between each point and its nearest neighbor, the distance aggregation weight is calculated, and the initial embedding representation of the nearest neighbor is weighted and aggregated to generate the distance embedding; Step 3, Low-order neighborhood attention aggregation: In the three-dimensional physical space, based on the geometric properties between each point and its nearest neighbor, a low-order attention score is calculated, and the distance embedding is weighted and aggregated according to the low-order attention score to generate a low-order attention embedding to represent geometric consistency. Step 4, High-order Neighborhood Attention Aggregation: In the high-dimensional embedding space, based on the semantic feature similarity between each point and its neighboring points, a high-order attention score is calculated, and the distance embedding is weighted and aggregated according to the high-order attention score to generate a high-order attention embedding to represent semantic consistency. Step 5, Point Existence Probability Prediction: The low-order attention embedding and the high-order attention embedding are fused, and the existence probability of each point in the point cloud is predicted by the decoder. Noise points are then filtered out based on the existence probability.

[0028] The multiple geometric attributes mentioned in step 1 of this embodiment include the three-dimensional coordinates, normal vector, and curvature of a point.

[0029] The normal vector in this embodiment is obtained in the following way: For any point in the point cloud, find its k nearest neighbors in the three-dimensional physical space neighborhood and calculate the covariance matrix of the neighbors. Perform eigenvalue decomposition on the covariance matrix and take the eigenvector corresponding to the smallest eigenvalue as the normal vector; The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is the curvature at that point. The calculation formula is: ; λ1 ≤ λ2 ≤ λ3; in, Represents the smallest eigenvalue; Indicates intermediate eigenvalues; Represents the largest eigenvalue; In step 2 of this embodiment, the distance aggregation weight... The calculation formula is: , , in, Indicates distance Recent One point; Indicates the current point; Represents point clouds; This means finding the largest value from a set of data. One element; Represents the relationship between two vectors distance; This indicates normalization.

[0030] In this embodiment, the calculation method of the low-order attention score in step 3 is as follows: In the three-dimensional physical space, the distance embedding is mapped to query, key and value respectively through a trainable matrix. The low-order attention weight is calculated based on the similarity between query and key, and the distance embedding is weighted and aggregated using the weight.

[0031] In this embodiment, the calculation method of the higher-order attention score in step 4 is as follows: In the high-dimensional embedding space, the distance embedding is mapped to query, key and value respectively through a trainable matrix. Based on the similarity between query and key, the neighboring points of the top k largest attention scores are selected, the higher-order attention weight is calculated, and the distance embedding is weighted and aggregated using the weight.

[0032] In step 5 of this embodiment, the fusion method is to concatenate the low-order attention embedding and the high-order attention embedding and then input them into the multilayer perceptron decoder to output the existence probability of each point.

[0033] This embodiment also includes a model training step: using supervised learning, the predicted loss is calculated using the cross-entropy loss function, and the loss is backpropagated to update the trainable parameters in the model; the formula for calculating the cross-entropy loss function is: , in, Indicates the model loss; Represents point clouds; This represents the model's predicted probability of the true point; This represents the model's predicted probability for noise points; For real points in a point cloud, These are noise points in the point cloud.

[0034] The point cloud noise filtering system with geometric-semantic joint embedding provided in this embodiment includes: The initial feature construction and embedding generation module is configured to acquire multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; The nearest neighbor aggregation module is configured to calculate the distance aggregation weight based on the geometric distance between each point and its nearest neighbor in the three-dimensional physical space, and to perform weighted aggregation on the initial embedding representation of the nearest neighbor to generate a distance embedding. The low-order neighborhood attention aggregation module is configured to calculate a low-order attention score in a three-dimensional physical space based on the geometric attributes between each point and its nearest neighbor, and to perform weighted aggregation on the distance embedding based on the low-order attention score to generate a low-order attention embedding to characterize geometric consistency. The high-order neighborhood attention aggregation module is configured to calculate a high-order attention score in a high-dimensional embedding space based on the semantic feature similarity between each point and its neighboring points, and to perform weighted aggregation on the distance embedding based on the high-order attention score to generate a high-order attention embedding to represent semantic consistency. The point existence probability prediction module is configured to fuse the low-order attention embedding and the high-order attention embedding, predict the existence probability of each point in the point cloud through the decoder, and filter out noise points based on the existence probability.

[0035] In this embodiment, the low-order neighborhood attention aggregation module and the high-order neighborhood attention aggregation module share the same distance embedding output by the nearest neighbor aggregation module as input.

[0036] Example 2

[0037] like Figure 1 As shown, Figure 1This embodiment describes a point cloud noise filtering method based on geometric-semantic joint embedding. The implementation process of this method is detailed below: Application Scenarios and Problems: With the rapid development of 3D perception technology, point cloud data is widely used in fields such as autonomous driving, robot navigation, augmented reality, digital twins, and industrial inspection. However, due to limitations in sensor accuracy, environmental interference, or poor acquisition conditions, raw point clouds often contain a large number of noisy points (such as outliers, floating points, and density anomalies). This noise can severely affect the performance of subsequent high-level tasks such as 3D reconstruction, target recognition, and semantic segmentation. In practical applications, point cloud noise filtering faces two core challenges: 1) Coupling of geometric structure and semantic information: Effective points in real-world scenes must satisfy both local geometric consistency (such as smooth surfaces and sharp edges) and overall semantic logic (such as belonging to the same object). Traditional methods rely only on low-order geometric features, making it difficult to distinguish between details and noise in complex structures; while pure data-driven deep learning methods, if lacking explicit geometric constraints, are prone to over-smoothing or structural distortion.

[0038] 2) Insufficient multi-scale contextual modeling: Existing deep models typically aggregate neighborhood information in a single space (such as the original coordinate space or fixed embedding space), which cannot simultaneously capture the low-level geometric proximity and high-level semantic similarity, resulting in insufficient ability to distinguish small structures, sparse regions or boundary points.

[0039] Therefore, there is an urgent need for a point cloud noise filtering technology that can jointly model geometric consistency and deep semantic features and has multi-level context awareness capabilities to achieve high-fidelity and robust denoising effects.

[0040] The basic idea of ​​the point cloud noise filtering system method based on geometric-semantic joint embedding provided in this embodiment is as follows: For each point, extract its 3D coordinates, normal vector, and curvature, and concatenate them to form initial features, which are then used as network input. The initial features are mapped to a high-dimensional embedding space through a multilayer perceptron (MLP) encoder to obtain the initial point embedding representation; We perform weighted aggregation of the nearest neighbor set for each point based on Euclidean distance to generate a preliminary distance aggregation embedding while preserving local geometric proximity. In the three-dimensional physical space, we calculate the attention weights of the nearest neighbor set and perform weighted aggregation to obtain a low-order attention embedding that represents the underlying geometric consistency. In high-dimensional embedding spaces, computation before The embeddings of the maximum attention scores (based on semantic similarity) are weighted and aggregated to obtain the high-order attention embeddings that encode deep semantic features; The low-order attention embedding and the high-order attention embedding are cascaded and fused, then input into the MLP decoder to output the existence probability of each point, thereby eliminating noise points with low confidence.

[0041] By employing a dual-space (physical space and embedding space) and dual-order (low-order geometry and high-order semantics) attention aggregation mechanism, we have achieved collaborative modeling of geometric structure and semantic context, which significantly improves the ability to distinguish between noise and effective details in complex scenes.

[0042] The point cloud noise filtering method provided in this embodiment has the following detailed steps: 1. Initial Feature Construction and Initial Embedding Generation For point clouds any point in Find its neighborhood in three-dimensional physical space Calculate the covariance matrix of the nearest neighbors. : , , in, express The average of the coordinates of the nearest neighbors; Indicates the current point; express A point among its nearest neighbors; For covariance matrix Eigenvalue decomposition yields eigenvalues ​​and eigenvectors: , , , in, This represents an orthogonal matrix composed of eigenvectors; This represents the eigenvector corresponding to the smallest eigenvalue; This represents the eigenvector corresponding to the intermediate eigenvalue; This represents the eigenvector corresponding to the largest eigenvalue; Represents a diagonal matrix composed of eigenvalues; like Figure 2 As shown, Figure 2 Initial features are constructed and initial embeddings are generated. By normalizing and concatenating various initial attributes of the points, the initial features of the points are obtained. Then, the initial features are input into the MLP encoder to map the initial features to a high-dimensional embedding space to obtain the initial embedding.

[0043] The eigenvector corresponding to the smallest eigenvalue is the normal vector of that point. : , The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is the curvature at that point. : ; in, Represents the smallest eigenvalue; Indicates intermediate eigenvalues; Represents the largest eigenvalue; coordinates of the point Normal vector With curvature After normalization and concatenation, the resulting data is input into an MLP encoder to obtain the initial embedding. : , in, For activation function, This represents the vector concatenation operator. For the number of layers in the MLP encoder, It is a trainable matrix.

[0044] Indicates the initial embedding; Represents point clouds; Represents the normal vector; Indicates the number of layers in the MLP encoder; 2. Nearest Neighbor Aggregation For point clouds any point in Find its neighborhood in three-dimensional physical space Calculate the nearest neighbor points and aggregate the distances between them using the nearest neighbor points. : , , in, This means finding the largest value from a set of data. One element, Represents the relationship between two vectors distance, This indicates normalization.

[0045] in, Indicates distance Recent One point; Indicates the current point; Represents point clouds; The initial embedding of weighted aggregate neighbors is used to obtain the distance embedding. : , in, Indicates the distance aggregation layer.

[0046] in, Represents distance embedding; Indicates the transpose of the initial embedding; 3. Low-order neighborhood attention aggregation For point clouds any point in Find its neighborhood in three-dimensional physical space Calculate the low-order neighbor attention aggregation score for each nearest neighbor. : , , , , , in, Indicates the current point; Indicates distance Recent Distance embedding of points; This means finding the largest value from a set of data. One element; Represents the relationship between two vectors distance; express The low-order query vector of neighboring nodes; express The lower-order key vectors of neighboring nodes; express The lower-order value vector of neighboring points; This represents a low-order query mapping matrix; Represents a low-order bond mapping matrix; This represents a low-order value mapping matrix.

[0047] Map the distance embeddings to respectively , , .

[0048] like Figure 3 As shown, Figure 3 For low-order neighborhood attention aggregation; select distance in three-dimensional physical space Recent For each neighboring node, a low-order attention score is calculated using distance embedding, followed by a weighted aggregation of the low-order value vectors of the neighbors. Low-order attention embeddings.

[0049] We weight and aggregate the low-order value vectors of neighbors to obtain the low-order attention embedding. : ; in, This represents the transpose of a lower-order value vector; Indicates the first Low-order attention scores for layer aggregation; This indicates the number of low-order attention aggregation layers.

[0050] 4. Higher-order neighborhood attention aggregation For any point in the point cloud Find its preceding position in the high-dimensional embedding space. Calculate the higher-order neighbor attention aggregation score for the nearest neighbor with the highest attention score. : , , , , in, Represents a higher-order query mapping matrix; Represents a higher-order bond mapping matrix; Represents a higher-order value mapping matrix. Map the distance embeddings to respectively , , .

[0051] like Figure 4 As shown, Figure 4 For high-order neighborhood attention aggregation, use distance embedding to calculate the high-order attention score of neighbors, and select the top neighbors with the highest attention score in the high-dimensional embedding space. 1. Neighboring points, then weighted and aggregated the higher-order value vectors of the neighbors to obtain Higher-order attention embeddings.

[0052] We obtain higher-order attention embeddings by weighted aggregation of neighbor distances. : , in, This indicates the number of higher-order attention aggregation layers.

[0053] in, This represents a higher-order attention embedding; Indicates the first Higher-order attention scores for layer aggregation; 5. Probability of point existence Embed low-level attention With higher-order attention embedding After normalization and concatenation, the data is input into an MLP decoder to obtain the existence probability of the points. : , in, This represents a low-order attention embedding; This represents a higher-order attention embedding; This represents the vector concatenation operator; This represents the number of layers in the MLP decoder. This represents a trainable matrix.

[0054] 6. Model Training Since this proposal uses supervised learning, the cross-entropy loss function is used to calculate the training loss of the model. The loss is calculated as follows: , in, Indicates the model loss; This represents the model's predicted probability of the true point; This represents the model's predicted probability for noise points; For real points in a point cloud, These are noise points in the point cloud.

[0055] Calculate the predicted loss and store it. In the process, after one batch of training, the results will be obtained. Backpropagation updates the trainable parameters in the model using gradients. To prevent gradient explosion or vanishing, it is necessary to... Multiply by the adjustable learning rate Then propagate in reverse.

[0056] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for point cloud noise filtering based on geometry-semantic joint embedding, characterized in that, Includes the following steps: Step 1, Initial Feature Construction and Embedding Generation: Obtain multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; Step 2, nearest neighbor aggregation: In the three-dimensional physical space, based on the geometric distance between each point and its nearest neighbor, the distance aggregation weight is calculated, and the initial embedding representation of the nearest neighbor is weighted and aggregated to generate the distance embedding; Step 3, Low-order neighborhood attention aggregation: In the three-dimensional physical space, based on the geometric properties between each point and its nearest neighbor, a low-order attention score is calculated, and the distance embedding is weighted and aggregated according to the low-order attention score to generate a low-order attention embedding to represent geometric consistency. Step 4, High-order Neighborhood Attention Aggregation: In the high-dimensional embedding space, based on the semantic feature similarity between each point and its neighboring points, a high-order attention score is calculated, and the distance embedding is weighted and aggregated according to the high-order attention score to generate a high-order attention embedding to represent semantic consistency. Step 5, Point Existence Probability Prediction: The low-order attention embedding and the high-order attention embedding are fused, and the existence probability of each point in the point cloud is predicted by the decoder. Noise points are then filtered out based on the existence probability.

2. The method of claim 1, wherein, The multiple geometric properties mentioned in step 1 include the three-dimensional coordinates of the point, the normal vector, and the curvature.

3. The method of claim 2, wherein, The normal vector is obtained in the following way: For any point in the point cloud, find its k nearest neighbors in the three-dimensional physical space neighborhood and calculate the covariance matrix of the neighbors. Perform eigenvalue decomposition on the covariance matrix and take the eigenvector corresponding to the smallest eigenvalue as the normal vector; The ratio of the minimum eigenvalue to the sum of all eigenvalues is the curvature of the point The formula is: ； λ1 ≤ λ2 ≤ λ3; in, Represents the smallest eigenvalue; Indicates intermediate eigenvalues; This represents the largest eigenvalue.

4. The method according to claim 1, characterized in that, The distance aggregation weight described in step 2 The calculation formula is: , , in, Indicates distance Recent One point; Indicates the current point; Represents point clouds; This means finding the largest value from a set of data. One element; Represents the relationship between two vectors distance; This indicates normalization.

5. The method according to claim 1, characterized in that, The calculation method for the low-order attention score in step 3 is as follows: In the three-dimensional physical space, the distance embedding is mapped to query, key and value respectively through a trainable matrix. The low-order attention weight is calculated based on the similarity between query and key, and the distance embedding is weighted and aggregated using the weight.

6. The method according to claim 1, characterized in that, The calculation method for the higher-order attention score in step 4 is as follows: In the high-dimensional embedding space, the distance embedding is mapped to query, key and value respectively through a trainable matrix. Based on the similarity between query and key, the neighboring points of the top k largest attention scores are selected, the higher-order attention weight is calculated, and the distance embedding is weighted and aggregated using the weight.

7. The method according to claim 1, characterized in that, The fusion method described in step 5 is to concatenate the low-order attention embedding with the high-order attention embedding and then input the result into a multilayer perceptron decoder to output the existence probability of each point.

8. The method according to claim 1, characterized in that, It also includes a model training step: employing supervised learning, using the cross-entropy loss function to calculate the prediction loss, and backpropagating the loss to update the trainable parameters in the model; the formula for calculating the cross-entropy loss function is: , in, Indicates the model loss; Represents point clouds; This represents the model's predicted probability of the true point; This represents the model's predicted probability for noise points; For real points in a point cloud, These are noise points in the point cloud.

9. A point cloud noise filtering system with geometric-semantic joint embedding, characterized in that, include: The initial feature construction and embedding generation module is configured to acquire multiple geometric attributes of each point in the point cloud, construct initial features, and map the initial features to a high-dimensional embedding space to generate an initial embedding representation for each point; The nearest neighbor aggregation module is configured to calculate the distance aggregation weight based on the geometric distance between each point and its nearest neighbor in the three-dimensional physical space, and to perform weighted aggregation on the initial embedding representation of the nearest neighbor to generate a distance embedding. The low-order neighborhood attention aggregation module is configured to calculate a low-order attention score in a three-dimensional physical space based on the geometric attributes between each point and its nearest neighbor, and to perform weighted aggregation on the distance embedding based on the low-order attention score to generate a low-order attention embedding to characterize geometric consistency. The high-order neighborhood attention aggregation module is configured to calculate a high-order attention score in a high-dimensional embedding space based on the semantic feature similarity between each point and its neighboring points, and to perform weighted aggregation on the distance embedding based on the high-order attention score to generate a high-order attention embedding to represent semantic consistency. The point existence probability prediction module is configured to fuse the low-order attention embedding and the high-order attention embedding, predict the existence probability of each point in the point cloud through the decoder, and filter out noise points based on the existence probability.

10. The system according to claim 9, characterized in that, The low-order neighborhood attention aggregation module and the high-order neighborhood attention aggregation module share the same distance embedding output by the nearest neighbor aggregation module as input.

Images