An explicit surface reconstruction system and method for noisy sparse point clouds

Abstract

The application discloses a display surface reconstruction system and method for a noise-containing sparse point cloud. The system comprises a feature extraction module, an anchor point generation module, one upsampling module or multiple cascaded upsampling modules, and a grid extraction module. The feature extraction module, the anchor point generation module, and the multiple cascaded upsampling modules are jointly trained. The grid extraction module is separately trained. The application realizes high-quality point cloud densification and surface grid extraction by designing an anchor point guided point cloud upsampling module and an octree-based grid extraction module. The application captures the topological structure information of the point cloud surface through the anchor points, provides key structure guidance for point cloud upsampling, and realizes progressive feature enhancement of "structure guidance-adaptive refinement" through the collaborative mechanism of AGA and SVA, thereby effectively improving the restoration degree of the upsampling point cloud to the target geometric details.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and more specifically, relates to an explicit surface reconstruction system and method for sparse point clouds. Background Technology

[0002] 3D surface reconstruction aims to recover the surface geometry model of an object or scene from a discrete point cloud. It is a fundamental task in the field of computer vision. How to reconstruct a high-quality surface is crucial for fields such as unmanned system navigation, entertainment, advanced manufacturing, and medical imaging.

[0003] The point clouds obtained directly by existing 3D perception technologies usually have characteristics such as noise, sparsity and non-uniform distribution. In order to meet the application requirements of display surface reconstruction, it is usually necessary to perform high-quality densification processing on the directly obtained noisy and sparse point clouds.

[0004] In recent years, many learning-based studies have focused on point cloud upsampling for point cloud densification, such as GeoUDF and Grad-PU methods. However, due to the severe lack of local contextual information in noisy sparse point clouds, existing methods often struggle to effectively maintain correct surface topological details during upsampling. This limitation leads to the presence of holes or outliers in the upsampled point cloud, significantly affecting the quality and integrity of subsequent surface reconstruction: GeoUDF-sampled point clouds often contain outliers, while Grad-PU-upsampled point clouds often have holes. These methods struggle to maintain correct topological details during upsampling, resulting in poor surface reconstruction performance. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides an explicit surface reconstruction system and method for noisy sparse point clouds. The aim is to record the surface topology, and during the upsampling process, the model continuously utilizes anchor point information to refine the topology, ultimately generating a high-quality, dense point cloud with well-preserved topology and uniform distribution. Mesh extraction is then performed, and the reconstructed surface retains correct topological details. This solves the technical problem of existing technologies failing to maintain correct topological details during upsampling, resulting in poor surface reconstruction performance.

[0006] To achieve the above objectives, according to one aspect of the present invention, an explicit surface reconstruction system for noisy sparse point clouds is provided, comprising: a feature extraction module, an anchor point generation module, an upsampling module or multiple cascaded upsampling modules, and a mesh extraction module. The feature extraction module is an encoder, comprising multiple stacked vector self-attention layers, which are connected via residuals; its input is a noisy sparse point cloud of the reconstructed target. The output is the feature. ,in As a feature dimension, it is submitted to the anchor point generation module; The anchor point generation module is a decoder, including a nonlinear transformation submodule composed of a multilayer perceptron and a convolutional layer, and a decoding submodule composed of multiple cascaded multilayer perceptrons, which outputs a uniform anchor point cloud covering the target surface to be reconstructed to the upsampling module. For each of the multiple cascaded upsampling modules, its input is the point cloud output of the previous upsampling module. and the point cloud features of the previous level The output is the point cloud of this level. and the features of this level of point cloud The input to the upsampling module in the first episode is the anchor cloud. and anchor point features ,Right now , Multiple cascaded upsampling modules will densify the point cloud at multiple levels. Output to the network extraction module; The mesh extraction module is used to extract meshes from point clouds to obtain reconstructed surfaces.

[0007] Preferably, the explicit surface reconstruction system for noisy sparse point clouds serves as a decoder for the anchor point generation module, characterized by... The nonlinear transformation submodule is input respectively, and its output is related to the characteristics. After channel splicing, the data is input to the decoding submodule; the output of the final-level multilayer perceptron of the decoding submodule is then compared with the noisy sparse point cloud. Merging into anchor cloud The anchor point cloud Number of midpoints and noisy sparse point clouds The number of points is the same, and the points on the anchor point cloud uniformly cover the target surface to be reconstructed; the output of the multilayer perceptron at the second-to-last stage of the decoding submodule is used as the anchor point features. ; Anchor point cloud and anchor point features Submit to the upsampling module.

[0008] Preferably, in the explicit surface reconstruction system for noisy sparse point clouds, the first... The level upsampling module includes an anchor-guided attention submodule, an autovector attention submodule, and a dense submodule; Point cloud features of the previous level And the point cloud of the previous level Input anchor points guide the attention submodule to extract attention features. The output is fed to the self-vector attention submodule; Point cloud of the previous level And the attention features output by the anchor-guided attention submodule Input the self-vector attention submodule to extract upsampled features. Output to dense submodules; Point cloud of the previous level and the upsampled features output by the self-vector attention submodule Input the dense submodule to obtain the point cloud at this level of density. and local dense point cloud Point cloud features .

[0009] Preferably, the explicit surface reconstruction system for noisy sparse point clouds, for each level of point cloud... Its attention characteristics The topological features used to characterize anchor-guided structures are denoted as... The calculation method is as follows:

[0010] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, The output of the multi-head sensor MLP; The calculation method is as follows:

[0011] in, It is a nonlinear projection function implemented using MLP. , This refers to each upsampling point In anchor point cloud The search was performed using the KNN algorithm. Neighbor, The noisy sparse point cloud features extracted by the encoder. For this level of point cloud points and anchor point Relative position information, relative position coordinates ; Nonlinear projection function , for from to higher dimensional space The mapping is as follows:

[0012] in As a hyperparameter, it can be set to... 2L is the dimension of the spatial mapping.

[0013]

[0014] in, The output of the multi-head sensor MLP; This is for concatenation operations.

[0015] Preferably, the explicit surface reconstruction system for noisy sparse point clouds, for each level of point cloud... Its upsampling features Used to characterize structural features The features after local adaptive thinning are denoted as The calculation method is as follows:

[0016] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, This is the output of the multi-head sensor MLP.

[0017] Preferably, in the explicit surface reconstruction system for noisy sparse point clouds, the dense submodule includes a super-resolution block composed of multilayer perceptrons and deconvolutional layers, and a decoding block composed of multiple cascaded multilayer perceptrons; the sampling features of the previous level. Input the super-resolution block and the sampling features of the previous level respectively. conduct After being copied multiple times, it is concatenated with the output channel of the super-resolution submodule and input into the decoding block; the stride of the deconvolution layer is... The output of the final multilayer perceptron of the decoding block is compared with the dense point cloud upsampled from the previous stage. The points are merged into a denser point cloud upsampled at this level. The output of the multilayer perceptron at the last stage of the decoding submodule is used as the dense point cloud feature at that stage. .

[0018] Preferably, in the explicit surface reconstruction system for noisy sparse point clouds, the network extraction module adopts an octree-based U-Net architecture.

[0019] Preferably, in the explicit surface reconstruction system for noisy sparse point clouds, the network extraction module has the following specific structure: The output of the octree-based U-Net is fed into two output heads: a Tanh output head for predicting vertex position offsets and a Sigmoid output head for predicting edge classification labels. The input of the octree-based U-Net is an expanded, multi-level dense point cloud. The multi-level dense point cloud Perform morphological dilation so that the leaf nodes of the octree constructed based on it can completely cover the object surface.

[0020] According to another aspect of the invention, a training method for the explicit surface reconstruction system of the noisy sparse point cloud is provided, which employs ground truth supervised training, wherein: The feature extraction module, anchor point generation module, and multiple cascaded upsampling modules are trained together. The network extraction module is trained separately.

[0021] Preferably, the training method for the explicit surface reconstruction system of noisy sparse point clouds uses the following total loss function for the joint training of the feature extraction module, anchor point generation module, and multiple cascaded upsampling modules:

[0022] in, , For loss weights; The constraint chamfer distance loss of the anchor point generation module The calculation method is as follows:

[0023] in, For radix operations; A template point cloud with an ideal distribution, obtained by uniformly sampling the same number of training samples; The upsampling module uses the current level's output point cloud. and a corresponding number of GT surface uniform sampling point clouds chamfer distance The calculation method is as follows:

[0024] in, Calculate the chamfer distance.

[0025] .

[0026] Overall, the above-described technical solutions conceived by this invention can achieve the following beneficial effects compared with the prior art.

[0027] The present invention provides a display surface reconstruction system and method for noisy sparse point clouds. By designing an anchor-guided point cloud upsampling module and an octree-based mesh extraction module, it achieves high-quality point cloud densification and surface mesh extraction. This invention captures the topological structure information of the point cloud surface through anchor points, providing crucial structural guidance for point cloud upsampling. Furthermore, through the collaborative mechanism of AGA and SVA, it achieves progressive feature enhancement of "structure guidance-adaptive refinement," effectively improving the accuracy of the upsampled point cloud in restoring the geometric details of the target.

[0028] Furthermore, this invention utilizes an octree-based mesh extraction module, which eliminates the traditional implicit reconstruction process and supports the direct and accurate extraction of high-quality meshes from dense point clouds. Experiments on several widely used public benchmarks demonstrate that AUPOME exhibits significant advantages in watertight, non-watertight, or real scanned surface reconstruction, surpassing existing methods in geometric fidelity and topological accuracy. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the sparse point cloud display surface reconstruction system provided in an embodiment of the present invention; Figure 2 This is a structural diagram of a submodule of the sparse point cloud display surface reconstruction system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the mesh extraction module structure of the sparse point cloud display surface reconstruction system provided in an embodiment of the present invention; Figure 4 This is a demonstration of the test results for watertight surface reconstruction according to an embodiment of the present invention; Figure 5 This is a demonstration of the test results for clothing reconstruction on a non-watertight curved surface according to an embodiment of the present invention; Figure 6 This is a demonstration of the test results for automobile body reconstruction in a non-watertight curved surface according to an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0031] Due to the low resolution and uneven distribution of sparse point clouds, the high-frequency geometric information and contextual information of the surface are severely lacking. In order to achieve more robust surface reconstruction, we adopted a reconstruction process of cascaded upsampling and mesh extraction.

[0032] The sparse point cloud display surface reconstruction system provided by the present invention includes: a feature extraction module, an anchor point generation module, an upsampling module or multiple cascaded upsampling modules, and a mesh extraction module; The feature extraction module is an encoder, comprising multiple stacked vector self-attention layers (SVA modules), which are connected via residuals; its input is the noisy sparse point cloud of the reconstructed target. The output is the feature. ,in As a feature dimension, it is submitted to the anchor point generation module; The anchor point generation module is a decoder, comprising a nonlinear transformation submodule consisting of a multilayer perceptron and convolutional layers, and a decoding submodule consisting of multiple cascaded multilayer perceptrons. It outputs a uniform anchor point cloud covering the target surface to be reconstructed to the upsampling module. The nonlinear transformation submodule is input respectively, and its output is related to the characteristics. After channel splicing, the data is input to the decoding submodule; the output of the final-level multilayer perceptron of the decoding submodule is then compared with the noisy sparse point cloud. Merging into anchor cloud The anchor point cloud Number of midpoints and noisy sparse point clouds The number of points is the same, and the points on the anchor point cloud uniformly cover the target surface to be reconstructed; the output of the multilayer perceptron at the second-to-last stage of the decoding submodule is used as the anchor point features. ; Anchor point cloud and anchor point features Submit to the upsampling module.

[0033] The anchor point generation module trains and converges with the goal of minimizing the loss function, using constrained chamfer distance loss as the loss function; the constrained chamfer distance loss The calculation method is as follows:

[0034] in, For radix operations; This refers to a template point cloud with an ideal distribution, obtained by uniformly sampling the same number of samples from the training samples.

[0035] For each of the multiple cascaded upsampling modules, its input is the point cloud output of the previous upsampling module. and the point cloud features of the previous level The output is the point cloud of this level. and the features of this level of point cloud The input to the upsampling module in the first episode is the anchor cloud. and anchor point features ,Right now , Multiple cascaded upsampling modules will densify the point cloud at multiple levels. Output to the network extraction module; No. The upsampling module includes the Anchor Guided Attention (AGA) submodule, the Self Vector Attention (SVA) submodule, and the Dense Submodule (UP). Point cloud features of the previous level And the point cloud of the previous level Input anchor points guide the attention submodule to extract attention features. The output is fed to the self-vector attention submodule; for each level of point cloud... Its attention characteristics The topological features used to characterize anchor-guided structures are denoted as... The calculation method is as follows:

[0036] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, The output of the multi-head sensor MLP; The calculation method is as follows:

[0037] in, It is a nonlinear projection function implemented using MLP. , This refers to each upsampling point In anchor point cloud The search was performed using the KNN algorithm. Neighbor, The noisy sparse point cloud features extracted by the encoder. For this level of point cloud points and anchor point Relative position information, relative position coordinates ; Nonlinear projection function , for from to higher dimensional space The mapping is as follows:

[0038] in As a hyperparameter, it can be set to... 2L is the dimension of the spatial mapping.

[0039]

[0040] in, The output of the multi-head sensor MLP; This is for concatenation operations.

[0041] Point cloud of the previous level And the attention features output by the anchor-guided attention submodule Input the self-vector attention submodule to extract upsampled features. Output to the dense submodule; for each level of point cloud Its upsampling features Used to characterize structural features The features after local adaptive thinning are denoted as The calculation method is as follows:

[0042] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, This is the output of the multi-head sensor MLP.

[0043] Point cloud of the previous level and the upsampled features output by the self-vector attention submodule Input the dense submodule to obtain the point cloud at this level of density. and local dense point cloud Point cloud features The dense submodule includes a super-resolution block composed of multilayer perceptrons and deconvolutional layers, and a decoding block composed of multiple cascaded multilayer perceptrons; the sampling features of the previous level. Input the super-resolution block and the sampling features of the previous level respectively. conduct After being copied multiple times, it is concatenated with the output channel of the super-resolution submodule and input into the decoding block; the stride of the deconvolution layer is... The output of the final multilayer perceptron of the decoding block is compared with the dense point cloud upsampled from the previous stage. The points are merged into a denser point cloud upsampled at this level. The output of the multilayer perceptron at the last stage of the decoding submodule is used as the dense point cloud feature at that stage. .

[0044] The upsampling module uses the current level's output point cloud. and a corresponding number of GT surface uniform sampling point clouds chamfer distance The calculation method is as follows:

[0045] in, Calculate the chamfer distance.

[0046]

[0047] The total loss function used in the joint training of the feature extraction module, anchor point generation module, and multiple cascaded upsampling modules is:

[0048] in, , This is the loss weight.

[0049] The mesh extraction module is used to extract meshes from the point cloud to obtain the reconstructed surface; preferably, the network extraction module adopts an octree-based U-Net architecture, with the following specific structure: The output of the octree-based U-Net is fed into two output heads: a Tanh output head for predicting vertex position offsets and a Sigmoid output head for predicting edge classification labels. The input of the octree-based U-Net is an expanded, multi-level dense point cloud. The multi-level dense point cloud Perform morphological dilation so that the leaf nodes of the octree constructed based on it can completely cover the object surface.

[0050] The network extraction module is trained separately, and it uses network extraction error. This includes the mean square error of the offset displacement and the binary cross-entropy of the edge classification labels.

[0051] The following is an example: The sparse point cloud display surface reconstruction system provided in this embodiment, such as Figure 1 As shown, it includes: a feature extraction module, an anchor point generation module, two cascaded upsampling modules, and a grid extraction module.

[0052] We first extract features from noisy sparse point clouds using a feature extraction module. Extracting features And use the anchor point generation module to generate an anchor point cloud. and anchor point features Subsequently, a cascaded upsampling module was used to densify the point cloud twice. Finally, an octree mesh extraction module (OME) was used to directly extract the surface mesh from the dense point cloud. The arrows in the figure indicate the directions of feature transfer and location information transfer.

[0053] The feature extraction module is an encoder, comprising four stacked SelfVectorAttention (SVA) layers. These four self-attention modules are connected via residuals to extract features from the noisy sparse point cloud. Features are extracted from the target object; the input is the noisy sparse point cloud of the target object. The output is the feature. ,in These features are submitted to the anchor point generation module as feature dimensions; these features can effectively capture local geometric patterns and their regional context information.

[0054] The anchor point generation module is a decoder (TAG, Topo-aware AnchorGenerator), comprising a nonlinear transformation submodule composed of multilayer perceptrons and convolutional layers, and a decoding submodule composed of multiple cascaded multilayer perceptrons; features The nonlinear transformation submodule is input respectively, and its output is related to the characteristics. After channel splicing, the data is input to the decoding submodule; the output of the final-level multilayer perceptron of the decoding submodule is then compared with the noisy sparse point cloud. Merging into anchor cloud The anchor point cloud Number of midpoints and noisy sparse point clouds The number of points is the same, and the points on the anchor point cloud uniformly cover the target surface to be reconstructed; the output of the multilayer perceptron at the second-to-last stage of the decoding submodule is used as the anchor point features. ; Anchor point cloud and anchor point features Submit to the upsampling module.

[0055] The anchor point generation module trains and converges with the goal of minimizing the loss function, using constrained chamfer distance loss as the loss function; the constrained chamfer distance loss The calculation method is as follows:

[0056] in, For radix operations; This refers to a template point cloud with an ideal distribution, obtained by uniformly sampling the same number of samples from the training samples.

[0057] A set of points with the same number as the original point cloud is generated using TAGs. This set of points completely and uniformly covers the surface, forming the topological basis of the surface and providing crucial prior geometric information for subsequent upsampling. We call this set of points anchor points and use them as the initial positions for the subsequent upsampling process. Subsequently, we employ a cascaded coarse-to-fine upsampling framework to progressively upsample a high-fidelity surface point cloud.

[0058] anchor point cloud Quantity settings and noisy sparse point clouds The quantities are consistent, and the anchor point cloud is obtained by learning point-by-point displacement. In this embodiment, the structure of the TAG is as follows: Figure 2 As shown in (a), the input features are first processed by a single-layer biased multilayer perceptron (MLP) and a convolutional module (Conv). A nonlinear transformation is performed, followed by channel concatenation of the transformed features with the original input features. This concatenation is then processed through a two-layer MLP to obtain the anchor features. Finally, the anchor cloud is obtained by predicting the displacement of the input point cloud using a 3-layer regression MLP. To establish effective supervision, we uniformly sample the same number of point clouds on the GT mesh. As an ideal distribution template, the anchor cloud is then guided by constrained Chamfer Distance (CD) loss. The generation of .

[0059] For each of the multiple cascaded upsampling modules, its input is the point cloud output of the previous upsampling module. and the point cloud features of the previous level The output is the point cloud of this level. and the features of this level of point cloud The input to the upsampling module in the first episode is the anchor cloud. and anchor point features ,Right now , Multiple cascaded upsampling modules will densify the point cloud at multiple levels. The output is sent to the network extraction module.

[0060] During the upsampling process at each level, we first enhance the current point cloud using an attention mechanism. Upsampling features: the first Hierarchical input features First, the anchor-guided attention submodule (AGA) is used to fuse anchor information to enhance topology preservation. Then, the self-vector attention submodule (SVA) is used to achieve adaptive refinement at the hierarchical level. The resulting enhanced features are then upsampled through the dense submodule (UP).

[0061] No. The upsampling module includes the Anchor Guided Attention (AGA) submodule, the Self Vector Attention (SVA) submodule, and the Dense Submodule (UP). Point cloud features of the previous level And the point cloud of the previous level Input anchor points guide the attention submodule to extract attention features. The output is fed to the self-vector attention submodule; for each level of point cloud... Its attention characteristics The topological features used to characterize anchor-guided structures are denoted as... The calculation method is as follows:

[0062] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, The output of the multi-head sensor MLP; The calculation method is as follows:

[0063] in, It is a nonlinear projection function implemented using MLP. , This refers to each upsampling point In anchor point cloud The search was performed using the KNN algorithm. Neighbor, The noisy sparse point cloud features extracted by the encoder. For this level of point cloud points and anchor point Relative position information, relative position coordinates ; This level of dense point cloud features.

[0064] Nonlinear projection function , for from to higher dimensional space The mapping is as follows:

[0065] in As a hyperparameter, it can be set to... 2L is the dimension of the spatial mapping.

[0066]

[0067] in, The output of the multi-head sensor MLP; This is for concatenation operations.

[0068] This embodiment uses AGA to enhance the topology preservation capability of upsampled features. The structure of the AGA in this embodiment is as follows: Figure 2As shown in (c). Given the previous level. Coordinates of the corresponding point cloud and point cloud features To achieve topology preservation during the upsampling process, the anchor cloud is fully utilized. and encoder output features The information is aggregated using a local attention mechanism.

[0069] Point cloud of the previous level And the attention features output by the anchor-guided attention submodule Input the self-vector attention submodule to extract upsampled features. Output to the dense submodule; for each level of point cloud Its upsampling features Used to characterize structural features The features after local adaptive thinning are denoted as The calculation method is as follows:

[0070] in, To normalize the weights, , The neighborhood attention score, For Hadamard products, This is the output of the multi-head sensor MLP.

[0071] Upsampled features are obtained by adaptively refining features using SVA. SVA and AGA structures are very similar. The SVA structure is as follows: Figure 2 As shown in (d), the output characteristics of the encoder in AGA. Replaced with attention features AGA is then converted to SVA. In this embodiment of the KNN algorithm... .

[0072] Point cloud of the previous level and the upsampled features output by the self-vector attention submodule Input the dense submodule to obtain the point cloud at this level of density. and local dense point cloud Point cloud features The dense submodule includes a super-resolution block composed of multilayer perceptrons and deconvolutional layers, and a decoding block composed of multiple cascaded multilayer perceptrons; the sampling features of the previous level. Input the super-resolution block and the sampling features of the previous level respectively. conduct After being copied multiple times, it is concatenated with the output channel of the super-resolution submodule and input into the decoding block; the stride of the deconvolution layer is... The output of the final multilayer perceptron of the decoding block is compared with the dense point cloud upsampled from the previous stage. The points are merged into a denser point cloud upsampled at this level. The output of the multilayer perceptron at the last stage of the decoding submodule is used as the dense point cloud feature at that stage. .

[0073] Upsampling of point cloud points is achieved through a dense submodule, with the structure as follows: Figure 2 As shown in (b), this can be obtained by replacing Conv in TAG with a deconvolution module (Deconv), which first receives upsampled features through a dense submodule. And point cloud As input, the features are then processed using MLP and Deconv. resolution expansion Multiply, to obtain sub-features To establish multi-scale feature associations, we will use the original features conduct After double replication and The data is stitched together along the channel dimension, and then the displacement is predicted using MLP followed by a regression MLP. Then you can get the coordinates. , For copying operations, the coordinates and dense point cloud features As input for the next level.

[0074] The upsampling module uses the current level's output point cloud. and a corresponding number of GT surface uniform sampling point clouds chamfer distance The calculation method is as follows:

[0075] in, Calculate the chamfer distance.

[0076]

[0077] The total loss function used in the joint training of the feature extraction module, anchor point generation module, and multiple cascaded upsampling modules is:

[0078] in, , This is the loss weight.

[0079] The mesh extraction module is used to extract meshes from the point cloud to obtain the reconstructed surface; preferably, the network extraction module adopts an octree-based U-Net architecture with the following specific structure: The output of the octree-based U-Net is fed into two output heads: a Tanh output head for predicting vertex position offsets and a Sigmoid output head for predicting edge classification labels. The input of the octree-based U-Net is an expanded, multi-level dense point cloud. The multi-level dense point cloud Perform morphological dilation so that the leaf nodes of the octree constructed based on it can completely cover the object surface.

[0080] The structure of the OME in this embodiment is as follows: Figure 3 As shown, the octree-based U-Net (OUNet) architecture is used as the backbone for voxel feature encoding, with two output heads added to predict vertex position offsets and edge classification labels. Details of the same color modules are shown on the right. The U-Net network used in the model corresponds to each level of the octree; edge classification uses a Sigmoid output; vertex position regression uses a scaled Tanh (×0.5) to constrain vertex position predictions. In the mesh extraction and decoding stage, we decode and predict the voxel features of the leaf nodes of the octree. If the binary classification label of an edge of a voxel is predicted as 1, we connect the vertices of its four adjacent voxels to form the surface reconstruction mesh. A classification label of 1 for a leaf node voxel edge means that this edge passes through the surface to be reconstructed, so we connect the vertices of its adjacent voxels to form the mesh.

[0081] The network extraction module is trained separately, and it uses network extraction error. This includes the mean square error of the offset displacement and the binary cross-entropy of the edge classification labels.

[0082] After obtaining high-quality dense point clouds, an octree-based mesh extraction module was constructed to support direct extraction of surface meshes from dense point clouds. Addressing the numerical instability caused by the reliance on the quadratic error function (QEF) in traditional Dual Contouring (DC) methods, and the incomplete surface coverage problem caused by traditional octree modeling which only samples the area occupied by the point cloud, this invention proposes an end-to-end mesh extraction framework based on dilated octrees and voxel edge classification (OME). This invention normalizes the point cloud space to... For cubes, a morphological dilation operation is introduced to construct a fully covered octree. Then, using the octree-based U-Net architecture, the intersection state of voxel edges and vertex offsets are directly predicted, avoiding the complex QEF solution process and significantly improving reconstruction efficiency and topological integrity.

[0083] Input dense point cloud as Construct an octree with a maximum depth of 8. .set up For the first There are 1 leaf node voxels, and their center coordinates are 1. , For the first Feature map of the layer .

[0084] The encoder employs an octree-based U-Net structure, extracting multi-scale geometric features through octree sparse convolution. For the... Input features of layer +1 , Hierarchical feature extraction is performed using the octree sparse convolution module (ConV_OC):

[0085]

[0086] : Tensors corresponding to the convolution weights of the three voxels. * indicates the convolution operation. This represents the octree grouping normalization function, with ReLU as the activation function.

[0087] No. The features of the octagonal leaf nodes of the layer are encoded as follows after passing through the network: By characteristics After octree convolution, it is connected to the skip connection feature. It is pieced together. This is a splicing operation at the channel dimension.

[0088]

[0089] The edge classification head decoding module is responsible for predicting the intersection probability of a specific edge of a voxel with the surface. The probability mapping function is as follows: ,here

[0090] Symbol explanation: : Voxel convolution weights are used for feature transformation. Sigmoid activation function Map the output to Interval. S: Predicted edge classification probability vector, with dimension 1. (The three axes of the corresponding voxel).

[0091] The vertex regression head decoding module is responsible for predicting the normalized offset of vertices relative to the voxel center. ,here ; ; : The voxel convolution weights are set, and the last layer has 3 output channels. Hyperbolic tangent activation function, output range is . Scaling factor, used to constrain the output to This ensures that the vertex falls inside the voxel. : The predicted vertex position offset. The final vertex coordinates are obtained by adding the voxel center to the offset:

[0092] After decoding and prediction, if the prediction probability of a certain edge of a voxel is greater than 0.5, the binary classification label is predicted as 1. We then connect the vertices corresponding to its four adjacent voxels to form a surface reconstruction mesh.

[0093] The mesh extractor is trained by supervising the prediction of offset displacements using mean squared error (MSE) and by supervising the prediction of edge cross labels using binary cross entropy.

[0094]

[0095] in, It is a binary mask that takes a value of 1 only at voxel blocks that pass through the real surface, and 0 at the rest; It is an indicator function; This indicates element-wise multiplication; and These represent the coordinates of the predicted vertex and the actual vertex, respectively. Represents the set of all edges to be classified; and These represent the predicted edge classification probabilities and the true labels, respectively. The total loss function is a weighted sum of the location regression loss and the classification loss:

[0096] This embodiment's test experiment was conducted using a single NVIDIA GeForce RTX 4090 GPU. The overall training process was divided into two stages: the first stage focused on point cloud upsampling. In this stage, the model performed upsampling twice, with upsampling rates of 2 and 8, respectively, achieving a 16-fold increase in the input point cloud size. This stage used the Adam optimizer, with an initial learning rate set to 1e-3, and applied a cosine annealing strategy to gradually reduce the learning rate to 5e-6. Training consisted of 60 epochs, and the weighted coefficients of the loss function were... The values ​​were set to 10, 10, and 200 respectively. The second stage focused on training the grid extraction module. In this stage, the full depth of the octree was set to 2, the maximum depth to 8, and the dimensions of the features in each layer of U-Net were set to [256, 256, 128, 128, 64, 64, 32]. When constructing the GT octree, we additionally dilated the voxels occupied by the GT surface by a radius of 1. The model in this stage used the AdamW optimizer, with an initial learning rate of 5e-4 and a multinomial decay strategy, with an exponent of 0.9 and a weight decay of 0.05, for a total of 50 training epochs.

[0097] Watertight Surface Reconstruction: We used the ShapeNetCore dataset processed by DISN, which removes non-manifold structures and internal meshes from the original shapes. The dataset was split into training, validation, and test sets according to the 3D-R2N2 standard. For each shape, we randomly sampled 3000 points from its surface as the input raw point cloud. Experiments were conducted in two settings: raw point cloud and noisy point cloud (Gaussian noise with a standard deviation of 0.005 was added to the raw point cloud).

[0098] To ensure fair comparison, the resolution of the MC used for all comparison methods was set to 128. Since the maximum depth of the octree constructed by our method is 8, and the theoretical resolution of the extracted mesh is 256, for fair comparison, we pre-scaled the input point cloud to half its original size (so that the voxel mesh resolution corresponding to the leaf nodes of the final octree at the maximum depth is 128), and then scaled it back to its original size after mesh extraction for performance evaluation. To quantitatively measure reconstruction quality, we followed GeoUDF.

[25] Evaluation criteria: 100,000 points are sampled from both the reconstructed surface and the GT surface, and the chamfer distance (CDL1) between these two point clouds is calculated. The distance thresholds for the F-Score were set to 0.5% and 1.0% of the longest side of the bounding box of the GT surface mesh, respectively. We compared Ours with existing methods including GridFormer, Ditto, NVF, POCO, ONet, ConvONet, DOG, NDF, GIFS, and GeoUDF.

[0099] As shown in Table 1, our method significantly outperforms the comparison methods in both clean and noisy scenes in terms of reconstruction accuracy. In clean, noise-free scenes, Ours' mean and median CD values ​​are 0.219 and 0.214, respectively, which are 6.4% and 4.7% lower than the second-place method's mean of 0.234 and median of 0.224. Meanwhile, the F-Score is also significantly lower at strict thresholds (…). At a precision of 0.961, it is significantly higher than Ditto's 0.946, an improvement of 1.5 percentage points; at a more lenient threshold ( Furthermore, it approaches the theoretical limit of 1 with an accuracy of 0.998. In noisy scenarios, the mean and median CD of this embodiment are 0.262 and 0.254, respectively, which is lower than Ditto's 0.277 and 0.262; while Our metric of 0.994 surpasses Ditto's 0.987 and GeoUDF's 0.987. These results demonstrate that our method provides stable and reliable reconstruction results in both clean and noisy environments.

[0100]

[0101] like Figure 4 As shown, POCO and GridFormer can generate relatively smooth surface meshes, but they cannot effectively predict thin-walled structural regions (such as aircraft wings, chair armrests, lamp wires, and sailboat masts), resulting in the loss of key structures. Although Ditto has made some improvements in this regard, it still exhibits the same limitations to some extent. GIFS can construct relatively complete surfaces, but the mesh surface exhibits obvious roughness (especially evident in the texture of chair backs and aircraft fuselages). GeoUDF can reconstruct complete surface topology, but its output mesh shows fractured and discretized characteristics (manifested as broken aircraft wings and holes in sailboat sails). Ours' reconstructed surface mesh is significantly better than the comparative methods in terms of structural integrity and surface smoothness, and its fine geometric structure closely approximates the real GT surface.

[0102] Non-watertight surface reconstruction To evaluate AUPOME's performance on open surface reconstruction, the MGN dataset was used. One hundred shapes were randomly selected from the MGN dataset for testing, with 3000 points randomly sampled from each shape as the input point cloud. Furthermore, to test AUPOME's reconstruction capabilities on shapes with complex geometries (such as multi-layered internal structures), the "Car" category, which contains the most internal structures in the original, unprocessed ShapeNet dataset, was selected. Similarly, 100 shapes were randomly selected from this category for testing, with 6000 points randomly sampled from each shape as the input point cloud.

[0103] To verify the model's generalization ability, we performed inference directly using a model pre-trained on the ShapeNet (watertight) dataset. We compared our method with two representative methods: UDF-based methods (GIFS, GeoUDF, and NVF) and the BOF-based method Ditto.

[0104] The quantitative results shown in Table 2 demonstrate that, on the MGN dataset, our method outperforms the suboptimal method. The performance improved from 0.973 to 0.984 (a 1.1% improvement); the average CD decreased from 0.199 to 0.189 (a 5% decrease). On the ShapeNet “Car” dataset, our method outperformed the suboptimal approach. This represents a 3% improvement, with the average CD decreasing from 0.305 to 0.287 (a 5.9% reduction). Our reconstruction results significantly outperformed the comparison methods across all evaluation metrics, demonstrating that our model maintains strong performance even when generalized to different datasets.

[0105]

[0106] In respectively Figure 5 , Figure 6 The visualization showcases the reconstruction results of the car body and its internal structure using MGN and ShapeNet "Car". For example... Figure 5 As shown, for garment reconstruction, GIFS and NVF produce coarse surfaces, while GeoUDF and Ditto produce smoother results. However, GeoUDF suffers from mesh fragmentation issues in fine structures such as collars, hems, and cuffs, while Ditto incorrectly seals open boundaries that should remain open. In contrast, this embodiment accurately recovers fine wrinkles and sharp boundary contours. Figure 6 The car reconstruction results revealed significant artifacts in existing methods. GIFS produces uneven, bumpy surfaces on the vehicle body. GeoUDF exhibits noticeable deformation in areas such as the front bumper and rearview mirrors. NVF is affected by mesh surface damage, while Ditto produces false artifacts. In contrast, this embodiment reconstructs a smooth, continuous vehicle body with a clear internal structure, achieving significantly higher geometric fidelity.

[0107] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An explicit surface reconstruction system for noisy sparse point clouds, characterized in that, include: The module includes a feature extraction module, an anchor point generation module, an upsampling module or multiple cascaded upsampling modules, and a grid extraction module. The feature extraction module is an encoder, comprising multiple stacked vector self-attention layers, which are connected via residuals; its input is a noisy sparse point cloud of the reconstructed target. The output is the feature. ,in As a feature dimension, it is submitted to the anchor point generation module; The anchor point generation module is a decoder, including a nonlinear transformation submodule composed of a multilayer perceptron and a convolutional layer, and a decoding submodule composed of multiple cascaded multilayer perceptrons, which outputs a uniform anchor point cloud covering the target surface to be reconstructed to the upsampling module. For each of the multiple cascaded upsampling modules, its input is the point cloud output of the previous upsampling module. and the point cloud features of the previous level The output is the point cloud of this level. and the features of this level of point cloud The input to the upsampling module in the first episode is the anchor cloud. and anchor point features ,Right now , Multiple cascaded upsampling modules will densify the point cloud at multiple levels. Output to the network extraction module; The mesh extraction module is used to extract meshes from point clouds to obtain reconstructed surfaces.

2. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 1, characterized in that, For the anchor point generation module to be a decoder, the feature is... The nonlinear transformation submodule is input respectively, and its output is related to the characteristics. After channel splicing, the data is input to the decoding submodule; the output of the final-level multilayer perceptron of the decoding submodule is then compared with the noisy sparse point cloud. Merging into anchor cloud The anchor point cloud Number of midpoints and noisy sparse point clouds The number of points is the same, and the points on the anchor point cloud uniformly cover the target surface to be reconstructed; the output of the multilayer perceptron at the second-to-last stage of the decoding submodule is used as the anchor point features. ; Anchor point cloud and anchor point features Submit to the upsampling module.

3. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 1, characterized in that, No. The level upsampling module includes an anchor-guided attention submodule, an autovector attention submodule, and a dense submodule; Point cloud features of the previous level And the point cloud of the previous level Input anchor points guide the attention submodule to extract attention features. The output is fed to the self-vector attention submodule; Point cloud of the previous level And the attention features output by the anchor-guided attention submodule Input the self-vector attention submodule to extract upsampled features. Output to dense submodules; Point cloud of the previous level and the upsampled features output by the self-vector attention submodule Input the dense submodule to obtain the point cloud at this level of density. and local dense point cloud Point cloud features .

4. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 3, characterized in that, For each level of point cloud Its attention characteristics The topological features used to characterize anchor-guided structures are denoted as... The calculation method is as follows: ； in, To normalize the weights, , The neighborhood attention score, For Hadamard products, The output of the multi-head sensor MLP; The calculation method is as follows: ； in, It is a nonlinear projection function implemented using MLP. , This refers to each upsampling point In anchor point cloud The search was performed using the KNN algorithm. Neighbor, The noisy sparse point cloud features extracted by the encoder. For this level of point cloud points and anchor point Relative position information, relative position coordinates ; Nonlinear projection function , for from to higher dimensional space The mapping is as follows: ； in As a hyperparameter, it can be set to... 2L is the dimension of the spatial mapping; ； in, The output of the multi-head sensor MLP; This is for concatenation operations.

5. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 3, characterized in that, For each level of point cloud Its upsampling features Used to characterize structural features The features after local adaptive thinning are denoted as The calculation method is as follows: ； in, To normalize the weights, , The neighborhood attention score, For Hadamard products, This is the output of the multi-head sensor MLP.

6. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 3, characterized in that, The dense submodule includes a super-resolution block composed of multilayer perceptrons and deconvolutional layers, and a decoding block composed of multiple cascaded multilayer perceptrons; the sampling features of the previous level. Input the super-resolution block and the sampling features of the previous level respectively. conduct After being copied multiple times, it is concatenated with the output channel of the super-resolution submodule and input into the decoding block; the stride of the deconvolution layer is... The output of the final multilayer perceptron of the decoding block is compared with the dense point cloud upsampled from the previous stage. The points are merged into a denser point cloud upsampled at this level. The output of the multilayer perceptron at the last stage of the decoding submodule is used as the dense point cloud feature at that stage. .

7. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 1, characterized in that, The network extraction module adopts an octree-based U-Net architecture.

8. The explicit surface reconstruction system for noisy sparse point clouds as described in claim 7, characterized in that, The specific structure of the network extraction module is as follows: The output of the octree-based U-Net is fed into two output heads: a Tanh output head for predicting vertex position offsets and a Sigmoid output head for predicting edge classification labels. The input of the octree-based U-Net is an expanded, multi-level dense point cloud. The multi-level dense point cloud Perform morphological dilation so that the leaf nodes of the octree constructed based on it can completely cover the object surface.

9. The training method for an explicit surface reconstruction system of noisy sparse point clouds as described in any one of claims 1 to 8, characterized in that, Truth-supervised training is employed, where: The feature extraction module, anchor point generation module, and multiple cascaded upsampling modules are trained together. The network extraction module is trained separately.

10. The training method for the explicit surface reconstruction system of noisy sparse point clouds as described in claim 9, characterized in that, The total loss function used in the joint training of the feature extraction module, anchor point generation module, and multiple cascaded upsampling modules is: ； in, , For loss weights; The constraint chamfer distance loss of the anchor point generation module The calculation method is as follows: ； in, For radix operations; A template point cloud with an ideal distribution, obtained by uniformly sampling the same number of training samples; The upsampling module uses the current level's output point cloud. and a corresponding number of GT surface uniform sampling point clouds chamfer distance The calculation method is as follows: ； in, Calculate the chamfer distance; 。

Images