Point cloud surface implicit reconstruction method based on a slice learning strategy
By adopting an implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy, we have solved the shortcomings of traditional methods in local feature representation and high-resolution reconstruction. This method has achieved effective reconstruction of the overall structure and local details of 3D shapes, improving reconstruction performance and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU NORMAL UNIVERSITY
- Filing Date
- 2022-12-13
- Publication Date
- 2026-05-01
AI Technical Summary
Existing surface reconstruction methods based on deep neural networks have shortcomings in handling noise and local feature representation, resulting in unsatisfactory reconstruction performance, especially in the generation of local structural information and high computational resource requirements in high-resolution model reconstruction.
An implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy is adopted. The network is designed through a piecewise learning strategy, and the sampling point neighborhood point aggregation and fully connected point cloud residual feature decoding are used to generate a three-dimensional shape mesh model by combining the Marching Cube algorithm, which preserves local structural details and improves robustness.
It effectively reconstructs the overall structure and local details of 3D shapes, improves reconstruction performance, reduces computational resource requirements, and enhances robustness to noise and point cloud normals.
Smart Images
Figure CN115830271B_ABST
Abstract
Description
An Implicit Reconstruction Method for Point Cloud Surfaces Based on Piece Learning Strategy Technical Field
[0001] This invention belongs to the field of surface reconstruction technology and relates to an implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy. Background Technology
[0002] Point cloud surface reconstruction is an important problem in computer graphics and 3D vision, with wide applications in computer-aided design, 3D printing, augmented reality, and mixed reality. Traditional geometry-based 3D reconstruction methods are often susceptible to noise in the model data itself, leading to a decline in reconstruction performance. Currently, surface reconstruction methods based on deep neural networks, which are less affected by model noise and have higher reconstruction performance, have received widespread attention from industry and academia.
[0003] In deep neural network-based surface reconstruction methods, learning strategies targeting the sign distance function (SDF) or occupancy field, which learn a continuous latent function field, can effectively represent surfaces with arbitrarily complex topologies, making implicit surface reconstruction more flexible. However, traditional global feature extraction and learning methods only use a single latent vector in the latent space to represent and learn the overall features of the object. This results in poor representation and characterization of local features of different shapes, leading to less than ideal reconstruction performance. Essentially, these two typical methods utilize deep neural networks to learn the overall shape features of the shape to be reconstructed without fully considering the effective representation and learning of local shape features. Consequently, they lack the generation of local surface details in shape surface reconstruction, easily leading to the loss of local structural information of the original shape and the appearance of breaks in fine structures due to sparse sampling points.
[0004] Learning the implicit representation of the symbolic distance field of a 3D shape using deep neural networks can typically generate shape surfaces with arbitrary topology and extract the shape surface at arbitrary resolution during surface reconstruction. However, surface reconstruction based on the implicit field learning of the global surface of a shape usually lacks the generation of local details of the shape surface, while 3D shape representation and reconstruction based on spatial voxels have extremely high computer resource requirements, making it difficult to reconstruct high-resolution 3D models. Summary of the Invention
[0005] To overcome the limitation of preserving the original local structural information in point cloud shape surface reconstruction, this invention proposes an implicit point cloud surface reconstruction method based on a piecewise learning strategy. An implicit point cloud surface reconstruction network is designed based on this strategy. This network uses piecewise surface representation and trains the local symbolic distance field of the 3D shape. For the input 3D point cloud model data, it leverages neighborhood point aggregation of sampling points and fully connected point cloud residual feature decoding to reconstruct the overall structure of the point cloud shape while effectively preserving the fine details of the 3D shape. Furthermore, it exhibits robustness against point cloud normals and noise.
[0006] The present invention specifically includes the following steps:
[0007] Step 1: Using discrete point cloud data as input, firstly, the initial center position and radius of the spherical neighborhood patches representing the shape of the point cloud are obtained by adopting the farthest point sampling strategy to generate initial patches; then, the offset value of each sampling point in each patch relative to the patch center is calculated as the relative position of the sampling point. Among them, the sampling points of patches in the overlapping area will have multiple relative positions.
[0008] Step 2: In the neural network encoder, the relative position coordinates of each sampling point are encoded into a 256-dimensional latent vector, and the latent vector of the sampling point in each shape patch is maximized to represent the latent features of the patch. Then, in the neural network decoder, a multi-layer fully connected residual feature decoding module is used to decode to obtain the shape symbol distance value of the relative position of each sampling point. The relative positions of sampling points located in the overlapping areas of different patches will be weighted and summed to obtain their corresponding symbol distance values.
[0009] Step 3: Use the Marching Cube algorithm to extract the zero isosurface of the symbolic distance values of the sampling points to obtain a three-dimensional mesh model.
[0010] In neural network encoders, in order to make better use of local feature information of point cloud during point cloud reconstruction, when encoding features of point cloud data, a multilayer perceptron stacked structure with gradually increasing dimensions is first used to asymptotically encode the original local feature information of the input point cloud. Then, a neighborhood aggregation layer is used to obtain the point cloud neighborhood aggregation latent feature containing local neighborhood information of the point cloud.
[0011] In the neural network decoder, a multi-layer fully connected residual feature decoding module is used to decode and obtain the shape symbol distance value of the relative position of each sampling point. Specifically, the module is implemented with three fully connected layers, each of which uses the ReLU activation function and performs residual learning only on the last two layers. That is, the output of the first layer is merged with the output of the third layer. This ensures the efficiency of neural network training while preventing gradient vanishing in neural network training and accelerating the convergence of neural network training by utilizing residual learning. Attached Figure Description
[0012] Figure 1 is a diagram of the implicit reconstruction network structure of point cloud surface according to the present invention;
[0013] Figure 2 is a structural diagram of the encoder in Figure 1;
[0014] Figure 3 is a diagram of the decoder structure in Figure 1;
[0015] Figure 4 shows an example of the effect of implicit surface reconstruction for point cloud data of different models in the embodiment;
[0016] Figure 5 shows an example of the effect of implicit surface reconstruction for point cloud data of different fine models in the embodiment. Detailed Implementation
[0017] The technical method and implicit reconstruction effect of point cloud surface of the present invention will be further described and explained below with reference to the accompanying drawings.
[0018] As shown in Figure 1, an implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy specifically includes the following steps:
[0019] Step 1: Using discrete point cloud data as input, firstly, the initial center position and radius of the spherical neighborhood patches representing the shape of the point cloud are obtained by adopting the farthest point sampling strategy to generate initial patches; then, the offset value of each sampling point in each patch relative to the patch center is calculated as the relative position of the sampling point. Among them, the sampling points of patches in the overlapping area will have multiple relative positions.
[0020] As shown in Figure 1, the original input of the implicit reconstruction network for point cloud surfaces is p1,…,p n For discrete point cloud data, the initial center positions c1, c2, ..., c of each patch are first determined using a farthest point sampling strategy. 30 The corresponding patch radius is determined, and these two patch parameters are optimized using gradient backpropagation during network training. Next, the offset of each sampling point within each patch relative to the patch center is calculated as the relative position of the sampling point, such as sampling point p. i The relative position of the facet with respect to the center c1 is c1p. i The patch sampling points within the overlapping region will have multiple relative positions (e.g., sampling point p). k There are two relative positions c1p relative to the center of the facet c1 and c2 respectively. k and C2P k Then, the neural network encoder in the following steps is used to obtain the hidden features of the patch, and the neural network decoder is used to obtain the shape symbol distance value f(c) of the relative position of each sampling point. i p k Finally, a weighted sum is performed to obtain the corresponding signed distance value f(p).k ), where k = 1, 2, 3, ..., n.
[0021] Step 2: In the neural network encoder, the relative position coordinates of each sampling point are encoded into a 256-dimensional latent vector, and the latent vector of the sampling point in each shape patch is maximized to represent the latent features of the patch. Then, in the neural network decoder, a multi-layer fully connected residual feature decoding module is used to decode to obtain the shape symbol distance value of the relative position of each sampling point. The relative positions of sampling points located in the overlapping areas of different patches will be weighted and summed to obtain their corresponding symbol distance values.
[0022] Step 3: Finally, the Marching Cube algorithm is used to extract the zero isosurface of the symbolic distance values of the sampling points to obtain a three-dimensional mesh model.
[0023] As shown in Figure 2, in the neural network encoder, when encoding features for point cloud data, a multilayer perceptron stacked structure with gradually increasing dimensions is first used to asymptotically encode the original local feature information of the input point cloud. Then, a neighborhood point aggregation layer is used to obtain the point cloud neighborhood aggregation features containing the local neighborhood information of the point cloud. Specifically, the encoder input is the coordinate information of N discrete sampling points of the three-dimensional point cloud shape. First, N×256-dimensional point cloud local feature information is obtained through a three-layer perceptron (64, 128, 256). Then, the neighborhood point aggregation module is used to aggregate the extracted local feature information to obtain the aggregation result of the point cloud neighborhood features.
[0024] In the neighborhood point aggregation module, the local feature information of the point cloud is first processed by two branches: the first branch aggregates 32 neighborhood points of each sampling point using the k-NN algorithm to obtain an N×32×256 feature matrix P1; the second branch expands the local feature information of the point cloud by a factor of 32 through repeated operations to obtain an N×32×256 feature matrix P2; then, the subtraction operation between P1 and P2 is performed to obtain an N×32×256 feature matrix P3; then, the feature matrices P2 and P3 are concatenated to obtain an N×32×512 feature matrix, and max pooling is performed on the second dimension of the feature matrix to obtain N×512-dimensional point cloud neighborhood feature aggregation information; finally, this aggregation information is processed through a four-layer perceptron (512, 512, 256, 256) to obtain N×256-dimensional point cloud neighborhood feature information. Repeat the above process of neighborhood point aggregation from N×256-dimensional point cloud local feature information to N×512-dimensional point cloud neighborhood feature aggregation information, and finally use a four-layer perceptron stacked structure to obtain the final N×256-dimensional encoding result of the point cloud neighborhood aggregation feature.
[0025] As shown in Figure 3, in the neural network decoder, a multi-layer fully connected residual feature decoding module is used to decode and obtain the shape symbol distance value of the relative position of each sampling point. This module is implemented with three fully connected layers, where each layer uses the ReLU activation function and only performs residual learning on the last two layers. That is, only the output of the first layer is merged with the output of the third layer. In this way, while ensuring the training efficiency of the neural network, residual learning can be used to prevent gradient vanishing in the training of the neural network and accelerate the convergence of the neural network training. Specifically, the decoder input consists of N×3-dimensional relative coordinates of the sampling points and N×256-dimensional latent features of the patch containing the sampling points, resulting in N×259-dimensional positional feature information. The decoder first uses four fully connected residual feature decoding modules to obtain an N×256-dimensional feature matrix. Then, this feature matrix is concatenated with the decoder input to obtain an N×515-dimensional feature matrix. This operation allows the decoder to reuse its input feature information and enhance feature propagation, accelerating network training and reducing feature information loss during network propagation. Next, this matrix is passed through a three-layer perceptron (256, 128, 64) to progressively extract feature information and obtain N×64-dimensional features. Finally, a linear layer with an output dimension of 1 decodes the feature information to obtain the symbolic distance value of each sampling point.
[0026] To improve the effectiveness of point cloud surface reconstruction methods, the loss function of the implicit point cloud surface reconstruction network based on a piecewise learning strategy proposed in this invention includes the following parts: First, the error between the symbolic distance value of the sampling points predicted by the reconstruction method and the true distance value of each sampling point is compared; second, each patch of the same object is as close as possible to the object surface; third, the points on the object surface are located within at least one patch; fourth, the patches are as small as possible to prevent significant overlap between different patches; and fifth, all patches are of similar size to prevent the surface from being reconstructed using only a few large patches.
[0027] Figure 4 shows the effect of implicit surface reconstruction for different model point cloud data in this embodiment. Figure 4a shows the point cloud data of a recliner and the effect of implicit surface reconstruction; Figure 4b shows the point cloud data of a street lamp and the effect of implicit surface reconstruction; Figure 4c shows the point cloud data of a wall-mounted lamp and the effect of implicit surface reconstruction. Figure 4 demonstrates that this method can reconstruct not only the overall shape of an object but also the local component structure of the 3D shape, with good reconstruction results. Figure 5 shows the effect of implicit surface reconstruction for different detailed model point cloud data in this embodiment. Figure 5a shows the input point cloud model; Figure 5b shows the effect after implicit surface reconstruction; Figure 5c shows the actual 3D model. Figure 5 demonstrates that this method can reconstruct not only the overall shape of an object but also the fine details of the 3D shape, with good reconstruction results.
Claims
1. A method for implicit reconstruction of point cloud surfaces based on a piecewise learning strategy, characterized in that: Specifically, the steps are as follows: Step 1: Using discrete point cloud data as input, the initial center position and radius of the spherical neighborhood patches representing the shape of the point cloud are obtained by adopting the farthest point sampling strategy to generate initial patches; then, the offset value of each sampling point in each patch relative to the patch center is calculated as the relative position coordinate of the sampling point, where the sampling points in the overlapping area will have multiple relative positions; Step 2: In the neural network encoder, the relative position coordinates of each sampling point are encoded into a 256-dimensional latent vector, and the latent vector of the sampling points in each shape patch is maximized to represent the latent features of the patch; then, in the neural network decoder, a multi-layer fully connected residual feature decoding module is used to decode to obtain the shape symbolic distance value of the relative position of each sampling point, and the relative positions of sampling points located in the overlapping areas of different patches will be weighted and summed to obtain their corresponding symbolic distance values; Step 3: The zero isosurface of the symbolic distance value of the sampling points is extracted using the Marching Cube algorithm to obtain a three-dimensional shape mesh model.
2. The implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy as described in claim 1, characterized in that: In the neural network encoder described in step two, when performing feature encoding on point cloud data, a multilayer perceptron stacked structure with gradually increasing dimensions is first used to asymptotically encode the original local feature information of the input point cloud. Then, a neighborhood aggregation layer is used to obtain the point cloud neighborhood aggregation latent feature containing the local neighborhood information of the point cloud.
3. The implicit reconstruction method for point cloud surfaces based on a piecewise learning strategy as described in claim 1, characterized in that: In the neural network decoder described in step two, a multi-layer fully connected residual feature decoding module is used to decode and obtain the shape symbol distance value of the relative position of each sampling point. Specifically, it is implemented using three fully connected layers, where each layer uses the ReLU activation function and residual learning is only performed on the last two layers. That is, the output of the first layer is merged with the output of the third layer. This ensures the efficiency of neural network training while preventing gradient vanishing during neural network training and accelerating the convergence of neural network training through residual learning.