A method and apparatus for implicit reconstruction of point cloud surfaces based on surface accuracy optimization strategy

Abstract

This invention discloses an implicit reconstruction method and apparatus for point cloud surfaces based on a surface accuracy optimization strategy. For the input point cloud shape, the method first generates initial shape patches based on spherical patch representation, and calculates the offset value of each sampling point within each initial shape patch relative to the patch center as the relative position of the sampling point. Secondly, in the network encoding stage, the implicit features representing the shape patch are obtained based on the point cloud feature information in the neighborhood of each sampling point. Then, in the network decoding stage, a tolerance parameter is introduced based on the surface accuracy optimization strategy to control the accuracy of the neural network in surface learning, obtaining the symbol distance value of each sampling point. Finally, based on the obtained symbol distance values ​​of each sampling point, its zero isosurface is extracted to obtain the surface model of the three-dimensional shape. This invention reconstructs the overall structure of the point cloud shape while effectively preserving its local fine structure, overcoming the limitation of traditional methods that struggle to maintain the local fine structure of the shape.

Description

Technical Field

[0001] This invention belongs to the field of surface reconstruction technology, and relates to an implicit reconstruction method and apparatus for point cloud surfaces based on a surface accuracy optimization strategy. This invention can effectively reconstruct the local structure of a 3D point cloud shape and reduce adhesion between components of the reconstructed model, while preserving its fine local structure well during the reconstruction of the overall point cloud shape structure. Background Technology

[0002] Implicit reconstruction of point cloud surfaces is a crucial problem in computer graphics and 3D vision, with wide applications in embodied intelligence, humanoid robots, augmented reality, and mixed reality. Traditional geometry-based 3D reconstruction methods require high-quality point cloud data; reconstructing point cloud data with noise, missing data, or uneven distribution leads to a significant decrease in reconstruction performance. Currently, 3D point cloud reconstruction methods based on deep neural networks are receiving widespread attention from both industry and academia.

[0003] In point cloud reconstruction based on deep neural networks, learning strategies targeting the sign distance function or occupancy field, which learn a continuous implicit function field, can effectively represent the surface of objects with arbitrarily complex topologies, making the reconstruction method quite flexible and exhibiting good generalization performance. However, when using deep neural networks to learn the overall shape features of the object to be reconstructed, the effective representation and learning of local shape features are not fully considered. This results in a lack of representation and generation of local fine structures in surface reconstruction, which can easily lead to the loss of original shape structure information and the fragmentation of the fine structure of the 3D shape due to the sparse distribution of sampling points. Summary of the Invention

[0004] To effectively overcome the limitation of preserving the fine local structure of the original shape in point cloud shape surface reconstruction, this invention proposes a method and apparatus for implicit point cloud surface reconstruction based on a surface accuracy optimization strategy. This is achieved by controlling the accuracy of surface learning in a neural network through tolerance parameters, progressively learning and reconstructing the implicit surface of the point cloud shape "from coarse to fine." The invention employs a piecewise implicit surface learning network based on neighborhood point aggregation and fully connected residual features. This network directly uses point cloud data as input, capturing point cloud information within its scale region using neighborhood point aggregation modules at different scales. The obtained single-scale features are combined into multi-scale implicit features to extract local feature information from the sampled point cloud. Simultaneously, based on the surface accuracy optimization strategy and using parameters to control the accuracy of the network's surface learning, only shallow layers are used at the beginning of training, and the number of network layers is gradually increased with the number of training epochs. This strategy allows the network to progressively learn increasingly complex shape surface structures. This invention can better reconstruct the local structure of a 3D point cloud shape and reduce the adhesion between the components of the reconstruction model. While effectively reconstructing the overall structure of the point cloud shape, it can also maintain its local fine structure.

[0005] In one aspect, this invention provides an implicit reconstruction method for point cloud surfaces based on a surface accuracy optimization strategy, specifically including the following steps: Step 1: Use the farthest point sampling strategy to obtain the initial center position and radius of the spherical patch representing the shape of the point cloud to generate the initial shape patch; and calculate the offset value of each sampling point in each initial shape patch relative to the patch center as the relative position of the sampling point. The sampling points of the patch in the overlapping area will have multiple relative positions. Step 2: In the network encoding stage, two neighborhood point aggregation modules at different scales are used to extract the point cloud feature information in the neighborhood of the sampling points in each patch and extract the sampling point features at the corresponding scale. The features of the sampling points at different scales are spliced ​​to form multi-scale features. Then, the multi-scale features are passed through an MLP perceptron to obtain the latent vector corresponding to the sampling point. The latent vector of each sampling point in each shape patch is maximized to obtain the latent features representing the shape patch. Step 3: In the network decoding stage, a tolerance parameter is introduced based on the surface accuracy optimization strategy to control the accuracy of the neural network in learning the surface. At the beginning of training, only a shallow network is used, and the number of layers of the neural network is gradually increased as the number of training rounds increases. During the network learning process, the relative coordinates and their corresponding shape patch latent features are used as the decoder input to obtain the symbolic distance value of the relative position of each sampling point. For sampling points located in the overlapping area of ​​different patches, the symbolic distance values ​​of all their relative positions are weighted and summed to obtain their symbolic distance value. Step 4: Based on the obtained symbol distance values ​​of each sampling point, use the Marching Cube algorithm to extract its zero isosurface to obtain a three-dimensional surface model, thus realizing implicit surface reconstruction.

[0006] In a neural network encoder, for the input containing... When performing feature encoding on point cloud data from each sampling point, the first step is to obtain the feature value through a perceptron. The system first extracts 256-dimensional latent vectors from the sampling points. Then, it uses two neighborhood point aggregation modules at different scales to encode the relative coordinates of each sampling point into 256-dimensional latent vectors to extract the feature information of adjacent sampling points (the first module aggregates 8 neighboring points around each sampling point, and the other aggregates 16 neighboring points). Finally, the latent features of the sampling points output by the two neighborhood point aggregation modules are concatenated to obtain the latent features of the sampling points. Multi-scale latent features are obtained; then, they are passed through an MLP perceptron to obtain the latent vector corresponding to the sampling point, and the latent vector of the sampling point in each shape patch is maximized to finally obtain the point cloud surface neighborhood point aggregation features. Dimensional encoding result.

[0007] A surface accuracy optimization strategy is introduced into the neural network decoder. First, a shallow network is used to learn a surface accuracy error at the beginning of training. The rough shape of the value, the surface accuracy error increases with the number of training rounds. As the value decreases, more layers are gradually added to improve surface accuracy, eventually forming a complete decoder structure. When adding untrained new layers to the decoder, the previously trained shallow networks are easily affected. Therefore, a weight is assigned to the new layer within a set number of training epochs after its addition. The weights of the old layers of the network before the addition are: and weight The number of training rounds increases linearly from 0 to 1.

[0008] In another aspect, the present invention also provides an implicit reconstruction device for point cloud surfaces based on a surface accuracy optimization strategy, comprising the following modules: The patch sampling module generates initial shape patches based on spherical patches representing the shape of the point cloud, and calculates the offset value of each sampling point in each initial shape patch relative to the patch center as the relative position of the sampling point.

[0009] The latent feature extraction module is used in the network encoding stage to obtain the latent features that characterize the shape of the face based on the point cloud feature information in the neighborhood of the sampling points in each facet.

[0010] The symbol distance value acquisition module is used in the network decoding stage to introduce tolerance parameters based on the surface accuracy optimization strategy to control the accuracy of the neural network in surface learning and obtain the symbol distance value of each sampling point.

[0011] The implicit surface reconstruction output module is used to extract the zero isosurface of each sample point based on the obtained symbol distance value and the MarchingCube algorithm to obtain a three-dimensional surface model, thereby realizing implicit surface reconstruction.

[0012] Beneficial effects of the present invention: Based on the three-dimensional shape surface accuracy optimization strategy, the present invention proposes a piecewise implicit surface reconstruction network for three-dimensional point cloud shapes, which can effectively reconstruct the local structure of three-dimensional point cloud shapes and reduce the adhesion phenomenon between various components of the reconstruction model. While reconstructing the overall structure of the point cloud shape, it can well maintain its local fine structure, thereby overcoming the limitation of traditional methods that are difficult to maintain the local fine structure of the shape. Attached Figure Description

[0013] Figure 1 This is a diagram of the implicit reconstruction network structure of point cloud surfaces according to the present invention; Figure 2 for Figure 1 Encoder structure diagram; Figure 3 for Figure 1 Decoder structure diagram; Figure 4 The image shows an example of the effect of implicit surface reconstruction for different models in the embodiment. Detailed Implementation

[0014] The technical method and implicit reconstruction effect of point cloud surface of the present invention will be further described and explained below with reference to the accompanying drawings.

[0015] like Figure 1 As shown, an implicit reconstruction method for point cloud surfaces based on a surface accuracy optimization strategy specifically includes the following steps: 1) The initial center position and radius of the spherical facet representing the shape are obtained by using the farthest point sampling strategy to generate the initial shape facet; then the offset value of each sampling point in each initial shape facet relative to the center of the facet is calculated as the relative position of the sampling point. Among them, the sampling points of the facet in the overlapping area will have multiple relative positions. 2) In the network encoding stage, two neighborhood point aggregation modules at different scales are used to encode the relative coordinates of each sampling point into a 256-dimensional latent vector to extract the feature information of adjacent sampling points. The first module aggregates 8 neighborhood points around each sampling point, while the other aggregates 16. Then, the sampling point feature information output by the two modules is concatenated to obtain the multi-scale feature of the sampling point. This feature is then passed through a 4-layer MLP perceptron (512, 512, 256, 256) to obtain the latent vector corresponding to the sampling point. The latent vector of each sampling point in each shape patch is maximized to represent the latent features of the shape patch. 3) In the network decoding stage, based on the surface accuracy optimization strategy, the accuracy of the neural network for surface learning is controlled by introducing tolerance parameters. At the beginning of training, only shallow networks are used, and the number of layers of the neural network is gradually increased as the number of training rounds increases. During the network learning process, the relative coordinates and their corresponding latent features of the surface are used as the input of the decoder to obtain the symbolic distance value of the relative position of each sampling point. The symbolic distance value of the sampling points located in the overlapping areas of different surfaces is obtained by weighted summation of the symbolic distance values ​​of all their relative positions. 4) Based on the obtained symbol distance values ​​of each sampling point, the zero isosurface of the symbol distance values ​​of the sampling points is extracted using the Marching Cube algorithm to obtain a three-dimensional shape surface model, thereby realizing implicit reconstruction of the curved surface.

[0016] like Figure 2 As shown, in the neural network encoder, the encoder input is the coordinate information of the sampling points, which is first obtained through a 2-layer perceptron (64, 128). First, the system extracts point feature information; then, it uses two neighborhood point aggregation modules at different scales to extract local feature information and concatenates them to obtain the combined multi-scale feature. For example... Figure 2 As shown, in the neighborhood point aggregation module, the local features of the point cloud are processed in two branches: The first branch aggregates the features of each sampling point from the surrounding area using the k-NN algorithm. The neighboring points are obtained 3D feature matrix The second branch augments the local feature information of the point cloud through repeated operations. Doubled 3D feature matrix Then, subtracting these two feature matrices yields... 3D feature matrix Then the feature matrix and Perform a connection operation to obtain The feature matrix is ​​divided into two dimensions, and finally, max pooling is performed on the second dimension of the feature matrix to obtain the final feature matrix. The second neighborhood point aggregation module also performs the above-mentioned aggregation of neighborhood point features. Dimensional feature information to The process involves aggregating neighborhood points using neighborhood feature aggregation information from two point cloud modules. Then, the single-scale features output from both modules are concatenated to obtain... Multi-scale feature information is obtained, and finally, a 4-layer perceptron (512, 512, 256, 256) stacked structure is used to obtain the final point cloud neighborhood aggregation features. Dimensional encoding results. Among them, the neighborhood point aggregation modules at two different scales... Set them to 16 and 8 respectively.

[0017] like Figure 3 As shown, a surface accuracy optimization strategy is introduced into the neural network decoder. First, a shallow network is used to learn a surface accuracy error at the beginning of training. The rough shape of the value, the surface accuracy error increases with the number of training rounds. As the value decreases, more layers are gradually added to improve surface accuracy, eventually forming a complete decoder structure. When adding untrained new layers to the decoder, the previously trained shallow networks are easily affected; therefore, a weight is assigned to the new layer within the first 200 training iterations after its addition. The weights of the old layers of the network before the addition are: and weight As the number of training epochs linearly increases from 0 to 1 to allow the original network to smoothly adapt to the newly added layer, the weighted output of the network in the new layer is as follows: in , This represents the feature matrix output by the previous layer. express The feature matrix obtained after this new layer This represents the feature matrix that the network ultimately outputs at this layer. For the number of training rounds of the network, Pick .

[0018] To improve the effectiveness of point cloud surface reconstruction methods, the loss function in the implicit point cloud surface reconstruction network proposed in this invention includes the following: First, each patch of the same object should be as close as possible to the object surface. in It is an object The set of all sampling points on the surface This refers to the number of objects in a batch during network training, only if the distance between the object's surface and the patch is greater than a threshold. The loss function is used only when the threshold is set in time. It is 0.06.

[0019] Second, at least one point on the object's surface must lie within a single facet: in It is an object The set of all sampling points that are not located within any patch. For the first Sampling points of an object Located in the The weight of each facet.

[0020] Third, the dough pieces should be as small as possible to prevent significant overlap between different pieces: Represents objects The Middle The radius of each facet.

[0021] Fourth, all facets should be similar in size to prevent the surface from being reconstructed using only a small number of large facets.

[0022] in It is an object The average radius of the facet in the diagram.

[0023] The final loss function is: In the experiment, the weights of each item were taken as follows: , , , .

[0024] In another aspect, the present invention also provides an implicit reconstruction device for point cloud surfaces based on a surface accuracy optimization strategy, comprising the following modules: The patch sampling module generates initial shape patches based on spherical patches representing the shape of the point cloud, and calculates the offset value of each sampling point in each initial shape patch relative to the patch center as the relative position of the sampling point.

[0025] The latent feature extraction module is used in the network encoding stage to obtain the latent features that characterize the shape of the face based on the point cloud feature information in the neighborhood of the sampling points in each facet.

[0026] The symbol distance value acquisition module is used in the network decoding stage to introduce tolerance parameters based on the surface accuracy optimization strategy to control the accuracy of the neural network in surface learning and obtain the symbol distance value of each sampling point.

[0027] The implicit surface reconstruction output module is used to extract the zero isosurface of each sample point based on the obtained symbol distance value and the MarchingCube algorithm to obtain a three-dimensional surface model, thereby realizing implicit surface reconstruction.

[0028] In this embodiment, the effect of implicit surface reconstruction on point cloud data of different shapes is as follows: Figure 4 As shown, Figure 4 The first column shows the different input point cloud models. Figure 4 The second column shows the effect after implicit reconstruction using a curved surface. Figure 4The third column shows the actual 3D shape. As can be seen, this invention can not only completely reconstruct the overall shape of an object, but also effectively reconstruct the local structure of components such as pillows on a sofa, lamp holders of table lamps, and backrests of chairs. At the same time, the reconstructed surface model has very few artifacts and adhesions. Figure 4 This demonstrates that the point cloud surface reconstruction method proposed in this embodiment can reconstruct the local structure of the three-dimensional point cloud shape well and reduce the adhesion phenomenon between the components of the reconstruction model. While effectively reconstructing the overall structure of the point cloud shape, it can also maintain its local fine structure well.

Claims

1. A method for implicit reconstruction of point cloud surfaces based on a surface accuracy optimization strategy, characterized in that, Includes the following steps: Step 1: For the input point cloud shape, generate an initial shape patch based on the spherical patch representation, and calculate the offset value of each sampling point in each initial shape patch relative to the patch center as the relative position of the sampling point. Step 2: In the network encoding stage, based on the point cloud feature information in the neighborhood of the sampling points in each patch, the latent features representing the shape of the patch are obtained. Step 3: In the network decoding stage, a tolerance parameter is introduced based on the surface accuracy optimization strategy to control the accuracy of the neural network in surface learning, and the symbol distance value of each sampling point is obtained. Step 4: Based on the obtained symbol distance values ​​of each sampling point, use the Marching Cube algorithm to extract its zero isosurface to obtain a three-dimensional surface model, thus realizing implicit surface reconstruction.

2. The implicit reconstruction method for point cloud surfaces based on surface accuracy optimization strategy according to claim 1, characterized in that, In step one, the generation of the initial shape patch specifically involves: using the farthest point sampling strategy to obtain the initial center position and radius of the spherical patch representing the shape of the point cloud, and generating the initial shape patch; Each sampling point within the shape patch will have multiple relative positions if it is located within the overlapping area.

3. The implicit reconstruction method for point cloud surfaces based on surface accuracy optimization strategy according to claim 1, characterized in that, The second step is specifically implemented as follows: In the network coding stage, two neighborhood point aggregation modules of different scales are used to extract the point cloud feature information in the neighborhood of the sampling points in each patch and extract the sampling point features at the corresponding scale. The features of the sampling points at different scales are then spliced ​​together to form multi-scale features. The multi-scale feature is then passed through an MLP perceptron to obtain the latent vector corresponding to the sampling point. The latent vector of each sampling point in each shape patch is maximized to obtain the latent feature representing the shape patch.

4. The implicit reconstruction method for point cloud surfaces based on surface accuracy optimization strategy according to claim 3, characterized in that, The specific implementation process of the network coding is as follows: For the input containing When performing feature encoding on point cloud data from each sampling point, the feature is first obtained through an MLP perceptron. The system first extracts the feature information of each sampling point; then, it uses two neighborhood point aggregation modules at different scales to encode the relative coordinates of each sampling point into a latent vector to extract the feature information of adjacent sampling points. The first module aggregates 8 neighborhood points around each sampling point, while the other aggregates 16 neighborhood points. Finally, the latent features of the sampling points output by the two neighborhood point aggregation modules are concatenated to obtain the feature information of the sampling point. Multi-scale latent features are obtained; then, they are passed through an MLP perceptron to obtain the latent vector corresponding to the sampling point, and the latent vector of the sampling point in each shape patch is maximized to finally obtain the point cloud surface neighborhood point aggregation features. Dimensional encoding result.

5. The implicit reconstruction method for point cloud surfaces based on surface accuracy optimization strategy according to claim 1, characterized in that, Step three is specifically implemented as follows: In the network decoding stage, a tolerance parameter is introduced based on the surface accuracy optimization strategy to control the accuracy of the neural network in learning the surface. At the beginning of training, only a shallow network is used, and the number of layers of the neural network is gradually increased as the number of training rounds increases. During the network learning process, the relative coordinates and their corresponding shape patch latent features are used as the decoder input to obtain the symbolic distance value of the relative position of each sampling point. The sampling points located in the overlapping areas of different patches are weighted and summed to obtain their symbolic distance value.

6. The implicit reconstruction method for point cloud surfaces based on surface accuracy optimization strategy according to claim 5, characterized in that, The specific implementation process of network decoding is as follows: A surface accuracy optimization strategy is introduced into the neural network decoder. First, the network learns surface accuracy errors at the beginning of training. The rough shape of the value, the surface accuracy error increases with the number of training rounds. As the value decreases, more layers are gradually added to improve surface accuracy, eventually forming a complete decoder structure. When adding untrained new layers to the decoder, the previously trained shallow networks are easily affected. Therefore, a weight is assigned to the new layer within a set number of training epochs after its addition. The weights of the old layers of the network before the addition are: and weight As the number of training epochs increases linearly from 0 to 1, the output of the network in this new layer after weighted calculation is: ,in , This represents the feature matrix output by the previous layer. express The feature matrix obtained after this new layer This represents the feature matrix that the network ultimately outputs at this layer. This is the number of training rounds for the network at this point.

7. A point cloud surface implicit reconstruction device based on a surface accuracy optimization strategy, used to implement the point cloud surface implicit reconstruction method according to any one of claims 1 to 6, characterized in that, Includes the following modules: The patch sampling module generates initial shape patches based on spherical patches representing the shape of the point cloud, and calculates the offset value of each sampling point in each initial shape patch relative to the patch center as the relative position of the sampling point; The latent feature extraction module is used to obtain the latent features representing the shape of the facet based on the point cloud feature information in the neighborhood of the sampling points in each facet during the network encoding stage. The symbol distance value acquisition module is used in the network decoding stage to introduce tolerance parameters based on the surface accuracy optimization strategy to control the accuracy of the neural network in surface learning and obtain the symbol distance value of each sampling point. The implicit surface reconstruction output module is used to extract the zero isosurface of each sample point based on the obtained symbol distance value and the Marching Cube algorithm to obtain a three-dimensional surface model, thereby realizing implicit surface reconstruction.

Images