Point cloud three-dimensional reconstruction method based on UDF combined cross product geometric prior

Abstract

The invention relates to the technical field of computer vision and three-dimensional reconstruction, particularly discloses a neural unsigned distance field (UDF) learning method and a surface reconstruction method based on cross product geometric prior, and aims to solve the technical problems that the gradient direction is unstable and the reconstructed surface is fragmented and discontinuous due to the fact that an existing neural UDF cannot be differentiated at a zero level set. The method is characterized in that a geometric prior constraint mechanism based on an original point cloud cross product is introduced; a local surface normal vector is estimated by calculating the cross product robustness of three adjacent points in an input point cloud, and a neural UDF gradient direction learned by constructing a loss function forced constraint is strictly aligned with a normal vector direction; in the optimal scheme, near surface optimization is focused in combination with an adaptive weight strategy, and a network is jointly driven to learn a more accurate and smoother UDF in combination with unsupervised chamfering distance loss and surface distance constraint. According to the method, the fuzziness of the UDF gradient near a zero level set is effectively overcome, the continuity and integrity of a reconstructed surface are remarkably improved, the method is particularly suitable for high-quality reconstruction of an open surface and an object with a complex internal structure, and the key indexes of the method are superior to those of an existing advanced technology through verification of multiple standard data sets.

Description

Technical Field

[0001] This invention relates to the research and application of a point cloud 3D reconstruction method based on UDF joint cross product geometric prior, and a professional method for point cloud 3D reconstruction using UDF joint cross product geometric prior. Existing technology

[0002] In the field of 3D reconstruction, especially in the modeling of open surfaces (such as internal mechanical structures and clothing models) and non-watertight geometries, unsigned distance fields (UDFs) have become a key technology because they can overcome the limitations of closed surfaces (Chibane et al., NeurIPS 2020). Traditional methods based on signed distance fields (SDFs) (Park et al., CVPR 2019), while having the advantage of differentiability, cannot handle open geometric topologies, while UDFs solve this bottleneck by abandoning sign constraints.

[0003] However, existing neural UDF learning methods still have serious shortcomings. Mainstream methods such as CAP-UDF (Zhou et al., TPAMI 2024) rely on unsupervised optimization using chamfer distance loss, but the non-differentiability of UDFs at the zero level set leads to disordered gradient directions, causing error propagation and resulting in holes in the reconstructed surface. LevelSetUDF (Zhou et al., ICCV 2023) attempts to guide the direction through pseudo-gradient assumptions, but the assumed directions deviate from the normals of the real surface, resulting in breaks in complex structures (such as the joints of a car chassis). In addition, DEUDF (Xu et al., AAAI 2025) requires predefined fixed displacement patterns to generate training point sets, which is difficult to adapt to irregular surfaces in real scan data, leading to the loss of texture details. The above methods are particularly insufficient in reconstructing accuracy for thin-walled open surfaces (such as clothing folds) because their constraint mechanisms do not focus on key areas of the surface.

[0004] To address the aforementioned shortcomings, this invention proposes a UDF optimization method based on cross-product geometric prior. First, for each query point, the three nearest neighbors are extracted from the original point cloud. A local surface normal estimate is generated by calculating the cross product of vectors, and consistency correction is performed based on the relative orientation of the points and the surface. Second, the UDF gradient direction predicted by the neural network is forced to be strictly aligned with the corrected normal. Simultaneously, a distance-adaptive dynamic weighting strategy is introduced, prioritizing optimization of the critical region near the surface. Finally, a multi-objective optimization framework is constructed by combining chamfer distance loss, surface zero-value constraint loss, and normal alignment loss, driving the neural network to learn a continuous and accurate unsigned distance field. Summary of the Invention

[0005] 1. Purpose of the invention

[0006] The purpose of this invention is to provide a point cloud 3D reconstruction method based on UDF and cross-product geometric prior. This method aims to solve the problems of surface fragmentation, loss of geometric details, and insufficient generalization in the reconstruction of open surfaces and complex internal structures due to the non-differentiability of UDF at the zero level set. By introducing cross-product geometric prior constraints, this invention can stabilize the UDF gradient direction and significantly improve the accuracy of distance field learning near the surface. In addition, this method combines an adaptive weighting strategy and a multi-objective optimization framework to effectively focus on key reconstruction regions, achieving high-fidelity reconstruction of complex topological structures while maintaining the algorithm's generalization ability. Compared with existing technologies, this invention can not only generate continuous and complete open surfaces, but also accurately restore thin-walled structures and internal mechanical details, providing a more reliable solution for 3D reconstruction applications in industrial design, virtual reality, and other fields.

[0007] 2. Technical Solutions

[0008] This method is mainly divided into five parts: neural UDF representation, unsupervised UDF learning, cross product geometric constraints, training steps, and surface extraction algorithm.

[0009] like Figure 1 As shown, UDF refers to any point in space. Shortest absolute distance to the surface And if and only if On the surface, =0, that is:

[0010]

[0011] Any surface can be implicitly represented using a UDF. Neural UDFs refer to those represented using an MLP. The network accepts point cloud data as input and outputs the UDF value corresponding to each point. The network employs an 8-layer MLP with 256 parameters per layer and uses geometric initialization (Atzmon et al., CVPR 2020). The UDF-based 3D reconstruction method involves continuous training and optimization to ensure the network accurately fits the surface's UDF, and finally reconstructs the surface using the learned neural UDF. The overall process of this method is as follows: Figure 2 As shown.

[0012] Current neural UDF learning is mainly divided into supervised learning and unsupervised learning. Supervised learning trains the network by minimizing the difference between the predicted UDF and the true UDF. However, UDFs are not differentiable on the surface, which leads to the instability and loss of UDF gradients near the zero level set. Supervised learning, which relies on gradient backpropagation optimization, does not perform well in reconstructing the surface, and the reconstructed surface often exhibits high fragmentation. Therefore, this method first adopts unsupervised learning for training, using chamfer distance loss. Optimize the network without relying on specific truth values.

[0013] For any point We go in the opposite direction of UDF ,move Distance reached .if Perfect fit to UDF It will be projected onto the surface, then Composition of moving point set and original surface point cloud The chamfer distance will be minimized, that is:

[0014]

[0015] This optimization method can mitigate the impact of surface nondifferentiability to some extent by implicitly incorporating gradients into the optimization process, but there is still room for improvement in reconstruction accuracy.

[0016] By analyzing the geometric properties of the UDF gradient, we find that the UDF gradient direction is consistent with the surface normal vector direction. Therefore, we propose a gradient constraint method based on the geometric prior of the cross product. First, we estimate the surface normal vector by calculating the cross product of the original point cloud. Next, cosine similarity is used to force alignment of UDF gradients. and normal vector estimation Furthermore, the UDF gradient exhibits instability only near the surface. We address this by introducing a weighting function. This allows the network to focus on optimizing gradients near the zero level set. Cross product geometric constraints. The formula is as follows:

[0017]

[0018] In practice, this method employs a single-shape training approach, allowing only one point cloud to be input at a time. The network can only fit a single point cloud's UDF, lacking generalization ability. Before training, we first normalize the point cloud and sample a certain number of original point clouds, adding normally distributed noise to construct query points. We then input the query points and the original point clouds into the network, learning the UDF by optimizing chamfer distance loss, cross product constraints, and other parameters.

[0019] After training, this method divides the entire space into The grid is used to calculate the UDF value and gradient of each grid point. When the gradients of two points are opposite, the surface will be between the two points. The UDF value is used to perform linear interpolation to determine the specific location of the surface and complete the reconstruction of the surface.

[0020] 3. Technical Effects

[0021] The point cloud 3D reconstruction method proposed in this invention, based on UDF learning and joint cross product geometric priors, achieves significant technical results. Firstly, by introducing cross product geometric prior constraints, it effectively solves the gradient direction ambiguity problem caused by the non-differentiability of the zero-level set in UDF, significantly improving the continuity of the reconstructed surface. On the DeepFashion dataset, the normal consistency reaches 99.23%, as shown in the reconstruction results. Figure 4 As shown in the figure. Secondly, the adaptive weighting strategy employed in this method can intelligently focus on key regions of the surface, achieving a reduction in reconstruction error of complex internal structures to 0.00255 and 0.00320 on the ShapeNet-Cars and Synthetic Rooms datasets, respectively. The reconstruction results are shown in the figure. Figure 5 As shown. Furthermore, the mechanism of directly utilizing the original point cloud to construct constraints enhances the algorithm's generalization ability, achieving superior reconstruction accuracy on real SRB scan data compared to existing methods, and accurately preserving subtle features such as texture. In summary, this invention demonstrates outstanding technical advantages in the field of 3D reconstruction of open surfaces and complex structures, providing a more reliable solution for applications such as industrial design and virtual try-on.

[0022] V. Attached Figures and Brief Descriptions

[0023] Figure 1 Explain the differentiability of the UDF. The gradients of the UDF on different sides of the surface are opposite, and therefore it is not differentiable on the surface.

[0024] Figure 2 Our method is summarized below. First, for any query point, we estimate the normal vector using the cross product of the three closest non-collinear original points and redirect it according to the direction from surface to surface. Second, we construct the cross product constraint by aligning the gradient sums. Finally, we optimize the chamfer distance loss between the moving point and the original point to achieve unsupervised UDF learning.

[0025] Figure 3 The effect of gradients on UDF learning: (a) With the correct gradient, the neural network learns the correct UDF value. (b) When there is bias, minimizing the chamfer distance loss will cause the network to learn the wrong UDF value.

[0026] Figure 4 The reconstruction results of DeepFashion.

[0027] Figure 5 Reconstruction results of ShapeNet Cars. Detailed Implementation

[0028] This invention relates to a neural network method for learning unsigned distance fields (UDFs), particularly suitable for reconstructing complex, non-closed 3D surface structures from dense point clouds. Its core innovation lies in introducing a geometric prior constraint mechanism based on cross product to improve the gradient instability problem of UDFs near the zero level set, significantly enhancing the continuity and geometric consistency of the reconstruction results.

[0029] I. UDF Neural Network Modeling

[0030] This invention first employs a multilayer perceptron (MLP) neural network to approximate the UDF function in the input space. This network takes three-dimensional coordinate points as input and outputs the shortest unsigned distance from those points to a certain implicit surface.

[0031] Given a set of original point clouds We construct a query point set using a certain number of original points and adding normally distributed noise. The formulaic representation of neural UDFs is as follows:

[0032]

[0033] in, This represents the network parameters. The value of this function is in the zero level set (i.e., An implicit surface is defined on the network, and the network objective is to approximate the true unsigned distance of these points.

[0034] II. Gradient-Driven Chamfer Distance Optimization

[0035] This invention optimizes the predicted UDF using the Chamfer distance loss function under an unsupervised learning setting.

[0036] First, we use points Predicted gradient By Move in the opposite direction of the gradient. The distance finally reached ,Right now:

[0037]

[0038] Under rational conditions, if the neural network accurately learns the surface UDF, then the set of moving points... It should be precisely projected onto the target surface. Therefore, by minimizing the set of moving points... and original point cloud The chamfered distance between the points allows for network optimization without requiring actual UDF values. The Chamfer distance loss function is defined as follows:

[0039]

[0040] This loss function effectively utilizes the distribution information of the point cloud, guiding the network to converge to the true implicit surface. However, if... Figure 3 As shown, when gradient estimation is flawed, minimizing the chamfer distance loss will cause the network to learn an incorrect UDF. To address the potential for misleading optimizations introduced by unstable gradients, additional geometric constraints are required.

[0041] III. Construction of geometrical prior constraints for cross product

[0042] To address the gradient uncertainty problem caused by the non-differentiability of UDF at the zero level set, this invention proposes a gradient constraint strategy that integrates geometric priors. The local normal vector is calculated based on the three-point cross product and aligned with the gradient direction of the UDF.

[0043] 1. Normal vector estimation method:

[0044] For any query point We select the three nearest non-collinear points from the point cloud based on KDTree. Construct two edge vectors:

[0045]

[0046] The normal vector is obtained by calculating the cross product:

[0047]

[0048] To ensure the estimated direction value and surface to Since the direction is consistent, we use arrive redirecting the direction :

[0049]

[0050] 2. Gradient alignment loss construction:

[0051] The gradient of a UDF should align with the direction of the fastest change, and the geometric normal vector also reflects the direction of the fastest surface change; therefore, they should be consistent. Based on this principle, we use cosine similarity to construct the cross product geometric constraint. By aligning the predicted UDF gradient and the estimated cross product normal vector, we effectively solve the problems of instability and vanishing gradients at the zero level set. The formula for the cross product geometric constraint is as follows:

[0052]

[0053] in, It is a weighting function used to enhance the constraint weights of the network in the region near the surface, and is defined as follows:

[0054]

[0055] in The hyperparameters are used to control the influence of the cross product constraint during training. This strategy ensures that the network is more compliant with gradient alignment near UDF is 0, effectively alleviating the gradient oscillation problem in the zero level set region.

[0056] IV. Zero Level Set Constraint

[0057] To further ensure that the implicit surface output by the network is consistent with the input point cloud, this invention uses the original point cloud. As input, an explicit zero-level set constraint term was constructed:

[0058]

[0059] This constraint ensures that the original point cloud is accurately mapped to zero points on the implicit surface, enhancing the surface fitting accuracy of the reconstructed surface.

[0060] V. Joint Optimization of Objective Function

[0061] The three losses mentioned above work together to form the final training optimization objective:

[0062]

[0063] in, is a hyperparameter used to balance the importance of the three loss terms, and is usually determined by optimization on the validation set in experiments.

[0064] VI. Network Structure and Training

[0065] Our network takes a point cloud as input and outputs the predicted unsigned distance for each point. The intermediate network consists of 8 MLP layers, each with 256 parameters, and applies geometric initialization. Geometric initialization is used to accelerate model convergence and avoid getting trapped in local optima. During initialization, we initialize the MLP output as a sphere, and the gradient is always 1, expressed mathematically as:

[0066]

[0067] in Let be the radius of the sphere.

[0068] During training, we use a cosine annealing strategy, initialized to 0.001, and gradually reduced to 0 during the training period.

[0069] To validate the effectiveness of our method, we systematically evaluated it on four typical datasets: ShapeNetCars, Synthetic Rooms, DeepFashion, and SRB. ShapeNet Cars contains various vehicle models, whose complex internal structures pose a significant challenge to the reconstruction algorithm. Synthetic Rooms contains 5000 synthetic interior scenes, constructed by randomly combining ShapeNet furniture models (such as tables, chairs, and sofas) to create multi-object scenes. DeepFashion, as a representative open surface dataset, provides 3D models for multiple clothing categories. The SRB dataset includes five real-world scan models, allowing us to verify the algorithm's robustness on real-world scan data.

[0070] During the experiments, we randomly selected a 3D model for each dataset, generated the input point cloud through uniform sampling, and applied a normal distribution perturbation to create the query point set. In the evaluation phase, we assessed the reconstruction quality by uniformly sampling 100,000 points from both the original and reconstructed surfaces.

[0071] VII. Surface Extraction

[0072] After training the neural network, we employ the CAP-UDF reconstruction algorithm to extract surfaces from the learned neural UDFs. This method is an improvement on March Cubes, primarily utilizing gradient direction to determine surface location. When the gradient directions of two points are opposite, the surface lies between them, and the precise surface location is determined by the interaction of the UDF values ​​at these two points.

[0073] In the surface extraction process, we divide the three-dimensional space into The mesh is constructed, and the learned UDF is used to calculate the UDF values ​​and gradients of all mesh points. Finally, a reconstruction algorithm is used to traverse the mesh points to extract the surface.

Claims

1. A method for learning an unsigned distance field using a neural network, characterized in that, Includes the following steps: a) Data preparation: Receive the raw point cloud dataset From this, a set of query points is generated. and to a) Add Gaussian perturbations to the points to enhance generalization ability; b) Unsigned distance prediction: Utilize a multilayer perceptron (MLP) neural network to establish an unsigned distance function. , using three-dimensional coordinate points c) Gradient-driven point projection: For each query point, the output is the shortest distance to the implicit surface; Calculate its corresponding gradient And move that point along the gradient direction to obtain The amount of movement is To approximate the real surface; d) Normal vector estimation: for each query point Select the three nearest non-collinear points from the original point cloud. Calculate the cross product vector The unit normal vector is obtained by adjusting the direction of the normal vector based on the directional relationship between the point and the triangle face. ; e) Gradient alignment constraint: This will adjust the predicted gradient... With estimation of normal vector Alignment, constructing a gradient consistency loss term to guide the network to output a reasonable gradient direction near the zero level set; f) Zero level set constraint: for each point in the original point cloud ,make g) Enhance the consistency between the reconstructed surface and the input data; g) Joint optimization: through the total loss function Optimize network parameters, among which This represents the Chamfer distance loss. This represents the gradient consistency loss. h) Surface reconstruction: An improved gradient-direction-based Marching Cubes algorithm is used to extract a 3D surface structure model from the learned UDF.

2. The method according to claim 1, characterized in that, The multilayer perceptron described in step b) is a neural network with 8 layers, each with 256 neurons, and uses a geometric initialization strategy.

3. The method according to claim 1, characterized in that, In step e), when constructing the gradient consistency loss, a weight function is introduced. To enhance the constraint strength of the zero-level set neighborhood, where It is an adjustable hyperparameter.

4. The method according to claim 1, characterized in that, The three points used to estimate the normal vector in step d) are the query points. The three points closest to Euclidean distance.

5. The method according to claim 1, characterized in that, In step c), the Chamfer distance loss is defined as the sum of the symmetric minimum distances between the predicted point set and the point cloud: .

6. The method according to claim 1, characterized in that, In step h), when the UDF signs of two adjacent grid points are opposite and their gradient directions are opposite, it is determined that there is a surface between them, and the intersection point of the surface is calculated by linear interpolation.

7. The method according to claim 1, characterized in that, This method is applicable to indoor scene modeling, 3D reconstruction of clothing, restoration of mechanical internal structures, and reconstruction of real scanned point cloud data.

8. The method according to claim 1, characterized in that, This method is a completely unsupervised learning framework that does not require any real distance or label information.

9. A point cloud 3D reconstruction method based on UDF joint cross product geometric prior, characterized in that, This method realizes a point cloud 3D reconstruction method based on UDF joint cross product geometric prior as described in any one of claims 1 to 8.

Images