A robust three-dimensional surface reconstruction method based on multi-scale fusion

Abstract

The present application belongs to the technical field of three-dimensional surface reconstruction, and particularly relates to a robust three-dimensional surface reconstruction method based on multi-scale fusion. Three-dimensional point cloud data is acquired, a local implicit surface of neighborhood points is fitted by using a weighted least square method, and a geometric reconstruction grid model is generated; geometric features under different receptive fields are extracted through multi-scale convolution, channel attention, spatial attention and a gating mechanism are introduced to weight and fuse the geometric features under different receptive fields, feature mixing operations are used to enhance global dependency, and a deep learning grid model is generated; the geometric reconstruction grid model and the deep learning grid model are fused in a double-branch manner, a sensitive geometric area is highlighted, and an enhanced feature after fusion is obtained; the enhanced feature after fusion is input into an implicit function decoder, signed distance function values of each vertex are predicted, a triangular mesh model is generated based on the predicted signed distance function field by using an isosurface extraction algorithm, and three-dimensional surface reconstruction is completed.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional surface reconstruction technology, specifically relating to a robust three-dimensional surface reconstruction method based on multi-scale fusion. Background Technology

[0002] 3D surface reconstruction is one of the core problems in computer vision and graphics, and it is widely used in various fields such as autonomous driving, virtual reality, cultural heritage digitization, and industrial inspection. With the development of 3D scanning technology, point cloud data has gradually become the main input form for 3D surface reconstruction. However, actual point cloud data often suffers from problems such as noise, sparse sampling, complex topology, and local missing data. These problems make it difficult for traditional 3D reconstruction methods to guarantee high accuracy, smoothness, and robustness.

[0003] Existing 3D surface reconstruction methods can be broadly classified into two categories: traditional geometric methods and deep learning-based methods.

[0004] Traditional geometric methods: Traditional geometric reconstruction methods typically rely on explicit or implicit mathematical modeling, such as weighted least squares, polynomial fitting, and radial basis functions. These methods can generate accurate geometric surfaces through local fitting, and they perform well, especially in cases with low noise, sparse data, and regular topology. However, traditional methods have drawbacks, including reliance on manually defined features and priors, and limited ability to handle noise, missing data, and large-scale point clouds. They often struggle to guarantee both accuracy and efficiency, particularly when dealing with complex 3D shapes.

[0005] Deep learning-based methods: In recent years, deep learning-based 3D reconstruction methods have made significant progress. Deep neural networks, by automatically learning complex geometric patterns from large-scale data, are better able to handle noisy data and irregular sampling. Many studies have employed techniques such as convolutional neural networks and generative adversarial networks for 3D surface reconstruction, achieving stronger robustness and generalization capabilities. However, deep learning methods often lack explicit geometric constraints, which limits their ability to model details and recover local geometric structures. Furthermore, the "black box" nature of deep networks also results in poor interpretability, failing to provide an intuitive understanding for engineering applications.

[0006] To overcome these problems, some hybrid methods combining traditional geometric methods and deep learning techniques have been proposed in recent years, attempting to leverage both the high accuracy of traditional geometric methods and the powerful feature learning capabilities of deep learning methods. However, existing fusion methods typically only achieve a certain degree of balance between the advantages of both, and still have shortcomings in handling large-scale point clouds, complex geometries, and high-precision modeling.

[0007] Therefore, how to effectively combine traditional geometric fitting methods with the feature learning capabilities of deep learning to propose a three-dimensional surface reconstruction method with high accuracy, robustness, and interpretability has become an important research direction. Summary of the Invention

[0008] To overcome the problems in the prior art, this invention proposes a robust three-dimensional surface reconstruction method based on multi-scale fusion.

[0009] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0010] This invention provides a robust three-dimensional surface reconstruction method based on multi-scale fusion, comprising the following steps:

[0011] Acquire 3D point cloud data, select target points and their neighboring points, use weighted least squares method to fit the local implicit surface of the neighboring points, extract local geometric priors, and generate a geometric reconstruction mesh model;

[0012] We acquire 3D point cloud data, extract geometric features under different receptive fields through multi-scale convolution, introduce channel attention, spatial attention and gating mechanism to perform weighted fusion of geometric features under different receptive fields, and use feature fusion operation to enhance global dependency to generate a deep learning grid model.

[0013] By fusing the geometric reconstruction mesh model and the deep learning mesh model in a dual-branch manner, and combining the comprehensive indices of Gaussian curvature and adjacent edge length of each vertex in the geometric reconstruction mesh model and the deep learning mesh model, the geometrically sensitive areas are highlighted, and feature aggregation is achieved in the spatial neighborhood to obtain the enhanced features after fusion.

[0014] The fused enhanced features are input into an implicit function decoder to predict the signed distance function value of each vertex; based on the predicted signed distance function field, an isosurface extraction algorithm is used to generate a triangular mesh model, thereby completing the 3D surface reconstruction.

[0015] Further, 3D point cloud data is acquired, target points and their neighboring points are selected, and the local implicit surface of the neighboring points is fitted using the weighted least squares method. Local geometric priors are extracted, and a geometric reconstruction mesh model is generated, including:

[0016] Select a target point in the 3D point cloud data, define its local neighborhood point range, and construct a local implicit surface for the neighboring points within the local neighborhood point range;

[0017] We use the weighted least squares method to fit the local implicit surface of the neighborhood points to form the global implicit surface; and then sample the global implicit surface to generate the initial triangular mesh model through meshing.

[0018] Furthermore, 3D point cloud data is acquired, and geometric features under different receptive fields are extracted through multi-scale convolution. Channel attention, spatial attention, and gating mechanisms are introduced to weightedly fuse the geometric features under different receptive fields. Feature fusion operations are then employed to enhance global dependencies, generating a deep learning mesh model, including:

[0019] Acquire 3D point cloud data and extract geometric features under different receptive fields by setting multi-scale convolutional paths in parallel.

[0020] The importance of geometric features under different receptive fields is weighted using the channel attention mechanism to obtain the geometric features after channel attention weighting.

[0021] Based on the geometric features weighted by the channel attention mechanism, the fusion of channel attention and spatial attention is adaptively controlled by combining the spatial attention mechanism and introducing a gating factor, resulting in features fused by the channel attention and spatial attention mechanisms.

[0022] The features fused by channel attention and spatial attention are stitched together with the acquired 3D point cloud data and passed as input to a multilayer perceptron. The corresponding directed distance function value is regressed point by point, and a deep learning grid model is generated using the MarchingCubes method.

[0023] Furthermore, by combining spatial attention mechanisms and introducing gating factors, the fusion of channel attention and spatial attention is adaptively controlled, resulting in features after the fusion of channel attention and spatial attention, including:

[0024] ;

[0025] In the above formula, This represents the features after the fusion of channel attention and spatial attention; Represents the learnable channel attention weights; Represents learnable spatial attention weights; Indicates the gating factor; This represents the geometric features after being weighted by the channel attention mechanism.

[0026] Furthermore, learnable spatial attention weights for:

[0027] ;

[0028] In the above formula, Indicates the first i The feature vector of each point is the original information input into the network; GAP represents global average pooling. This represents the LeakyReLU activation function. This represents the Sigmoid activation function; and The weight matrix of the fully connected layer helps the model capture and adjust the relationships between features, thereby improving the model's expressive power and performance.

[0029] Furthermore, learnable spatial attention weights for:

[0030] ;

[0031] In the above formula, This represents the Sigmoid activation function; Indicates the first i Input features of 1 point; This represents a depthwise separable convolution.

[0032] Furthermore, the geometric reconstruction mesh model and the deep learning mesh model are fused in a two-branch manner. By combining the comprehensive indices of Gaussian curvature and adjacent edge length of each vertex in both models, geometrically sensitive regions are highlighted, and feature aggregation is achieved within the spatial neighborhood to obtain the enhanced fused features, including:

[0033] The Gaussian curvature and edge saliency of vertices in the deep learning mesh model and the geometric reconstruction mesh model are calculated respectively. After concatenating the Gaussian curvature and edge saliency, a nonlinear mapping is performed to obtain the nonlinear mapping features.

[0034] Based on deep learning grid models and geometric reconstruction grid models, attention subgraphs based on nearest neighbor retrieval are constructed in the spatial neighborhood, and the correlation weights between the target point and the neighborhood points are calculated using these attention subgraphs. Based on the correlation weights, the features in the neighborhood are weighted and aggregated to obtain local perception enhancement features.

[0035] Nonlinear mapping features are fused with local perception enhancement features, and the importance of different features is adaptively adjusted through a gated channel-spatial attention mechanism to obtain enhanced local features, including multi-scale perception gated fusion enhancement features and geometric surface reconstruction enhancement features.

[0036] The multi-scale perception gated fusion enhancement features and the geometric surface reconstruction enhancement features are respectively subjected to global average pooling to obtain the global representation vector of multi-scale perception gated fusion and the global representation vector of geometric surface reconstruction.

[0037] The multi-scale perception gated global representation vector and the geometric surface reconstruction global representation vector are concatenated and then fed into two fully connected layers and a nonlinear activation function to obtain the fused enhanced features.

[0038] Furthermore, based on deep learning grid models and geometric reconstruction grid models, attention subgraphs based on nearest neighbor retrieval are constructed within the spatial neighborhood, and the relevance weights between the target point and its neighboring points are calculated using these attention subgraphs, including:

[0039] For each target point p Search for nearest neighbors within its spatial neighborhood to form a spatial neighborhood point set. Within the spatial neighborhood set, an attention subgraph is constructed; the relevance weights between the target point and each neighboring point are calculated using the attention subgraph, and these relevance weights reflect the strength of the correlation between local features within the neighborhood; among them, the relevance weights... for:

[0040] ;

[0041] In the above formula, d The scaling factor is the feature dimension. For target point p The set of neighborhood points; Indicates the target point p A neighborhood index in the set of neighbors; Indicates the target point p The representation of the feature vector in the projection space, i.e., the query vector; Representing neighborhood points q The eigenvectors are represented in the projection space as key vectors.

[0042] Furthermore, the fused enhanced features are input into the implicit function decoder to predict the signed distance function value for each vertex, including:

[0043] The fused enhanced features are then stitched together with the corresponding original 3D point cloud data to form stitched features.

[0044] The concatenated features are input into an implicit function decoder, which employs a multilayer perceptron structure to perform nonlinear mapping and feature regression on each point, predicting the signed distance function value corresponding to that point point by point.

[0045] Furthermore, it also includes: introducing a joint loss function, which includes SDF regression loss, gradient consistency loss and marginal constraint loss.

[0046] Compared with the prior art, the present invention has the following technical effects:

[0047] (1) Starting from the complex geometric features of point cloud data, this invention proposes a robust 3D surface reconstruction method based on multi-scale fusion. Combining the advantages of traditional geometric fitting and deep learning feature modeling, high-precision, end-to-end reconstruction of point cloud data is achieved through the collaborative design of a geometric surface reconstruction branch and a scale-aware gating fusion branch. In the geometric surface reconstruction branch, a weighted least squares method is used to perform a second implicit surface fitting on the local neighborhood of the point cloud, extracting local geometric priors, and generating a geometric reconstruction mesh model through sampling and meshing, thereby maintaining high fitting accuracy and interpretability in complex local regions. At the same time, a scale-aware gating fusion branch is proposed. The scale-aware gating fusion branch captures geometric features under different receptive fields through parallel multi-scale convolution, and combines channel attention, spatial attention and gating mechanism to perform weighted fusion of geometric features. At the same time, feature mixing operation is introduced to enhance global dependencies, so that the model can take into account both local structure and global consistency.

[0048] (2) In order to better handle geometrically sensitive regions and local context relationships in point cloud data, this embodiment designs a dual-branch fusion module. This dual-branch fusion module introduces curvature and edge saliency fusion units, which highlight geometric details by combining Gaussian curvature and edge intensity indicators. At the same time, it combines local perception attention units to construct attention subgraphs in the neighborhood, realizing dynamic weighted aggregation and consistent modeling of features, thereby effectively improving the robustness of the model to complex geometric details and noise points.

[0049] (3) Based on the fused features, this embodiment inputs the enhanced representation and point coordinates into the implicit function decoder, and uses a multilayer perceptron to predict the signed distance function value point by point. Subsequently, at the zero level set, an isosurface extraction algorithm is used to generate a topologically continuous and smooth triangular mesh model. This process can transform the implicit results predicted by the network into an explicit geometric structure, taking into account both geometric accuracy and renderability.

[0050] (4) To ensure stable training and accurate reconstruction of the model, this embodiment introduces a joint loss function, including SDF regression loss, gradient consistency loss, and edge constraint loss. The SDF regression loss minimizes the difference between the predicted and true values, the gradient consistency loss improves the accuracy of the predicted normal vector, and the edge constraint loss assigns higher weights to high curvature and boundary regions to highlight geometric details. Through these multiple constraints, the network achieves a good balance between detail recovery and global consistency. Attached Figure Description

[0051] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the process of the present invention;

[0053] Figure 2 This is a comparison diagram of the reconstruction effect of the present invention with other methods;

[0054] Figure 3 This is a second comparison diagram of the reconstruction effect of the present invention with other methods. Detailed Implementation

[0055] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0056] In this embodiment, refer to Figure 1 This paper presents a robust 3D surface reconstruction method based on multi-scale fusion, comprising the following steps:

[0057] Acquire 3D point cloud data, select target points and their neighborhood points, fit a quadratic implicit surface in the local neighborhood of the point cloud based on the weighted least squares method, extract local geometric priors, and generate a geometric reconstruction mesh model.

[0058] We acquire 3D point cloud data, extract geometric features under different receptive fields through multi-scale convolution, introduce channel attention, spatial attention and gating mechanism to perform weighted fusion of geometric features under different receptive fields, and use feature fusion operation to enhance global dependency to generate a deep learning grid model.

[0059] By fusing the geometric reconstruction mesh model and the deep learning mesh model in a two-branch manner, and combining the comprehensive index of Gaussian curvature of each vertex and the length of adjacent edges, the geometrically sensitive areas are highlighted, and feature aggregation is achieved in the spatial neighborhood to obtain the enhanced features after fusion.

[0060] The fused enhanced features are input into an implicit function decoder to predict the signed distance function value of each vertex; based on the predicted signed distance function field, an isosurface extraction algorithm is used to generate a triangular mesh model, thereby completing the 3D surface reconstruction.

[0061] The following is a detailed explanation of each of the above steps:

[0062] Step 100: Acquire 3D point cloud data. In the geometric surface reconstruction branch, select the target point and its neighborhood points. Use the weighted least squares method to fit the local implicit surface of the neighborhood points, extract the local geometric prior, and generate the geometric reconstruction mesh model, which is the output of the geometric surface reconstruction branch.

[0063] As an example, step 100 specifically includes:

[0064] Step 110: Select the target point in the 3D point cloud data, define its local neighborhood point range, and construct the local implicit surface of the neighboring points within the local neighborhood point range.

[0065] For each target point in the 3D point cloud Define the first-order neighborhood points of the target point. For local neighborhood points, where, Represents the set of local neighborhood points related to the target point; Represents local neighborhood points; This represents the set of first-order local neighborhood points of the target point.

[0066] Based on the defined local neighborhood of the target point, a local implicit surface is constructed for the ternary quadratic neighborhood points:

[0067] ;

[0068] In the above formula, Indicates the first i A local implicit surface function constructed centered on a target point; , , , , , , , , , The coefficients to be fitted together determine the specific shape and position of the implicit surface; Indicates the target point The coordinates.

[0069] Step 120: Fit the local implicit surface of the neighborhood points using the weighted least squares method to form the global implicit surface; and sample the global implicit surface to generate the initial triangular mesh model through meshing operation.

[0070] The method of fitting the local implicit surface of neighboring points using weighted least squares to form a global implicit surface includes:

[0071] ;

[0072] In the above formula, Point Compared to the first i The weights of the local implicit surface functions reflect the degree of influence of the local surface on that location; S represents the total number of neighborhood points involved in the fitting. Represents a global implicit surface function.

[0073] The weighting function employs a spherical decay model, where only points falling within the effective influence range of the curved surface participate in the weighted fusion. The weighting function is defined as follows:

[0074] ;

[0075] in: Point With the target point The distance; The target point The distance from a first-order neighbor point to the farthest point in its own first-order neighborhood. A point can only participate in weighted fusion if it falls within the effective influence range of a certain surface.

[0076] Based on this, multiple local implicit surfaces are weighted and fused to form a globally consistent and continuous implicit surface. Sampling is then performed on the global implicit surface, and an initial triangular mesh model is generated through meshing operations.

[0077] Step 200: Acquire 3D point cloud data. In the multi-scale perception gated fusion branch, extract geometric features under different receptive fields through multi-scale convolution. Introduce channel attention, spatial attention and gate mechanism to perform weighted fusion of geometric features under different receptive fields. Further, use feature mixing operation to enhance global dependency and generate a deep learning grid model, which is the output of the multi-scale perception gated fusion branch.

[0078] As an example, step 200 specifically includes:

[0079] Step 210: Acquire 3D point cloud data Then, multi-scale convolutional paths are set in parallel to extract geometric features under different receptive fields:

[0080] ;

[0081] In the above formula, Represents the geometric features under different receptive fields; This indicates concatenation, which combines the output features of three convolutional paths at different scales. For three parallel convolutional paths with different kernel sizes (1×1, 3×3, 5×5); This indicates that a 1×1 convolution kernel is used on the input point cloud. x The result of performing a convolution operation; This indicates that a 3×3 convolution kernel is used on the input point cloud. x The result of performing a convolution operation; This indicates that a 5×5 convolution kernel is used on the input point cloud. x The result of performing a convolution operation; This represents the activation function, used to... Perform nonlinear transformations or further extract features; This represents the activation function, used to... Perform nonlinear transformations or further extract features; This represents the activation function, used to... Perform nonlinear transformations or further extract features; N represents the number of points in the point cloud; D represents the dimension of the feature vector of each point.

[0082] Smaller convolutional kernels (such as 1×1) can capture subtle geometric features in point clouds, such as edges and corners. Larger convolutional kernels (such as 5×5) can capture broader contextual information, helping the model understand the overall structure of the point cloud. By stitching together features at different scales, the model can simultaneously utilize local details and global contextual information, thereby enhancing its understanding and representation of point cloud geometric features.

[0083] Step 220: Use channel attention mechanism to weight the importance of geometric features under different receptive fields to obtain the weighted geometric features after channel attention mechanism:

[0084] ;

[0085] In the above formula, This represents the geometric features after weighting by the channel attention mechanism; These are learnable weight vectors; This indicates an element-wise multiplication operation.

[0086] Step 230: Based on the geometric features weighted by the channel attention mechanism, the fusion of channel attention and spatial attention is adaptively controlled by combining the spatial attention mechanism and introducing a gating factor to obtain the features after the fusion of channel attention and spatial attention.

[0087] The weighted geometric features are combined with a spatial attention mechanism to highlight regions with significant local geometric changes, and a 1×1 convolution is used to generate a gating factor at each point location. This is used to adaptively adjust the fusion ratio of channel attention and spatial attention, enhancing their complementarity and expressive balance. A gating mechanism is used to adaptively adjust the fusion ratio of channel attention and spatial attention.

[0088] ;

[0089] In the above formula, This represents the features after the fusion of channel attention and spatial attention; Represents the learnable channel attention weights; This represents the learnable spatial attention weights.

[0090] Specifically, by employing global average pooling and a fully connected network, the expressive power of each channel's features in the global semantics can be captured, thereby establishing global dependencies along the channel dimension and generating a set of learnable channel attention weights for each channel. :

[0091] ;

[0092] In the above formula, Indicates the first i The feature vector of each point is the original information input into the network.

[0093] GAP stands for Global Average Pooling; This represents the LeakyReLU activation function. This represents the Sigmoid activation function; and It is the weight matrix of the fully connected layer, which helps the model capture and adjust the relationships between features, thereby improving the model's expressive power and performance.

[0094] By applying depthwise separable convolution to the feature vector, the response intensity of each point in its local neighborhood is extracted, and a corresponding spatial attention weight is generated for each point. To highlight areas with significant geometric changes Represented as:

[0095] ;

[0096] In the above formula, This indicates that a depthwise separable convolution with a kernel size of 7×7 applies a feature mixing operation to the fused features to achieve non-local information interaction between different points, thereby enhancing global dependencies and geometric representation capabilities.

[0097] Step 240: Combine the features fused by channel attention and spatial attention with the acquired 3D point cloud data. The data is concatenated and passed as input to a multilayer perceptron (MLP). The corresponding SDF (Signed Distance Function) values ​​are regressed point by point, and the Marching Cubes method is used to generate a deep learning grid model.

[0098] Marching Cubes is a classic algorithm for extracting isosurfaces from three-dimensional scalar fields, such as volume data composed of SDF values.

[0099] Step 300: Input the output of the geometric surface reconstruction branch (geometric reconstruction mesh model) and the output of the multi-scale perception gating fusion branch (deep learning mesh model) into the dual-branch fusion module at the same time. In the dual-branch fusion module, the comprehensive index of Gaussian curvature of each vertex and adjacent edge length is combined to highlight the expressive ability of geometrically sensitive regions.

[0100] As an example, step 300 specifically includes:

[0101] Step 310: Calculate the Gaussian curvature and edge saliency of the vertices, concatenate the Gaussian curvature and edge saliency, and then perform nonlinear mapping to obtain the nonlinear mapping features.

[0102] Gaussian curvature and edge saliency are two important indicators for measuring the geometric characteristics of a 3D model.

[0103] Gaussian curvature is a scalar value describing the degree of curvature of a surface at a point. For target points in deep learning mesh models and geometric reconstruction mesh models... p Its Gaussian curvature It can be calculated using the included angle between adjacent triangular faces:

[0104] ;

[0105] In the above formula, For this vertex on the triangle face t The included angle within; This represents the set of triangles adjacent to this vertex.

[0106] Marginal salience It is an indicator that measures the change in length between adjacent edges of a vertex and is used to capture edge features in the model:

[0107] ;

[0108] In the above formula, Let be the set of adjacent edges of a vertex; Indicates the coordinates of the target point; Represents the coordinates of neighboring points; p, q Represents a vertex; It represents Euclidean distance.

[0109] After concatenating the Gaussian curvature and edge saliency, a nonlinear mapping is performed:

[0110] ;

[0111] In the above formula, This represents the characteristics of a nonlinear mapping.

[0112] By combining Gaussian curvature and edge saliency, the curvature of the surface and edge features can be comprehensively utilized to describe the geometric properties of the vertices. Nonlinear mapping can further extract and enhance these geometric features.

[0113] Step 320: Construct an attention subgraph based on K-nearest neighbor retrieval within the spatial neighborhood, where K=16, and use the attention subgraph to calculate the correlation weight between the target point and the K nearest neighbor points; based on the correlation weight, perform weighted aggregation of the features within the neighborhood to obtain local perception enhancement features, that is, obtain multi-scale perception gating fusion enhancement features and geometric surface reconstruction enhancement features.

[0114] An attention subgraph based on nearest neighbor retrieval is constructed within the spatial neighborhood using local perceptual attention units, and the relevance weights between the target point and its neighboring points are calculated using this attention subgraph, including:

[0115] For each target point p Search for nearest neighbors within its spatial neighborhood to form a spatial neighborhood point set. Within the spatial neighborhood set, an attention subgraph is constructed; the relevance weights between the target point and each neighboring point are calculated using the attention subgraph, and these relevance weights reflect the strength of the correlation between local features within the neighborhood; among them, the relevance weights... for:

[0116] ;

[0117] In the above formula, d The scaling factor is the feature dimension. For target point p The set of neighborhood points; Indicates the target point p A neighborhood index in the set of neighbors; Indicates the target point p The representation of the feature vector in the projection space, i.e., the query vector; Representing neighborhood points qThe eigenvectors are represented in the projection space as key vectors.

[0118] By utilizing the calculated relevance weights, the features of neighboring points are weighted and aggregated to obtain locally enhanced perceptual features. Weighted aggregation enhances the local consistency of overall features while improving the ability to express details. Among these, the locally enhanced perceptual features... for:

[0119] ;

[0120] In the above formula, Indicates the first p The feature vector of each point.

[0121] This mechanism can effectively introduce spatially highly correlated neighborhood information while preserving the characteristics of the vertices themselves, thereby improving the smoothness and structural consistency of feature representation.

[0122] Step 330: Map the nonlinear mapping features With local perception enhancement features The system performs fusion and adaptively adjusts the importance of different features through a gated channel-spatial attention mechanism to obtain enhanced local features. express:

[0123] ;

[0124] In the above formula, Simultaneously, attention weights are allocated in the channel and spatial dimensions to highlight geometrically critical areas.

[0125] Step 340: Perform global average pooling on the multi-scale perception gated fusion enhancement features and the geometric surface reconstruction enhancement features respectively to obtain the multi-scale perception gated global representation vector and the geometric surface reconstruction global representation vector; concatenate the multi-scale perception gated global representation vector and the geometric surface reconstruction global representation vector and feed them into two fully connected mapping layers and a nonlinear activation function to obtain the fused enhancement features.

[0126] For the multi-scale perception gated fusion branch and the geometric surface reconstruction branch, the multi-scale perception gated fusion enhancement features are respectively applied. Global average pooling is performed on the geometric surface reconstruction enhancement features to obtain a multi-scale perceptual gating global representation vector. With global representation vector ;

[0127] Gated global representation vector for multi-scale sensing With global representation vector After splicing, the data is fed into two fully connected layers and a nonlinear activation function. To learn the complementary semantics between the two reconstruction mechanisms, the fused enhanced features are obtained. :

[0128] ;

[0129] In the above formula, This represents the enhanced features of multi-scale perception gating fusion; This represents the geometric surface reconstruction enhancement feature; This indicates a feature concatenation operation. , A learnable mapping matrix, To fuse feature dimensions.

[0130] Step 400: Input the fused enhanced features into the implicit function decoder to predict the signed distance function value of each vertex.

[0131] The fused enhanced features are then stitched together with the corresponding original 3D point cloud data to form stitched features.

[0132] The concatenated features are input into an implicit function decoder; wherein the implicit function decoder adopts a multilayer perceptron structure, performs nonlinear mapping and feature regression on each point, and predicts the signed distance function value corresponding to the point point by point.

[0133] By introducing a nonlinear activation function to transform the features of each layer, the expressive power of the network is enhanced. At the same time, the end-to-end regression training method ensures that the predicted signed distance function value can accurately characterize the spatial geometric distribution of the point cloud data, thereby realizing implicit modeling and continuous expression of the three-dimensional surface.

[0134] Step 500: Based on the predicted signed distance function field, a continuous and smooth triangular mesh model is generated using an isosurface extraction algorithm, thereby completing the 3D surface reconstruction.

[0135] Based on the signed distance function field predicted by the implicit function decoder, the voxel space is progressively traversed and judged at the zero level set using the isosurface extraction algorithm. The intersection points of the voxel boundaries and the isosurfaces are calculated to generate a topologically continuous surface structure. The intersection points are then connected to form triangular patches to construct a mesh, ultimately obtaining a continuous, smooth, and high-precision triangular mesh model, thus achieving complete 3D surface reconstruction.

[0136] This embodiment introduces a joint loss function, including SDF regression loss, gradient consistency loss, and edge constraint loss. The SDF regression loss minimizes the difference between the predicted and true values, the gradient consistency loss improves the accuracy of the predicted normal vector, and the edge constraint loss assigns higher weights to high curvature and boundary regions to highlight geometric details.

[0137] During the training phase, a joint loss function is used. The model is constrained in the following form:

[0138] ;

[0139] In the above formula, the first term is the SDF regression loss, which calculates the mean squared error between the predicted SDF value and the true SDF value to minimize the difference between the predicted and true values; the second term is the gradient consistency constraint loss, which improves the accuracy of the surface normal vector by predicting the similarity of the angle between the gradient and the normal vector; the third term is the edge constraint loss, which assigns higher weights to regions with high curvature or geometric details. This enhances the model's ability to reconstruct boundaries and details.

[0140] in, Represents the actual SDF value. Indicates the predicted SDF value. Indicates the number of training samples; This represents the weights of the gradient consistency constraint loss. Indicates the angle between the normal vectors; Indicates the loss weight of the edge constraint term; This represents the gradient of the predicted SDF value; The edge weights are calculated based on information such as local curvature and edge saliency.

[0141] In summary, the robust 3D surface reconstruction method based on multi-scale fusion proposed in this embodiment can not only handle large-scale, noisy point cloud data, but also accurately recover the geometric structure of high-curvature and irregular boundary regions. Experimental results show that this method significantly enhances the robustness and detail representation of the model while improving reconstruction accuracy, providing an efficient and feasible solution for high-fidelity reconstruction of point cloud data. The reconstruction effect is visualized as follows: Figure 2 and Figure 3 As shown.

[0142] Reference Figures 2-3 The proposed methods include Sinkhorn distance-based Sinkhorn Bar Optimization (SDRO), HotSpot, OffsetOPT, Divergence-Guided Shape Implicit Neural Representation (DIGS), Implicit Geometric Regularization (IGR), Predictable Context Prior (PCP), Filtered Poisson Surface Reconstruction (SPSR), Unsigned Learning (SAL), Shape as Point (SAP), Point Convolution for Surface Reconstruction (POCO), and Ground Values ​​(GT), where ground values ​​are the real data used as the baseline.

[0143] Figure 2This paper demonstrates the reconstruction performance of our method compared to other methods. Most other methods suffer from ghosting, generating false surfaces in inappropriate locations or resulting in disconnected reconstructions. In contrast, our method significantly mitigates these issues while maintaining topological consistency. Although the OffsetOptimization (OffsetOPT) and Divergence-Guided Shape Implicit Neural Representation (DIGS) methods perform close to our method on some objects, their results significantly degrade in the third, more challenging reconstruction task of industrial parts, while our method maintains stable robustness and high accuracy.

[0144] Figure 3 This demonstrates a visual comparison of our reconstruction results with those of other methods, such as... Figure 3 As shown, other methods still have significant shortcomings in visual effects. The divergence-guided shape implicit neural representation (DIGS), point convolution for surface reconstruction (POCO), hotspot method, and offset optimization method (OffsetOPT) result in large areas of missing surfaces. Predictable context prior (PCP), unsigned learning method (SAL), shape as point method (SAP), and Sinkhorn distance-based sub-Brow bar optimization method (SDRO) are insufficient in geometric detail representation, while the screened Poisson surface reconstruction method (SPSR) produces false closures. Overall, our reconstruction results significantly outperform other methods in both accuracy and stability.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A robust three-dimensional surface reconstruction method based on multi-scale fusion, characterized in that, Includes the following steps: Acquire 3D point cloud data, select target points and their neighboring points, use weighted least squares method to fit the local implicit surface of the neighboring points, extract local geometric priors, and generate a geometric reconstruction mesh model; 3D point cloud data is acquired, and geometric features under different receptive fields are extracted through multi-scale convolution. Channel attention, spatial attention, and gating mechanisms are introduced to weightedly fuse the geometric features under different receptive fields. Feature fusion operations are then used to enhance global dependencies, generating a deep learning mesh model, including: Acquire 3D point cloud data and extract geometric features under different receptive fields by setting multi-scale convolutional paths in parallel. The importance of geometric features under different receptive fields is weighted using the channel attention mechanism to obtain the geometric features after weighting by the channel attention mechanism; Based on the geometric features weighted by the channel attention mechanism, the fusion of channel attention and spatial attention is adaptively controlled by combining the spatial attention mechanism and introducing a gating factor, resulting in features fused by the channel attention and spatial attention mechanisms. The features fused by channel attention and spatial attention are stitched together with the acquired 3D point cloud data and passed as input to the multilayer perceptron. The corresponding directed distance function value is regressed point by point, and the Marching Cubes method is used to generate a deep learning grid model. By fusing the geometric reconstruction mesh model and the deep learning mesh model in a dual-branch manner, and combining the comprehensive indices of Gaussian curvature and adjacent edge length of each vertex in the geometric reconstruction mesh model and the deep learning mesh model, the geometrically sensitive areas are highlighted, and feature aggregation is achieved in the spatial neighborhood to obtain the enhanced features after fusion. The fused enhanced features are input into an implicit function decoder to predict the signed distance function value of each vertex; based on the predicted signed distance function field, an isosurface extraction algorithm is used to generate a triangular mesh model, thereby completing the 3D surface reconstruction.

2. The robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 1, characterized in that, Acquire 3D point cloud data, select target points and their neighborhood points, fit the local implicit surface of the neighborhood points using the weighted least squares method, extract local geometric priors, and generate a geometrically reconstructed mesh model, including: Select a target point in the 3D point cloud data, define its local neighborhood point range, and construct a local implicit surface for the neighboring points within the local neighborhood point range; We use the weighted least squares method to fit the local implicit surface of the neighborhood points to form the global implicit surface; and then sample the global implicit surface to generate the initial triangular mesh model through meshing.

3. The robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 1, characterized in that, By combining spatial attention mechanisms and introducing gating factors, the fusion of channel attention and spatial attention is adaptively controlled, resulting in features after the fusion of channel attention and spatial attention, including: ； In the above formula, This represents the features after the fusion of channel attention and spatial attention; Represents the learnable channel attention weights; Represents learnable spatial attention weights; Indicates the gating factor; This represents the geometric features after being weighted by the channel attention mechanism.

4. The robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 3, characterized in that, Learnable Spatial Attention Weights for: ； In the above formula, Indicates the first i The feature vector of a point; GAP represents global average pooling; This represents the LeakyReLU activation function. This represents the Sigmoid activation function; and This is the weight matrix of the fully connected layer.

5. A robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 4, characterized in that, Learnable Spatial Attention Weights for: ； In the above formula, This represents the Sigmoid activation function; Indicates the first i Input features of 1 point; This represents depthwise separable convolution.

6. A robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 1, characterized in that, A dual-branch fusion of the geometric reconstruction mesh model and the deep learning mesh model is performed. By combining the Gaussian curvature of each vertex and the length of adjacent edges in both models, geometrically sensitive regions are highlighted, and feature aggregation is achieved within the spatial neighborhood to obtain the enhanced fused features, including: The Gaussian curvature and edge saliency of vertices in the deep learning-based mesh model and the geometric reconstruction mesh model are calculated respectively. The Gaussian curvature and edge saliency are concatenated and then nonlinearly mapped to obtain the nonlinear mapping features. Based on deep learning grid models and geometric reconstruction grid models, attention subgraphs based on nearest neighbor retrieval are constructed in the spatial neighborhood, and the correlation weights between the target point and the neighborhood points are calculated using these attention subgraphs. Based on the correlation weights, the features in the neighborhood are weighted and aggregated to obtain local perception enhancement features. Nonlinear mapping features are fused with local perception enhancement features, and the importance of different features is adaptively adjusted through a gated channel-spatial attention mechanism to obtain enhanced local features, including multi-scale perception gated fusion enhancement features and geometric surface reconstruction enhancement features. The multi-scale perception gated fusion enhancement features and the geometric surface reconstruction enhancement features are respectively subjected to global average pooling to obtain the global representation vector of multi-scale perception gated fusion and the global representation vector of geometric surface reconstruction. The multi-scale perception gated global representation vector and the geometric surface reconstruction global representation vector are concatenated and then fed into two fully connected layers and a nonlinear activation function to obtain the fused enhanced features.

7. A robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 6, characterized in that, Based on deep learning grid models and geometric reconstruction grid models, attention subgraphs based on nearest neighbor retrieval are constructed within the spatial neighborhood. These attention subgraphs are then used to calculate the relevance weights between the target point and its neighboring points, including: For each target point p Search for nearest neighbors within its spatial neighborhood to form a spatial neighborhood point set. Within the spatial neighborhood set, construct an attention subgraph; use the attention subgraph to calculate the relevance weights between the target point and each neighboring point. for: ； In the above formula, d The scaling factor is the feature dimension. For target point p The set of neighborhood points; Indicates the target point p A neighborhood index in the set of neighbors; Indicates the target point p The representation of the feature vector in the projection space, i.e., the query vector; Representing neighborhood points q The eigenvectors are represented in the projection space as key vectors.

8. A robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 1, characterized in that, The fused enhanced features are input into the implicit function decoder to predict the signed distance function value for each vertex, including: The fused enhanced features are then stitched together with the corresponding original 3D point cloud data to form stitched features. The concatenated features are input into an implicit function decoder, which employs a multilayer perceptron structure to perform nonlinear mapping and feature regression on each point, predicting the signed distance function value corresponding to that point point by point.

9. A robust three-dimensional surface reconstruction method based on multi-scale fusion according to claim 1, characterized in that, Also includes: A joint loss function is introduced, which includes SDF regression loss, gradient consistency loss, and marginal constraint loss.

Images