End-to-end field optimization object surface reconstruction method based on Gaussian sputtering guidance

Abstract

The invention discloses an end-to-end field optimization surface reconstruction method based on Gaussian sputtering guidance, and belongs to the field of computer vision and computer graphics. Comprising the following steps: 1) Gaussian guidance field optimization: defining an implicit symbol distance field by using a parameterized MLP network, and initializing and optimizing a surface through a microcosmic contour surface extraction method; gaussian units are bound with the extracted surface triangular patches, the positions of the patches are parameterized through the vertexes of the patches, rotation and scaling are carried out, regularization constraints based on the minimum wrapped ellipse and the distribution entropy are introduced, it is ensured that the Gaussian units are accurately attached to the surface and evenly distributed on the surface, and a result obtained through rendering guides field optimization. And 2) local multi-resolution: identifying surface voxels based on the optimized symbol distance field to carry out adaptive octree subdivision so as to realize multi-resolution improvement. And 3) adopting an inner and outer double-layer cyclic training architecture, optimizing the symbolic distance field and the grid geometric parameters on the outer layer, and optimizing a bound Gaussian unit on the inner layer to realize end-to-end joint optimization of surface geometry and Gaussian rendering.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision and computer graphics, and relates to an end-to-end field optimization method for object surface reconstruction based on Gaussian sputtering guidance. Background Technology

[0002] Multi-view synthesis and geometric reconstruction are highly challenging and crucial tasks in computer vision. In recent years, the rapid development of technologies such as autonomous driving, virtual reality, and digital twins has created an urgent need for high-fidelity, high-efficiency 3D scene reconstruction. Currently, widely applied tasks include: in industrial manufacturing and reverse engineering, using surface reconstruction technology to perform high-precision digital modeling of physical component scan data, quickly generating standardized digital models for process optimization; in medical imaging and precision medicine, this technology can be used for 3D organ reconstruction, lesion localization, and surgical path planning; in cultural heritage protection and digitization, multi-view reconstruction technology can perform high-precision scanning and modeling of ancient buildings and cultural relics, fully preserving their texture details and geometric structural features, providing precise data support for cultural relic restoration work, and also meeting the scanning and reconstruction needs of large cities and underwater scenes from aerial or underwater perspectives.

[0003] However, traditional 3D modeling relies on a two-stage paradigm of Structure from Motion (SFM) and Multi-View Stereo (MVS) or a purely manual modeling process, which has several insurmountable bottlenecks: the SFM-MVS paradigm heavily relies on manually designed feature matching, is not differentiable, and cannot fully utilize the advantages of deep learning, exhibiting extremely poor robustness in sparse viewpoints, textureless regions, or wide baseline scenes; manual modeling not only requires a large investment of manpower and resources from professional artists, but also typically takes weeks or even months, making it difficult to adapt to the rapid modeling needs of dynamic or large-scale scenes. Furthermore, the shortcomings of traditional methods in dynamic target reconstruction and real-time response further limit their application in emerging technology scenarios.

[0004] With the iterative updates of technology, numerous researchers have proposed new reconstruction techniques. In 2020, Stanford University and Google collaborated to introduce the concept of radiation fields for the first time, primarily aiming to solve the problem of 3D scene reconstruction based on 2D image rendering methods. In 2021, Mildenhall et al. proposed the first neural radiance field (Nerf) method based on neural networks and machine learning techniques. This method describes three-dimensional space as an implicit continuous function, achieving the goal of representing the color and density of any point in space. It mainly covers how light interacts with object surfaces, materials, and the surrounding environment, thereby reconstructing the behavior of light in space to reproduce the real world. Although the continuity of this method helps optimize the process, the random sampling required for rendering is costly and prone to noise. In 2023, Kerbl et al. proposed the 3D Gaussian splatting (3DGS) method. This method optimizes explicit 3D Gaussian distribution parameters (position, rotation, scale, and appearance) and combines them with efficient alpha-blend rendering technology, achieving minute-level training time and millisecond-level rendering speed, opening up new possibilities for real-time applications. Although 3DGS excels in rendering efficiency, the inherent irregularity and disorder of the Gaussian distribution makes it difficult to accurately fit real-world object surfaces, thus limiting its applicability in downstream tasks. Extracting high-quality, controllable meshes from millions of discrete Gaussian points remains a challenge. Building on this, some studies, such as SuGaR et al., have proposed methods for extracting surfaces from trained Gaussian scenes. However, these methods generally suffer from problems: surface extraction is a post-processing step and does not support end-to-end training; the generated mesh is decoupled from the Gaussian rendering process and cannot be continuously optimized during training; and limitations imposed by the irregular shape of the Gaussian shape and the viewpoint input result in noise and areas of failed reconstruction.

[0005] Existing patents related to 3D Gaussian sputtering surface reconstruction generally have certain shortcomings. Patents such as "A Method for Large-Scene 3D Reconstruction, Update, and Editing Based on 3DGS" and "A Method for 3D Reconstruction of Panoramic Images Based on 3DGS" employ traditional Gaussian sputtering methods, which cannot reconstruct sparsely populated regions. The surface extraction process in existing patents is a post-processing step, which cannot correct surface errors during training, resulting in significant noise and degenerate triangular patches. Furthermore, existing patents are limited by the uncertainty of Gaussian generation, making it difficult to flexibly control the resolution of the reconstructed surface. Summary of the Invention

[0006] This invention presents an end-to-end field optimization method for object surface reconstruction based on Gaussian sputtering guidance, aiming to solve problems such as difficulty in reconstructing sparse viewpoint regions, inconsistent triangular mesh quality, and high noise levels in existing surface reconstruction techniques. First, a neural network is used to implicitly represent the Signed Distance Field (SDF). Combined with the improved isosurface extraction technique of this invention, surface structure is extracted in real time during model training, providing a refined surface foundation for subsequent optimization. Second, after accurately fitting the Gaussian ellipsoid to the current triangular facet, rendering is performed. Through a "Gaussian-mesh binding mechanism," surface quality information can be fed back in real time using the rendered image, thereby achieving closed-loop continuous optimization of the signed distance field. Furthermore, this invention innovatively designs a Gaussian shape regularization strategy, which effectively suppresses problems such as "exceeding the target surface range" or "uneven spatial distribution" of Gaussian elements during deformation. The core technical design of this invention includes: 1) Employing a neural network to implicitly represent the symbolic distance field and iteratively optimize it, combined with Gaussian sputtering technology and differentiable isosurface extraction methods, which can gradually improve surface noise interference areas and sparse viewpoint missing areas during the optimization process, significantly enhancing the surface reconstruction quality. 2) Using an internal and external loop training method to control the gradient, enabling joint optimization of Gaussian sputtering and the surface. 3) Employing a local multi-resolution computing mechanism, which significantly reduces system computational overhead while enhancing the surface detail representation capability.

[0007] The technical solution adopted in this invention is as follows:

[0008] An end-to-end field optimization method for object surface reconstruction based on Gaussian sputtering guidance includes the following steps:

[0009] Step 1: Obtain multi-source input data of the target object: Use a high-precision camera to perform a surround-sound shooting of the target object to obtain RGB images of the target object from multiple perspectives; Simultaneously obtain the camera intrinsic parameters (including focal length and pixel size) and extrinsic parameters (including pose and world coordinate system coordinates) corresponding to each RGB image; Then obtain the initial depth true value of the target object (collected by high-precision equipment such as a structured light scanner) for subsequent optimization and calibration.

[0010] Step 2: Preprocess the RGB images: Perform denoising and sharpening processing on each RGB image in sequence to obtain multiple preprocessed RGB images; Based on the preprocessed RGB images, generate a binary mask of the target object, and optionally calculate the depth map and normal vector map of the target object.

[0011] Step 3: Initialization of the Symbolic Distance Field and Surface Extraction: A parameterized Multilayer Perceptron (MLP) network is used to define the implicit symbolic distance field. An initial voxel mesh is created according to the resolution, and the 3D coordinates of the voxel mesh vertices are mapped to symbolic distance values ​​and associated high-dimensional features. The voxel creation process and the symbolic distance field SDF calculation process are integrated with an adaptive block-based computation mechanism to optimize memory usage. The training process uses a differentiable isosurface extraction method to extract surfaces from the symbolic distance field as intermediate results. Three key parameters are introduced: mesh cell interpolation weights, mesh cell splitting weights, and mesh vertex deformation vectors. These parameters are used to achieve gradient-based optimization and fit the surface features of the target object.

[0012] Step 4: Binding and Regularization Constraints of Gaussian Elements: Based on the surface extracted in Step 3, Gaussian parameters are defined using the vertex coordinates and topological information of the surface. One or more Gaussian elements are initialized on each triangular facet of the surface, and the position of the Gaussian element is parameterized by the vertex coordinates of the triangular facet. The normal vector of the triangular facet and the vector from the centroid to the vertex are calculated. Gram-Schmidt orthogonalization is performed on the vectors to define a local coordinate system and generate the rotation matrix of the Gaussian element. The scaling factor is compressed along the direction perpendicular to the surface of the triangular facet to flatten the Gaussian ellipsoid into a structure aligned with this direction, and the scaling factor matrix is ​​calculated according to the facet size.

[0013] Specific methods: possess An object with a surface, each triangular facet containing three vertices. and A number of Gaussian units, each with a Gaussian center location Determined by the convex combination of the three vertices:

[0014]

[0015] The formula for calculating the centroid coordinate coefficient is:

[0016]

[0017] in, Represents the coordinates of the vertices of the triangular facet; For the corresponding number vertex The centroid coordinate parameters in the direction control the movement of the Gaussian center position, ensuring that each Gaussian center is strictly located within the convex hull of the triangular facet. for An intermediate form that can be optimized during training. This is a number greater than 0 to prevent the denominator from being zero during calculation. The covariance matrix of each Gaussian is reparameterized based on the geometric information of the triangular facets. ,in, It is a Gaussian rotation matrix. To use a Gaussian scaling matrix, the specific steps are as follows: Calculate the normal vectors of the triangular facets. Centroid to Vertex Vector and put and Gram-Schmidt orthogonalization was performed to obtain This defines the local coordinate system of the triangle, and simultaneously generates the Gaussian rotation matrix: R=[ r 0 , r 1 , r 2 ] The Gaussian ellipsoid is flattened into a plane aligned with this axis by a scale factor perpendicular to the surface direction, and the scaling matrix is ​​calculated based on the patch size. , It is a tiny constant. This is the first tangential scale parameter. For the second tangential scale parameter:

[0018]

[0019] in, It is the centroid of the triangular facet.

[0020] For Gaussians with unreasonable coverage, a Gaussian scaling constraint based on the minimum wrapping ellipse is introduced to force the range of each Gaussian ellipsoid to be within the boundary of the minimum wrapping ellipse of its triangular facet, thus avoiding Gaussian units from exceeding the surface range or being unevenly distributed.

[0021]

[0022] in, and As a local 2D coordinate variable, it represents the x and y coordinates of any point on the Gaussian ellipsoid in the local 2D coordinate system of its triangular patch; and This is the semi-axis length parameter of the minimum enclosing ellipse of the triangular facet, which is the same as the first and second tangential scale parameters. To encourage a more uniform distribution of Gaussian centers on the surface of the triangular facets, a Gaussian distribution constraint based on the distribution parameter is introduced. For the centroid distribution on each triangular facet... ,in , , The distribution regularization loss is defined as follows:

[0023] Boundary avoidance of loss To prevent the Gaussian distribution from being too close to the edge of the triangular facet:

[0024]

[0025] Spacing constraint loss Encourage an increase in the center-to-center spacing of Gaussians:

[0026]

[0027] Among them, minimum distance Target distance .

[0028] Distribution range loss This encourages a more uniform Gaussian distribution.

[0029]

[0030] Final loss The definition is as follows:

[0031]

[0032] Step 5: Use the Gaussian units bound to the mesh in Step 4 for training and rendering to optimize the symbolic distance field. The rendering method involves calculating the superposition effect of all Gaussian units under the current training result at a certain camera viewpoint, ultimately synthesizing a 2D image. Training employs a two-layer loop architecture. The outer loop trains the symbolic distance field and mesh geometry parameters, while the inner loop individually optimizes the Gaussian units bound to the current surface, ensuring that the Gaussian units accurately reflect the surface quality. The final loss obtained from the inner loop is then fed back to the outer loop; the loss function of the inner loop... The Structure Similarity Index Measure (SSIM) is used. This loss function is derived from the RGB information of the real object image obtained in step two and the RGB information of the image rendered during training in this step. The specific calculation method is shown in formula (9). This loss function assists Gaussian in quickly rendering an RGB image that matches the current surface; the loss function of the outer loop... A phased design is adopted. In the early stage of training, a mask loss is used, which is derived by comparing the binary mask information of the real object image obtained in step two with the binary mask information of the image rendered in step six. The specific calculation method is shown in formula (10). In the later stage of training, the structural loss is calculated by comparing the RGB image generated by the inner loop rendering with the real RGB image. This loss is then backpropagated to the mesh geometry parameters to achieve accurate optimization of model details.

[0033]

[0034]

[0035] in The mean absolute error is calculated by summing the absolute differences of all pixels (rendered values ​​- actual values) and then dividing by the total number of pixels. Structural similarity difference loss is an index proposed based on the human eye's perception of image brightness, contrast, and structure, used to measure the similarity between two images. Weights are manually assigned for control. middle The proportion; The total pixel size is determined by the image resolution. and They represent the first, second, and third parts respectively. For each image, the rendered masking information and the actual masking information are shown, with 0 corresponding to the background and 1 corresponding to the foreground target area; Loss can quickly adjust the external contour of the model.

[0036] Step Six: Local Multi-Resolution Implementation. Based on the symbolic distance field optimized in Step Five, voxels containing the surface of the target object are identified. Octree partitioning is performed on this type of voxel and its directly adjacent voxels to complete the object surface reconstruction.

[0037] The beneficial effects of this invention are:

[0038] (1) The present invention adopts an end-to-end optimization architecture based on the symbolic distance field, which enables the model to achieve continuous iterative optimization during training. The numerical value of the field and the grid parameters are dynamically adjusted through gradient feedback, which effectively improves the surface integrity of the three-dimensional reconstruction model. At the same time, it dynamically suppresses surface noise interference, solving the problem that traditional reconstruction technology is difficult to continuously optimize and has significant noise residue.

[0039] (2) This invention ensures the spatial consistency between Gaussian units and surfaces by binding constraints between Gaussian units and surfaces. Combined with the inner and outer layer loop optimization architecture, it achieves the collaborative optimization of structured surface extraction and Gaussian fast rendering, taking into account both surface topology accuracy and rendering feedback efficiency.

[0040] (3) The present invention adopts a local multi-resolution optimization method, which can reduce the overall training computation load on the one hand, and improve the representation accuracy of local surface details on the other hand, while realizing the controllable adjustment of model resolution and the regularization of triangular patches; effectively solving the problem of irregular reconstruction surface caused by the randomness of ellipsoid in traditional Gaussian sputtering, so that the reconstruction results can be adapted to downstream model editing tasks and expand the application scenarios of the technology. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall process of the method of the present invention.

[0042] Figure 2 This is a schematic diagram of the Gaussian scaling constraint proposed in this invention, wherein (a) is a schematic diagram of the error case where the Gaussian ellipsoid exceeds the surface, and (b) is a schematic diagram of the ideal Gaussian size after constraint.

[0043] Figure 3 (a) ~ Figure 3 (b) is a schematic diagram of the Gaussian distribution position constraint proposed in this invention, wherein (a) is a schematic diagram of the error case where the Gaussian positions overlap too much, and (b) is a schematic diagram of the ideal Gaussian distribution after constraint.

[0044] Figure 4 This is a schematic diagram of voxel partitioning according to the present invention, wherein (a) and (b) are schematic diagrams before and after voxel partitioning, respectively.

[0045] Figure 5 This is a schematic diagram of the local resolution enhancement described in this invention, where (a) and (b) are schematic diagrams before and after the resolution enhancement, respectively. The resolution enhancement causes the symbol distance field density to increase, and the points in the figure are a visual representation of the symbol distance field density.

[0046] Figure 6 The figures show a comparative ablation experiments of the Gaussian constraint module proposed in this invention, where (a) represents the results using the Gaussian constraint method. Figure 6 (b) represents the unused results.

[0047] Figures 7-9 This is a comparison chart of the results of this invention on the selected dataset in the experiment, wherein, Figure 7 The reconstruction quality of sparse parts of the viewpoint was compared with existing methods. Figure 8 The surface noise was compared with existing methods. Figure 9 The rendering of model details of the present invention at different resolutions was compared. Detailed Implementation

[0048] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0049] like Figure 1 As shown, the present invention provides an end-to-end field optimization method for object surface reconstruction based on Gaussian sputtering guidance, comprising the following steps:

[0050] Step 1: Acquire multi-source input data of the target object: Use a high-precision camera to perform surround shooting on the target object to acquire RGB images of the target object from multiple perspectives; simultaneously acquire the camera intrinsic and extrinsic parameters corresponding to each RGB image; when hardware conditions permit, additionally acquire the initial ground truth depth of the target object. In this embodiment, the commonly used 3D reconstruction datasets DTU and Nerf_Synthetic dataset are used. The former contains 128 scenes in a controlled laboratory environment, and its models are captured using a structured light scanner, containing 49 or 64 camera positions, RGB images with a resolution of 1200×1600 pixels, and corresponding camera parameters and binary mask images for the poses; the latter models are all generated by Blender rendering, with clean backgrounds and accurate camera parameters, including eight scenes, each containing 100 training images, 100 validation images, and 200 test images, as well as corresponding pose transformation matrices, with an image resolution of 800×800.

[0051] Step 2: Preprocess the object image by removing blurry images, calculating a binary mask, and calculating the depth map and normal map. Different data preprocessing methods are developed for different dataset formats.

[0052] Step 3: Estimate the 3D model size of the target object using 3D software such as Meshlab to obtain the bounding box coordinates. Divide the model into a uniform voxel mesh in 3D space, input the vertices of the voxel mesh into a neural network, and set the network block calculation size according to the model size. The neural network implicitly represents the numerical distribution of the signed distance field in space. Extract the surface using a differentiable surface extraction method.

[0053] Step 4: Bind a fixed number of Gaussian elements (usually between 1 and 100) to each surface according to accuracy requirements. Based on the surfaces extracted in Step 3, use Gaussian rendering to provide feedback on the current surface reconstruction quality. Introduce Gaussian scaling constraints and Gaussian distribution constraints to improve the quality of the Gaussian elements.

[0054] Step 5: Use the Gaussian units bound to the mesh in Step 4 for training to optimize the signed distance field. Training employs a two-layer loop architecture. When starting the outer training layer, Gaussian units and the rendering scene are initialized. Upon entering the inner training layer, gradient propagation is cut off, and a dedicated rendering scene is created. After the inner Gaussian units complete training, their parameters are copied to the outer Gaussian units. Then, rendering operations are performed through the outer rendering scene to obtain RGB images and mask images, which are compared with the real images to calculate the corresponding losses. After the entire training process is completed, the surface model of the target object, the corresponding rendered image of the model, and the Gaussian sputtering scene are output. The structural loss is calculated using the RGB image, and the masking loss is calculated using the mask image.

[0055]

[0056]

[0057] Step Six: Based on the symbolic distance field optimized in Step Five, identify voxels containing the surface of the target object in 3D space. Use an octree partitioning method to divide these surface-containing voxels into 8 sub-voxels, ensuring each sub-voxel inherits the mesh geometry parameters of the original voxel, such as mesh cell interpolation weights and splitting weights. To ensure the accuracy of the dual vertex calculation results, perform the same octree partitioning operation on voxels directly adjacent to the surface-containing voxels. In the initial stage of model training, iterative optimization is performed based on a fixed low-resolution voxel mesh. Later in training, the resolution of the surface-containing voxel region is gradually increased through octree partitioning to optimize more local detail features of the target object. Simultaneously, it supports receiving input target object bounding box range parameters and performing a coarse local resolution enhancement operation based on the bounding box range. This reduces the number of voxel partitioning operations in unnecessary regions during the initial training stage, thus reducing system computational overhead.

[0058] The Gaussian constraint proposed in step four of this method is as follows: Figures 2-3 As shown.

[0059] (1) Gaussian scaling constraint mainly targets the excessive Gaussian distribution beyond the surface to prevent the Gaussian distribution from exceeding the boundary of the triangular facet and generating false surface artifacts in the vacuum region, making it difficult to fill the missing surface region.

[0060] (2) Gaussian distribution constraint encourages Gaussian uniformity to cover the entire surface. This avoids excessive concentration of distribution, which can lead to surface sampling sparsity, resulting in holes in the rendering and disrupting geometric continuity.

[0061] Step six of this method is a local multi-resolution method, specifically as follows: Figures 4-5 As shown.

[0062] Based on the sign change of the SDF value, voxels containing the surface of the target object are selected. The surface voxels and their adjacent voxels are then subdivided to achieve local resolution improvement and make the SDF distribution denser.

[0063] The specific implementation steps are as follows:

[0064] Experimental setup: Ubuntu 22.04 operating system, NVIDIA GeForce RTX 3090 GPU. Images at 1200×1600 resolution from the DTU dataset and 800×800 resolution from the Nerf_Synthetic dataset were used as input. Stochastic gradient descent with 0.937 momentum was employed for optimization. The batch size used during training was 16, the initial learning rate for the grid parameters was 0.01, and the initial learning rate for SDF was set to 0.0001 to 0.00005 depending on the model size. The learning rate was dynamically adjusted using the `lr_schedule` function (max(0.0, 10^(- iterations × 0.00002)))) based on the LambdaLR scheduler.

[0065] The symbolic distance field neural network has an input dimension of 3 and an output dimension of 257 (256-dimensional features + 1-dimensional SDF value), containing 10 hidden layers (hidden dimension 256). A skip connection is set in the 4th layer, and a 6-level multi-resolution coordinate encoding is used. The network adopts a geometrically aware initialization and weight normalization strategy, and enables a gradient checkpointing mechanism to optimize memory usage. It is deployed on a specified computing device.

[0066] Training process:

[0067] Input the resolution parameters, create a voxel mesh within the object's bounding box, normalize it, input it into the SDF network, calculate the symbolic distance function value for each vertex, and form the initial SDF field to be optimized.

[0068] Initialize the mesh deformation parameters and segmentation weight parameters, create an optimizer, and input the obtained SDF values ​​into the differentiable surface extraction module to obtain the process surface results.

[0069] Initialize Gaussian cells, create a scene using surface vertices and Gaussians, and calculate the position, rotation, scaling, opacity, color, and other properties of the Gaussians bound to the mesh.

[0070] The initial mask image is obtained by rendering the scene and compared with the real mask image to calculate the mask loss; based on the loss, the SDF field and mesh parameters are optimized, and the surface rough outer contour is quickly formed.

[0071] Initiate internal loop optimization, temporarily freeze SDF gradient propagation, and create inner training-specific Gaussian units and scenes. After the inner Gaussian units are trained, copy the parameters used for rendering to the outer Gaussian units to render the current surface RGB image.

[0072] Return to the outer loop, calculate the SSIM loss by comparing the current surface RGB image result with the real image RGB, and optimize the SDF and mesh parameters to obtain a detailed surface.

[0073] The final training process yields a high-quality surface, a rendered image, and a Gaussian sputtering scene corresponding to the final model.

[0074] Evaluation process:

[0075] The following steps are used to calculate the Chamfer Distance index, which measures the difference between the model and the true model.

[0076] Obtain the surface mesh data to be evaluated, scene identifiers, an instance catalog containing scene images and masks, and a reference dataset containing observation masks, bounding boxes, resolution, and standard point clouds.

[0077] Read the zoom matrix and world matrix of each viewpoint from the camera parameter file in the instance directory, and parse the camera intrinsic and extrinsic parameters through matrix operations.

[0078] Read the mask images from the mask folder in the instance directory for each viewpoint, and use them to identify the visible area of ​​the surface from the corresponding viewpoint.

[0079] If the input is a clipped mesh, the mesh triangles are sampled uniformly (the sampling density is controlled by the downsampling threshold), and the sampled points are merged with the mesh vertices to form an initial point cloud; if the input is a point cloud, the data is loaded directly.

[0080] The KD tree algorithm is used to downsample the initial point cloud, and points with a spacing greater than the downsampling threshold are retained to remove redundancy and improve efficiency.

[0081] Load the observation mask, bounding box, and resolution of the reference dataset, and filter out the point cloud located within the bounding box extension region (extension size is patch_size); convert it to the observation mask grid coordinate system, and retain the points located within the valid region as valid point clouds to be evaluated.

[0082] Load the reference dataset standard point cloud, calculate the nearest neighbor Euclidean distance from the valid point cloud to be evaluated to the standard point cloud, and count the average distance less than the maximum threshold; based on the ground plane parameters of the reference dataset, filter the points above the ground plane in the standard point cloud, calculate the nearest neighbor Euclidean distance from these points to the valid point cloud to be evaluated, and count the average distance less than the maximum threshold; calculate the bidirectional average error.

[0083] Experimental results:

[0084] At a resolution of 128×128×128:

[0085] Figure 6The left side shows a smooth, noise-free surface after Gaussian constraint, while the right side shows pits and holes on the surface without Gaussian constraint. Figure 7 This invention demonstrates that the method can, to some extent, compensate for the problem of other methods failing to reconstruct sparse areas such as the back and base. Figure 8 This invention demonstrates that it can reduce surface noise caused by Gaussian irregularities in other methods. Figure 9 The rendering of model details of the present invention at different resolutions was compared.

[0086] The comparison results at a resolution of 256×256×256 are shown in Table 1. As can be seen from the data in Table 1, this method can improve the surface quality compared with the original Gaussian sputtering surface reconstruction method.

[0087] Table 1. Comparison of Chamfer Distance between the original Gaussian sputtering method and the method of this invention.

[0088]

Claims

1. A method for reconstructing the surface of an object based on end-to-end field optimization guided by Gaussian sputtering, characterized in that, Includes the following steps: Step 1: Obtain multi-source input data of the target object: Use a high-precision camera to perform surround shooting on the target object to obtain RGB images of the target object from multiple perspectives; simultaneously obtain the camera intrinsic and extrinsic parameters corresponding to each RGB image; then obtain the initial depth true value of the target object for subsequent optimization and calibration; Step 2: Preprocess the RGB images: Perform denoising and sharpening processing on each RGB image in sequence to obtain multiple preprocessed RGB images; Based on the preprocessed RGB image, a binary mask of the target object is generated; Step 3: Initialize the symbolic distance field and extract the surface: Use a parameterized multilayer perceptron (MLP) network to define the implicit symbolic distance field, create an initial voxel grid according to the resolution, and map the three-dimensional coordinates of the voxel grid vertices to symbolic distance values ​​and associated high-dimensional features; The voxel creation process and the symbolic distance field SDF calculation process are integrated with an adaptive block calculation mechanism to optimize video memory usage; The training process employs a differentiable isosurface extraction method to extract the surface as an intermediate result from the symbolic distance field. Three key parameters are introduced: grid cell interpolation weight, grid cell splitting weight, and grid vertex deformation vector. These parameters are used to achieve gradient-based optimization and fit the surface features of the target object. Step 4: Binding and Regularization Constraints of Gaussian Elements: Based on the surface extracted in Step 3, Gaussian parameters are defined using the vertex coordinates and topological information of the surface. One or more Gaussian elements are initialized on each triangular facet of the surface, and the position of the Gaussian element is parameterized by the vertex coordinates of the triangular facet. The normal vector of the triangular facet and the vector from the centroid to the vertex are calculated. Gram-Schmidt orthogonalization is performed on the vectors to define a local coordinate system and generate the rotation matrix of the Gaussian element. The scale factor is compressed along the direction perpendicular to the surface of the triangular facet to flatten the Gaussian ellipsoid into a structure aligned with this direction, and the scaling factor matrix is ​​calculated according to the facet size. Step 5: Use the Gaussian units bound to the mesh in Step 4 for training and rendering to optimize the symbolic distance field; the rendering method is to calculate the superposition effect of all Gaussian units under the current training result at a certain camera view, and finally synthesize a 2D image. The training adopts a two-layer loop architecture. The outer loop is used to train the symbolic distance field and mesh geometry parameters, while the inner loop optimizes the Gaussian unit bound to the current surface to ensure that the Gaussian unit can accurately reflect the surface quality. The final loss obtained from the inner training is then fed back to the outer loop. Step 6: Local multi-resolution implementation; Based on the symbolic distance field optimized in Step 5, voxels containing the surface of the target object are identified, and octree partitioning is performed on this type of voxel and its directly adjacent voxels to complete the reconstruction of the object surface.

2. The method for reconstructing an object surface based on Gaussian sputtering guidance through end-to-end field optimization according to claim 1, characterized in that, Step four is as follows: have An object with a surface, each triangular facet containing three vertices. and A number of Gaussian units, each with a Gaussian center location Determined by the convex combination of the three vertices: ； The formula for calculating the centroid coordinate coefficient is: ； in, Represents the coordinates of the vertices of the triangular facet; For the corresponding number vertex The centroid coordinate parameters in the direction control the movement of the Gaussian center position, ensuring that each Gaussian center is strictly located within the convex hull of the triangular facet. for An intermediate form that can be optimized during training. The value is greater than 0 to prevent the denominator from being zero during calculation; the covariance matrix of each Gaussian is reparameterized based on the geometric information of the triangular facets. ,in, It is a Gaussian rotation matrix. To use a Gaussian scaling matrix, the specific steps are as follows: Calculate the normal vectors of the triangular facets. Centroid to Vertex Vector and put and Gram-Schmidt orthogonalization was performed to obtain This defines the local coordinate system of the triangle, and simultaneously generates the Gaussian rotation matrix: R=[ r 0 , r 1 , r 2 ] The Gaussian ellipsoid is flattened into a plane aligned with the axis by a scale factor perpendicular to the surface direction, and the scaling matrix is ​​calculated based on the patch size. , It is a tiny constant. This is the first tangential scale parameter. For the second tangential scale parameter: ； in, It is the centroid of the triangular facet; For Gaussians with unreasonable coverage, a Gaussian scaling constraint based on the minimum wrapping ellipse is introduced to force the range of each Gaussian ellipsoid to be within the boundary of the minimum wrapping ellipse of its triangular facet, thus avoiding Gaussian units from exceeding the surface range or being unevenly distributed. ； in, and As a local 2D coordinate variable, it represents the x and y coordinates of any point on the Gaussian ellipsoid in the local 2D coordinate system of its triangular patch; and The semi-axis length parameter of the minimum enclosing ellipse of the triangular facet is the same as the first and second tangential scale parameters; to encourage a more uniform distribution of Gaussian centers on the surface of the triangular facet, a Gaussian distribution constraint based on the distribution parameter is introduced; for the centroid distribution on each triangular facet... ,in , , The distribution regularization loss is defined as follows: Boundary avoidance of loss To prevent the Gaussian distribution from being too close to the edge of the triangular facet: ； Spacing constraint loss Encourage an increase in the center-to-center spacing of Gaussians: ； Among them, minimum distance Target distance ; Distribution range loss This encourages a more uniform Gaussian distribution. ； Final loss The definition is as follows:

3. The method for reconstructing an object surface based on Gaussian sputtering guidance through end-to-end field optimization according to claim 1, characterized in that, In step five, the loss function of the inner loop The Structural Similarity Index (SSIM) is used. This loss function is derived from the RGB information of the real object image obtained in step two and the RGB information of the image rendered in this training step. The specific calculation method is shown in formula (9). This loss function assists Gaussian in quickly rendering an RGB image that matches the current surface. The loss function of the outer loop... A phased design is adopted. In the early stage of training, a mask loss is used. This loss function is derived by comparing the binary mask information of the real object image obtained in step two with the binary mask information of the image rendered in step six during training. The specific calculation method is shown in formula (10); in the later stage of training, the RGB image generated by the inner loop rendering and the real RGB image are used to calculate the structural loss, and the loss is back-propagated to the mesh geometry parameters to achieve accurate optimization of model details; ； ； in The mean absolute error is calculated by summing the absolute differences of all pixels and then dividing by the total number of pixels. Structural similarity difference loss is a metric used to measure the similarity between two images. Weights are manually assigned for control. middle The proportion; The total pixel size is determined by the image resolution. and They represent the first, second, and third parts respectively. Each image contains rendered masking information and actual masking information; 0 corresponds to the background, and 1 corresponds to the foreground target area.

Images