Multi-view reconstruction method based on three-dimensional Gaussian splash and sign distance function

Abstract

The invention relates to the technical field of three-dimensional reconstruction and the like, in particular to a multi-view reconstruction method based on three-dimensional Gaussian splashing and a signed distance function, which comprises the following steps: initializing a three-dimensional Gaussian set according to a multi-view image and internal and external parameters of a camera thereof, and adding a signed distance for each three-dimensional Gaussian atom as a Gaussian parameter; performing micro-Gaussian rasterization at any training view angle; whether densification and pruning are needed or not is judged by taking the average gradient of the three-dimensional Gaussian atoms as a density control factor, and a densification strategy and a pruning strategy are executed on the three-dimensional Gaussian set according to the size and transparency of the three-dimensional Gaussian atoms; obtaining indexes and contribution values of each pixel and the first K three-dimensional Gaussian atoms with the highest contribution, and performing weighted aggregation on the Gaussian-level learnable signed distance to obtain a pixel-level signed distance; during color rendering, the pixel-level signed distance participates in color synthesis together, and a reconstructed image is obtained through calculation. According to the invention, the precision of the reconstructed object is higher.

Description

Technical Field

[0001] This invention relates to the fields of three-dimensional reconstruction and other technologies, and in particular to a multi-view reconstruction method based on three-dimensional Gaussian splashing and a signed distance function. Background Technology

[0002] Multiview Figure 3 The field of 3D reconstruction is mainly divided into two parts: implicit neural representation and explicit representation. In implicit representation, methods represented by NeRF utilize differentiable volume rendering and MLP networks to map 3D position and viewpoint to color and volume density. Views are generated by projecting the implicit representation and trained using real image loss. Subsequent work has also seen methods integrating SDF into volume rendering for surface reconstruction, followed by mesh extraction using methods such as MarchingCubes. In explicit representation, 3D Gaussian Splatting (3DGS) uses 3D anisotropic Gaussian ellipses as basic primitives. After projection onto the screen, transparency is used to quickly synthesize images with new perspectives. Subsequent work mainly focuses on optimizing the Gaussian ellipsoid itself and surface reconstruction, such as replacing the original 3D Gaussian ellipsoid with a 2D Gaussian plane for training. In recent years, there has been a clear trend of combining SDF and explicit Gaussian in implicit methods, such as adding 3DGS as a guide in implicit training, or adding SDF implicit training to 3DGS for surface guidance from a rendering quality perspective.

[0003] However, existing 3D reconstruction methods do not fully integrate SDF and 3DGS. They basically use a two-branch training method for joint training, which requires additional resources to handle the consistency of the two branches during training and cannot guarantee the balance of the two-branch effects.

[0004] As can be seen from the above analysis, the existing methods, due to their inability to fully combine implicit SDF and explicit Gaussian, all exhibit a certain degree of inconsistency or imbalance between reconstruction accuracy and rendering results during the training process. That is, there are situations where reconstruction accuracy is high but rendering results are poor, or rendering results are good but reconstruction accuracy is low. Summary of the Invention

[0005] To address the problem of the ineffective combination of implicit SDF and explicit Gaussian in existing technologies, and in order to effectively balance reconstruction accuracy and rendering results, this invention proposes a multi-view reconstruction method based on 3D Gaussian splashing and the sign distance function, specifically including the following steps:

[0006] A three-dimensional Gaussian set is initialized based on the multi-view image and its camera intrinsic and extrinsic parameters. A signed distance is added to each three-dimensional Gaussian atom as a Gaussian parameter. This distance is used to represent the relative proximity of the three-dimensional Gaussian atom to the implicit surface. The smaller the absolute value of this distance, the closer the current three-dimensional Gaussian atom is to the implicit surface.

[0007] Perform differentiable Gaussian rasterization at any training viewpoint to obtain the color and opacity of each three-dimensional Gaussian atom;

[0008] The average gradient of the three-dimensional Gaussian atoms is used as a density control factor to determine whether densification and pruning are needed. Densification and pruning strategies are applied to the three-dimensional Gaussian set based on the size and transparency of the three-dimensional Gaussian atoms.

[0009] Obtain the index and contribution value of each pixel and its K highest-contributing 3D Gaussian atoms, and then perform weighted aggregation of the Gaussian-level learnable signed distances to obtain pixel-level signed distances.

[0010] During color rendering, pixel-level signed distances are also involved in color synthesis to calculate the reconstructed image.

[0011] Furthermore, when the density control factor of the i-th 3D Gaussian atom is greater than a set threshold, the densification and pruning strategies are determined. The densification strategy determines the scale vector; if the scaling scale is less than the set threshold, a copy of the 3D Gaussian atom is made at its center position. Otherwise, the direction of the zero horizontal plane is estimated using the signed distance distribution of the neighborhood Gaussians, and the center position of the Gaussian atom is moved in this direction. | / 2, and at the symmetrical position, that is, the position with a signed distance of 0 as the symmetrical point, a new three-dimensional Gaussian atom is generated at the position opposite to the current three-dimensional Gaussian atom. Then the scale vector of the two three-dimensional Gaussian atoms is changed to the original 0.75. The pruning strategy determines that if the average opacity of the three-dimensional Gaussian atom is greater than the set threshold, then the redundant three-dimensional Gaussian atom is removed. Let be the signed distance of the i-th three-dimensional Gaussian atom.

[0012] Furthermore, the density control factor is updated based on the signed distance. The update process includes:

[0013]

[0014] in, The density control factor is updated based on the signed distance; The average gradient of a three-dimensional Gaussian atom; Weights for signed distance in density control; Let be the proximity function of the signed distance to the i-th three-dimensional Gaussian atom, denoted as , This is a hyperparameter.

[0015] Furthermore, when implementing the pruning strategy, the average opacity of the three-dimensional Gaussian atoms is updated based on the signed distance. The update process includes:

[0016]

[0017] in, The average opacity of three-dimensional Gaussian atoms updated based on signed distance; The weights of signed distance in the pruning strategy.

[0018] Compared to existing technologies, to ensure the effective participation of implicit SDF in Gaussian training, this invention first embeds SDF scalar values ​​into the 3D Gaussian representation, forming a new 3D Gaussian form that enables the 3D Gaussian to acquire geometric awareness during training. Secondly, this invention uses the implicit SDF as a reference standard in the density control strategy during Gaussian training, modifying the original threshold evaluation criteria to make the movement and density changes of the Gaussian during training geometrically influenced. Finally, this invention proposes an SDF-based rendering enhancement method, returning the highly involved Gaussian in the rasterization process as a mapping matrix and adding the corresponding Gaussian SDF weights to perform geometrically aware color correction during image rendering. Compared to more successful 3D reconstruction methods in recent years, the method proposed in this invention achieves higher accuracy in reconstructed objects, improving both objective evaluation metrics and subjective visual effects. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the training process for a multi-view reconstruction method based on three-dimensional Gaussian splashing and symbolic distance function according to the present invention;

[0020] Figure 2 This is a schematic diagram of the splitting based on Gaussian shift of SDF according to the present invention;

[0021] Figure 3 This is a comparison of visual effects on the TNT dataset of this invention. From bottom to top, each row represents the ground truth (GT), the reconstruction results of the implicit method NeuS, the reconstruction results of the explicit method 2DGS, the reconstruction results of SuGar, the reconstruction results of VCR-Gaus, and the reconstruction results of this invention. From left to right, the reconstruction objects are the garage (Barn), the truck (truck), and the bulldozer (caterpillar). Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] This invention proposes a multi-view reconstruction method based on three-dimensional Gaussian splashing and the sign distance function, specifically including the following steps:

[0024] A three-dimensional Gaussian set is initialized based on the multi-view image and its camera intrinsic and extrinsic parameters. A signed distance is added to each three-dimensional Gaussian atom as a Gaussian parameter. This distance is used to represent the relative proximity of the three-dimensional Gaussian atom to the implicit surface. The smaller the absolute value of this distance, the closer the current three-dimensional Gaussian atom is to the implicit surface.

[0025] Perform differentiable Gaussian rasterization at any training viewpoint to obtain the color and opacity of each three-dimensional Gaussian atom;

[0026] The average gradient of the three-dimensional Gaussian atoms is used as a density control factor to determine whether densification and pruning are needed. Densification and pruning strategies are applied to the three-dimensional Gaussian set based on the size and transparency of the three-dimensional Gaussian atoms.

[0027] Obtain the index and contribution value of each pixel and its K highest-contributing 3D Gaussian atoms, and then perform weighted aggregation of the Gaussian-level learnable signed distances to obtain pixel-level signed distances.

[0028] During color rendering, pixel-level signed distances are also involved in color synthesis to calculate the reconstructed image.

[0029] This invention employs a three-dimensional Gaussian field to effectively indicate near-surface geometry under multi-view conditions, thereby improving rendering stability. By introducing a learnable signed distance (SDF) scalar into the three-dimensional Gaussian ellipsoid and implementing pixel-to-Gaussian mapping for pixel-domain projection and weighting of the SDF, cross-view consistency and noise resistance are improved without altering the original renderer structure. The specific process is as follows:

[0030] First, a three-dimensional Gaussian set is initialized based on the multi-view images and their camera intrinsic and extrinsic parameters. In addition to conventional parameters such as center position, scale, rotation, and opacity, an SDF scalar is added to each Gaussian to characterize its relative proximity to the implicit surface (the smaller the absolute value, the closer it is).

[0031] Subsequently, differentiable Gaussian rasterization is performed under any training viewpoint to obtain results such as color and opacity. At the same time, the index and weight of each pixel and its top K major contributing Gaussians are recorded. Based on this, the Gaussian-level SDF is weighted and aggregated according to the local contribution of pixels to obtain the pixel-level SDF map.

[0032] Based on SDFMap, a monotonically adjustable weight function is constructed and applied as a multiplicative factor to the raster color to achieve a steady-state enhancement of rendering where "the contribution is greater in areas close to the surface and less in areas far from the surface".

[0033] During the training phase, real images are used as supervision to minimize the difference between the raster reconstruction map and the robust weighted map relative to the ground truth. At the same time, at the structural level, SDF absolute value and visibility statistics are combined, and density control is performed by setting a threshold rule for annealing during training. Under the combined effect of the above supervision and scheduling, SDF, visibility and Gaussian structure converge end-to-end, and finally output a Gaussian field representation containing SDF and rendering results.

[0034] like Figure 1 The input image data is initialized into a set of three-dimensional Gaussian atoms using a three-dimensional Gaussian sputtering technique. Each three-dimensional Gaussian atom in this invention has a signed distance parameter embedded in its parameters. Differentiable Gaussian rasterization is performed on the three-dimensional Gaussian atoms, and it is determined whether each three-dimensional Gaussian atom needs a densification strategy or a pruning strategy based on the embedded signed distance parameter. Then, the three-dimensional Gaussian atoms are mapped to the pixel domain through differentiable Gaussian rasterization to obtain the rendering result.

[0035] This invention directly embeds a learnable SDF scalar into a three-dimensional Gaussian ellipsoid, thereby enabling the explicit Gaussian ellipsoid to express the degree of closeness to an implicit surface. In this embodiment, the i-th three-dimensional Gaussian atom is defined as:

[0036]

[0037] in, Represents the i-th three-dimensional Gaussian atom; It is the three-dimensional center position of the i-th three-dimensional Gaussian atom. Representing three-dimensional real space, This means that the value space of the three-dimensional center position of the i-th three-dimensional Gaussian atom belongs to a three-dimensional real number space; It is the three-dimensional anisotropic covariance of the i-th three-dimensional Gaussian atom, derived from the scale vector of the i-th three-dimensional Gaussian atom. The rotation matrix of the i-th three-dimensional Gaussian atom Parameterization; It is the opacity of the i-th three-dimensional Gaussian atom constrained by the Sigmoid; Let SDF be the Scalar value of the i-th 3D Gaussian atom. The SDF scalar represents the shortest distance from a point in 3D space to the surface of an object. The sign is positive when the point is outside the object and negative when it is inside. This value is a new addition in this invention. The smaller its absolute value, the closer the Gaussian atom is to the implicit surface. The implicit surface is composed of points with an SDF value of 0, representing the ideal geometry of the object. However, this surface does not actually exist in the model. The SDF parameter added to the 3D Gaussian atom is to allow the atom to perceive the implicit surface and approach it during training. During training, SDF, along with position, scale, rotation, and opacity, participates in the forward and backward propagation generated by differentiable rasterization. Simultaneously, SDF, as a learnable quantity, is stably projected onto the pixel domain by the pixel-to-Gaussian mapping matrix to construct the pixel-level SDF graph and subsequent occupancy-weighted calculation.

[0038] This invention utilizes the newly added SDF attribute to enhance the geometric perception capability of the 3D Gaussian ellipsoid and proposes an SDF-based density control strategy. In existing density control strategies, the densification aspect typically involves using the Gaussian average gradient (…). The density control strategy proposed in this invention uses two reference standards, namely, the SDF value and the Gaussian scaling scale, as the trigger for density control, and then uses the major axis direction of the ellipsoid as the default direction of movement. For pruning, only the Gaussian transparency is used as the trigger for pruning. The SDF-based density control strategy incorporates the SDF value into the reference standards: on the one hand, the degree of proximity to the zero level set is used as the trigger condition, and on the other hand, the gradient direction of the SDF is used as the direction of movement. The specific SDF-based density control strategy includes the following:

[0039] 101. For the densification strategy, this invention is based on the current i-th three-dimensional Gaussian atom. SDF value A function that approximates this approximation is calculated:

[0040] ,

[0041] in, Let be a function of the proximity of the i-th three-dimensional Gaussian atom. It is a hyperparameter; the new density control factor is updated according to the proximity function as follows:

[0042]

[0043] in, This is a new density control factor in this invention; The average gradient of all three-dimensional Gaussian atoms; In this embodiment, the weighting factor for SDF is set to 0.75;

[0044] use To replace the original As a reference standard, if and only if When the value exceeds a preset threshold, densification is triggered; once densification is triggered, the scale vector of the i-th 3D Gaussian atom is used as the basis for the process. Determine whether to perform replication or splitting: when the scale vector of the i-th 3D Gaussian atom... When the value is less than a preset threshold, the scale vector of the i-th three-dimensional Gaussian atom central position Copy the Gaussian; otherwise, trigger a split, i.e., first estimate the direction of the zero horizontal plane using the SDF distribution of the neighboring Gaussians, and then move the center position of the Gaussian in this direction. | / 2, and generate a new Gaussian at the symmetrical position, i.e., the position where the SDF sign is opposite, and then change the scaling scale of the two Gaussians to the original 0.75 (e.g. Figure 2 As shown in the figure, this invention adds SDF information to determine how each atom should operate, so that density control has geometric information control; in the moving part, guided by SDF, the split atoms are directly brought closer to the surface of the object, avoiding invalid splitting that may be caused by the original random movement.

[0045] The direction of the zero horizontal plane is estimated using the SDF distribution of the neighborhood Gaussians. Specifically, based on the position of the Gaussian to be split, all Gaussians within a preset radius are selected according to the Euclidean distance in three-dimensional space. If there is a set of neighborhood Gaussians with SDF values ​​opposite to those of the Gaussian to be split, the intersection point with zero SDF is estimated by linear interpolation on the line connecting the split target to the center of this part of Gaussians. The direction with the fastest change in SDF value to 0 is selected as the direction of the zero horizontal plane. If all SDF values ​​of the neighborhood Gaussians have the same sign, the rate of change of SDF per unit length from the center of the split target to the center of each Gaussian is calculated. The direction with the largest rate of change that reduces the absolute value of the SDF value is selected as the direction of the zero horizontal plane.

[0046] 102. Regarding the pruning strategy, this invention modifies the opacity setting to:

[0047]

[0048] in, The triggering factor for the pruning strategy is the average opacity of the three-dimensional Gaussian atom. The weight of the signed distance in the pruning strategy is set to 0.5 in this embodiment.

[0049] This invention effectively improves the consistency of geometry and appearance by adaptively focusing the newly added Gaussian onto the object surface without affecting the original rendering backbone.

[0050] In Gaussian differentiable rasterization, the color of each pixel p is obtained by superimposing a ray along the viewing direction with a set of ordered 3D Gaussian atoms. For the 3D Gaussian atom hit by the ray, the contribution of the i-th 3D Gaussian atom to the pixel is:

[0051]

[0052] in, Let be the contribution of the i-th 3D Gaussian atom to pixel p; Let be the opacity of the i-th three-dimensional Gaussian atom; Let be the cumulative transmittance of the ray before it reaches the i-th three-dimensional Gaussian atom, with the observation direction as the starting point of the ray.

[0053] When the renderer performs color overlay, this invention records the IDs and contribution values ​​of the top K largest contributing 3D Gaussian atoms for each pixel from the current viewpoint, thereby obtaining a pixel-level ID-contribution value mapping matrix. The mapping matrices are all three-dimensional matrices H*W*K, where H and W represent the height and width of the corresponding pixel image, and K represents the top K Gaussian atoms in terms of total contribution (i.e., weights), sorted from largest to smallest. Based on the mapping matrix, this invention can obtain the SDF image corresponding to the current viewpoint. During color rendering, the SDF weights are included in color synthesis: for the i-th three-dimensional Gaussian atom in each pixel p, its color contribution is changed from the original... Change to Therefore, the color of each pixel in the rendered image can be obtained as follows:

[0054]

[0055] in, To render the pixel value of pixel p in the image; The contribution value of the i-th three-dimensional Gaussian atom among the K three-dimensional Gaussian atoms that contribute the most to pixel p; The signed distance weight of the i-th 3D Gaussian atom among the K 3D Gaussian atoms that contribute the most to the i-th pair of pixels p is denoted as: , Let be the signed distance of the i-th three-dimensional Gaussian atom. Let i be the color of the i-th three-dimensional Gaussian atom.

[0056] This invention compares the obtained rendered image with the real image to obtain a loss function. The loss function is then used to train the forward and backward propagation parameters (including signed distance, center position, scale factor, rotation factor, and opacity) of the three-dimensional Gaussian atoms involved in differentiable Gaussian rasterization by minimizing the loss function. The loss function is expressed as:

[0057]

[0058] in, The loss function; , As a weighting factor; To render the image; It is a real image; This indicates the calculation of the L1 norm; Indicates the computational rendering of the image With real images The similarity.

[0059] To verify the effectiveness of the method proposed in this invention, this embodiment evaluates the performance of the method on the TNT (Tanks and Temples) dataset and compares it with some of the most representative 3D reconstruction methods in recent years, comparing them from multiple aspects such as subjective visual effects and objective evaluation indicators, and achieves very good results.

[0060] To verify the superiority of the proposed method, the reconstructed surface was compared with the following four 3D reconstruction methods. Table 1 shows the F1 score of the reconstructed surface: First, based on a given distance threshold, the point-pair matching relationship between the predicted surface and the real surface is converted into a binary hit / miss judgment, then the precision and recall of the predicted surface are calculated, and finally the sum of the two is used as the overall reconstruction quality. As shown in the table, it can be seen that the proposed method achieves better results in most scenarios on this dataset. Figure 3 This involves a qualitative comparison of the visual reconstruction of objects using different methods. For the Truck dataset, it's clear that other methods cannot effectively separate all the barriers, while for the Caterpillar dataset, other methods are almost unable to clearly reconstruct the nameplates on the vehicles.

[0061] Table 1. F1 scores of reconstructed surfaces and their comparison

[0062] Neus SuGaR 2DGS Ours Barn 0.29 0.14 0.36 0.65 Caterpillar 0.29 0.16 0.23 0.31 Courthouse 0.17 0.08 0.13 0.14 Ignatius 0.83 0.33 0.44 0.67 Meetingroom 0.24 0.15 0.16 0.14 Truck 0.45 0.26 0.26 0.55 Mean 0.38 0.19 0.26 0.41

[0063] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function, characterized in that, Specifically, the following steps are included: A three-dimensional Gaussian set is initialized based on the multi-view image and its camera intrinsic and extrinsic parameters. A signed distance is added to each three-dimensional Gaussian atom as a Gaussian parameter. This distance is used to represent the relative proximity of the three-dimensional Gaussian atom to the implicit surface. The smaller the absolute value of this distance, the closer the current three-dimensional Gaussian atom is to the implicit surface. Perform differentiable Gaussian rasterization at any training viewpoint to obtain the color and opacity of each three-dimensional Gaussian atom; The average gradient of the three-dimensional Gaussian atoms is used as a density control factor to determine whether densification and pruning are needed. Densification and pruning strategies are applied to the three-dimensional Gaussian set based on the size and transparency of the three-dimensional Gaussian atoms. Obtain the index and contribution value of each pixel and its K highest-contributing 3D Gaussian atoms, and then perform weighted aggregation of the Gaussian-level learnable signed distances to obtain pixel-level signed distances. During color rendering, pixel-level signed distances are also involved in color synthesis to calculate the reconstructed image.

2. The multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 1, characterized in that, When the density control factor of the i-th 3D Gaussian atom is greater than a set threshold, the densification and pruning strategies are determined. The densification strategy determines the scale vector size; if the scaling scale is less than the set threshold, a copy of the 3D Gaussian atom is made at its center position. Otherwise, the direction of the zero horizontal plane is estimated using the signed distance distribution of the neighborhood Gaussians, and the center position of the Gaussian atom is moved in this direction. | / 2, and at the symmetrical position, that is, the position with a signed distance of 0 as the symmetrical point, a new three-dimensional Gaussian atom is generated at the position opposite to the current three-dimensional Gaussian atom. Then the scale vector of the two three-dimensional Gaussian atoms is changed to the original 0.

75. The pruning strategy determines that if the average opacity of the three-dimensional Gaussian atom is greater than the set threshold, then the redundant three-dimensional Gaussian atom is removed. Let be the signed distance of the i-th three-dimensional Gaussian atom.

3. The multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 2, characterized in that, The density control factor is updated based on the signed distance. The update process includes: in, The density control factor is updated based on the signed distance; The average gradient of a three-dimensional Gaussian atom; Weights for signed distance in density control; Let be the proximity function of the signed distance to the i-th three-dimensional Gaussian atom, denoted as , This is a hyperparameter.

4. The multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 2, characterized in that, When implementing the pruning strategy, the average opacity of the three-dimensional Gaussian atoms is updated based on the signed distance. The update process includes: in, The average opacity of three-dimensional Gaussian atoms updated based on signed distance; Weights for signed distance in the pruning strategy; Let be the proximity function of the signed distance to the i-th three-dimensional Gaussian atom, denoted as , This is a hyperparameter.

5. The multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 1, characterized in that, In Gaussian differentiable rasterization, the color of each pixel p is obtained by superimposing a ray along the viewing direction with a set of ordered 3D Gaussian atoms. For the 3D Gaussian atom hit by the ray, the contribution of the i-th 3D Gaussian atom to pixel p is: in, Let be the contribution of the i-th 3D Gaussian atom to pixel p; Let be the opacity of the i-th three-dimensional Gaussian atom; Let be the cumulative transmittance of the ray before it reaches the i-th three-dimensional Gaussian atom, with the observation direction as the starting point of the ray.

6. A multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 1 or 5, characterized in that, During color rendering, pixel-level signed distances are also involved in color composition, including: in, To render the pixel value of pixel p in the image; The contribution value of the i-th three-dimensional Gaussian atom among the K three-dimensional Gaussian atoms that contribute the most to pixel p; The signed distance weight of the i-th 3D Gaussian atom among the K 3D Gaussian atoms that contribute the most to the i-th pair of pixels p is denoted as: , Let be the signed distance of the i-th three-dimensional Gaussian atom. Let i be the color of the i-th three-dimensional Gaussian atom.

7. The multi-view reconstruction method based on three-dimensional Gaussian splashing and signed distance function according to claim 1, characterized in that, The Gaussian parameters of a 3D Gaussian atom, namely the signed distance, center position, scale factor, rotation factor, and opacity, participate in the forward and backward propagation during differentiable Gaussian rasterization. The loss function during training is expressed as: in, The loss function; , As a weighting factor; To render the image; It is a real image; This indicates the calculation of the L1 norm; Indicates the computational rendering of the image With real images The similarity.

Images