A Gaussian Splash Covariance Matrix Optimization Method Based on Hamiltonian Monte Carlo Algorithm

Abstract

This invention discloses a method for optimizing the covariance matrix of Gaussian splashes based on the Hamiltonian Monte Carlo algorithm, belonging to the field of 3D Gaussian splash modeling technology. This invention constructs a dynamic system using the Hamiltonian Monte Carlo algorithm, and further constructs dynamic equations to improve the ability and accuracy of the covariance matrix in optimizing Gaussian ellipsoid parameters. By constructing Hamiltonian quantities, introducing momentum variables, and using the frog-leap method to simulate dynamics, a novel covariance matrix optimized by the mass matrix is ​​obtained, providing an optimization and analysis method for the Gaussian splash covariance matrix based on the Hamiltonian Monte Carlo algorithm. Simultaneously, a peak signal-to-noise ratio (PSNR) evaluation mechanism is proposed to quantify the optimization results.

Description

Technical Field

[0001] This invention relates to the field of 3D Gaussian splash modeling technology, specifically to a Gaussian splash covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm. Background Technology

[0002] To address the shortcomings of existing optimization techniques for the covariance matrix in 3D Gaussian splashing, traditional 3D Gaussian splashing methods, which use gradient descent to optimize the Gaussian ellipsoid parameters of the covariance matrix, are prone to getting trapped in local optima. The covariance matrix needs to remain positive semi-definite, but direct optimization using gradient descent easily violates this constraint, potentially leading to physically invalidated parameters. This increases the difficulty of optimizing these parameters. Standard gradient descent converges slowly for high-dimensional parameters (such as 9-dimensional covariance), and anisotropic optimization is prone to jagged edges, affecting the quality of 3D scene reconstruction. (Covariance matrix...) The rotation matrix R (quaternion) ) and scaling matrix S (vector) The problem involves strong nonlinear constraints (unit quaternions, positive definiteness). Stochastic gradient descent is sensitive to hyperparameters, and covariance matrix degradation can easily lead to rendering distortion. Subsequently, researchers used Markov chain Monte Carlo algorithms (such as the Metropolis-Hastings algorithm) to further optimize the covariance matrix, but such methods are inefficient in random walks in the parameter space, and existing optimizers struggle to balance rendering accuracy (loss function nonconvexity) with computational efficiency.

[0003] The aforementioned problems urgently need to be solved. To address this, this invention proposes an optimization and analysis method for the Gaussian splash covariance matrix based on the Hamiltonian Monte Carlo algorithm. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to solve the problems of gradient descent being prone to local optima and violating semidefinite constraints, and the low efficiency of Markov chain Monte Carlo algorithm. This invention provides a Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm. This method proposes to construct a dynamic system by Hamiltonian Monte Carlo algorithm, and combine data acquisition and preprocessing, covariance matrix construction, gradient calculation, hybrid optimization strategy and quantitative evaluation to achieve efficient and accurate optimization of covariance matrix.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution, and the present invention includes the following steps:

[0006] S1: Data Acquisition and Preprocessing

[0007] Building image data is collected by drones, and the building image data is preprocessed to obtain building image data at a set high resolution.

[0008] S2: Construction of the covariance matrix

[0009] Construct a covariance matrix to describe the shape and orientation of the Gaussian ellipsoid;

[0010] S3: Introducing momentum variables

[0011] Construct a Hamiltonian dynamical system as ( , Introducing momentum variables, we construct phase space. , , , ), , They are respectively with The corresponding momentum vector, For unit quaternions, For a positive definite scaling vector, an adaptive momentum decay strategy is designed for the momentum vector;

[0012] S4: Define the rendering loss function

[0013] Define the rendering loss function as a comprehensive loss function formed by the basic rendering loss function, edge loss function, SSIM loss function and multi-scale loss function;

[0014] S5: Constructing the Hamiltonian

[0015] Constructing the Hamiltonian based on Hamiltonian dynamics The canonical equation is derived from the Hamiltonian, that is, the Hamiltonian equation is obtained, and the Hamiltonian equation is expressed continuously.

[0016] S6: Optimization Process

[0017] Constraining unit quaternions using the Leapfrog integrator With positive definite scaling vector and calculate and The gradient is then calculated; the quality matrix is ​​adaptively set, and then random subsampling is performed.

[0018] S7: GPU Parallel Acceleration

[0019] The GPU is used to update the rendering and dynamics of each Gaussian ellipsoid as an independent unit;

[0020] S8: Hybrid Optimization Strategy

[0021] First, run multiple epochs using gradient descent. Then, use the Markov chain Monte Carlo algorithm to optimize the covariance matrix. Finally, use the covariance matrix optimized by the Hamiltonian Monte Carlo algorithm in steps S3 to S7 to evaluate the data.

[0022] Furthermore, in step S2, the expression for the covariance matrix is ​​as follows:

[0023] ;

[0024] in, is a 3×3 covariance matrix for a single Gaussian ellipsoid, used to describe the shape and orientation of the Gaussian ellipsoid; From unit quaternion The resulting 3×3 rotation matrix; It is related to the positive definite scaling vector The relevant scale matrix.

[0025] Furthermore, in step S3, the adaptive momentum decay strategy is specifically as follows:

[0026] S31: Calculate the rate of change of the comprehensive loss function in the current iteration step in step S4, that is, the ratio of the absolute value of the difference between the current loss and the previous step loss to the corresponding time difference;

[0027] S32: Adjust the momentum decay coefficient based on the rate of change of the loss function. When the rate of change is greater than or equal to a set threshold, it indicates fast convergence, so increase the momentum decay coefficient. When the rate of change is less than the set threshold, it indicates slow convergence, so decrease the momentum decay coefficient. The update formula for the momentum decay coefficient is as follows:

[0028] ;

[0029] in, The current momentum decay coefficient, It's about adjusting hyperparameters. It is the rate of change of the loss function. This is the difference between the current loss and the loss from the previous step. This represents the corresponding time difference. This is the updated momentum decay coefficient;

[0030] S33: To prevent momentum decay from being too fast or too slow, set upper and lower limits for the momentum decay coefficient;

[0031] S34: Update the equation based on the new momentum decay coefficient. Corresponding momentum vector , .

[0032] Furthermore, in step S4, the expression for the comprehensive loss function is as follows:

[0033] ;

[0034] in, To optimize the objective, which is also the potential energy term in the Hamiltonian; , , , For hyperparameters, Based on the rendering loss function, For edge loss function, For SSIM loss function, For multi-scale loss functions;

[0035] The basic rendering loss function is defined as follows:

[0036] ;

[0037] in, It is an image rendered from the current Gaussian ellipsoid parameters, i.e., a rendered image; It is a real image. , These are the height and width of the rendered image and the actual image, respectively.

[0038] The edge loss function is defined as follows:

[0039] ;

[0040] in, and These are the edge images of the rendered image and the real image, respectively.

[0041] The SSIM loss function is defined as follows:

[0042] ;

[0043] in, This indicates the calculation of structural similarity;

[0044] The multi-scale loss function is defined as follows:

[0045] ;

[0046] in, Indicates different scales. It is the weight of each scale. It is the pixel-by-pixel loss at each scale.

[0047] Furthermore, in step S5, the expression for the Hamiltonian is as follows:

[0048] ;

[0049] in, It is the Hamiltonian, which is equal to the sum of the potential energy and kinetic energy terms; , For the quality matrix, , It represents its inverse matrix, and the mass matrix is ​​used to adjust the kinetic energy term.

[0050] Furthermore, in step S5, the Hamiltonian equation is expressed as a continuity equation as follows:

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] in, They are right , gradient, Right now .

[0056] Furthermore, in step S6, during constraint processing, for unit quaternions... After the position is updated, it is re-regularized to a unit quaternion; for positive definite scaling vectors The calculation is performed in the logarithmic field, the update is performed in the unconstrained space, and then mapped back to the positive definite field.

[0057] Furthermore, in step S6, when performing gradient calculation, for unit quaternions... , the original gradient Projected onto the tangent space of the unit quaternion The gradient calculation is implemented as follows:

[0058] ;

[0059] in, It projects a vector onto a unit quaternion. The tangent space operation of the institution guarantees that the gradient is in the feasible direction. Right now , Right now , The adjustment factor is used to adjust the accuracy of the quaternion projection, and its calculation formula is as follows:

[0060] ;

[0061] in, For the region The level of detail complexity, γ and It is a hyperparameter, γ, used to control the degree to which detail complexity affects projection accuracy. This is a constant term used to set the minimum projection precision;

[0062] For positive definite scaling vectors Using a logarithmic mapping, the gradient used for dynamics is calculated as follows:

[0063] ;

[0064] Where ⊙ represents element-wise multiplication, that is, matrix multiplication... The elements within are multiplied one by one. Right now .

[0065] Furthermore, in step S6, when adaptively setting the quality matrix, the elements of the quality matrix are set to the reciprocal of the historical variance of the corresponding parameter.

[0066] Furthermore, in step S6, each calculation Randomly subsample image blocks or Gaussian subsets with gradient.

[0067] Furthermore, in step S8, peak signal-to-noise ratio (PSNR) and mean square error (MSE) are used as metrics to evaluate the degree of optimization of the covariance matrix, wherein the mean square error... Peak signal-to-noise ratio , To render image pixels in Strength at that location, For real images in The pixel value at that location, where MAX is the maximum pixel value.

[0068] Compared with existing technologies, this invention has the following advantages: The Gaussian splash covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm constructs a dynamic system using the Hamiltonian Monte Carlo algorithm, and further constructs dynamic equations to improve the processing capability and accuracy of the covariance matrix for Gaussian ellipsoid parameters in 3D Gaussian splash optimization. By constructing Hamiltonian quantities, introducing momentum variables, and using the Leapfrog integrator to simulate dynamics, a novel covariance matrix optimized by the mass matrix is ​​obtained, providing an optimization and analysis method for the Gaussian splash covariance matrix based on the Hamiltonian Monte Carlo algorithm. Simultaneously, a peak signal-to-noise ratio (PSNR) evaluation mechanism is proposed to quantify the optimization results. Attached Figure Description

[0069] Figure 1 This is a flowchart illustrating the Gaussian splash covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm in an embodiment of the present invention.

[0070] Figure 2 This is a schematic diagram of the experimental real image of the Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm in an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram showing the result of adding random noise to real experimental images in an embodiment of the present invention;

[0072] Figure 4 This is a schematic diagram showing the results of the Gaussian splash covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm of this invention;

[0073] Figure 5 This is a schematic diagram showing the result of optimizing the covariance matrix based on the Markov chain Monte Carlo algorithm.

[0074] Figure 6 This is a schematic diagram showing the results of optimizing the covariance matrix based on gradient descent.

[0075] Figure 7 This is a line graph comparing the PSNR results of three methods. The three methods are the Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm, the covariance matrix optimization method based on Markov chain Monte Carlo algorithm, and the covariance matrix optimization method based on gradient descent method. Detailed Implementation

[0076] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0077] like Figure 1 As shown, this embodiment provides a technical solution: a Gaussian splash covariance matrix optimization method based on the Hamiltonian Monte Carlo algorithm, including the following steps:

[0078] Step S1: Collect small building image data using a drone, preprocess the small building image data to obtain high-resolution small building image data.

[0079] In this step, small buildings refer to buildings with an area of ​​less than 500 square meters, and high resolution refers to spatial resolution of 1 meter or less.

[0080] In this step, image data preprocessing is performed, mainly including geometric correction and radiometric correction. Geometric correction corrects geometric distortions in the image, ensuring that the image pixels correspond one-to-one with their actual spatial locations on the ground. Radiometric correction eliminates radiometric deviations caused by factors such as sensors, weather, and solar angle, ensuring data consistency and accuracy, and removing noise.

[0081] Step S2: Construct the covariance matrix ,in, It is a 3×3 covariance matrix for a single Gaussian ellipsoid, used to describe the shape and orientation of the Gaussian ellipsoid (symmetric positive definite matrix). A unit quaternion used to describe the rotation of a Gaussian ellipsoid, satisfying... ; It is a positive definite scaling vector that controls the length of the semi-axis of the Gaussian ellipsoid in the base direction; From unit quaternion The resulting 3×3 rotation matrix; It is related to the positive definite scaling vector The relevant scaling matrix is ​​usually a diagonal matrix.

[0082] Step S3: Construct the Hamiltonian dynamical system as ( , Introducing momentum variables, we construct phase space. , , , ), , They are respectively with The corresponding momentum vector.

[0083] To improve the convergence stability of the Hamiltonian dynamics system, an adaptive momentum decay strategy is designed for the momentum vector. The specific scheme is as follows: Calculate the rate of change of the loss function: First, calculate the rate of change of the loss function (i.e., the comprehensive loss function below) in the current iteration step, which is the ratio of the absolute value of the difference between the current loss and the previous loss to the corresponding time difference. A faster decrease in loss indicates faster convergence of the optimization process, while a slower change indicates slower convergence. Define the momentum decay coefficient update formula: Dynamically adjust the momentum decay coefficient according to the rate of change of the loss function; if the rate of change of the loss function is greater than or equal to a certain threshold, it indicates faster convergence, and the decay coefficient should be increased; if it is less than a certain threshold, it indicates slower convergence, and the decay coefficient should be decreased. Update the decay coefficient: The momentum decay coefficient is updated using the following formula:

[0084]

[0085] in, This is the updated momentum decay coefficient. The current momentum decay coefficient, It's about adjusting hyperparameters. It is the change in the loss function (the difference between the current loss and the previous loss). It is the rate of change of the loss function. This indicates the corresponding time difference. Set the momentum variation range: To prevent momentum decay from being too fast or too slow, set upper and lower limits for the momentum decay coefficient to ensure it is adjusted within a reasonable range. Update momentum: Based on the new decay coefficient, update the momentum... Corresponding momentum vector , ( Since these are coefficients, you can simply multiply them. , direct multiplication This improves the system's stability and convergence speed.

[0086] Step S4: First, define the basic rendering loss function as the rendered image. With real images The measure of pixel-by-pixel difference is:

[0087]

[0088] in, It is an image rendered from the current Gaussian ellipsoid parameters, i.e., a rendered image; It is a real image; , These represent the height and width of the two images (used to represent the number of pixels).

[0089] The edges and contours of building images often contain important structural information. Therefore, edge loss is introduced, and edge detection operators are used to extract edge features from the image, thereby calculating the differences between edges. Edge loss can be defined as:

[0090]

[0091] in, and These are the edge images of the rendered image and the real image, respectively, obtained through edge detection operators.

[0092] Structural similarity loss (SSIM) is an important metric for evaluating the structural similarity of images, particularly suitable for preserving detailed structural features. SSIM loss calculates the brightness, contrast, and structural similarity of an image:

[0093]

[0094] SSIM calculates the structural similarity between two images; the closer the value is to 1, the more similar the image structures. Therefore, 1-SSIM is used as the loss function.

[0095] Multi-scale loss can help the optimization process capture architectural features at different scales, especially in building imagery where details may be revealed at different scales. By processing the image at different scales and then calculating the loss at each scale, the rendering performance of the image at different scales can be improved.

[0096]

[0097] in, Indicates different scales. It represents the weight of each scale. It is the pixel-by-pixel loss at each scale.

[0098] The final rendering loss function combines all the above losses into a comprehensive loss function:

[0099]

[0100] in, To optimize the objective (which is also the potential energy term in the Hamiltonian), i.e., to render the loss function, , , , These are hyperparameters used to adjust the weights of various loss parameters. By adjusting these weights, the effects of pixel-wise loss, edge loss, structural similarity loss, and multi-scale loss can be balanced, thereby optimizing the quality of the rendered image.

[0101] Step S5: Construct the Hamiltonian based on the Hamiltonian dynamics system:

[0102] ;

[0103] in, The Hamiltonian (total energy) is equal to the sum of potential energy and kinetic energy; , For the quality matrix, , It represents its inverse matrix, and the mass matrix is ​​used to adjust the kinetic energy term, affecting the rate of dynamic evolution.

[0104] Step S6: Based on the Hamiltonian derivation in Step S5, the canonical equation is obtained, i.e., the Hamiltonian equation is obtained. The Hamiltonian equation is expressed as a continuity equation as follows:

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] in, They are right , gradient, Right now .

[0110] Step S7: Use the Leapfrog integrator to constrain the quaternary parameters With scaling parameters For unit quaternions After the position is updated, it must be re-regularized to a unit quaternion; It can be computed in the logarithmic field, allowing updates to be performed in an unconstrained space and then mapped back to the positive definite region.

[0111] In this step, updates using the Leapfrog integrator are performed by alternately updating position and momentum. Here, "position" is a quaternion. and scaling vector Momentum is a randomly initialized momentum vector. During each update, a new gradient is calculated and used to update the momentum. The updated momentum is then used to update the position (quaternion and scaling vector).

[0112] Step S8: Perform gradient calculation, provided by the differentiable rendering module. For any parameter θ (here, θ is θ) , The derivative of ) with respect to : The original gradient Projected onto the tangent space of the quaternion : ;right : Using a logarithmic mapping, the gradient actually used for dynamics is (⊙ represents element-wise multiplication); where It projects a vector onto a quaternion. The tangent space operation of the institution guarantees that the gradient is in the feasible direction. Right now .

[0113] The level of detail in building images (such as the details of walls, windows, doors, and roofs) is crucial for the optimization process. The richness of detail in a region can be evaluated based on the local structural complexity of the image. Based on this level of detail, different projection accuracies can be applied to different regions. Higher projection accuracies are used for regions with complex details (such as windows, doors, and roofs), while lower accuracies can be used for regions with simpler structures (such as large walls) to improve computational efficiency. An adjustment factor is defined. This factor is used to control the precision of the quaternion projection. It is dynamically adjusted based on the detail complexity of the image. For regions with complex details (such as edges, windows, doors, etc.), it will... Increasing the size improves projection accuracy; for simpler areas, the size can be decreased. This reduces computational precision and optimizes computational efficiency.

[0114] It can be done for each pixel or region Calculation detail complexity Then adjust the projection accuracy based on this complexity:

[0115]

[0116] in, For the region The level of detail complexity is typically measured by edge strength, gradient, etc. γ and It is a hyperparameter, γ, used to control the degree to which detail complexity affects projection accuracy. This is a constant term used to set the minimum projection accuracy.

[0117] Then, the projection operation is adjusted according to the projection accuracy adjustment factor, and the projection formula is:

[0118] .

[0119] In this step, the differentiable rendering module's role is to handle each parameter (quaternion). and scaling vector Calculate the gradients and ensure that these gradient updates are within a reasonable range. Specifically: use tangent space projection on the quaternion to ensure that the gradient does not violate the unit constraint of the quaternion, use logarithmic mapping on the scaling vector, and then process the gradient updates by element-wise multiplication to keep them always within the positive definite region.

[0120] Step S9: For the mass matrix ( , To accelerate convergence, adaptive settings can be implemented, and the elements of the quality matrix can be set to the reciprocals of the historical variances of the parameters, specifically: ,here for Historical variance estimation helps to scale step sizes across different dimensions, ensuring consistent learning rates for parameters with different dimensions.

[0121] Step S10: Perform random subsampling, calculating each time. By randomly subsampling image blocks or Gaussian subsets during gradient calculation, the computational cost per step is reduced and the gradient is approximated in multiple batches.

[0122] Step S11: Use the GPU to update the rendering and dynamics of each Gaussian ellipsoid into independent units, which can be mapped to CUDA threads / blocks.

[0123] Step S12: Evaluate the results using a hybrid optimization strategy. First, run gradient descent for several epochs (quickly reduces high errors, but prone to local convergence). Then, use a Markov chain Monte Carlo algorithm (such as the Metropolis-Hastings algorithm) to optimize the covariance matrix. Finally, use the covariance matrix optimized by the Hamiltonian Monte Carlo algorithm described above to evaluate the data.

[0124] Step S13: Add random noise to the experimental data (see...) Figure 3 The peak signal-to-noise ratio (PSNR) and mean squared error (MSE) are used as metrics to evaluate the degree of optimization of the covariance matrix. Peak signal-to-noise ratio . : Rendering image pixels in The strength at that location. : True value of the image pixel. MAX: Maximum pixel value.

[0125] Step S14: Implement the method of the present invention and compare it with the traditional methods for optimizing the covariance matrix based on gradient descent and Markov chain Monte Carlo algorithm. The PSNR of the method based on gradient descent is 39.77 dB, and the MSE is 6.63. The PSNR of the method based on Markov chain Monte Carlo algorithm is 47.31 dB, and the MSE is 1.00. The PSNR of the Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm in this invention is 58.93 dB, and the MSE is 0.12. The results show (e.g.) Figure 7 (As shown) The covariance matrix optimization of the present invention is better.

[0126] like Figures 4 to 6 As shown in the results, the method of this invention has a stronger ability to process data and achieves better results. Compared with the other two methods, the results are closer to the real image. Among them, the Markov chain Monte Carlo algorithm will sharpen local features when processing images, thus deviating from the details of the real image. The gradient descent method has a poor overall effect when processing images, and its results are closer to the real image. Figure 2 The difference is significant, and the overall ambiguity will be more obvious.

[0127] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm, characterized in that, Includes the following steps: S1: Data Acquisition and Preprocessing Building image data is collected by drones, and the building image data is preprocessed to obtain building image data at a set high resolution. S2: Construction of the covariance matrix Construct a covariance matrix to describe the shape and orientation of the Gaussian ellipsoid; S3: Introducing momentum variables Construct a Hamiltonian dynamical system as ( , Introducing momentum variables, we construct phase space. , , , ), , They are respectively with The corresponding momentum vector, For unit quaternions, For a positive definite scaling vector, an adaptive momentum decay strategy is designed for the momentum vector; S4: Define the rendering loss function Define the rendering loss function as a comprehensive loss function formed by the basic rendering loss function, edge loss function, SSIM loss function and multi-scale loss function; S5: Constructing the Hamiltonian Constructing the Hamiltonian based on Hamiltonian dynamics The canonical equation is derived from the Hamiltonian, that is, the Hamiltonian equation is obtained, and the Hamiltonian equation is expressed continuously. S6: Optimization Process Constraining unit quaternions using the Leapfrog integrator With positive definite scaling vector and calculate and The gradient is then calculated; the quality matrix is ​​adaptively set, and then random subsampling is performed. S7: GPU Parallel Acceleration The GPU is used to update the rendering and dynamics of each Gaussian ellipsoid as an independent unit; S8: Hybrid Optimization Strategy First, run multiple epochs using gradient descent, then use the Markov chain Monte Carlo algorithm to optimize the covariance matrix, and finally use the covariance matrix optimized by the Hamiltonian Monte Carlo algorithm in steps S3 to S7 to evaluate the data. In step S2, the expression for the covariance matrix is ​​as follows: ； in, is a 3×3 covariance matrix for a single Gaussian ellipsoid, used to describe the shape and orientation of the Gaussian ellipsoid; From unit quaternion The resulting 3×3 rotation matrix; It is related to the positive definite scaling vector The relevant scaling matrix; In step S4, the expression for the comprehensive loss function is as follows: ； in, To optimize the objective, which is also the potential energy term in the Hamiltonian; , , , For hyperparameters, Based on the rendering loss function, For edge loss function, For SSIM loss function, For multi-scale loss functions; The basic rendering loss function is defined as follows: ； in, It is an image rendered from the current Gaussian ellipsoid parameters, i.e., a rendered image; It is a real image. , These are the height and width of the rendered image and the actual image, respectively. Two-dimensional coordinates for both the rendered image and the pixels of the real image; The edge loss function is defined as follows: ； in, and These are the edge images of the rendered image and the real image, respectively. The SSIM loss function is defined as follows: ； in, This indicates the calculation of structural similarity; The multi-scale loss function is defined as follows: ； in, Indicates different scales. It is the weight of each scale. It is the pixel-by-pixel loss at each scale; In step S5, the expression for the Hamiltonian is as follows: ； in, It is the Hamiltonian, which is equal to the sum of the potential energy and kinetic energy terms; , For the quality matrix, , This represents its inverse matrix; the mass matrix is ​​used to adjust the kinetic energy term. In step S5, the Hamiltonian equation is expressed as a continuous expression as follows: ； ； ； ； in, They are right , gradient, Right now ; In step S6, during constraint processing, for unit quaternions... After the position is updated, it is re-regularized to a unit quaternion; for positive definite scaling vectors The calculation is performed in the logarithmic field, the update is performed in the unconstrained space, and then mapped back to the positive definite field. In step S6, when performing gradient calculation, for unit quaternions... , the original gradient Projected onto the tangent space of the unit quaternion The gradient calculation is implemented as follows: ； in, It projects a vector onto a unit quaternion. The tangent space operation of the institution guarantees that the gradient is in the feasible direction. Right now , Right now , The adjustment factor is used to adjust the accuracy of the quaternion projection, and its calculation formula is as follows: ； in, For the region The level of detail complexity, γ and It is a hyperparameter, γ, used to control the degree to which detail complexity affects projection accuracy. This is a constant term used to set the minimum projection precision; For positive definite scaling vectors Using a logarithmic mapping, the gradient used for dynamics is calculated as follows: ； Where ⊙ represents element-wise multiplication, that is, matrix multiplication... The elements within are multiplied one by one. Right now .

2. The Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm according to claim 1, characterized in that, In step S3, the adaptive momentum decay strategy is as follows: S31: Calculate the rate of change of the comprehensive loss function in the current iteration step in step S4, that is, the ratio of the absolute value of the difference between the current loss and the previous step loss to the corresponding time difference; S32: Adjust the momentum decay coefficient based on the rate of change of the loss function. When the rate of change is greater than or equal to a set threshold, it indicates fast convergence, so increase the momentum decay coefficient. When the rate of change is less than the set threshold, it indicates slow convergence, so decrease the momentum decay coefficient. The update formula for the momentum decay coefficient is as follows: ； in, The current momentum decay coefficient, It's about adjusting hyperparameters. It is the rate of change of the loss function. This is the difference between the current loss and the loss from the previous step. This represents the corresponding time difference. This is the updated momentum decay coefficient; S33: To prevent momentum decay from being too fast or too slow, set upper and lower limits for the momentum decay coefficient; S34: Update the equation based on the new momentum decay coefficient. Corresponding momentum vector , .

3. The Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm according to claim 1, characterized in that, In step S6, when adaptively setting the quality matrix, the elements of the quality matrix are set to the reciprocal of the historical variance of the corresponding parameter.

4. The Gaussian splash covariance matrix optimization method based on Hamiltonian Monte Carlo algorithm according to claim 1, characterized in that, In step S8, peak signal-to-noise ratio (PSNR) and mean square error (MSE) are used as metrics to evaluate the degree of optimization of the covariance matrix, wherein the mean square error... Peak signal-to-noise ratio , To render image pixels in Strength at that location, For real images in The pixel value at that location, where MAX is the maximum pixel value.

Images