Method for optimizing geometric structure and rendering effect of point cloud based on 3DGS

Abstract

The invention discloses a 3DGS-based point cloud geometric structure and rendering effect optimization method, and relates to the field of three-dimensional reconstruction. The method comprises the following steps: carrying out data acquisition on a target scene to obtain a scene image set, and preprocessing data in the scene image set; sparse reconstruction is carried out based on the preprocessed scene image set, and sparse point clouds of the scene are recovered; constructing a three-dimensional Gaussian point cloud based on the sparse point cloud obtained in the step 2, and flattening the three-dimensional Gaussian point cloud to obtain a two-dimensional Gaussian point cloud; carrying out two-dimensional Gaussian point cloud projection; creating a filter based on scale adaptation and acting the filter on the projected two-dimensional Gaussian point cloud to adaptively modify the distribution size of the two-dimensional Gaussian projection; rendering the two-dimensional Gaussian point cloud adjusted by the filter to obtain a rendered image; and according to a comparison result of the rendered image and the corresponding image in the preprocessed scene image set, carrying out back propagation to carry out two-dimensional Gaussian point cloud optimization. According to the invention, the scene reconstruction precision and the reality of the rendering effect can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a method for optimizing point cloud geometry and rendering effects based on 3DGS. Background Art

[0002] With the rapid development of technologies such as virtual reality (VR), augmented reality (AR), autonomous driving, and robotic navigation, the demand for efficient and realistic rendering of 3D scenes is growing. Traditional 3D scene reconstruction methods, such as those based on laser scanning or multi-view images, and 3D modeling using specialized software, suffer from low efficiency and sub-realistic rendering effects.

[0003] 3D Gaussian Splatting (3DGS) is an emerging 3D scene reconstruction method that explicitly models the scene by using a set of 3D Gaussian ellipsoids to form a point cloud. These Gaussian ellipsoids can be dynamically added and removed during the optimization process of scene reconstruction to cover the entire surface of the reconstructed scene and represent high-frequency details.

[0004] 3DGS still faces challenges in practical applications. The point cloud geometry of 3DGS is represented by a three-dimensional Gaussian ellipsoid. Its non-zero thickness along all three axes significantly hinders its close alignment with scene surfaces, making it particularly difficult to achieve a good fit when modeling thin surfaces. There is ambiguity in the direction of the normal of each 3D Gaussian ellipsoid, meaning that during the scene reconstruction optimization process, the normal direction can vary at different scales. This variation can lead to inaccuracies in the reconstruction of scenes with fine detail. Furthermore, 3DGS incorporates a 2D dilation during the projection of the 3D Gaussian ellipsoid to ensure the stability of the 2D Gaussian basis primitives projected onto the image plane, but this can cause artifacts when the rendering resolution is changed. 2D dilation involves dilating the 2D Gaussian projected onto the image plane during projection to create the rendered image. Lowering the rendering resolution results in a reduction in the size of the projected 2D Gaussian on the image plane. Applying the same 2D dilation results in dilation artifacts, making the rendered scene appear unrealistically dilated. Conversely, increasing the resolution of the rendering will lead to erosion artifacts. This is because the size of the projected 2D Gaussian increases, but the 2D dilation remains unchanged. This erosion artifact causes the scene's rendering to shrink in a needle-like manner. Therefore, further exploration of a technical solution that can achieve realistic rendering effects without artifacts based on the 3D Gaussian splatter rendering method is urgently needed. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a point cloud geometry structure and rendering effect optimization method based on 3DGS, aiming to improve the accuracy of scene reconstruction and the realism of rendering effects by exploring the three-dimensional Gaussian ellipsoid compression method and the artifact optimization strategy guided by Shannon sampling theorem.

[0006] The technical solution of the present invention is:

[0007] A method for optimizing point cloud geometry and rendering effects based on 3DGS, the method comprising the following steps:

[0008] Step 1: Collect data of the target scene to obtain a scene image set, and perform relevant preprocessing on the data in the scene image set;

[0009] Step 2: Perform sparse reconstruction based on the preprocessed scene image set to restore the sparse point cloud of the scene;

[0010] Step 3: Construct a 3D Gaussian point cloud based on the sparse point cloud obtained in step 2, and flatten the 3D Gaussian point cloud to obtain a 2D Gaussian point cloud;

[0011] Step 4: 2D Gaussian point cloud projection;

[0012] Step 5: Create a scale-adaptive filter and apply it to the projected two-dimensional Gaussian point cloud to adaptively modify the size of the two-dimensional Gaussian projection distribution;

[0013] Step 6: Render the two-dimensional Gaussian point cloud after filter adjustment to obtain a rendered image;

[0014] Step 7: Based on the comparison results of the rendered image obtained in step 6 and the corresponding image in the preprocessed scene image set, back propagate and perform two-dimensional Gaussian point cloud optimization.

[0015] Furthermore, according to the 3DGS-based point cloud geometry structure and rendering effect optimization method, in step 2, SfM technology is used for sparse reconstruction.

[0016] Furthermore, according to the point cloud geometry structure and rendering effect optimization method based on 3DGS, the method for constructing a three-dimensional Gaussian point cloud based on the sparse point cloud obtained in step 2 described in step 3 is: generating a three-dimensional Gaussian ellipsoid corresponding one-to-one to the points in the sparse point cloud; initializing the parameters of the three-dimensional Gaussian ellipsoid: using the three-dimensional coordinates of a point in the sparse point cloud as the center of the three-dimensional Gaussian ellipsoid corresponding to the point, and initializing the position of the three-dimensional Gaussian ellipsoid; calculating the average distance from the center point position of each three-dimensional Gaussian ellipsoid to the center point position of the nearest neighbor by the K-nearest neighbor algorithm, as the initial radius of the three-dimensional Gaussian ellipsoid; using the covariance matrix Σ to record the position information and initial radius of the three-dimensional Gaussian ellipsoid; initializing the opacity α of the three-dimensional Gaussian ellipsoid to a fixed value; using spherical harmonics to represent the color of the three-dimensional Gaussian ellipsoid; and forming a three-dimensional Gaussian point cloud with a large number of initialized three-dimensional Gaussian ellipsoids corresponding to the sparse point cloud.

[0017] Furthermore, according to the 3DGS-based point cloud geometry and rendering effect optimization method, the method for flattening the three-dimensional Gaussian point cloud described in step 3 is as follows: decompose the covariance matrix Σ of the three-dimensional Gaussian ellipsoid into a rotation matrix R and a diagonal scaling matrix S, and satisfy Σ=RSS T R T ; where R is the rotation matrix represented by the quaternion, and the diagonal elements of the diagonal scaling matrix S are s x ,s y ,s z , represents the scaling factor of three dimensions; compress the three-dimensional Gaussian ellipsoid along the direction with the smallest scale factor, specifically, in the covariance matrix of the three-dimensional Gaussian ellipsoid, compress the third dimension of the diagonal scaling matrix to 0, that is, S=(s x ,s y ,0), and extract the third column of the rotation matrix, that is, the z-axis direction vector after rotation, as the normal direction of the plane where the Gaussian ellipse is located, and flatten the three-dimensional Gaussian ellipsoid into a plane closest to its original shape, that is, a two-dimensional Gaussian ellipse; according to the above method, all three-dimensional Gaussian ellipsoids in the three-dimensional Gaussian point cloud are flattened into two-dimensional Gaussian ellipses, thereby obtaining a two-dimensional Gaussian point cloud formed by a large number of two-dimensional Gaussian ellipses.

[0018] Furthermore, according to the 3DGS-based point cloud geometry and rendering effect optimization method, a scale-adaptive scaling factor r is introduced in step 5 to create a scale-adaptive filter as shown in formula (4);

[0019] The scale-adaptive scaling factor r=ΔRp / ΔDc, where ΔRp is the ratio of the resolution of the current rendering camera to the resolution of the training camera; ΔDc is the ratio of the focal length of the current rendering camera to the focal length of the training camera closest to the rendering camera;

[0020]

[0021] in represents the distribution of the two-dimensional Gaussian point cloud in the two-dimensional image plane coordinate system after projection; x is the function independent variable, representing the initial point of the Gaussian distribution; k represents the number of ellipses in the point cloud; ∑ k represents the covariance matrix of the k-th two-dimensional Gaussian ellipse; p k represents the position of the kth Gaussian ellipse; I is a two-dimensional identity matrix; s is a scalar dilation hyperparameter.

[0022] Furthermore, according to the 3DGS-based point cloud geometry structure and rendering effect optimization method, the two-dimensional Gaussian point cloud rendering after filter adjustment in step 6 includes two-dimensional Gaussian point cloud projection rasterization, and supersampling the initial rendering image obtained by the two-dimensional Gaussian point cloud projection rasterization to obtain the final rendering image.

[0023] Furthermore, according to the 3DGS-based point cloud geometry structure and rendering effect optimization method, in step 7, based on the principle of the existing 3DGS back-propagation adaptive Gaussian densification scheme, two-dimensional Gaussian point cloud optimization is performed by back-propagation according to the comparison result between the rendered image obtained in step 6 and the corresponding image in the preprocessed scene image set.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] Based on 3DGS, the present invention proposes an optimization method to solve the problem that its point cloud geometric structure cannot fit the reconstructed scene surface well and artifacts appear in rendering at multiple resolutions: (1) The three-dimensional Gaussian ellipsoid is compressed along the direction with the smallest scale factor, effectively flattening the ellipsoid into a plane closest to its original shape. The geometric structure of the two-dimensional Gaussian ellipse solves the problem that the three-dimensional Gaussian ellipsoid cannot fit the scene surface well, thereby reducing the accuracy of scene reconstruction; (2) It proposes to create a scale-adaptive filter in the rendering process, adaptively modify the distribution size of the two-dimensional Gaussian projection, keep the two-dimensional Gaussian projection size consistent at multiple resolutions, thereby removing artifacts and optimizing the rendering effect; (3) Based on the two-dimensional Gaussian point cloud splatter after scale optimization, a supersampling method is introduced to improve the anti-aliasing ability and thus optimize the rendering effect. The present invention improves the original 3DGS method and effectively improves the scene reconstruction rendering effect at multiple resolutions through a two-dimensional Gaussian splatter method based on scale-adaptive filtering. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of the method for optimizing the geometric structure and rendering effect based on 3DGS in this embodiment;

[0027] Figure 2A set of rendering effect comparison images, where (a) is the rendering image obtained using the original 3DGS method, and (b) is the rendering image obtained using the method of the present invention;

[0028] Figure 3 This is another set of rendering effect comparison pictures, where (a) is the rendering picture obtained by the original 3DGS method, and (b) is the rendering picture obtained by the method of the present invention. DETAILED DESCRIPTION

[0029] To facilitate understanding of the present application, the present application will be described more comprehensively below with reference to the relevant drawings.

[0030] Figure 1 This is a flow chart of the method for optimizing the geometric structure and rendering effect based on 3DGS in this embodiment. Figure 1 As shown, the 3DGS-based geometric structure and rendering effect optimization method includes the following steps:

[0031] Step 1: Collect data of the target scene to obtain a scene image set, and perform relevant preprocessing on the data in the scene image set;

[0032] In this embodiment, a visible light camera or drone is selected to capture or record the target scene from multiple angles, circling the center of the scene, to obtain a scene image set. The image set contains at least 30 but no more than 200 consecutive angle-captured images or video slices. Those skilled in the art should ensure the accuracy of camera parameters (such as focal length, distortion coefficient, etc.) during the capture process to facilitate subsequent processing.

[0033] Next, this embodiment performs a memory reduction process on the collected original scene image set, controls the memory occupied by a single image within 5MB, and sets the maximum image resolution to no more than 2K.

[0034] Step 2: Perform sparse reconstruction based on the preprocessed scene image set to restore the sparse point cloud of the scene;

[0035] In this implementation, the preprocessed scene image set is sparsely reconstructed using SfM (Structure from Motion) technology to convert it into a point cloud format suitable for 3DGS training. The preprocessed scene image set and the sparse point cloud with camera parameters are then combined into a single file structure for easy use during training.

[0036] Step 3: Construct a 3D Gaussian point cloud based on the sparse point cloud obtained in step 2, and flatten the 3D Gaussian point cloud to obtain a 2D Gaussian point cloud;

[0037] Step 3.1: Construct a 3D Gaussian point cloud based on the sparse point cloud obtained in step 2;

[0038] The method is as follows: generate a three-dimensional Gaussian ellipsoid corresponding to each point in the sparse point cloud; initialize the parameters of the three-dimensional Gaussian ellipsoid: use the three-dimensional coordinates of a point in the sparse point cloud as the center of the three-dimensional Gaussian ellipsoid corresponding to the point, and initialize the position (p) of the three-dimensional Gaussian ellipsoid; calculate the average distance from the center point of each three-dimensional Gaussian ellipsoid to the center point of the nearest neighbor by the K-nearest neighbor (KNN) algorithm, as the initial radius of the three-dimensional Gaussian ellipsoid; use the covariance matrix (Σ) to record the position information and initial radius of the three-dimensional Gaussian ellipsoid; initialize the opacity (α) of the three-dimensional Gaussian ellipsoid to a fixed value; use spherical harmonics (SH) to represent the color of the three-dimensional Gaussian ellipsoid; and a large number of initialized three-dimensional Gaussian ellipsoids corresponding to the sparse point cloud form a three-dimensional Gaussian point cloud.

[0039] Step 3.2: Flatten the 3D Gaussian point cloud to obtain a 2D Gaussian point cloud;

[0040] In this embodiment, the method for flattening the three-dimensional Gaussian point cloud is as follows: decompose the covariance matrix Σ of the three-dimensional Gaussian ellipsoid into a rotation matrix R and a diagonal scaling matrix S, and satisfy Σ=RSS T R T ; where R is the rotation matrix represented by the quaternion, and the diagonal elements of the diagonal scaling matrix S are s x ,s y ,s z , represents the scaling factor of three dimensions; compress the three-dimensional Gaussian ellipsoid along the direction with the smallest scale factor, specifically in the covariance matrix of the three-dimensional Gaussian ellipsoid, compress the third dimension of the scaling matrix to 0, that is, S=(s x ,s y ,0), extract the third column of the rotation matrix (i.e., the z-axis direction vector after rotation) as the normal direction of the plane where the Gaussian ellipse lies. Flatten the 3D Gaussian ellipsoid into a plane closest to its original shape, i.e., a 2D Gaussian ellipse. Follow the above method to flatten all 3D Gaussian ellipsoids in the 3D Gaussian point cloud into 2D Gaussian ellipses, thus obtaining a 2D Gaussian point cloud formed by a large number of 2D Gaussian ellipses.

[0041] Step 4: 2D Gaussian point cloud projection;

[0042] Projection is a key step in the rendering process. It refers to the process of mapping objects in three-dimensional space onto a two-dimensional plane. The two-dimensional Gaussian point cloud projection process is consistent with 3DGS. Specifically, the two-dimensional Gaussian point cloud projection process is the process of projecting the two-dimensional Gaussian ellipse in the two-dimensional Gaussian point cloud onto the image plane space, as follows:

[0043]

[0044] G(x) is the two-dimensional Gaussian point cloud distribution in the three-dimensional world coordinate system, where x is the function independent variable, representing the initial point of the Gaussian distribution; ∑ represents the position information and rotation and scaling information of the two-dimensional Gaussian point cloud. The new two-dimensional Gaussian distribution located in the two-dimensional image plane coordinate system after projection is ∑′. In formula (2), W is the view transformation matrix, and J is the affine approximation of the projection transformation matrix. Given the view transformation matrix W and the two-dimensional Gaussian point cloud covariance matrix ∑, the radial approximation matrix J, the projected 2D covariance matrix is as follows:

[0045] ∑′=JW∑W T J T (2)

[0046] The center position and color of the projected 2D Gaussian ellipse can be directly obtained from the parameters of the 2D Gaussian point cloud in the 3D world coordinate system. The opacity of the projected 2D Gaussian ellipse needs to be adjusted based on the opacity and covariance matrix of the 2D Gaussian ellipse in the 3D world coordinate system.

[0047] In 3DGS, in order to avoid the degenerate situation where the projected Gaussian point cloud distribution is too small in the image plane, that is, smaller than one pixel, the projected Gaussian point cloud distribution containing k two-dimensional Gaussian ellipses is expanded as follows:

[0048]

[0049] in represents the distribution of the kth two-dimensional Gaussian ellipse in the two-dimensional Gaussian point cloud located in the two-dimensional image plane coordinate system after projection; k represents the number of ellipses traversed in the point cloud; where ∑ k represents the covariance matrix of the k-th two-dimensional Gaussian ellipse; p k represents the position of the kth Gaussian ellipse; I is a two-dimensional identity matrix; s is a scalar dilation hyperparameter, which is 1.64 in this embodiment. This parameter adjusts the scale of the two-dimensional Gaussian in the image plane while keeping its maximum value unchanged, but this leads to non-uniform scale changes under different rendering settings.

[0050] Step 5: Construct a scale-adaptive filter and apply it to the projected two-dimensional Gaussian point cloud:

[0051] The non-uniform scale change of 3DGS during the rendering phase and the artifacts caused by changes in rendering resolution together lead to a decrease in view quality under different rendering parameters. A scale-adaptive scaling factor r = ΔRp / ΔDc is constructed, where ΔRp is the ratio of the resolution of the current rendering camera to the resolution of the training camera; and ΔDc is the ratio of the focal lengths of the current rendering camera and the training camera closest to the rendering camera. By introducing the scale-adaptive scaling factor r, a scale-adaptive filter is created as shown in Equation (4) and applied to the projected 2D Gaussian point cloud. This filter adaptively modifies the size of the 2D Gaussian projection distribution. Under different rendering parameter settings, the Gaussian primitives have consistent scale and distribution in the camera space, thus matching the training settings.

[0052]

[0053] Step 6: Rendering of the 2D Gaussian point cloud after filter adjustment;

[0054] Step 6.1: Two-dimensional Gaussian point cloud projection rasterization: After each two-dimensional Gaussian ellipse projection from near to far, the two-dimensional Gaussian point cloud forms an image area. Rasterization fusion is performed in the overlapping area. After the projections in each area are fused, the image can be obtained.

[0055] The specific process is as follows: given a pixel position x, the distance between the pixel position x and all overlapping two-dimensional Gaussian ellipses is calculated through the view transformation matrix W, that is, a sorted list N of two-dimensional Gaussian ellipses is formed according to the depth of these two-dimensional Gaussian ellipses; image blocks are used instead of pixel-level accuracy, and each image block contains 16*16 pixels; spherical harmonics are used to represent the color c involved in image rendering. k Modeling is performed, and alpha blending is performed based on the depth order 1, .., K of the two-dimensional Gaussian ellipse to obtain the final color information of the entire image. The alpha value represents opacity, and alpha blending calculation is the process of superimposing the opacity values of the two-dimensional Gaussian ellipse to obtain the rendered image. This embodiment creates the alpha blending calculation formula shown in formula (5), which is applied to the pixels after the blocks to calculate the final color information of the rendered image:

[0056]

[0057] In the above formula, represents the j-th two-dimensional Gaussian ellipse distribution in the two-dimensional Gaussian point cloud located in the two-dimensional image plane coordinate system after projection; c(x) represents the final color information of the rendered image; α represents the opacity value, which is superimposed from depth sorting 1 to K and depth sorting 1 to k-1, where k and j are two variables, namely α k is the opacity value corresponding to the depth sort k, α jThe opacity value corresponding to the depth sorting j.

[0058] Step 6.2: Supersample the initial rendered image obtained in step 6.1 to obtain the final rendered image;

[0059] The method is: given a pixel P in a rendered image t , when traversing the ordered classified two-dimensional Gaussian ellipses in the sorted list N, the pixel P is calculated respectively t The distance between the center of each sub-pixel in the next u×u sub-pixels and the center of the traversed two-dimensional Gaussian ellipse, these sub-pixels have independent alpha blending process and cumulative transparency T u . The pixel P in the final rendered image t Color C t It is determined by weighted averaging the spherical harmonic coefficients based on the distance between the sub-pixel and the center of the corresponding two-dimensional Gaussian ellipse, and a clearer rendered image is obtained after supersampling. Where G is the number of two-dimensional Gaussian ellipses, SH i are the spherical harmonic coefficients of the i-th two-dimensional Gaussian ellipse. Function For converting spherical harmonic coefficients to colors:

[0060]

[0061] in Represents the cumulative transparency of the i-th two-dimensional Gaussian ellipse; Indicates the opacity of the i-th two-dimensional Gaussian ellipse; represents the opacity of the th two-dimensional Gaussian ellipse; u is an integer greater than 1, and in this embodiment, u=3.

[0062] Step 7: Based on the principle of the existing 3DGS back-propagation Gaussian adaptive density control scheme, back-propagate and perform 2D Gaussian point cloud optimization based on the comparison results between the final rendered image obtained in step 6 and the corresponding image in the pre-processed scene image set;

[0063] In this implementation, no fundamental changes are made to the back-propagation Gaussian adaptive density control scheme for 3DGS. During the back-propagation optimization process, two-dimensional Gaussian ellipses with large gradients are subject to under-reconstruction and over-reconstruction issues. The gradient is passed without updating any parameters. The parameter group used when constructing the two-dimensional Gaussian point cloud in the three-dimensional world coordinate system is used to determine whether the current two-dimensional Gaussian ellipse is under-reconstructed or over-reconstructed. If so, it is copied or split:

[0064] ① In under-reconstructed regions with a small variance of the 2D Gaussian ellipse, a copy operation is performed: Based on the gradient threshold and the point cloud scaling parameters, a mask is generated to select points. Based on the masked points, new point cloud coordinates, feature parameters, opacity, scaling matrix, and rotation matrix are generated to construct a new point cloud. The newly generated point cloud and features are appended to the original point cloud.

[0065] ② The variance of the two-dimensional Gaussian ellipse in the over-reconstructed area is large, so a splitting operation is performed; obtain the number of two-dimensional Gaussian point clouds in the initial three-dimensional world coordinate system. Create a zero tensor of the same size as the gradient and fill the gradient value into the zero tensor. Based on the threshold condition of the gradient and the scaling factor of the point cloud, a mask that can select points is generated. The points selected based on the mask are copied, and the new point cloud coordinates, feature parameters, opacity, scaling matrix, and rotation matrix are calculated. The newly generated point cloud and features are appended to the original point cloud. Create a filter for pruning, which includes the masks corresponding to the original point cloud and the newly generated point cloud. Delete unnecessary points according to the pruning filter.

[0066] ③ Point pruning: Points with opacity below a certain threshold are subtracted, as are points with scaling factors exceeding a certain threshold. Furthermore, after a certain number of training iterations of all steps following the 3D Gaussian point cloud flattening operation, 2D Gaussian point clouds that are too close to the training camera are set to be almost transparent. This allows for a controlled increase in the necessary Gaussian point cloud density while removing redundant points. A pruning mask is generated based on the minimum opacity and maximum screen size conditions, and the point cloud is pruned.

[0067] Figure 2 and Figure 3 This is a comparison chart of two sets of rendering effects, Figure 2 (a) and (b) are the renderings obtained using the original 3DGS method and the renderings obtained using the method of the present invention, respectively, under the same lens zoom condition. Comparing the two images, it can be found that the object edges in image (b) are more natural and free of artifacts. Figure 3 (a) and (b) are respectively the renderings obtained using the original 3DGS method and the renderings obtained using the method of the present invention under the same camera advance. Comparing the two images, it can be found that the sofa details in Figure (b) are more natural and free of artifacts.

Claims

1. A method for optimizing point cloud geometry and rendering effects based on 3DGS, characterized in that: The method comprises the following steps: Step 1: Collect data of the target scene to obtain a scene image set, and perform relevant preprocessing on the data in the scene image set; Step 2: Perform sparse reconstruction based on the preprocessed scene image set to restore the sparse point cloud of the scene; Step 3: Construct a 3D Gaussian point cloud based on the sparse point cloud obtained in step 2, and flatten the 3D Gaussian point cloud to obtain a 2D Gaussian point cloud; Step 4: 2D Gaussian point cloud projection; Step 5: Create a scale-adaptive filter and apply it to the projected two-dimensional Gaussian point cloud to adaptively modify the size of the two-dimensional Gaussian projection distribution; Step 6: Render the two-dimensional Gaussian point cloud after filter adjustment to obtain a rendered image; Step 7: Based on the comparison results of the rendered image obtained in step 6 and the corresponding image in the preprocessed scene image set, back propagate and perform two-dimensional Gaussian point cloud optimization.

2. The method for optimizing point cloud geometry and rendering effects based on 3DGS according to claim 1, characterized in that: In step 2, SfM technology is used for sparse reconstruction.

3. The method for optimizing point cloud geometry and rendering effect based on 3DGS according to claim 1, characterized in that: The method described in step 3 for constructing a three-dimensional Gaussian point cloud based on the sparse point cloud obtained in step 2 is as follows: generating a three-dimensional Gaussian ellipsoid corresponding one-to-one to the points in the sparse point cloud; initializing the parameters of the three-dimensional Gaussian ellipsoid: using the three-dimensional coordinates of a point in the sparse point cloud as the center of the three-dimensional Gaussian ellipsoid corresponding to the point, and initializing the position of the three-dimensional Gaussian ellipsoid; calculating the average distance from the center point position of each three-dimensional Gaussian ellipsoid to the center point position of the nearest neighbor by the K-nearest neighbor algorithm, as the initial radius of the three-dimensional Gaussian ellipsoid; using the covariance matrix Σ to record the position information and initial radius of the three-dimensional Gaussian ellipsoid; initializing the opacity α of the three-dimensional Gaussian ellipsoid to a fixed value; using spherical harmonics to represent the color of the three-dimensional Gaussian ellipsoid; and forming a three-dimensional Gaussian point cloud with a large number of initialized three-dimensional Gaussian ellipsoids corresponding to the sparse point cloud.

4. The method for optimizing point cloud geometry and rendering effects based on 3DGS according to claim 3, characterized in that: The method for flattening the three-dimensional Gaussian point cloud described in step 3 is as follows: decompose the covariance matrix Σ of the three-dimensional Gaussian ellipsoid into a rotation matrix R and a diagonal scaling matrix S, and satisfy Σ=RSS T R T ; where R is the rotation matrix represented by the quaternion, and the diagonal elements of the diagonal scaling matrix S are s x ,s y ,s z , represents the scaling factor of three dimensions; compress the three-dimensional Gaussian ellipsoid along the direction with the smallest scale factor, specifically, in the covariance matrix of the three-dimensional Gaussian ellipsoid, compress the third dimension of the diagonal scaling matrix to 0, that is, S=(s x ,s y ,0), and extract the third column of the rotation matrix, that is, the z-axis direction vector after rotation, as the normal direction of the plane where the Gaussian ellipse is located, and flatten the three-dimensional Gaussian ellipsoid into a plane closest to its original shape, that is, a two-dimensional Gaussian ellipse; according to the above method, all three-dimensional Gaussian ellipsoids in the three-dimensional Gaussian point cloud are flattened into two-dimensional Gaussian ellipses, thereby obtaining a two-dimensional Gaussian point cloud formed by a large number of two-dimensional Gaussian ellipses.

5. The method for optimizing point cloud geometry and rendering effect based on 3DGS according to claim 3, characterized in that: In step 5, the scale-adaptive scaling factor r is introduced to create the scale-adaptive filter shown in formula (4); The scale adaptation factor r=ΔRp / ΔDc, where ΔRp is the ratio of the resolution of the current rendering camera to the resolution of the training camera; ΔDc is the ratio of the focal lengths of the current rendering camera to the training camera that faces the closest rendering camera; then in represents the distribution of the kth two-dimensional Gaussian ellipse in the two-dimensional Gaussian point cloud located in the two-dimensional image plane coordinate system after projection; x is the function independent variable, representing the initial point of the Gaussian distribution; k represents the number of ellipses in the point cloud; Σ k represents the covariance matrix of the k-th two-dimensional Gaussian ellipse; p k represents the position of the kth Gaussian ellipse; I is a two-dimensional identity matrix; s is a scalar dilation hyperparameter.

6. The method for optimizing point cloud geometry and rendering effects based on 3DGS according to claim 1, characterized in that: The two-dimensional Gaussian point cloud rendering after filter adjustment described in step 6 includes two-dimensional Gaussian point cloud projection rasterization and supersampling the initial rendering image obtained by the two-dimensional Gaussian point cloud projection rasterization to obtain a final rendering image.

7. The method for optimizing point cloud geometry and rendering effect based on 3DGS according to claim 1, characterized in that: In step 7, based on the existing 3DGS back-propagation Gaussian adaptive density control scheme, the two-dimensional Gaussian point cloud is optimized by back-propagation according to the comparison results between the rendered image obtained in step 6 and the corresponding image in the pre-processed scene image set.

Images