Efficient three-dimensional representation primitive generation method based on truncation and tailoring

Abstract

The invention belongs to the field of computer graphics and computer vision, provides an efficient three-dimensional representation primitive generation method based on truncation and tailoring, and aims to improve the expression ability and rendering efficiency of three-dimensional Gaussian splash representation in modeling high-frequency details, sharp boundaries and complex geometric structures. In order to solve the problems of boundary blur and modeling redundancy caused by low-pass filtering characteristics of traditional Gaussian primitives, a novel representation primitive is constructed through spatial intersection of a plurality of coplanar two-dimensional Gaussian primitives, smooth transition is achieved by introducing a parameter learnable boundary attenuation function, and boundary continuity and expression diversity are effectively enhanced. Therefore, while the rendering effect is maintained, the number of elements required by modeling is remarkably reduced, and the recovery capability of high-frequency textures and complex edge structures is improved. The method is suitable for scenes of new view angle synthesis, three-dimensional reconstruction, real-time rendering, augmented / virtual reality, automatic driving, robot vision and the like, and has the advantages of high combinability, high expressive power, low calculation overhead and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer graphics and computer vision, and relates to a high-efficiency three-dimensional representation primitive generation method based on truncated clipping. BACKGROUND

[0002] With the development of neural rendering and three-dimensional reconstruction technology, the explicit rendering method based on Gaussian splatting has been widely used in many fields such as novel view synthesis, virtual reality, augmented reality, robot navigation and digital content production due to its high efficiency and differentiable rendering characteristics. 3D Gaussian Splatting (3DGS) realizes continuous, differentiable and real-time image synthesis by representing the scene as a set of three-dimensional Gaussian distribution bodies with position, scale, direction, color and transparency parameters, and projecting them onto a two-dimensional image plane.

[0003] However, the core feature of the traditional Gaussian primitive, i.e. its kernel function is an anisotropic two-dimensional / three-dimensional Gaussian distribution, has the inherent low-pass filtering property, making it difficult to capture sharp edges, high-frequency textures and complex shape structures in images or geometry during modeling. As shown in FIG. 1, to improve the representation ability, existing methods usually increase the density of Gaussian primitives (such as local stacking at the edge) or introduce additional control points (such as Bezier control points, anchor points, cutting surfaces) to construct complex structures. Figure 1

[0004] These improvements, although to some extent improve the modeling accuracy, but also significantly increase the rendering and storage overhead. For example, multiple Gaussian primitives are stacked in space to approximate curves or sharp edges, which often introduces several times or even orders of magnitude of parameter amount. In addition, the introduction of control points also leads to complex representation structure and slow reasoning speed, making it difficult to meet the dual demands of high precision and real-time performance.

[0005] Therefore, there is an urgent need for a new Gaussian representation method that can effectively improve the boundary expression ability and realize complex structure modeling without significantly increasing the number of primitives and computational cost, in order to balance the expression ability, rendering efficiency and system scalability. SUMMARY

[0006] In view of the problems of insufficient representation ability, low modeling efficiency and high computational cost of existing three-dimensional Gaussian splatting technology in expressing high-frequency details, sharp boundaries and complex geometric shapes, the present application provides a new truncated modified Gaussian primitive, which can enhance the description ability of Gaussian primitive to structural boundaries while maintaining low redundancy, thereby effectively improving the expressiveness and efficiency of three-dimensional scene modeling and image rendering.

[0007] The technical solution of the present application is as follows: ​

[0008] An efficient three-dimensional representation primitive generation method based on truncated clipping, steps as follows:

[0009] First, the definition and construction of composite truncated Gaussian:

[0010] 1.1 Truncated Gaussian construction: The core idea is to introduce a Mahalanobis distance gating function in the two-dimensional image projection space, and form a clear boundary structure by truncating its response range. The truncated Gaussian kernel is defined as follows: TG(p) = exp(-0.5f(p)), if f(p)≤β, else 0. Where f(p) = (p-μ′) T ∑′ -1 (p-μ′), μ′ and ∑′ represent the mean and covariance of the Gaussian distribution in the two-dimensional projection respectively;

[0011] 1.2 Truncated Gaussian coplanar rotation: The coplanar rotation method based on Rodrigues rotation formula is adopted to realize it, which allows the two-dimensional Gaussian distribution to be rotated around the normal vector. Given the unit rotation axis n and the rotation angle ω, the rotation matrix ΔR(n, ω) is expressed as: Where I represents the unit matrix, [n] ×

[0012] is the anti-symmetric cross product matrix of the normal vector. For the truncated Gaussian normal vector r3, the rotation matrix R = [r1, r2, r3] is obtained, and the coplanar splash rotation matrix can be written as:

[0013] Further, the projection of the covariance matrix of the new Gaussian can be calculated by the following formula

[0014] 1.3 Composite truncated Gaussian: As Figure 4 shown, the intersection of multiple coplanar rotated truncated Gaussians at the same position is taken, and the distribution value in the intersection is weighted and averaged to obtain a composite truncated Gaussian:

[0015]

[0016] Second step, adjustable attenuation mechanism: In order to avoid the discontinuity caused by hard truncation and improve the smooth transition ability of the model, the invention further introduces a smooth attenuation function to control the transition process of the boundary region.

[0017] 2.1 Boundary attenuation factor. A simple form of attenuation factor Γ(p) is selected, which is defined as:

[0018] Γ(p) = exp(γ[β-f(p)] 3 ) if f(p)>β else 1

[0019] where γ is a learnable control coefficient. The value of the function at the boundary is set to 1 to achieve seamless transition, while its radial derivative is zero at the boundary to avoid abruptness. The cubic term in the exponential expression [β-f(p)] 3 which guarantees smooth flattening of the function near the boundary, while enabling rapid decay after exceeding the boundary. The parameter γ controls the decay rate: a larger γ will produce a steep but smooth truncation effect, while a smaller γ brings a more moderate decay, equivalent to a low-pass filter;

[0020] 2.2 Adaptive Soft Truncated Gaussian. Based on this decay mechanism, it can be written as:

[0021]

[0022] This expression can dynamically adjust between sharpness and smoothness according to the value of the parameter γ;

[0023] 2.3 Truncated Clipped Gaussian. As Figure 5 shown, the soft truncated Gaussian function is extended to multiple intersecting Gaussian distributions. Based on the previous composite truncated Gaussian, a truncated clipped Gaussian is further defined, whose expression is:

[0024]

[0025] where Γ(p; ω i ) is the decay factor corresponding to the i-th Gaussian direction, and SG(p; ω i ) is its corresponding soft truncated Gaussian function. This formula fuses information from multiple directions and achieves joint boundary control through the product term, thereby balancing the smoothness and continuity of the edge while maintaining the complexity of the shape. As Figure 8 , 9 , 10 shows, when fitting common arc-shaped objects, convex polygons, and non-convex polygons, compared with different primitives, the error value of the primitives generated by the present application is lower than that of other SOTA methods when using the same number of primitives, indicating that the primitive representation capability of the present application method is stronger.

[0026] As Figure 3 shown, the core of the efficient three-dimensional representation primitive generation method based on truncated clipping proposed by the present application is:

[0027] 1. Use multiple two-dimensional primitives to intersect and truncate each other to obtain new primitives;

[0028] 2. Expand the receptive field and sharpness adjustment capability of the new primitive through boundary smoothing decay.

[0029] As Figure 4 shown and Figure 5As shown, compared with traditional 3DGS systems, the truncation and adjustable primitives proposed in this invention have the following advantages:

[0030] 1. Significantly reduces the number of primitives required for modeling;

[0031] 2. Significantly improves the realism of model details;

[0032] The beneficial effects of this invention are:

[0033] This paper proposes a novel representation primitive generation method based on an existing two-dimensional Gaussian representation model, using projection truncation. This method achieves efficient representation of complex shapes and sharp boundaries through Gaussian kernel truncation, significantly improving the accuracy and efficiency of traditional Gaussian splashing methods in high-frequency detail modeling.

[0034] This invention presents a novel Gaussian representation structure with composability, high expressiveness, and low computational burden, which is suitable for various scenarios such as 3D reconstruction, new perspective synthesis, real-time rendering, AR / VR, autonomous driving perception, and robot vision. Attached Figure Description

[0035] Figure 1 This is an existing schematic diagram of primitives.

[0036] Figure 2 This is a schematic diagram of the synthesis principle of new primitives.

[0037] Figure 3 This is an illustration of the core steps of the present invention.

[0038] Figure 4 This is a schematic diagram of the basic shape fitting of the primitives in this invention.

[0039] Figure 5 This is a multi-scale comparison result image of an application example of the present invention.

[0040] Figure 6 This is a schematic diagram of the application of the present invention to the truncation of GSSurfels primitives.

[0041] Figure 7 This is a schematic diagram of the boundary decay function.

[0042] Figure 8 The number of primitives used to fit the ellipse by each method is represented by a line graph showing the error.

[0043] Figure 9 The number of primitives for fitting convex polygons (hexagons) by each method is represented by a line graph of the error.

[0044] Figure 10 The number of primitives used to fit the non-convex polygon (pentagram) by each method is represented by a line graph of the error.

[0045] Figure 11 are the new perspective synthesis visualization results of each method in the outdoor open scene. DETAILED DESCRIPTION

[0046] The application can be applied to various Gaussian primitives, such as 2D GS and GSSurfels distributed in two dimensions, and improves the high-frequency information expression capability of the Gaussian primitives. In order to better understand the technical solutions of the application, the following will take the primitive of GSSurfels as an example, combine the drawings and specific mathematical formulas, and specifically describe the high-efficiency three-dimensional representation primitive generation method based on truncation clipping proposed by the application.

[0047] A high-efficiency three-dimensional representation primitive generation method based on truncation clipping, the steps are as follows:

[0048] First, the definition and construction of the composite truncated Gaussian:

[0049] 1.1 Truncated Gaussian construction: the core idea is to introduce a Mahalanobis distance gating function in the two-dimensional image projection space, form a clear boundary structure by truncating the response range, and the truncated Gaussian kernel is defined as follows: TG(p) = exp(-0.5f(p)), if f(p)≤β, else 0. Where f(p) = (p-μ′) T ∑′ -1 (p-μ′), μ′ and ∑′ represent the mean and covariance of the Gaussian distribution in the two-dimensional projection respectively;

[0050] 1.2 Truncated Gaussian coplanar rotation: the coplanar rotation method based on the Rodrigues rotation formula is adopted to realize it, which allows the two-dimensional Gaussian distribution to be rotated around the normal vector. Given the unit rotation axis n and the rotation angle ω, the rotation matrix ΔR(n, ω) is expressed as: Where, I represents the unit matrix, [n] ×

[0051] is the skew-symmetric cross product matrix of the normal vector. For the truncated Gaussian normal vector r3, the rotation matrix R = [r1, r2, r3] is obtained, and the coplanar rotation matrix can be written as: Further, the projection of the covariance matrix of the new Gaussian can be calculated by the following formula

[0052] 1.3 Composite truncated Gaussian: as shown in the following formula, the intersection of multiple coplanar rotated truncated Gaussians at the same position is taken, and the distribution values in the intersection are weighted and averaged to obtain the composite truncated Gaussian: Figure 4

[0053] ​

[0054] Second step, adjustable attenuation mechanism: in order to avoid the discontinuity caused by hard cutting and improve the smooth transition ability of the model, the present application further introduces a smooth attenuation function to control the transition process of the boundary region.

[0055] 2.1 Boundary attenuation factor. A simple form of attenuation factor Γ(p) is selected, which is defined as:

[0056] Γ(p) = exp(γ[β-f(p)] 3 )if f(p)>βelse1

[0057] Where γ is a learnable control coefficient. The value of the function at the boundary is set to 1 to achieve seamless transition, and its radial derivative is zero at the boundary to avoid sudden changes. The cubic term [β-f(p) 3 in the exponential expression ensures the smooth flattening of the function near the boundary, and can quickly decay after exceeding the boundary. The parameter γ controls the decay rate: a larger γ$ will produce a steep but smooth cutting effect, while a smaller γ will bring a more moderate decay, equivalent to a low-pass filter;

[0058] 2.2 Adaptive soft-cut Gaussian. Based on the attenuation mechanism, it can be written as:

[0059]

[0060] This expression can dynamically adjust between sharpness and smoothness according to the value of the parameter γ;

[0061] 2.3 Truncated clipping Gaussian. As Figure 5 shown, the soft-cut Gaussian function is extended to multiple intersecting Gaussian distributions. Based on the previous composite truncated Gaussian, a truncated clipping Gaussian is further defined, and its expression is:

[0062]

[0063] Where Γ(p;ω i ) is the attenuation factor corresponding to the i-th Gaussian direction, and SG(p;ω i ) is the corresponding soft-cut Gaussian function. This formula integrates information from multiple directions and achieves joint boundary control through the product term, thereby balancing the smoothness and continuity of the edge while maintaining the complexity of the shape. As Figure 8 , 9 , 10 shows that when fitting common arc-shaped objects, convex polygons, and non-convex polygons, the error value of the primitives generated by the present application is lower than that of other SOTA methods when using the same number of primitives, indicating that the primitive representation ability of the present application method is stronger.

[0064] To further illustrate the representation ability of the primitives generated based on the present application, and the superiority of the performance of the present application. The truncated clipped Gaussian (TNT) generated by the present application based on GSSurfels in the above embodiment is used in the standard Gaussian reconstruction process to reconstruct the Family and Horse scenes in the Tanks and Temples dataset. The reconstruction of different scenes is carried out according to the following steps, wherein steps 1 to 6 are the reconstruction process of the standard Gaussian splashing, and the reconstruction primitives used in steps 1 and 5 are the primitives generated by the present method in the embodiment:

[0065] Step 1, initialization: initialize the scene using TNT and colmap, step number j = 0;

[0066] Step 2, calculate the loss function: calculate the loss function using the rendered images and the ground truth images under different perspectives;

[0067] Step 3, optimization: optimize the TNT parameters using the loss function, j = j + 1;

[0068] Step 4, process control: when 500 < j < 15000 and j mod 100 == = 0, jump to step 5; when j > 30000, jump to step 6, otherwise jump to step 2;

[0069] Step 5, densification: select TNT primitives for splitting and cloning operations according to the gradient in the training process. Jump to step 2;

[0070] Step 6, training is completed, save the TNT parameters.

[0071] The reconstruction results are shown in Figure 7 Compared with the SOTA method, the reconstruction results using the primitives generated by the present method have clearer textures and more explicit structures in terms of visual effects.

Claims

1. A method for generating efficient 3D representation primitives based on truncation and clipping, characterized by the following steps: Step 1, definition and construction of composite truncated Gaussians: 1.1 Truncation of Gaussian Structure: The truncated Gaussian kernel is defined as follows: TG(p)=exp(-0.5f(p)),if f(p)≤β,else0. Where f(p)=(p-μ′) T ∑′ -1 (p-μ′), where μ′ and ∑′ represent the mean and covariance of the Gaussian distribution in the two-dimensional projection, respectively; 1.2 Truncation of Gaussian coplanar rotation: Given a unit rotation axis n and a rotation angle ω, the rotation matrix ΔR(n,ω) is expressed as: in, I represents the identity matrix, [n] × It is the antisymmetric cross product matrix of the normal vector; for the truncated Gaussian normal vector r3, it is obtained by the rotation matrix R = [r1, r2, r3]. The coplanar splash rotation matrix is ​​written as: Furthermore, the projection of the new Gaussian covariance matrix is ​​given by the formula Calculated; 1.3 Composite truncated Gaussian: By intersecting multiple coplanar truncated Gaussians at the same position and taking their intersection, and then weighted averaging the values ​​within the intersection, a composite truncated Gaussian is obtained. The second step is an adjustable attenuation mechanism: a smooth attenuation function is introduced to control the transition process in the boundary region. 2.1 Boundary Attenuation Factor; Choose a simple attenuation factor Γ(p), defined as: Γ(p)=exp(γ[β-f(p)] 3 )if f(p)>else 1 Here, γ is a learnable control coefficient; the function is set to 1 at the boundary to achieve a seamless transition, while its radial derivative is zero at the boundary to avoid abrupt changes; The cubic term [β-f(p)] in the exponential expression 3 This ensures the function flattens out smoothly near the boundary and decays rapidly after exceeding the boundary; the parameter γ controls the decay rate. 2.2 Adaptive Soft-Truncation Gaussian; Based on this decay mechanism, it can be written as: This expression dynamically adjusts between sharpness and smoothness based on the value of the parameter γ; 2.3 Truncation-Clipped Gaussian; Extending the soft-truncation Gaussian function to multiple intersecting Gaussian distributions; Based on the previous composite truncation Gaussian, defining the truncation-clipping Gaussian, its expression is as follows: Wherein, Γ(p;ω i ) is the attenuation factor corresponding to the i-th Gaussian direction, SG(p; ω) i ) is its corresponding soft-truncation Gaussian function.

Images