Model appearance synthesis method based on three-dimensional Gaussian and shell mapping

Abstract

The invention relates to a model appearance synthesis method based on three-dimensional Gaussian and shell mapping, in particular to a method for synthesizing an appearance model for a known grid model by utilizing shell mapping and geometric transformation. According to the method, UV mapping and shell mapping are mainly used for processing a three-dimensional Gaussian material, a position corresponding relation in a UVW space and a shell space, rotation, scaling and other parameters, and the three-dimensional Gaussian material is converted into an appearance model of a grid through two times of conversion. The method has the advantages that the appearance model with strong sense of reality can be synthesized for the known grid without manually processing chartlets and materials, a small-scale geometric structure can be changed near the surface of the model, and the visual effect is enhanced.

Description

Technical Field

[0001] The invention relates to the fields of computer graphics and geometric processing, and in particular to a method for synthesizing an appearance model for an area near a surface of a grid model based on three-dimensional Gaussian and shell mapping. Background Art

[0002] A complete 3D model usually includes two aspects: geometry and appearance. The most commonly used method to represent geometry is the mesh model. Modeling appearance based on the mesh model is usually done by texture mapping and setting surface materials. Moreover, in the traditional model production pipeline, geometry and appearance are usually independent, that is, modeling geometry and modeling materials and mapping do not affect each other.

[0003] Texture mapping usually maps a two-dimensional pixel map on the surface of a three-dimensional model to give the model color and surface texture. It usually contains maps such as color reflectivity, specularity, normal, and bumpiness. Material-based appearance modeling usually requires the establishment of a physically rendered bidirectional reflectance distribution function (BRDF) for the model surface, or the adjustment of preset material model parameters to achieve the effect. These appearance modeling methods usually require a lot of labor costs for artists to manually process, or the measurement costs of precision instruments, and cannot be easily obtained from reality. At the scale of details, geometry and appearance are not completely separated. For example, some detailed geometric structures will be replaced by textures because the modeling cost is too high. However, as mentioned above, existing appearance modeling methods are almost completely independent of geometry. Therefore, if a complex appearance needs to be built on a relatively simple geometric model, so that geometric details need to be modified or added to the existing model, the existing methods are relatively inconvenient. Recently, the 3D Gaussian Splatting (3DGS) method provides a new way to reconstruct real 3D objects from photos only. It outputs a point cloud file with attributes such as position, scale, rotation, color, opacity, etc., and reconstructs a 3D model with real color and geometry. This model can actually be used as the material of the appearance model. Shell mapping, as a spatial mapping method, maps the texture space to the space near the model surface, providing technical support for assigning 3D texture to the model surface. Summary of the invention

[0004] The purpose of the present invention is to provide a method for synthesizing the appearance of a known mesh model by using three-dimensional Gaussian as texture material and utilizing shell mapping and other geometric transformations, and this appearance contains geometric structures at a detail scale.

[0005] In order to achieve the above objectives, the technical solution of the present invention is as follows: a model appearance synthesis method based on three-dimensional Gaussian and shell mapping, the method comprising the following steps:

[0006] Input: A mesh with UV mapping, a 3D Gaussian inside a rectangular bounding box.

[0007] The specific steps are as follows:

[0008] S1. Process the 3D Gaussian bounding box and exclude the 3D Gaussian points that are too dispersed.

[0009] S2. triangulate the mesh surface, select the mesh surface, and obtain the corresponding UV bounding box.

[0010] S3. Calculate filling parameters based on the three-dimensional Gaussian bounding box and the UV bounding box, and fill the three-dimensional Gaussian into the augmented UV space.

[0011] S4. Calculate the parameters of the shell space on the model surface according to the bounding box of the three-dimensional Gaussian in the augmented UV space, and expand the shell space on the model surface.

[0012] S5. Calculate synthetic appearance transformation parameters:

[0013] S5.1 Calculate the three-dimensional Gaussian position parameters in shell space based on the shell mapping;

[0014] S5.2 calculates three-dimensional Gaussian rotation parameters based on UV mapping;

[0015] S5.3 Calculate three-dimensional Gaussian scaling parameters based on surface area.

[0016] S6. Fill the shell space with the three-dimensional Gaussian in the augmented UV space to obtain a synthesized mesh model appearance.

[0017] Furthermore, the processing of the three-dimensional Gaussian bounding box in step S1 specifically includes: eliminating Gaussian points with scattered positions by counting the quantiles of the three-dimensional Gaussian position coordinates. The purpose of this step is to ensure that the shape of the three-dimensional Gaussian material is suitable for subsequent processing.

[0018] Furthermore, the mesh surface is selected in step S2, including a method of calculating the mesh surface normal, selecting a partial directional surface, or a method previously marked and specified by other model processing programs.

[0019] Furthermore, the augmented UV space in step S3 refers to a three-dimensional space that removes the U and V coordinates provided by the UV mapping of the mesh model file and adds the height W coordinate, or is called UVW space. The filling parameters calculated in step 3 specifically include the translation, rotation, scaling, and number of copies of the three-dimensional Gaussian. Filling the three-dimensional Gaussian into the augmented UV space specifically includes: modifying the opacity attribute of the three-dimensional Gaussian, calculating the position attribute of the three-dimensional Gaussian after filling through affine transformation and replication, calculating its rotation attribute through quaternion rotation, calculating the scaling attribute of the three-dimensional Gaussian through the scaling coefficient, and calculating the color attribute of the three-dimensional Gaussian through the rotating spherical harmonic function coefficient.

[0020] Modifying the three-dimensional Gaussian opacity specifically includes: specifying the overlap degree p of the copied fill texture overlap , and adjust the three-dimensional Gaussian opacity within the overlapping boundary range, with a coefficient f o As follows

[0021]

[0022] Where SDF(μ) is the value of the signed distance function SDF of the 3D Gaussian center μ on the 3D Gaussian bounding box, and l is the length of the long side of the bounding box. The purpose of this step is to eliminate the mutation caused by the texture material boundary after texture filling.

[0023] Furthermore, the parameter of the model surface shell space to be calculated in step S4 is the shell space thickness, and the specific calculation formula is:

[0024]

[0025] In the formula, h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the bottom area of ​​the shell space, that is, the area of ​​the selected surface in the mesh model, A uv is the bottom area of ​​the three-dimensional Gaussian bounding box in UV space. In step 4, the shell space is expanded on the model surface, specifically, all vertices on the selected surface are expanded toward their outer normal by the thickness h of the shell space. s distance.

[0026] Further, step S5.1 first establishes a shell mapping, and then calculates the three-dimensional Gaussian position parameters based on the shell mapping. The shell mapping is a First, create several prisms in the augmented UV space. The bases of these prisms are model surface triangles located in the w=0 plane. For example, a V A =(u A ,v A,0) T ,V B ,V C The composition of ΔV A V B V c The top surfaces of these triangular prisms are located at w = h uv The corresponding points of the triangle, such as V A ′=(u A ,v A ,h uv ) T ,V B ′,V C ΔV A ′V B ′V c ′. Divide each triangular prism into three tetrahedrons according to the following vertex order, namely

[0027] V A V B V c -V A ′,V A ′V B ′V c ′-V B ,V A ′V B V c -V C ′.

[0028] At any point (u,v,w) in the space covered by these triangular prisms, we can find the tetrahedron V X V Y V Z -V W , and find its center of gravity coordinates in the tetrahedron Make

[0029]

[0030] Each triangular prism in the augmented UV space corresponds to a triangular prism in the shell space that expands from the triangular surface. These triangular prisms are divided into tetrahedrons in the same vertex order. Recovering these barycentric coordinates in the shell space can obtain the position of the point in the shell space.

[0031]

[0032] V Xs Yes V X at the corresponding point in shell space.

[0033] The calculation process of step S5.1 specifically includes the following steps:

[0034] S5.1.1 Locate the surface triangle where the UV coordinates of the three-dimensional Gaussian in each augmented UV space are located, that is, find which triangle of the model it is on when the three-dimensional Gaussian is projected onto the UV plane. Specifically include: calculating the geometric center of the UV bottom triangle, establishing a kd tree of geometric center coordinates, traversing the three-dimensional Gaussian points to find the k nearest triangles, traversing these triangles, solving the linear equations to calculate the barycentric coordinates, and checking whether the barycentric coordinates are legal. Check if a 3D Gaussian is inside a triangle. Remove 3D Gaussians that are not inside any surface triangle.

[0035] S5.1.2 Locate the tetrahedrons where these three-dimensional Gaussians are located and calculate the barycentric coordinates. That is, given the triangular prism where the three-dimensional Gaussian is located, find out which of the three tetrahedrons it is inside. Specifically, it includes: splitting each triangular prism according to the above rules and establishing an index, solving the linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position, and judging whether the barycentric coordinates are legal Returns legal barycenter coordinates and indices.

[0036] S5.1.3 Restore the coordinates of the three-dimensional Gaussian in shell space. Specifically, based on the triangle index and tetrahedron index, use the formula Compute the coordinates in shell space.

[0037] Furthermore, step S5.2 calculates the rotation parameters according to the UV mapping. Specifically, for each surface triangle, the rotation matrix of how to rotate from the UV space to the shell space can be calculated by singular value decomposition and mirror processing:

[0038]

[0039] U,S,V T =SVD(H),

[0040] R=VU T ,

[0041] Where, T uv and T s They are 3×3 matrices, each column is the 3D coordinates of a vertex of a triangle in UV and shell space. SVD is singular value decomposition, R is the rotation matrix. If det(R)<0, mirror processing is required, and the matrix V T The third line is inverted, and then R=VU is recalculated T Finally, each 3D Gaussian rotation parameter is assigned according to the index of the triangle where the 3D Gaussian is located.

[0042] Further, step S5.3 calculates the scaling parameter according to the surface area, specifically including: 5.3.1 Scaling coefficient

[0043]

[0044] The variables in the formula are the same as those in step S4; 5.3.2 The scaling factor required for shell space expansion is calculated as follows:

[0045]

[0046] Where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the area of ​​the selected surface in the mesh model, A′ s is the outer surface area of ​​the shell formed after the vertex expansion in step 4. Therefore, the scaling factor of each 3D Gaussian is

[0047] S=S uv→s ·S Se .

[0048] Furthermore, step S6 transforms the three-dimensional Gaussian using the parameters calculated in steps S3, S4, and S5 to synthesize the appearance model in the shell space. Specifically, it includes: assigning the three-dimensional Gaussian position attribute, that is, filling the parameters obtained in step S5.1 into the three-dimensional Gaussian model; assigning other attributes to the three-dimensional Gaussian, firstly copying the other attributes of the three-dimensional Gaussian material except the position according to the number of copies and fills in step S2, and converting the opacity and scaling attributes according to the following formula:

[0049]

[0050]

[0051] o, s are the opacity and scaling property values ​​after conversion, x o ,x s is the opacity and scaling attribute value in the 3D Gaussian PLY file. Multiply the opacity by the coefficient f in step S3 O Modification, scaling is calculated by multiplying the coefficient S of step S5.3, rotation is calculated by the rotation matrix R of S5.2, the color is first reorganized into a spherical harmonic function coefficient matrix, and then the coefficient corresponding to the rotation matrix R is adjusted to complete the rotation. After the above steps, the synthesized appearance model is obtained, which is still in a three-dimensional Gaussian form.

[0052] The benefits of this method are as follows: 1. The present invention converts the powerful ability of 3D Gaussian to restore reality into the ability to model the appearance of the mesh model, and can directly obtain the appearance model from reality without the need for artists to manually process maps and materials. That is, this method does not require modelers to edit details on 3D modeling software. Human work only requires cutting the 3D Gaussian once before input, and collecting videos for 3D Gaussian modeling, while our method uses 3D Gaussian as input. Subsequent steps are all completed automatically by computer programs, which can greatly reduce the workload of modelers. 2. The appearance model established by this method has a strong sense of reality in color and texture visual effects, and can have small-scale geometry. It has the ability to change the geometry near the model surface that traditional maps and materials do not have, that is, to show the bumps and holes near the surface, and can even add complex small geometric shapes, such as adding petals, fluff, and dandruff on the surface of the object to further enhance the visual effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The following is the overall flow chart;

[0054] Figure 2 For example result 1, the input mesh model is sofa 1, and the input 3DGS is four types: leather, cloth fiber, flower, and bread;

[0055] Figure 3 For example result 2, the input mesh model is sofa 2, and the input 3DGS is four types: bread, grass, embossed, and flower;

[0056] Figure 4 For example result 3, there are two input mesh models: vase 1 and vase 2, and the input 3DGS is flowers;

[0057] Figure 5 For example result 4, two 3DGS are assigned to different surfaces of the same mesh model. DETAILED DESCRIPTION

[0058] The following is a specific implementation case with reference to the accompanying drawings. Figure 1 As shown in the process, the mesh model of the sofa and the 3DGS model of a piece of bread are input and processed according to this method.

[0059] We execute step 1 to process the 3DGS point cloud first: the length, width and height of the bounding box of the 3DGS point cloud are respectively in the positive direction of the X, Y and Z axes, the quantiles are set to 5% and 95%, the three-dimensional Gaussian position coordinates are counted, and the scattered Gaussian points outside the 5% or 95% of the boundary are excluded to ensure that the shape of the three-dimensional Gaussian material is suitable for subsequent processing.

[0060] Then we perform step 2 to process the mesh model: triangulate its surface, and then read its UV map for surface selection. Figure 1 All surfaces are selected in the process.

[0061] Then we execute step 3, as shown in the figure. The augmented UV space refers to the three-dimensional space with the height W coordinate added instead of the U and V coordinates. Calculate the filling parameters, translation and rotation of the three-dimensional Gaussian: move a vertex of the 3DGS bounding box to the position (u,v,w)=(0,0,0), and specify to rotate the positive direction of the XYZ axis to the positive direction of UVW (or the corresponding order of the reverse or scroll axis); scale and copy the number of fill copies: if the U-axis fill number is specified as 4, calculate the scaling X-axis coefficient to satisfy that copying the original 3DGS four times can fill the length of the bounding box in the U-axis direction, calculate the number of V-axis fill copies under this scaling coefficient, calculate the Y-axis scaling coefficient after rounding, and average the XY-axis scaling coefficients to calculate the Z-axis scaling coefficient. Fill the three-dimensional Gaussian into the augmented UV space and modify the three-dimensional Gaussian opacity attribute: specify the overlap p of the copied fill texture. overlap 5%, and adjust the three-dimensional Gaussian opacity within the overlapping boundary as follows

[0062]

[0063] Where SDF(μ) is the value of the signed distance function SDF of the 3D Gaussian center μ on the 3D Gaussian bounding box, and l is the length of the long side of the bounding box, so as to eliminate the mutation caused by the texture material boundary after texture filling. Then, through affine transformation and replication, the position attribute of the 3D Gaussian after filling is calculated, the rotation attribute is calculated through quaternion rotation, the scaling attribute of the 3D Gaussian is calculated through the scaling coefficient, and the color attribute of the 3D Gaussian is calculated through the rotation spherical harmonic function coefficient.

[0064] Then, the parameter of the model surface shell space calculated in step 4 is the shell space thickness, as shown in the following formula:

[0065]

[0066] In the formula, h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the bottom area of ​​the shell space, that is, the area of ​​the selected surface in the mesh model, A uv is the bottom area of ​​the three-dimensional Gaussian bounding box in UV space. Then expand the shell space on the model surface, and expand the shell space thickness h of all vertices on the selected surface toward their outward normal. s distance.

[0067] Then execute step 5.1 to establish the shell mapping, and calculate the three-dimensional Gaussian position parameters based on the shell mapping. The shell mapping is a First, create several prisms in the augmented UV space. The bases of these prisms are model surface triangles located in the w=0 plane. For example, a V A =(u A ,v A ,0) T ,V B ,V C The composition of ΔV A V B V c The top surfaces of these triangular prisms are located at w = h uv The corresponding points of the triangle, such as V A ′=(u A ,v A ,h uv ) T ,V B ′,V C ΔV A ′V B ′V c '. Divide each triangular prism into three tetrahedrons according to the following vertex order (the flowchart shows that the triangular prism is divided into tetrahedrons), namely

[0068] V A V B V c -V A ′,V A ′V B ′V c ′-V B ,V A ′V B V c -V C ′.

[0069] At any point (u,v,w) in the space covered by these triangular prisms, we can find the tetrahedron V X V Y V Z -V W , and find its center of gravity coordinates in the tetrahedron Make

[0070]

[0071] Each triangular prism in the augmented UV space corresponds to a triangular prism in the shell space that expands from the triangular surface. These triangular prisms are divided into tetrahedrons in the same vertex order. Recovering these barycentric coordinates in the shell space can obtain the position of the point in the shell space.

[0072]

[0073] V Xs Yes V X at the corresponding point in shell space.

[0074] Follow the steps below to perform the calculation: 5.1.1 Locate the surface triangle where the UV coordinates of the 3D Gaussian in each augmented UV space are located, that is, find which triangle of the model it is on when the 3D Gaussian is projected onto the UV plane. Calculate the geometric center of the UV bottom triangle, establish a kd tree of geometric center coordinates, traverse the 3D Gaussian points to find the k nearest triangles, traverse these triangles, solve the linear equations to calculate the barycentric coordinates, and check whether the barycentric coordinates are legal. Check if a 3D Gaussian is inside a triangle. Remove 3D Gaussians that are not inside any surface triangle.

[0075] 5.1.2 Locate the tetrahedrons where these three-dimensional Gaussians are located and calculate the barycentric coordinates at the same time. That is, given the triangular prism where the three-dimensional Gaussian is located, find out which of the three tetrahedrons it is inside. Split each triangular prism according to the shell mapping rule and create an index, solve the linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position, and determine whether the barycentric coordinates are legal Returns legal barycenter coordinates and indices.

[0076] 5.1.3 Recover the coordinates of the three-dimensional Gaussian in shell space. According to the triangle index and tetrahedron index, use the formula Compute the coordinates in shell space.

[0077] Then execute step 5.2 to calculate the rotation parameters based on the UV mapping. For each surface triangle, the rotation matrix from UV space to shell space can be calculated by singular value decomposition and mirror processing according to the following formula:

[0078]

[0079] U,S,V T =SVD(H),

[0080] R=VU T ,

[0081] Where, T uv and T s They are 3×3 matrices, each column is the 3D coordinates of a vertex of a triangle in UV and shell space. SVD is singular value decomposition, R is the rotation matrix. If det(R)<0, mirror processing is required, and the matrix V T The third line is inverted, and then R=VU is recalculated TFinally, each 3D Gaussian rotation parameter is assigned according to the index of the triangle where the 3D Gaussian is located.

[0082] Then perform step 5.3 to calculate the scaling factor based on the surface area. 5.3.1 Calculate the scaling factor

[0083]

[0084] The variables in the formula are the same as those in step 4; 5.3.2 The scaling factor required for shell space expansion is calculated as follows:

[0085]

[0086] Where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the area of ​​the selected surface in the mesh model, A′ s is the outer surface area of ​​the shell formed after the vertex expansion in step 4. Therefore, the scaling factor of each 3D Gaussian is

[0087] S=S uv→s ·S se .

[0088] Finally, execute step 6, and transform the 3D Gaussian using the parameters calculated in steps 3, 4, and 5 to synthesize the appearance model in the shell space. Specifically, it includes: assigning the 3D Gaussian position attribute, that is, filling the parameters obtained in step 5.1 into the 3D Gaussian model; assigning other attributes to the 3D Gaussian, first copying the other attributes of the 3D Gaussian material except the position according to the number of copies and fills in step 2, and converting the opacity and scaling attributes according to the following formula:

[0089]

[0090] o, s are the opacity and scaling property values ​​after conversion, x o ,x s is the opacity and scaling attribute value in the 3D Gaussian PLY file. Multiply the opacity by the coefficient f in step S3 O Modification, scaling is calculated by multiplying the coefficient S of step S5.3, and the rotation is calculated by the rotation matrix R of S5.2. The color is first reorganized into a spherical harmonic function coefficient matrix, and then the coefficients corresponding to the rotation matrix R are adjusted to complete the rotation.

[0091] After the above steps, the synthesized appearance model is obtained, which is still in the form of a three-dimensional Gaussian. As shown in the figure, the sofa with a bread appearance surface not only has the color appearance of bread, but also has a porous geometric surface on the bread surface, which is provided by 3DGS.

[0092] Case 2: The implementation process is similar to Case 1, and the results are as follows: Figure 2 As shown in Figure 2, please note that the appearance geometry of the appearance model in the result is significantly different from that of the mesh model. This is due to the fact that the method of the present invention successfully utilizes the ability of 3DGS to change the mesh surface geometry. Case 3, the implementation process is similar to that of Case 1, and the results are shown in Figure 2. Figure 3 As shown, please also note the obvious difference between the appearance geometry of the appearance model and the mesh model in the result.

[0093] Case 4: In step 2, instead of selecting all surfaces, we first calculate the surface normal vectors, select the surfaces that are within -30 to +30 degrees from the positive direction of the X-axis to perform the flower 3DGS subsequent steps, select other surfaces to perform the embossing 3DGS subsequent steps, and finally merge their results. The final result is as follows: Figure 4 shown.

[0094] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A model appearance synthesis method based on three-dimensional Gaussian and shell mapping, characterized in that: The following steps are involved: S1. Process the 3D Gaussian bounding box and exclude the 3D Gaussian points that are too dispersed. S2. Triangulate the mesh surface, select the mesh surface, and obtain the corresponding UV bounding box. S3. Calculate the filling parameters from the 3D Gaussian bounding box and the UV bounding box, and fill the 3D Gaussian into the augmented UV space. S4. Calculate the parameters of the shell space on the model surface according to the bounding box of the three-dimensional Gaussian in the augmented UV space, and expand the shell space on the model surface. S5. Calculate synthetic appearance transformation parameters: S5.1 Calculate the three-dimensional Gaussian position parameters in shell space based on the shell mapping; S5.2 calculates three-dimensional Gaussian rotation parameters based on UV mapping; S5.3 Calculate the three-dimensional Gaussian scaling parameters based on the surface area, S6. Fill the shell space with the three-dimensional Gaussian in the augmented UV space to obtain a synthesized mesh model appearance.

2. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: The processing of the three-dimensional Gaussian bounding box in step S1 specifically includes: eliminating Gaussian points with scattered positions by counting the quantiles of the three-dimensional Gaussian position coordinates, so as to ensure that the shape of the three-dimensional Gaussian material is suitable for subsequent processing.

3. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1 is characterized in that: The mesh surface is selected in step S2, including calculating the mesh surface normal, selecting a partial directional surface method, or a method previously marked and specified by other model processing programs.

4. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: The filling parameters calculated in step 3 specifically include the translation, rotation, scaling, and number of copies of the three-dimensional Gaussian. The three-dimensional Gaussian is filled into the augmented UV space, specifically including: modifying the opacity attribute of the three-dimensional Gaussian, calculating the position attribute of the three-dimensional Gaussian after filling through affine transformation and replication, calculating its rotation attribute through quaternion rotation, calculating the scaling attribute of the three-dimensional Gaussian through the scaling coefficient, and calculating the color attribute of the three-dimensional Gaussian through the rotating spherical harmonic function coefficient. Modifying the three-dimensional Gaussian opacity specifically includes: specifying the overlap degree p of the copied fill texture overlap , and adjust the three-dimensional Gaussian opacity within the overlapping boundary range as follows Where SDF(μ) is the value of the three-dimensional Gaussian center μ on the signed distance function SDF of the three-dimensional Gaussian bounding box, and l is the length of the long side of the bounding box. This step eliminates the mutation caused by the texture material boundary after texture filling.

5. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: The parameter of the model surface shell space to be calculated in step S4 is the shell space thickness, and the specific calculation formula is: In the formula, h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the bottom area of ​​the shell space, that is, the area of ​​the selected surface in the mesh model, A uv is the bottom area of ​​the three-dimensional Gaussian bounding box in UV space. In step 4, a shell space is expanded on the model surface. Specifically, all vertices on the selected surface are expanded toward their outer normals by a shell space thickness h. s distance.

6. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S5.1 first establishes a shell map, and then calculates the three-dimensional Gaussian position parameters based on the shell map. The shell map is a The mapping is to input the (u, v, w) coordinates of an augmented UV space and output the (x, y, z) coordinates of a shell space. First, several triangular prisms are established in the augmented UV space. The bottom surfaces of these triangular prisms are the model surface triangles located in the w=0 plane. Recovering these barycentric coordinates in shell space gives the position of the point in shell space V Xs Yes V X At the corresponding point in shell space, are the barycentric coordinates.

7. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 6, characterized in that: The calculation process of step S5.1 specifically includes the following steps: S5.1.1 Locate the surface triangle where the UV coordinates of the three-dimensional Gaussian in each augmented UV space are located, that is, find which triangle of the model it is on when the three-dimensional Gaussian is projected onto the UV plane. Specifically, it includes: calculating the geometric center of the UV bottom triangle, establishing a kd tree of geometric center coordinates, traversing the three-dimensional Gaussian points to find the k nearest triangles, traversing these triangles, solving the linear equations to calculate the barycentric coordinates, and checking whether the barycentric coordinates are legal. Check if the 3D Gaussian is inside a triangle, remove the 3D Gaussian that is not inside any surface triangle, S5.1.2 Locate the tetrahedrons where these three-dimensional Gaussians are located and calculate the barycentric coordinates at the same time. That is, given the triangular prism where the three-dimensional Gaussian is located, find out which of the three tetrahedrons it is inside. Specifically, it includes: splitting each triangular prism according to the above rules and establishing an index, solving the linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position, and judging whether the barycentric coordinates are legal Returns the legal center of gravity coordinates and index, S5.1.3 Recover the coordinates of the three-dimensional Gaussian in shell space, specifically including: Based on the triangle index and tetrahedron index, use the formula Compute the coordinates in shell space.

8. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S5.2 calculates the rotation parameters according to the UV mapping. Specifically, for each surface triangle, the rotation matrix of how to rotate from UV space to shell space can be calculated by singular value decomposition and mirror processing: U,S,V T =SVD(H), R=VU T , Where, T uv and T s They are 3×3 matrices, each column is the three-dimensional coordinates of a vertex of a triangle in UV and shell space, SVD is singular value decomposition, R is the rotation matrix, if det(R)<0, mirror processing is required, and the matrix V T The third line is inverted, and then R=VU is recalculated T , and finally assign each 3D Gaussian rotation parameter according to the index of the triangle where the 3D Gaussian is located.

9. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S5.3 calculates the scaling parameters based on the surface area, specifically including: 5.3.1 Scaling coefficient The variables in the formula are the same as those in step S4; 5.3.2 The scaling factor required for shell space expansion is calculated as follows: Where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the area of ​​the selected surface in the mesh model, A′ s is the outer surface area of ​​the shell formed after the vertex expansion in step 4, so the scaling factor of each three-dimensional Gaussian is S=S uv→s ·S se 。 10. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S6 transforms the three-dimensional Gaussian using the parameters calculated in steps S3, S4, and S5 to synthesize the appearance model in the shell space, specifically including: assigning the three-dimensional Gaussian position attribute, that is, filling the parameters obtained in step S5.1 into the three-dimensional Gaussian model; assigning other attributes to the three-dimensional Gaussian, first copying the other attributes of the three-dimensional Gaussian material except the position according to the number of copies and fills in step S2, and converting the opacity and scaling attributes according to the following formula: o, s are the opacity and scaling property values ​​after conversion, x o ,x s is the opacity and scaling attribute value in the 3D Gaussian PLY file. Multiply the opacity by the coefficient f in step S3. O Modification, scaling is calculated by multiplying the coefficient S of step S5.3, and the rotation is calculated by the rotation matrix R of S5.

2. The color is first reorganized into a spherical harmonic function coefficient matrix, and then the coefficients corresponding to the rotation matrix R are adjusted to complete the rotation.

Images