Model Appearance Synthesis Method Based on 3D Gaussian and Shell Mapping

Abstract

The present invention relates to a method for synthesizing a model appearance based on three-dimensional Gaussian and shell mapping, which is a method for synthesizing an appearance model for a known mesh model by using shell mapping and geometric transformation. This method mainly uses UV mapping and shell mapping to process the positional correspondence, rotation, scaling and other parameters in the three-dimensional Gaussian material, UVW space and shell space, and transforms the three-dimensional Gaussian material into the appearance model of the mesh through two transformations. The advantage of this method is that it can synthesize a realistic appearance model for a known mesh without manual processing of textures and materials, and can change the small-scale geometric structure near the model surface to enhance the visual effect.

Description

Technical Field

[0001] The present invention relates to the fields of computer graphics and geometric processing, and particularly to a method for synthesizing an appearance model for the area near the surface of a mesh model based on 3D Gaussian and shell mapping. Background Art

[0002] A complete 3D model usually includes two aspects: its geometry and appearance. The most commonly used geometric structure representation method is the Mesh model. Modeling the appearance based on the mesh model is usually completed by means of texture mapping and setting surface materials. Moreover, in the traditional model production pipeline, geometry and appearance are usually independent, that is, modeling geometry and modeling materials and textures do not affect each other.

[0003] Texture mapping usually maps a 2D pixel map on the surface of a 3D model to endow the model with color and surface texture. It usually includes maps such as color reflectance, specular degree, normal, and bumpiness. Appearance modeling based on materials usually requires establishing a bidirectional reflectance distribution function (BRDF) rendered based on a physical method for the model surface, or achieving the effect by adjusting the parameters of a preset material model. These appearance modeling methods usually require a large amount of human cost of manual processing by artists, or the measurement cost of precision instruments, and cannot be conveniently obtained from reality. At the detail scale, geometry and appearance are not completely separated. For example, some detailed geometric structures will be approximately replaced by textures due to the too high modeling cost. However, as mentioned above, the existing appearance modeling methods are almost completely independent of geometry. Therefore, if it is necessary to establish a complex appearance on a relatively simple geometric model, that is, to modify or add geometric details to an existing model, the existing methods are relatively inconvenient. Recently, the 3D Gaussian (3DGaussianSplatting, abbreviated as 3DGS) method provides a new way to reconstruct 3D objects in reality only from photos. It outputs a point cloud file with attributes such as position, scale, rotation, color, and opacity, and reconstructs a 3D model with both real colors and geometry. This model can actually be used as the material for the appearance model. As a spatial mapping method, shell mapping maps the texture space to the space near the model surface, providing technical support for endowing the 3D texture to the model surface. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for synthesizing an appearance for a known mesh model using 3D Gaussian as texture material, utilizing shell mapping and other geometric transformations, and this appearance includes geometric structures at the detail scale.

[0005] To achieve the above object, the technical solution of the present invention is as follows: A method for synthesizing the appearance of a model based on 3D Gaussian and shell mapping, the method comprising the following steps:

[0006] Input: A mesh model with UV mapping and a 3D Gaussian material located within a rectangular bounding box.

[0007] The specific steps are as follows:

[0008] S1. Process the 3D Gaussian bounding box to exclude 3D Gaussian points that are too scattered in position.

[0009] S2. Triangulate the mesh surface, select the mesh surface, and obtain the corresponding UV bounding box.

[0010] S3. Calculate the filling parameters from the 3D Gaussian bounding box and the UV bounding box, and fill the 3D Gaussian into the augmented UV space.

[0011] S4. Calculate the parameters of the model surface shell space based on the bounding box of the 3D Gaussian in the augmented UV space, and inflate the shell space on the model surface.

[0012] S5. Calculate the synthetic appearance transformation parameters:

[0013] S5.1 Calculate the 3D Gaussian position parameters in the shell space according to the shell mapping;

[0014] S5.2 Calculate the 3D Gaussian rotation parameters according to the UV mapping;

[0015] S5.3 Calculate the 3D Gaussian scaling parameters according to the surface area.

[0016] S6. Fill the 3D Gaussian in the augmented UV space into the shell space to obtain the synthetic mesh model appearance.

[0017] Furthermore, the processing of the 3D Gaussian bounding box in step S1 specifically includes: excluding the scattered Gaussian points by statistically calculating the quantiles of the 3D Gaussian position coordinates. The purpose of this step is to ensure that the shape of the 3D Gaussian material is suitable for subsequent processing.

[0018] Furthermore, the selection of the mesh surface in step S2 includes calculating the mesh surface normal, the method of selecting partial direction surfaces, or the method marked and specified in advance by other model processing programs.

[0019] Further, the augmented UV space in step S3 refers to a three-dimensional space that removes the U and V coordinates provided by the UV mapping of the mesh model file and adds a height W coordinate, or is also called the UVW space. The filling parameters calculated in step 3 specifically include the translation, rotation, scaling, and copy filling times of the three-dimensional Gaussian. Filling the three-dimensional Gaussian into the augmented UV space specifically includes: modifying the opacity attribute of the three-dimensional Gaussian, calculating the position attribute after filling the three-dimensional Gaussian through affine transformation and copying, calculating its rotation attribute through quaternion rotation, calculating the scaling attribute of the three-dimensional Gaussian through the scaling coefficient, and calculating the color attribute of the three-dimensional Gaussian through the rotation spherical harmonic function coefficient.

[0020] Among them, modifying the opacity of the three-dimensional Gaussian specifically includes: specifying the overlap degree p overlap , and adjusting the opacity of the three-dimensional Gaussian within the overlapping boundary range, with its coefficient f O as follows

[0021]

[0022] In the formula, SDF(μ) is the value of the three-dimensional Gaussian center μ on the signed distance function SDF of the three-dimensional Gaussian bounding box, and l is the length of the long side of the bounding box. The purpose of this step is to eliminate the mutation caused by the texture material boundary after texture filling.

[0023] Further, the parameter of the model surface shell space to be calculated in step S4 is the shell space thickness, and the specific calculation formula is:

[0024]

[0025] In the formula, h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in the UV space, A s is the bottom area of the shell space, that is, the area of the selected surface in the mesh model, A uv is the bottom area of the three-dimensional Gaussian bounding box in the UV space. In step 4, expanding the shell space on the model surface specifically means expanding all the vertices on the selected surface by a distance of the shell space thickness h s .

[0026] Further, in step S5.1, first establish a shell mapping, and then calculate the three-dimensional Gaussian position parameters according to the shell mapping. This shell mapping is a mapping, that is, inputting the (u, v, w) coordinates of the augmented UV space and outputting the (x, y, z) coordinates of the shell space. First, establish several triangular prisms in the augmented UV space. The bottom surfaces of these triangular prisms are the model surface triangles located on the w = 0 plane. For example, a triangle with V A =(u A , v A,0) T ,V B ,V C composed of ΔV A V B V C , the top surfaces of these triangular prisms are triangles composed of corresponding points located at w = h uv , such as V A ′ = (u A , v A , h uv ). T , V B ′, V C ′ composed of ΔV A ′V B ′V C ′. Each triangular prism is divided into three tetrahedrons according to the following vertex order, that is

[0027] V A V B V C -V′ A , V′ A V′ B V′ C -V B , V′ A V B V C -V′ C ,

[0028] At any point (u, v, w) in the space covered by these triangular prisms, the tetrahedron V X V Y V Z -V W where it is located can be found, and the centroid coordinates of this point within the tetrahedron can be calculated such that

[0029]

[0030] Each triangular prism in the augmented UV space corresponds to a triangular prism expanded from the surface of this triangle in the shell space. These triangular prisms are divided into tetrahedrons in the same vertex order. By restoring these centroid coordinates in the shell space, the position of this point in the shell space can be obtained

[0031]

[0032] V Xs is the corresponding point of V X in the shell space.

[0033] The specific calculation process of step S5.1 includes the following steps:

[0034] S5.1.1 Locate the surface triangle where the UV coordinates of the three-dimensional Gaussian in each augmented UV space are located, that is, find which triangular patch of the model the three-dimensional Gaussian projects onto when projected onto the UV plane. Specifically, it includes: calculating the geometric center of the UV bottom triangle, building a geometric center coordinate k-d tree, traversing the three-dimensional Gaussian points to find the k nearest triangles, traversing these triangles, solving a system of linear equations to calculate the barycentric coordinates, and checking whether the barycentric coordinates are legal Check whether the three-dimensional Gaussian is inside the triangle. Delete the three-dimensional Gaussian that is not inside any surface triangle.

[0035] S5.1.2 Locate the tetrahedron where these three-dimensional Gaussians are located and calculate the barycentric coordinates at the same time, that is, given the triangular prism where the three-dimensional Gaussian is located, find which one of the three tetrahedrons it is inside. Specifically, it includes: splitting each triangular prism according to the above rules and building an index, solving a system of linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position, and judging whether the barycentric coordinates are legal Return the legal barycentric coordinates and the index.

[0036] S5.1.3 Restore the coordinates of the three-dimensional Gaussian in the shell space. Specifically, it includes: according to the triangle index and the tetrahedron index, using the formula Calculate the coordinates in the shell space.

[0037] Furthermore, step S5.2 calculates the rotation parameters according to the UV mapping. Specifically, for each surface triangle, the rotation matrix for rotating from the UV space to the shell space can be calculated through singular value decomposition and mirror processing:

[0038]

[0039] U, S, V T = SVD(H),

[0040] R = VU T ,

[0041] In the formula, T uv and T s are 3×3 matrices respectively, and each column is the three-dimensional coordinates of a vertex of the triangle in the UV space and the shell space. SVD is the singular value decomposition, and R is the rotation matrix. If det(R) < 0, mirror processing is required. Take the negative of the third row of the matrix V T , and then recalculate R = VU T . Finally, according to the index of the triangle where the three-dimensional Gaussian is located, assign the rotation parameters to each three-dimensional Gaussian.

[0042] Furthermore, step S5.3 calculates the scaling parameters according to the surface area, specifically including: 5.3.1 Scaling coefficient

[0043]

[0044] The variables in the formula are the same as those in step S4; 5.3.2 Scaling factor required for the shell space expansion, calculated as follows:

[0045]

[0046] where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the three-dimensional Gaussian bounding box in the UV space, A s is the area of the selected surface in the mesh model, A′ s is the outer surface area of the shell formed after vertex inflation in step 4. Therefore, the scaling factor for each three-dimensional Gaussian is

[0047] S = S uv→s ·S se

[0048] Furthermore, step S6 uses the parameters calculated in steps S3, S4, and S5 to transform the three-dimensional Gaussian and synthesize the appearance model in the shell space. Specifically, it includes: assigning the position attribute to the three-dimensional Gaussian, that is, filling the parameters obtained in step S5.1 into the three-dimensional Gaussian model; assigning other attributes to the three-dimensional Gaussian. First, copy and fill the other attributes of the three-dimensional Gaussian material except the position according to the copy and fill number in step S2, and convert the opacity and scaling attributes according to the following formula:

[0049]

[0050] o, s are the converted opacity and scaling attribute values, x o , x s are the opacity and scaling attribute values in the three-dimensional Gaussian PLY file. Multiply the opacity by the coefficient f of step S3 O for modification, multiply the scaling by the coefficient S of step S5.3 for calculation, calculate the rotation by the rotation matrix R of S5.2, and first reorganize the color into a spherical harmonic function coefficient matrix, and then perform coefficient adjustment corresponding to the rotation matrix R to complete the rotation. After the above steps, the synthesized appearance model is obtained, and this appearance model is still in the form of a three-dimensional Gaussian.

[0051] The advantages of this method are as follows: 1. The present invention converts the powerful reality restoration ability of the three-dimensional Gaussian into the ability to model the appearance of a mesh model, enabling the direct acquisition of the appearance model from reality without the need for artists to manually process textures and materials. That is, this method does not require modelers to edit details in 3D modeling software. The work of humans only involves performing a single cropping on the three-dimensional Gaussian before input and collecting the video for three-dimensional Gaussian modeling, and our method uses the three-dimensional Gaussian as the input. The subsequent steps are all automatically completed by computer programs, which can greatly reduce the workload of modelers. 2. The appearance model established by this method has a strong sense of reality in terms of color and texture visual effects, and can also carry small-scale geometries. It has the ability to change the geometry near the model surface, which traditional textures and materials do not possess, that is, it can represent the bumps, holes near the surface, and even add complex small geometric shapes, such as adding petals, fluff, and dandruff on the object surface to further enhance the visual effect. Description of the Drawings

[0052] Figure 1 is the overall flowchart;

[0053] Figure 2 is Example Result 1, the input mesh model is Sofa 1, and the input 3DGS are four types: leather, cloth fiber, flower, and bread;

[0054] Figure 3 is Example Result 2, the input mesh model is Sofa 2, and the input 3DGS are four types: bread, grassland, embossed pattern, and flower;

[0055] Figure 4 is Example Result 3, the input mesh models are two types: Vase 1 and Vase 2, and the input 3DGS is flower;

[0056] Figure 5 is Example Result 4, where two types of 3DGS are assigned to different surfaces of the same mesh model. Detailed Implementation Manner

[0057] The following combines the drawings to illustrate specific implementation cases. Case 1, as Figure 1 shown in the process, input the mesh model of the sofa and the 3DGS model of a piece of bread, and process it according to this method.

[0058] We first execute Step 1 to process the 3DGS point cloud: Align the length, width, and height of the bounding box of the 3DGS point cloud with the positive directions of the X, Y, and Z axes respectively, set the quantiles to 5% and 95%, count the position coordinates of the three-dimensional Gaussian, and exclude the Gaussian points scattered outside the 5% or 95% of the boundary to ensure that the shape of the three-dimensional Gaussian material is suitable for subsequent processing.

[0059] Then we execute Step 2 to process the mesh model: Triangulate its surface, and then read its UV mapping for surface selection.Figure 1 All surfaces are selected in the process of

[0060] Then we execute Step 3, as Figure 1 , the augmented UV space refers to a three-dimensional space that, in addition to the U and V coordinates, adds a height W coordinate. Calculate the filling parameters, translation and rotation of the three-dimensional Gaussian: Move one vertex of the 3DGS bounding box to the position (u, v, w) = (0, 0, 0), and specify to rotate the positive directions of the XYZ axes to the positive directions of the UVW (it is also possible to reverse or roll the corresponding order of the axes); Scale and copy the number of filling copies: If the number of U-axis fillings is specified as 4, then calculate the scaling coefficient of the X-axis to satisfy that copying four original 3DGSs can fill the length of the U-axis direction bounding box. Calculate the number of V-axis fillings under this scaling coefficient, and after rounding, calculate the scaling coefficient of the Y-axis. Average the XY-axis scaling coefficients to calculate the scaling coefficient of the Z-axis. Fill the three-dimensional Gaussian into the augmented UV space and modify the opacity attribute of the three-dimensional Gaussian: Specify the overlap degree p overlap of 5%, and adjust the opacity of the three-dimensional Gaussian within the overlapping boundary range as follows

[0061]

[0062] where SDF(μ) is the value of the three-dimensional Gaussian center μ on the signed distance function SDF of the three-dimensional Gaussian bounding box, and l is the length of the long side of the bounding box, so as to eliminate the mutation caused by the texture material boundary after texture filling. Then, through affine transformation and copying, calculate the position attribute of the three-dimensional Gaussian after filling, calculate its rotation attribute through quaternion rotation, calculate the scaling attribute of the three-dimensional Gaussian through the scaling coefficient, and calculate the color attribute of the three-dimensional Gaussian through the rotation spherical harmonic function coefficient.

[0063] Then execute Step 4. The parameter of the model surface shell space calculated is the shell space thickness, as follows:

[0064]

[0065] where h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in the UV space, A s is the bottom area of the shell space, that is, the area of the selected surface in the mesh model, A uv is the bottom area of the three-dimensional Gaussian bounding box in the UV space. Then expand the shell space on the model surface, and expand all vertices on the selected surface by a distance of the shell space thickness h s from their outer normal directions.

[0066] Then execute Step 5.1 to establish the shell mapping, and calculate the three-dimensional Gaussian position parameters according to the shell mapping. This shell mapping is a First, create several prisms in the augmented UV space. The bases of these prisms are model surface triangles located in the w=0 plane. For example, a V A =(u A ,v A ,0) T ,V B ,V C The composition of ΔV A V B V C The top surfaces of these triangular prisms are located at w = h uv The corresponding points of the triangle, such as V A ′=(u A ,v A ,h uv ) T ,V B ′,V C ΔV A ′V B ′V C '. Divide each triangular prism into three tetrahedrons according to the following vertex order (process Figure 1 It can be seen that the triangular prism is divided into a tetrahedron), that is,

[0067] V A V B V C -V′ A ,V′ A V′ B V′ C -V B ,V′ A V B V C -V′ C

[0068] At any point (u,v,w) in the space covered by these triangular prisms, we can find the tetrahedron V X V Y V Z -V W , and find its center of gravity coordinates in the tetrahedron Make

[0069]

[0070] Each triangular prism in the augmented UV space corresponds to a triangular prism in the shell space that expands from the triangular surface. These triangular prisms are divided into tetrahedrons in the same vertex order. Recovering these barycentric coordinates in the shell space can obtain the position of the point in the shell space.

[0071]

[0072] V Xs is V X The corresponding point in the shell space.

[0073] Perform calculations according to the following steps: 5.1.1 Locate the surface triangle where the UV coordinates of the three-dimensional Gaussian in each augmented UV space are located, that is, find which triangular patch of the model the three-dimensional Gaussian is on when projected onto the UV plane. Calculate the geometric center of the UV bottom triangle, establish a geometric center coordinate k-d tree, traverse the three-dimensional Gaussian points to find the k nearest triangles, traverse these triangles, solve the linear equations to calculate the barycentric coordinates, and check whether the barycentric coordinates are legal Check whether the three-dimensional Gaussian is inside the triangle. Delete the three-dimensional Gaussian that is not inside any surface triangle.

[0074] 5.1.2 Locate the tetrahedron where these three-dimensional Gaussians are located and calculate the barycentric coordinates at the same time, that is, given the triangular prism where the three-dimensional Gaussian is located, find which one of the three tetrahedrons it is inside. Split each triangular prism according to the shell mapping rule and establish an index, solve the linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position, and judge whether the barycentric coordinates are legal Return the legal barycentric coordinates and the index.

[0075] 5.1.3 Restore the coordinates of the three-dimensional Gaussian in the shell space. According to the triangle index and the tetrahedron index, use the formula Calculate the coordinates in the shell space.

[0076] Then perform step 5.2 and calculate the rotation parameters according to the UV mapping. For each surface triangle, the rotation matrix for rotating from the UV space to the shell space can be calculated through singular value decomposition and mirror processing according to the following formula:

[0077]

[0078] U, S, V T = SVD(H),

[0079] R = VU T ,

[0080] where, T uv and T s are 3×3 matrices respectively, and each column is the three-dimensional coordinates of a vertex of the triangle in the UV space and the shell space. SVD is the singular value decomposition, and R is the rotation matrix. If det(R) < 0, mirror processing is required. Take the negative of the third row of the matrix V T and then recalculate R = VU T. Finally, according to the index of the triangle where the three-dimensional Gaussian is located, rotation parameters are assigned to each three-dimensional Gaussian.

[0081] Then perform Step 5.3 to calculate the scaling parameter according to the surface area. 5.3.1 Calculate the scaling coefficient

[0082]

[0083] The variables in the formula are the same as those in Step 4; 5.3.2 The scaling coefficient required for the shell space expansion is calculated as follows:

[0084]

[0085] where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the bounding box of the three-dimensional Gaussian in the UV space, A s is the area of the selected surface in the mesh model, A′ s is the outer surface area of the shell formed after the vertex expansion in Step 4. Therefore, the scaling coefficient of each three-dimensional Gaussian is

[0086] S = S uv→s ·S se

[0087] Finally, perform Step 6. Using the parameters calculated in Steps 3, 4, and 5, transform the three-dimensional Gaussian to synthesize the appearance model in the shell space. Specifically, it includes: assigning the position attribute to the three-dimensional Gaussian, that is, filling the parameters obtained in Step 5.1 into the three-dimensional Gaussian model; assigning other attributes to the three-dimensional Gaussian. First, copy and fill the other attributes of the three-dimensional Gaussian material except the position according to the number of copies filled in Step 2, and convert the opacity and scaling attributes according to the following formula:

[0088]

[0089] o, s are the converted opacity and scaling attribute values, x o , x s are the opacity and scaling attribute values in the three-dimensional Gaussian PLY file. Multiply the opacity by the coefficient f O in Step S3 for modification, calculate the scaling by multiplying with the coefficient S in Step S5.3, calculate the rotation by the rotation matrix R in S5.2, and first reorganize the color into the spherical harmonic function coefficient matrix and then perform coefficient adjustment corresponding to the rotation matrix R to complete the rotation.

[0090] After the above steps, the synthesized appearance model is obtained, and this appearance model is still in the form of a three-dimensional Gaussian. For example Figure 1 the obtained sofa with a bread-like appearance surface not only has the color appearance of bread but also has a porous geometric surface on the bread surface, and this geometry is provided by 3DGS.

[0091] Case 2. The implementation process is similar to that of Case 1, and the result is as Figure 2 shown. Please note the obvious difference in the external geometry of the appearance model and the mesh model in the result, which is attributed to the successful utilization of 3DGS by the method of the present invention to change the surface geometry of the mesh.

[0092] Case 3. The implementation process is similar to that of Case 1, and the result is as Figure 3 shown. Similarly, please note the obvious difference in the external geometry of the appearance model and the mesh model in the result.

[0093] Case 4. In step 2, instead of selecting all surfaces, first calculate the surface normal vectors, select the surfaces within -30 to +30 degrees from the positive X-axis direction to perform the subsequent steps of flower 3DGS, select other surfaces to perform the subsequent steps of embossing 3DGS, and finally merge their results. The final result is as Figure 4 shown.

[0094] It should be noted that the above embodiments are not used to limit the protection scope of the present invention. Any equivalent transformation or substitution made on the basis of the above technical solutions falls within the protection scope of the claims of the present invention.

Claims

1. A model appearance synthesis method based on three-dimensional Gaussian and shell mapping, characterized in that: The following steps are involved: S1. Process the 3D Gaussian bounding box and exclude the 3D Gaussian points that are too dispersed. S2. Triangulate the mesh surface, select the mesh surface, and obtain the corresponding UV bounding box. S3. Calculate the filling parameters from the three-dimensional Gaussian bounding box and the UV bounding box, and fill the three-dimensional Gaussian into the augmented UV space, which refers to the three-dimensional space with the height W coordinate added instead of the U and V coordinates. S4. Calculate the parameters of the shell space on the model surface according to the bounding box of the three-dimensional Gaussian in the augmented UV space, and expand the shell space on the model surface. S5. Calculate synthetic appearance transformation parameters: S5.1 Calculate the three-dimensional Gaussian position parameters in shell space based on the shell mapping; S5.2 calculates three-dimensional Gaussian rotation parameters based on UV mapping; S5.3 Calculate the three-dimensional Gaussian scaling parameters based on the surface area, S6. Fill the three-dimensional Gaussian in the augmented UV space into the shell space to obtain a synthesized mesh model appearance; Among them, the filling parameters calculated in step S3 specifically include the translation, rotation, scaling, and copy filling number of the three-dimensional Gaussian, and filling the three-dimensional Gaussian into the augmented UV space, specifically including: modifying the opacity attribute of the three-dimensional Gaussian, calculating the position attribute of the three-dimensional Gaussian after filling through affine transformation and replication, calculating its rotation attribute through quaternion rotation, calculating the scaling attribute of the three-dimensional Gaussian through the scaling coefficient, and calculating the color attribute of the three-dimensional Gaussian through the rotating spherical harmonic function coefficient. Modifying the three-dimensional Gaussian opacity specifically includes: specifying the overlap degree p of the copied fill texture overlap , and adjust the three-dimensional Gaussian opacity within the overlapping boundary range as follows Where SDF(μ) is the value of the three-dimensional Gaussian center μ on the signed distance function SDF of the three-dimensional Gaussian bounding box, and l is the length of the long side of the bounding box. This step eliminates the mutation caused by the texture material boundary after texture filling.

2. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: The processing of the three-dimensional Gaussian bounding box in step S1 specifically includes: eliminating Gaussian points with scattered positions by counting the quantiles of the three-dimensional Gaussian position coordinates, so as to ensure that the shape of the three-dimensional Gaussian material is suitable for subsequent processing.

3. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1 is characterized in that: The mesh surface is selected in step S2, including calculating the mesh surface normal, selecting a partial directional surface method, or a method previously marked and specified by other model processing programs.

4. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: The parameter of the model surface shell space to be calculated in step S4 is the shell space thickness, and the specific calculation formula is: In the formula, h s is the shell space thickness, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the bottom area of ​​the shell space, that is, the area of ​​the selected surface in the mesh model, A uv is the bottom area of ​​the three-dimensional Gaussian bounding box in UV space. In S4, the shell space is expanded on the model surface. Specifically, all vertices on the selected surface are expanded toward their outer normal by the thickness h of the shell space. s distance.

5. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S5.1 first establishes a shell map, and then calculates the three-dimensional Gaussian position parameters based on the shell map. The shell map is a The mapping is to input an augmented UV space (u, v, w) coordinate and output a shell space (x, y, z) coordinate.

6. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 5, characterized in that: The calculation process of step S5.1 specifically includes the following steps: S5.1.1 Locate the surface triangle where the UV coordinates of the three-dimensional Gaussian in each augmented UV space are located, that is, find which triangle patch of the model it is on when the three-dimensional Gaussian is projected onto the UV plane. Specifically, it includes: calculating the geometric center of the UV bottom triangle, establishing a kd tree of geometric center coordinates, traversing the three-dimensional Gaussian points to find the k nearest triangles, traversing these triangles, solving the linear equations to calculate the barycentric coordinates, and checking whether the barycentric coordinates are legal, that is, Check if the 3D Gaussian is inside a triangle, remove the 3D Gaussian that is not inside any surface triangle, S5.1.2 Locate the tetrahedrons where these three-dimensional Gaussians are located and calculate the barycentric coordinates at the same time. That is, given the triangular prism where the three-dimensional Gaussian is located, find out which of the three tetrahedrons it is inside. Specifically, it includes: establishing several triangular prisms in the augmented UV space. The bases of these triangular prisms are model surface triangles located in the w=0 plane. Split each triangular prism according to the rules and establish an index. Solve the linear equations to calculate the barycentric coordinates of the three-dimensional Gaussian position and determine whether the barycentric coordinates are legal, that is, Returns the legal center of gravity coordinates and index, S5.1.3 Recovering the coordinates of the three-dimensional Gaussian in the shell space, specifically comprising: calculating the coordinates in the shell space according to the index.

7. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S5.2 calculates the rotation parameters according to the UV mapping. Specifically, for each surface triangle, the rotation matrix of how to rotate from UV space to shell space can be calculated by singular value decomposition and mirror processing: U,S,V T =SVD(H), R=VU T , Where, T uv and T s They are 3×3 matrices, each column is the three-dimensional coordinates of a vertex of a triangle in UV and shell space, SVD is singular value decomposition, R is the rotation matrix, if det(R)<0, mirror processing is required, and the matrix V T The third line is inverted, and then R=VU is recalculated T , and finally assign each 3D Gaussian rotation parameter according to the index of the triangle where the 3D Gaussian is located.

8. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 4, characterized in that: Step S5.3 calculates the scaling parameter according to the surface area, specifically comprising: 5.3.1 Scaling Factor 5.3.2 The scaling factor required for shell space expansion is calculated as follows: Where w is the W coordinate value of the three-dimensional Gaussian in the augmented UV space, h uv is the height of the three-dimensional Gaussian bounding box in UV space, A s is the area of ​​the selected surface in the mesh model, A′ s is the outer surface area of ​​the shell formed after the vertex expansion in step S4, so the scaling factor of each three-dimensional Gaussian is S=S uv→s ·S se 。 9. The model appearance synthesis method based on three-dimensional Gaussian and shell mapping according to claim 1, characterized in that: Step S6 transforms the three-dimensional Gaussian using the parameters calculated in steps S3, S4, and S5 to synthesize the appearance model in the shell space, specifically including: assigning the three-dimensional Gaussian position attribute, that is, filling the parameters obtained in step S5.1 into the three-dimensional Gaussian model; assigning other attributes to the three-dimensional Gaussian, first copying the other attributes of the three-dimensional Gaussian material except the position according to the number of copies and fills in step S3, and converting the opacity and scaling attributes according to the following formula: o, s are the opacity and scaling property values ​​after conversion, x o ,x s is the opacity and scaling attribute value in the 3D Gaussian PLY file. Multiply the opacity by the coefficient f in step S3. O Modification, scaling is calculated by multiplying the scaling factor S in step S5.3, and the rotation is calculated by the rotation matrix R in S5.

2. The color is first reorganized into a spherical harmonic function coefficient matrix, and then the coefficients corresponding to the rotation matrix R are adjusted to complete the rotation.

Images