A tree model material decoupling and light separation method and system based on three-dimensional Gaussian splash

Abstract

The application belongs to the technical field of geographic information systems, and discloses a tree model material decoupling and light separation method and system based on three-dimensional Gaussian splashing, which comprises the following steps: obtaining an initial image set of a target tree and performing pretreatment to obtain a first main image sequence; constructing a three-dimensional Gaussian radiation field corresponding to the first main image sequence based on a motion recovery structure algorithm, initializing an ambient light map of the three-dimensional Gaussian radiation field and geometric parameters and material parameters of a three-dimensional Gaussian primitive; performing differentiable rendering on the initialized three-dimensional Gaussian primitive to generate a geometric buffer, and obtaining a second main image sequence after rendering according to the geometric buffer; constructing a joint loss function, and optimizing and outputting the ambient light map and the geometric parameters and material parameters of the three-dimensional Gaussian primitive according to the joint loss function, the first main image sequence and the second main image sequence; and the application can efficiently and realistically separate the inherent material of a tree from dynamic ambient light in an image.

Description

Technical Field

[0001] This invention relates to the field of geographic information system technology, and in particular to a method and system for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing. Background Technology

[0002] In fields such as film and television production, game development, virtual reality, and digital twins, there is a growing demand for highly realistic and editable digital tree models.

[0003] High-quality tree assets not only require realistic geometric shapes, but also physically realistic material properties (such as diffuse reflectance, specular reflectance, roughness, etc.) and the ability to interact realistically with any lighting environment (relighting). Traditional methods based on manual modeling and material painting are cumbersome, time-consuming, and difficult to guarantee physical realism.

[0004] Among existing tree modeling techniques, multi-view stereo reconstruction methods can recover geometry from photographs, but cannot separate intrinsic materials and lighting, resulting in poor reusability. Neural radiation fields and their variants can reconstruct complex scenes with high quality, but their implicit representation makes it difficult to directly decouple and edit lighting properties. 3D Gaussian splashing technology, as a display scene representation method, achieves real-time rendering and high-quality reconstruction, but its original representation does not explicitly separate materials and lighting, limiting its application in downstream tasks such as material editing and lighting replacement.

[0005] Existing technologies suffer from problems such as difficulty in reuse, insufficient universality, and difficulty in intuitively controlling and editing decoupling results. Therefore, existing technologies need further improvement. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for material decoupling and lighting separation of tree models based on three-dimensional Gaussian splashing, in order to solve the problems of high reusability, insufficient universality and difficulty in intuitive control and editing of decoupling results in existing tree modeling technology.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including: Obtain an initial image set of the target tree, and preprocess the initial image set to obtain a first main image sequence; The three-dimensional Gaussian radiation field corresponding to the first subject image sequence is constructed based on the motion recovery structure algorithm, and the ambient lighting map and geometric parameters and material parameters of the three-dimensional Gaussian radiation field are initialized. Differentiable rendering is performed on the initialized 3D Gaussian primitives to generate a geometric buffer, and the rendered second main image sequence is obtained based on the geometric buffer. Construct a joint loss function, and optimize the ambient lighting map and the geometric and material parameters of the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence; Output optimized ambient lighting map and geometric and material parameters of 3D Gaussian elements.

[0008] In one implementation, the preprocessing of the initial image set to obtain the first main image sequence includes: The initial image set is semantically segmented using a segmentation model to obtain the initial mask of the target tree. The initial mask is subjected to morphological closing operation and connected component analysis to obtain an optimized binary mask; Generate the main image sequence of the target tree based on the binary mask.

[0009] In one implementation, constructing the three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion reconstruction structure algorithm includes: Sparse 3D point cloud and camera pose of the first subject image sequence are extracted based on the structure-of-motion reconstruction algorithm. A three-dimensional Gaussian radiation field is constructed based on the extracted sparse three-dimensional point cloud and the camera pose.

[0010] In one implementation, initializing the ambient lighting map of the three-dimensional Gaussian radiation field and the geometric and material parameters of the three-dimensional Gaussian primitives includes: Initialize the ambient light map by uniformly sampling and scaling the RGB values ​​of each pixel in the ambient light map as the initial representation of global illumination; Initialize the geometric parameters of the three-dimensional Gaussian quaternion; wherein the geometric parameters include any one or more combinations of position parameters, rotation quaternions, scaling vectors, and opacity; Initialize the material parameters of the three-dimensional Gaussian primitive; wherein the material parameters include any one or more combinations of diffuse color, specular color, roughness, and surface normal.

[0011] In one implementation, the step of performing differentiable rendering on the initialized 3D Gaussian primitives to generate a geometry buffer, and obtaining the rendered second main image sequence based on the geometry buffer, includes: Based on the camera pose, the initialized 3D Gaussian primitives are rendered using differentiable methods to generate a geometry buffer containing the geometric parameters and the material parameters. Based on the geometry buffer and the ambient light map, calculate the final color of each pixel in the ambient light map; The ambient light map is updated based on the final color of each pixel; Based on the updated ambient irradiance map, a rendered second subject image sequence corresponding to the first subject image sequence is generated.

[0012] In one implementation, the joint loss function includes any one or more combinations of: a reconstruction loss function, a distance regularization loss function, a normal consistency loss function, and a mask loss function.

[0013] In one implementation, optimizing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence includes: With the ambient lighting map and material parameters fixed, the geometric parameters are optimized based on a preset first learning rate, the reconstruction loss function, and the distance regularization loss function. With the optimized geometric parameters fixed, the ambient lighting map and the material parameters are optimized based on a preset second learning rate and the joint loss function.

[0014] Secondly, the present invention provides a material decoupling and lighting separation system for a tree model based on three-dimensional Gaussian splashing, comprising: The data acquisition module is used to acquire an initial image set of the target tree and preprocess the initial image set to obtain a first main image sequence. The parameter initialization module is used to construct the three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion reconstruction structure algorithm, and to initialize the ambient lighting map and the geometric parameters and material parameters of the three-dimensional Gaussian radiation field. The image rendering module is used to perform differentiable rendering on the initialized three-dimensional Gaussian primitives, generate a geometric buffer, and obtain the rendered second main image sequence based on the geometric buffer. The loss constraint module is used to construct a joint loss function and optimize the geometric and material parameters of the ambient lighting map and the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence. The results output module is used to output the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian elements.

[0015] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing, and the program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing, when executed by the processor, is used to implement the operation of the method for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing as described in the first aspect.

[0016] Fourthly, the present invention also provides a computer-readable storage medium storing a program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing, wherein the program, when executed by a processor, is used to implement the operation of the method for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing as described in the first aspect.

[0017] The present invention, by employing the above technical solution, has the following effects: This invention provides a method and system for material decoupling and lighting separation of tree models based on 3D Gaussian splashing, comprising: acquiring an initial image set of the target tree; preprocessing the initial image set to obtain a first main image sequence; constructing a 3D Gaussian radiation field corresponding to the first main image sequence based on the structure-of-motion reconstructing algorithm; initializing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives of the 3D Gaussian radiation field; performing differentiable rendering on the initialized 3D Gaussian primitives to generate a geometric buffer; obtaining a rendered second main image sequence based on the geometric buffer; constructing a joint loss function; optimizing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives based on the joint loss function, the first main image sequence, and the second main image sequence; and outputting the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives. This invention can efficiently and realistically separate the inherent material of trees from the dynamic lighting of the environment from images, providing a data foundation for high-quality relighting, material editing, and asset reuse, without relying on complex professional knowledge and modeling processes, effectively reducing the threshold for the creation and use of high-quality digital tree assets. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0019] Figure 1This is a flowchart of the method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing in this invention.

[0020] Figure 2 This is a schematic diagram illustrating the process of processing a Bodhi tree using a three-dimensional Gaussian splashing-based tree model material decoupling and lighting separation method in one implementation of the present invention.

[0021] Figure 3 This is a schematic diagram illustrating the process of processing purple-leaf plum using a tree model material decoupling and illumination separation method based on three-dimensional Gaussian splashing in one implementation of the present invention.

[0022] Figure 4 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0023] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0025] Exemplary methods In existing tree modeling techniques, multi-view stereo reconstruction methods can recover geometry from photographs, but cannot separate intrinsic materials and lighting, resulting in models being bound to specific lighting conditions and having poor reusability. Neural radiation fields and their variants can reconstruct complex scenes with high quality, but their implicit representation makes it difficult to directly decouple and edit lighting attributes. 3D Gaussian splashing technology, as a display scene representation method, achieves real-time rendering and high-quality reconstruction, but its original representation does not explicitly separate materials and lighting, limiting its application in downstream tasks such as material editing and lighting replacement.

[0026] Existing technologies suffer from the following drawbacks: high reliance on specialized knowledge and manual parameter tuning, low automation, and difficulty in large-scale production; the lighting and materials of the reconstructed model are interdependent, making it impossible to achieve the goal of "sequential reconstruction and reuse in multiple locations"; most methods rely on carefully designed capture equipment and lighting conditions, resulting in insufficient universality; the optimization process for implicit representation methods is lengthy, and the decoupling results are difficult to directly control and edit. Therefore, existing technologies require further improvement.

[0027] To address the above technical problems, this invention provides a method for decoupling material and separating illumination in tree models based on 3D Gaussian splashing. The method includes: acquiring an initial image set of the target tree; preprocessing the initial image set to obtain a first main image sequence; constructing a 3D Gaussian radiation field corresponding to the first main image sequence based on a motion recovery structure algorithm; initializing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives of the 3D Gaussian radiation field; performing differentiable rendering on the initialized 3D Gaussian primitives to generate a geometric buffer; obtaining a rendered second main image sequence based on the geometric buffer; constructing a joint loss function; optimizing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives based on the joint loss function, the first main image sequence, and the second main image sequence; and outputting the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives. This invention can efficiently and realistically separate the inherent material of trees from dynamic environmental illumination in images, providing a data foundation for high-quality relighting, material editing, and asset reuse. Furthermore, it does not rely on complex professional knowledge and modeling processes, effectively lowering the threshold for creating and using high-quality digital tree assets.

[0028] like Figure 1 As shown, this embodiment of the invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including the following steps: Step S100: Obtain an initial image set of the target tree, and preprocess the initial image set to obtain a first main image sequence.

[0029] Specifically, in one implementation of this embodiment, step S100 includes the following steps: Step S101: Obtain the initial image set of the target trees.

[0030] In this embodiment, an initial image set of the target tree is obtained. The initial image set of the target tree needs to contain color images of the target tree from multiple different perspectives under natural lighting.

[0031] In this embodiment, an initial image set of the target tree can be obtained by taking pictures; alternatively, a set of color images of the target tree from multiple different perspectives under natural light can be directly obtained from a publicly available database as the initial image set of the target tree.

[0032] Step S102: Use a segmentation model to perform semantic segmentation on each image in the initial image set to obtain the initial mask of the target tree.

[0033] In this embodiment, the initial image set is preprocessed to remove complex background interference, ensuring that subsequent processing focuses on the main tree. First, a segmentation model is used to perform semantic segmentation on each image from multiple perspectives in the initial image set to obtain the initial mask / foreground mask of the tree. In this embodiment, the SAM model (Segment Anything Mode, a general image segmentation basic model) is used to perform semantic segmentation on each image from multiple perspectives in the initial image set.

[0034] Step S103: Perform morphological closing operation and connected component analysis on the initial mask to obtain the optimized binary mask.

[0035] In this embodiment, after semantic segmentation of the multi-view images in the initial image set to obtain the initial mask, the preprocessing steps further include: performing morphological closing operations and connected component analysis on the initial mask to obtain an optimized binary mask. Specifically, morphological closing operations are performed on the initial mask to fill the internal holes caused by the gaps between leaves, and maximum connected component analysis is performed to retain only the main body region with the largest area, thus obtaining the optimized binary mask.

[0036] Step S104: Generate the main image sequence of the target tree based on the binary mask.

[0037] In this embodiment, the optimized binary mask is used to perform image matting on each image from different perspectives in the initial image sequence to obtain a clean tree subject image sequence with the background removed. This completes the preprocessing step of the initial image set and obtains the first subject image sequence, providing clean and consistent input data for subsequent 3D reconstruction and material decoupling.

[0038] like Figure 1 As shown, this embodiment of the invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including the following steps: Step S200: Construct a three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion recovery structure algorithm, and initialize the ambient lighting map and geometric parameters and material parameters of the three-dimensional Gaussian radiation field.

[0039] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201: Extract the sparse 3D point cloud and camera pose of the first subject image sequence based on the structure-of-motion reconstruction algorithm.

[0040] In this embodiment, the first subject image sequence is processed based on the Structure from Motion (SfM) algorithm. The SfM process automatically matches feature points between multi-view images in the first subject image sequence. Through incremental or global bundled adjustment, the precise camera pose and sparse 3D point cloud of the scene corresponding to each image are simultaneously optimized. The sparse 3D point cloud provides the basic geometric skeleton of the trees, while the camera pose can serve as an indispensable observation parameter for differentiable rendering in subsequent steps.

[0041] Step S202: Construct a three-dimensional Gaussian radiation field based on the extracted sparse three-dimensional point cloud and camera pose.

[0042] In this embodiment, a three-dimensional Gaussian primitive is initialized with each feature point in the sparse three-dimensional point cloud generated by SfM as the center, which together constitute the displayed three-dimensional Gaussian radiation field; wherein, each three-dimensional Gaussian primitive contains geometric parameters and material parameters.

[0043] In this embodiment, the geometric parameters of the three-dimensional Gaussian element are defined as follows: ;in, For position parameters, It is a real number; It is a rotation quaternion; For scaling vectors; Opacity.

[0044] The material parameters of a 3D Gaussian element are defined as follows: ;in, It is a diffuse color; The color is a specular reflection. For roughness; For surface normals.

[0045] In this embodiment, the spatial influence range of each Gaussian unit is described by a three-dimensional Gaussian distribution, whose covariance matrix is ​​constructed from the rotation quaternion and scaling vector in the geometric parameters, as shown in the following formula: ; in, For rotation quaternions The converted Rotation matrix, To scale vectors It is a diagonal matrix composed of diagonal elements.

[0046] Step S203: Initialize the ambient light map by uniformly sampling and scaling the RGB values ​​of each pixel in the ambient light map as the initial representation of global illumination.

[0047] In this embodiment, the initialization steps for the ambient lighting map are specifically as follows: initialize a resolution of... The cube map is a global illumination map in which each pixel represents ambient light in one direction. The RGB values ​​of each pixel are sampled and played from a uniform distribution as an initial representation of global illumination. The ambient map is used to participate in subsequent physically based rendering calculations.

[0048] Step S204: Initialize the geometric parameters of the three-dimensional Gaussian quaternion; wherein the geometric parameters include any one or more combinations of position parameters, rotation quaternions, scaling vectors, and opacity.

[0049] In this embodiment, the geometric parameters include any one or more combinations of position parameters, rotation quaternions, scaling vectors, and opacity. The geometric parameters of the 3D Gaussian primitives are initialized: the position parameters are initialized to the points in the corresponding sparse 3D point cloud; the rotation quaternions are initialized to unit quaternions; the scaling vector is initialized to a preset small value; and the opacity is initialized to a fixed value of 0.1.

[0050] Step S205: Initialize the material parameters of the three-dimensional Gaussian elements; wherein the material parameters include any one or more combinations of diffuse color, specular color, roughness, and surface normal.

[0051] In this embodiment, the material parameters include any one or more of the following: diffuse color, specular color, roughness, and surface normal. The material parameters of the three-dimensional Gaussian primitive are initialized as follows: diffuse color is initialized to white; specular color is initialized to gray; roughness is initialized to 0.5; and surface normal is initialized to a zero vector.

[0052] like Figure 1 As shown, this embodiment of the invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including the following steps: Step S300: Perform differentiable rendering on the initialized 3D Gaussian primitives to generate a geometric buffer, and obtain the rendered second main image sequence based on the geometric buffer.

[0053] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301: Based on the camera pose, perform differentiable rendering on the initialized 3D Gaussian primitives to generate a geometry buffer containing the geometric parameters and the material parameters.

[0054] In this embodiment, differentiable rendering of the initialized 3D Gaussian primitives is performed using a two-stage differentiable deferred rendering pipeline. Specifically, in the first stage, based on the acquired camera pose, all 3D Gaussian primitives in the 3D Gaussian radiation field are rendered to screen space through a differentiable perspective projection and splatting process, generating a geometry buffer (G-Buffer). Each pixel in the geometry buffer stores aggregated information from the top K contributing Gaussian primitives, including but not limited to: diffuse color. Specular reflection color Roughness World Space Normal The process is fully differentiable, allowing gradients to propagate back from the image space to the parameters of each Gaussian unit.

[0055] Step S302: Based on the geometry buffer and the ambient light map, calculate the final color of each pixel in the ambient light map.

[0056] In this embodiment, in the second stage of 3D Gaussian primitive differentiable rendering, physically based lighting calculations are performed based on the geometric buffer generated in the first stage. Specifically, for each pixel in the ambient light map, the Cook-Torrance microsurface model is used as the bidirectional reflectance distribution function (BRDF) to calculate its final color.

[0057] In this embodiment, the diffuse reflection component The Lambertian model is used to obtain irradiance by performing a cosine-weighted hemispherical integral on the lowest level Mipmap (a texture mapping technique) of the ambient irradiance map.

[0058] Specular reflection component The calculation formula is: ; in, For the normal distribution function, the GGX model (Trowbridge-Reitz distribution model, micro-surface distribution model) is selected. For Fresnel terms, the Schlick approximation is chosen; The geometric occlusion function is represented by the Smith model. and These are the incident light and the line of sight, respectively. For surface normal, To extract from the environment map based on roughness The incident light intensity obtained from the corresponding Mipmap level of dynamic sampling.

[0059] Final color for: ; in, and These are the weighting coefficients derived from the material parameters.

[0060] Step S303: Update the ambient light map according to the final color of each pixel.

[0061] In this embodiment, the ambient light map is updated based on the final color calculated in step S302.

[0062] Step S304: Based on the updated ambient lighting map, generate the rendered second subject image sequence corresponding to the first subject image sequence.

[0063] In this embodiment, a rendered second subject image sequence corresponding to the first subject image sequence is generated based on the updated ambient lighting map.

[0064] This embodiment can combine decoupled material properties with independent ambient lighting to synthesize a rendered second main image sequence corresponding to the first main image sequence.

[0065] like Figure 1 As shown, this embodiment of the invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including the following steps: Step S400: Construct a joint loss function, and optimize the ambient lighting map and the geometric and material parameters of the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence.

[0066] Specifically, in one implementation of this embodiment, step S400 includes the following steps: Step S401: Construct a joint loss function; wherein the joint loss function includes any one or more combinations of the following: reconstruction loss function, distance regularization loss function, normal consistency loss function, and mask loss function.

[0067] In this embodiment, by constructing a joint loss function, the geometric and material parameters of the ambient lighting map and the three-dimensional Gaussian primitives are optimized based on the joint loss function, thereby decoupling the material from the lighting and making the synthesized image based on the ambient lighting map and the geometric and material parameters closer to the real image.

[0068] In this embodiment, the joint loss function includes any one or more combinations of the following: reconstruction loss function, distance regularization loss function, normal consistency loss function, and mask loss function.

[0069] In one implementation of this embodiment, the formula for calculating the joint loss function is as follows: ; in, To reconstruct the loss function, L1 loss and structural similarity (SSIM) loss are combined to ensure reconstruction quality at both the pixel and structural levels. The distance regularization loss function penalizes Gaussian points for having too small an average distance to their K nearest neighbors, preventing excessive clustering of Gaussian points and encouraging them to uniformly cover the surface. The normal consistency loss function encourages the smooth and consistent local surface normal directions fitted by adjacent Gaussian point positions. The mask loss function is used to compare the Alpha channel of the rendered image with the optimized binary mask obtained in step S103, constraining the geometric range.

[0070] Step S402: Fix the ambient lighting map and the material parameters, and optimize the geometric parameters based on a preset first learning rate, the reconstruction loss function and the distance regularization loss function.

[0071] In this embodiment, the step of optimizing the ambient lighting map and the geometric and material parameters of the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence uses a hierarchical adaptive strategy.

[0072] Specifically, in the first stage, the ambient lighting map and the material parameters are fixed, and the geometric parameters are optimized based on a preset first learning rate, the reconstruction loss function, and the distance regularization loss function; wherein, the preset first learning rate is a relatively high learning rate, for example, the position parameter is set to 0.00016; the first stage only uses the reconstruction loss function. and distance regularization loss function Under the constraints of precise geometry, physically realistic material properties and global illumination are finely optimized.

[0073] Step S403: Fix the optimized geometric parameters, and optimize the ambient lighting map and the material parameters based on the preset second learning rate and the joint loss function.

[0074] In this embodiment, in the second stage of hierarchical adaptive strategy optimization, the optimized geometric parameters are fixed, and the ambient lighting map and material parameters are optimized based on a preset second learning rate and the joint loss function. The preset second learning rate is the disclosure learning rate, for example, the diffuse color is set to 0.0025 and the ambient map pixel is set to 0.001. The second stage uses the complete joint loss function to finely optimize physically realistic material properties and global illumination under the constraints of precise geometry.

[0075] In this embodiment, the Adam optimizer is used throughout the optimization process of the hierarchical adaptive strategy, with differentiated learning rates set for different parameter types and exponential decay scheduling employed.

[0076] like Figure 1 As shown, this embodiment of the invention provides a method for material decoupling and lighting separation of a tree model based on three-dimensional Gaussian splashing, including the following steps: Step S500: Output the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian elements.

[0077] In this embodiment, through the optimization of steps S402 and S403, the optimized ambient lighting map is finally output, which is optimized to capture the image of the real lighting of the shooting scene; the output set of three-dimensional Gaussian elements has material parameters that are the inherent physical property materials of trees after being decoupled from the specific lighting environment.

[0078] This embodiment achieves the following technical effects through the above technical solution: The physically based shading model BRDF is explicitly embedded into the differentiable rendering pipeline of 3D Gaussian splashing. In addition to geometric parameters, each 3D Gaussian primitive is explicitly associated with a set of physical material parameters, including diffuse color, specular color, roughness, and normals. A material-independent global environment lightmap is introduced as the lighting source. This fundamentally solves the problem of material and lighting information coupling in traditional 3D Gaussian splashing and other methods, and provides a differentiable physical basis for end-to-end joint optimization decoupling. This framework makes it possible to recover physically meaningful material properties from ordinary images. A multi-objective optimization function was constructed that integrates reconstruction loss, distance regularization loss, normal consistency loss, and mask loss. This loss function not only constrains rendering fidelity at the pixel and structural levels, but also ensures that the surfaces to which the materials are attached have reasonable spatial distribution and smoothness through geometric regularization. While driving the optimization of material and lighting parameters, it effectively prevents ill-conditioned solutions such as disordered diffusion of Gaussian points and abrupt changes in material properties in space, ensuring that the final decoupled material assets have physical rationality and geometric consistency, thus improving the robustness of the method. A two-stage optimization process is adopted: geometry first, then materials and lighting. The first stage uses a high learning rate to quickly and stably reconstruct the geometry under simplified loss constraints. The second stage, with the geometry fixed, finely optimizes the material parameters and lighting maps jointly using a lower learning rate, and enables all distance regularization losses. This strategy significantly improves the stability and convergence quality of the complex optimization process. Establishing an accurate geometric foundation first avoids local optima caused by mutual interference between materials and lighting in the early stages of optimization, thus achieving decoupled results with higher fidelity.

[0079] In summary, this invention enables efficient and realistic separation of the inherent material of trees from dynamic environmental lighting in images, providing a data foundation for high-quality relighting, material editing, and asset reuse. Moreover, it does not require complex professional knowledge and modeling processes, effectively lowering the threshold for creating and using high-quality digital tree assets.

[0080] Exemplary device Based on the above embodiments, the present invention also provides a material decoupling and lighting separation system for a tree model based on three-dimensional Gaussian splashing, comprising: The data acquisition module is used to acquire an initial image set of the target tree and preprocess the initial image set to obtain a first main image sequence. The parameter initialization module is used to construct the three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion reconstruction structure algorithm, and to initialize the ambient lighting map and the geometric parameters and material parameters of the three-dimensional Gaussian radiation field. The image rendering module is used to perform differentiable rendering on the initialized three-dimensional Gaussian primitives, generate a geometric buffer, and obtain the rendered second main image sequence based on the geometric buffer. The loss constraint module is used to construct a joint loss function and optimize the geometric and material parameters of the ambient lighting map and the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence. The results output module is used to output the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian elements.

[0081] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 2 As shown.

[0082] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a computer-readable storage medium and internal memory; the computer-readable storage medium stores an operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the computer-readable storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0083] When executed by the processor, this computer program is used to implement a method for material decoupling and lighting separation of a tree model based on 3D Gaussian splashing.

[0084] It will be understood by those skilled in the art that Figure 2The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0085] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a program for decoupling and separating the material of a tree model based on 3D Gaussian splashing, the program being executed by the processor to implement the operation of the above-described method for decoupling and separating the material of a tree model based on 3D Gaussian splashing.

[0086] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a material decoupling and lighting separation program for a tree model based on 3D Gaussian splashing, which, when executed by a processor, is used to implement the operation of the above-described material decoupling and lighting separation method for a tree model based on 3D Gaussian splashing.

[0087] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0088] In summary, this invention provides a method for decoupling material and separating illumination in tree models based on 3D Gaussian splashing, comprising: acquiring an initial image set of the target tree; preprocessing the initial image set to obtain a first main image sequence; constructing a 3D Gaussian radiation field corresponding to the first main image sequence based on the structure-of-motion reconstructing algorithm; initializing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives of the 3D Gaussian radiation field; performing differentiable rendering on the initialized 3D Gaussian primitives to generate a geometric buffer; obtaining a rendered second main image sequence based on the geometric buffer; constructing a joint loss function; optimizing the ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives based on the joint loss function, the first main image sequence, and the second main image sequence; and outputting the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian primitives. This invention can efficiently and realistically separate the inherent material of trees from the dynamic environmental illumination from images, providing a data foundation for high-quality relighting, material editing, and asset reuse, without relying on complex professional knowledge and modeling processes, effectively reducing the threshold for creating and using high-quality digital tree assets.

[0089] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for material decoupling and lighting separation of a tree model based on 3D Gaussian splashing, characterized in that, include: Obtain an initial image set of the target tree, and preprocess the initial image set to obtain a first main image sequence; The three-dimensional Gaussian radiation field corresponding to the first subject image sequence is constructed based on the motion recovery structure algorithm, and the ambient lighting map and geometric parameters and material parameters of the three-dimensional Gaussian radiation field are initialized. Differentiable rendering is performed on the initialized 3D Gaussian primitives to generate a geometric buffer, and the rendered second main image sequence is obtained based on the geometric buffer. Construct a joint loss function, and optimize the ambient lighting map and the geometric and material parameters of the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence; Output optimized ambient lighting map and geometric and material parameters of 3D Gaussian elements.

2. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The preprocessing of the initial image set to obtain the first main image sequence includes: The initial image set is semantically segmented using a segmentation model to obtain the initial mask of the target tree. The initial mask is subjected to morphological closing operation and connected component analysis to obtain an optimized binary mask; Generate the main image sequence of the target tree based on the binary mask.

3. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The construction of the three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion reconstruction structure algorithm includes: Sparse 3D point cloud and camera pose of the first subject image sequence are extracted based on the structure-of-motion reconstruction algorithm. A three-dimensional Gaussian radiation field is constructed based on the extracted sparse three-dimensional point cloud and the camera pose.

4. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The initialization of the environment lighting map and the geometric and material parameters of the three-dimensional Gaussian radiation field includes: Initialize the ambient light map by uniformly sampling and scaling the RGB values ​​of each pixel in the ambient light map as the initial representation of global illumination; Initialize the geometric parameters of the three-dimensional Gaussian quaternion; wherein the geometric parameters include any one or more combinations of position parameters, rotation quaternions, scaling vectors, and opacity; Initialize the material parameters of the three-dimensional Gaussian primitive; wherein the material parameters include any one or more combinations of diffuse color, specular color, roughness, and surface normal.

5. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 3, characterized in that, The process of performing differentiable rendering on the initialized 3D Gaussian primitives to generate a geometric buffer, and obtaining the rendered second main image sequence based on the geometric buffer, includes: Based on the camera pose, the initialized 3D Gaussian primitives are rendered using differentiable methods to generate a geometry buffer containing the geometric parameters and the material parameters. Based on the geometry buffer and the ambient light map, calculate the final color of each pixel in the ambient light map; The ambient light map is updated based on the final color of each pixel; Based on the updated ambient irradiance map, a rendered second subject image sequence corresponding to the first subject image sequence is generated.

6. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 1, characterized in that, The joint loss function includes any one or more combinations of the following: reconstruction loss function, distance regularization loss function, normal consistency loss function, and mask loss function.

7. The method for material decoupling and illumination separation of a tree model based on three-dimensional Gaussian splashing as described in claim 6, characterized in that, The optimization of the ambient lighting map and the geometric and material parameters of the 3D Gaussian elements based on the joint loss function, the first subject image sequence, and the second subject image sequence includes: With the ambient lighting map and material parameters fixed, the geometric parameters are optimized based on a preset first learning rate, the reconstruction loss function, and the distance regularization loss function. With the optimized geometric parameters fixed, the ambient lighting map and the material parameters are optimized based on a preset second learning rate and the joint loss function.

8. A system for decoupling material and separating illumination of a tree model based on three-dimensional Gaussian splashing, used to implement the method for decoupling material and separating illumination of a tree model based on three-dimensional Gaussian splashing as described in any one of claims 1-7, characterized in that, include: The data acquisition module is used to acquire an initial image set of the target tree and preprocess the initial image set to obtain a first main image sequence. The parameter initialization module is used to construct the three-dimensional Gaussian radiation field corresponding to the first subject image sequence based on the motion reconstruction structure algorithm, and to initialize the ambient lighting map and the geometric parameters and material parameters of the three-dimensional Gaussian radiation field. The image rendering module is used to perform differentiable rendering on the initialized three-dimensional Gaussian primitives, generate a geometric buffer, and obtain the rendered second main image sequence based on the geometric buffer. The loss constraint module is used to construct a joint loss function and optimize the geometric and material parameters of the ambient lighting map and the three-dimensional Gaussian primitives based on the joint loss function, the first subject image sequence, and the second subject image sequence. The results output module is used to output the optimized ambient lighting map and the geometric and material parameters of the 3D Gaussian elements.

9. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing, and the program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing, when executed by the processor, is used to implement the method for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing. When executed by a processor, the program is used to implement the method for decoupling and separating the material of a tree model based on three-dimensional Gaussian splashing as described in any one of claims 1-7.

Images