Three-dimensional gaussian-based power grid digital space modeling and inverse rendering method

Abstract

The application discloses a power grid digital space modeling and inverse rendering method based on three-dimensional Gauss, and the application divides an image into transparent object Gauss and opaque environment Gauss through a finely adjusted SAM2 model, estimates the transparent object Gauss information by using a finely adjusted DiffusionRenderer model and a finely adjusted VGGT model, then jointly optimizes and reconstructs the transparent object Gauss and a traditional triangular mesh, and jointly optimizes and reconstructs the opaque environment Gauss and the transparent object Gauss. A new scene is composed of the reconstructed transparent object, the reconstructed environment and other reconstructed power grid scenes and traditional triangular meshes made by an art designer, so that virtual-real fusion and mixed real-time rendering are realized. The application guides Gauss ray tracing by using triangular mesh ray tracing, realizes precalculation and table optimization of light transmission paths by combining neural Gauss light transmission fields, and models environment light by using a neural Gauss incident light field, so that the calculation cost during rendering is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of power system information technology and 3D modeling technology, specifically a method for digital space modeling and inverse rendering of power grids based on 3D Gaussian. Background Technology

[0002] As the energy system develops towards low-carbon and intelligent directions, the informatization of the power system is rapidly advancing, and digitalization and intelligence have become inevitable trends. In this context, the digital twin of the power grid, as a precise mapping of the physical power grid into the digital space, has become the technical support for key operations such as distribution dispatching, emergency command, asset management, and inspection and maintenance. Among these, rendering is the core component of 3D scene construction, its core function being to transform virtual 3D models into visually realistic and perceptible 2D images or dynamic scenes.

[0003] In existing technologies, methods typically employ the reconstruction of 3D scenes from 2D images and the generation of new perspectives to achieve realistic rendering effects. Related techniques, such as Neural Radiance Fields (NeRF), implicitly represent the scene using a multilayer perceptron (MLP) and combine this with volumetric ray casting to synthesize new perspectives, achieving high rendering quality. However, these methods generally rely on deep network structures, resulting in lengthy training and rendering times, making them unsuitable for real-time applications. To improve efficiency, existing solutions have attempted to introduce auxiliary data structures such as hash grids, point clouds, or voxels; however, their real-time performance remains limited due to the need for dense sampling along the ray direction.

[0004] Among them, 3D Gaussian Splatting (3D-GS) proposes a novel explicit scene representation method. This method no longer relies on implicit modeling in MLPs, but instead uses Gaussian functions as graphical primitives to represent the scene, thus avoiding the high overhead of dense point sampling in volumetric rendering and achieving a significant improvement in rendering performance. Specifically, after acquiring multi-view images calibrated by camera parameters, 3DGS achieves explicit scene modeling by optimizing a set of 3D Gaussian functions, thereby ensuring realistic effects while possessing real-time rendering capabilities.

[0005] However, 3D Gaussian mapping is difficult to integrate effectively with traditional rendering pipelines to achieve seamless blending of reality and virtuality. Furthermore, handling complex materials, especially transparent materials based on physical refraction, remains challenging. Existing 3D reconstruction techniques for transparent objects typically either heavily rely on dedicated hardware or incur significant training and inference time overhead, falling far short of real-time rendering requirements. Summary of the Invention

[0006] The purpose of this invention is to provide a method for digital spatial modeling and inverse rendering of power grids based on 3D Gaussian splatting. This method integrates 3D Gaussian splatting, neural rendering technology, and physically based optical transmission for digital spatial modeling and real-time rendering of power grids. It enables high-fidelity, lightweight modeling of complex power grid scenes and, combined with a physically based inverse rendering mechanism for complex materials (such as reflective and transparent materials), achieves a realistic reproduction of the optical properties of the real world.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] The steps of the method for digital spatial modeling and inverse rendering of power grids based on 3D Gaussian are as follows:

[0009] Image data of the target scene is acquired and preprocessed. Keyframes are extracted from the acquired image data at fixed time intervals, and image quality is automatically filtered. This invention utilizes two types of transparent object datasets: one is an existing publicly available transparent object dataset, and the other is a virtual transparent object dataset additionally constructed using 3D modeling and rendering software (such as Blender). The virtual dataset ensures the diversity and coverage of training data by parametrically controlling geometric shapes and material properties (such as refractive index, thickness, and smoothness) and supplementing them with precise label information (such as object segmentation, geometric ground truth, and lighting conditions).

[0010] Fine-tuning of the SAM2 model, DiffusionRenderer model, and VGGT model using a transparent object dataset;

[0011] The fine-tuned SAM2 model segments the image into Gaussian segments for transparent objects and Gaussian segments for opaque environments. The fine-tuned DiffusionRenderer model and the fine-tuned VGGT model are then used to estimate the Gaussian information for transparent objects.

[0012] Joint optimization and reconstruction of Gaussian mesh for transparent objects and triangular mesh; joint optimization and reconstruction of Gaussian mesh for opaque environment and transparent object.

[0013] A new scene is formed by reconstructed transparent objects, reconstructed environment, other reconstructed power grid scenes, and triangular meshes created by artists, achieving virtual-real fusion and mixed real-time rendering.

[0014] In the fine-tuning using a transparent object dataset, for the SAM2 model, the focus was on improving its robustness and accuracy in transparent object segmentation scenarios, ensuring accurate separation of the boundaries between transparent and non-transparent objects. For the DiffusionRenderer model, the original material parameter space based on metallicity and roughness was modified to introduce refractive index as the main characterization parameter, to better reflect the physical properties of transparent materials. For the VGGT model, additional transparent object training data was used to enhance its joint geometry and lighting modeling capabilities, enabling it to maintain high-fidelity inference of point clouds and camera poses under transparent materials.

[0015] According to the above technical solution, the multi-view image obtained from the first step of SAM2 segmentation is fine-tuned to obtain the transparent object part and the opaque environment part. The opaque environment part can be directly estimated by using a Structure-from-Motion (SfM) tool (such as COLMAP) to estimate the camera intrinsic and extrinsic parameters (pose, focal length, etc.) of each image, and initialize the opaque environment Gaussian E-3DGS. E-3DGS includes the basic information of traditional 3DGS, such as mean, covariance Σ, opacity o, and appearance information spherical harmonic function K.

[0016] The fine-tuned DiffusionRenderer model estimates the material of the transparent object and initializes the Gaussian SVBSDF material information, refractive index Σ, and additional geometric information and normal σ of the transparent object. The fine-tuned VGGT estimates the geometry of the transparent object and initializes the mean, covariance Σ, and opacity σ of the Gaussian object.

[0017] According to the above technical solution, the joint optimization and reconstruction steps of the transparent object Gaussian mesh and triangular mesh include:

[0018] S1. Gaussian extracts triangular meshes from transparent objects using a GPU-based differentiable network extraction algorithm;

[0019] S2. Gaussian extracts the spatially varying bidirectional scattering density function of a transparent object using a GPU-based differentiable SVBSDF extraction algorithm.

[0020] S3. Based on the triangular mesh and spatially varying bidirectional scattering density function, Gaussian mixing of transparent objects is performed using alpha mixing to obtain a G-buffer, and the loss function is calculated to constrain the physical consistency of geometric and material properties.

[0021] In step S4, the G-buffer calculates the outgoing rays based on the incident rays using differentiable ray tracing. The position and direction of the outgoing rays serve as guidance for the Gaussian ray tracing of the transparent object in step S3, acting similarly to a view frustum. This avoids redundant computational overhead caused by Gaussian rays outside the view frustum participating in the calculation, thus improving the efficiency of Gaussian ray tracing. By constructing a constraint chain of "surface properties → outgoing rays → Gaussian ray tracing of transparent objects" through the G-buffer and differentiable ray tracing, the rendering of transparent objects conforms to physical laws and can be efficiently integrated into the scene. The key lies in using the position and direction of the outgoing rays to reduce the sampling space of the Gaussian body, optimize the scattering direction, and achieve joint parameter optimization through differentiability, ultimately improving the realism and efficiency of transparent scene rendering.

[0022] S5. Outgoing Ray Guidance: The neural Gaussian light transmission field is generated through a dial-up process to train and calculate the accelerated light transmission within a transparent object. During the dial-up stage, priority is given to sampling the beam transmission path related to the outgoing ray (such as the beam from the light source to the surface and then refracted into the interior of the transparent object), so that the sample distribution is focused on the effective transmission path, reducing redundant sampling in irrelevant high-dimensional spaces.

[0023] During neural network training, the transmission results corresponding to the outgoing light rays are used as "supervision signals" to force the network to prioritize fitting the transmission patterns of these key paths, avoiding blind learning in high-dimensional space and accelerating convergence.

[0024] The accelerated results of the transport field feed back into Gaussian ray tracing (O-3DGS), enabling it to calculate outgoing rays more efficiently; and the new outgoing rays can further optimize the sampling and training of the transport field, forming a closed loop of "accelerated transport field → more accurate outgoing rays → more efficient transport field training".

[0025] Based on the principle that when the geometry and material are determined, the same incident light will always produce a definite outgoing light when the optimization of a transparent object converges, mesh and SVBSDF obtain a G-buffer that calculates the outgoing light (such as reflected and refracted rays) based on the incident light rays through differentiable ray tracing. According to the above technical solution, the G-buffer:

[0026] ;

[0027] In the formula, Indicates cumulative transmittance. Indicates mixed weights, Specifically, it includes SVBSDF material information, refractive index ν, additional geometric information, and normal ν. v is the pixel coordinate; i is the subscript, representing the Gaussian index within pixel v. express G-buffer.

[0028] G-buffer stores the geometric and material properties of the scene surface (such as depth, normal vector, position, roughness, etc.), which is the basis for differentiable ray tracing to calculate the interaction between incident rays and the surface.

[0029] According to the above technical solution, the triangular mesh and the spatially varying bidirectional scattering density function obtain two types of G-buffers through a differentiable rendering pipeline, and the Volume-to-Surface Consistency loss is calculated using these two types of G-buffers. This aims to constrain the physical consistency of geometric and material properties (such as surface position, normal vectors, and conservation of scattering energy), ensuring that the two match each other during the rendering process. Specifically, for vectors such as normals, cosine similarity loss is used.

[0030] ;

[0031] For other G-buffer scalars, an L1-based smoothing loss function is used, taking refractive index as an example:

[0032] ;

[0033] In the formula, The cosine similarity loss represents the normal vector. This represents the Gaussian normal to a transparent object. This represents the normal obtained from the mesh of a transparent object. The L1-based smoothing loss represents the refractive index. This represents the refractive index of a transparent object calculated by Gauss. This represents the refractive index of a transparent mesh.

[0034] According to the above technical solution, the steps of joint optimization and reconstruction of opaque environment Gaussian and transparent object Gaussian include:

[0035] An opaque Gaussian environment is transformed into camera space using a world-to-camera transformation matrix W, and then projected into two-dimensional image space using a locally affine approximation Jacobian matrix J to obtain a projected two-dimensional Gaussian environment. The rendering environment background image is obtained using a volumetric alpha blending method. ;

[0036] The opaque Gaussian environment is manifolded every t rounds to obtain or update the neural Gaussian incident light field;

[0037] The estimated outgoing ray from the neural Gaussian light transmission field samples the neural Gaussian incident light field, obtains the RGB or radiance of the intersection point of the ray and the manifold, and then performs coloring on the transparent object to render the transparent object image. Since the neuro-Gaussian incident light field can obtain the RGB or radiance of the intersection point of the light ray and the manifold based on any given position and direction (5 degrees of freedom);

[0038] render the environment background image and rendering transparent object images Blending is performed to obtain a blended image. .

[0039] Among them, the acceleration path based on the light field:

[0040] The outgoing light rays (i.e., the direction and position of the transmitted light rays emitted from a point on the surface of a transparent object) are estimated using the neural Gaussian light transmission field.

[0041] By querying the neural Gaussian incident light field using the "position + direction" of the outgoing ray, the radiance incident on the object surface in that direction (including all illumination contributions such as light source, ambient light, and reflected light from other objects) can be directly obtained.

[0042] By combining the optical properties of the object's surface (such as transmittance and dispersion coefficient), the incident radiance is converted into the final pixel color, thus completing the coloring process.

[0043] According to the above technical solution, the rendering environment background image :

[0044] ;

[0045] In the formula, N represents the number of environment Gaussian pixels within a single pixel. Indicates the index of the environment Gaussian pixel within a single pixel. Indicates cumulative transmittance. Indicates that the pixel v index is The Gaussian spherical harmonic function at v.

[0046] According to the above technical solution, the mixed image :

[0047] ;

[0048] In the formula, A mask representing a transparent object.

[0049] According to another aspect of the present invention, the present invention further provides an electronic device, comprising: one or more processors; and a storage device for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the three-dimensional Gaussian-based digital space modeling and inverse rendering method for power grids as described in any of the above technical solutions.

[0050] According to another aspect of the present invention, the present invention further provides a storage medium storing at least one instruction, which is loaded and executed by a processor to implement the three-dimensional Gaussian-based digital space modeling and inverse rendering method for power grids as described in any of the above technical solutions.

[0051] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0052] 1. This invention significantly outperforms existing technologies in the geometric reconstruction and optical property recovery of transparent objects. By combining physical optics laws with prior knowledge from neural networks, this invention can faithfully reproduce the refraction, reflection, and illumination characteristics of transparent materials. Simultaneously, by utilizing explicit scene representation based on Gaussian functions, ray tracing algorithms, and the organic integration with triangular meshes, a more efficient optimization process is achieved, thereby significantly reducing reconstruction time while maintaining high quality.

[0053] 2. This invention breaks through the dependence of traditional transparent object reconstruction methods on special hardware (such as polarization cameras, dedicated optical devices, etc.). It can reconstruct complex materials and transparent objects in power grid scenarios using only ordinary consumer-grade cameras. This is thanks to the introduction of a modeling skeleton based on end-to-end optimization of 3DGS and neural network priors, which effectively avoids complex and expensive acquisition devices and has greater universality and application value.

[0054] 3. This invention achieves a breakthrough improvement in rendering performance. By employing triangular mesh ray tracing to guide Gaussian ray tracing, and combining it with a neural Gaussian light transport field to pre-calculate and optimize light transport paths, while simultaneously using a neural Gaussian incident light field to manifold model ambient light, the computational overhead during rendering is significantly reduced. This technology not only supports realistic rendering of large-scale power grid scenes but also meets the needs of applications with extremely high real-time requirements, such as power grid operation monitoring and emergency command.

[0055] 4. This invention breaks through the bottleneck of traditional methods' difficulty in integrating transparent materials and triangular meshes by efficiently unifying geometry, materials, and lighting modeling. It achieves realistic mixed rendering of traditional mesh models, reconstructed meshes, and 3DGS-based representations within the same scene. This capability not only supports the natural integration of virtual power grid models with real power grid environments but also enables physical interactions between virtual and real objects (such as the interaction of reflection and refraction), significantly enhancing the immersiveness and practicality of digital twin systems. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0057] Figure 1 This is a flowchart of the steps of the three-dimensional Gaussian digital space modeling and inverse rendering method for power grids in this invention;

[0058] Figure 2 These are images of the Stanford bunny (a transparent object).

[0059] Figure 3 It is a rendered image of the Stanford Bunny (a transparent object). Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] Taking the example of recombining the reconstructed power grid scene (environment) with the reconstructed transparent objects—a mouse, a triangular mesh sphere, and a Stanford bunny (transparent objects)—to create a new scene, this paper explains the 3D Gaussian digital space modeling and inverse rendering method for power grids, detailing its specific steps. Figure 1 )include:

[0062] Step 1: Using a regular consumer-grade camera, capture images of transparent objects such as mice, triangular mesh balls, and Stanford bunnies (transparent objects) from multiple angles and perspectives. The captured images of the Stanford bunnies (transparent objects) are shown below. Figure 2 As shown. During the acquisition process, the camera should be set to high-resolution mode to ensure complete preservation of geometric details and lighting characteristics in the subsequent reconstruction process. The acquired data can be either multiple still images or a continuous video sequence to meet the needs of different scene scales and reconstruction tasks.

[0063] Step 2: Fine-tune the SAM2 model, DiffusionRenderer model, and VGGT model using the transparent object dataset. Specifically:

[0064] Two types of transparent object datasets are used: one is an existing publicly available transparent object dataset, and the other is a virtual transparent object dataset constructed additionally using 3D modeling and rendering software (such as Blender). The virtual dataset ensures the diversity and coverage of training data by parametrically controlling the geometry and material properties (such as refractive index, thickness, smoothness, etc.) and supplementing them with accurate label information (such as object segmentation, geometric ground truth, lighting conditions).

[0065] A large dataset containing transparent objects was input into network models such as SAM2 (Segment Anything Model 2), DiffusionRenderer, and VGGT (Visual Geometry Grounded Transformer) for fine-tuning. For SAM2, the focus was on improving its robustness and accuracy in transparent object segmentation scenarios, ensuring accurate separation of the boundaries between transparent and non-transparent objects. For DiffusionRenderer, the original material parameter space based on metallicity and roughness was modified to introduce refractive index as the main characterization parameter to better reflect the physical properties of transparent materials. For VGGT, additional transparent object training data was used to enhance its joint geometry and lighting modeling capabilities, enabling it to maintain high-fidelity inference of point clouds and camera poses under transparent materials.

[0066] Step 3: The fine-tuned SAM2 model segments the image into Gaussian segments for transparent objects and opaque environments. The fine-tuned DiffusionRenderer and VGGT models are then used to estimate the Gaussian information for the transparent objects. Specifically:

[0067] The fine-tuned SAM2 segmentation step one yields multi-view images, thus obtaining transparent object parts and opaque environment parts. The opaque environment parts can be directly estimated using Structure-from-Motion (SfM) tools (such as COLMAP) to estimate the camera intrinsic and extrinsic parameters (pose, focal length, etc.) of each image, and the opaque environment Gaussian E-3DGS is initialized. E-3DGS includes the traditional 3DGS basic information mean φ, covariance Σ, opacity o, and appearance information spherical harmonic function K.

[0068] The opaque environment portion is input to the DiffusionRenderer and VGGT obtained from the fine-tuning in step two. The DiffusionRenderer estimates the material of the transparent object, initializes the Gaussian SVBSDF material information of the transparent object, the refractive index φ, and the additional geometric information normal φ. The VGGT estimates the geometry of the transparent object, initializes the basic information of the Gaussian object, the mean φ, the covariance Σ, and the opacity ω.

[0069] Step 4: Joint optimization and reconstruction of Gaussian mesh for transparent objects and triangular mesh, and joint optimization and reconstruction of Gaussian mesh for opaque environment and transparent objects. Specific steps include:

[0070] S4.1. The opaque environment Gaussian is transformed into camera space through the world-to-camera transformation matrix W, and then projected into the two-dimensional image space through the Jacobian matrix J of the local affine approximation to obtain the projected two-dimensional Gaussian. The rendering environment background image is obtained using a volumetric alpha blending method. ;

[0071] ;

[0072] In the formula, N represents the number of environment Gaussian pixels within a single pixel. Indicates the index of the environment Gaussian pixel within a single pixel. Indicates cumulative transmittance. Indicates that the pixel v index is The Gaussian spherical harmonic function at v.

[0073] S4.2. The neuro-Gaussian incident light field is obtained or updated by iterating the opaque Gaussian environment every t rounds.

[0074] S4.3, Transparent Object Gaussian Extraction: Triangular meshes are extracted using a GPU-based differentiable network extraction algorithm.

[0075] S4.4. For transparent objects, Gaussian extracts the spatially varying bidirectional scattering density function using a GPU-based differentiable SVBSDF extraction algorithm.

[0076] S4.5 Based on triangular mesh and spatially varying bidirectional scattering density function, Gaussian mixing of transparent objects is performed using alpha mixing to obtain G-buffer, and loss function is calculated to constrain the physical consistency of geometric and material properties.

[0077] ;

[0078] In the formula, Indicates cumulative transmittance. Indicates mixed weights, Specifically, it includes SVBSDF material information, refractive index, additional geometric information, and normals. v is the pixel coordinate; i is the subscript, indicating the Gaussian index within pixel v. express G-buffer.

[0079] The extracted mesh and SVBSDF are processed through a differentiable rendering pipeline to obtain a G-buffer. The volume-to-surface consistency loss is calculated using these two G-buffers, specifically using cosine similarity loss for vectors such as normals.

[0080] ;

[0081] For other G-buffer scalars, an L1-based smoothing loss function is used, taking refractive index as an example:

[0082] ;

[0083] In the formula, The cosine similarity loss represents the normal vector. This represents the Gaussian normal to a transparent object. This represents the normal obtained from the mesh of a transparent object. The L1-based smoothing loss represents the refractive index. This represents the refractive index of a transparent object calculated by Gauss. This represents the refractive index of a transparent mesh.

[0084] In step S4.6, the mesh and SVBSDF obtain a G-buffer and calculate the outgoing ray based on the incident ray using differentiable ray tracing. The position and direction of the outgoing ray are used as a guide for Gaussian ray tracing of the transparent object in step S4.5. Specifically, it plays a role similar to the view frustum, avoiding redundant computational overhead caused by Gaussian rays outside the view frustum participating in the calculation, thereby improving the efficiency of Gaussian ray tracing.

[0085] S4.7 Based on the principle that when the optimization of a transparent object converges, and given the geometry and material, the same incident light will always produce a definite outgoing light, the outgoing light ray is obtained through a combination of mesh and O-3DGS ray tracing. To avoid redundant ray tracing overhead, a neural Gaussian light transport field is generated by table generation. This transport field is generated based on O-3DGS and is used to further accelerate the calculation process of light transport within the transparent object. Since the Light Transport Field is a very high-dimensional function, to ensure its training convergence, the outgoing light ray is guided by the neural Gaussian light transport field generated by table generation.

[0086] S4.8 The outgoing ray sampled from the neural Gaussian incident light field of the estimated neural Gaussian light transmission field is used to obtain the RGB or radiance of the intersection point of the ray and the manifold, and then the transparent object is colored to render the transparent object image. ;

[0087] S4.9, render the background image of the environment. and rendering transparent object images Mixing images Among them, the Stanford bunny (a transparent object) is like Figure 3 As shown;

[0088] ;

[0089] In the formula, The mask representing a transparent object is part of the G-buffer.

[0090] Step 5: A new scene is formed by the reconstructed transparent objects, the reconstructed environment, other reconstructed power grid scenes, and triangular meshes created by the artist, achieving virtual-real fusion and mixed real-time rendering.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0092] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for digital spatial modeling and inverse rendering of power grids based on 3D Gaussian geometry, characterized in that, The steps include: Collect image data of the target scene and preprocess the collected image data; Fine-tuning of the SAM2 model, DiffusionRenderer model, and VGGT model using a transparent object dataset; The fine-tuned SAM2 model segments the image into Gaussian segments for transparent objects and Gaussian segments for opaque environments. The fine-tuned DiffusionRenderer model and the fine-tuned VGGT model are then used to estimate the Gaussian information for transparent objects. The joint optimization and reconstruction of transparent objects using Gaussian and triangular meshes involves the following steps: S1. Gaussian extracts triangular meshes from transparent objects using a GPU-based differentiable network extraction algorithm; S2. Gaussian extracts the spatially varying bidirectional scattering density function of a transparent object using a GPU-based differentiable SVBSDF extraction algorithm. S3. Based on the triangular mesh and spatially varying bidirectional scattering density function, Gaussian mixing of transparent objects is performed using alpha mixing to obtain a G-buffer, and the loss function is calculated to constrain the physical consistency of geometric and material properties. S4. G-buffer calculates the outgoing ray based on the incident ray using differentiable ray tracing, and uses the position and direction of the outgoing ray as a guide for Gaussian ray tracing of the transparent object in step S3. S5. The outgoing light beam is trained by generating a neural Gaussian light transmission field through a dial indicator, which calculates and accelerates the transmission of light beams within a transparent object. The steps of joint Gaussian optimization and reconstruction of opaque environments and transparent objects include: An opaque Gaussian environment is transformed into camera space using a world-to-camera transformation matrix W, and then projected into the two-dimensional image space using a locally affine approximation Jacobian matrix J to obtain a projected two-dimensional Gaussian environment. The rendering environment background image is obtained using a volumetric alpha blending method. ; The opaque Gaussian incident light field is obtained or updated by iterating the Gaussian light field in the opaque environment every t rounds. The estimated outgoing ray from the neural Gaussian light transmission field samples the neural Gaussian incident light field, obtains the RGB or radiance of the intersection point of the ray and the manifold, and then performs coloring on the transparent object to render the transparent object image. ; render the environment background image and rendering transparent object images Blending is performed to obtain a blended image. ; A new scene is formed by reconstructed transparent objects, reconstructed environment, other reconstructed power grid scenes, and triangular meshes created by artists, achieving virtual-real fusion and mixed real-time rendering.

2. The method for digital spatial modeling and inverse rendering of power grids based on three-dimensional Gaussian as described in claim 1, characterized in that, The fine-tuned DiffusionRenderer model estimates the material of the transparent object and initializes the Gaussian SVBSDF material information, refractive index Σ, and additional geometric information and normal σ of the transparent object. The fine-tuned VGGT estimates the geometry of the transparent object and initializes the mean, covariance Σ, and opacity σ of the Gaussian object.

3. The method for digital spatial modeling and inverse rendering of power grids based on three-dimensional Gaussian as described in claim 1, characterized in that, The G-buffer: ； In the formula, Indicates cumulative transmittance. Indicates mixed weights, This represents the SVBSDF material information, refractive index ν, additional geometric information, and normal ν. v is the pixel coordinate; i is the subscript, representing the Gaussian index within pixel v. express G-buffer.

4. The method for digital spatial modeling and inverse rendering of power grids based on three-dimensional Gaussian as described in claim 1, characterized in that, The triangular mesh and the spatially varying bidirectional scattering density function are used to obtain two types of G-buffers through a differentiable rendering pipeline.

5. The method for digital spatial modeling and inverse rendering of power grids based on three-dimensional Gaussian as described in claim 1, characterized in that, The rendering environment background image : ； In the formula, N represents the number of environment Gaussian pixels within a single pixel. Indicates the index of the environment Gaussian pixel within a single pixel. Indicates cumulative transmittance. Indicates that the pixel v index is The Gaussian spherical harmonic function at v.

6. The method for digital spatial modeling and inverse rendering of power grids based on three-dimensional Gaussian as described in claim 1, characterized in that, The mixed image : ； In the formula, A mask representing a transparent object.

7. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method according to any one of claims 1-6.

8. A storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method as described in any one of claims 1-6.

Images