Fluid material reconstruction method and system based on three-dimensional Gaussian sputtering and physical prior

Abstract

The invention provides a fluid material reconstruction method and system based on three-dimensional Gaussian sputtering and physical prior, and the method comprises the steps: S1, reconstructing a time sequence three-dimensional geometric model of fluid through three-dimensional Gaussian representation based on a multi-frame image sequence of an input video, and estimating a velocity field of the surface of the fluid; s2, constructing an initial three-dimensional fluid volume velocity field, and constraining a physical boundary to obtain a physically reasonable three-dimensional fluid volume velocity field; and S3, taking the physically reasonable three-dimensional fluid volume velocity field as a supervision target, optimizing physical parameters of the fluid, further coupling the optimized physical parameters and the time sequence three-dimensional geometric model, and generating dynamic fluid assets capable of physical simulation. According to the method, full-automatic reconstruction of the three-dimensional dynamic fluid material assets from the single-view video is realized, and compared with the existing technology depending on multiple views, manual adjustment or a non-physical method, the process automation degree and the physical consistency of reconstruction are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics and physical simulation technology, specifically relating to a method and system for fluid material reconstruction based on 3D Gaussian sputtering and physical priors. More specifically, it is a method and system for automatically extracting high-quality, editable, physically consistent 3D dynamic fluid assets from single-view videos. Background Technology

[0002] 3D content generation technology has garnered significant attention in recent years. It aims to combine traditional 3D modeling with cutting-edge generation techniques to reduce production costs, improve creative efficiency, and bring a new paradigm to digital asset production. Particularly in fields such as film, games, and virtual reality, there is a growing demand for fluid assets with realistic physical dynamics, such as smoke and water flow.

[0003] Manually creating such fluid assets is time-consuming and labor-intensive, requiring precise control of complex fluid dynamics and simulation parameters. Therefore, automatically generating physically consistent dynamic fluid assets from real videos is of great significance. Existing research has made some progress in combining 3D generation with physical simulation, but significant limitations remain. In 3D generation, methods based on neural radiation fields, such as NeRF and its dynamic extensions, D-NeRF and HyperNeRF, can achieve dynamic scene modeling, but they largely rely on multi-view input or synthetic data and are difficult to directly couple with physical simulation.

[0004] The recently emerging 3D Gaussian splashing technology, with its explicit representation and real-time rendering capabilities, has been widely used in static scene reconstruction. Its generative methods, such as DreamGaussian and TriplaneGaussian, while supporting the generation of 3D content from single images, are mostly designed for rigid or elasto-plastic bodies, and have limited ability to handle dynamic objects like fluids, which have no fixed shape and complex internal motion. In terms of physical simulation, traditional methods such as Eulerian meshes, Lagrange particle methods, and hybrid methods can achieve high-quality fluid simulations, but they require pre-setting geometric and physical parameters and boundary conditions, and cannot directly invert simulateable fluid assets from real videos.

[0005] In recent years, some works have attempted to combine physical simulation with 3D representation. For example, PhysGaussian introduces the matter point method into Gaussian particles, GaussianSplashing combines positional dynamics to achieve fluid appearance and motion synthesis, and PhysDreamer optimizes material parameters through differentiable simulation to match input videos. However, most of these methods still rely on 3D geometric priors or multi-view observations, or can only reconstruct fluids for specific types. They have not yet achieved end-to-end automatic extraction of editable, simulateable, and physically consistent 3D dynamic fluid assets from single-view videos.

[0006] Therefore, there is an urgent need to propose a method that can automatically reconstruct physically consistent 3D dynamic fluid assets from single-view videos to address the shortcomings of existing technologies in terms of geometric reconstruction integrity, motion estimation accuracy, physical parameter optimizability, and asset editability.

[0007] This problem urgently needs to be solved. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide a fluid material reconstruction method and system based on three-dimensional Gaussian sputtering and physical priors.

[0009] According to the present invention, a fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical prior is provided, comprising: step S1: based on a multi-frame image sequence of an input video, a temporal three-dimensional geometric model of the fluid is reconstructed using three-dimensional Gaussian representation, and the velocity field of the fluid surface is estimated; Step S2: Based on the velocity field of the fluid surface, construct an initial three-dimensional fluid volume velocity field, and obtain a physically reasonable three-dimensional fluid volume velocity field after constraining the physical boundaries; Step S3: Using the physically reasonable three-dimensional fluid volume velocity field as the supervision target, optimize the physical parameters of the fluid, and then couple the optimized physical parameters with the temporal three-dimensional geometric model to generate a dynamic fluid asset capable of physical simulation.

[0010] Preferably, in step S1, consecutive [processes] are merged. The three-dimensional Gaussian geometric representation generated from the frame image sequence yields a temporal three-dimensional geometric model; The The frame rate is set based on the degree of dynamic change in image content between adjacent video frames, and the expression is:

[0011] in, Indicates the number of frames. This is the proportionality coefficient. For frame index, Indicates mean square error; This represents the peak signal-to-noise ratio.

[0012] Preferably, in step S1, estimating the velocity field of the fluid surface includes: calculating the displacement vector of each pixel in the two-dimensional screen space between adjacent video frames using an optical flow algorithm, and then combining this with the depth information provided by the Gaussian point cloud to inversely deduce the displacement in three-dimensional space and the three-dimensional velocity estimate; the depth information is... , Represents pixels Depth in the camera coordinate system; The displacement vector in the two-dimensional screen space is expressed as:

[0013] in, This represents a displacement vector in two-dimensional screen space. This represents the horizontal displacement component of the optical flow, i.e., the x-axis. This represents the displacement component of the optical flow in the vertical direction, i.e., the y-axis. The displacement in the three-dimensional space is expressed as:

[0014] in, Represents displacement in three-dimensional space. This represents the camera intrinsic parameter matrix used for video acquisition. These are pixel coordinates; The three-dimensional velocity estimation is expressed as follows:

[0015] in, This represents a three-dimensional velocity estimate. This is the inter-frame time interval.

[0016] Preferably, in step S2, the constrained physical boundary includes applying a boundary layer velocity decay model to the fluid particles of the rigid boundary, and the expression of the boundary layer velocity decay model is:

[0017] in, For the target point velocity, For the magnitude of the surface velocity, The distance to the boundary. For boundary layer thickness, This is the proportionality coefficient; The physically reasonable three-dimensional fluid volume velocity field is obtained by solving the divergence-free condition, that is, by estimating the velocity field of the fluid surface through the divergence-free projection of the Poisson pressure equation, and thus obtaining the physically reasonable initial volume velocity field. The expression for the Poisson pressure equation is as follows:

[0018] in, Represents the gradient operator; The symbol · represents the density of the fluid; the symbol · represents the product.

[0019] Preferably, in step S3, the physical parameters of the fluid are optimized using a gradient descent loss function; the gradient descent loss function is:

[0020] in, This represents the loss function of gradient descent. Represents the velocity field of the simulated volume; This represents the summation of all mesh elements used in the differentiable physics simulation; Represents a physically plausible three-dimensional fluid volume velocity field; Represents the L2 norm; and All are weighting coefficients; The optimized physical parameters are coupled with the temporal three-dimensional geometric model, including updating the affine transformation matrix of the corresponding Gaussian particle using the affine deformation matrix of the particle in the matter point method simulation, to achieve dynamic driving; the update expression is:

[0021] in, Represents the affine transformation matrix of a Gaussian particle. For the first The affine transformation matrix of a Gaussian particle at time step. For the first The affine deformation matrix corresponding to the material point at each step. For time step, It is the identity matrix. This represents the affine deformation matrix of particles in the material point method simulation.

[0022] A fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors, provided by the present invention, includes: Module M1: Based on a multi-frame image sequence of the input video, it reconstructs a temporal three-dimensional geometric model of the fluid using three-dimensional Gaussian representation and estimates the velocity field on the fluid surface; Module M2: Based on the velocity field of the fluid surface, an initial three-dimensional fluid volume velocity field is constructed, and a physically reasonable three-dimensional fluid volume velocity field is obtained after constraining the physical boundaries; Module M3: Using the physically reasonable three-dimensional fluid volume velocity field as the supervision target, optimize the physical parameters of the fluid, and then couple the optimized physical parameters with the temporal three-dimensional geometric model to generate dynamic fluid assets that can be physically simulated.

[0023] Preferably, in module M1, consecutive [processes] are merged. The three-dimensional Gaussian geometric representation generated from the frame image sequence yields a temporal three-dimensional geometric model; The The frame rate is set based on the degree of dynamic change in image content between adjacent video frames, and the expression is:

[0024] in, Indicates the number of frames. This is the proportionality coefficient. For frame index, Indicates mean square error; This represents the peak signal-to-noise ratio.

[0025] Preferably, in module M1, estimating the velocity field of the fluid surface includes: calculating the displacement vector of each pixel in the two-dimensional screen space between adjacent video frames using an optical flow algorithm, and then combining this with the depth information provided by the Gaussian point cloud to inversely deduce the displacement in three-dimensional space and the three-dimensional velocity estimate; the depth information is... , Represents pixels Depth in the camera coordinate system; The displacement vector in the two-dimensional screen space is expressed as:

[0026] in, This represents a displacement vector in two-dimensional screen space. This represents the horizontal displacement component of the optical flow, i.e., the x-axis. This represents the displacement component of the optical flow in the vertical direction, i.e., the y-axis. The displacement in the three-dimensional space is expressed as:

[0027] in, Represents displacement in three-dimensional space. This represents the camera intrinsic parameter matrix used for video acquisition. These are pixel coordinates; The three-dimensional velocity estimation is expressed as follows:

[0028] in, This represents a three-dimensional velocity estimate. This is the inter-frame time interval.

[0029] Preferably, in module M2, the constrained physical boundary includes applying a boundary layer velocity decay model to fluid particles at a rigid boundary, the expression of which is:

[0030] in, For the target point velocity, For the magnitude of the surface velocity, The distance to the boundary. For boundary layer thickness, This is the proportionality coefficient; The physically reasonable three-dimensional fluid volume velocity field is obtained by solving the divergence-free condition, that is, by estimating the velocity field of the fluid surface through the divergence-free projection of the Poisson pressure equation, and thus obtaining the physically reasonable initial volume velocity field. The expression for the Poisson pressure equation is as follows:

[0031] in, Represents the gradient operator; The symbol · represents the density of the fluid; the symbol · represents the product.

[0032] Preferably, in module M3, the physical parameters of the fluid are optimized using a gradient descent loss function; the gradient descent loss function is:

[0033] in, This represents the loss function of gradient descent. Represents the velocity field of the simulated volume; This represents the summation of all mesh elements used in the differentiable physics simulation; Represents a physically plausible three-dimensional fluid volume velocity field; Represents the L2 norm; and All are weighting coefficients; The optimized physical parameters are coupled with the temporal three-dimensional geometric model, including updating the affine transformation matrix of the corresponding Gaussian particle using the affine deformation matrix of the particle in the matter point method simulation, to achieve dynamic driving; the update expression is:

[0034] in, Represents the affine transformation matrix of a Gaussian particle. For the first The affine transformation matrix of a Gaussian particle at time step. For the first The affine deformation matrix corresponding to the material point at each step. For time step, It is the identity matrix. This represents the affine deformation matrix of particles in the material point method simulation.

[0035] Compared with the prior art, the present invention has the following beneficial effects: 1. The end-to-end generation framework based on generative 3D Gaussian splashing and physical prior proposed in this invention realizes fully automatic reconstruction of 3D dynamic fluid material assets from single-view videos. Compared with existing technologies that rely on multiple views, manual adjustments or non-physical methods, its process automation and physical consistency of reconstruction are significantly improved.

[0036] 2. The physical parameters of the fluid assets constructed by this invention can be automatically optimized to match multiple fluid types, and have high editability and scene interaction capabilities. Compared with the existing four-dimensional asset generation model, it can effectively reduce the production and iteration costs of three-dimensional fluid effects and reduce the complexity of subsequent parameter adjustments while ensuring physical realism.

[0037] 3. This invention enables the fully automated generation of high-quality, physically consistent, and editable 3D dynamic fluid assets from single-view videos, significantly reducing the cost of fluid content production. Attached Figure Description

[0038] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A simplified flowchart illustrating the fluid material reconstruction method provided by this invention; Figure 2 A schematic diagram of the system architecture flow of the fluid material reconstruction method provided by the present invention; Figure 3 A schematic diagram of the denoising and interior filling results of the three-dimensional Gaussian geometric representation provided by the present invention; Figure 4 A schematic diagram of the boundary layer velocity decay model provided by the present invention; wherein, BC Indicates a rigid boundary. Indicates the boundary layer; Figure 5 A schematic diagram of the three-dimensional volume velocity field estimation results provided by the present invention; Figure 6 A schematic diagram illustrating the changes in the loss function during the three fluid optimization processes provided by this invention; Figure 7 A schematic diagram of the fluid assets after river reconstruction provided by this invention; Figure 8 A schematic diagram comparing the smoke reconstruction results provided by this invention with the FluidNexus method; Figure 9 A comparative schematic diagram of viscous fluids generated by different three-dimensional Gaussian splash generation models provided by the present invention; Figure 10 This is a schematic diagram comparing the SSIM with the baseline SSIM in all embodiments provided by the present invention; Figure 11 A comparison chart showing the percentage of execution time in different system stages for the three fluids provided in this invention. Detailed Implementation

[0039] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0040] To address the challenge of automatically generating physically consistent, editable 3D dynamic fluid assets from single-view videos in existing technologies, this invention aims to provide a fluid asset generation method based on generative 3D Gaussian splashing and differentiable physics simulation. This method can automatically reconstruct high-quality dynamic fluid assets end-to-end from readily available single-view videos, significantly reducing the production cost of 3D fluid effects.

[0041] This invention provides a method for generating 3D dynamic fluid assets from single-view video, comprising: reconstructing the 3D geometric structure of the fluid from the single-view video using generative 3D Gaussian splashing technology, and filling and aligning the Gaussian point cloud generated from multiple frames; extracting the 3D velocity field of the fluid surface based on optical flow and depth information, and obtaining the 3D volume velocity field as a physical prior by interpolation with the fluid velocity model; converting the velocity field into a mesh representation, and optimizing parameters such as fluid density, viscosity, and boundary conditions through differentiable physical simulation to make the simulation behavior consistent with the video dynamics; and finally outputting an editable 3D fluid asset with physical properties, supporting material replacement, scene fusion, and interactive expansion.

[0042] Specifically, according to the present invention, a fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors includes: Step S1: Reconstructing fluid geometry and surface motion based on generative Gaussian splash; using an image-conditional generative 3D Gaussian splash model, an initial 3D fluid geometry representation is independently generated for each frame of the input video; through voxel filling and multi-frame geometric union operations, a dense and temporally consistent 3D Gaussian point cloud model is obtained; combining inter-frame optical flow and depth information, the 3D velocity field of the fluid surface is estimated and corrected.

[0043] Step S2: Construction of three-dimensional fluid volume velocity field and injection of physical constraints; interpolation and propagation of the surface velocity field into the fluid internal volume to initialize the overall motion; application of corresponding physical boundary constraints for different types of fluids.

[0044] For example, for liquids flowing along a riverbed, the near-wall velocity decay is modeled based on boundary layer theory; finally, a physically reasonable initial volumetric velocity field is obtained by solving for the divergence-free condition. The different types of fluids include gases and viscous liquids.

[0045] Step S3: Fluid parameter optimization based on differentiable physical simulation; based on the material point method, the velocity field on the Gaussian particle is transformed into a regular mesh, and a differentiable pressure solution and convection calculation module is integrated; using the volumetric velocity field obtained in the previous step as the supervised target, the physical parameters of the fluid are optimized through gradient descent; the physical parameters include density. Viscosity Inlet / outlet velocity boundary conditions , and gravity .

[0046] Step S4: Generation, rendering, and editing of dynamic fluid assets; The optimized physical parameters are coupled with 3D Gaussian geometry to generate dynamic fluid assets carrying physical properties; This asset supports real-time visualization through a standard 3D Gaussian splash renderer and allows users to edit the fluid's physical properties or import it into a new scene for physical interaction simulation with other objects.

[0047] The multi-frame geometric union operation in step S1 includes: combining consecutive frames... The generated and padded 3D Gaussian geometric representations of the frames are merged to form a unified 3D Gaussian point cloud model with higher geometric consistency; among which, the number of merged frames... The determination is dynamically based on the degree of dynamic change in image content between adjacent video frames.

[0048] The degree of dynamic change is assessed by calculating the mean square error of the peak signal-to-noise ratio of adjacent frames, and the number of merged frames is considered. It is directly proportional to the degree of dynamic change:

[0049] in, For frame index.

[0050] Specifically, the voxelization filling operation in step S1 includes: voxelizing the three-dimensional space, and for voxel centers determined to be inside the fluid, inserting new Gaussian particles to enhance geometric integrity.

[0051] Specifically, the degree of dynamic change is evaluated by calculating the mean square error of the peak signal-to-noise ratio of adjacent frames. The number of merged frames N is proportional to the degree of dynamic change, and the formula is as follows:

[0052] in, This is the proportionality coefficient. For frame index. Indicates mean square error; This represents the peak signal-to-noise ratio.

[0053] Specifically, the surface velocity field estimation in step S1 includes: calculating the two-dimensional displacement vector between adjacent video frames using an optical flow algorithm, combining the depth information corresponding to the pixel, calculating the three-dimensional spatial displacement using the camera back projection formula, and then obtaining the three-dimensional velocity estimate of the surface point.

[0054] Specifically, the surface velocity field estimation in step S1 includes: calculating the displacement vector of each pixel in the two-dimensional screen space between adjacent video frames using an optical flow algorithm. Combined with the depth information provided by Gaussian point clouds Inversely calculate the displacement in three-dimensional space The specific formula is as follows:

[0055] in, For the camera intrinsic parameter matrix, Pixel coordinates. 3D velocity estimation of surface points in camera coordinates. , This is the inter-frame time interval.

[0056] Specifically, in step S2, a boundary layer velocity decay model is applied to the fluid particles at the rigid boundary, i.e., for particles at a distance to the rigid boundary less than a preset boundary layer thickness. For fluid particles, a boundary layer velocity decay model is applied, wherein the boundary layer velocity decay model is:

[0057] in, For the target point velocity, For the magnitude of the surface velocity, This is a constant proportionality coefficient, i.e., the proportionality constant. The distance to the boundary. The boundary layer thickness is set differently based on the fluid type.

[0058] Specifically, solving the divergence-free condition in step S2 includes: solving the Poisson pressure equation. The initial volume velocity field is obtained by performing a divergence-free projection on the interpolated velocity field. In other words, through Poisson's pressure equation By performing a divergence-free projection on the velocity field, a physically plausible initial volume velocity field is obtained. . Represents the gradient operator; This indicates the density of the fluid.

[0059] Specifically, the differentiable pressure solution and convection calculation module in step S3 includes: Taichi's DiffMPM module, which replaces the traditional iterative solver to achieve efficient gradient propagation and optimization convergence. The differentiable pressure solution and convection calculation module is implemented based on the differentiable material point method.

[0060] Specifically, the optimization algorithm in step S3 adopts the gradient descent method, and the loss function of the gradient descent method is: in, To simulate the velocity field of a volume, , These are the weighting coefficients.

[0061] Specifically, the coupling of physical parameters with three-dimensional Gaussian geometry in step S4 includes: the affine deformation matrix of particles in the matter point method simulation. Used to update the affine transformation matrix corresponding to the Gaussian particle. This enables the dynamic driving of physical motion on Gaussian geometry. The specific update formula is as follows: in, For the first The affine transformation matrix of a Gaussian particle at time step. For the first The affine deformation matrix corresponding to the material point at each step. For time step, It is an identity matrix.

[0062] The following specific embodiments illustrate the applicability of the method of the present invention to different types of fluids, such as ordinary liquids, viscous fluids, and gases, as well as the specific implementation process and effects.

[0063] Example 1: This embodiment demonstrates how the method of the present invention can automatically generate a physically consistent and interactive 3D dynamic river asset from a fixed-viewpoint video of river flow. The input to this embodiment is a 5-second, 1920×1080 resolution, 30fps fixed-viewpoint video of river flow in MP4 format. The output is a fluid asset that can be rendered in a 3D scene, whose water flow dynamics match the video, and is interactive. The overall implementation process of this embodiment follows... Figure 2 The framework shown is illustrated, and the specific steps are as follows: Step S100: Prepare the video data of the river and generate the corresponding 3D Gaussian geometry. In this step, the input video is first preprocessed, and the 3D fluid geometry of each frame is reconstructed using a generative 3D Gaussian model. Step S110: Preprocess the video, decoding it into a continuous sequence of image frames. To improve subsequent generation efficiency, each frame is downsampled to 512×512 resolution, and camera intrinsic parameters are extracted. If not provided in the video metadata, estimation is performed. Step S120: Use the Transformer-based single-graph generation model TriplaneGaussian as the base generator. Generate a 3D Gaussian geometric model frame by frame. For each frame of image... enter To obtain the initial three-dimensional Gaussian representation This includes position, opacity, covariance matrix, and spherical harmonic features. Step S130, generating for each frame... Post-processing involves geometric denoising and filling. Denoising includes removing all opacities. Or, an anomalous Gaussian particle whose largest eigenvalue of the covariance matrix is ​​more than 5 times the average eigenvalue.

[0064] The filling process includes: defining a voxel grid, where the voxel size in this embodiment is 0.01 world units, and traversing the grid cells. For each cell center point... Calculate the number of intersections between the ray and the outer envelope of all Gaussian particles. If the number of intersections is odd, then... Inside the fluid, a new Gaussian particle with average characteristics is inserted at that location. Step S140: To obtain a stable and complete Gaussian geometric representation of the river, a multi-frame geometric union is performed, selecting continuous frames from the video. Gaussian set after frame processing Perform union operation This serves as the input geometry for subsequent processes. The value is dynamically determined by the degree of dynamic change between adjacent frames, and the specific formula is as follows:

[0065] in, , For frame index.

[0066] Step S200: Estimate the three-dimensional fluid velocity field of the river and inject physical constraints. In this step, a physically plausible three-dimensional volumetric velocity field is reconstructed based on the generated geometry and video sequence. Step S210: Surface velocity estimation, calculating the dense two-dimensional optical flow field between consecutive frames using the Farnebäck optical flow method. Using the 3D Gaussian representation with depth obtained in step S130, the 2D optical flow is combined with pixel depth variation, and the 3D displacement in screen space is calculated through camera back projection. The specific formula is as follows:

[0067] in, For the camera intrinsic parameter matrix, Pixel coordinates. 3D velocity estimation of surface points in camera coordinates. , In this embodiment, the inter-frame time interval is... .

[0068] To address the issue of optical flow failure in areas with uniform river surface texture, the Canny edge detector is used to extract the main edge directions of water flow in video frames, thus determining the fluid flow direction. For regions lacking optical flow Its speed is determined by the neighborhood Internal optical flow velocity It is obtained by interpolation projection along the fluid flow direction, and the weight depends on the consistency of the direction. The specific formula is as follows:

[0069] in, Indicates regions with missing optical flow The optical flow velocity.

[0070] Step S220: Perform volume velocity field interpolation and boundary constraints, and convert the surface point velocity field obtained in step S210 into... Bilinear interpolation to The nearest surface Gaussian particle. For all internally filled Gaussian particles, a boundary layer model is applied based on their distance to the nearest surface particle and their normal direction. For particles close to the "riverbed," i.e., the plane fitted by the geometric bottom point cloud, their velocity magnitude is attenuated according to the laminar boundary layer formula, while the direction maintains the main flow direction of the river. The specific formula is as follows:

[0071] in, To correspond to the most recent surface velocity magnitude, The boundary layer thickness is the vertical distance to the riverbed. Based on the river scale, it is set at 0.5 world units. is an empirical constant, i.e., a proportionality coefficient, set to 0.874. For particles in the inner non-boundary layer, their velocities are obtained by radial basis interpolation based on a Gaussian kernel function, propagating from the surface and the calculated particle velocities.

[0072] Finally, a fast divergence-free projection is performed on the velocity field composed of all Gaussian particles, and a simplified Poisson equation is solved using the preprocessed conjugate gradient method to obtain the initial, physically more reasonable volume velocity field. .

[0073] Step S300: Optimize fluid parameters based on differentiable physical simulation. In this step, the velocity field is converted to a mesh, and the physical parameters are optimized through differentiable simulation to dynamically approximate video observations. Step S310: Construct the mesh and differentiable simulator: Create an Eulerian simulation mesh with a resolution of 640×320×160, covering... The bounding box is defined. A differentiable material point method simulator, DiffMPM, is implemented using the Taichi language. This simulator is a differentiable solid line of the traditional material point method, capable of differentiable solutions for pressure and convection processes. In this embodiment, the pressure solver uses a modified physically encoded recurrent convolutional neural network, whose input is the divergence field of the velocity field and whose output is the pressure field. Through training, it is made to approximate the results of a traditional iterative solver. The convection term calculation module directly calculates the convection derivative. The analytical gradient.

[0074] Step S320: Optimize the material parameters of the fluid, defining the optimizable parameter of the fluid as density. Viscosity Inlet / outlet velocity boundary conditions , and gravity The result obtained in step S200 The target true value is used and transferred to the mesh as the initial state. DiffMPM simulation is then run to obtain the simulated velocity field. Specifically, in this embodiment, a fluid region mask is introduced. To distinguish between the fluid region and the spatial region, the loss function is modified as follows:

[0075] in, Masking to distinguish between fluid and airspace, , Using the Adam optimizer with a learning rate of 0.01, the parameters in step S320 were optimized for 500 iterations. The optimization process lasted approximately 4 hours on a device with an Intel Core i7-13700K and an NVIDIA GeForce RTX 3090 (24GB).

[0076] Step S400: Asset Integration, Rendering, and Parameter Editing. In this step, the optimized physical parameters are combined with a 3D Gaussian model to generate the final asset. Step S410: Dynamic Gaussian Asset Generation. The optimized parameters are injected into the DiffMPM simulator. As the initial particle position As the initial velocity, run the complete simulation. During the simulation, each Gaussian particle... Affine matrix The covariance matrix is ​​calculated in real time and used to update its rendering. The specific formula is as follows: Step S420: Import the dynamic Gaussian sequence into a real-time renderer that supports 3DGS, such as Unity's UnityGaussianSplatting plugin. Render using the standard Gaussian splash rasterization pipeline. Step S430: Modify the fluid's physical parameters, such as density. Viscosity Inlet / outlet velocity boundary conditions , This process generates fluid assets with different properties, resulting in new dynamic Gaussian sequences. To quantify the effectiveness of this embodiment, the fluid assets output by this invention are simulated and rendered in a 30-frame sequence from the same viewpoint. The structural similarity (SSIM) and optical flow endpoint error (EPE) are calculated against the input video. This serves as a baseline for comparison with methods that only use optical flow to drive Gaussian particles without physics optimization.

[0077] The results show that the average SSIM of the rendered sequence and the input video obtained by the method of this invention is 0.89, and the EPE is 1.2 pixels; while the baseline method has an SSIM of 0.76 and an EPE of 2.8 pixels. This indicates that the fluid dynamics generated by the present invention are more consistent with real observations in terms of both vision and motion. The fully automated processing time from single-view video to the generation of interactive assets is approximately 2 hours, excluding parameter optimization, with most of the time spent on 3D Gaussian generation and velocity field estimation.

[0078] Example 2: This embodiment demonstrates how the method of the present invention can automatically generate a physically consistent and interactive 3D dynamic smoke asset from a video of smoke rising from a fixed perspective. The input to this embodiment is a 6-second video of smoke rising, with a resolution of 1920×1080 and a frame rate of 30fps, in MP4 format. The output of this embodiment is a smoke volumetric asset that can be rendered in a 3D scene, whose dynamics match the video, and is interactive. The overall implementation process follows... Figure 2 The general framework shown only involves parameter adjustments and technical adaptations for key steps related to the physical properties of gases. Specific differences are explained below: Step S100: Prepare the video data of the smoke and generate the corresponding 3D Gaussian geometry. In this step, the input video is first preprocessed, and the 3D smoke geometry of each frame is reconstructed using a generative 3D Gaussian model.

[0079] Step S110 is the same as in Example 1. Step S120 selects ML-Sharp, which is better at generating complex shapes and volumetric effects, as the base generator. The three-dimensional Gaussian geometric model is generated frame by frame, and the other steps are the same as in Example 1. In step S130, since the smoke is a sparse volume rather than a continuous medium, strict voxelization is not used for internal filling. Instead, in the low-resolution density field obtained from the initial three-dimensional Gaussian rendering, new Gaussian particles are inserted probabilistically in regions where the density value is higher than the dynamic threshold, with the probability positively correlated with the density value. This enhances details in visually dense regions, such as the smoke core, and maintains reasonable sparsity in sparse regions. The other steps are the same as in Example 1. Step S140 is the same as in Example 1.

[0080] Step S200: Estimate the three-dimensional fluid velocity field of the smoke and inject physical constraints. In this step, a physically plausible three-dimensional volumetric velocity field is reconstructed based on the generated geometry and video sequence. Step S210: Consistent with Example 1.

[0081] Step S220: Unlike rivers, smoke is a free fluid and does not use a boundary layer model. Its internal velocity is obtained directly by interpolation of the nearest surface point, without considering attenuation and boundary conditions. The other steps are the same as in Example 1.

[0082] Step S300 is the same as in Example 1.

[0083] Step S400 is consistent with Example 1. To quantify the effect of this example, the fluid assets output by this invention are simulated and rendered in a 30-frame sequence from the same viewpoint. The structural similarity (SSIM) and optical flow endpoint error (EPE) are calculated with the input video. This serves as a baseline for comparison with a method that only uses optical flow to drive Gaussian particles without physics optimization.

[0084] The results show that the average SSIM of the rendered sequence and the input video obtained by the method of this invention is 0.92, and the EPE is 1.1 pixels; while the baseline method has an SSIM of 0.91 and an EPE of 1.3 pixels. This indicates that the fluid dynamics generated by the present invention are more consistent with real observations in terms of both vision and motion. The fully automated processing time from single-view video to the generation of interactive assets is approximately 1.5 hours, excluding parameter optimization, with most of the time spent on 3D Gaussian generation and velocity field estimation.

[0085] Example 3:This embodiment demonstrates how the method of the present invention can automatically generate a physically consistent and interactive 3D dynamic viscous fluid asset from a fixed-viewpoint video of viscous fluid flow. The input to this embodiment is an 8-second video of lava flow with a resolution of 1920×1080 and a frame rate of 30fps, in MP4 format. The output is a fluid asset that can be rendered in a 3D scene, whose viscous flow and morphological evolution highly match the video, and is interactive. The overall implementation process follows... Figure 2 The general framework shown has been adjusted in the following key ways to account for the properties of viscous fluids: Step S100: Prepare video data of the viscous fluid and generate the corresponding 3D Gaussian geometry. In this step, the input video is first preprocessed, and the 3D viscous fluid Gaussian geometry of each frame is reconstructed using a generative 3D Gaussian model.

[0086] Step S110 is the same as in Example 1. Step S120 is the same as in Example 1. Step S130 is the same as in Example 1. Step S140, because the viscous fluid moves slowly and the geometric changes between adjacent frames are small, a fixed number of 40 consecutive frames are selected, i.e. We process the Gaussian set and perform a union operation to obtain an extremely dense static fused geometry with a sufficient dynamic range.

[0087] Step S200: Estimate the three-dimensional volume velocity field of the viscous fluid and inject physical constraints.

[0088] Step S210: The TV-L1 optical flow algorithm is adopted, which has better robustness to changes in illumination and produces a smoother optical flow field that conforms to the characteristics of viscous flow. Other steps are the same as in Example 1.

[0089] Step S220, unlike rivers and smoke, applies a no-slip boundary condition to viscous fluid particles in contact with the boundary, i.e., the velocity is set to 0. For internal particles, the velocity is modeled using a viscous boundary layer model, which is an extension of the formula in Example 1 to reflect the effect of high viscosity. The specific formula is as follows:

[0090] in, The fluid viscosity proportionality coefficient is set to 1.5. Set to 0.3 world units, and the other steps are the same as in Example 1.

[0091] Step S300: Optimize fluid parameters based on differentiable physical simulation. In this step, the velocity field is converted to a mesh, and the physical parameters are optimized through differentiable simulation to make the simulation dynamically approximate video observations. Step S310: The mesh resolution is adjusted to 1280×320×640; other steps are the same as in Example 1.

[0092] In step S320, since viscous fluids move slowly and have rich texture details, relying solely on the velocity field as the optimization objective is insufficient. Therefore, structural similarity loss and peak signal-to-noise ratio loss are additionally introduced into the loss function, as shown in the following formulas:

[0093] in, This represents the velocity field matching loss term; The weighting coefficients represent the structural similarity loss term; The weighting coefficients for the peak signal-to-noise ratio loss term are indicated.

[0094] in, , , .

[0095] Step S400 is the same as in Example 1. Results show that the average SSIM of the rendered sequence of the method of the present invention is 0.87, and the EPE is 0.93 pixels; while the baseline method has an SSIM of 0.84 and an EPE of 1.2 pixels. This indicates that the fluid dynamics generated by the present invention are more visually and kinematically consistent with real-world observations. The fully automated processing time from single-view video to the generation of interactive assets is approximately 3 hours, excluding parameter optimization.

[0096] The present invention also provides a fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors. The fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors can be implemented by executing the process steps of the fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors. That is, those skilled in the art can understand the fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors as a preferred embodiment of the fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors.

[0097] A fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors, provided by the present invention, includes: Module M1: Based on a multi-frame image sequence of the input video, it reconstructs a temporal three-dimensional geometric model of the fluid using three-dimensional Gaussian representation and estimates the velocity field on the fluid surface; Module M2: Based on the velocity field of the fluid surface, an initial three-dimensional fluid volume velocity field is constructed, and a physically reasonable three-dimensional fluid volume velocity field is obtained after constraining the physical boundaries; Module M3: Using the physically reasonable three-dimensional fluid volume velocity field as the supervision target, optimize the physical parameters of the fluid, and then couple the optimized physical parameters with the temporal three-dimensional geometric model to generate dynamic fluid assets that can be physically simulated.

[0098] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0099] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for fluid material reconstruction based on three-dimensional Gaussian sputtering and physical priors, characterized in that, include: Step S1: Based on the multi-frame image sequence of the input video, reconstruct the temporal three-dimensional geometric model of the fluid using three-dimensional Gaussian representation, and estimate the velocity field on the fluid surface; Step S2: Based on the velocity field of the fluid surface, construct an initial three-dimensional fluid volume velocity field, and obtain a physically reasonable three-dimensional fluid volume velocity field after constraining the physical boundaries; Step S3: Using the physically reasonable three-dimensional fluid volume velocity field as the supervision target, optimize the physical parameters of the fluid, and then couple the optimized physical parameters with the temporal three-dimensional geometric model to generate a dynamic fluid asset capable of physical simulation.

2. The fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors according to claim 1, characterized in that, In step S1, consecutive ones are merged. The three-dimensional Gaussian geometric representation generated from the frame image sequence yields a temporal three-dimensional geometric model; The The frame rate is set based on the degree of dynamic change in image content between adjacent video frames, and the expression is: in, Indicates the number of frames. This is the proportionality coefficient. For frame index, Indicates mean square error; This represents the peak signal-to-noise ratio.

3. The fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors according to claim 1, characterized in that, In step S1, estimating the velocity field of the fluid surface includes: calculating the displacement vector of each pixel in the two-dimensional screen space between adjacent video frames using an optical flow algorithm, and then combining this with the depth information provided by the Gaussian point cloud to inversely deduce the displacement in three-dimensional space and the three-dimensional velocity estimate; the depth information is... , Represents pixels Depth in the camera coordinate system; The displacement vector in the two-dimensional screen space is expressed as: in, This represents a displacement vector in two-dimensional screen space. This represents the horizontal displacement component of the optical flow, i.e., the x-axis. This represents the displacement component of the optical flow in the vertical direction, i.e., the y-axis. The displacement in the three-dimensional space is expressed as: in, Represents displacement in three-dimensional space. This represents the camera intrinsic parameter matrix used for video acquisition. These are pixel coordinates; The three-dimensional velocity estimation is expressed as follows: in, This represents a three-dimensional velocity estimate. This is the inter-frame time interval.

4. The fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors according to claim 1, characterized in that, In step S2, the constrained physical boundary includes applying a boundary layer velocity decay model to fluid particles at a rigid boundary, the expression of which is: in, For the velocity of the target point, For the magnitude of the surface velocity, The distance to the boundary. Boundary layer thickness, This is the proportionality coefficient; The physically reasonable three-dimensional fluid volume velocity field is obtained by solving the divergence-free condition, that is, by estimating the velocity field of the fluid surface through the divergence-free projection of the Poisson pressure equation, and thus obtaining the physically reasonable initial volume velocity field. The expression for the Poisson pressure equation is as follows: in, Represents the gradient operator; The symbol · represents the density of the fluid; the symbol · represents the product.

5. The fluid material reconstruction method based on three-dimensional Gaussian sputtering and physical priors according to claim 4, characterized in that, In step S3, the physical parameters of the fluid are optimized using a gradient descent loss function; the gradient descent loss function is: in, This represents the loss function of gradient descent. Represents the velocity field of the simulated volume; This represents the summation of all mesh elements used in the differentiable physics simulation; Represents a physically plausible three-dimensional fluid volume velocity field; Represents the L2 norm; and All are weighting coefficients; The optimized physical parameters are coupled with the temporal three-dimensional geometric model, including updating the affine transformation matrix of the corresponding Gaussian particle using the affine deformation matrix of the particle in the matter point method simulation, to achieve dynamic driving; the update expression is: in, Represents the affine transformation matrix of a Gaussian particle. For the first The affine transformation matrix of a Gaussian particle at time step. For the first The affine deformation matrix corresponding to the material point at each step, i.e., the affine deformation matrix of the particle in the material point method simulation. For time step, It is an identity matrix.

6. A fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors, characterized in that, include: Module M1: Based on a multi-frame image sequence of the input video, it reconstructs a temporal three-dimensional geometric model of the fluid using three-dimensional Gaussian representation and estimates the velocity field on the fluid surface; Module M2: Based on the velocity field of the fluid surface, an initial three-dimensional fluid volume velocity field is constructed, and a physically reasonable three-dimensional fluid volume velocity field is obtained after constraining the physical boundaries; Module M3: Using the physically reasonable three-dimensional fluid volume velocity field as the supervision target, optimize the physical parameters of the fluid, and then couple the optimized physical parameters with the temporal three-dimensional geometric model to generate dynamic fluid assets that can be physically simulated.

7. The fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors according to claim 6, characterized in that, In module M1, consecutive [processes] are merged. The three-dimensional Gaussian geometric representation generated from the frame image sequence yields a temporal three-dimensional geometric model; The The frame rate is set based on the degree of dynamic change in image content between adjacent video frames, and the expression is: in, Indicates the number of frames. This is the proportionality coefficient. For frame index, Indicates mean square error; This represents the peak signal-to-noise ratio.

8. The fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors according to claim 6, characterized in that, In module M1, estimating the velocity field of the fluid surface includes: calculating the displacement vector of each pixel in the two-dimensional screen space between adjacent video frames using an optical flow algorithm, and then combining this with the depth information provided by the Gaussian point cloud to infer the displacement and three-dimensional velocity estimate in three-dimensional space; the depth information is... , Represents pixels Depth in the camera coordinate system; The displacement vector in the two-dimensional screen space is expressed as: in, This represents a displacement vector in two-dimensional screen space. This represents the horizontal displacement component of the optical flow, i.e., the x-axis. This represents the displacement component of the optical flow in the vertical direction, i.e., the y-axis. The displacement in the three-dimensional space is expressed as: in, Represents displacement in three-dimensional space. This represents the camera intrinsic parameter matrix used for video acquisition. These are pixel coordinates; The three-dimensional velocity estimation is expressed as follows: in, This represents a three-dimensional velocity estimate. This is the inter-frame time interval.

9. The fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors according to claim 6, characterized in that, In module M2, the constrained physical boundary includes applying a boundary layer velocity decay model to fluid particles at a rigid boundary, the expression of which is: in, For the velocity of the target point, For the magnitude of the surface velocity, The distance to the boundary. Boundary layer thickness, This is the proportionality coefficient; The physically reasonable three-dimensional fluid volume velocity field is obtained by solving the divergence-free condition, that is, by estimating the velocity field of the fluid surface through the divergence-free projection of the Poisson pressure equation, and thus obtaining the physically reasonable initial volume velocity field. The expression for the Poisson pressure equation is as follows: in, Represents the gradient operator; The symbol · represents the density of the fluid; the symbol · represents the product.

10. The fluid material reconstruction system based on three-dimensional Gaussian sputtering and physical priors according to claim 9, characterized in that, In module M3, the physical parameters of the fluid are optimized using a gradient descent loss function; the gradient descent loss function is: in, This represents the loss function of gradient descent. Represents the velocity field of the simulated volume; This represents the summation of all mesh elements used in the differentiable physics simulation; Represents a physically plausible three-dimensional fluid volume velocity field; Represents the L2 norm; and All are weighting coefficients; The optimized physical parameters are coupled with the temporal three-dimensional geometric model, including updating the affine transformation matrix of the corresponding Gaussian particle using the affine deformation matrix of the particle in the matter point method simulation, to achieve dynamic driving; the update expression is: in, Represents the affine transformation matrix of a Gaussian particle. For the first The affine transformation matrix of a Gaussian particle at time step. For the first The affine deformation matrix corresponding to the material point at each step, i.e., the affine deformation matrix of the particle in the material point method simulation. For time step, It is an identity matrix.

Images