A three-dimensional scene simulation method and system based on 3D Gaussian sputtering
By improving 3D Gaussian sputtering technology, and using image annotation, implicit integration, and finite element methods to simulate deformation and motion, the problem of insufficient accuracy in 3D scene simulation in existing technologies has been solved, achieving high-precision and timely simulation results.
Patent Information
- Application Number
- CN202510015140.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2045-01-06
AI Technical Summary
Existing scene simulation technologies based on 3D Gaussian sputtering have shortcomings in terms of accuracy and collision detection and response, making it difficult to achieve high-precision 3D scene simulation.
By acquiring and annotating real-world images of the simulation target from different perspectives, the initial point cloud model is reconstructed and expanded into a triangular mesh model. Physical properties are configured, and deformation and motion are simulated using implicit integration and finite element methods. Collision detection is performed by combining penalty force or IPC, and a 3D Gaussian sputtering model is simultaneously deformed to render the simulation results.
It significantly improves the accuracy and timeliness of 3D scene simulation, effectively handles the collision detection and response problem between multiple reconstructed targets, and provides realistic simulation results.
Smart Images
Figure CN119903705B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of physical simulation, and particularly relates to a three-dimensional scene simulation method and system based on 3D Gaussian spattering. BACKGROUND
[0002] The emergence of 3D Gaussian spattering technology successfully revolutionizes the traditional three-dimensional reconstruction method. The 3D Gaussian spattering not only significantly improves the calculation speed of reconstructing a scene from multiple-view pictures, but also provides a novel geometric expression of a three-dimensional scene.
[0003] However, the current scene simulation based on 3D Gaussian spattering only focuses on the change of a scene over time, ignoring the physical laws behind the change. Some focus on the motion and deformation based on physics, but the simulation results are difficult to achieve high authenticity, and also cannot well handle the collision detection and response problems between multiple reconstruction targets, that is, the simulation accuracy of the current scene simulation based on 3D Gaussian spattering is poor.
[0004] Therefore, how to provide a three-dimensional scene simulation method and system based on 3D Gaussian spattering to improve the simulation accuracy of a three-dimensional scene has become a technical problem to be solved. SUMMARY
[0005] The technical problem to be solved by the present application is to provide a three-dimensional scene simulation method and system based on 3D Gaussian spattering to improve the simulation accuracy of a three-dimensional scene.
[0006] In a first aspect, the present application provides a three-dimensional scene simulation method based on 3D Gaussian spattering, comprising the following steps:
[0007] Step S1, obtaining real object images of a simulation target at different viewing angles, labeling the simulation target in each real object image to obtain a mask label, and reconstructing an initial point cloud model based on each real object image;
[0008] Step S2, reconstructing a triangular mesh model of the simulation target based on the initial point cloud model and the mask label, and extending each triangular mesh model to obtain a tetrahedral mesh model;
[0009] Step S3, creating a corresponding 3D Gaussian spattering model based on the triangular mesh model through a 3D Gaussian kernel function;
[0010] Step S4, configuring the physical properties of the simulation target, simulating the deformation and motion effect of the simulation target based on the physical properties through implicit integration and finite element method, and collecting the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model during the deformation process.
[0011] Step S5, synchronously deforming the 3D Gaussian sputtering model based on the deformation gradient;
[0012] Step S6, mapping the deformed 3D Gaussian sputtering model to the tetrahedral mesh model based on the vertex coordinates, and rendering to obtain a simulation result.
[0013] Further, the step S1 is specifically:
[0014] Real object images of the simulation target at different viewing angles are obtained through a camera, and each real object image is stored in a preset image storage path in real time. The simulation target in each real object image is labeled to obtain a mask label, and each mask label is stored in a preset label storage path in real time.
[0015] Image features are extracted from each real object image through COLMAP, and each image feature is input into Structure-from-Motion, and then an initial point cloud model is reconstructed, and the initial point cloud model is stored in a preset model storage path in real time.
[0016] Further, the step S4 is specifically:
[0017] The physical properties of the simulation target are configured based on a JSON file, and the physical properties at least include material type, material parameters, and force conditions;
[0018] Through implicit integration and finite element method, the deformation and motion effect of the simulation target are simulated based on the physical properties. At each time step in the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving an energy equation.
[0019] Further, the step S5 is specifically:
[0020] The deformation gradient is synchronously transmitted to a corresponding 3D Gaussian kernel function of the triangular mesh model, and the covariance matrix and center point of the 3D Gaussian kernel function are modified, so as to synchronously deform the 3D Gaussian sputtering model based on the deformation gradient.
[0021] Further, the step S6 is specifically:
[0022] The deformed 3D Gaussian sputtering model is mapped to the tetrahedral mesh model based on the vertex coordinates, and a simulation result is rendered. Collision detection and response are performed between different simulation results through the triangular mesh model and in combination with a penalty force or IPC, so as to interact between different simulation results.
[0023] In a second aspect, the present application provides a three-dimensional scene simulation system based on 3D Gaussian sputtering, comprising the following modules:
[0024] An initial point cloud model creation module is configured to obtain real object images of the simulation target from different perspectives, label the simulation target in each of the real object images to obtain mask labels, and reconstruct an initial point cloud model based on each of the real object images.
[0025] A tetrahedral mesh model creation module is configured to reconstruct a triangular mesh model of the simulation target based on the initial point cloud model and the mask labels, and extend each of the triangular mesh models to obtain a tetrahedral mesh model.
[0026] A 3D Gaussian sputtering model creation module is configured to create a corresponding 3D Gaussian sputtering model based on the triangular mesh model through a 3D Gaussian kernel function.
[0027] A tetrahedral mesh model deformation module is configured to configure physical properties of the simulation target, control the deformation and movement of the tetrahedral mesh model to simulate the simulation target through implicit integration and finite element methods based on the physical properties, and collect the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model during the deformation process.
[0028] A 3D Gaussian sputtering model deformation module is configured to deform the 3D Gaussian sputtering model based on the deformation gradient.
[0029] A simulation rendering module is configured to map the deformed 3D Gaussian sputtering model to the tetrahedral mesh model based on the vertex coordinates, and render a simulation result.
[0030] Further, the initial point cloud model creation module is specifically configured to:
[0031] The camera is used to obtain real object images of the simulation target from different perspectives, and each of the real object images is stored in a preset image storage path in real time. The simulation target in each of the real object images is labeled to obtain mask labels, and each of the mask labels is stored in a preset label storage path in real time.
[0032] COLMAP is used to extract image features from each of the real object images, and each of the image features is input into Structure-from-Motion to reconstruct an initial point cloud model, and the initial point cloud model is stored in a preset model storage path in real time.
[0033] Further, the tetrahedral mesh model deformation module is specifically configured to:
[0034] The physical properties of the simulation target are configured based on a JSON file, and the physical properties at least include a material type, material parameters, and a force condition;
[0035] By implicit integration and finite element, the deformation and motion effect of the tetrahedral mesh model simulating the simulation target are controlled based on the physical properties.
[0036] Further, the 3D Gaussian sputtering model deformation module is specifically used for:
[0037] The deformation gradient is synchronously transmitted to the corresponding 3D Gaussian kernel function of the triangular mesh model, and the covariance matrix and the center point of the 3D Gaussian kernel function are modified, so as to synchronously deform the 3D Gaussian sputtering model based on the deformation gradient.
[0038] Further, the simulation rendering module is specifically used for:
[0039] The deformed 3D Gaussian sputtering model is mapped to the tetrahedral mesh model based on the vertex coordinates, and a simulation result is rendered.
[0040] The advantage of the present application is that:
[0041] 1、Through acquiring real object images of different perspectives of a simulation target, mask labels are obtained by labeling the simulation target in each real object image, and an initial point cloud model is obtained based on each real object image; then, a triangular mesh model of the simulation target is reconstructed based on the initial point cloud model and the mask labels, each triangular mesh model is expanded to obtain a tetrahedral mesh model; then, a corresponding 3D Gaussian sputtering model is created based on the triangular mesh model through a 3D Gaussian kernel function; then, the physical properties of the simulation target are configured, and the deformation and motion effects of the simulation target are simulated based on the physical properties by means of implicit integration and finite elements, and the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are collected during the deformation process; then, the 3D Gaussian sputtering model is synchronously deformed based on the deformation gradient, and finally, the deformed 3D Gaussian sputtering model is mapped onto the tetrahedral mesh model based on the vertex coordinates, and a simulation result is rendered; collision detection and response between different simulation results are performed through the triangular mesh model in combination with a penalty force or IPC, so as to interact between different simulation results; that is, by labeling the simulation target, the reconstruction of the simulation target is focused on, and the interference of redundant background information is ignored; by configuring the physical properties, the deformation and motion effects of the simulation target are simulated based on the implicit integration and finite elements, and the physical laws behind the changes of the simulation target are fully considered; collision detection and response between multiple reconstruction targets (simulation results) can be well handled through the penalty force or IPC, and the three-dimensional scene simulation accuracy is greatly improved.
[0042] 2、At each time step during the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving the energy equation, and the deformation gradient is synchronously transmitted to the corresponding 3D Gaussian kernel function of the triangular mesh model to synchronously deform the 3D Gaussian sputtering model, thereby greatly improving the timeliness of the simulation.
[0043] 3、The tetrahedral mesh model is obtained by expanding the triangular mesh model, and the tetrahedral mesh model is used as the main model for simulation, which can be applied to the simulation method of implicit integration and finite elements, so as to as accurately as possible solve the physical motion and deformation equations under the condition of supporting large time steps and ensuring system stability, thereby obtaining a realistic simulation result. BRIEF DESCRIPTION OF DRAWINGS
[0044] The application will be further described below with reference to the embodiments and the accompanying drawings.
[0045] Figure 1 is a flowchart of a three-dimensional scene simulation method based on 3D Gaussian sputtering.
[0046] Figure 2It is a structure schematic diagram of a three-dimensional scene simulation system based on 3D Gaussian sputtering of the application.
[0047] Figure 3 It is a schematic diagram of a multi-view physical image of the application.
[0048] Figure 4 It is a schematic diagram of labeling a simulation target of the application.
[0049] Figure 5 It is a schematic diagram of a triangular mesh model of the application.
[0050] Figure 6 It is a schematic diagram of a tetrahedral mesh model of the application.
[0051] Figure 7 It is a schematic diagram of a 3D Gaussian sputtering model of the application.
[0052] Figure 8 It is a schematic diagram of a simulation result of deforming a simulation target of the application. DETAILED DESCRIPTION
[0053] The technical scheme in the embodiments of the application has the following general idea: by labeling a simulation target, the reconstruction of the simulation target is focused on, and the interference of background redundant information is ignored; by configuring physical properties, combining implicit integration and finite element methods, the deformation and motion effects of a tetrahedral mesh model simulating a simulation target are controlled, and the physical laws behind the changes of the simulation target are fully considered; by using a penalty force or an IPC method for collision detection and response, the collision detection and response problems among multiple reconstructed targets can be well handled, and the three-dimensional scene simulation accuracy is improved.
[0054] Please refer to Figures 1 to 8 The preferred embodiment of the three-dimensional scene simulation method based on 3D Gaussian sputtering of the application comprises the following steps:
[0055] Step S1: Obtain physical images of a simulation target from different perspectives, label the simulation target in each physical image to obtain a mask label, and reconstruct an initial point cloud model based on each physical image, i.e., learn the details of a real scene;
[0056] Step S2: Reconstruct a triangular mesh model of the simulation target based on the initial point cloud model and the mask label, and extend each triangular mesh model to obtain a tetrahedral mesh model;
[0057] Step S3: Create a corresponding 3D Gaussian sputtering model based on the triangular mesh model through a 3D Gaussian kernel function;
[0058] The geometric models required by the physical simulation module include a tetrahedral mesh model and a 3D Gaussian sputtering model; the tetrahedral mesh model can be extracted from a triangular mesh model, and the 3D Gaussian sputtering model needs to be matched with a triangular surface proxy of the triangular mesh model, so the triangular mesh model is first reconstructed;
[0059] In step S4, the physical properties of the simulation target are configured, the deformation and motion effect of the simulation target are simulated by the tetrahedral mesh model based on the physical properties through implicit integration and finite element method, and the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are collected during the deformation process;
[0060] In step S5, the 3D Gaussian sputtering model is synchronously deformed based on the deformation gradient.
[0061] In step S6, the deformed 3D Gaussian sputtering model is mapped to the tetrahedral mesh model based on the vertex coordinates, and a simulation result is rendered.
[0062] The step S1 specifically includes:
[0063] Real object images of the simulation target at different perspectives are acquired by a camera, for example, 100-300 real object images in RGB format are acquired, scene details at all angles are captured as much as possible during shooting, and each real object image is stored in real time to a preset image storage path; mask labels are obtained by labeling the simulation target in each real object image, and each mask label is stored in real time to a preset label storage path.
[0064] Image features are extracted from each real object image by COLMAP, each image feature is input into Structure-from-Motion (SfM), and then an initial point cloud model in ply format is reconstructed, and the initial point cloud model is stored in real time to a preset model storage path.
[0065] The step S4 specifically includes:
[0066] The physical properties of the simulation target are configured based on a JSON file, and the physical properties at least include a material type, material parameters and a force condition; the material parameters at least include a Young's modulus for simulating the shape of different substances; and the force condition at least includes a gravitational acceleration for driving the motion and deformation of the simulation target.
[0067] The deformation and motion effect of the simulation target are simulated by the tetrahedral mesh model based on the physical properties through implicit integration and finite element method, and the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving an energy equation at each time step during the deformation process.
[0068] That is, the physical grid used for solving the simulation equation and the preset physical parameters, output the position sequence information (deformation gradient and vertex coordinates) of the physical grid at each time step.
[0069] The step S5 is specifically:
[0070] Synchronously transfer the deformation gradient to the corresponding 3D Gaussian kernel function of the triangular mesh model, modify the covariance matrix and the center point of the 3D Gaussian kernel function, and deform the 3D Gaussian sputtering model based on the deformation gradient.
[0071] That is, the deformation gradient is transferred based on the correspondence between the mesh elements of the tetrahedral mesh model and the triangular mesh model.
[0072] The step S6 is specifically:
[0073] Map the deformed 3D Gaussian sputtering model to the tetrahedral mesh model based on the vertex coordinates, and render the simulation results; the different simulation results are detected and responded through the triangular mesh model, combined with the penalty force or IPC, to interact between different simulation results. That is, the 3D Gaussian sputtering model is bound to the surface of the triangular mesh model, which is used to cooperate with the vertex change of the tetrahedral mesh model, and realizes the rendering of the restored real scene texture.
[0074] A preferred embodiment of a three-dimensional scene simulation system based on 3D Gaussian sputtering according to the application comprises the following modules:
[0075] An initial point cloud model creation module is used to obtain real object images of different perspectives of a simulation target, label the simulation target in each real object image to obtain a mask label, and reconstruct an initial point cloud model based on each real object image, that is, learn the details of the real scene.
[0076] A tetrahedral mesh model creation module is used to reconstruct a triangular mesh model of a simulation target based on the initial point cloud model and the mask label, and extend each triangular mesh model to obtain a tetrahedral mesh model.
[0077] A 3D Gaussian sputtering model creation module is used to create a corresponding 3D Gaussian sputtering model based on the triangular mesh model through a 3D Gaussian kernel function.
[0078] The geometric models required by the physical simulation module include a tetrahedral mesh model and a 3D Gaussian sputtering model; wherein the tetrahedral mesh model can be extracted from the triangular mesh model, and the 3D Gaussian sputtering model needs to be matched with the triangular surface proxy of the triangular mesh model, so the triangular mesh model is first reconstructed.
[0079] The tetrahedral mesh model deformation module is configured to configure physical properties of the simulation target, control deformation and motion effects of the tetrahedral mesh model simulating the simulation target based on the physical properties through implicit integration and finite element method, and collect deformation gradients and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model during the deformation process.
[0080] The 3D Gaussian sputtering model deformation module is configured to deform the 3D Gaussian sputtering model based on the deformation gradients.
[0081] The simulation rendering module is configured to map the deformed 3D Gaussian sputtering model to the tetrahedral mesh model based on the vertex coordinates, and render a simulation result.
[0082] The initial point cloud model creation module is configured to:
[0083] The real object images of the simulation target at different angles are obtained by a camera, for example, 100-300 real object images in RGB format are obtained, scene details at all angles are captured as much as possible during shooting, and each real object image is stored in a preset image storage path in real time; the simulation target in each real object image is labeled to obtain a mask label, and each mask label is stored in a preset label storage path in real time.
[0084] Image features are extracted from each real object image by COLMAP, each image feature is input into Structure-from-Motion (SfM), and then an initial point cloud model in ply format is reconstructed, and the initial point cloud model is stored in a preset model storage path in real time.
[0085] The tetrahedral mesh model deformation module is configured to:
[0086] The physical properties of the simulation target are configured based on a JSON file, the physical properties at least include a material type, material parameters and a force condition; the material parameters at least include a Young's modulus for simulating the shape of different substances; and the force condition at least includes a gravitational acceleration for driving the motion and deformation of the simulation target.
[0087] The deformation and motion effects of the tetrahedral mesh model simulating the simulation target are controlled based on the physical properties through implicit integration and finite element method, and at each time step during the deformation process, the deformation gradients and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving an energy equation.
[0088] That is, the physical grid and the preset physical parameters used for solving the simulation equation are output to obtain the position sequence information (deformation gradient and vertex coordinates) of the physical grid at each time step.
[0089] The 3D Gaussian sputtering model deformation module is specifically used for:
[0090] The deformation gradient is synchronously transmitted to the corresponding 3D Gaussian kernel function of the triangular mesh model, and the covariance matrix and the center point of the 3D Gaussian kernel function are modified, so that the 3D Gaussian sputtering model is synchronously deformed based on the deformation gradient.
[0091] That is, the deformation gradient is transmitted based on the corresponding relationship between the mesh surface elements of the tetrahedral mesh model and the triangular mesh model.
[0092] The simulation rendering module is specifically used for:
[0093] The deformed 3D Gaussian sputtering model is mapped onto the tetrahedral mesh model based on the vertex coordinates, and a simulation result is rendered; different simulation results are subjected to collision detection and response through the triangular mesh model in a manner of a penalty force or IPC, so as to realize interaction between different simulation results. That is, the 3D Gaussian sputtering model is bound to the surface of the triangular mesh model, used for cooperating with the vertex change of the tetrahedral mesh model, and used for realizing rendering of a restored real scene texture.
[0094] In summary, the application has the following advantages:
[0095] 1. By obtaining physical images of the simulation target from different perspectives, the simulation target in each physical image is labeled to obtain a mask label, and an initial point cloud model is reconstructed based on each physical image; then, based on the initial point cloud model and the mask label, a triangular mesh model of the simulation target is reconstructed, and each triangular mesh model is extended to obtain a tetrahedral mesh model; then, based on the triangular mesh model, a corresponding 3D Gaussian sputtering model is created through a 3D Gaussian kernel function; then, the physical properties of the simulation target are configured, and the deformation and motion effects of the simulation target are simulated based on the physical properties by means of implicit integration and finite elements, and the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are collected during the deformation process; then, the 3D Gaussian sputtering model is deformed based on the deformation gradient, and finally the deformed 3D Gaussian sputtering model is mapped onto the tetrahedral mesh model based on the vertex coordinates, and the simulation result is rendered; collision detection and response between different simulation results are performed through the triangular mesh model combined with a penalty force or IPC to interact between different simulation results; that is, by labeling the simulation target, the reconstruction of the simulation target is focused on, and the interference of redundant background information is ignored; by configuring the physical properties, the deformation and motion effects of the simulation target are simulated based on the implicit integration and finite element method, and the physical laws behind the changes of the simulation target are fully considered; collision detection and response between multiple reconstruction targets (simulation results) can be well handled through the penalty force or IPC, which greatly improves the simulation accuracy of the three-dimensional scene.
[0096] 2. At each time step during the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving the energy equation, and the deformation gradient is synchronously transmitted to the corresponding 3D Gaussian kernel function of the triangular mesh model to deform the 3D Gaussian sputtering model, thereby greatly improving the timeliness of the simulation.
[0097] 3. By extending the triangular mesh model to obtain a tetrahedral mesh model, the tetrahedral mesh model can be used as the main model for simulation, which can be applied to the simulation method of implicit integration and finite elements to solve the physical motion and deformation equations as accurately as possible while supporting large time steps and ensuring system stability, thereby obtaining realistic simulation results.
[0098] Although the specific embodiments of the present application are described above, those skilled in the art should understand that the specific embodiments described are only illustrative, and are not intended to limit the scope of the present application, and equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present application should be covered by the scope of the claims of the present application.
Claims
1. A three-dimensional scene simulation method based on 3D Gaussian sputtering, characterized in that: Includes the following steps: Step S1: Obtain real-object images of the simulation target from different perspectives, label the simulation target in each real-object image to obtain mask labels, and reconstruct the initial point cloud model based on each real-object image; Step S2: Based on the initial point cloud model and mask labels, reconstruct the triangular mesh model of the simulation target, and expand each of the triangular mesh models to obtain a tetrahedral mesh model; specifically: The camera acquires real-object images of the simulated target from different perspectives, and stores each real-object image in real time to a preset image storage path; the simulated target in each real-object image is labeled to obtain mask tags, and each mask tag is stored in real time to a preset tag storage path; Image features are extracted from each of the physical object images using COLMAP, and the image features are input into Structure-from-Motion to reconstruct an initial point cloud model. The initial point cloud model is then stored in real time to a preset model storage path. Step S3: Create a corresponding 3D Gaussian sputtering model based on the triangular mesh model using the 3D Gaussian kernel function; Step S4: Configure the physical properties of the simulation target. Using implicit integration and finite element method, control the tetrahedral mesh model based on the physical properties to simulate the deformation and motion effect of the simulation target. During the deformation process, collect the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model. Step S5: Based on the deformation gradient synchronous deformation 3D Gaussian sputtering model; Step S6: Map the deformed 3D Gaussian sputtering model onto the tetrahedral mesh model based on vertex coordinates, and render the simulation results.
2. The three-dimensional scene simulation method based on 3D Gaussian sputtering as described in claim 1, characterized in that: Step S4 specifically involves: The physical properties of the simulation target are configured based on a JSON file, and the physical properties include at least the material type, material parameters, and stress conditions. By using implicit integration and finite element method, the deformation and motion of the target are simulated based on the physical properties controlled by the tetrahedral mesh model. At each time step in the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving the energy equation.
3. The three-dimensional scene simulation method based on 3D Gaussian sputtering as described in claim 1, characterized in that: Step S5 specifically involves: The deformation gradient is synchronously passed to the 3D Gaussian kernel function corresponding to the triangular mesh model, and the covariance matrix and center point of the 3D Gaussian kernel function are modified to synchronously deform the 3D Gaussian sputtering model based on the deformation gradient.
4. The three-dimensional scene simulation method based on 3D Gaussian sputtering as described in claim 1, characterized in that: Step S6 specifically involves: The deformed 3D Gaussian sputtering model is mapped onto a tetrahedral mesh model based on vertex coordinates, and the simulation results are rendered. Different simulation results are used to perform collision detection and response through a triangular mesh model, combined with penalty force or IPC, so as to enable interaction between different simulation results.
5. A three-dimensional scene simulation system based on 3D Gaussian sputtering, characterized in that: Includes the following modules: The initial point cloud model creation module is used to acquire real-world images of the simulation target from different perspectives, annotate the simulation target in each of the real-world images to obtain mask labels, and reconstruct the initial point cloud model based on each of the real-world images. The tetrahedral mesh model creation module is used to reconstruct the triangular mesh model of the simulation target based on the initial point cloud model and mask labels, and to expand each of the triangular mesh models to obtain a tetrahedral mesh model. The 3D Gaussian sputtering model creation module is used to create a corresponding 3D Gaussian sputtering model based on the triangular mesh model using a 3D Gaussian kernel function. The tetrahedral mesh model deformation module is used to configure the physical properties of the simulation target. Based on the physical properties, it controls the tetrahedral mesh model to simulate the deformation and motion effects of the simulation target through implicit integration and finite element method. During the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are collected. A 3D Gaussian sputtering model deformation module is used to synchronously deform the 3D Gaussian sputtering model based on the deformation gradient. The simulation rendering module is used to map the deformed 3D Gaussian sputtering model onto a tetrahedral mesh model based on vertex coordinates, and render the simulation results. The initial point cloud model creation module is specifically used for: The camera acquires real-object images of the simulated target from different perspectives, and stores each real-object image in real time to a preset image storage path; the simulated target in each real-object image is labeled to obtain mask tags, and each mask tag is stored in real time to a preset tag storage path; Image features are extracted from each of the physical object images using COLMAP, and the image features are input into Structure-from-Motion to reconstruct an initial point cloud model. The initial point cloud model is then stored in real time to a preset model storage path.
6. The three-dimensional scene simulation system based on 3D Gaussian sputtering as described in claim 5, characterized in that: The tetrahedral mesh model deformation module is specifically used for: The physical properties of the simulation target are configured based on a JSON file, and the physical properties include at least the material type, material parameters, and stress conditions. By using implicit integration and finite element method, the deformation and motion of the target are simulated based on the physical properties controlled by the tetrahedral mesh model. At each time step in the deformation process, the deformation gradient and vertex coordinates of each tetrahedral mesh in the tetrahedral mesh model are obtained by solving the energy equation.
7. The three-dimensional scene simulation system based on 3D Gaussian sputtering as described in claim 5, characterized in that: The 3D Gaussian sputtering model deformation module is specifically used for: The deformation gradient is synchronously passed to the 3D Gaussian kernel function corresponding to the triangular mesh model, and the covariance matrix and center point of the 3D Gaussian kernel function are modified to synchronously deform the 3D Gaussian sputtering model based on the deformation gradient.
8. The three-dimensional scene simulation system based on 3D Gaussian sputtering as described in claim 5, characterized in that: The simulation rendering module is specifically used for: The deformed 3D Gaussian sputtering model is mapped onto a tetrahedral mesh model based on vertex coordinates, and the simulation results are rendered. Different simulation results are used to perform collision detection and response through a triangular mesh model, combined with penalty force or IPC, so as to enable interaction between different simulation results.
Citation Information
Patent Citations
High-fidelity simulation environment establishment system and method based on multi-view video supervision
CN118596137A
High-fidelity simulation model construction method based on monocular video
CN118886250A