Detail level rendering method based on Gaussian splashing

Abstract

The invention discloses a detail level rendering method based on Gaussian splash, which can dynamically change the detail level of a model according to the actual needs of a user in an interaction process, realize high-fidelity and low-resource-loss rendering based on Gaussian splash, and reduce the loss required by rendering. The technical scheme of the invention comprises a preparation stage and a real-time rendering stage. In the preparation stage, a multi-view image is acquired, a corresponding three-dimensional model or four-dimensional model is constructed according to the multi-view image, and Gaussian splash models with different detail levels are generated. And in the real-time rendering stage, the Gaussian splash models of different detail levels are selected for real-time rendering according to the real-time data of the user by using the Gaussian splash models of different detail levels.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and human-computer interaction, and in particular to a detail level rendering method based on Gaussian splashing. Background Art

[0002] Gaussian splatting is a 3D volume rendering technique that can directly render 3D volume data without converting the 3D data into line segments and planes. With the advancement of computer graphics, 3D and 4D Gaussian splatting techniques have been developed and used to provide real-time radiation field rendering and dynamic scene rendering. 3D Gaussian splatting (3DGS) can display models in real time while improving rendering quality. This method integrates the sparse points generated in camera calibration and represents the scene with a 3D Gaussian function, retaining the characteristics of a continuous volume radiation field. In addition, alternating optimization and density control of the 3D Gaussian function are introduced to support anisotropic Gaussian splatting. 4D Gaussian splatting adds time to 3D to achieve dynamic scene rendering.

[0003] The patent with publication number CN117671108A provides a dynamic human body modeling method based on three-dimensional Gaussian. The invention discloses a dynamic human body modeling method based on three-dimensional Gaussian. The method first establishes a three-dimensional Gaussian expression of the human body by inputting a multi-view human body video. The method attaches a hidden code and a set of mixing weights to each Gaussian. Subsequently, a multi-layer perceptron is used to make corrections according to a given target human body posture. The corrected Gaussian splash is then Gaussian transformed through the mixing weights and transformed to the target posture in a linear mixed skinning manner. Finally, realistic human body images at new perspectives or new postures can be rendered in real time by Gaussian splashing.

[0004] The invention proposes a method for realizing dynamic Gaussian splashing of human body, which can realize dynamic human body rendering process under Gaussian splashing rendering pipeline. However, since the method is not based on the model in the initialization process of Gaussian splashing, the number and position of Gaussian splashing are uncontrollable, and thus Gaussian splashing rendering with multi-level details cannot be realized. In the invention, since there is no change in the level of detail, it is impossible to realize adjustable level of detail in rendering and interaction.

[0005] Paper: GaussianAvatars: Photorealistic Head Avatars with Rigged3DGaussians.

[0006] This paper proposes a dynamic 3D representation method for human head based on Gaussian splash, which fixes Gaussian splash on a deformable 3D face model. First, a 3D Gaussian splash is initialized at the center of each triangle of the model. Each Gaussian splash is translated, rotated and scaled according to the triangle face on which it is located. For each frame of the collected 4D data, the Gaussian splash is converted from the face space on the local triangle to the world space, and then rendered using the 3D Gaussian splash. During network training, the Gaussian splash in the local space is optimized by minimizing the color loss during rendering. At the same time, new Gaussian splashes are adaptively added and removed according to the binding relationship between the existing Gaussian splashes and the triangles, and the position and scale of the 3D Gaussian splashes are regularized. Then, the radiation field is formed on the surface of the model by training the 3D Gaussian splash. This method creates a movable head avatar by binding the 3D Gaussian splash to a parameterized mesh model while ensuring the high rendering quality of Gaussian rendering. This method provides a solution for 4D dynamic Gaussian splash rendering, which focuses on how to make the 3D Gaussian splash move, but does not consider the loss of rendering. The positions of the generated Gaussian splashes in the results of this method are obtained during the training process, and the number is not controllable, which makes it impossible to adjust the level of detail of the trained 3D Gaussian.

[0007] It can be seen that the existing Gaussian splash rendering methods are unable to adjust the level of detail. In actual virtual reality or augmented reality applications, this rendering method has a large loss. There is currently no technical solution to this problem. Summary of the invention

[0008] In view of this, the present invention provides a detail level rendering method based on Gaussian splashing, which can dynamically change the detail level of the model according to the needs of the user in the actual interaction process, thereby achieving high-realism, low-resource loss-based Gaussian splashing rendering and reducing the loss required for rendering.

[0009] To achieve the above-mentioned purpose, the technical solution of the present invention includes a preparation stage and a real-time rendering stage.

[0010] In the preparation stage, multi-view images are acquired, and corresponding three-dimensional models or four-dimensional models are constructed based on the multi-view images to generate Gaussian splash models with different levels of detail.

[0011] In the real-time rendering stage, Gaussian splash models with different levels of detail are used and selected according to the user's real-time data for real-time rendering.

[0012] Furthermore, in the preparation stage, multi-view images are images of the same object taken at multiple different viewpoints, which are used to represent objects that require the use of Gaussian splatter radiation fields. A three-dimensional model or a four-dimensional model is composed of a series of vertices and connecting lines, and the model is divided into a series of model faces for representation.

[0013] Furthermore, the multi-view image is a single frame or a sequence of frames; if it is a single-frame multi-view image, it is used to construct a Gaussian splash detail level three-dimensional model; if it is a sequence of frame multi-view images, it is used to construct a Gaussian splash detail level four-dimensional model.

[0014] Furthermore, in the preparation stage, a Gaussian splash model with different levels of detail is generated, which is specifically generated by the following steps:

[0015] Step 101: Initialize a Gaussian splash at each vertex and the center of gravity of each model surface of a three-dimensional model or a four-dimensional model, and each Gaussian splash has five basic parameters, including position, rotation, scale, opacity, and a set of spherical harmonic functions representing color information; for the Gaussian splash at the center of gravity of each model surface, its spatial position, spatial rotation, and spatial scale are obtained from the Gaussian splash information of the vertex position in the same plane corresponding to the center of gravity position, thereby obtaining the position of the initialized Gaussian splash.

[0016] Step 102: The initialized Gaussian splash position is used for training the Gaussian splash network, and the parameters of the Gaussian points are continuously optimized to generate different levels of detail, thereby obtaining a Gaussian splash model with different levels of detail.

[0017] Furthermore, in step 102, the position of the initialized Gaussian splash is used for training the Gaussian splash network; the training process is specifically as follows: optimizing the position, rotation, scaling, opacity attributes and color information of the Gaussian points; during the training process, for the Gaussian splash at the vertices of the model, its spatial position is not changed, only its rotation, scaling, opacity attributes and color information are changed; for the Gaussian splash at the center of gravity, its position, rotation, scaling, opacity and color information are all changed; during the network training process, the number of Gaussian splashes is not changed; after optimizing all Gaussian splashes through the training process, a Gaussian splash model with a detail level is obtained.

[0018] Further, in step 102, different levels of detail are generated to obtain Gaussian splash models with different levels of detail, specifically: the model surface is subdivided; the specific method is: for the three-dimensional model or the four-dimensional model, the Gaussian splash at the center of gravity position of each model surface is moved to a new position after network training; by connecting the new position points and the original vertices, a series of new model surfaces are formed.

[0019] While keeping the position, rotation, scale, opacity and color information of the existing Gaussian splash unchanged, initialize a series of new Gaussian splashes at the center of gravity of the new model surface.

[0020] By repeating the training process of the Gaussian splash network and continuously optimizing the new Gaussian splash, new details can be added to the existing level of detail, thereby obtaining a Gaussian splash model with a new level of detail.

[0021] The Gaussian splash model of the new detail level is evaluated, that is, the Gaussian splash model of the new detail level is reprojected to the camera position. If the image similarity with the original multi-view image reaches the set index range, the Gaussian splash model of the new detail level is saved, otherwise the Gaussian splash model of the new detail level is deleted; the camera position is the position of the camera for obtaining the multi-view image.

[0022] Furthermore, in the real-time rendering stage, the real-time data of the user includes: real-time data of the user's position, the user's head orientation, and the user's hand position.

[0023] Furthermore, the real-time rendering stage selects different levels of detail levels according to the user's real-time data, specifically: using a decision tree model, a neural symbolic decision model or a multi-layer perceptron to map the user's real-time data to a Gaussian splatter model of different detail levels.

[0024] Beneficial effects:

[0025] 1; The present invention proposes a detail level rendering method based on Gaussian splashing, which can dynamically change the detail level of the model according to the needs of the user in the actual interaction process, and realize Gaussian splashing-based rendering with high realism and low resource loss. The present invention introduces detail levels in Gaussian splashing rendering, which can generate three-dimensional or four-dimensional models based on Gaussian splashing with different precisions, and can be applied to a wider range of application scenarios. And by changing the detail level of the model in real-time interaction, the loss required for rendering is reduced.

[0026] 2: After the method of the present invention obtains the Gaussian splash three-dimensional or four-dimensional model with different levels of detail, it selects and replaces the models with different levels of detail according to the real-time data of the user during the real-time interaction process. During the user's use, first obtain real-time data including but not limited to the user's position, the user's head orientation, the user's hand position, etc. Subsequently, select the level of detail to be rendered in the current situation according to the user's real-time data. The selection method includes but is not limited to decision trees, neural symbolic decisions, multi-layer perceptrons, etc. For example, one possible method is to use a decision tree model to map the user's real-time information to models with different levels of detail. When the user is far away from the model, a model with a low level of detail is used for rendering; when the user is close to the model, a higher level of detail is used. This method can reduce the cost required for rendering calculations while ensuring the user's subjective perception of high rendering quality, thereby realizing multi-level detail rendering based on Gaussian splashing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of the detail level rendering method based on Gaussian splatting;

[0028] Figure 2 Schematic diagram of model surface subdivision and Gaussian splash network training. DETAILED DESCRIPTION

[0029] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0030] The present invention aims to introduce detail levels into Gaussian splash rendering to achieve high-quality, low-computational-cost Gaussian splash rendering. Three-dimensional and four-dimensional Gaussian splash techniques can render highly realistic three-dimensional or four-dimensional data and have currently the best rendering performance. However, the number and position of points in the Gaussian splash model are not controllable in the model, which raises two problems: first, in some scenarios, the Gaussian splash is too dense, and sometimes it is not necessary to fully render all the details; second, due to the aforementioned problems, it is impossible to generate a Gaussian splash model with different levels of detail, and it is also impossible to achieve dynamic level of detail replacement.

[0031] Based on this, the present invention provides a detail level rendering method based on Gaussian splashing. The process of the present invention is as follows Figure 1 As shown, the method includes two parts: detail level preparation stage and real-time rendering stage.

[0032] In the preparation stage, a multi-view image is obtained, and a corresponding three-dimensional model or four-dimensional model is constructed based on the multi-view image to generate a Gaussian splash model with different levels of detail. The multi-view image is an image of the same object taken from multiple different viewpoints, which is used to represent the object that needs to use the Gaussian splash radiation field; the multi-view image is a single frame or a sequence of frames; if it is a single-frame multi-view image, it is used to construct a three-dimensional model of the detail level of Gaussian splash; if it is a sequence of frame multi-view images, it is used to construct a four-dimensional model of the detail level of Gaussian splash. The three-dimensional model or four-dimensional model is composed of a series of vertices and connecting lines, and the model is divided into a series of model faces for representation.

[0033] In an embodiment of the present invention, the detail level preparation stage is a pre-offline process for preparing Gaussian splashes of different detail levels required for real-time rendering. In this stage, different Gaussian splashes are first generated according to the three-dimensional model of the object to be rendered, and these Gaussian splashes are bound to the vertices and the center of gravity positions of the faces on the model surface (taking triangular faces as an example). Subsequently, the Gaussian splash network is trained using the collected multi-view images of the sequence frames. During the training process, the splitting and deletion of the Gaussian splashes are restricted, the number of Gaussian splashes is controlled, and a Gaussian splash model with a low level of detail is obtained. Then, the surface of the three-dimensional model is meshed and subdivided. The existing Gaussian splash is maintained, the Gaussian splash is reinitialized at the center of gravity position of each face of the subdivided three-dimensional model, and the optimization process of the Gaussian splash network training is repeated to obtain a Gaussian splash model with a higher level of detail.

[0034] Generate a Gaussian splatter model with different levels of detail using the following steps:

[0035] Step 101: Initialize Gaussian splash at each vertex and the center of gravity of each model surface of the three-dimensional model or the four-dimensional model, and each Gaussian splash has five basic parameters, including position, rotation, scale, opacity and a set of spherical harmonic functions representing color information; for the Gaussian splash at the center of gravity of each model surface, its spatial position, spatial rotation and spatial scale are obtained from the Gaussian splash information of the vertex position in the same plane corresponding to the center of gravity position, thereby obtaining the position of the initialized Gaussian splash;

[0036] Step 102: The initialized Gaussian splash position is used for training the Gaussian splash network, and the parameters of the Gaussian points are continuously optimized to generate different levels of detail, thereby obtaining a Gaussian splash model with different levels of detail.

[0037] The specific training process is as follows: optimizing the position, rotation, scaling, opacity attributes and color information of Gaussian points; during the training process, for the Gaussian splashes at the vertices of the model, their spatial positions are not changed, but their rotation, scaling, opacity attributes and color information are changed; for the Gaussian splashes at the center of gravity, their position, rotation, scaling, opacity and color information are all changed; during the network training process, the number of Gaussian splashes is not changed; after optimizing all Gaussian splashes during the training process, a Gaussian splash model with a detailed level is obtained.

[0038] During training, the accuracy of the Gaussian splatter model is improved by limiting the splitting and deletion of Gaussian splatter and by subdividing the model faces to obtain models with different levels of detail.

[0039] The Gaussian splatter model at the lowest level of detail is obtained after training with the initialized Gaussian splatter positions.

[0040] The model surface is subdivided; specifically, for a three-dimensional model or a four-dimensional model, the Gaussian splash at the center of gravity of each model surface is moved to a new position after network training; a series of new model surfaces are formed by connecting the new position points and the original vertices;

[0041] Initialize a series of new Gaussian splashes at the centroid of the new model surface while keeping the position, rotation, scale, opacity and color information of the existing Gaussian splashes unchanged;

[0042] Repeat the training process of the Gaussian splash network. By continuously optimizing the new Gaussian splash, new details can be added to the existing detail level, thereby obtaining a Gaussian splash model with a new detail level.

[0043] The Gaussian splash model of the new detail level is evaluated, that is, the Gaussian splash model of the new detail level is reprojected to the camera position. If the image similarity with the original multi-view image reaches the set index range, the Gaussian splash model of the new detail level is saved, otherwise the Gaussian splash model of the new detail level is deleted; the camera position is the position of the camera for obtaining the multi-view image.

[0044] In the real-time rendering stage, Gaussian splash models with different levels of detail are used and selected according to the user's real-time data for real-time rendering.

[0045] In the embodiment of the present invention, in the real-time rendering stage, the models of different detail levels that have been acquired are dynamically selected and replaced according to the real-time data of the user, so as to realize the real-time rendering of detail levels based on Gaussian splashing. The core of this research is to introduce detail levels into Gaussian splashing technology. This method can reduce the resource consumption in real-time rendering while maintaining high-quality rendering. This method is applicable to a variety of real-time interactive scenarios based on Gaussian splashing rendering.

[0046] Based on the existing level of detail, during real-time rendering and use, by capturing the user's real-time data, the model in the virtual environment is selected and replaced to achieve real-time rendering of the level of detail based on Gaussian splashing. In this way, the computational loss of rendering is reduced while ensuring the rendering quality required by the user.

[0047] Example 1

[0048] The process of this embodiment is as follows Figure 1 As shown, it includes two parts: a pre-detail level preparation process and a real-time rendering process. In the detail level preparation process, the present invention generates a Gaussian splash model with different detail levels through a three-dimensional or four-dimensional multi-view image and a corresponding three-dimensional model. In the real-time rendering process, the above-mentioned different detail level models are used to select different levels of detail levels according to the user's real-time data to achieve replacement of different detail levels, thereby achieving real-time rendering of detail levels based on Gaussian splashing.

[0049] Data preparation and initialization of Gaussian splatting

[0050] In the detail level preparation stage, it is first necessary to obtain the multi-view images required for Gaussian splashing and the corresponding three-dimensional model. Multi-view images are images of the same object taken at multiple different perspectives, and are used to represent objects that require the use of Gaussian splashing radiation fields. Since the more multi-view images used for Gaussian splashing, the better the viewing effect at different angles, preferably, these images need to capture information about the object at all angles and directions in three-dimensional space. Multi-view images can be single frames or sequence frames. Single-frame multi-view images are used for the detail level model of three-dimensional Gaussian splashing, while sequence frame multi-view images are used for the detail level model of four-dimensional Gaussian splashing.

[0051] A three-dimensional model is composed of a series of vertices and connecting lines. Usually, such a model can be represented by a series of triangular faces or quadrilateral faces. In the method of obtaining a three-dimensional model of an object represented by a Gaussian splatter radiation field, the three-dimensional model can be obtained by hand-made, image-based generation (such as FLAME[1]), or using an existing standard model (such as SMPL[2]).

[0052] Then, the position of Gaussian splash is initialized. Gaussian splash needs to be initialized at each vertex and the center of gravity of each surface of this three-dimensional model, such as Figure 2 As shown on the left. Each Gaussian splash has five basic parameters, including position, rotation, scale, opacity and a set of spherical harmonic functions representing color information. For each Gaussian splash at the center of gravity position of the surface, its spatial position, spatial rotation and spatial scale can be obtained from the Gaussian splash information of the vertex position in the same plane corresponding to the center of gravity position. The position of the initialized Gaussian splash can then be used to train the Gaussian splash network, continuously optimizing the parameters of the Gaussian points and generating different levels of detail.

[0053] Gaussian splatting network training

[0054] After obtaining the initialized Gaussian splash spatial position, the position, rotation, scale, opacity and color information of all Gaussian splashes are optimized according to the collected single-frame or multi-frame multi-view images. This method adopts the method in the document "3D Gaussian Splatting for Real-Time Radiance Field Rendering" [3], projects the existing Gaussian splash to the camera imaging plane position corresponding to each frame of the multi-view image, calculates the loss, and uses the gradient descent method to optimize the Gaussian points. In this scheme, there are two main steps: the first step is to optimize the position, rotation, scale, opacity attribute and color information of the Gaussian points, and the second step is adaptive density control. Different from this scheme, in the present invention, during the training process, in the first step, for the Gaussian splash at the model vertex, its spatial position is not changed, only its rotation, scale, opacity attribute and color information are changed; for the Gaussian splash at the center of gravity, its position, rotation, scale, opacity and color information can all be changed. The present invention cancels the adaptive density control, that is, cancels the steps of Gaussian splash splitting and deletion, so that the number of Gaussian splashes does not change during the network training process. After the network training optimizes all Gaussian splashes, a Gaussian splash model with the lowest detail level can be obtained.

[0055] Model surface subdivision and detail level implementation

[0056] By subdividing the 3D model surface, more surfaces can be generated in the model, so as to reinitialize the position of Gaussian splash on the new surface, increase the number of Gaussian splash, and realize the model division of different levels of detail, such as Figure 2 shown.

[0057] The subdivision method of the model surface can be not limited to loop subdivision. Taking the triangular surface model as an example, the Gaussian splash at the centroid position of each surface will move to a new position after network training. By connecting these points and the positions of the original vertices, a series of new triangular surfaces can be formed. While keeping the position, rotation, scale, opacity and color information of the existing Gaussian splash unchanged, a series of new Gaussian splashes are initialized at the centroid position of the newly generated triangular surface. Then, the network training is repeated, and by continuously optimizing the new Gaussian splash, new details can be added to the existing level of detail, thereby obtaining a higher level of detail.

[0058] When evaluating the detail level of the Gaussian splash model, image similarity metrics can be used, including but not limited to peak signal-to-noise ratio, structural similarity, and learning-perceptual image block similarity. In actual use, the range of image similarity metrics for the detail level can be preset in advance. When the Gaussian splash model in the iteration is reprojected to the camera position and the image similarity with the original input image reaches the index range, the model is saved as a detail level model. In this way, new details are continuously added to the existing low detail level.

[0059] For example, this embodiment may use an image similarity measurement indicator - structural similarity, where the value range of structural similarity is between -1 and 1, and a larger value indicates a better image quality.

[0060] After the first network training step of the Gaussian splash model, the resulting level of detail is called LOD 0, assuming that its structural similarity is 0.2. Subsequently, the structural similarity index range is artificially set, for example, when the structural similarity is 0.4, it is LOD 1, when the structural similarity is 0.6, it is LOD 2, and when the structural similarity is 0.8, it is LOD 3. Iterations are performed continuously, and the structural similarity of the current model is judged during the iteration process. When the structural similarity reaches 0.4, the model is saved as the LOD 1 model, and so on, so as to obtain Gaussian splash models with different details.

[0061] Real-time selection replacement and rendering of level of detail

[0062] After obtaining Gaussian splash three-dimensional or four-dimensional models with different levels of detail, this method selects and replaces models with different levels of detail according to the user's real-time data during real-time interaction. During the user's use, first obtain real-time data including but not limited to the user's position, the user's head orientation, the user's hand position, etc. Subsequently, select the level of detail to be rendered in the current situation according to the user's real-time data. The selection method includes but is not limited to decision trees, neural symbolic decisions, multi-layer perceptrons, etc. For example, one possible method is to use a decision tree model to map the user's real-time information to models with different levels of detail. When the user is far away from the model, a model with a low level of detail is used for rendering; when the user is close to the model, a higher level of detail is used. This method can reduce the cost required for rendering calculations while ensuring the user's subjective perception of high rendering quality, thereby realizing multi-level detail rendering based on Gaussian splash.

[0063] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A detail level rendering method based on Gaussian splashing, characterized in that: Includes preparation stage and real-time rendering stage; In the preparation stage, multi-view images are acquired, and corresponding three-dimensional models or four-dimensional models are constructed according to the multi-view images to generate Gaussian splash models with different levels of detail; In the real-time rendering stage, the Gaussian splash models at different detail levels are used to select Gaussian splash models at different detail levels for real-time rendering according to the real-time data of the user.

2. A detail level rendering method based on Gaussian splatting as claimed in claim 1, characterized in that: In the preparation stage, the multi-view images are images of the same object taken at multiple different view angles, and are used to represent the object that needs to use the Gaussian splash radiation field; The three-dimensional model or four-dimensional model is composed of a series of vertices and connecting lines, and the model is divided into a series of model faces.

3. A detail level rendering method based on Gaussian splashing as claimed in claim 2, characterized in that: The multi-view image is a single frame or a sequence of frames; If it is a single-frame multi-view image, it is used to construct a detail level 3D model of Gaussian splatter; If it is a sequence frame multi-view image, it is used to construct a four-dimensional model of the detail level of Gaussian splashing.

4. A detail level rendering method based on Gaussian splatting as claimed in claim 2, characterized in that: In the preparation stage, the generation of Gaussian splash models with different levels of detail is specifically performed by the following steps: Step 101: Initialize Gaussian splash at each vertex and the center of gravity position of each model surface of the three-dimensional model or the four-dimensional model, each Gaussian splash having five basic parameters, including position, rotation, scaling, opacity and a set of spherical harmonic functions representing color information; for the Gaussian splash at the center of gravity position of each model surface, its spatial position, spatial rotation and spatial scaling are obtained from the Gaussian splash information of the vertex position in the same plane corresponding to the center of gravity position, thereby obtaining the position of the initialized Gaussian splash; Step 102: The initialized Gaussian splash position is used for training the Gaussian splash network, and the parameters of the Gaussian points are continuously optimized to generate different levels of detail, thereby obtaining a Gaussian splash model with different levels of detail.

5. A detail level rendering method based on Gaussian splatting as claimed in any one of claims 1 to 4, characterized in that: In step 102, the position of the initialized Gaussian splash is used for training the Gaussian splash network; The training process is specifically as follows: optimizing the position, rotation, scaling, opacity attribute and color information of Gaussian points; during the training process, for Gaussian splashes at the vertices of the model, their spatial positions are not changed, and only their rotation, scaling, opacity attribute and color information are changed; for Gaussian splashes at the center of gravity position, their position, rotation, scaling, opacity and color information are all changed; during the network training process, the number of Gaussian splashes is not changed; after all Gaussian splashes are optimized through the training process, a Gaussian splash model with a detailed level is obtained.

6. A detail level rendering method based on Gaussian splashing as claimed in claim 5, characterized in that: In step 102, different detail levels are generated to obtain Gaussian splash models with different detail levels, specifically: The model surface is subdivided; specifically, for the three-dimensional model or the four-dimensional model, the Gaussian splash at the center of gravity of each model surface is moved to a new position after network training; a series of new model surfaces are formed by connecting the new position points and the original vertices; Initialize a series of new Gaussian splashes at the centroid of the new model surface while keeping the position, rotation, scale, opacity and color information of the existing Gaussian splashes unchanged; Repeat the training process of the Gaussian splash network. By continuously optimizing the new Gaussian splash, new details can be added to the existing detail level, thereby obtaining a Gaussian splash model with a new detail level. The Gaussian splash model of the new detail level is evaluated, that is, the Gaussian splash model of the new detail level is reprojected to the camera position. If the image similarity with the original multi-view image reaches a set index range, the Gaussian splash model of the new detail level is saved, otherwise the Gaussian splash model of the new detail level is deleted; the camera position is the position of the camera for acquiring the multi-view image.

7. The detail level rendering method based on Gaussian splashing according to claim 1, characterized in that: In the real-time rendering stage, the real-time data of the user includes: real-time data of the user's position, the user's head orientation, and the user's hand position.

8. A method for rendering level of detail based on Gaussian splatting as claimed in claim 7, characterized in that: The real-time rendering stage selects different levels of detail according to the user's real-time data, specifically: A decision tree model, a neural symbolic decision model or a multi-layer perceptron is used to map the user's real-time data to Gaussian splash models at different levels of detail.

Images