Method and system for text generation of three-dimensional assets based on three-dimensional Gaussian sputtering

Abstract

The invention provides a text generation three-dimensional asset method and system based on three-dimensional Gaussian sputtering. According to the method, firstly, text description is input into a pre-trained three-dimensional point cloud model to generate an initial point cloud, and the initial point cloud is converted into a three-dimensional Gaussian model after densification processing and random color perturbation; then, a two-stage optimization strategy is adopted: in the first stage, a multi-view normal diffusion model is utilized to focus on geometric shape optimization, and only key parameters influencing the shape are updated; and in the second stage, color texture optimization is realized by combining a general text graph diffusion model and geometric regularization loss. According to the method, a periodic square root annealing time step sampling mechanism is innovatively introduced, and semantic expression richness and optimization stability are balanced. Compared with the prior art, the method has the remarkable advantages that the rendering efficiency and the geometric expression capability are improved through three-dimensional Gaussian expression; the multi-view normal diffusion model enhances the geometric detail quality and multi-view consistency; and an innovative time step sampling strategy ensures the stability of the optimization process.

Description

Technical Field

[0001] The present invention relates to the field of computer graphics and artificial intelligence content generation, and in particular to a method and system for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering. Background Art

[0002] With the vigorous development of emerging technologies such as the metaverse, virtual reality, and augmented reality, the demand for high-quality three-dimensional digital assets has exploded and has become the core infrastructure of the digital content industry. Traditional three-dimensional asset production methods mainly rely on manual modeling by artists, which not only requires professional modeling skills and artistic skills, but also has a long production cycle and high labor costs, making it difficult to meet the market's growing and diversified demand for three-dimensional content. To this end, the industry has successively proposed a variety of automated three-dimensional asset model generation methods based on artificial intelligence, but these methods still have many technical bottlenecks in practical applications.

[0003] Existing Neural Radiance Fields (NeRF)-based methods have limited rendering quality and visual quality. Figure 1 Significant progress has been made in terms of consistency, but there are some limitations. First, the method has high computational complexity, which leads to slow model training and rendering speeds, making it difficult to meet the needs of real-time applications. Second, due to its implicit representation characteristics, it is difficult to fine-tune the optimization process. Another type of method based on explicit representation (such as triangular meshes, point clouds, etc.) has the advantages of being intuitive, easy to edit and render, but it is difficult to accurately reconstruct the fine geometric structure and real texture details of the object during the generation process, resulting in the visual quality and realism of the generated results being less than ideal.

[0004] As an emerging 3D content representation method, 3D Gaussian Splatting (3DGS) technology has significant advantages in rendering efficiency and expression ability by representing 3D content as a series of 3D Gaussian functions with spatial distribution characteristics. Each 3D Gaussian function can be regarded as a learnable spatial particle, whose properties include mean vector (indicating spatial position), covariance matrix (characterizing shape and direction), color, and opacity.

[0005] However, when applying 3D Gaussian sputtering technology to the task of converting text to 3D assets, existing technologies still face multiple technical challenges. First, since existing methods tend to optimize the geometric shape and other properties of 3D Gaussian at the same time, it is difficult to accurately capture and reconstruct the fine geometric details of objects from text descriptions; second, the problem of uneven Gaussian distribution often occurs during the optimization process, resulting in excessive dispersion or local aggregation in certain areas, which seriously affects the surface quality and geometric continuity of the generated results; finally, the optimization method based on random single-view sampling is difficult to ensure that the generated results maintain consistency in geometry and appearance under multiple viewpoints.

[0006] In view of the above technical difficulties, the present invention makes full use of the explicit characteristics of three-dimensional Gaussian and innovatively proposes a geometric shape independent optimization method based on a multi-view normal diffusion model. By introducing a time step sampling strategy of periodic square root annealing and combining it with a carefully designed geometric regularization loss function, the present invention significantly improves the overall quality and multi-view consistency of the generated results. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for generating 3D assets from text based on 3D Gaussian sputtering, so as to solve the problems of insufficient geometric details and poor multi-view consistency in the prior art. The present invention adopts 3D Gaussian as the basic 3D representation unit and combines multiple pre-trained diffusion models to achieve automatic and efficient generation from text prompt words to high-quality 3D assets. Compared with the prior art, the present invention has the following technical advantages:

[0008] 1) The three-dimensional Gaussian representation method is used to significantly improve rendering efficiency and geometric expression capabilities;

[0009] 2) Innovatively introduces a multi-view normal diffusion model, which effectively improves the quality of geometric details and multi-view consistency;

[0010] 3) A unique periodic square root annealing sampling strategy was designed to achieve the stability of the optimization process.

[0011] The present invention adopts the following technical solution:

[0012] A method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering, characterized by comprising the following steps:

[0013] Step 1: 3D Gaussian initialization: input the target 3D asset description text into the pre-trained 3D point cloud model to generate the initial point cloud of the object, perform densification and random color perturbation on the initial point cloud, and convert it into a 3D Gaussian model.

[0014] Step 2, iterative optimization of geometric shape, iterative optimization of geometric shape of the three-dimensional Gaussian model, specifically including the following steps:

[0015] S21, performing multi-view sampling on the three-dimensional Gaussian model in an iterative process according to a preset camera view, rendering to obtain depth image samples under the multi-view, and calculating normal image samples according to the depth image samples;

[0016] S22, optimizing the geometric shape of the three-dimensional Gaussian model through a pre-trained multi-view normal diffusion model according to the normal image sample, fixing the color and transparency parameters of the three-dimensional Gaussian model during the optimization process, and only iteratively updating the parameters that affect the geometric shape;

[0017] Step 3: iterative optimization of color texture. On the basis of ensuring the quality of geometric shape, the color texture of the 3D Gaussian model is further iteratively optimized. Specifically, the following steps are included:

[0018] S31, sampling the three-dimensional Gaussian model in an iterative process according to the random camera perspective, rendering to obtain natural image samples and depth image samples, and estimating normal image samples according to the depth image samples;

[0019] S32, based on natural image samples, through pre-trained general language learning Figure 2 dimensional diffusion model, calculating the difference loss between the sample and the diffusion model prior;

[0020] S33, calculating a geometric regularization loss based on the normal image according to the normal image sample, and iteratively optimizing the parameters of the three-dimensional Gaussian model by combining the geometric regularization loss with the diffusion model prior difference loss.

[0021] Furthermore, in step S22 and step S32, during the iterative optimization of the geometric shape and the iterative optimization of the color texture, the present invention adopts a periodic square root annealing time step sampling strategy, and achieves an effective balance between the richness of semantic information and the stability of optimization by dynamically adjusting the time step in each optimization cycle. This strategy significantly improves the overall quality of the generated results.

[0022] Further, in the color texture optimization step in step S33, the geometric regularization loss function based on the normal image includes: normal flatness loss L NS ,This indicator measures the flatness of the object surface by calculating the difference between adjacent normal vectors; the adjacent normal angle difference loss L AV , which evaluates the continuity of the surface by calculating the angle between adjacent normal vectors, and can better capture the mutation characteristics of the surface; local normal consistency loss L LC ,This metric is used to evaluate the consistency of normals in a local area, which helps to maintain the local smoothness of the surface while retaining important geometric features.

[0023] The beneficial effects of the present invention include: using three-dimensional Gaussian as the basic representation unit, which not only ensures rendering efficiency but also can express rich geometric and appearance details; through the guidance of the multi-view normal map diffusion model, the geometric shape details of the generated results are significantly improved, and the geometric consistency under different view angles is improved; the normal map-based geometric regularization loss is introduced to effectively maintain the geometric shape quality while optimizing the texture; the annealing sampling strategy is used to dynamically adjust the time step, balancing the richness of semantic information and optimization stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the process of the present invention;

[0025] Figure 2 It is a schematic diagram of the overall system framework of the present invention;

[0026] Figure 3 It is a comparison diagram of the results of the present invention under different optimization methods;

[0027] Figure 4 It is an actual effect diagram of the present invention generating corresponding results according to the text prompt words. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] like Figure 1 and Figure 2 As shown, a method for generating a text to a three-dimensional asset based on three-dimensional Gaussian sputtering comprises the following steps:

[0030] Step S100: input the description text of the three-dimensional asset into the pre-trained three-dimensional point cloud model to generate a three-dimensional initial point cloud.

[0031] Step S200: Using a preset method, perform densification and random color perturbation operations on the initial point cloud to convert it into an initial three-dimensional Gaussian model.

[0032] Step S300, according to the preset camera viewing angle, multi-view sampling is performed on the three-dimensional Gaussian model in an iterative process, depth image samples under the multi-view are obtained by rendering, and normal image samples are calculated according to the depth image samples.

[0033] Step S400, optimizing the geometric surface shape of the three-dimensional Gaussian model through a pre-trained multi-view normal diffusion model according to the multi-view normal image samples, and updating the mean and covariance parameters of the three-dimensional Gaussian model;

[0034] Step S500, performing random perspective sampling on the three-dimensional Gaussian model according to the random camera perspective, rendering to obtain natural image samples and depth image samples under the random perspective, and calculating normal image samples according to the depth image samples;

[0035] Step S600: Based on the natural image sample, the pre-trained general language Figure 2 dimensional diffusion model, calculate the difference loss value between the current sample and the diffusion model prior;

[0036] Step S700: Calculate the geometric regularization loss value of the current sample based on the normal image sample. Combine the geometric regularization loss value with the prior difference loss value of the two-dimensional diffusion model to jointly optimize and update all parameters of the three-dimensional Gaussian model, and iterate to obtain the final three-dimensional asset generation result.

[0037] In step S200, the densification process and random color perturbation operation are performed on the initial point cloud, and the specific implementation method is as follows:

[0038] The spatial boundary value of the initial point cloud is obtained, and a uniformly distributed set of candidate spatial points is randomly generated within the boundary range; a KD tree index structure is constructed based on the spatial position information of the initial point cloud, and the distance value between all spatial points in the candidate spatial point set and the nearest neighbor point in the KD tree is calculated. When the distance value is less than a preset threshold, the corresponding candidate spatial point is retained, otherwise it is discarded. For each retained spatial point, a random perturbation value is superimposed on the color of the corresponding nearest initial point cloud point.

[0039] The initialization process of the three-dimensional Gaussian model is as follows: the spatial point position coordinates obtained by the above processing are merged with the original point cloud position information to initialize the mean parameters of the three-dimensional Gaussian; the spatial point color attributes are used to initialize the color attributes of the three-dimensional Gaussian; the distance between each three-dimensional Gaussian and its nearest neighbor is calculated, and the distance is used to initialize the covariance parameters of the corresponding three-dimensional Gaussian; the opacity parameters of all three-dimensional Gaussians are uniformly set to a predetermined initial value.

[0040] The present invention obtains technical inspiration from in-depth research and analysis of traditional sculpture techniques, and creatively proposes a novel phased optimization method. In traditional sculpture techniques, sculptors first focus on shaping the basic form of the object, and then perform coloring and texturing after the form is completely determined. This process flow inspires us to divide the optimization process of the three-dimensional Gaussian model into two independent stages: shape optimization and texture optimization. In the shape optimization stage, the present invention fixes the color parameters and transparency parameters so that the optimization process focuses on adjusting the geometric shape of the object. This technical means can effectively avoid the interference of color and transparency parameters on shape optimization, and ensure the accuracy and stability of shape optimization. This phased optimization strategy avoids mutual interference between parameters, improves the controllability of the optimization process, and at the same time ensures the stable maintenance of shape features, significantly improving the quality of the final optimization results.

[0041] In step S400, the geometric surface shape of the three-dimensional Gaussian model is optimized according to the multi-view normal image samples by using a pre-trained multi-view normal diffusion model. The specific method includes:

[0042] For the multi-view normal image samples, the noisy samples at time step t and time step s=t-Δt are calculated by the cyclic inversion process of the denoising diffusion implicit model (DDIM). and The 3D Gaussian parameter update is guided by the following optimization objective gradient formula:

[0043]

[0044] where θ g represents the parameters that affect the geometric shape in the three-dimensional Gaussian model; g n (·) represents the rendering process of the normal map; each viewing angle c i The normal map below is represented as L geo is the multi-view normal map sampling loss function, φ n is the pre-trained multi-view normal map diffusion model, y is the input text prompt word, ω(t) is the weight parameter associated with time step t;

[0045] To ensure that the optimization process focuses on the improvement of geometric shape, only the mean and covariance parameters of the 3D Gaussian model are updated in the iterative optimization stage, thereby achieving accurate optimization of geometric features.

[0046] It should be noted that the pre-trained multi-view normal diffusion model used in the present invention is obtained by fine-tuning the Objaverse 3D object dataset based on the general Vincent graph pre-training model. Through the fine-tuning process of the professional 3D dataset, the model obtains more accurate 3D shape perception capabilities. Compared with directly using the general Vincent graph model, it can better adapt to the special data distribution characteristics of the normal map, thereby providing a more stable and realistic geometric shape optimization effect.

[0047] The model fine-tuning method specifically includes: presetting 48 observation angles around the object, the camera positions of these angles are distributed in the radial distance range of 1.5 to 3.0 units, the elevation angle covers the range of -10 to 30 degrees, and the azimuth angle covers 0 to 360 degrees. Rendering obtains the natural image of the object and its corresponding normal image at each viewing angle position. To ensure data quality, samples with rough surface details and samples with low CLIP correlation between the rendered image and the text description of the object are eliminated. Noise prediction loss is used as the optimization loss function in the model training stage:

[0048]

[0049] Where c represents the camera pose, y represents the text description of the object, and z t represents the noise-added sample after noise addition at a specific time step t, ∈ θ represents the noise prediction result, ∈ represents random noise.

[0050] It should be noted that although the multi-view pre-training model contains rich geometric prior knowledge, this global constraint also limits the detail expression of the generated results to a certain extent. Considering that a stable geometric shape has been obtained in the geometric optimization stage, the present invention adopts a two-dimensional diffusion model of a general Vincent graph in the subsequent color and texture optimization stage, and cooperates with a random single-view sampling strategy. This optimization scheme utilizes the characteristics of the two-dimensional diffusion model in detail generation, and improves the richness and realism of texture details while maintaining the stability of the geometric shape.

[0051] In step S600, the natural image samples are pre-trained using a general language Figure 2 dimensional diffusion model, calculate the difference loss value between the current sample and the diffusion model prior, including:

[0052] The natural image samples z rendered from random camera perspectives 0 , the noisy image z at time step t and time step s = t-Δt is calculated through the cyclic inversion process of the denoising diffusion implicit model t and z s ; Input the noised image into the pre-trained Figure 2The noise prediction is performed in the dimensional diffusion model φ, and the noise prediction result under the text guidance y is obtained. Noise prediction results without text guidance The difference loss is calculated by the following loss formula:

[0053]

[0054] Among them, c represents the camera position parameter, ω(t) represents the weight parameter related to the time step, and y represents the input text prompt word.

[0055] It should be noted that the present invention does not fix the geometric parameters of the three-dimensional Gaussian model in the color texture optimization stage, but retains the possibility of fine-tuning the geometric details. The consideration of this design is that there may be finer geometric features that need to be adjusted during the color texture optimization process. However, this design also brings some technical challenges: the three-dimensional Gaussian model may have unnatural deformation of the geometric shape, or produce blurred edges and rough surfaces during the texture optimization stage. In order to effectively solve these problems, the present invention introduces a regularized loss function based on normal calculation. This loss function can effectively suppress unnecessary shape deformation while maintaining the freedom of geometric detail optimization, maintain the smoothness and continuity of the object surface, and ensure that the final generated result has a natural geometric form and delicate surface texture by constraining the local surface normal.

[0056] The geometric regularization loss is calculated according to the found image samples in step S700, and the specific calculation method includes:

[0057] The loss function consists of three items: the first item is the normal flatness loss L used to measure the surface flatness NS This indicator measures the flatness of the object surface by calculating the difference between adjacent normal vectors. The smaller the value, the smoother the surface. The calculation formula is as follows:

[0058]

[0059] The second term is the adjacent normal angle difference loss L for evaluating surface continuity. AV This indicator evaluates the continuity of the surface by calculating the angle between adjacent normal vectors, which can better capture the mutation characteristics of the surface. The calculation formula is as follows

[0060]

[0061] The third term is the local area normal consistency loss L LC , which evaluates the consistency of normals in a local area, where N K(i,j) represents a K×K neighborhood centered at (i,j). Adding this indicator helps maintain the local smoothness of the surface while retaining important geometric features.

[0062]

[0063] By weighted combination of the above three losses, the final geometric regularization loss function is obtained:

[0064]

[0065] where ω 1 ,ω 2 and ω 3 is the corresponding weight coefficient. This geometric regularization loss function can effectively suppress unnatural geometric deformation during the optimization process. Figure 3 The effect of adding geometric regularization loss on the generated results is shown in.

[0066] In the color texture iterative optimization stage, the present invention adopts a weighted combination of the total loss function L texture Optimize. The loss function is composed of the color optimization loss L rgb With geometric regularization loss L norm It consists of two parts, and the specific formula is:

[0067]

[0068] where λ rgb and λ norm are the weight coefficients of the corresponding loss terms respectively. By adjusting the ratio of the weight coefficients, a balance between color texture optimization and geometric shape preservation can be achieved.

[0069] In step 400 and step 600, before obtaining the prior guidance through the diffusion model, it is necessary to perform noise processing on the input samples, and in the noise adding process, the noise intensity needs to be accurately controlled through the time step parameter.

[0070] After a large number of experimental verifications, it is found that the choice of time step parameters has a significant impact on the final generation effect. Specifically, a larger time step parameter is conducive to the model capturing rich high-level semantic features, but due to the large difference between the noised samples and the original samples, the continuous use of a large time step will lead to an unstable optimization process, making it difficult for the three-dimensional Gaussian model parameters to converge effectively, and ultimately causing the blurring of detail features. Relatively speaking, under a smaller time step parameter, the difference between the noised samples and the original samples is small. Continuously using a smaller time step parameter for optimization can ensure the stable convergence of the model parameters and obtain clearer surface detail results, but it often leads to insufficient expression of semantic features, affecting the overall quality of the generation results.

[0071] To solve the above problems, the present invention adopts a periodic square root annealing strategy to dynamically adjust the noise adding time step. Specifically, during the optimization iteration process, the calculation formula of the time step t is:

[0072]

[0073] Among them, t max and t min Represent the preset maximum and minimum time steps, i represents the current iteration number, N represents the total iteration number, and n represents the set number of optimization cycles. Through this calculation method, the time step is gradually reduced from a larger value to a smaller value at a square root rate in each optimization cycle.

[0074] The sampling method effectively balances the richness of semantic information and shape consistency by dynamically adjusting the time step, avoiding the semantic simplification problem caused by continuous use of smaller time steps and the optimization instability and detail blurring caused by continuous use of larger time steps. Figure 3 As shown, compared with the random sampling strategy, the periodic square root annealing sampling strategy proposed in the present invention has obvious advantages in generation effect, and can simultaneously ensure the semantic integrity and geometric detail quality of the model.

[0075] Figure 4 FIG. 2 shows the experimental results obtained by the method for generating three-dimensional assets from text according to an embodiment of the present invention. Figure 4 As shown, by simply inputting the corresponding text description prompt words, the technical solution of the present invention can automatically generate the corresponding high-quality three-dimensional model. The generation process does not require manual intervention, nor does it require other auxiliary information such as reference images or three-dimensional models. Experimental results show that the technical solution provided by the present invention can accurately understand the text semantics and convert it into a three-dimensional asset with fine geometric details and realistic texture effects, which fully verifies the feasibility and effectiveness of the present invention.

Claims

1. A method for generating three-dimensional assets based on text from three-dimensional Gaussian sputtering, characterized in that: The build process consists of the following steps: Step 1: Three-dimensional Gaussian initialization. Receive the text prompt words input by the user and input them into the pre-trained 3D point cloud model. The model generates point cloud data based on the text prompt words. Then, the point cloud data is converted into a 3D Gaussian model. To improve the initialization quality, the 3D Gaussian model is subjected to densification and random color perturbation. The densification operation is used to enhance the ability to express geometric details, and the random color perturbation is used to improve the robustness of subsequent optimization. Step 2: Iterative optimization of geometry. After completing the initialization, the method optimizes the geometric shape of the three-dimensional Gaussian model. Specifically, a set of camera perspective parameters covering different angles are first predefined, and multi-perspective sampling is performed on the three-dimensional Gaussian model based on these preset camera perspectives. Subsequently, the sampling results are processed, depth map samples under multiple perspectives are rendered and generated, and the corresponding normal map is calculated based on the acquired depth map. Finally, the multi-perspective normal map is input into a pre-trained multi-perspective normal map diffusion model, and the geometric shape optimization is guided by the prior knowledge of the model to iteratively update the geometric surface shape of the three-dimensional Gaussian model. The pre-trained multi-perspective normal map diffusion model can effectively ensure the consistency of the generated results under different perspectives by processing normal maps of multiple perspectives at the same time, and avoid geometric shape mutations when the perspective is switched. Step 3: Iterative optimization of color texture. On the basis of obtaining a stable geometric shape, this step further iteratively optimizes the color texture of the three-dimensional Gaussian model. First, multiple camera view positions are randomly sampled, and two-dimensional images are rendered from these sampled viewpoints. Subsequently, the rendered image is input into the pre-trained Vincent graph two-dimensional diffusion model, the difference between the diffusion model prior is calculated, and gradient optimization is performed based on the difference. To ensure that the optimized geometric surface shape is not affected during the color texture optimization process, this step introduces a geometric regularization loss based on the normal map, which effectively suppresses unnatural geometric deformations by evaluating normal flatness, calculating the difference in adjacent normal angles, and measuring local normal consistency, while maintaining smooth surface quality and key geometric features.

2. The method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: The three-dimensional Gaussian initialization module is specifically implemented in the following steps: First, the text prompt word is input into the pre-trained three-dimensional diffusion model to generate an initial point cloud set containing point cloud positions and color attributes. Subsequently, the initial point cloud data is densified. Specifically, the boundary value of the initial point cloud is obtained, and spatial points that obey a uniform distribution are randomly generated within the boundary range; a KD tree structure is constructed based on the initial point cloud position, and the distance between the spatial point and the nearest point in the KD tree is calculated, and the spatial point is retained when the distance is less than a preset threshold. The screened spatial points are subjected to color perturbation. For each retained spatial point, a random perturbation value is added based on the color of its nearest initial point cloud point to obtain the perturbed color attribute. Finally, the initial point cloud data and the processed spatial point data are converted into a three-dimensional Gaussian model. Specifically, the point cloud position and the spatial point position are merged and initialized as the mean of the three-dimensional Gaussian, and the point cloud color and the perturbed spatial point color are merged and initialized as the color attribute of the three-dimensional Gaussian; the distance between each three-dimensional Gaussian and the nearest neighbor is calculated, and the distance value is initialized as the covariance of the three-dimensional Gaussian; at the same time, the opacity of all three-dimensional Gaussians is uniformly initialized to a preset value.

3. The method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: The specific implementation process of the geometric shape optimization step is as follows: First, based on the preset multiple camera poses c i The three-dimensional Gaussian model is sampled from multiple perspectives and rendered at each perspective to obtain the corresponding normal map, where θ g represents the parameters that affect the geometric shape in the three-dimensional Gaussian model; g n (·) represents the rendering process of the normal map; each viewing angle c i The normal map below is represented as Secondly, for the normal map at each viewing angle, the noise addition results at time step t and time step s = t-Δt are calculated through the cyclic inversion process of the denoising diffusion implicit model. and Subsequently, the 3D Gaussian parameter update is guided by the following optimization objective gradient formula: Among them, L geo is the multi-view normal map sampling loss function, φ n is a pre-trained multi-view normal map diffusion model, y is the input text prompt word, ω(t) is the weight parameter related to the time step t; in order to optimize the geometric shape, only the mean and covariance parameters of the three-dimensional Gaussian model are updated during the iteration process.

4. The method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: The calculation process of the color optimization loss function in the color texture optimization step is as follows: The 2D image z0 is rendered by the 3D Gaussian model through random camera poses; the noisy image z at time step t and time step s = t-Δt is calculated through the cyclic inversion process of the denoising diffusion implicit model. t and z s ; Input the noisy image into the pre-trained two-dimensional diffusion model φ and optimize it using the following loss function: Among them, ω(t) is the weight parameter related to the time step, and y is the input text prompt word.

5. The method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: In the geometric shape iterative optimization step and the color texture iterative optimization step, before the sample is input into the diffusion model to obtain the prior guidance, the input sample needs to be denoised. The time step parameter used in the denoising process is determined by a periodic square root annealing sampling method, and the method content is as follows: Considering the differences in the characteristics of the diffusion model at different time steps, the present invention uses a periodic square root annealing sampling method to dynamically adjust the noise time step parameter. Specifically, during the optimization iteration process, the time step t is calculated as follows: Among them, t max and t min Represent the preset maximum and minimum time steps, i represents the current iteration number, N represents the total iteration number, and n represents the set number of optimization cycles. Through this calculation method, the time step parameter will gradually decrease from a larger value to a smaller value at a square root rate in each optimization cycle.

6. The method for generating three-dimensional assets from text based on three-dimensional Gaussian sputtering according to claim 1, characterized in that: The color texture iterative optimization step introduces a geometric regularization loss function based on the normal map, and the specific implementation method is as follows: The loss function consists of three items: the normal flatness loss L used to measure the flatness of the surface NS , adjacent normal angle difference loss L for evaluating surface continuity AV , and the loss L that measures the consistency of local region normals LC , the calculation formula is as follows: This indicator measures the flatness of the surface by calculating the difference between adjacent normal vectors. The smaller the value, the smoother the surface. This indicator evaluates the continuity of the surface by calculating the angle between adjacent normal vectors, which can better capture the mutation characteristics of the surface. This indicator evaluates the consistency of normals in a local area, where N K (i,j) represents a K×K neighborhood centered at (i,j). Adding this indicator helps maintain the local smoothness of the surface while retaining important geometric features. By weighted combination of the above three losses, the final geometric regularization loss function is obtained: Among them, ω1, ω2 and ω3 are the corresponding weight coefficients. The geometric regularization loss function can effectively suppress unnatural geometric deformation during the optimization process.

7. The method for generating three-dimensional assets based on three-dimensional Gaussian sputtering under text drive according to claim 1, characterized in that: The color texture optimization step adopts a weighted combination of the total loss function L texture Optimize, the loss function is the color optimization loss L described in claim 6 rgb The geometric regularization loss L according to claim 7 norm The specific expression is: where λ rgb and λ norm are the weight coefficients of the corresponding loss terms respectively. By adjusting the ratio of the weight coefficients, a balance between color texture optimization and geometric shape preservation can be achieved.

8. A text-to-3D asset generation system based on 3D Gaussian sputtering, characterized in that: The system includes the following modules: Initialization module, which accepts text prompt words as input, generates a point cloud model through a pre-trained 3D diffusion model, and converts it into a 3D Gaussian model. In order to improve the initialization quality, the generated 3D Gaussian model is densified and randomly perturbed during the conversion process. The geometric shape optimization module accepts a three-dimensional Gaussian model and a text prompt word as input, performs multi-view sampling on the three-dimensional Gaussian model according to the preset camera pose, renders to obtain multi-view depth map samples, estimates the normal map based on the depth map, and iteratively optimizes the geometric surface shape of the three-dimensional Gaussian model through a pre-trained multi-view normal map diffusion model. Color texture optimization module, which accepts 3D Gaussian model and text prompt words as input, obtains 2D renderings based on randomly sampled camera perspectives, and optimizes the model by minimizing the difference with the prior of the 2D diffusion model of the Vincent graph. At the same time, in order to ensure the geometric surface quality during the color texture optimization process, a geometric regularization loss function based on normal map calculation is introduced to effectively suppress the unnatural geometric deformation and surface roughness of the 3D Gaussian model. The rendering module can rasterize the three-dimensional Gaussian model into corresponding two-dimensional natural images and depth images according to a given camera perspective.

Images