Newton-Simpson-based text 3D rapid generation method and system

Abstract

The invention provides a Newton-Simpson-based text 3D rapid generation method and system, belongs to the technical field of cross fusion of computer graphics and artificial intelligence, and aims to improve the efficiency and precision of a three-dimensional generation process. According to the method, 3D Gaussian point rendering is adopted as a three-dimensional model representation mode, and high-quality conversion from text semantics to a three-dimensional structure is realized through combination of projection, rendering and a diffusion model. In the training stage, a score distillation sampling (SDS) or variational score distillation (VSD) algorithm is introduced to construct three-dimensional noise prediction loss, and a parallel acceleration framework of banded nonlinear solution is constructed based on a Newton-Simpson optimization method, so that the convergence speed and stability of the training process are remarkably improved. Compared with a traditional method, the method has the advantages that the overall training process is optimized through a sliding window parallel strategy and a self-adaptive stop mechanism, and the method has higher expression ability, higher generation speed and higher expandability.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of computer graphics and artificial intelligence, specifically relating to a method and system for rapid generation of textual 3D based on Newton-Simpson. Background Technology

[0002] Text-to-3D technology aims to automatically generate 3D models from natural language descriptions, representing a crucial direction for the cross-fertilization of computer graphics and artificial intelligence (especially generative AI). Previously, 3D models primarily relied on manual modeling (such as Maya, Blender, and 3ds Max) or 3D scanning techniques, which were time-consuming and highly dependent on specialized skills, creating production bottlenecks in fields like game development, film and television production, virtual reality, augmented reality, and digital twins. In recent years, with the rise of large-scale text-to-image pre-trained models (such as CLIP and Stable Diffusion) and breakthroughs in technologies like neural fields, 3D Gaussian splatting, and implicit representations, generating high-quality 3D content directly from text has become a trend. This trend has driven a paradigm shift from manual modeling to automated generation, significantly lowering the barrier to 3D asset production and potentially becoming the core infrastructure for next-generation content production.

[0003] To achieve high-quality 3D model generation, current mainstream methods largely rely on neural representations such as NeRF (Neural Radiation Field) or volumetric rendering. However, neural volumetric rendering methods are typically computationally expensive, slow, and resource-intensive during training, limiting their scalability in real-time applications. To overcome these issues, the 3D Gaussian point rendering method, proposed in recent years, has attracted increasing attention as a novel 3D representation. This method expresses the scene structure in 3D space through a set of parameterized Gaussian distributions. Its rendering process does not rely on volumetric sampling or neural network forward propagation; instead, it is based on differentiable rasterization operations for synthesis, offering high efficiency and flexibility.

[0004] Meanwhile, the outstanding performance of diffusion probabilistic models (DPMs) in image generation tasks has spurred a surge in their application in the field of 3D generation. Since existing large-scale, high-quality image datasets far outnumber 3D model data, the academic community has proposed converting 3D models into 2D images through projection, and then using well-trained 2D diffusion models for indirect 3D modeling (e.g., patent CN120318382A discloses a real-time 3D human avatar generation method and system based on 3D Gaussian Splatting). Such methods typically map 3D features to 2D images from multiple perspectives, achieving inverse optimization and 3D reconstruction through noise learning and denoising prediction in the image space. However, this process involves complex nonlinear optimization problems, especially in high-resolution, multi-view, and multi-sample training scenarios, facing technical bottlenecks such as low training efficiency, unstable gradients, and slow convergence speed.

[0005] Furthermore, existing 3D generative models mostly employ a stepwise update strategy based on static optimizers, failing to fully utilize the potential temporal coupling characteristics and parallel capabilities. This results in difficulty in improving computational efficiency during training, making it hard to meet the requirements for efficient generation. Therefore, how to construct an efficient, stable, and parallelizable 3D generation optimization mechanism while ensuring generation quality has become a key research focus and technical challenge in this field. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a rapid textural 3D generation method and system based on Newton-Simpson optimization. This method combines a diffusion probability model with 3D Gaussian point rendering representation and introduces a Newton-Simpson optimization method to construct a parallel, strip-shaped nonlinear solution framework. It systematically solves the problems of slow convergence, high computational resource consumption, and unstable noise optimization inherent in traditional generation methods for high-dimensional 3D modeling. By combining SDS or VSD algorithms for noise-driven training and designing a sliding window parallel strategy and adaptive stopping criteria, the training efficiency and accuracy of the diffusion model in 3D generation tasks are significantly improved, providing an innovative technical path for high-quality, high-efficiency textural 3D generation.

[0007] The technical solution adopted in this invention is as follows:

[0008] In a first aspect, this invention proposes a rapid textural 3D generation method based on Newton-Simpson union, comprising the following steps:

[0009] (1) Obtain the 3D shape parameters and their corresponding text prompts represented by the initialized 3D Gaussian point rendering method;

[0010] (2) The dimensions of the 3D shape parameters are unified by the projection algorithm to generate 3D shape parameters with fixed dimensions, and then converted into a two-dimensional image with a fixed viewpoint by the rendering function;

[0011] (3) Input the two-dimensional image, text prompt and fixed viewpoint parameters into the pre-trained diffusion model, combine the score distillation sampling SDS or variational score distillation VSD algorithm to construct the loss function, optimize the fixed-dimensional 3D shape parameters according to the gradient of the loss function, and establish a parallel acceleration framework for strip nonlinear solution based on the Newton-Simpson optimization method to optimize the training process.

[0012] (4) Output the final optimized fixed-dimensional 3D shape parameters to generate the 3D model.

[0013] Furthermore, the projection algorithm in step (2) achieves dimensionality compression by deleting a subset of the Gaussian point set.

[0014] Furthermore, the rendering function in step (2) uses fixed viewpoint parameters to project fixed-dimensional 3D shape parameters to generate a two-dimensional image with two-dimensional feature representation.

[0015] Furthermore, a parallel acceleration framework for solving strip-shaped nonlinear problems is established based on the Newton-Simpson optimization method, including:

[0016] A nonlinear equation system with localized band coupling at time steps is established, and the recursive form of the equation is expressed as:

[0017]

[0018]

[0019] The general formula for iterative solution is:

[0020]

[0021] in, This represents an update function that includes gradients from the SDS or VSD algorithm. Indicates the first , The three-dimensional shape parameters of the time step. This represents the gradient of the SDS or VSD algorithm. This represents the derivative with respect to the three-dimensional shape parameters. Indicates the step size. Representing a nonlinear equation system, This represents the initial estimate of the parameters in the (k+1)th iteration. In the k-th iteration, at time step The approximate solution of the parameters, This represents the initial parameter vector for the iteration, i.e., the initial conditions. This indicates that at the k-th iteration, the update function is applied at the j-th time step. The calculated increment, Indicates a time step. Indicates a time step. This represents the approximate solution of the parameters at the j-th time step in the k-th iteration.

[0022] Furthermore, the parallel acceleration framework also includes:

[0023] The preprocessing mechanism in the startup phase generates initial values ​​of 3D shape parameters for several time steps based on the fixed-dimensional 3D shape parameters in step (2) through single-threaded optimization or existing coarse estimation.

[0024] The sliding window parallel strategy, within a fixed-length window [ , , …, The residuals of the 3D shape parameters between two adjacent iterations are calculated in parallel within the [internal] module, and then the window is advanced according to the updated parameters. Indicates the first in the window Time step parameters, It is a parameter related to window size;

[0025] Adaptive stopping criteria, for time steps within a sliding window The norm difference before and after parameter update is dynamically determined to converge. If convergence is achieved, the window is advanced. The threshold value used to determine the norm difference is dynamically adjusted using the moving average estimate of the error within the window.

[0026] Furthermore, the convergence criterion for the adaptive stopping criterion is as follows:

[0027]

[0028] in, The parameter dimension representing the 3D shape parameter. Indicates the threshold value. This represents the time step within the window during the (k+1)th iteration. 3D shape parameters, Represents the time step within the window in the k-th iteration. 3D shape parameters.

[0029] Furthermore, in the parallel acceleration framework, when the sliding window reaches the last time step and converges, the update result of the last time step under the latest iteration is output as the final optimized fixed-dimensional 3D shape parameter.

[0030] Furthermore, the formula for calculating the gradient of the SDS algorithm is:

[0031]

[0032] in, Represents the gradient of the SDS algorithm. Expressing expectations, This represents a weighting function with respect to diffusion time. Represents the predicted two-dimensional noise. Indicates time step The noisy image at that location, Indicates a time step. Indicates a text description. This represents Gaussian random noise. Indicates the viewpoint parameter. This represents the rendering function.

[0033] Furthermore, the formula for calculating the gradient of the VSD algorithm is:

[0034]

[0035] in, Represents the gradient of the VSD algorithm. Expressing expectations, This represents a weighting function with respect to diffusion time. Represents the predicted two-dimensional noise. Indicates time step The noisy image at that location, Indicates a time step. Indicates a text description. This represents Gaussian random noise. This represents a learnable noise estimation network. Indicates the viewpoint parameter. This represents the rendering function.

[0036] Secondly, this invention proposes a Newton-Simpson-based rapid text-based 3D generation system to implement the aforementioned text-based 3D generation method.

[0037] The beneficial effects of this invention are:

[0038] This invention effectively reduces the sequential dependency problem of parameter updates during 3D generation, enabling parallel computation and consistency correction of multi-step gradients. Compared with existing sequential optimization methods, this invention significantly shortens training time and improves computational efficiency while ensuring the quality of generated results and global convergence. In summary, this invention can rapidly generate 3D models with highly consistent semantics and text descriptions, and realistic and stable geometry and appearance under limited computing resources, significantly enhancing the practical value of 3D content generation. Attached Figure Description

[0039] Figure 1This is a schematic diagram of the rapid generation method for textual 3D based on Newton-Simpson proposed in this invention.

[0040] Figure 2 This is a structural block diagram of the Newton-Simpson-based rapid generation method for textual 3D proposed in this invention.

[0041] Figure 3 This is a schematic diagram of the diffusion model training and generation algorithm of the present invention;

[0042] Figure 4 This is a schematic diagram of the parallel acceleration framework algorithm of the present invention;

[0043] Figure 5 This is a schematic diagram of the structure of the Newton-Simpson-based rapid text generation system for 3D text based on the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0045] like Figure 1 and Figure 2 As shown in the embodiment, the rapid generation method for textual 3D based on Newton-Simpson's algorithm includes the following steps:

[0046] (1) Obtain the 3D shape parameters and text hints represented by the initialized 3D Gaussian point rendering method.

[0047] 3D Gaussian point rendering uses a set of parameterizable 3D Gaussian distributions to describe the geometric structure and visual attributes of spatial points in a scene, and employs differentiable rasterization operations to project and blend each Gaussian point. 3D Gaussian point rendering is an efficient and flexible 3D representation method, gradually becoming one of the mainstream choices for 3D generation guided by diffusion models. In this method, the number, position, scale, and orientation of points are dynamically adjusted during training to ensure 3D representation capability and efficiency. Each Gaussian point consists of attributes such as a position vector, covariance matrix, color, transparency, and density. During rendering, these 3D Gaussian points are projected onto the image plane and synthesized to form the final image. During training, these Gaussian points are split and cropped to enhance expressive power and control computational scale. When some Gaussian points cover too large an area or no longer contribute effective gradients, the system can selectively delete them or subdivide them into multiple new points, thereby controlling model complexity without sacrificing accuracy.

[0048] Text prompt It is a text description entered by the user.

[0049] (2) For 3D shape parameters with different Gaussian point dimensions (the Gaussian point dimensions will also change during training), the projection algorithm is used to convert them into 3D shape parameters with fixed dimensions.

[0050] In this embodiment, it is assumed that the 3D shape parameters of a sample, expressed using the 3D Gaussian point rendering method, are as follows: Because each 3D model sample has different features, their number of Gaussian points is different, i.e. The dimensions vary. To ensure consistent input dimensions during model training, a projection algorithm is used. By input Delete a set as The point set implementation of the subset from arrive The projection transforms the sample into a 3D shape with fixed dimensions. 3D feature samples. The projection algorithm is expressed as:

[0051]

[0052] in, It is a 3D shape parameter represented by a 3D Gaussian point rendering method from a certain sample.

[0053] (3) For a fixed-dimensional 3D shape parameter, the three-dimensional shape is projected into a two-dimensional image from a certain camera view (randomly generated) through a rendering function.

[0054] In the embodiment, 3D shape parameters It cannot be directly used on existing 2D diffusion models; viewpoint parameters must be used. Through the rendering function The 3D features are projected and transformed into 2D features. (Rendering function) Algorithms such as parallel projection, orthographic projection, and perspective projection can be used.

[0055] (4) Add noise to the rendered 2D image to obtain a noisy image. .

[0056] (5) Add noise to the image As training data, it is input into the pre-trained diffusion model (combined with time steps). Text prompts (etc.) to obtain the predicted noise .

[0057] In this embodiment, as Figure 3As shown, Diffusion Probabilistic Models (DPMs) are a class of generative models based on noise injection and denoising. The forward diffusion process of the diffusion model is used to generate training data. For clean image samples, at each time step... The current state with added noise to the sample Through drift items For the current state Make changes and add a diffusion term to it. Controlled standard Brownian motion noise Generate a noisy image Through gradual addition of noise, the original samples eventually become random samples that approximate a standard normal distribution. This process is modeled using a stochastic differential equation (SDE) or an ordinary differential equation (ODE). The diffusion model can be written as:

[0058]

[0059] in, This is the current state. and These are the drift term and the diffusion term, respectively. It is standard Brownian motion. The model's back-diffusion process is used for model training and is the final generation process used for production. Starting from a state of pure Gaussian noise, it follows the gradient of the data distribution. This process guides the model step-by-step to remove noise and generate high-quality data samples. The inverse process takes the following form:

[0060]

[0061] In the diffusion distillation framework for text-to-3D generation, the diffusion model acts as a guiding module, playing a crucial role in transforming 2D priors into 3D supervisory signals. This is given by a multi-view 2D image generated by a differentiable renderer. and the corresponding camera parameters (Elevation angle, azimuth angle, line-of-sight distance) This module first uses a pre-trained VAE encoder to map the image to the latent space to obtain the latent variable z. Then, at time step... By applying forward diffusion noise to the hidden variables, we obtain the noisy representation. By using a UNet network with frozen parameters, the model conditionally predicts noise based on the text embedding of the text prompt. The denoising direction is calculated accordingly. The diffusion model is a pre-trained model.

[0062] (6) The loss is calculated using the Score Distillation Sampling (SDS) or Variational Score Distillation (VSD) algorithm. Backward gradient propagation is performed, and the 3D shape parameters are optimized sequentially using the optimizer. The overall inference process is accelerated by the parallel acceleration framework of the strip nonlinear solution. After multiple iterations, the process converges to obtain the optimized 3D shape parameters.

[0063] In the embodiment, to guide the optimizer to update the 3D shape in a direction that "conforms to the textual semantics and is more likely under the 2D diffusion model distribution," and to reduce the diffusion model prediction noise... The ability to inversely apply to optimize 3D parameters. Through... Differentiate and use The gradient of the function, weighted by the diffusion time, is calculated as follows:

[0064]

[0065] in, It is a weighted function of diffusion time, i.e., weights, which control the importance of different training stages. This is a noise prediction from a pre-trained 2D diffusion model, where the input to the diffusion model is a noisy image. diffusion time step and text description The output is the predicted noise; It is real Gaussian noise sampled from the standard normal distribution. In the loss function, it is used as a reference signal and compared with the noise predicted by the diffusion model. It's a rendering function that takes 3D parameters. and camera parameters Convert it into a two-dimensional image. This is achieved through... Differentiation The optimizer will push the 3D shape to be updated in a direction that is "more likely to conform to the semantics of the text and under the 2D diffusion model distribution".

[0066] Similarly, the more sophisticated but time-consuming VSD algorithm can be used as an alternative. The VSD algorithm introduces a noise estimation network with learnable parameters. This replaces the static Gaussian noise term, enhancing the model's ability to fit the distribution of the rendered image. The VSD algorithm typically employs a LoRA (Low-Rank Adaptation) structure, enabling it to quickly adapt to the current scene distribution. The optimization gradient expression for VSD is as follows:

[0067]

[0068] in, It is a parameter-learnable noise estimation network, and the input is a noisy image. Time step Text description and viewpoint parameters The output is the estimated noise, which enhances the model's ability to fit the distribution of the rendered image.

[0069] The model returns SDS / VSD loss using reparameterization techniques. This loss drives the 3D shape parameter update of 3D Gaussian point rendering through a chain of differentiable rendering backpropagation, achieving end-to-end distillation from 2D diffusion prior to 3D geometry and appearance.

[0070] In traditional serial execution processes, the iterative optimization steps described above require sequential execution of all time steps. Traditional serial updates have a strict time dependency, meaning each step must wait for the previous step to complete before proceeding. Its main drawback is that the wall clock time and inference time steps increase linearly, making it difficult to utilize the parallel computing power of multiple GPUs / cores, thus limiting overall throughput. Therefore, this invention proposes to accelerate the generation process by introducing a strip-shaped nonlinear solution parallel acceleration framework based on Newton-Raphson. Specifically, the parallel solution framework breaks down the "rendering-diffusion model scoring-gradient evaluation" part into parallelizable computational units, simultaneously evaluating candidate updates for multiple time steps within a sliding window; subsequently, it performs sequential consistency correction (Newton-Simpson correction), coordinating momentum state and structural changes in one go within the window to generate a self-consistent 3D shape parameter trajectory. This strategy transforms the original O(T) time cost serial latency into a small number of iterations of O(K), significantly improving hardware utilization and convergence speed.

[0071] In the embodiments, such as Figure 4 As shown, this parallel acceleration framework for solving nonlinear equations in a strip shape uses the Newton-Simpson optimization method to construct the general term formula for the nonlinear equation system with local strip coupling at the time step, and uses three key mechanisms to ensure that the model can converge quickly and stably.

[0072] make The recursive form of the nonlinear equation system with local band coupling at this time step is expressed as:

[0073]

[0074] The system's Jacobian matrix possesses a lower-band sparse structure, where non-zero terms exist only in the main diagonal and its next band. The parallel acceleration framework uses Jacobian approximation or identity matrix substitution, significantly reducing inversion complexity. The iterative update form is:

[0075]

[0076] Rearranging the above equations, we derive the general term formula for the parallel acceleration framework:

[0077]

[0078] in, This represents an update function that includes gradients from the SDS or VSD algorithm. Indicates the first , The three-dimensional shape parameters of the time step. This represents the gradient of the SDS or VSD algorithm. This represents the derivative with respect to the three-dimensional shape parameters. Indicates the step size. Representing a nonlinear equation system, This represents the initial estimate of the parameters in the (k+1)th iteration. In the k-th iteration, at time step The approximate solution of the parameters, This represents the initial parameter vector for the iteration, i.e., the initial conditions. This indicates that at the k-th iteration, the update function is applied at the i-th time step. The calculated increment, Indicates a time step. Indicates a time step. This represents the approximate solution of parameters in the i-th iteration substep during the k-th iteration.

[0079] To ensure that the model can converge quickly and stably, the parallel acceleration framework should utilize three key mechanisms, including preconditioning during the startup phase, sliding window parallelism, and adaptive stopping criterion.

[0080] Mechanism 1 involves a preprocessing mechanism during the startup phase. This involves running a small number of single-threaded optimization steps or initializing the model using existing coarse estimates to generate a more stable sequence of initial parameters, thereby improving gradient stability and convergence speed during the main processes of SDS and VSD. This mechanism is particularly effective in VSD scenarios, mitigating the instability of the LoRA sub-model in the early stages of training.

[0081] Mechanism 2, sliding window parallel strategy, using a fixed-length window [ , , …, The sliding window parallel strategy computes each residual in parallel within the window, and then advances the window according to the updated state. This strategy achieves an efficient computational process that combines partial parallelism with local advancement, solving the memory bottleneck and computational redundancy caused by full-time parallelism.

[0082] Mechanism three, the adaptive stopping criterion, introduces a convergence judgment mechanism based on dynamic adjustment of window error. Specifically, for each time step within the sliding window... Calculate the norm difference (e.g., L2 distance) between the parameters before and after the update, and compare it with a threshold value. Compare the results. If all time steps within the window satisfy:

[0083]

[0084] This indicates that the current window has converged and can be swiped forward. Here, This represents the time step within the window during the (k+1)th iteration. The parameters, Represents the time step within the window in the k-th iteration. 3D shape parameters.

[0085] To accommodate different inputs and variations in model performance, this threshold... Without the need for fixed settings, dynamic adjustments can be made using the moving average (mean or median) of the error within the window. For example, the update step size can be controlled by the exponential moving average (EMA), thereby achieving automatic adjustment of the convergence tolerance.

[0086] In the later stages of training, the parallel architecture gradually exhibits stable convergence characteristics. With multiple parallel steps, the obtained gradient signals continuously optimize the 3D shape parameters, and the parameter trajectory gradually approaches the fixed-point solution. Specifically, through continuous iteration of the "parallel computation—sequential correction—error evaluation" process, the entire trajectory stabilizes within a finite number of iterations. At this point, the error evaluation result tends to be below the threshold, and the rendering result remains stable across all viewpoints, indicating that the entire parallel optimization process has converged to the required fixed-point solution. The 3D shape parameters then infinitely approach the ideal solution that best fits the semantics. The final optimized fixed-dimensional 3D shape parameters are output, thus generating the 3D model.

[0087] This method effectively improves generation efficiency and accuracy, solves the problems of slow training and poor stability of traditional methods, and has good scalability and engineering application value.

[0088] The Newton-Simpson-based textual 3D rapid generation method provided in the above embodiments achieves a generation process with stronger expressive power, faster generation speed and higher scalability.

[0089] Based on the same inventive concept, the embodiments also provide a Newton-Simpson-based textural 3D rapid generation system, such as... Figure 5 As shown, the system consists of five top-level modules: a 3D representation module, a rendering module, a diffusion prior module, an SDS / VSD loss head module, and a parallel acceleration solution module based on the Newton-Simpson algorithm (containing three sub-mechanisms: preprocessing, sliding window, and adaptive stopping). Each module has a clear responsibility and stable interfaces, progressing forward according to the data flow and closing the loop backward according to the gradient flow.

[0090] The 3D representation module is used to obtain the 3D shape parameters and their corresponding text prompts represented by the initialized 3D Gaussian point rendering method; to unify the dimensions of the 3D shape parameters through a projection algorithm to generate fixed-dimensional 3D shape parameters; and to maintain the learnable 3D parameters. In this embodiment, the implementation supports variable-dimensional representation of 3D Gaussian points and provides projection / re-projection submodules to maintain the parameter dimensions consistent with the optimizer state during structural changes such as splitting / clipping.

[0091] The rendering module is used to convert the image into a 2D image with a fixed viewpoint through the rendering function. It takes (θ, c) as input (c is the camera viewpoint / parameter), serves both the forward evaluation (generating a 2D image for scoring) and the Jacobian transfer of the reverse link, and acts as a bridge between 3D and 2D priors.

[0092] The diffusion prior module uses a pre-trained diffusion model, taking a 2D image, text prompts, and fixed viewpoint parameters as inputs, and iteratively optimizes the fixed-dimensional 3D shape parameters based on the gradient of the loss function. In this embodiment, it includes text encoding, VAE encoder, U-Net denoising network, etc., and also includes a lightweight LoRA submodule in the VSD scenario.

[0093] The SDS / VSD loss head module is used to construct a loss function by combining the score distillation sampling (SDS) or variational score distillation (VSD) algorithms, transforming the output of the diffusion prior into a gradient available for the three-dimensional parameters. The responsibility of this module is to accurately map the 2D prior signal back to the 3D parameter space.

[0094] The parallel acceleration solution module, based on the Newton-Simpson optimization method, constructs the general term formula for a time-step locally coupled nonlinear equation system to accelerate the training speed and convergence stability of the diffusion model. To improve efficiency and optimize the process, three sub-mechanisms are added: a preprocessing mechanism to obtain a certain amount of better initialization to significantly improve the gradient stability and early convergence of SDS / VSD; a sliding window parallel strategy that uses a fixed-length window to evaluate the residuals of each time step in parallel, advancing the window sequentially with the update state to achieve "partial parallelism + local advancement," avoiding the memory bottleneck and computational redundancy caused by full-time parallelism while maintaining throughput; and an adaptive stopping criterion that uses the norm difference of parameter updates within the window as the convergence criterion (convergence is judged when the update magnitude of each step is lower than the threshold), and dynamically adjusts the threshold through moving / exponential moving averages to adapt to the differences in different inputs and models, stably and efficiently controlling the window advancement.

[0095] It should be noted that the Newton-Simpson-based textual 3D rapid generation system provided in the above embodiments should be illustrated using the above-described functional module divisions when performing Newton-Simpson-based textual 3D rapid generation. The above functions can be assigned to different functional modules as needed. In addition, the Newton-Simpson-based textual 3D rapid generation system and the Newton-Simpson-based textual 3D rapid generation method embodiments are based on the same concept. For details of its implementation process, please refer to the Newton-Simpson-based textual 3D rapid generation embodiments, which will not be repeated here.

[0096] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0097] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A rapid 3D generation method based on Newton-Simpson's method, characterized in that, Includes the following steps: (1) Obtain the 3D shape parameters and their corresponding text prompts represented by the initialized 3D Gaussian point rendering method; (2) The dimensions of the 3D shape parameters are unified by the projection algorithm to generate 3D shape parameters with fixed dimensions, and then converted into a two-dimensional image with a fixed viewpoint by the rendering function; (3) Input the two-dimensional image, text prompt and fixed viewpoint parameters into the pre-trained diffusion model, combine the score distillation sampling SDS or variational score distillation VSD algorithm to construct the loss function, optimize the fixed-dimensional 3D shape parameters according to the gradient of the loss function, and establish a parallel acceleration framework for strip nonlinear solution based on the Newton-Simpson optimization method to optimize the training process. (4) Output the final optimized fixed-dimensional 3D shape parameters to generate the 3D model.

2. The rapid textural generation method for 3D graphics based on Newton-Simpson's method according to claim 1, characterized in that, The projection algorithm in step (2) achieves dimensionality compression by deleting a subset of the Gaussian point set.

3. The rapid textural generation method for 3D models based on Newton-Simpson's method according to claim 1, characterized in that, The rendering function in step (2) uses fixed viewpoint parameters to project fixed-dimensional 3D shape parameters to generate a two-dimensional image with two-dimensional feature representation.

4. The rapid textural generation method for 3D models based on Newton-Simpson's method according to claim 1, characterized in that, A parallel acceleration framework for solving strip-shaped nonlinear problems based on the Newton-Simpson optimization method is established, including: A nonlinear equation system with localized band coupling at time steps is established, and the recursive form of the equation is expressed as: ； ； The general formula for iterative solution is: ； in, This represents an update function that includes gradients from the SDS or VSD algorithm. Indicates the first , The three-dimensional shape parameters of the time step. This represents the gradient of the SDS or VSD algorithm. This represents the derivative with respect to the three-dimensional shape parameters. Indicates the step size. Representing a nonlinear equation system, This represents the initial estimate of the parameters in the (k+1)th iteration. This indicates that in the k-th iteration, at time step... The approximate solution of the parameters, This represents the initial parameter vector for the iteration, i.e., the initial conditions. This indicates that at the k-th iteration, the update function is applied at the j-th time step. The calculated increment, Indicates a time step. Indicates a time step. This represents the approximate solution of the parameters at the j-th time step in the k-th iteration.

5. The rapid textural generation method for 3D models based on Newton-Simpson's algorithm according to claim 1 or 4, characterized in that, The parallel acceleration framework also includes: The preprocessing mechanism in the startup phase generates initial values ​​of 3D shape parameters for several time steps based on the fixed-dimensional 3D shape parameters in step (2) through single-threaded optimization or existing coarse estimation. The sliding window parallel strategy, within a fixed-length window [ , , …, The residuals of the 3D shape parameters between two adjacent iterations are calculated in parallel within the [internal] module, and then the window is advanced according to the updated parameters. Indicates the first in the window Time step parameters, It is a parameter related to window size; Adaptive stopping criteria, for time steps within a sliding window The norm difference before and after parameter update is dynamically determined to converge. If convergence is achieved, the window is advanced. The threshold value used to determine the norm difference is dynamically adjusted using the moving average estimate of the error within the window.

6. The rapid textural generation method for 3D models based on Newton-Simpson's method according to claim 5, characterized in that, The convergence criterion for the adaptive stopping criterion is: ； in, The parameter dimension representing the 3D shape parameter. Indicates the threshold value. This represents the time step within the window during the (k+1)th iteration. 3D shape parameters, Represents the time step within the window in the k-th iteration. 3D shape parameters.

7. The method for rapid generation of textual 3D based on Newton-Simpson's algorithm according to claim 5, characterized in that, In the parallel acceleration framework based on Newton-Simpson, when the sliding window reaches the last time step and converges, the update result of the last time step under the latest iteration is output as the final optimized fixed-dimensional 3D shape parameter.

8. The rapid generation method for textual 3D based on Newton-Simpson's method according to claim 4, characterized in that, The formula for calculating the gradient of the SDS algorithm is: ； in, Represents the gradient of the SDS algorithm. Expressing expectations, This represents a weighting function with respect to diffusion time. Represents the predicted two-dimensional noise. Indicates time step The noisy image at that location, Indicates a time step. Indicates a text description. This represents Gaussian random noise. Indicates the viewpoint parameter. This represents the rendering function.

9. The rapid textural generation method for 3D models based on Newton-Simpson's method according to claim 4, characterized in that, The formula for calculating the gradient of the VSD algorithm is: ； in, This represents the gradient of the VSD algorithm. Expressing expectations, This represents a weighting function with respect to diffusion time. Represents the predicted two-dimensional noise. Indicates time step The noisy image at that location, Indicates a time step. Indicates a text description. This represents Gaussian random noise. This represents a learnable noise estimation network. Indicates the viewpoint parameter. This represents the rendering function.

10. A rapid textural 3D generation system based on Newton-Simpson's method, used to implement the textural 3D generation method of claim 1, characterized in that, The system includes: The 3D representation module is used to obtain the 3D shape parameters represented by the initialized 3D Gaussian point rendering method and their corresponding text prompts; and to unify the dimensions of the 3D shape parameters through a projection algorithm to generate 3D shape parameters with fixed dimensions. The rendering module is used to convert images into a two-dimensional image with a fixed viewpoint through rendering functions; The diffusion prior module uses a pre-trained diffusion model, taking a 2D image, text prompts, and fixed viewpoint parameters as input, and iteratively optimizes the fixed-dimensional 3D shape parameters based on the gradient of the loss function. The SDS / VSD loss head module is used to construct a loss function by combining the scoring distillation sampling (SDS) or variational scoring distillation (VSD) algorithms. The parallel acceleration solution module is used to establish a parallel acceleration framework for strip nonlinear solution based on the Newton-Simpson optimization method to optimize the training process; The output module is used to output the final optimized fixed-dimensional 3D shape parameters to generate the 3D model.

Images