Geometric perception ADMM-based training-free text-based three-dimensional digital model generation method, system, terminal and storage medium

Abstract

This invention relates to the field of image processing technology and discloses a method, system, terminal, and storage medium for generating 3D digital models of text without training based on geometry-aware ADMM. The method includes: in the latent space, ADMM-style topological constraints regularize the global structure to alleviate polyhedral geometry problems; in the normal vector space, constraints based on denoising regularization suppress high-frequency surface noise while preserving semantic details. Both types of constraints rely on lightweight classical operators, avoiding the computational overhead of 3D diffusion models. This invention can be seamlessly integrated into various SDS pipelines without retraining the diffusion model, significantly improving the geometric integrity and surface quality of the generated 3D digital model while maintaining semantic fidelity.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system, terminal, and computer-readable storage medium for generating three-dimensional digital models of text without training based on geometrically perceptual ADMM. Background Technology

[0002] Text-driven 3D content generation is a key technology in gaming, virtual reality, and digital content creation, used to generate 3D digital models with length, width, and height. In recent years, methods based on pre-trained text-to-image diffusion models have leveraged strong visual priors to synthesize 3D objects using natural language without relying on large-scale 3D datasets. The core enabling technology is Score Distillation Sampling (SDS), which transfers knowledge from 2D diffusion models to 3D representations through differentiable rendering, significantly lowering the barrier to entry for 3D content creation.

[0003] However, SDS-based optimization methods still have problems. The 2D diffusion model only provides appearance supervision signals per viewpoint and lacks the ability to perceive the global 3D topology, which may cause the optimization process to converge to degenerate solutions. Furthermore, due to the randomness of the SDS gradient (caused by noise injection and fraction estimation), high-frequency geometric components are overemphasized, resulting in rough and physically unreasonable surface textures.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, terminal, and computer-readable storage medium for generating 3D digital models from training-free text based on geometry-aware ADMM. This aims to solve the problem that in the prior art, the generated 3D models exhibit inconsistent or repetitive surface texture features under different viewpoints, resulting in frequent roughness and pitting on the geometric surface of the 3D models.

[0006] To achieve the above objectives, this invention provides a method for generating 3D digital models from training-free text based on geometry-aware ADMM. The method includes the following steps:

[0007] A low-level 3D representation is constructed based on the user's text prompts for the target object. The low-level 3D representation is parameterized to obtain multiple parameters. Based on all the parameters, the low-level 3D representation is rendered into a 2D image and a depth image.

[0008] The diffusion latent variables of the 2D image are extracted using a variational autoencoder, and the diffusion latent variables are downsampled at multiple scales to obtain feature maps of different resolutions. An SDS gradient is then constructed based on the diffusion latent variables.

[0009] The pseudo normal of the depth image is calculated, and the pseudo normal is smoothed using a bilateral filter to obtain the target normal. The surface regularization gradient is then calculated based on the target normal.

[0010] Obtain the scaling dual variable, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variable and the surface regularization gradient, obtain the test text prompt of the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target 3D digital model of the target object or the new target object.

[0011] Optionally, the method for generating a training-free 3D digital model of text based on geometry-aware ADMM, wherein the step of constructing a low-level 3D representation based on user text prompts for the target object, parameterizing the low-level 3D representation to obtain multiple parameters, and rendering the low-level 3D representation into a 2D image and a depth image based on all parameters, specifically includes:

[0012] Obtain text prompts input by the user, wherein the text prompts represent textual descriptions of the target object;

[0013] Assign values ​​to multiple 3D Gaussian primitives according to the text prompts, and obtain the position parameters, scaling parameters, rotation quaternions, opacity and color parameters of each 3D Gaussian primitive;

[0014] Based on all parameters assigned to all 3D Gaussian primitives, construct the underlying 3D representation of the target object;

[0015] Render all the 3D Gaussian elements in the underlying 3D representation to obtain the 2D image and depth image of the target object.

[0016] Optionally, the method for generating a 3D digital model of text without training based on geometry-aware ADMM, wherein rendering all the 3D Gaussian primitives in the underlying 3D representation to obtain a 2D image and a depth image of the target object specifically includes:

[0017] Calculate the residual sum of squares based on the scaling parameter and the rotation quaternion for each of the 3D Gaussian elements, and calculate the covariance matrix based on the residual sum of squares:

[0018] ;

[0019] in, Let R denote the covariance matrix, R denote the rotation matrix derived from the rotation quaternion, S denote the scaling diagonal matrix composed of scaling parameters, and T denote the transpose.

[0020] Calculate the density at each of the 3D Gaussian elements based on the covariance matrix:

[0021] ;

[0022] in, This represents the density of p. Indicates an exponential mapping. This represents the position parameter, where p represents the 3D Hexagonal element;

[0023] For each of the 3D Gaussian primitives, corresponding scene parameters are constructed based on the position parameter, the scaling parameter, the rotation quaternion, the opacity, and the color parameter:

[0024] ;

[0025] in, Indicates scene parameters, This represents the position parameter of the i-th 3D Gaussian element. This represents the scaling parameter of the i-th 3D Gaussian element. Denotes the rotation quaternion of the i-th 3D Gaussian element. Let represent the opacity of the i-th 3D Gaussian element. Let N represent the camera viewpoint of the i-th 3D Gaussian element, and N represent the total number of 3D Gaussian elements.

[0026] Obtain the camera viewpoint that samples the target object, concatenate the scene parameters into a single vector, and construct the 2D image and the depth image based on the single vector and the camera viewpoint.

[0027] Optionally, the method for generating a 3D digital model of text without training based on geometrically perceptual ADMM, wherein the step of extracting the diffusion latent variables of the 2D image using a variational autoencoder, performing multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and constructing an SDS gradient based on the diffusion latent variables specifically includes:

[0028] The 2D image is input into a variational autoencoder for feature extraction, and the output is a diffusion latent variable:

[0029] ;

[0030] Where f represents the diffusion latent variable, Let x represent a 2D image, c represent a single vector, and c represent the camera viewpoint from which the target object is sampled. Indicates to Feature extraction is performed, where R represents the rotation matrix derived from the rotation quaternion;

[0031] During the diffusion process, Gaussian random noise is added to the latent variables corresponding to the 2D image to obtain auxiliary variables;

[0032] Based on the camera viewpoint, the SDS gradient is constructed according to the auxiliary variables at each time step:

[0033] ;

[0034] in, Indicates the direction of the update for x. Represents the SDS gradient. Let represent the expectation with respect to time step t, Gaussian random noise, and camera viewpoint c. This represents the weighting function at diffusion time step t. Let y represent the auxiliary variable at diffusion time step t, and let y represent the text prompt. Represents a diffusion model. Indicates to Differentiate, This represents Gaussian random noise;

[0035] In the ADMM framework, auxiliary variables and scaling dual variables are introduced. The diffusion latent variable is added to the scaling dual variable and then projected through a plug-and-play geometric denoising device to obtain the updated auxiliary variable.

[0036] In the ADMM framework, the states of the auxiliary variables and the scaling dual variables at different resolution scales are kept independent and iteratively updated.

[0037] Optionally, the method for generating a 3D digital model from training-free text based on geometry-aware ADMM, wherein calculating the pseudo-normals of the depth image, smoothing the pseudo-normals using a bilateral filter to obtain the target normals, and calculating the surface regularization gradient based on the target normals, specifically includes:

[0038] The pseudo-normals of the depth image are calculated using the Sobel operator:

[0039] ;

[0040] Where n represents the pseudonormal. This indicates the calculation of pseudonormals. This represents the Sobel operator in the horizontal direction. This represents the Sobel operator in the vertical direction, where D represents the rendering depth of the depth image;

[0041] The pseudo-normal is smoothed using a bilateral filter to obtain the target normal, and the cosine distance loss between the target normal and the pseudo-normal is calculated.

[0042] ;

[0043] in, This represents the cosine distance loss, where p represents the 3D Gaussian element. This represents the pseudo-normal at point p. Indicates the target normal. Represents the cosine value;

[0044] The surface regularization gradient is obtained by differentiating the cosine distance loss.

[0045] Optionally, the method for generating a training-free 3D digital model of text based on geometry-aware ADMM, wherein the steps of obtaining the scaling dual variable, optimizing the constructed ADMM framework according to the SDS gradient, the scaling dual variable, and the surface regularization gradient, obtaining the test text prompt of the target object or a new target object, and analyzing the test text prompt using the optimized ADMM framework to obtain the target 3D digital model of the target object or the new target object, specifically include:

[0046] The process of constructing the underlying 3D representation is equivalent to a geometrically constrained optimization process:

[0047] ;

[0048] ;

[0049] in, This represents finding the minimum value for a single vector x. This represents a text alignment function. Represents the balance parameters. The geometric manifold projection representing a single vector. represents a 3D geometric manifold, s represents the scaling parameter, and t represents the time step;

[0050] The optimization process is decomposed into multiple sub-problems, and an augmented Lagrangian function is constructed based on the auxiliary variables and the obtained scaling dual variables:

[0051] ;

[0052] in, This represents the augmented Lagrange function. This represents the force of the geometric constraints, where z represents the auxiliary variable. Indicates scaling dual variables;

[0053] Iterative updates are performed on the single vector, auxiliary variables, and scaled dual variables based on the augmented Lagrangian function;

[0054] Based on the updated single vector, auxiliary variables, and scaling dual variables, a total gradient function is constructed according to the SDS gradient, the scaling dual variables, and the surface regularization gradient. The ADMM framework is then optimized using this total gradient function.

[0055] ;

[0056] in, Represents the total gradient function. For SDS gradient, For surface regularization gradient, Represents the gradient of geometric consistency constraints;

[0057] The optimized ADMM framework is used to analyze the description information of the target object to obtain the target 3D model of the target object to be tested.

[0058] Optionally, the method for generating a training-free 3D digital model of text based on geometry-aware ADMM, wherein the iterative update of the single vector, auxiliary variable, and scaling dual variable based on the augmented Lagrangian function specifically includes:

[0059] Gradient descent is applied to the single vector based on the augmented Lagrangian function to obtain the single vector after the current iteration:

[0060] ;

[0061] in, Represents a single vector in the (k+1)th iteration;

[0062] The auxiliary variable is then subjected to plug-and-play geometric projection and updated using a momentum mechanism to obtain the auxiliary variable after the current iteration:

[0063] ;

[0064] in, Describes the auxiliary variable for the (k+1)th iteration. Indicates an indicator function, This represents the geometric manifold projection of a single vector in the (k+1)th iteration. Describe the geometric manifold of z;

[0065] The scaling dual variable is updated to obtain the scaling dual variable after the current iteration:

[0066] ;

[0067] in, Denotes the scaling dual variable for the (k+1)th iteration. Let represent the scaling dual variable for the k-th iteration.

[0068] Furthermore, to achieve the above objectives, the present invention also provides a training-free 3D digital model generation system for text based on geometry-aware ADMM, wherein the training-free 3D digital model generation system for text based on geometry-aware ADMM includes:

[0069] The image rendering module is used to construct a low-level 3D representation based on the user's text prompts for the target object, parameterize the low-level 3D representation to obtain multiple parameters, and render the low-level 3D representation into a 2D image and a depth image based on all parameters.

[0070] The noise addition module is used to extract the diffusion latent variables of the 2D image using a variational autoencoder, perform multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and construct the SDS gradient based on the diffusion latent variables.

[0071] The geometric constraint module is used to calculate the pseudo normals of the depth image, smooth the pseudo normals using a bilateral filter to obtain the target normals, and calculate the surface regularization gradient based on the target normals.

[0072] An optimization generation module is used to obtain scaling dual variables, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variables, and the surface regularization gradient, obtain the test text prompt of the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target 3D digital model of the target object or the new target object.

[0073] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a three-dimensional digital model generation program for training-free text based on geometry-aware ADMM stored in the memory and executable on the processor. When the three-dimensional digital model generation program for training-free text based on geometry-aware ADMM is executed by the processor, it implements the steps of the three-dimensional digital model generation method for training-free text based on geometry-aware ADMM as described above.

[0074] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a three-dimensional digital model generation program for training-free text based on geometry-aware ADMM, and when the three-dimensional digital model generation program for training-free text based on geometry-aware ADMM is executed by a processor, it implements the steps of the three-dimensional digital model generation method for training-free text based on geometry-aware ADMM as described above.

[0075] In this invention, a low-level 3D representation is constructed based on the user's text prompts for the target object. This low-level 3D representation is parameterized to obtain multiple parameters, and then rendered into a 2D image and a depth image based on all parameters. A variational autoencoder (VAE) is used to extract the diffusion latent variables from the 2D image. These latent variables are then downsampled at multiple scales to obtain feature maps at different resolutions. An SDS gradient is constructed based on the diffusion latent variables, and the auxiliary variables and scaling dual variables introduced into the ADMM framework are iteratively updated. The pseudo-normals of the depth image are calculated, and the pseudo-normals are smoothed using a bilateral filter to obtain the target normals. The surface regularization gradient is then calculated based on the target normals. A total gradient function is constructed based on the SDS gradient, the scaling dual variables, and the surface regularization gradient. After optimizing the ADMM framework using the total gradient function, the text prompts to be tested are analyzed to obtain the target 3D model of the target object. This invention can be seamlessly integrated into multiple SDS pipelines without retraining the diffusion model, significantly improving the geometric integrity and surface quality of the generated 3D model while maintaining semantic fidelity. Attached Figure Description

[0076] Figure 1 This is a flowchart of a preferred embodiment of the method for generating 3D digital models of text without training based on geometric perception ADMM of the present invention;

[0077] Figure 2 This is a flowchart of the Geo-ADMM dual-space framework of a preferred embodiment of the method for generating three-dimensional digital models of text without training based on geometry-aware ADMM of the present invention.

[0078] Figure 3 These are comparison images of 3D generation from a preferred embodiment of the method for generating 3D digital models of text without training based on geometrically perceptual ADMM of the present invention.

[0079] Figure 4 These are qualitative comparison and ablation experiment diagrams of a preferred embodiment of the method for generating three-dimensional digital models of text without training based on geometric perception ADMM of the present invention.

[0080] Figure 5This is a first gradient decomposition diagram of a preferred embodiment of the method for generating three-dimensional digital models of text without training based on geometrically perceptual ADMM of the present invention.

[0081] Figure 6 This is the second gradient decomposition diagram of a preferred embodiment of the method for generating three-dimensional digital models of text without training based on geometrically perceptual ADMM of the present invention.

[0082] Figure 7 This is a structural diagram of a preferred embodiment of the three-dimensional digital model generation system for training-free text based on geometrically perceptual ADMM of the present invention.

[0083] Figure 8 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0085] Text-driven 3D content generation has rapidly become a key technology in the fields of gaming, virtual reality, and digital content creation. However, existing SDS-based optimization methods suffer from two persistent geometric pathologies. First, multifaceted structures independently satisfy appearance constraints from each viewpoint, even with lower loss than a single coherent face compromised across multiple views. This results in generated 3D objects exhibiting inconsistent or repetitive facial features across different viewpoints. Second, due to the randomness of SDS gradients (caused by noise injection and fraction estimation), high-frequency geometric components are overemphasized, leading to rough and physically illogical surface textures.

[0086] To address these issues, this invention discloses a training-free text-to-3D generation framework based on the Geo-ADMM (Geographical Alternating Direction Method of Multipliers). This framework fundamentally models text-to-3D generation as a geometrically constrained optimization problem and solves it using ADMM combined with plug-and-play priors.

[0087] Polyhedral artifacts and surface noise both stem from the same fundamental cause: SDS optimization is essentially an ill-conditioned, under-constrained inverse problem, where the 2D diffusion prior only provides view-by-view appearance supervision and is unaware of global 3D topology and surface continuity. Based on this diagnosis, this invention does not modify the diffusion model or redesign the distillation loss, but instead directly applies geometric constraints during the optimization process.

[0088] The preferred embodiment of the present invention describes a method for generating 3D digital models of text without training based on geometry-aware ADMM. Figure 1 As shown, the method for generating a 3D digital model of text without training based on geometry-aware ADMM includes the following steps:

[0089] Step S10: Construct a low-level 3D representation based on the user's text prompts for the target object, parameterize the low-level 3D representation to obtain multiple parameters, and render the low-level 3D representation into a 2D image and a depth image based on all parameters.

[0090] Specifically, the text prompts input by the user are obtained, wherein the text prompts represent a textual description of the target object;

[0091] Assign values ​​to multiple 3D Gaussian primitives according to the text prompts, and obtain the position parameters, scaling parameters, rotation quaternions, opacity and color parameters of each 3D Gaussian primitive;

[0092] Based on all parameters assigned to all 3D Gaussian primitives, construct the underlying 3D representation of the target object;

[0093] Render all the 3D Gaussian elements in the underlying 3D representation to obtain the 2D image and depth image of the target object.

[0094] Among them, such as Figure 2 As shown (with the input text "A golden retriever dog, 8K, HDR"), in the embodiments disclosed in this invention, 3D Gaussian Splatting (3DGS) is used as the underlying 3D representation. 3DGS represents the scene using a set of anisotropic 3D Gaussian primitives. Each Gaussian primitive consists of position parameters, scaling parameters, rotation quaternions, opacity, and color parameters. Based on these parameters, initial 3D scene parameters are constructed.

[0095] Among them, the color parameter is a color that is only related to the viewpoint, that is, the camera viewpoint used; starting from the randomly selected camera viewpoint, the current 3D Gaussian primitives are rendered into a 2D image through a differentiable rasterizer, and then a depth image can be rendered simultaneously for the 2D image.

[0096] Explicit parameterization of 3D representations (such as position, covariance, color, opacity, etc.) not only facilitates subsequent optimization and fine-tuning, but also supports editing operations such as cloning, deleting, and repositioning objects, avoiding the shortcomings of implicit methods that are difficult to modify. The parameterized structure ensures the consistency of multi-view rendering and reduces artifacts and floating effects.

[0097] Further, the residual sum of squares is calculated based on the scaling parameter and the rotation quaternion for each of the 3D Gaussian elements, and the covariance matrix is ​​calculated based on the residual sum of squares:

[0098] ;

[0099] in, Let R denote the covariance matrix, R denote the rotation matrix derived from the rotation quaternion, S denote the scaling diagonal matrix composed of scaling parameters, and T denote the transpose.

[0100] Calculate the density at each of the 3D Gaussian elements based on the covariance matrix:

[0101] ;

[0102] in, This represents the density of p. Indicates an exponential mapping. This represents the position parameter, where p represents the 3D Hexagonal element;

[0103] For each of the 3D Gaussian primitives, corresponding scene parameters are constructed based on the position parameter, the scaling parameter, the rotation quaternion, the opacity, and the color parameter:

[0104] ;

[0105] in, Indicates scene parameters, This represents the position parameter of the i-th 3D Gaussian element. This represents the scaling parameter of the i-th 3D Gaussian element. Denotes the rotation quaternion of the i-th 3D Gaussian element. Let represent the opacity of the i-th 3D Gaussian element. Let N represent the camera viewpoint of the i-th 3D Gaussian element, and N represent the total number of 3D Gaussian elements.

[0106] Obtain the camera viewpoint that samples the target object, concatenate the scene parameters into a single vector, and construct the 2D image and the depth image based on the single vector and the camera viewpoint.

[0107] In order to ensure positive semi-qualitative optimization of the ADMM framework, the covariance matrix can be constructed using parameterized forms of scaling parameters and rotation quaternions, and the density of different spatial points can be defined using the covariance matrix. At this time, for each 3D Gaussian element, information such as its unknown parameters, scaling parameters, rotation quaternions, opacity and color parameters are obtained, so that the corresponding scene parameters can be constructed.

[0108] Among them, the parameterized 3D representation is uniformly rendered as 2D images and depth images, providing complete perceptual input for downstream tasks; the depth map can be used for spatial understanding and obstacle avoidance, and the 2D image is used for visual presentation. The combination of the two achieves "one building for multiple uses", improving data utilization and system integration efficiency.

[0109] Step S20: Extract the diffusion latent variables of the 2D image using a variational autoencoder, perform multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and construct the SDS gradient based on the diffusion latent variables.

[0110] Specifically, the 2D image is input into a variational autoencoder for feature extraction, and the output is a diffusion latent variable:

[0111] ;

[0112] Where f represents the diffusion latent variable, Let x represent a 2D image, c represent a single vector, and c represent the camera viewpoint from which the target object is sampled. Indicates to Feature extraction is performed, where R represents the rotation matrix derived from the rotation quaternion;

[0113] During the diffusion process, Gaussian random noise is added to the latent variables corresponding to the 2D image to obtain auxiliary variables;

[0114] Based on the camera viewpoint, the SDS gradient is constructed according to the auxiliary variables at each time step:

[0115] ;

[0116] in, Indicates the direction of the update for x. Represents the SDS gradient. Let represent the expectation with respect to time step t, Gaussian random noise, and camera viewpoint c. This represents the weighting function at diffusion time step t. Let y represent the auxiliary variable at diffusion time step t, and let y represent the text prompt. Represents a diffusion model. Indicates to Differentiate, This represents Gaussian random noise;

[0117] In the ADMM framework, auxiliary variables and scaling dual variables are introduced. The diffusion latent variable is added to the scaling dual variable and then projected through a plug-and-play geometric denoising device to obtain the updated auxiliary variable.

[0118] In the ADMM framework, the states of the auxiliary variables and the scaling dual variables at different resolution scales are kept independent and iteratively updated.

[0119] In the case of feature extraction in the manifold latent space, given an image rendered from the perspective, a variational autoencoder is used to extract diffusion latent variables. While preserving the semantic structure, it can also attenuate high-frequency textures, making it suitable for topology-aware regularization.

[0120] Furthermore, using deep guidance, joint bilateral filter instantiation is performed:

[0121] ;

[0122] in, D Let h denote a bilateral filter, h denote the latent feature map, and p and q denote 3D Gaussian elements. Represents the Gaussian kernel. Represents a local neighborhood. This represents the normalized weight.

[0123] In this invention, depth guidance preserves discontinuities at geometric boundaries while smoothing homogeneous regions. To better suppress coarse, multi-faceted artifacts, a multi-scale scheme is employed in the disclosed embodiments, where 's' takes values ​​of 64, 32, and 16, each with independent ADMM states. This increases the enforcement of geometric constraints at lower resolutions, thereby imposing stronger low-frequency topological constraints. The design of maintaining independent ADMM states at the {64, 32, 16} scales and imposing stronger constraints at lower resolutions effectively suppresses low-frequency global topological errors (such as Janus artifacts). Strong constraints at scale effectively suppress detail loss, while weak constraints at high-resolution scales do not excessively restrict detail preservation.

[0124] Furthermore, since large-scale paired text-3D data is limited, in the embodiments disclosed in this invention, random noise is added to the latent space features to obtain auxiliary variables, and then a pre-trained 2D diffusion model is used as a priori optimization to render the image under the given camera viewpoint and construct its SDS gradient, which provides a supervisory signal for semantic alignment.

[0125] The encoder maps 2D images into continuous, structured diffusion latent variables, effectively compressing redundant pixel information and preserving high-level semantic features. Compared to operating directly in the pixel space, the latent space has a lower dimension and a more regular distribution (approximately normal), which greatly reduces the learning difficulty of the noise prediction network and improves the convergence speed and stability of the diffusion process.

[0126] Furthermore, multi-scale downsampling of latent variables generates feature maps of different resolutions (such as 8×8, 16×16, 32×32, etc.) to construct a pyramid-shaped representation structure. This allows the model to quickly optimize the overall layout at a coarse-grained level, while refining local textures and edge details at a fine-grained level. This strategy is widely used in accelerating sampling and super-resolution tasks and also provides rich spatial hierarchical information for subsequent gradient calculations.

[0127] Finally, by constructing gradients through multi-scale latent representations, the text guidance signal can be backpropagated to the input image or latent variable space, enabling tasks such as "text-driven editing" or "image style transfer." Since SDS utilizes the score function of the diffusion model as a supervision signal, it can achieve high-quality semantic consistency optimization without requiring real labeled data.

[0128] Step S30: Calculate the pseudo normal of the depth image, smooth the pseudo normal using a bilateral filter to obtain the target normal, and calculate the surface regularization gradient based on the target normal.

[0129] Specifically, by using a rendered depth map, pseudo-normals are calculated using the Sobel operator, and then the normals are smoothed using a bilateral filter to obtain the target normals. The cosine distance loss of the normals and its surface regularization gradient are calculated to suppress high-frequency noise on the surface.

[0130] Specifically, the pseudo-normals of the depth image are calculated using the Sobel operator:

[0131] ;

[0132] Where n represents the pseudonormal. This indicates the calculation of pseudonormals. This represents the Sobel operator in the horizontal direction. This represents the Sobel operator in the vertical direction, where D represents the rendering depth of the depth image;

[0133] The pseudo-normal is smoothed using a bilateral filter to obtain the target normal, and the cosine distance loss between the target normal and the pseudo-normal is calculated.

[0134] ;

[0135] in, This represents the cosine distance loss, where p represents the 3D Gaussian element. This represents the pseudo-normal at point p. Indicates the target normal. Represents the cosine value;

[0136] The surface regularization gradient is obtained by differentiating the cosine distance loss.

[0137] The cosine distance loss is used to penalize deviations from the smooth normal field while preserving meaningful ensemble details. By using the Sobel operator to differentiate the depth image in the x and y directions, the depth change rate of each pixel can be quickly estimated, thus synthesizing a pseudo-normal map. Since the depth map itself is single-channel geometric data, the Sobel operator, based on first-order gradients, can effectively capture abrupt surface changes (such as object edges and depressions), providing initial geometric guidance for subsequent normal optimization.

[0138] Furthermore, pseudo-normal maps are often perturbed by noise or uneven sampling in the depth map. Introducing a bilateral filter for smoothing simultaneously considers both spatial proximity between pixels and normal value similarity, achieving the effect of "smoothing noise while preserving realistic edges." In regions with drastic geometric changes (such as object contours), where normal differences are large, the value weights automatically reduce the influence of dissimilar neighbors, preventing edge blurring. In flat regions, the filtering effect approximates Gaussian smoothing, effectively suppressing high-frequency jitter. The final output target normal is more stable and better conforms to the real surface structure, providing a reliable geometric prior for subsequent regularization.

[0139] Finally, by calculating the gradient through the target normal, a geometry-aware regularization term can be constructed, constraining the optimization process towards convergence to a solution with "smooth surfaces and continuous normals". For example, in 3DGS training, this gradient can be used to penalize unnatural protrusions or holes in the arrangement of Gaussian spheres, improving the realism and physical plausibility of the rendering results.

[0140] Step S40: Obtain the scaling dual variable, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variable and the surface regularization gradient, obtain the text prompt to be tested for the target object or a new target object, analyze the text prompt to be tested using the optimized ADMM framework, and obtain the target three-dimensional digital model of the target object or the new target object.

[0141] This involves updating the dual variable at each scale, accumulating constraint violation residuals, and then calculating the SDS gradient (i.e., ...). Figure 2 In ), surface regularization gradient (Right now Figure 2 The RED residual term in the data. ,in Indicates the prediction result. (representing the target result), ADMM penalized gradient (i.e.) Figure 2 In The three are added together to obtain the total gradient, and the Adam optimizer is used to perform a gradient descent update on the scene parameters.

[0142] Specifically, the process of constructing the underlying 3D representation is equivalent to a geometrically-aware constraint optimization process:

[0143] ;

[0144] ;

[0145] in, This represents finding the minimum value for a single vector x. This represents a text alignment function. Represents the balance parameters. The geometric manifold projection representing a single vector. represents a 3D geometric manifold, s represents the scaling parameter, and t represents the time step;

[0146] The optimization process is decomposed into multiple sub-problems, and an augmented Lagrangian function is constructed based on the auxiliary variables and the obtained scaling dual variables:

[0147] ;

[0148] in, This represents the augmented Lagrange function. This represents the force of the geometric constraints, where z represents the auxiliary variable. Indicates scaling dual variables;

[0149] Iterative updates are performed on the single vector, auxiliary variables, and scaled dual variables based on the augmented Lagrangian function;

[0150] Based on the updated single vector, auxiliary variables, and scaling dual variables, a total gradient function is constructed according to the SDS gradient, the scaling dual variables, and the surface regularization gradient. The ADMM framework is then optimized using this total gradient function.

[0151] ;

[0152] in, Represents the total gradient function. For SDS gradient, For surface regularization gradient, Represents the gradient of geometric consistency constraints;

[0153] The optimized ADMM framework is used to analyze the description information of the target object to obtain the target 3D model of the target object to be tested.

[0154] In the embodiments disclosed in this invention, the text-to-3D generation reconstruction is first transformed into a geometrically perceptual constraint optimization problem, and the geometry is regularized in two complementary spaces: to alleviate the instability of random SDS updates, a cosine distance loss is applied in the normal vector domain using the RED framework to suppress high-frequency noise while preserving semantically relevant details; the geometric manifold projection of a single vector is constrained to a reasonable 3D geometric manifold through plug-and-play priors.

[0155] This method transforms the optimization problem from unconstrained to constrained, theoretically solving the fundamental problems of ill-conditioned and under-constrained SDS optimization. Ablation experiments show that a guiding mechanism from degenerate solutions to reasonable geometries can be established simply by introducing a constraint framework (without specific implementation).

[0156] Furthermore, to solve the geometrically perceptual constraint problem, the augmented Lagrange method is used (where, for example, ... Figure 2 As shown, the gradient of the augmented Lagrange function is By combining variable splitting and optimizing through ADMM, this process decouples semantic guidance, local surface refinement and global geometric feasibility into tractable subproblems.

[0157] Furthermore, gradient descent is applied to the single vector based on the augmented Lagrangian function to obtain the single vector after the current iteration:

[0158] ;

[0159] in, Represents a single vector in the (k+1)th iteration;

[0160] The auxiliary variable is then subjected to plug-and-play geometric projection and updated using a momentum mechanism to obtain the auxiliary variable after the current iteration:

[0161] ;

[0162] in, Describes the auxiliary variable for the (k+1)th iteration. Indicates an indicator function, This represents the geometric manifold projection of a single vector in the (k+1)th iteration. Describe the geometric manifold of z;

[0163] The scaling dual variable is updated to obtain the scaling dual variable after the current iteration:

[0164] ;

[0165] in, Denotes the scaling dual variable for the (k+1)th iteration. Let represent the scaling dual variable for the k-th iteration.

[0166] In this process, auxiliary variables and scaling dual variables are introduced to construct the augmented Lagrangian function, and quadratic terms facilitate the extraction of geometry. Consistency with auxiliary variables is maintained, and in each iteration, Geo-ADMM alternately performs update steps:

[0167] (1) Apply gradient descent to the scene parameter x to minimize the SDS loss, surface regularization, and ADMM penalty term (i.e. In practice, the solution is approximated by a single gradient descent step.

[0168] (2) Project the auxiliary variable z onto the geometric manifold; because For implicit manifolds, a plug-and-play approximation is used:

[0169] ;

[0170] Where D is the latent space projection operator; to improve time stability, momentum is further applied:

[0171] ;

[0172] in, The momentum coefficient is denoted as 0.9 in the embodiments disclosed in this invention. The momentum mechanism provides temporal smoothness for the geometric projection step, preventing drastic jumps in z-updates from causing optimization instability. Theoretical analysis shows that a larger momentum coefficient produces a more conservative (generally more stable) projection, but the worst-case bound is relaxed. A momentum coefficient of 0.9 achieves a good balance between stability and convergence speed.

[0173] (3) Update the dual variables to accumulate constraint violation residuals.

[0174] In the embodiments disclosed in this invention, the SDS gradient provides strong semantic supervision to ensure that the generated 3D content is highly matched with the text description. The surface regularization gradient can constrain geometric rationality, suppress artifacts and unnatural protrusions, and scale the dual variable to coordinate the constraint satisfaction between subproblems, thereby improving convergence stability. Through the weighted fusion of the three, the model can not only "understand" the text, "perceive" the spatial structure, but also "obey" mathematical constraints during the optimization process, thus achieving multi-objective balanced optimization.

[0175] Furthermore, this invention provides theoretical support for convergence under three mild assumptions:

[0176] Assumption 1: The geometric denoiser satisfies robustness and non-scalability, that is, for all variables... and Both exist:

[0177] ;

[0178] in, This indicates the use of a geometric denoiser on variables. The result after instantiation This indicates the use of a geometric denoiser on variables. The result after instantiation;

[0179] Assumption 2: The momentum parameter takes values ​​in the range [0, 1), sequence Bounded;

[0180] Assumption 3: Operator perturbation induced by one-step original update (Adam) (i.e. , It satisfies the properties of summability and square summability.

[0181] Based on assumptions 1, 2, and 3, this invention proposes the following theoretical results:

[0182] (1) Fejér-type descent lemma (an important tool in approximation theory and harmonic analysis, used to study the convergence of function sequences): The momentum plug-and-play operator T satisfies:

[0183] ;

[0184] in, Represents the current iteration vector. This represents the optimal vector solution. express The mapping, This indicates the use of a geometric denoiser for... The result after instantiation; ensuring a monotonically decreasing geometric residual at each iteration.

[0185] (2) Non-traversal Geometric residual bounds: This ensures that in the first K iterations, there exists at least one geometric residual of... Magnitude (where, (This represents the initial iteration vector).

[0186] (3) Asymptotic geometric consistency: This ensures that the iterative asymptotically satisfies the PnP geometric constraints (where, This indicates the use of a geometric denoiser to denoise the sequence. (Result after instantiation).

[0187] (4) Characterization of the limit point: The limit point satisfies Satisfying the PnP fixed point condition: Simultaneously satisfying the first-order optimality equilibrium condition, it formalizes the intuition of Geo-ADMM to decouple appearance-driven supervision from geometric verification and converge to a geometrically consistent equilibrium point (where, (These represent the optimal scene parameters, the optimal auxiliary variable, and the optimal scaling dual variable, respectively).

[0188] Furthermore, this invention conducts a comprehensive evaluation on the T3Bench benchmark (Benchmarking Current Progress in Text-to-3D Generation, a comprehensive evaluation benchmark for the progress of text-to-3D generation technology), thereby demonstrating the effectiveness of the Geo-ADMM framework disclosed in this invention.

[0189] Among them, such as Figure 3 As shown, the results of our method are compared with those of different baseline methods in the prior art (where DreamFusion represents 3D Gaussian Splatting for Text-to-3D Generation, a text-to-3D generation model; Magic3D represents High-Resolution Text-to-3D Content Creation, a high-resolution text-to-3D content generation model; ProlificDreamer represents High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, a high-fidelity and diverse text-to-3D generation model based on variational fractional distillation; and LucidDreamer is a leading model of 3D Gaussian Splatting for Text-to-3D Generation). Under the same text prompts (i.e., inputting "A zoomed out DSLR photo of a baby bunny sitting on top of a stack of pancakes" and "A DSLR photo of a Schnauzer wearing a piratehat"), the 3D content generated by our method has more faithful text alignment and finer geometric details.

[0190] Furthermore, such as Figure 4As shown in (a) (with input texts "Zombie JOKER,head,photorealistic" and "Batman,head,photorealistic,SK,HDR"), this paper presents a multi-view consistency comparison between our method and other baseline methods, demonstrating that our method effectively eliminates the polyhedral problem; Figure 4 As shown in (b) (with input texts "A DSLR photo of a football helmet" and "A DSLR photo of alion head scupture"), the surface quality comparison between this method and other baseline methods is presented. The results show that this method significantly reduces visual artifacts and produces a smoother, more refined surface; Figure 4 As shown in (c), the ablation study results of this method and other baseline methods are presented. This method, combined with surface regularization terms and the ADMM framework, can improve structural integrity and achieve results that surpass those of all baseline methods.

[0191] Among them, the complete Geo-ADMM (ADMM+RED) achieved the best results—a Janus rate (evaluating geometric consistency defects in text-to-3D generation models) of 12%, a CLIP score of 31.02, and a user preference rate of 53.5%. The dual-space strategy effectively "cleans up" the reconstruction results: eliminating floating artifacts and forcing the generation of sharp, continuous surfaces.

[0192] Among them, the ablation experiment results showed that, using only ADMM constraints (excluding surface regularization terms) compared to the baseline VanillaSDS (Vanilla Switching Dynamical System), the Janus rate decreased from 42% to 38%, the CLIP score (a core indicator for evaluating the quality of text-to-image generation) increased from 29.25 to 29.82, and the user preference rate increased from 5.2% to 12.8%.

[0193] Furthermore, such as Figure 5 (a) Figure 5 (b) Figure 6 (a) and Figure 6 As shown in (b), the decomposition relationship of the original SDS gradient (high noise), ADMM correction term (positively correlated control variable), and effective gradient (low variance, high signal-to-noise ratio) is illustrated. Among them, the ADMM correction term is strongly positively correlated with the high variance fluctuation in the original SDS gradient and effectively acts as a control variable. Compared with the original SDS gradient, the effective gradient achieves significant variance reduction and signal-to-noise ratio improvement, ensuring robust convergence along the effective geometric manifold.

[0194] Among them, according to Figure 5 and Figure 6 As can be seen, the ADMM correction term in this invention exhibits a strong positive correlation with the high variance fluctuations in the original SDS gradient, effectively serving as a control variable. Compared to the original SDS gradient, the effective gradient achieves significant variance reduction and signal-to-noise ratio improvement, ensuring robust convergence along the effective geometric manifold.

[0195] This invention can be seamlessly integrated into various SDS pipelines without retraining the diffusion model, and significantly improves geometric integrity and surface quality while maintaining semantic fidelity.

[0196] Furthermore, such as Figure 7 As shown, based on the above-mentioned method for generating 3D digital models from training-free text using geometry-aware ADMM, this invention also provides a system for generating 3D digital models from training-free text using geometry-aware ADMM, wherein the system includes:

[0197] The image rendering module 51 is used to construct a low-level 3D representation based on the user's text prompts for the target object, parameterize the low-level 3D representation to obtain multiple parameters, and render the low-level 3D representation into a 2D image and a depth image based on all parameters.

[0198] The noise addition module 52 is used to extract the diffusion latent variable of the 2D image using the encoder of the variational autoencoder, perform multi-scale downsampling on the diffusion latent variable to obtain feature maps of different resolutions, construct the SDS gradient based on the diffusion latent variable, and iteratively update the auxiliary variable and scaling dual variable introduced into the ADMM framework.

[0199] The geometric constraint module 53 is used to calculate the pseudo normal of the depth image, smooth the pseudo normal using a bilateral filter to obtain the target normal, and calculate the surface regularization gradient based on the target normal.

[0200] The optimization generation module 54 is used to obtain the scaling dual variable, optimize the constructed ADMM framework according to the SDS gradient, the scaling dual variable and the surface regularization gradient, obtain the test text prompt of the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target three-dimensional digital model of the target object or the new target object.

[0201] Furthermore, such as Figure 8 As shown, based on the above-mentioned method and system for generating three-dimensional digital models of text without training based on geometric perception ADMM, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 8 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0202] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code installed on the terminal. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a three-dimensional digital model generation program 40 based on geometry-aware ADMM (Advanced Digital Modeling Model) for training-free text. This program 40 can be executed by the processor 10 to implement the three-dimensional digital model generation method based on geometry-aware ADMM for training-free text in this application.

[0203] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the method for generating a three-dimensional digital model of text without training based on geometric perception ADMM.

[0204] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0205] In one embodiment, when processor 10 executes the training-free text 3D digital model generation program 40 based on geometry-aware ADMM stored in memory 20, the following steps are performed:

[0206] A low-level 3D representation is constructed based on the user's text prompts for the target object. The low-level 3D representation is parameterized to obtain multiple parameters. Based on all the parameters, the low-level 3D representation is rendered into a 2D image and a depth image.

[0207] The diffusion latent variables of the 2D image are extracted using a variational autoencoder, and the diffusion latent variables are downsampled at multiple scales to obtain feature maps of different resolutions. An SDS gradient is then constructed based on the diffusion latent variables.

[0208] The pseudo normal of the depth image is calculated, and the pseudo normal is smoothed using a bilateral filter to obtain the target normal. The surface regularization gradient is then calculated based on the target normal.

[0209] Obtain the scaling dual variable, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variable and the surface regularization gradient, obtain the test text prompt of the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target 3D digital model of the target object or the new target object.

[0210] Specifically, the step of constructing a low-level 3D representation based on the user's text prompts for the target object, parameterizing the low-level 3D representation to obtain multiple parameters, and rendering the low-level 3D representation into a 2D image and a depth image based on all parameters includes:

[0211] Obtain text prompts input by the user, wherein the text prompts represent textual descriptions of the target object;

[0212] Assign values ​​to multiple 3D Gaussian primitives according to the text prompts, and obtain the position parameters, scaling parameters, rotation quaternions, opacity and color parameters of each 3D Gaussian primitive;

[0213] Based on all parameters assigned to all 3D Gaussian primitives, construct the underlying 3D representation of the target object;

[0214] Render all the 3D Gaussian elements in the underlying 3D representation to obtain the 2D image and depth image of the target object.

[0215] Specifically, rendering all the 3D Gaussian primitives in the underlying 3D representation to obtain the 2D image and depth image of the target object includes:

[0216] Calculate the residual sum of squares based on the scaling parameter and the rotation quaternion for each of the 3D Gaussian elements, and calculate the covariance matrix based on the residual sum of squares:

[0217] ;

[0218] in, Let R denote the covariance matrix, R denote the rotation matrix derived from the rotation quaternion, S denote the scaling diagonal matrix composed of scaling parameters, and T denote the transpose.

[0219] Calculate the density at each of the 3D Gaussian elements based on the covariance matrix:

[0220] ;

[0221] in, This represents the density of p. Indicates an exponential mapping. This represents the position parameter, where p represents the 3D Hexagonal element;

[0222] For each of the 3D Gaussian primitives, corresponding scene parameters are constructed based on the position parameter, the scaling parameter, the rotation quaternion, the opacity, and the color parameter:

[0223] ;

[0224] in, Indicates scene parameters, This represents the position parameter of the i-th 3D Gaussian element. This represents the scaling parameter of the i-th 3D Gaussian element. Denotes the rotation quaternion of the i-th 3D Gaussian element. Let represent the opacity of the i-th 3D Gaussian element. Let N represent the camera viewpoint of the i-th 3D Gaussian element, and N represent the total number of 3D Gaussian elements.

[0225] Obtain the camera viewpoint that samples the target object, concatenate the scene parameters into a single vector, and construct the 2D image and the depth image based on the single vector and the camera viewpoint.

[0226] Specifically, the step of extracting the diffusion latent variables of the 2D image using a variational autoencoder, performing multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and constructing an SDS gradient based on the diffusion latent variables includes:

[0227] The 2D image is input into a variational autoencoder for feature extraction, and the output is a diffusion latent variable:

[0228] ;

[0229] Where f represents the diffusion latent variable, Let x represent a 2D image, c represent a single vector, and c represent the camera viewpoint from which the target object is sampled. Indicates to Feature extraction is performed, where R represents the rotation matrix derived from the rotation quaternion;

[0230] During the diffusion process, Gaussian random noise is added to the latent variables corresponding to the 2D image to obtain auxiliary variables;

[0231] Based on the camera viewpoint, the SDS gradient is constructed according to the auxiliary variables at each time step:

[0232] ;

[0233] in, Indicates the direction of the update for x. Represents the SDS gradient. Let represent the expectation with respect to time step t, Gaussian random noise, and camera viewpoint c. This represents the weighting function at diffusion time step t. Let y represent the auxiliary variable at diffusion time step t, and let y represent the text prompt. Represents a diffusion model. Indicates to Differentiate, This represents Gaussian random noise;

[0234] In the ADMM framework, auxiliary variables and scaling dual variables are introduced. The diffusion latent variable is added to the scaling dual variable and then projected through a plug-and-play geometric denoising device to obtain the updated auxiliary variable.

[0235] In the ADMM framework, the states of the auxiliary variables and the scaling dual variables at different resolution scales are kept independent and iteratively updated.

[0236] Specifically, the calculation of the pseudo-normal of the depth image, smoothing the pseudo-normal using a bilateral filter to obtain the target normal, and calculating the surface regularization gradient based on the target normal, includes:

[0237] The pseudo-normals of the depth image are calculated using the Sobel operator:

[0238] ;

[0239] Where n represents the pseudonormal. This indicates the calculation of pseudonormals. This represents the Sobel operator in the horizontal direction. This represents the Sobel operator in the vertical direction, where D represents the rendering depth of the depth image;

[0240] The pseudo-normal is smoothed using a bilateral filter to obtain the target normal, and the cosine distance loss between the target normal and the pseudo-normal is calculated.

[0241] ;

[0242] in, This represents the cosine distance loss, where p represents the 3D Gaussian element. This represents the pseudo-normal at point p. Indicates the target normal. Represents the cosine value;

[0243] The surface regularization gradient is obtained by differentiating the cosine distance loss.

[0244] Specifically, the step of obtaining the scaling dual variable involves optimizing the constructed ADMM framework based on the SDS gradient, the scaling dual variable, and the surface regularization gradient; obtaining the test text prompt for the target object or a new target object; and analyzing the test text prompt using the optimized ADMM framework to obtain the target 3D digital model of the target object or the new target object.

[0245] The process of constructing the underlying 3D representation is equivalent to a geometrically constrained optimization process:

[0246] ;

[0247] ;

[0248] in, This represents finding the minimum value for a single vector x. This represents a text alignment function. Represents the balance parameters. The geometric manifold projection representing a single vector. represents a 3D geometric manifold, s represents the scaling parameter, and t represents the time step;

[0249] The optimization process is decomposed into multiple sub-problems, and an augmented Lagrangian function is constructed based on the auxiliary variables and the obtained scaling dual variables:

[0250] ;

[0251] in, This represents the augmented Lagrange function. This represents the force of the geometric constraints, where z represents the auxiliary variable. Indicates scaling dual variables;

[0252] Iterative updates are performed on the single vector, auxiliary variables, and scaled dual variables based on the augmented Lagrangian function;

[0253] Based on the updated single vector, auxiliary variables, and scaling dual variables, a total gradient function is constructed according to the SDS gradient, the scaling dual variables, and the surface regularization gradient. The ADMM framework is then optimized using this total gradient function.

[0254] ;

[0255] in, Represents the total gradient function. For SDS gradient, For surface regularization gradient, Represents the gradient of geometric consistency constraints;

[0256] The optimized ADMM framework is used to analyze the description information of the target object to obtain the target 3D model of the target object to be tested.

[0257] Specifically, the iterative update of the single vector, auxiliary variable, and scaling dual variable based on the augmented Lagrangian function includes:

[0258] Gradient descent is applied to the single vector based on the augmented Lagrangian function to obtain the single vector after the current iteration:

[0259] ;

[0260] in, Represents a single vector in the (k+1)th iteration;

[0261] The auxiliary variable is then subjected to plug-and-play geometric projection and updated using a momentum mechanism to obtain the auxiliary variable after the current iteration:

[0262] ;

[0263] in, Describes the auxiliary variable for the (k+1)th iteration. Indicates an indicator function, This represents the geometric manifold projection of a single vector in the (k+1)th iteration. Describe the geometric manifold of z;

[0264] The scaling dual variable is updated to obtain the scaling dual variable after the current iteration:

[0265] ;

[0266] in, Denotes the scaling dual variable for the (k+1)th iteration. Let represent the scaling dual variable for the k-th iteration.

[0267] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a three-dimensional digital model generation program for training-free text based on geometry-aware ADMM, and the three-dimensional digital model generation program for training-free text based on geometry-aware ADMM, when executed by a processor, implements the steps of the three-dimensional digital model generation method for training-free text based on geometry-aware ADMM as described above.

[0268] In summary, this invention provides a method and related equipment for generating a training-free 3D digital model of text based on geometrically perceptual ADMM. The method includes: constructing a low-level 3D representation based on user text prompts for a target object; parameterizing the low-level 3D representation to obtain multiple parameters; rendering the low-level 3D representation into a 2D image and a depth image based on all parameters; extracting the diffusion latent variables of the 2D image using a variational autoencoder; performing multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions; constructing an SDS gradient based on the diffusion latent variables; calculating the pseudo-normals of the depth image; smoothing the pseudo-normals using a bilateral filter to obtain the target normals; and calculating the surface regularization gradient based on the target normals; obtaining scaling dual variables; optimizing the constructed ADMM framework based on the SDS gradient, the scaling dual variables, and the surface regularization gradient; obtaining the text prompts for the target object or a new target object; analyzing the text prompts for the target object using the optimized ADMM framework; and obtaining the target 3D digital model of the target object or the new target object. This invention can be seamlessly integrated into various SDS pipelines without retraining the diffusion model, and significantly improves geometric integrity and surface quality while maintaining semantic fidelity.

[0269] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0270] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0271] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for training-free text-based 3D digital model generation based on geometric-aware ADMM, characterized in that, The method for generating 3D digital models of text without training based on geometry-aware ADMM includes: A low-level 3D representation is constructed based on the user's text prompts for the target object. The low-level 3D representation is parameterized to obtain multiple parameters. Based on all the parameters, the low-level 3D representation is rendered into a 2D image and a depth image. The diffusion latent variables of the 2D image are extracted using a variational autoencoder, and the diffusion latent variables are downsampled at multiple scales to obtain feature maps of different resolutions. An SDS gradient is then constructed based on the diffusion latent variables. The process involves calculating the pseudo-normals of the depth image, smoothing the pseudo-normals using a bilateral filter to obtain the target normals, and calculating the surface regularization gradient based on the target normals. Specifically, this includes: The pseudo-normals of the depth image are calculated using the Sobel operator: ； wherein n denotes a pseudo normal, denotes a calculation of a pseudo normal, denotes a horizontal Sobel operator, denotes a vertical Sobel operator, and D denotes a rendering depth of the depth image; The pseudo-normal is smoothed using a bilateral filter to obtain the target normal, and the cosine distance loss between the target normal and the pseudo-normal is calculated. ； in, This represents the cosine distance loss, where p represents the 3D Gaussian element. This represents the pseudo-normal at point p. Indicates the target normal. Represents the cosine value; Differentiating the cosine distance loss yields the surface regularization gradient; Obtain the scaling dual variable, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variable, and the surface regularization gradient, obtain the test text prompt for the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target 3D digital model of the target object or the new target object, specifically including: The process of constructing the underlying 3D representation is equivalent to a geometrically constrained optimization process: ； ； in, This represents finding the minimum value for a single vector x. This represents a text alignment function. Represents the balance parameters. The geometric manifold projection representing a single vector. represents a 3D geometric manifold, s represents the scaling parameter, and t represents the time step; The optimization process is decomposed into multiple sub-problems, and an augmented Lagrangian function is constructed based on the auxiliary variables and the obtained scaling dual variables: ； wherein, denotes the augmented Lagrangian function, denotes the control of the force degree of constraint, z denotes the auxiliary variable, denotes the scaling of the dual variable; Iterative updates are performed on the single vector, auxiliary variables, and scaled dual variables based on the augmented Lagrangian function; Based on the updated single vector, auxiliary variables, and scaling dual variables, a total gradient function is constructed according to the SDS gradient, the scaling dual variables, and the surface regularization gradient. The ADMM framework is then optimized using this total gradient function. ； wherein, denotes the total gradient function, is the SDS gradient, is the surface regularization gradient, denotes the geometric consistency constraint gradient; The optimized ADMM framework is used to analyze the description information of the target object to obtain the target 3D model of the target object to be tested.

2. The geometry-aware ADMM based training-free text 3D digital model generation method of claim 1, wherein, The process of constructing a low-level 3D representation based on user text prompts for the target object, parameterizing the low-level 3D representation to obtain multiple parameters, and rendering the low-level 3D representation into a 2D image and a depth image based on all parameters specifically includes: Obtain text prompts input by the user, wherein the text prompts represent textual descriptions of the target object; Assign values ​​to multiple 3D Gaussian primitives according to the text prompts, and obtain the position parameters, scaling parameters, rotation quaternions, opacity and color parameters of each 3D Gaussian primitive; Based on all parameters assigned to all 3D Gaussian primitives, construct the underlying 3D representation of the target object; Render all the 3D Gaussian elements in the underlying 3D representation to obtain the 2D image and depth image of the target object.

3. The geometry-aware ADMM based training-free text 3D digital model generation method of claim 2, wherein, The step of rendering all the 3D Gaussian primitives in the underlying 3D representation to obtain the 2D image and depth image of the target object specifically includes: Calculate the residual sum of squares based on the scaling parameter and the rotation quaternion for each of the 3D Gaussian elements, and calculate the covariance matrix based on the residual sum of squares: ； in, Let R denote the covariance matrix, R denote the rotation matrix derived from the rotation quaternion, S denote the scaling diagonal matrix composed of scaling parameters, and T denote the transpose. Calculate the density at each of the 3D Gaussian elements based on the covariance matrix: ； wherein, denotes the density of p, denotes an exponential map, denotes a position parameter, p denotes a 3D Gaussian cell; For each of the 3D Gaussian primitives, corresponding scene parameters are constructed based on the position parameter, the scaling parameter, the rotation quaternion, the opacity, and the color parameter: ； in, Indicates scene parameters, This represents the position parameter of the i-th 3D Gaussian element. This represents the scaling parameter of the i-th 3D Gaussian element. Denotes the rotation quaternion of the i-th 3D Gaussian element. Let represent the opacity of the i-th 3D Gaussian element. This represents the color parameter of the i-th 3D Gaussian pixel, and N represents the total number of 3D Gaussian pixels; Obtain the camera viewpoint that samples the target object, concatenate the scene parameters into a single vector, and construct the 2D image and the depth image based on the single vector and the camera viewpoint.

4. The geometry-aware ADMM based training-free text 3D digital model generation method of claim 1, wherein, The process of extracting the diffusion latent variables of the 2D image using a variational autoencoder, performing multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and constructing an SDS gradient based on the diffusion latent variables specifically includes: The 2D image is input into a variational autoencoder for feature extraction, and the output is a diffusion latent variable: ； wherein f denotes a diffusion latent variable, denotes a 2D image, x denotes a single vector, c denotes a camera perspective sampling the target object, denotes a feature extraction on denotes a rotation matrix derived from a rotation quaternion; During the diffusion process, Gaussian random noise is added to the latent variables corresponding to the 2D image to obtain auxiliary variables; Based on the camera viewpoint, the SDS gradient is constructed according to the auxiliary variables at each time step: ； in, Indicates the direction of the update for x. Represents the SDS gradient. Let represent the expectation with respect to time step t, Gaussian random noise, and camera viewpoint c. This represents the weighting function at diffusion time step t. Let y represent the auxiliary variable at diffusion time step t, and let y represent the text prompt. Represents a diffusion model. Indicates to Differentiate, This represents Gaussian random noise; In the ADMM framework, auxiliary variables and scaling dual variables are introduced. The diffusion latent variable is added to the scaling dual variable and then projected through a plug-and-play geometric denoising device to obtain the updated auxiliary variable. In the ADMM framework, the states of the auxiliary variables and the scaling dual variables at different resolution scales are kept independent and iteratively updated.

5. The method for generating a three-dimensional digital model of text without training based on geometrically perceptual ADMM according to claim 1, characterized in that, The iterative update of the single vector, auxiliary variable, and scaling dual variable based on the augmented Lagrangian function specifically includes: Gradient descent is applied to the single vector based on the augmented Lagrangian function to obtain the single vector after the current iteration: ； wherein, represents the single vector of the k+1 iteration; The auxiliary variable is then subjected to plug-and-play geometric projection and updated using a momentum mechanism to obtain the auxiliary variable after the current iteration: ； in, Describes the auxiliary variable for the (k+1)th iteration. Indicates an indicator function, This represents the geometric manifold projection of a single vector in the (k+1)th iteration. Describe the geometric manifold of z; The scaling dual variable is updated to obtain the scaling dual variable after the current iteration: ； in, Denotes the scaling dual variable for the (k+1)th iteration. Let represent the scaling dual variable for the k-th iteration.

6. A geometry-aware ADMM based training-free text 3D digital model generation system, characterized in that, The geometry-aware ADMM-based training-free 3D digital model generation system is used to implement the geometry-aware ADMM-based training-free 3D digital model generation method as described in any one of claims 1-5, wherein the geometry-aware ADMM-based training-free 3D digital model generation system comprises: The image rendering module is used to construct a low-level 3D representation based on the user's text prompts for the target object, parameterize the low-level 3D representation to obtain multiple parameters, and render the low-level 3D representation into a 2D image and a depth image based on all parameters. The noise addition module is used to extract the diffusion latent variables of the 2D image using a variational autoencoder, perform multi-scale downsampling on the diffusion latent variables to obtain feature maps of different resolutions, and construct the SDS gradient based on the diffusion latent variables. The geometric constraint module is used to calculate the pseudo normals of the depth image, smooth the pseudo normals using a bilateral filter to obtain the target normals, and calculate the surface regularization gradient based on the target normals. An optimization generation module is used to obtain scaling dual variables, optimize the constructed ADMM framework based on the SDS gradient, the scaling dual variables, and the surface regularization gradient, obtain the test text prompt of the target object or a new target object, analyze the test text prompt using the optimized ADMM framework, and obtain the target 3D digital model of the target object or the new target object.

7. A terminal, characterized by comprising: The terminal includes: a memory, a processor, and a three-dimensional digital model generation program for training-free text based on geometry-aware ADMM stored in the memory and executable on the processor. When the three-dimensional digital model generation program for training-free text based on geometry-aware ADMM is executed by the processor, it implements the steps of the three-dimensional digital model generation method for training-free text based on geometry-aware ADMM as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a three-dimensional digital model generation program for training-free text based on geometry-aware ADMM. When the three-dimensional digital model generation program for training-free text based on geometry-aware ADMM is executed by a processor, it implements the steps of the three-dimensional digital model generation method for training-free text based on geometry-aware ADMM as described in any one of claims 1-5.

Images