Digital human model generation method and apparatus

Abstract

The application discloses a digital human model generation method and device, and relates to the technical field of computer graphics. The method first determines an initial parameter set of a 3D Gaussian ellipsoid on an extensible skinning multi-person linear model, wherein the initial parameter set comprises initial positions and initial attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid. Then, a conditional feature vector of a target 3D digital human model is acquired, the initial parameter set and the conditional feature vector are processed, and parameter deviations corresponding to the initial parameters are obtained. Finally, the initial parameters are updated according to the parameter deviations, and a 3D digital human model is constructed by using the updated parameters. The method can efficiently and accurately generate a high-quality 3D digital human model conforming to a human structure, and is widely applied in the fields of games, films and television, virtual reality and the like.

Description

Technical Field

[0001] This application relates to the field of computer graphics technology, and in particular to a method and apparatus for generating digital human models. Background Technology

[0002] In today's rapidly accelerating digital transformation, 3D digital humans, with their unique advantages, have been widely applied in numerous fields such as the metaverse, virtual reality, gaming, and digital marketing. 3D digital humans add vibrant virtual characters to the metaverse, allowing users to have a more realistic interactive experience in the virtual world; in gaming, 3D digital humans can create lifelike and charismatic characters, enhancing the fun and immersion of games; in digital marketing, 3D digital humans can serve as an innovative medium for brands to communicate with consumers, strengthening brand influence.

[0003] Currently, to simplify the generation process of 3D digital human models and reduce production costs, technologies that utilize images or text descriptions to generate 3D digital humans have emerged. This technology aims to eliminate the heavy reliance on professional artists and complex modeling software in traditional production methods, and to quickly generate corresponding 3D digital human models by inputting image or text information.

[0004] However, existing image or text description-based 3D digital human generation technologies produce 3D digital human models with poor accuracy and insufficient detail, making it difficult to achieve a highly realistic effect that matches the actual scene and thus failing to meet the needs of some application scenarios with high model quality requirements. Summary of the Invention

[0005] To address the above problems, this application provides a method and apparatus for generating digital human models, comprising the following:

[0006] Firstly, this application provides a method for generating a digital human model, the method comprising:

[0007] Obtain the conditional feature vector of the target 3D digital human model;

[0008] The initial parameter set and the conditional feature vector are processed using a diffusion model to obtain the parameter deviations corresponding to each initial parameter; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on a scalable skinned multi-person linear model and a 3D Gaussian ellipsoid association, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid;

[0009] The parameters in the initial parameter set are updated based on the parameter deviation, and the target 3D digital human model is constructed using the updated parameters.

[0010] Optionally, the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid;

[0011] The initial attribute parameters are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0012] Optionally, after constructing the 3D digital human model using the updated parameters, the method further includes:

[0013] When it is necessary to render the constructed 3D digital human model, the constructed 3D digital human model is rendered into a 2D image based on the 3D Gaussian sputtering renderer.

[0014] Optionally, the initialization of attribute parameters for all Gaussian points on the 3D Gaussian ellipsoid includes:

[0015] Initialize the scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0016] Optionally, the conditional feature vector of the target 3D digital human model includes:

[0017] Obtain the expected conditions for generating the target 3D digital human model, the expected conditions including at least one of textual description or image;

[0018] Determine the type of the expected condition. When the expected condition is a text description or an image, use the condition feature vector extracted based on the text description or the image as the condition feature vector of the target 3D digital human model.

[0019] When the expected conditions are text descriptions and images, the conditional feature vectors extracted based on the text and the conditional feature vectors extracted based on the images are fused, and the fused conditional feature vectors are used as the conditional feature vectors of the target 3D digital human model.

[0020] Optionally, the step of processing the initial parameter set and the conditional feature vector using a diffusion model to obtain the parameter deviations corresponding to each initial parameter includes:

[0021] The initial parameter set is reduced to a latent space using an autoencoder based on a diffusion model. The reduced initial parameter set and the conditional feature vector are then processed in the latent space to obtain the parameter deviation.

[0022] Optionally, the autoencoder using the diffusion model reduces the dimensionality of the initial parameter set to a latent space, and processes the dimensionality-reduced initial parameter set and the conditional feature vector in the latent space to obtain the parameter bias, including:

[0023] The autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters.

[0024] The initial parameters in the latent space are subjected to forward noise addition to obtain the initial parameters with added noise.

[0025] Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters.

[0026] The parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before dimensionality reduction using the decoder of the autoencoder, thus obtaining the parameter deviation.

[0027] Optionally, updating the parameters in the initial parameter set based on the parameter deviation includes:

[0028] Each parameter in the initial parameter set is summed with its corresponding parameter deviation to update the initial parameters.

[0029] Secondly, this application provides a digital human model generation apparatus, the apparatus comprising:

[0030] The acquisition unit is used to acquire the conditional feature vector of the target 3D digital human model;

[0031] The parameter deviation generation unit is used to process the initial parameter set and the conditional feature vector using a diffusion model to obtain the parameter deviation corresponding to each initial parameter; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on the association between the scalable skinned multi-person linear model and the 3D Gaussian ellipsoid, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid;

[0032] The parameter update unit is used to update the parameters in the initial parameter set according to the parameter deviation;

[0033] The model building unit is used to build the target 3D digital human model using the updated initial parameters.

[0034] Optionally, the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid;

[0035] The initial attribute parameters are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0036] Optionally, the device further includes a rendering unit for rendering the constructed 3D digital human model into a 2D image based on a 3D Gaussian sputtering renderer when rendering is required.

[0037] Optionally, the initialization unit initializes the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid, including:

[0038] Initialize the scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0039] Optionally, the acquisition unit is specifically used to acquire the expected conditions for generating the target 3D digital human model, the expected conditions including at least one of text description or image;

[0040] Determine the type of the expected condition. When the expected condition is a text description or an image, use the condition feature vector extracted based on the text description or the image as the condition feature vector of the target 3D digital human model.

[0041] When the expected conditions are text descriptions and images, the conditional feature vectors extracted based on the text and the conditional feature vectors extracted based on the images are fused, and the fused conditional feature vectors are used as the conditional feature vectors of the target 3D digital human model.

[0042] Optionally, the parameter deviation generation unit is specifically used to reduce the dimensionality of the initial parameter set to the latent space using the autoencoder of the diffusion model, and to process the dimensionality-reduced initial parameter set and the conditional feature vector in the latent space to obtain the parameter deviation.

[0043] Optionally, the autoencoder using the diffusion model reduces the dimensionality of the initial parameter set to a latent space, and processes the dimensionality-reduced initial parameter set and the conditional feature vector in the latent space to obtain the parameter bias, including:

[0044] The autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters.

[0045] The initial parameters in the latent space are subjected to forward noise addition to obtain the initial parameters with added noise.

[0046] Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters.

[0047] The parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before dimensionality reduction using the decoder of the autoencoder, thus obtaining the parameter deviation.

[0048] Optionally, the parameter update unit is specifically used to perform a one-to-one summation operation between each parameter in the initial parameter set and its corresponding parameter deviation to complete the update of the initial parameters.

[0049] Thirdly, embodiments of this application provide a computer device, the computer device comprising: a processor and a memory; the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the steps of the digital human model generation method as described in the first aspect above according to the instructions in the program code.

[0050] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the digital human model generation method described in any of the implementations of the first aspect.

[0051] This application provides a method for generating a digital human model. When executing the method, firstly, a conditional feature vector of the target 3D digital human model is obtained; then, a diffusion model is used to process the initial parameter set and the conditional feature vector to obtain the parameter deviations corresponding to each initial parameter; the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on the association between the scalable skinned multi-person linear model and the 3D Gaussian ellipsoid, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid; finally, the parameters in the initial parameter set are updated according to the parameter deviations, and the updated parameters are used to construct the 3D digital human model.

[0052] In this way, by setting the initial position of the center point of the 3D Gaussian ellipsoid based on the vertices of the scalable skinned multi-person linear model, and since the scalable skinned multi-person linear model accurately depicts the human body's topology, this ensures a reasonable distribution of the 3D Gaussian ellipsoid, making the final generated 3D digital human model more consistent with the human body's topology and effectively avoiding problems such as limb breakage and adhesion. By using a diffusion model to process the conditional feature vector of the target 3D digital human model with the initial parameter set, the incremental deviation of the 3D Gaussian parameters is directly predicted, and the initial parameters are updated based on the deviation data. This results in a 3D digital human model generated based on the updated initial parameters with richer details. At the same time, the initial parameters can be adjusted according to specific needs, making the generated 3D digital human model more flexible and adaptable, meeting diverse needs for generating human features, and achieving the effect of accurately customizing the 3D digital human model. Thus, high-quality, compliant, and structurally reasonable 3D digital human models can be generated efficiently and accurately, and are widely used in games, film and television, virtual reality, and other fields. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart illustrating a method for generating a 3D digital human model provided in this application embodiment;

[0055] Figure 2 A schematic diagram illustrating the process of obtaining the conditional feature vector of a target 3D digital human model, provided in an embodiment of this application;

[0056] Figure 3 A schematic diagram illustrating the process of obtaining the conditional feature vector of another target 3D digital human model provided in an embodiment of this application;

[0057] Figure 4 A flowchart illustrating a parameter deviation generation process provided in an embodiment of this application;

[0058] Figure 5 A flowchart illustrating another method for generating a 3D digital human model provided in an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of the structure of a 3D digital human model generation device provided in an embodiment of this application;

[0060] Figure 7A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0061] Figure 8 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0064] Figure 1 This is a flowchart illustrating a method for generating a 3D digital human model as provided in an embodiment of this application. (In conjunction with...) Figure 1 As shown, the 3D digital human model generation method provided in this application embodiment may include:

[0065] S101. Determine the initial parameter set of the 3D Gaussian ellipsoid on the scalable skinned multi-person linear model. The initial parameter set includes the initial position and initial attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0066] The Skinned Multi-Person Linear Model (SMPLX) is a parametric 3D human body model that allows for the simultaneous modeling of the dynamic shape and pose of the entire human body, including the face and hands. Typically, a SMPLX model has 10,475 vertices, which are the core components of the model and define the geometry of the human body's surface. Each vertex represents a specific point on the human body, such as the tip of the nose, a fingertip, or a knee.

[0067] In one implementation of this application, the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid. The process of obtaining the initial position is described below: In this embodiment, to introduce anatomical constraints during the digital human generation process and enable the generated 3D digital human model to support dynamic driving, the vertices of the SMPLX model are used as the initial position of the center point of the 3D Gaussian ellipsoid. Specifically, SMPLX is invoked, where SMPLX has 10475 vertices, and its vertex coordinates v... i Directly used as the center point c of the 3D Gaussian ellipsoid i The initial position, where v i ∈R 3 (i=1, 2, 3, ..., 10475), each vertex of the SMPLX standard mesh corresponds to the center point of a Gaussian ellipsoid, forming an initial Gaussian point cloud. The 3D Gaussian ellipsoid is a three-dimensional probability distribution model, defined by the covariance matrix, used to describe the density or attributes (such as color and transparency) of local regions in space. Its center point c... i This determines the position of the ellipsoid in space. The center point c serves as the core control point of the 3D Gaussian ellipsoid. i The location is directly related to the topological structure of the human body surface, ensuring the geometric rationality of the generated digital human.

[0068] The SMPLX possesses parametric characteristics, including: shape parameter β, pose parameter θ, and expression parameter ψ. The vertices of the SMPLX are used as the initial positions of the center points of the 3D Gaussian ellipsoid. Simultaneously, the parametric characteristics of the SMPLX allow the dynamic information of the human body to be implicitly embedded within the spatial distribution of the Gaussian ellipsoid. For example, when the SMPLX skeleton deforms through the pose parameter θ, the center point c of the Gaussian ellipsoid... i With the corresponding vertex v i Synchronous movement, no additional registration required.

[0069] Thus, by initializing the center point of the 3D Gaussian ellipsoid based on the SMSLX vertices, an initial point cloud strongly bound to the topology of the human body surface is constructed. Since the SMSLX vertices themselves are learned from a large amount of human scan data through principal component analysis, their distribution naturally conforms to the human anatomical structure. This allows the initialized Gaussian point cloud to directly inherit the structural rationality of limb connections and proportions, thereby avoiding the geometric penetration problem caused by the lack of topological constraints in traditional unstructured point clouds, such as an arm penetrating the torso. Simultaneously, by binding the center point of the 3D Gaussian ellipsoid to the SMSLX model, the spatial position of the Gaussian center point is directly controlled by the parameters of the SMSLX model. When the parameters are adjusted, the SMSLX model drives vertex deformation and synchronously moves the Gaussian center point. Without additional registration, the linear hybrid skinning algorithm can ensure smooth local deformation, achieving efficient digital human animation generation. Finally, this initialization based on prior human knowledge makes the position of the Gaussian center point close to the optimal solution of the distribution on the real human body surface. In subsequent generation processes, the diffusion model only needs to learn a small range of positional deviations to complete high-precision modeling, significantly reducing the difficulty of model training and data dependence.

[0070] Furthermore, the initial attribute parameters mentioned in this application are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid. The process of obtaining the attribute parameters is described below: In one implementation of this application, the attribute parameters of the Gaussian points need to be initialized first. The purpose of this operation is to provide a unified and clear starting state for the subsequent modeling and rendering process, ensuring that each Gaussian point has consistent attributes in the initial stage, thereby facilitating unified processing and adjustment later, and ultimately achieving effective representation and efficient rendering of complex 3D scenes.

[0071] In one implementation of this application, initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid specifically includes: initializing the scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0072] Specifically, the following fixed parameters need to be initialized synchronously for each Gaussian point: scaling parameter s i =[0.01,0.01,0.01], indicating that the initial size of the Gaussian point is a tiny sphere, which helps to make precise size adjustments as needed in subsequent operations; rotation parameter q i =[1,0,0,0], this is a unit quaternion, indicating that the Gaussian point is initially unrotated, providing a clear reference for subsequent rotation operations; the transparency parameter α i =0.8, meaning the Gaussian point is semi-transparent, which helps to better represent the layering and depth of objects when rendering complex scenes; spherical harmonic coefficient sh i ∈R 16=0, which means that the initial color of the Gaussian point is white by default, which makes it easier to adjust the color according to specific needs in subsequent operations.

[0073] Through the initialization process in step S101 above, the center point of the 3D Gaussian ellipsoid is bound to the SMSLX vertices, making the center point of the Gaussian ellipsoid closely associated with the SMSLX skeleton. This binding method provides the foundation for subsequent pose and facial animation by adjusting SMSLX parameters, thereby enabling the efficient generation of dynamic 3D animation effects. Furthermore, by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid, each Gaussian point has clear and consistent attribute parameters in the initial stage, laying a solid foundation for subsequent modeling and rendering processes.

[0074] After obtaining the initial position and initial attribute parameters, the initial attribute parameters and the initial position are used as the initial parameter set.

[0075] The initial attribute parameters include scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters. The scaling parameters include three values, for example, the initialized scaling parameter s. i =[0.01,0.01,0.01]; The rotation parameter includes four values, for example, the initialized rotation parameter q. i =[1,0,0,0]; The transparency parameter includes one value, such as the initialized transparency parameter α. i =0.8; the spherical harmonic coefficients include 16 values, such as the initialized spherical harmonic coefficients sh. i ∈R 16 =0. In addition, there is the initial position of the center point of the 3D Gaussian ellipsoid obtained based on SMPLX, which includes three values. These parameters, in total, comprise 27 values, used to comprehensively describe the characteristics of each Gaussian point, thus obtaining the initial parameter set P. init , where P init ∈R 10475×27 10475 indicates that there are a total of 10475 Gaussian ellipsoids, 27 indicates that each Gaussian point has 27 attribute parameters, and R indicates that these parameters are real numbers.

[0076] S102. Obtain the conditional feature vector of the target 3D digital human model.

[0077] The target 3D digital human model refers to the final 3D digital human model that the user hopes to generate. The specific features and style of this model depend on the user's input conditions, such as a reference image, a text description, or specific parameter settings. These input conditions are used to generate a matching conditional feature vector, which contains the key information and features needed to generate the target 3D digital human model.

[0078] The following section describes the process of obtaining the conditional feature vectors of the target 3D digital human model.

[0079] In one implementation of this application, the conditional feature vector is obtained based on the image or text input by the user. Figure 2 This application provides a schematic diagram of a process for obtaining the conditional feature vector of a target 3D digital human model, as illustrated in the embodiments of this application. Figure 2 As shown, the conditional feature vector for obtaining the target 3D digital human model includes:

[0080] S201. Obtain the expected conditions for generating the target 3D digital human model, the expected conditions including text description or images.

[0081] The process involves collecting user expectations for the target 3D digital human model, provided in the form of textual descriptions or images. Textual descriptions can be detailed descriptions of the 3D digital human model's appearance, posture, clothing, etc.; images can be reference photos or design drawings provided by the user, showcasing the desired visual effect of the 3D digital human model. These input conditions form the basis for generating the target 3D digital human model and determine the direction and focus of subsequent feature extraction.

[0082] S202, Extract feature vectors from the text description or image.

[0083] When extracting feature vectors, processing is performed based on a preset feature model.

[0084] Specifically, when extracting feature vectors from images, the ViT-L / 14 model is used to extract deep features from images at a fixed resolution, yielding the extraction results. The fixed resolution is set by those skilled in the art based on actual conditions; in this embodiment, the fixed resolution is set to 512×512. ViT-L / 14 is a vision model based on the Transformer architecture, capable of capturing deep features in images, such as shape, texture, and color. Adjusting the image to a fixed resolution ensures input consistency and model compatibility. The extracted deep feature vectors contain key visual information of the image and can be used in the subsequent model generation process.

[0085] When the user inputs a text description, the CLIP text encoder is used to extract the semantic feature vector of the text. The CLIP text encoder converts text into semantic feature vectors, which capture the meaning and contextual information of the text. The extracted semantic feature vectors... It contains a detailed description of the user's expectations for the 3D digital human model.

[0086] S203. When the expected condition is either a text description or an image, the conditional feature vector extracted based on the text description or the image shall be used as the conditional feature vector of the target 3D digital human model.

[0087] If the user only provides a text description or an image, then the extracted feature vector can be directly used. or This serves as the conditional feature vector for the target 3D digital human model. This means that the generated 3D digital human model will be constructed primarily based on a single input condition provided by the user, ensuring that the model accurately reflects the user's expectations expressed through text or images.

[0088] In one implementation of this application, the conditional feature vector is obtained based on the user-input image and text. Figure 3 A schematic diagram illustrating another process for obtaining the conditional feature vector of a target 3D digital human model provided in this application embodiment, as shown below. Figure 3 As shown, the conditional feature vector for obtaining the target 3D digital human model includes:

[0089] S301. Obtain the expected conditions for generating the target 3D digital human model, the expected conditions including text description and images.

[0090] This step is similar to S201, but the user provides both textual descriptions and images as input. This provides a richer source of information for the generated 3D digital human model; the textual descriptions can supplement details not explicitly expressed in the images, while the images provide intuitive visual references for the textual descriptions. Combining these two input conditions allows for a more comprehensive capture of the user's expectations, improving the accuracy and quality of the generated model.

[0091] S302, Extract feature vectors from the text description and image.

[0092] When extracting feature vectors from user-provided images, the ViT-L / 14 model is used to extract deep features from images at a fixed resolution, yielding the extraction results. The fixed resolution is set by those skilled in the art based on actual conditions; in this embodiment, the fixed resolution is set to 512×512. ViT-L / 14 is a vision model based on the Transformer architecture, capable of capturing deep features in images, such as shape, texture, and color. Adjusting the image to a fixed resolution ensures input consistency and model compatibility. The extracted deep feature vectors contain key visual information of the image and can be used in the subsequent model generation process.

[0093] Simultaneously, for the text description entered by the user, the CLIP text encoder is used to extract the semantic feature vector of the text. The CLIP text encoder converts text into semantic feature vectors, which capture the meaning and contextual information of the text. The extracted semantic feature vectors... It contains a detailed description of the user's expectations for the 3D digital human model.

[0094] By generating feature vectors of the same dimension using image encoders such as ViT-L / 14 and text encoders such as CLIP, and injecting these feature vectors into a diffusion model through the same path, image reconstruction and text generation can be achieved simultaneously within a single framework. This approach eliminates redundancy in modality transformation, improves model efficiency and consistency, and enables seamless integration of image and text generation, thus achieving more efficient and accurate generation results in multimodal tasks.

[0095] S303. The conditional feature vector extracted based on text and the conditional feature vector extracted based on image are fused together, and the fused conditional feature vector is used as the conditional feature vector of the target 3D digital human model.

[0096] When a user provides both text descriptions and images, it is necessary to combine the two feature vectors. and The fusion process yields the fused feature vector F. c The purpose of fusion is to comprehensively consider the visual information of the image and the semantic information of the text, so that the generated 3D digital human model can more fully reflect the user's expectations. The fusion method can be simple concatenation, weighted summation, or other more complex fusion strategies, depending on the specific settings used by those skilled in the art. The final obtained fusion feature vector F c This will serve as the conditional feature vector for the target 3D digital human model, guiding the subsequent model generation process.

[0097] Through the above steps, this application can accurately extract conditional feature vectors based on user input conditions, including image or text descriptions, or a combination of both, providing key feature information for subsequent generation of the target 3D digital human model. This method not only supports single input conditions but can also comprehensively process multiple input conditions, improving the flexibility and accuracy of the generated model.

[0098] S103. The initial parameter set and the conditional feature vector are processed using a diffusion model to obtain the parameter deviations corresponding to each initial parameter.

[0099] Processing the initial parameter set and the conditional feature vectors requires the use of a diffusion model. A diffusion model is a generative model that generates data by progressively adding noise and then recovering the data by removing the noise through the reverse process. An autoencoder is a key factor in accelerating training and inference in the diffusion model. Internally, it consists of an encoder and a decoder. The encoder encodes high-dimensional feature vectors into a latent space, where the feature dimension is much smaller than the original dimensional space. A diffusion operation is then performed in the latent space, and finally, the decoder returns the data from the latent space to the original high-dimensional space.

[0100] Specifically, the initial parameter set is reduced to a latent space using an autoencoder based on a diffusion model. Within this latent space, the reduced initial parameter set and the conditional feature vector are processed to obtain the parameter bias. This process, by mapping the initial parameter set to a low-dimensional latent space, reduces computational complexity and improves processing efficiency.

[0101] The following is combined with Figure 4 The process of generating parameter deviations will be introduced. Figure 4 This is a flowchart illustrating a parameter deviation generation process provided in an embodiment of this application. Figure 4 As shown, the process includes:

[0102] S401. The encoder of the autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters.

[0103] An autoencoder is a neural network structure used for data dimensionality reduction and feature extraction, consisting of an encoder and a decoder. In this embodiment, the encoder converts the high-dimensional initial parameter set P... init ∈R 10475×27 Dimensionality reduction to the latent space yields low-dimensional initial parameters z. init , z init ∈R 1×256 This process significantly reduces the dimensionality of the parameters, thereby improving computational efficiency and making subsequent diffusion operations more efficient. The autoencoder takes a high-dimensional vector of 10475×27 as input and outputs a low-dimensional vector of 1×256. In this way, the tens of thousands of-dimensional P... init The parameters are compressed into a 256-dimensional latent space to obtain z. init The specific formula is: z init =Encoder(P init ).

[0104] S402. Perform forward noise processing on the initial parameters in the latent space to obtain the initial parameters after adding noise.

[0105] In the diffusion model, the forward noise addition process involves progressively adding noise to the initial parameters, gradually blurring them. This process takes place in the latent space, specifically by progressively adding Gaussian noise to the dimensionality-reduced initial parameters over 1000 steps (T=1000). During this process, the dimensionality-reduced initial parameters z... init Let z0 be the denoted z0, z0∈R 256 =z init After processing, the final parameter z with added noise is obtained. t , z t The noise follows a standard normal distribution N(0,I), where I is the identity matrix, indicating that the noise is independent and identically distributed. This process provides the basis for subsequent inverse denoising.

[0106] S403. Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters.

[0107] The inverse denoising process is the core of the diffusion model, aiming to recover clear parameters from noisy parameters. In this embodiment, the DiT (Diffusion Transformer) architecture is used to generate biases in the latent space. Specifically, the conditional feature F... c Using the time step t as a guide, predict noise This process is implemented using the CrossAttn module, with the specific formula as follows: Where Q is the query matrix, K is the key matrix, and V is the value matrix. The main function of the cross-attention module is to inject the semantic information extracted from the text and images into the generation of the diffusion model, enabling the model to generate data based on the given images and text.

[0108] The final parameter deviations generated through the diffusion process include the center coordinate deviation (δ). center ), scaling deviation (δ) scale ), rotational deviation (δ) rotation ), transparency deviation (δ) opacity ), spherical harmonic coefficient deviation (δ) sh ).

[0109] S404. Using the decoder of the autoencoder, the parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before the dimensionality reduction process, and the parameter deviation is obtained.

[0110] The final result of the latent space generated by the diffusion model is δ. z Then, the decoder in the autoencoder will convert δ zMapping back to the original parameter space yields the high-dimensional coefficient bias δ. This process restores the low-dimensional parameter bias to the same dimension as the initial parameter set for subsequent parameter updates. The specific formula is: δ = Decoder(δ... z ).

[0111] In the above process, the diffusion model, through pre-trained priors, can complete areas invisible in a single view, such as the back and hairstyle, solving the problem of missing details, improving cross-modal capabilities, and significantly enhancing training stability. Since the variance of the bias is much smaller than the variance of the absolute parameters, this makes the training process more stable and accelerates convergence. Experimental verification shows that the convergence speed is improved by more than 40%. By predicting incremental biases instead of absolute parameters, the diffusion model can converge faster, reducing training time and computational resource consumption. Furthermore, the diffusion model provides a unified framework that supports both image and text input. Text conditions align the semantic space through CLIP, enabling the model to better understand and generate 3D digital human models that match the text descriptions. By processing the initial parameter set and conditional feature vectors in the latent space, features from different modalities can be better integrated, improving the quality and consistency of the generated results.

[0112] S104. Update the parameters in the initial parameter set according to the parameter deviation, and construct the target 3D digital human model using the updated parameters.

[0113] The step of updating the parameters in the initial parameter set according to the parameter deviation includes performing a one-to-one summation operation between each parameter in the initial parameter set and its corresponding parameter deviation to complete the update of the initial parameters, i.e., the final 3D Gaussian parameters P. final From the initial parameter P init It is obtained by adding the generated deviation δ. The specific formula is: P final =P init +δ. For the location of the center point of the Gaussian ellipsoid, the final center point = SMPLX vertex position + δ. center For other initial parameters, including scaling, rotation, transparency, and spherical harmonic coefficient deviation, the final parameter = initial parameter + corresponding parameter deviation.

[0114] Finally, the updated parameter set P is used. final A 3D digital human model is constructed, and the parameters of each Gaussian point, such as position, scale, rotation, transparency, and spherical harmonic coefficients, are adjusted according to the updated parameters.

[0115] In this way, by adding the initial parameters to the generated deviations, the parameters of the 3D digital human model are accurately updated. This update method not only improves the model's detail but also ensures that the 3D digital human model generated based on the updated parameters is more realistic and natural in geometry and appearance.

[0116] The above embodiments describe a method for generating a 3D digital human model provided by this application. Based on this, this application also provides another method for generating a 3D digital human model, which is applicable to scenarios where, after constructing a 3D digital human model, it is necessary to render the constructed 3D digital human model. Figure 5 A flowchart illustrating another 3D digital human model generation method provided in this application embodiment, combined with... Figure 5 As shown, the method includes:

[0117] S501. Determine the initial parameter set of the 3D Gaussian ellipsoid on the scalable skinned multi-person linear model. The initial parameter set includes the initial position and initial attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0118] S502. Obtain the conditional feature vector of the target 3D digital human model.

[0119] S503. The initial parameter set and the conditional feature vector are processed using a diffusion model to obtain the parameter deviations corresponding to each initial parameter.

[0120] S504. Update the parameters in the initial parameter set according to the parameter deviation, and construct the target 3D digital human model using the updated parameters.

[0121] The implementation process of steps S501-S504 is similar to that of steps S101-104 above, and can be referred to the above implementation process. It will not be repeated here.

[0122] S505. Render the 3D digital human model into a 2D image using a 3D Gaussian sputtering renderer.

[0123] Specifically, all Gaussian points are sorted by depth based on the camera's viewpoint. In a 3D scene, the camera's viewpoint defines the observer's position and orientation. It determines from which angle the 3D model is viewed and how it is projected onto a 2D plane. The purpose of depth sorting is to ensure that Gaussian points closer to the camera are rendered first during the rendering process, thus correctly handling occlusion relationships.

[0124] Each Gaussian point is then projected as a 2D ellipse. Since each Gaussian point has attributes such as position, orientation, and size in 3D space, these Gaussian points are projected onto a 2D plane during rendering, forming 2D ellipses. The shape and size of the ellipse depend on the properties of the Gaussian point and the camera's viewpoint. Further, rasterization is performed to calculate the pixel coverage, i.e., calculating the coverage area of ​​each 2D ellipse on the pixel plane. Rasterization is the process of converting continuous geometry into discrete pixels; this step ensures the correct representation of each Gaussian point in the 2D image.

[0125] Next, the viewpoint-dependent RGB colors are calculated based on spherical harmonics, and alpha blending is performed in depth order. This involves calculating the RGB color of each Gaussian point at the current viewpoint using spherical harmonics. For example, the color of Gaussian point C is (255, 200, 150), the color of Gaussian point A is (150, 200, 255), and the color of Gaussian point B is (100, 150, 200). Alpha blending of these colors in depth order ensures the accuracy of the rendering result. Finally, a 2D image is generated using the 3D Gaussian Sputtering (3DGS) engine, which supports gradient backpropagation, allows end-to-end training, and can optimize rendering parameters to improve image quality and rendering efficiency.

[0126] Regarding the end-to-end training strategy, all network models used in this application employ end-to-end training, and the loss function consists of the diffusion loss L. diff Rendering loss L render And autoencoder reconstruction loss L ae It consists of three parts. The diffusion loss is responsible for supervising latent space noise prediction; the rendering loss includes L1 loss to maintain pixel accuracy and LPIPS perceptual loss to enhance visual details; and the autoencoder reconstruction loss ensures the reversibility of the latent space. Training optimization techniques include: progressive training, first fixing the autoencoder and training only the diffusion model, then jointly optimizing all modules; and conditional dropout, randomly discarding the conditional feature vector F with a 30% probability. c To enhance model robustness, geometric regularization is used to add constraints that ensure the deviation of the generated center point |δcenter|2 < 0.1, preventing excessive deviation from the human body topology. The combined application of these strategies and techniques further improves the model's performance and robustness, enabling the generated 3D digital human model to better conform to the conditions and features of user input, thereby achieving high-quality generation results in various application scenarios.

[0127] This end-to-end differentiable high-efficiency rendering pipeline can connect to a differentiable Gaussian sputtering renderer after 3D Gaussian parameter reconstruction, jointly optimizing diffusion loss and pixel-level rendering loss. Since 3D Gaussian sputtering supports GPU rasterization, it can achieve >60 FPS (consumer-grade graphics cards) at 1080p resolution, significantly improving rendering efficiency. Furthermore, end-to-end training enables the diffusion model to perceive the rendering results, avoiding the disconnect between geometry and appearance, ensuring the coherence and realism of the generated model. Finally, explicit Gaussian representation allows for modeling particle effects (such as hair strands and veils), enabling the model to more accurately represent complex geometric structures and appearance effects.

[0128] The above are some specific implementations of a 3D digital human model generation method provided in the embodiments of this application. The following describes some application scenarios of the 3D digital human model generation method in this application.

[0129] Scenario 1: Real-time generation of virtual idols or streamers

[0130] First, input a single real photo; the user uploads a real photo.

[0131] Then, a high-precision 3D digital human is generated using the method described in the above embodiments of this application: Using the method described in the above application (steps S501-S505), the vertices of the SMPLX are used as the initial positions of the center points of the 3D Gaussian ellipsoid. The attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid are initialized, and these parameters and positions are used as the initial parameter set. The conditional feature vector of the target 3D digital human model is obtained. The initial parameter set and the conditional feature vector are processed to obtain parameter deviations, and the initial parameters are updated based on these deviations to construct the 3D digital human model. Finally, the 3D digital human model is rendered into a 2D image using a 3D Gaussian sputtering renderer, generating a high-precision 3D digital human.

[0132] After generating a high-precision 3D digital human, expressions / movements can be driven by adjusting the parameters of the SMSLX model. Specifically, the parameter adjustment function of the SMSLX model can be used to drive the expressions and movements of the generated 3D digital human in real time, such as blinking and waving.

[0133] It can also be integrated into live streaming based on 3D Gaussian real-time rendering (60FPS+), allowing the real-time rendered 3D digital human to be integrated into the live streaming at a frame rate of 60FPS or higher, enabling real-time interaction between virtual idols or streamers.

[0134] In this application scenario, the method in this application can achieve real-time rendering, supporting frame rates of over 60 FPS to ensure smooth live streaming; by adjusting the SMPLX parameters, rich expressions and movements can be achieved, improving the interactivity and expressiveness of virtual idols / anchors.

[0135] Scenario 2: Batch generation of game characters

[0136] The user inputs a text description, such as "medieval female warrior, armor, red hair," and then, through steps S501-S505 in the above embodiments of this application, a 3D character matching the description is generated. The generated 3D character model is then automatically bound to the game engine's skeletal system to achieve character animation and interaction. This allows for the rapid generation of stylized 3D characters, improving game development efficiency. Because it can be automatically bound to the game engine, the development process for character animation and interaction is simplified.

[0137] Scene 3: Virtual Fitting Room

[0138] Users upload their own photos, and personalized 3D human body models are generated through steps S501-S505 in the above embodiments of this application. A 3D clothing Gaussian model provided by the clothing merchant is overlaid onto the generated 3D human body model. A 3D Gaussian sputtering renderer is used to render the try-on effect in real time, supporting fabric physics simulation to ensure the realism of the try-on effect. Real-time rendering of the try-on effect and support for fabric physics simulation improve the user experience. Furthermore, the generation of personalized 3D human body models ensures a good fit and realistic appearance during the try-on process.

[0139] Scenario 4: AR Virtual Shopping Assistant

[0140] Users input their product needs via voice, such as "recommend sunglasses suitable for the beach." Through steps S501-S505 in the above embodiments of this application, a digital persona with recommended products is generated. Using a device such as mobile AR technology, the effect of the digital persona trying on the recommended products is displayed, allowing users to intuitively see the appearance and suitability of the products. In this way, by utilizing AR technology, an immersive shopping experience can be provided, increasing users' willingness to purchase. Through real-time rendering and personalized recommendations, it is ensured that the product effects seen by the user match their needs and preferences.

[0141] Through the introduction of these application scenarios, it can be seen that the method in this application has broad application prospects and significant advantages in areas such as real-time generation of virtual idols / anchors, batch generation of game characters, virtual fitting rooms, and AR virtual shopping assistants.

[0142] The above are some specific implementations of a 3D digital human model generation method provided in the embodiments of this application. Based on this, the present application also provides a corresponding apparatus. The apparatus provided in the embodiments of this application will be described below from the perspective of functional modularity.

[0143] Figure 6 This is a schematic diagram of a 3D digital human model generation device provided in an embodiment of this application. (Combined with...) Figure 6 As shown, the 3D digital human model generation device 600 provided in this application embodiment includes:

[0144] Initialization unit 610 is used to determine the initial parameter set of the 3D Gaussian ellipsoid on the scalable skinned multi-person linear model. The initial parameter set includes the initial position and initial attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0145] The acquisition unit 620 is used to acquire the conditional feature vector of the target 3D digital human model;

[0146] The parameter deviation generation unit 630 is used to process the initial parameter set and the conditional feature vector using a diffusion model to obtain the parameter deviation corresponding to each initial parameter; the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid; the initial attribute parameters are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0147] Parameter update unit 640 is used to update the parameters in the initial parameter set according to the parameter deviation;

[0148] Model building unit 650 is used to build a 3D digital human model using updated initial parameters.

[0149] In one implementation of this application, the initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid.

[0150] The initial attribute parameters are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0151] In one implementation of this application, the apparatus further includes a rendering unit, used to render the constructed 3D digital human model into a 2D image based on a 3D Gaussian sputtering renderer when it is necessary to render the constructed 3D digital human model.

[0152] In one implementation of this application, the initialization unit initializes the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid, including:

[0153] Initialize the scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters of all Gaussian points on the 3D Gaussian ellipsoid.

[0154] In one implementation of this application, the acquisition unit is specifically used to acquire the expected conditions for generating the target 3D digital human model, the expected conditions including at least one of text description or image;

[0155] Determine the type of the expected condition. When the expected condition is a text description or an image, use the condition feature vector extracted based on the text description or the image as the condition feature vector of the target 3D digital human model.

[0156] When the expected conditions are text descriptions and images, the conditional feature vectors extracted based on the text and the conditional feature vectors extracted based on the images are fused, and the fused conditional feature vectors are used as the conditional feature vectors of the target 3D digital human model.

[0157] In one implementation of this application, the parameter deviation generation unit is specifically used to reduce the dimensionality of the initial parameter set to the latent space using the autoencoder of the diffusion model, and to process the dimensionality-reduced initial parameter set and the conditional feature vector in the latent space to obtain the parameter deviation.

[0158] In one implementation of this application, the initial parameter set is reduced to a latent space using an autoencoder based on a diffusion model. The reduced initial parameter set and the conditional feature vector are then processed in the latent space to obtain the parameter deviation, which includes:

[0159] The autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters.

[0160] The initial parameters in the latent space are subjected to forward noise addition to obtain the initial parameters with added noise.

[0161] Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters.

[0162] The parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before dimensionality reduction using the decoder of the autoencoder, thus obtaining the parameter deviation.

[0163] In one implementation of this application, the parameter update unit is specifically used to perform a one-to-one summation operation between each parameter in the initial parameter set and its corresponding parameter deviation to complete the update of the initial parameters.

[0164] Furthermore, the digital human model generation method provided in this application can be applied to computer devices, see [link to relevant documentation]. Figure 7 This figure is a schematic diagram of an application scenario provided by an embodiment of this application. In some embodiments, the computer device may include, but is not limited to, mobile phones, tablets, laptops, handheld computers, wearable electronic devices, smartwatches, etc., and this application embodiment does not impose special restrictions on the specific form of the above-mentioned electronic devices.

[0165] In this embodiment of the application, computer device 701 can interact with server 702. Computer device 701 can respond to user operations, obtain the expected conditions for generating the target 3D digital human model, and send them to server 702.

[0166] After receiving the expected conditions for generating the target 3D digital human model, server 702 obtains the conditional feature vector of the target 3D digital human model; it processes the initial parameter set and the conditional feature vector using a diffusion model to obtain the parameter deviations corresponding to each initial parameter; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on the association between a scalable skinned multi-person linear model and a 3D Gaussian ellipsoid, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid; the parameters in the initial parameter set are updated according to the parameter deviations, and the target 3D digital human model is constructed using the updated parameters. After generating the 3D digital human model, server 702 can send the 3D digital human model to computer device 701 so that computer device 701 can display the 3D digital human model to the user. It is understood that the server can be deployed separately from the computer device, or the server can be integrated into the computer device; this embodiment does not specifically limit this.

[0167] This application also provides corresponding computer devices, such as... Figure 8 As shown in the figure, this figure is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor 801 and a memory 802.

[0168] The memory 802 is used to store program code and transmit the program code to the processor 801; the processor 801 is used to execute the steps of the digital human model generation method described above according to the instructions in the program code.

[0169] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a digital human model generation method as described above.

[0170] It is understood that in the specific embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved need to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0171] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0172] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating a digital human model, characterized in that, The method includes: Obtain the conditional feature vector of the target 3D digital human model; The initial parameter set and the conditional feature vector are processed using a diffusion model to obtain the parameter deviations corresponding to each initial parameter; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on a scalable skinned multi-person linear model and a 3D Gaussian ellipsoid association, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid; The parameters in the initial parameter set are updated according to the parameter deviation, and the target 3D digital human model is constructed using the updated parameters. The process of using the diffusion model to process the initial parameter set and the conditional feature vector yields the parameter deviations corresponding to each initial parameter, including: The initial parameter set is reduced to a latent space using an autoencoder based on a diffusion model. The reduced initial parameter set and the conditional feature vector are then processed in the latent space to obtain the parameter bias. The autoencoder using the diffusion model reduces the dimensionality of the initial parameter set to a latent space. In this latent space, the reduced initial parameter set and the conditional feature vector are processed to obtain the parameter bias, which includes: The autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters. The initial parameters in the latent space are subjected to forward noise addition to obtain the initial parameters with added noise. Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters. The parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before dimensionality reduction using the decoder of the autoencoder, thus obtaining the parameter deviation.

2. The method according to claim 1, characterized in that, The initial position in the initial parameter set is obtained by using the vertices of the scalable skinned multi-person linear model as the initial position of the center point of the 3D Gaussian ellipsoid; The initial attribute parameters are obtained by initializing the attribute parameters of all Gaussian points on the 3D Gaussian ellipsoid.

3. The method according to claim 2, characterized in that, The attribute parameters for initializing all Gaussian points on the 3D Gaussian ellipsoid include: Initialize the scaling parameters, rotation parameters, transparency parameters, and spherical harmonic coefficient parameters of all Gaussian points on the 3D Gaussian ellipsoid.

4. The method according to claim 1, characterized in that, The conditional feature vector for obtaining the target 3D digital human model includes: Obtain the expected conditions for generating the target 3D digital human model, the expected conditions including at least one of textual description or image; Determine the type of the expected condition. When the expected condition is a text description or an image, use the condition feature vector extracted based on the text description or the image as the condition feature vector of the target 3D digital human model. When the expected conditions are text descriptions and images, the conditional feature vectors extracted based on the text and the conditional feature vectors extracted based on the images are fused, and the fused conditional feature vectors are used as the conditional feature vectors of the target 3D digital human model.

5. The method according to claim 1, characterized in that, The step of updating the parameters in the initial parameter set according to the parameter deviation includes: Each parameter in the initial parameter set is summed with its corresponding parameter deviation to update the initial parameters.

6. A digital human model generation device, characterized in that, The device includes: The acquisition unit is used to acquire the conditional feature vector of the target 3D digital human model; The parameter deviation generation unit is used to process the initial parameter set and the conditional feature vector using a diffusion model to obtain the parameter deviation corresponding to each initial parameter; the initial parameter set includes initial position and initial attribute parameters; wherein, the initial position is determined based on the association between the scalable skinned multi-person linear model and the 3D Gaussian ellipsoid, and the initial attribute parameters are the attribute parameters of Gaussian points on the 3D Gaussian ellipsoid; The parameter update unit is used to update the parameters in the initial parameter set according to the parameter deviation; A model building unit is used to construct the target 3D digital human model using the updated initial parameters; The parameter deviation generation unit is specifically used to reduce the dimensionality of the initial parameter set to the latent space using the autoencoder of the diffusion model, and to process the dimensionality-reduced initial parameter set and the conditional feature vector in the latent space to obtain the parameter deviation. The autoencoder using the diffusion model reduces the dimensionality of the initial parameter set to a latent space. In this latent space, the reduced initial parameter set and the conditional feature vector are processed to obtain the parameter bias, which includes: The autoencoder is used to reduce the dimensionality of the data in the initial parameter set to the latent space to obtain the dimensionality-reduced initial parameters. The initial parameters in the latent space are subjected to forward noise addition to obtain the initial parameters with added noise. Based on the conditional feature vector and the preset time step, the initial parameters after adding noise are subjected to inverse denoising processing using an attention mechanism to obtain the parameter deviation of the dimensionality-reduced initial parameters. The parameter deviation of the initial parameters after dimensionality reduction is restored to the dimension before dimensionality reduction using the decoder of the autoencoder, thus obtaining the parameter deviation.

7. A computer device, characterized in that, The computer device includes: a memory and a processor; The memory is used to store program code and transmit the program code to the processor; the processor is used to execute the steps of the digital human model generation method as described in any one of claims 1-5 according to the instructions in the program code.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the digital human model generation method as described in any one of claims 1 to 5.

Images