Digital human generation method and related device

Abstract

The embodiment of the invention provides a digital human generation method and a related device, and the method comprises the steps: generating a first grid model based on human body data in three-dimensional data, collected from a plurality of perspectives, of a user; generating a second grid model based on clothing data in the three-dimensional data; performing feature fusion on the first grid model and the second grid model to generate a third grid model; determining digital human parameters in a motion state based on the third grid model; the digital human parameters comprise action parameters and clothing wrinkle parameters corresponding to the action parameters; and generating a digital person based on the digital person parameters in the motion state. According to the embodiment of the invention, in the process of generating the digital human in combination with the action parameter and the clothing wrinkle parameter corresponding to the action parameter, the authenticity of the clothing wrinkle of the digital human can be improved, the display effect of complex clothing is ensured, and the overall display effect of the digital human is improved.

Description

Technical Field

[0001] This application relates to the field of digital human generation technology, and in particular to a digital human generation method and related apparatus. Background Technology

[0002] Digital humans, also known as virtual humans or digitized characters, refer to virtual characters that exist in digital space in digital form and have human-like or real-life appearances, behaviors, and characteristics.

[0003] Current digital human generation technology can reconstruct human posture, shape, and texture. However, it is not effective in reconstructing digital human clothing. For example, the wrinkles in the clothes of the generated digital human are not realistic, and the long skirt is damaged, which reduces the overall display effect of the digital human.

[0004] Therefore, there is an urgent need for a solution to address the aforementioned technical problems. Summary of the Invention

[0005] In view of the above problems, this application provides a digital human generation method and related apparatus, which aims to improve the overall display effect of digital humans.

[0006] The embodiments of this application disclose the following technical solutions: Firstly, this application provides a method for generating a digital human, comprising: generating a first mesh model based on human body data in three-dimensional data of a user collected from multiple perspectives; generating a second mesh model based on clothing data in the three-dimensional data; fusing features of the first mesh model and the second mesh model to generate a third mesh model; determining digital human parameters in motion based on the third mesh model; the digital human parameters including action parameters and clothing fold parameters corresponding to the action parameters; and generating a digital human based on the digital human parameters in motion.

[0007] In this embodiment, a third mesh model is generated by fusing features from a first mesh model representing human body characteristics with a second mesh model representing the user's clothing. Based on this third mesh model, a digital human is generated using parameters for a digital human in motion. These parameters include motion parameters and corresponding clothing fold parameters. In this embodiment, the process of generating a digital human by combining motion parameters and corresponding clothing fold parameters improves the realism of the folds in the digital human's clothing, ensuring the display effect of complex clothing and thus enhancing the overall display effect of the digital human.

[0008] In one possible implementation, determining the digital human parameters in motion based on the third mesh model includes: mapping the third mesh model onto a two-dimensional image plane to generate a pose mapping map; and determining the digital human parameters in motion based on the pose mapping map and the pose parameters of the first mesh model.

[0009] In this embodiment, by mapping the third mesh model onto a two-dimensional image plane, the posture details of the digital human in three-dimensional space can be presented intuitively and accurately in two-dimensional space, avoiding the complex geometric relationships and calculation errors that may occur when directly processing them in three-dimensional space. Based on this, and using the posture parameters of the posture mapping map and the first mesh model, the parameters of the digital human in motion are determined. This allows for more precise determination of the parameters of various parts of the digital human in motion postures, such as joint angles and limb positions, making the posture simulation of the digital human closer to reality and improving the accuracy of posture simulation. This, in turn, facilitates subsequent improvements to the overall display effect of the digital human.

[0010] In one possible implementation, determining the digital human parameters in motion based on the pose mapping map and the pose parameters of the first mesh model includes: inputting the pose mapping map and the pose parameters of the first mesh model into an RGBC fully convolutional network to obtain a Gaussian sputtering attribute map; and determining the digital human parameters in motion based on the attribute data, color data, rotation data, and scale data included in the Gaussian sputtering attribute map.

[0011] In this embodiment, the pose mapping map contains the projection information of the digital human's pose onto a two-dimensional plane, and the pose parameters of the first mesh model can reflect different motion characteristics of the digital human. Based on the RGBC fully convolutional network with powerful feature extraction capabilities, deep features from these two types of data can be extracted and effectively fused. Through this deep feature analysis and feature fusion, subtle changes in the digital human's pose can be accurately captured, thereby generating a high-precision Gaussian sputtering attribute map. The Gaussian sputtering attribute map includes attribute data, color data, rotation data, and scale data. Based on this Gaussian sputtering attribute map, the parameters of the digital human in motion can be accurately determined, facilitating subsequent improvements to the overall display effect of the digital human.

[0012] In one possible implementation, generating a digital human based on the digital human parameters in the motion state includes: generating Gaussian points on the surface of the third mesh model; and converting the Gaussian points from a static pose to a motion pose based on the digital human parameters in the motion state to generate the digital human.

[0013] In this embodiment, by generating Gaussian points on the surface of the third mesh model, subtle features and geometric shapes of the surface can be captured. Compared to traditional continuous surface representation methods, Gaussian points can more flexibly adapt to complex surface morphologies, thus providing a more refined modeling foundation for the digital human and making its appearance more realistic. Furthermore, based on the digital human's parameters in motion, converting the Gaussian points from static poses to dynamic poses ensures the realism of clothing folds in different motion states, improves the display effect of complex clothing, and ultimately enhances the overall display effect of the digital human.

[0014] In one possible implementation, the step of fusing features of the first mesh model and the second mesh model to generate a third mesh model includes: spatially aligning the first mesh model and the second mesh model to obtain an aligned first mesh model and an aligned second mesh model; and fusing features of the aligned first mesh model and the aligned second mesh model to generate a third mesh model.

[0015] In this embodiment, by spatially aligning the first mesh model and the second mesh model, the first mesh model and the second mesh model can be placed in a unified spatial coordinate system. Based on this, feature fusion of the aligned first mesh model and the aligned second mesh model can effectively improve the efficiency of generating the third mesh model.

[0016] In one possible implementation, the step of fusing features of the aligned first mesh model and the aligned second mesh model to generate a third mesh model includes adding an offset to the surface of the aligned first mesh model using the vertex coordinates in the aligned second mesh model as the offset target to generate a fourth mesh model; and dynamically adjusting the fourth mesh model based on the pose parameters of the aligned first mesh model to generate the third mesh model.

[0017] In this embodiment, by adding an offset to the surface of the aligned first mesh model and using the vertex coordinates in the aligned second mesh model as the offset target, the geometric details of fine structures such as clothing and hair can be supplemented. Based on this, combined with the posture parameters in the aligned first mesh model, the fourth mesh model is dynamically adjusted to ensure that parts such as clothing and hair can produce natural deformation during movement, thereby improving the display effect of the subsequently generated digital human.

[0018] In one possible implementation, generating a first mesh model based on human body data in the user's 3D data collected from multiple perspectives includes: processing the human body data in the user's 3D data collected from multiple perspectives using the 3D Human Body (SMPL eXpressive, SMPLX) algorithm to generate the first mesh model.

[0019] The SMPLX algorithm, based on large-scale human data statistics and learning, can meticulously capture the morphological features of various parts of the human body, including skeletal structure, muscle distribution, and body surface contours. In this embodiment, the SMPLX algorithm improves the accuracy of the generated first mesh model, facilitating subsequent enhancements to the overall display effect of the digital human.

[0020] In one possible implementation, generating a second mesh model based on clothing data in the three-dimensional data includes: processing the clothing data in the three-dimensional data using the Signed Distance Field (SDF) algorithm and the DeepMarching Tetrahedra (DMT) algorithm to generate the second mesh model.

[0021] The SDF algorithm accurately describes the geometry of clothing by calculating the directed distance from each point in space to the clothing surface, capturing subtle contour changes such as collar folds, cuff tightening, and skirt hem undulations. The DMT algorithm handles dynamic deformations of clothing caused by human movement during wear, such as stretching, compression, and folding. This embodiment combines the SDF and DMT algorithms to improve the accuracy of the generated second mesh model.

[0022] Second aspect: Embodiments of this application provide a digital human generation device, including: The unit comprises a first generation unit, a feature fusion unit, a determination unit, and a second generation unit. The first generation unit is used to generate a first mesh model based on human body data in the three-dimensional data of the user collected from multiple perspectives; and to generate a second mesh model based on clothing data in the three-dimensional data. The feature fusion unit is used to fuse the features of the first mesh model and the second mesh model to generate a third mesh model; The determining unit is used to determine the digital human parameters in motion state based on the third grid model; the digital human parameters include motion parameters and clothing fold parameters corresponding to the motion parameters; The second generation unit is used to generate a digital human based on the digital human parameters in the motion state.

[0023] In one possible implementation, the determining unit is specifically used for: The third mesh model is mapped onto a two-dimensional image plane to generate a pose mapping map; Based on the posture mapping and the posture parameters of the first mesh model, the parameters of the digital human in motion state are determined.

[0024] In one possible implementation, the determining unit is specifically used for: The attitude mapping map and the attitude parameters of the first mesh model are input into an RGBC fully convolutional network to obtain a Gaussian sputtering property map; Based on the attribute data, color data, rotation data, and scale data included in the Gaussian sputtering attribute map, the parameters of the digital human in motion state are determined.

[0025] In one possible implementation, the generating unit is specifically used for: Gaussian points are generated on the surface of the third mesh model; Based on the digital human parameters under the motion state, the Gaussian point is converted from a static pose to a motion pose to generate a digital human.

[0026] In one possible implementation, the feature fusion unit is specifically used for: The first mesh model and the second mesh model are spatially aligned to obtain the aligned first mesh model and the aligned second mesh model; The aligned first mesh model and the aligned second mesh model are feature fused to generate a third mesh model.

[0027] In one possible implementation, the feature fusion unit is specifically used for: Using the vertex coordinates in the aligned second mesh model as the offset target, an offset is added to the surface of the aligned first mesh model to generate a fourth mesh model; based on the pose parameters of the aligned first mesh model, the fourth mesh model is dynamically adjusted to generate a third mesh model.

[0028] In one possible implementation, the first generating unit is specifically used for: The SMPLX algorithm is used to process human body data in the 3D data of users collected from multiple perspectives to generate the first mesh model.

[0029] In one possible implementation, the first generating unit is specifically used for: The clothing data in the 3D data is processed using the SDF and DMT algorithms to generate a second mesh model.

[0030] Third aspect: This application provides a computer device, which includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of a digital human generation method as described above, according to the instructions in the program code.

[0031] Fourth aspect: This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a digital human generation method as described above.

[0032] Fifth aspect: This application provides a computer program product, which, when run on a computer, executes the steps of a digital human generation method as described above.

[0033] Sixth aspect: This application provides a chip including a processor coupled to a memory for executing a computer program or instructions stored in the memory, such that the chip implements the steps of a digital human generation method as described above. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram illustrating an application scenario of a digital human generation method provided in an embodiment of this application; Figure 2 A flowchart illustrating a digital human generation method provided in this application embodiment; Figure 3 This application provides a schematic diagram of the structure of a digital human generation device according to an embodiment of the present application; Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application.

[0037] The "multiple" mentioned in the embodiments of this application refers to two or more. It should be noted that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0038] Current digital human generation technology can reconstruct human posture, shape, and texture. However, it is not effective in reconstructing digital human clothing. For example, the wrinkles in the clothes of the generated digital human are not realistic, and the long skirt is damaged, which reduces the overall display effect of the digital human.

[0039] For example, currently, using a 3D human body (SMPL eXpressive, SMPLX) model as the geometric prior, and optimizing through multi-view images, it is possible to reconstruct the human body's pose, shape, and texture. However, the SMPLX model only includes the human torso, limbs, and facial features, and does not contain the geometric structure and topological information of clothing. This results in a lack of reasonable geometric priors for complex clothing (such as long skirt areas) during the Gaussian sputtering initialization stage. In this case, clothing can only be reconstructed by relying on sparse point clouds or random initialization, resulting in blurred clothing displays with unclear edges.

[0040] Meanwhile, existing digital human generation methods struggle to reproduce the wrinkles and folds of real clothing during movement (such as long skirts and clothing swaying), which reduces the overall display quality of digital humans.

[0041] Based on this, this application provides a digital human generation method. A third mesh model is generated by fusing features from a first mesh model representing the human body and a second mesh model representing the user's clothing. Based on the third mesh model, a digital human is generated according to digital human parameters in motion, including motion parameters and corresponding clothing fold parameters. In this application embodiment, by combining motion parameters and corresponding clothing fold parameters to generate the digital human, the realism of the folds in the digital human's clothing can be improved, ensuring the display effect of complex clothing, thereby improving the overall display effect of the digital human.

[0042] like Figure 1 As shown in the figure, this is a schematic diagram of an application scenario for a digital human generation method provided in an embodiment of this application. In this embodiment, the digital human generation method can be implemented by interacting between the acquisition device 101 and the digital human generation device 102.

[0043] The acquisition device 101 acquires the user's three-dimensional data from multiple perspectives and sends the user's three-dimensional data to the digital human generation device 102.

[0044] The digital human generation device 102 generates a first mesh model based on human body data in the three-dimensional data of the user collected from multiple perspectives; generates a second mesh model based on clothing data in the three-dimensional data; fuses the features of the first mesh model and the second mesh model to generate a third mesh model; and determines the digital human parameters in motion state based on the third mesh model, wherein the digital human parameters include action parameters and clothing fold parameters corresponding to the action parameters.

[0045] The digital human generation device 102 generates a digital human based on the digital human parameters in the motion state.

[0046] To facilitate understanding, the following will be combined with Figure 2 This application introduces a digital human generation method provided by an embodiment. Figure 2 The flowchart of a digital human generation method provided in the embodiments of this application includes S201-S205.

[0047] S201. Generate a first mesh model based on human body data from the user's 3D data collected from multiple perspectives.

[0048] In this embodiment, three-dimensional data of the user can be collected from multiple perspectives, and the human body data in the three-dimensional data can be processed based on the SMPLX algorithm to generate a first mesh model.

[0049] For example, human body data includes, but is not limited to, contour features and limb joint angle data. Based on this SMPLX algorithm, the human torso, limbs, and face are reconstructed, and the generated first mesh model can represent human posture, shape, and texture.

[0050] S202. Based on the clothing data in the three-dimensional data, generate a second mesh model.

[0051] In this embodiment of the application, after collecting three-dimensional data of the user from multiple perspectives, the clothing data in the three-dimensional data of the user collected from multiple perspectives can be processed using the Signed Distance Field (SDF) algorithm and the Deep Marching Tetrahedra (DMT) algorithm to generate a second mesh model.

[0052] Among them, the SDF algorithm is an implicit method for representing three-dimensional geometry. It does not directly store points, lines, or surfaces (such as meshes), but uses a mathematical function to describe the distance from each point in space to the surface of the object. The DMT algorithm is an algorithm for extracting three-dimensional meshes from implicit SDF functions.

[0053] In this embodiment, by deeply integrating the SDF algorithm and the DMT algorithm, and combining multi-view data segmentation technology, it is possible to achieve full-body mesh reconstruction from coarse to fine.

[0054] In one possible implementation, an implicit SDF field is constructed based on clothing data, and isosurface extraction is performed on the SDF field using the DMT algorithm to generate a second grid model representing the characteristics of clothing.

[0055] For example, in the process of extracting isosurfaces from an SDF field based on the DMT algorithm, the DMT algorithm can achieve high-precision, unbiased isosurface extraction of the SDF field by dividing the space into tetrahedrons and optimizing the vertex positions by differentiable rendering gradients, so as to generate a second mesh model representing clothing features.

[0056] It is understood that the execution order of S201 and S202 is not specifically limited in the embodiments of this application. For example, S201 can be executed before S202, after S202, or simultaneously with S202.

[0057] S203. The first mesh model and the second mesh model are fused to generate a third mesh model.

[0058] In this embodiment, by fusing the features of a first grid model representing human body features with a second grid model representing clothing features, a third grid model that represents both human body features and clothing features can be generated, so that a digital human can be generated based on the third grid model.

[0059] In this embodiment, feature fusion refers to the process of integrating and optimizing feature information from different mesh models through algorithms to generate new features. These new features can represent both human body features and clothing features, facilitating the subsequent determination of digital human parameters under motion conditions.

[0060] For example, spatial alignment of the first mesh model and the second mesh model can yield aligned first mesh model and aligned second mesh model. Based on this, a third mesh model can be generated by feature fusion of the aligned first mesh model and aligned second mesh model.

[0061] In this embodiment of the application, during the process of feature fusion of the aligned first mesh model and the aligned second mesh model to generate the third mesh model, an offset can be added to the surface of the aligned first mesh model. The vertex coordinates in the aligned second mesh model are used as the offset target to supplement the geometric details of fine structures such as clothing and hair, thereby generating the fourth mesh model.

[0062] Based on this, the fourth mesh model can be dynamically adjusted by combining the posture parameters in the aligned first mesh model to ensure that clothing, hair and other parts can produce natural deformation during movement, thereby improving the display effect of the subsequently generated digital human.

[0063] S204. Based on the third grid model, determine the parameters of the digital human in motion.

[0064] The digital human parameters include motion parameters and clothing fold parameters corresponding to the motion parameters.

[0065] Since the third grid model represents static human body features and clothing features, in order to ensure the display effect of digital human clothing in motion, the embodiments of this application determine the digital human parameters in motion based on the third grid model.

[0066] In one possible implementation, a pose mapping map is generated by mapping the third mesh model onto a two-dimensional image plane; based on the pose mapping map and the pose parameters of the first mesh model, the parameters of the digital human in motion can be determined.

[0067] For example, mapping the third mesh model onto a two-dimensional image plane can generate a pose map. The color value of each pixel in the pose map can directly correspond to the position information of that pixel in the world coordinate system, which can be represented in the form of coordinates (X, Y, Z).

[0068] The Pose Map representation method can encode three-dimensional spatial information into two-dimensional images, which facilitates the subsequent analysis and manipulation of three-dimensional models using image processing techniques.

[0069] By inputting the attitude map and the attitude parameters of the first mesh model into the RGBC fully convolutional network, a Gaussian sputtering property (GS Map) map can be obtained.

[0070] Among them, the RGBC fully convolutional network is a special neural network structure that can process input images with multiple channels, such as RGB color channels plus possible additional channels; the GS map is a rich image representation that combines spatial and attribute information. Through the GS map, the position of points on the surface of a 3D model and the Gaussian distribution properties around them can be obtained simultaneously.

[0071] In this embodiment, the GS Map and the Pose Map have a pixel-level correspondence, meaning their pixel positions are one-to-one. The GS Map not only contains the spatial information from the original Pose Map but also incorporates all attribute parameters from a Gaussian distribution, such as attribute data, color data, rotation data, and scale data. Based on the attribute data, color data, rotation data, and scale data contained in the GS Map, the parameters of the digital human in motion can be determined.

[0072] It is understood that in this embodiment, the pose map input to the RGBC fully convolutional network reflects static features, while the pose parameters of the first mesh model can reflect different action features. By using the pose map and the pose parameters of the first mesh model as input to the RGBC fully convolutional network, a connection is established between the pose parameters and clothing data (such as clothing folds, colors, etc.). The resulting GS Map can characterize the folds of the digital human's clothing under different pose parameters, which facilitates the subsequent generation of a digital human with better display effects.

[0073] S205. Generate a digital human based on the digital human parameters under the motion state.

[0074] In this embodiment, Gaussian points can be generated on the surface of the third mesh model; based on the digital human parameters in the motion state, the Gaussian points are converted from static poses to motion poses to generate a digital human.

[0075] For example, Gaussian points are generated on the surface of the third mesh model. Using the generated Gaussian points as fixed positions, the color information of each Gaussian point can be determined by calculating the triangular facet difference. This allows the vertex or facet information of the three-dimensional mesh to be converted into a continuous color field, providing a basis for the subsequent rendering process.

[0076] Among them, triangular facet interpolation can include, but is not limited to, barycentric coordinate interpolation. By calculating triangular facet interpolation, the color information of each Gaussian point can be determined, which can ensure that the sampling points are evenly distributed and avoid discontinuities in color calculation.

[0077] Using the skinning information from SMPLX for driven rendering, the Gaussian points are transformed from static poses to dynamic poses to generate a digital human. During the driven rendering process using SMPLX's skinning information, the skinning weights and pose parameters (θ) of SMPLX are used to drive mesh deformation, converting the third mesh model into a digital human corresponding to the target motion map.

[0078] The target motion graph can be understood as an image derived from the user in different action states, rather than an image obtained directly through actual collection.

[0079] In the process of converting the third grid model into a digital human corresponding to the target motion map, the target motion map can be compared with the real image at the pixel level, and loss calculation can be performed based on the difference between the target motion map and the real image to improve the generation effect of the digital human corresponding to the target motion map.

[0080] For example, when generating digital humans by using skin information from SMSLX for driving rendering, traditional rasterization, such as OpenGL, or neural rendering, such as Neural Radiation Field (NeRF), can be used to generate digital humans with different target actions.

[0081] In summary, this application provides a digital human generation method. A third mesh model is generated by fusing features from a first mesh model representing the human body and a second mesh model representing the user's clothing. Based on this third mesh model, a digital human is generated based on digital human parameters in motion. These parameters include motion parameters and corresponding clothing fold parameters. In this application, the process of generating a digital human by combining motion parameters and corresponding clothing fold parameters improves the realism of clothing folds, ensures the display effect of complex clothing, and thus enhances the overall display effect of the digital human.

[0082] This application provides a digital human generation device, see [link]. Figure 3 The figure is a schematic diagram of the structure of a digital human generation device provided in an embodiment of this application. Its specific implementation method is consistent with the implementation method and the technical effect achieved in the embodiments of the above method, and some contents will not be repeated.

[0083] This application provides a digital human generation device 3100, including: The first generation unit 3101, the feature fusion unit 3102, the determination unit 3103, and the second generation unit 3104; The first generation unit 3101 is used to generate a first mesh model based on human body data in the three-dimensional data of the user collected from multiple perspectives; and to generate a second mesh model based on clothing data in the three-dimensional data. The feature fusion unit 3102 is used to fuse the features of the first mesh model and the second mesh model to generate a third mesh model. The determining unit 3103 is used to determine the digital human parameters in motion state based on the third grid model; the digital human parameters include action parameters and clothing fold parameters corresponding to the action parameters; The second generation unit 3104 is used to generate a digital human based on the digital human parameters in the motion state.

[0084] In one possible implementation, the determining unit is specifically used for: The third mesh model is mapped onto a two-dimensional image plane to generate a pose mapping map; Based on the posture mapping and the posture parameters of the first mesh model, the parameters of the digital human in motion state are determined.

[0085] In one possible implementation, the determining unit is specifically used for: The attitude mapping map and the attitude parameters of the first mesh model are input into an RGBC fully convolutional network to obtain a Gaussian sputtering property map; Based on the attribute data, color data, rotation data, and scale data included in the Gaussian sputtering attribute map, the parameters of the digital human in motion state are determined.

[0086] In one possible implementation, the generating unit is specifically used for: Gaussian points are generated on the surface of the third mesh model; Based on the digital human parameters under the motion state, the Gaussian point is converted from a static pose to a motion pose to generate a digital human.

[0087] In one possible implementation, the feature fusion unit is specifically used for: The first mesh model and the second mesh model are spatially aligned to obtain the aligned first mesh model and the aligned second mesh model; The aligned first mesh model and the aligned second mesh model are feature fused to generate a third mesh model.

[0088] In one possible implementation, the feature fusion unit is specifically used for: Using the vertex coordinates in the aligned second mesh model as the offset target, an offset is added to the surface of the aligned first mesh model to generate a fourth mesh model; based on the pose parameters of the aligned first mesh model, the fourth mesh model is dynamically adjusted to generate a third mesh model.

[0089] In one possible implementation, the first generating unit is specifically used for: The SMPLX algorithm is used to process human body data in the 3D data of users collected from multiple perspectives to generate the first mesh model.

[0090] In one possible implementation, the first generating unit is specifically used for: The clothing data in the 3D data is processed using the SDF and DMT algorithms to generate a second mesh model.

[0091] In summary, this application provides a digital human generation device that generates a third mesh model by fusing features of a first mesh model representing the human body and a second mesh model representing the user's clothing. Based on this third mesh model, a digital human is generated using digital human parameters in motion, including motion parameters and corresponding clothing fold parameters. In this application, the process of generating a digital human by combining motion parameters and corresponding clothing fold parameters improves the realism of the folds in the digital human's clothing, ensuring the display effect of complex clothing and thus enhancing the overall display effect of the digital human.

[0092] This application provides a computer device, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0093] The computer device 400 includes: a processor 410 and a memory 411; The memory 411 is used to store program code and transmit the program code to the processor 410; the processor 410 is used to execute the steps of a digital human generation method as described above according to the instructions in the program code.

[0094] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a digital human generation method as described above.

[0095] This application provides a computer program product that, when run on a computer, executes the steps of a digital human generation method as described above.

[0096] This application provides a chip including a processor coupled to a memory for executing computer programs or instructions stored in the memory, such that the chip implements the steps of a digital human generation method as described above.

[0097] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0098] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating a digital human, characterized in that, include: The first mesh model is generated based on human body data from the user's 3D data collected from multiple perspectives; Based on the clothing data in the three-dimensional data, a second mesh model is generated; The first and second mesh models are fused to generate a third mesh model; Based on the third mesh model, the parameters of the digital human in motion state are determined; the digital human parameters include motion parameters and clothing fold parameters corresponding to the motion parameters. A digital human is generated based on the digital human parameters under the aforementioned motion state.

2. The method according to claim 1, characterized in that, The determination of digital human parameters in motion state based on the third grid model includes: The third mesh model is mapped onto a two-dimensional image plane to generate a pose mapping map; Based on the posture mapping and the posture parameters of the first mesh model, the parameters of the digital human in motion state are determined.

3. The method according to claim 2, characterized in that, The step of determining the digital human parameters in motion state based on the posture parameters of the posture mapping map and the first mesh model includes: The attitude mapping map and the attitude parameters of the first mesh model are input into an RGBC fully convolutional network to obtain a Gaussian sputtering property map; Based on the attribute data, color data, rotation data, and scale data included in the Gaussian sputtering attribute map, the parameters of the digital human in motion state are determined.

4. The method according to claim 1, characterized in that, The process of generating a digital human based on the parameters of the digital human in the motion state includes: Gaussian points are generated on the surface of the third mesh model; Based on the digital human parameters under the motion state, the Gaussian point is converted from a static pose to a motion pose to generate a digital human.

5. The method according to claim 1, characterized in that, The step of fusing features from the first mesh model and the second mesh model to generate a third mesh model includes: The first mesh model and the second mesh model are spatially aligned to obtain the aligned first mesh model and the aligned second mesh model; The aligned first mesh model and the aligned second mesh model are feature fused to generate a third mesh model.

6. The method according to claim 5, characterized in that, The step of fusing features of the aligned first mesh model and the aligned second mesh model to generate a third mesh model includes: Using the vertex coordinates in the aligned second mesh model as the offset target, an offset is added to the surface of the aligned first mesh model to generate the fourth mesh model. Based on the attitude parameters of the aligned first mesh model, the fourth mesh model is dynamically adjusted to generate the third mesh model.

7. The method according to any one of claims 1-6, characterized in that, The process of generating a first mesh model based on human body data from three-dimensional user data collected from multiple perspectives includes: The SMPLX algorithm is used to process human body data in the 3D data of users collected from multiple perspectives to generate the first mesh model.

8. The method according to any one of claims 1-6, characterized in that, The step of generating a second mesh model based on the clothing data in the three-dimensional data includes: The clothing data in the 3D data is processed using the SDF and DMT algorithms to generate a second mesh model.

9. A digital human generation device, characterized in that, include: The unit comprises a first generation unit, a feature fusion unit, a determination unit, and a second generation unit. The first generation unit is used to generate a first mesh model based on human body data in the three-dimensional data of the user collected from multiple perspectives; and to generate a second mesh model based on clothing data in the three-dimensional data. The feature fusion unit is used to fuse the features of the first mesh model and the second mesh model to generate a third mesh model; The determining unit is used to determine the digital human parameters in motion state based on the third grid model; the digital human parameters include motion parameters and clothing fold parameters corresponding to the motion parameters; The second generation unit is used to generate a digital human based on the digital human parameters in the motion state.

10. A computer device, characterized in that, The computer device includes: a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the steps of a digital human generation method as described in any one of claims 1-8 according to instructions in the program code.

Images