Cartoonized digital human generation method and device, medium and computer program product

Abstract

This application relates to the field of image processing technology, and more particularly to a method, device, medium, and computer program product for generating cartoon-style digital humans. In this method, an image including a user's face is acquired, and a cartoon-style face model is generated based on the image. The cartoon-style face model is combined with a cartoon-style torso model from a torso material library to generate a cartoon-style digital human. The cartoon-style torso model has already undergone skeletal rigging. This reduces the computational power required by electronic devices, making it applicable to devices with relatively low computational power, such as smartphones and tablets.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device, medium, and computer program product for generating cartoon-style digital humans. Background Technology

[0002] With the development of rendering and artificial intelligence technologies, digital avatars are continuously evolving in fields such as augmented reality (AR), games, and animation. Among them, cartoonish digital humans have gained a following due to their adorable appearance. Currently, in the creation of cartoonish digital humans, users can create basic geometric shapes in the drawing software on electronic devices and then perform operations such as stretching, extruding, and rotating these shapes to create the body, head, limbs, and other parts of the cartoonish digital human. Users can then add facial features (such as eyes, nose, and mouth), clothing folds, and hairstyle textures to complete the creation of the cartoonish digital human.

[0003] However, the above methods require high computing power from electronic devices, making them difficult to apply to devices with relatively weak computing power, such as mobile phones and tablets. Summary of the Invention

[0004] To address the problem that existing methods for creating cartoon-style digital humans are difficult to apply to electronic devices with relatively weak computing power, embodiments of this application provide a method, device, medium, and computer program product for generating cartoon-style digital humans, including:

[0005] In a first aspect, embodiments of this application provide a method for generating a cartoon-style digital human, applied to an electronic device, comprising: acquiring a first image, the first image including at least a user's face; generating a cartoon-style face model based on the user's face in the first image; and combining the cartoon-style face model with a first cartoon-style torso model in a torso material library to obtain a cartoon-style digital human.

[0006] Understandably, the first cartoonish torso model in the torso asset library has already completed skeletal rigging.

[0007] Based on the above solution, by directly selecting and combining a cartoonish torso model with pre-bound skeletons from the torso material library with a cartoonish face model, the computing power of electronic devices can be reduced, making it applicable to electronic devices with relatively weak computing power, such as smartphones and tablets.

[0008] In some optional implementations, the first cartoonized torso model can be a torso model obtained by rendering images of torso parameters selected by the user from a torso material library, or a torso model obtained by rendering images of torso parameters matched by the electronic device based on the user's full-body image, or a torso model obtained by rendering images of torso parameters recommended by the electronic device based on the user's habits. The embodiments of this application do not make specific limitations.

[0009] In some optional implementations of the first aspect, generating a cartoon-style face model based on the user's face in the first image includes: inputting the first image into a face recognition model to identify the region corresponding to the user's face in the first image; determining the face parameters of the user's face based on the region corresponding to the user's face in the first image; and rendering the image based on the face parameters to obtain the cartoon-style face model.

[0010] In this embodiment, by creating a cartoon-style digital avatar based on the area corresponding to the user's face in the first image, customization of the cartoon-style digital avatar can be achieved. Thus, a corresponding cartoon-style digital avatar can be customized and generated based on the user's face in any input image, bringing a unique and therapeutic experience.

[0011] In some optional implementations of the first aspect, the face recognition model includes a backbone network, a first convolutional set, a second convolutional set, a third convolutional set, a first upsampling module, a second upsampling module, a first prediction head, a second prediction head, and a third prediction head; the backbone network includes a first sub-network, a second sub-network, a third sub-network, a fourth sub-network, and a fifth sub-network, wherein the feature map output by the first sub-network is the input of the second sub-network, the feature map output by the second sub-network is the input of the third sub-network, the feature map output by the third sub-network is the feature map output by the fourth sub-network, and the feature map output by the fourth sub-network is the input of the fifth sub-network; the feature map output by the fifth sub-network is the input of the first convolutional set, and the feature map output by the first convolutional set is the input of the first prediction head, which is used to determine a first prediction result based on the feature map output by the first convolutional set. The first prediction result includes a first region in the first image and the probability that the first region is the region corresponding to the user's face in the first image; the first... The feature map output by the convolutional set is the input to the first upsampling module, and the feature map output by the first upsampling module is the input to the second convolutional set. The feature map output by the fourth sub-network is the input to the second convolutional set, and the feature map output by the second convolutional set is the input to the second prediction head. The second prediction head is used to determine a second prediction result based on the feature map output by the second convolutional set. The second prediction result includes a second region in the first image and the probability that the second region is the region corresponding to the user's face in the first image. The feature map output by the second convolutional set is the input to the second upsampling module, and the feature map output by the second upsampling module is the input to the third convolutional set. The feature map output by the third sub-network is the input to the third convolutional set, and the feature map output by the third convolutional set is the input to the third prediction head. The third prediction head is used to determine a third prediction result based on the feature map output by the third convolutional set. The third prediction result includes a third region in the first image and the probability that the third region is the region corresponding to the user's face in the first image.

[0012] In this embodiment, a fully convolutional neural network model is used for face detection. Since at least some of the convolutional layers in the fully convolutional neural network model are grouped convolutional layers and depthwise separable convolutional layers, the computational requirements of electronic devices can be reduced. Therefore, it can be applied to electronic devices with relatively weak computational capabilities, such as mobile phones and tablets.

[0013] In some alternative implementations of the first aspect, the region with the highest probability of being the region corresponding to the user's face in the first image among the first region, second region, and third region is taken as the region corresponding to the user's face.

[0014] In some alternative implementations of the first aspect, determining the facial parameters of the user's face based on the region corresponding to the user's face in the first image includes: inputting the first image into a reconstruction parameter prediction model to determine the facial parameters of the user's face.

[0015] In some alternative implementations of the first aspect, the reconstruction parameter prediction model is trained as follows: acquiring at least one sample image, inputting the sample image into the reconstruction parameter prediction model to be trained, and predicting the reconstruction parameters corresponding to the user's face in each sample image; rendering the image based on the reconstruction parameters corresponding to the user's face in each sample image to obtain a first cartoonized face model corresponding to each sample image; mapping each first cartoonized face model to the coordinate system of the corresponding sample image to obtain a mapped image corresponding to each sample image; adjusting the model parameters of the reconstruction parameter prediction model to be trained based on the overlap between the user's face in each sample image and the corresponding mapped image until the overlap meets the termination condition.

[0016] It is understandable that the termination condition can be based on the overlap of the user's face in each sample image and the corresponding mapped image being greater than the overlap threshold.

[0017] In some alternative implementations of the first aspect, the overlap of the user's face in each sample image and the corresponding mapped image includes one or more of the following: the overlap of facial key points, the overlap of eye key points, the overlap of face regions, and the overlap of facial features.

[0018] In a second aspect, this application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for executing the cartoon-style digital human generation method mentioned in the first aspect or any one of the first aspects of this application.

[0019] Thirdly, this application provides a readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the cartoon-style digital human generation method mentioned in the first aspect or any of the first aspects of this application.

[0020] Fourthly, embodiments of this application provide a computer program product, which includes computer instructions. When executed by an electronic device, the electronic device executes the computer program code of the cartoon-style digital human generation method mentioned in the first aspect or any one of the first aspects of this application. Attached Figure Description

[0021] Figure 1 A schematic diagram of an application scenario is shown;

[0022] Figure 2 According to some embodiments of this application, a flowchart of a method for generating cartoon-style digital humans is shown;

[0023] Figure 3According to some embodiments of this application, a flowchart of another method for generating cartoon-like digital humans is shown;

[0024] Figure 4 According to some embodiments of this application, a structural schematic diagram of a face recognition model is shown;

[0025] Figure 5 According to some embodiments of this application, a schematic diagram of the structure of a convolution set is shown;

[0026] Figure 6 According to some embodiments of this application, a schematic diagram of an upsampling module is shown;

[0027] Figure 7 According to some embodiments of this application, a schematic diagram of the structure of a prediction head is shown;

[0028] Figure 8 According to some embodiments of this application, a schematic diagram of the input and output of a reconstruction parameter prediction model is shown;

[0029] Figure 9 According to some embodiments of this application, a schematic diagram of the hardware structure of an electronic device is shown. Detailed Implementation

[0030] The embodiments of this application include, but are not limited to, a method, device, medium, and program product for generating cartoon-style digital humans.

[0031] It is understood that the cartoon-style digital human generation method mentioned in the embodiments of this application can be applied to electronic devices. These electronic devices can also be referred to as terminals, user equipment (UE), mobile terminals (MT), etc. In some specific implementations, the electronic device can be a smartphone, tablet, etc.

[0032] It is understood that the methods mentioned in the embodiments of this application can be applied to scenarios involving the customization of cartoon-style digital humans for users, as well as scenarios involving the customization of cartoon-style pets for animals. The embodiments of this application do not impose specific limitations. Figure 1 The diagram illustrates an application scenario. In practical applications, electronic devices can generate a cartoonish digital human, such as in image B, based on the user's face in input image A.

[0033] To address the aforementioned problems, this application provides a method for generating a cartoon-style digital human. In this method, a first image is acquired, which includes at least a user's face. A cartoon-style face model is generated based on the first image. In response to a user's selection of a cartoon-style torso model from a torso material library, the cartoon-style face model and the cartoon-style torso model are combined to generate a cartoon-style digital human. The cartoon-style torso models in the torso material library can be stored either as cartoon-style torso models or as torso parameters. Furthermore, different parts of the torso model (e.g., hands, upper body, lower body, etc.) can maintain different postures or perform different actions as the position of the skeleton changes; this is referred to as skeletal rigging. Thus, by directly selecting and combining a skeletally rigging cartoon-style torso model with a cartoon-style face model from the torso material library, the computational power of electronic devices can be reduced, making it applicable to electronic devices with relatively weak computational power, such as smartphones and tablets.

[0034] It is understandable that, in some optional implementations, cartoon-style face models can be generated in the following ways:

[0035] The electronic device can determine the coordinates of skeletal nodes in the image coordinate system corresponding to the first image, and determine the skinning weight of each pixel in the first image relative to its corresponding skeletal node based on the distance between each pixel and the skeletal node (e.g., pixels corresponding to the left and right corners of the eyes). The closer a pixel is to a skeletal node, the higher its skinning weight, and the greater the influence of the skeletal node on the pixel. Adjusting the coordinates of the skeletal nodes in the image coordinate system corresponding to the first image adjusts the corresponding vertices in the initial face mesh to generate a cartoonish face model.

[0036] Understandably, in some alternative implementations, the electronic device can perform face detection on a full-body image, including the user's face and body, to obtain the facial region. Then, the electronic device can extract the input image corresponding to the facial region from the full-body image. Thus, by obtaining the input image from the user's full-body image to create a cartoonish digital human corresponding to the user, customization of the cartoonish digital human can be achieved.

[0037] In some methods of generating cartoon-style digital humans, electronic devices need to perform artistic design and modeling. During the artistic design process, electronic devices can design cute and brightly colored cartoon-style digital humans for children, and fashionable and trendy cartoon-style digital humans for young people. During the modeling process, electronic devices can first create basic geometric shapes and then perform operations such as stretching, extruding, and rotating these shapes to create the head, body, limbs, and other parts of the cartoon-style digital human. Then, electronic devices can add facial features, clothing folds, hairstyle textures, and other features to complete the creation of the cartoon-style digital human.

[0038] Currently, users urgently need electronic devices capable of generating customized cartoon-style digital avatars based on their faces in any input image, providing a unique and therapeutic experience. However, using the aforementioned methods to generate cartoon-style digital avatars based on the user's face in an input image results in long production times and high costs.

[0039] Therefore, with the development of artificial intelligence technology, electronic devices can use 3D reconstruction or 3D generation techniques to customize a cartoonish digital human corresponding to the user's face in any input image. However, 3D reconstruction techniques, such as neural radiance fields (NeRF) and 3D Gaussian splatting (3DGS), place high demands on the computing power of electronic devices. Furthermore, 3D reconstruction techniques themselves tend to focus on realistic reconstruction. If it is necessary to change the style of the generated cartoonish digital human, i.e., to perform style transfer on the cartoonish digital human, it will increase the computational load on the electronic device.

[0040] Furthermore, the cartoon-style digital human corresponding to the user's face in any input image customized using 3D generation technology is not responsive, meaning the generated cartoon-style digital human cannot perform limb movements. To ensure that the generated cartoon-style digital human can perform smooth limb movements, manual skeletal binding is often required, which results in high computing power required for electronic devices.

[0041] Furthermore, considering the demand for customized cartoon-style digital humans corresponding to any user's face in an input image on mobile devices such as smartphones and tablets, the reconstruction time, complexity, and deployment difficulty of 3D reconstruction or generation technologies make them difficult to deploy on mobile devices with relatively weak computing power and limited supporting operators. Therefore, creating drivable cartoon-style digital humans on mobile devices remains a challenge in the field of digital character creation.

[0042] Therefore, this application provides a method for generating cartoon-style digital humans, which can be applied to mobile devices with relatively weak computing power and limited supporting operators.

[0043] The following describes the cartoon-style digital human generation method mentioned in the embodiments of this application.

[0044] like Figure 2 The diagram illustrates a flowchart of a method for generating cartoon-style digital humans. This method can be executed by electronic devices, such as the smartphones and tablets mentioned earlier. The specific process is as follows:

[0045] 201: Face detection.

[0046] In some alternative implementations, the electronic device can perform face detection on the input image to obtain the region corresponding to the user's face in the input image.

[0047] It is understood that the input image is the first image mentioned above, which may include at least the user's face. In some alternative implementations, the input image may be a full-body image including the user's face and body, a half-body image including the user's face and part of the user's body, or an input image that only includes the user's face.

[0048] In some specific implementations, electronic devices can extract facial keypoint features (such as eyes, nose, and mouth) from an input image. Then, the electronic device can predict the coordinates of these facial keypoints in the image coordinate system corresponding to the input image based on these features, and determine the region corresponding to the user's face based on these coordinates.

[0049] For example, based on facial key point features, the coordinates of the outer corner of the left eye are predicted to be (50, 80), the inner corner of the left eye to be (80, 80), the inner corner of the right eye to be (120, 80), the outer corner of the right eye to be (150, 80), the tip of the nose to be (100, 120), the left corner of the mouth to be (70, 150), and the right corner of the mouth to be (130, 150). Furthermore, the electronic device can determine a rectangle with a top-left vertex of (30, 45) and a bottom-right vertex of (170, 180) from the input image, based on the distance of the facial key points from the facial contours—for example, the outer corner of the left eye is 20 units away from the left side of the face contour, and the mouth is 30 units away from the bottom of the face contour—to represent the area corresponding to the user's face.

[0050] In some embodiments, electronic devices can also use a pre-trained face recognition model to detect faces in input images and obtain the region corresponding to the user's face in the input image.

[0051] It is understood that the specific structure and detection process of the face recognition model mentioned in the embodiments of this application will be explained in detail later. Figure 3 To avoid repetition, details will not be elaborated here.

[0052] 202: Face modeling.

[0053] Understandably, in some alternative implementations, electronic devices can perform face modeling on input images that identify the region corresponding to the user's face, thereby obtaining the face parameters corresponding to the user's face in the input image.

[0054] In other alternative implementations, smaller image scales result in fewer pixels and less detail. Therefore, to improve image detail while reducing computational load on the electronic device, the device can crop the region corresponding to the user's face from the input image to obtain the input image. Furthermore, after obtaining the input image, the electronic device can perform face modeling to obtain the facial parameters corresponding to the user's face in the input image.

[0055] It is understandable that facial parameters can be used to construct cartoonish facial models. In some alternative implementations, facial parameters may include shape parameters.

[0056] In some specific implementations, electronic devices can use a pre-trained reconstruction parameter prediction model to model faces from input images and obtain face parameters. The training process of the reconstruction parameter prediction model will be detailed below; to avoid repetition, it will not be repeated here.

[0057] 203: Select torso parameters from the torso material library and merge face parameters and torso parameters.

[0058] Then, the electronic device can select torso parameters from the torso material library and merge the face parameters and torso parameters to obtain complete human body parameters.

[0059] Understandably, torso parameters can be used to construct cartoonish torso models. The torso parameters in this library are divided into three parts: hands, upper body, and lower body. In some specific implementations, these torso parameters can be shape parameters generated using a skinned multi-person linear model—the expressive (SMPL-X) model. Furthermore, the cartoonish torso model generated based on these torso parameters has already undergone skeletal rigging.

[0060] 204: Select a texture map from the texture library.

[0061] Understandably, in some alternative implementations, electronic devices can select texture maps from a texture library.

[0062] In some specific implementations, the electronic device can respond to the user's selection of texture map materials from the texture material library, and combine the user-selected texture map material with the aforementioned complete human body parameters to obtain textured human body parameters. The texture map materials can be manually designed or generated using an AI model. Texture map materials can include skin, eyes, lip color, body hair, blush, eyelashes, eyebrows, beard, wrinkles, nails, etc., and this application embodiment does not impose specific limitations.

[0063] 205: Select accessory materials from the accessory material library.

[0064] It is understandable that, in some optional implementations, electronic devices can select accessory materials from an accessory material library.

[0065] In some specific implementations, the electronic device can respond to the user's selection of accessory materials from the accessory material library, combining the user-selected accessory material with the aforementioned textured human body parameters to obtain textured human body parameters with accessories. Furthermore, the electronic device can perform image rendering based on these textured human body parameters with accessories to obtain a digital human (i.e., the cartoonish digital human mentioned earlier). The accessory materials can be manually designed or generated using AI models. Accessory materials can include hair, glasses, hats, tops, pants, shoes, socks, hand accessories, neck accessories, backpacks, etc., and this application embodiment does not specifically limit the scope.

[0066] like Figure 3 The diagram illustrates another method for generating cartoon-like digital humans. This method can be executed by electronic devices, such as the smartphones and tablets mentioned earlier. The specific process is as follows:

[0067] 301: Obtain a first image, which includes at least the user's face.

[0068] It is understood that the first image can be any image that includes at least the user's face. For example, the first image can be a full-body image that includes the user's face and body, a half-body image that includes the user's face and part of the user's body, or a face image that includes the user's face.

[0069] 302: Generate a cartoon-style face model based on the user's face in the first image.

[0070] In some alternative implementations, when the first image is a full-body image including the user's face and body, the electronic device can use a pre-trained face recognition model to perform face detection on the input image, obtaining the region corresponding to the user's face in the first image. Then, the electronic device can crop the region corresponding to the user's face from the first image to obtain a face image. In this way, the electronic device can improve the detail contained in the image while reducing computational load, thereby improving the accuracy of face detection. Then, the electronic device can use a trained reconstruction parameter prediction model to perform face modeling on the face image, obtaining face parameters, and performing image rendering based on the face parameters to generate a cartoonish face model.

[0071] In some alternative implementations, when the first image is a face image including the user's face, the electronic device can directly use a trained reconstruction parameter prediction model to model the face image, obtain face parameters, and then render the image based on these parameters to generate a cartoonish face model. The specific structure and training process of the reconstruction parameter prediction model will be detailed below; to avoid repetition, they will not be elaborated here.

[0072] 303: Combine the cartoon-style face model with the first cartoon-style torso model from the torso material library to obtain a cartoon-style digital human.

[0073] It is understood that the first cartoon-style torso model can be a torso model obtained by rendering images of torso parameters selected by the user from a torso material library, or a torso model obtained by rendering images of torso parameters matched by the electronic device based on the user's full-body image, or a torso model obtained by rendering images of torso parameters recommended by the electronic device based on the user's habits. This application embodiment does not make specific limitations.

[0074] The following is about Figure 2 and Figure 3 The specific structure and detection process of the face recognition model mentioned in the article will be described in detail.

[0075] like Figure 4 The diagram shows a schematic representation of a face recognition model. This face recognition model may include a backbone network, convolution set 1, convolution set 2, convolution set 3, upsampling set 1, upsampling set 2, prediction head 1, prediction head 2, and prediction head 3.

[0076] In some alternative implementations, the backbone network can be an 18-layer residual network (ResNet-18). This backbone network can include five stages (i.e., five sub-networks): stage 1, stage 2, stage 3, stage 4, and stage 5.

[0077] Furthermore, the feature map output from stage 1 can be used as input to stage 2, the feature map output from stage 2 can be used as input to stage 3, the feature map output from stage 3 can be used as input to stage 4, and the feature map output from stage 4 can be used as input to stage 5. The feature map output from stage 5 can be used as input to convolution set 1, and the feature map output from convolution set 1 can be used as input to prediction head 1. Prediction head 1 can obtain prediction result 1 based on the feature map output from convolution set 1. The feature map output from convolution set 1 can also be used as input to upsampling 1, and the feature map output from upsampling 1 can be used as input to convolution set 2. The feature map output from stage 4 can be used as input to convolution set 2, and the feature map output from convolution set 2 can be used as input to prediction head 2. Prediction head 2 can obtain prediction result 2 based on the feature map output from convolution set 2. The feature map output from convolution set 2 can also be used as input to upsampling 2, and the feature map output from upsampling 2 can be used as input to convolution set 3. The feature map output by stage 3 can be used as the input of convolution set 3, and the feature map output by convolution set 3 can be used as the input of prediction head 3. Prediction head 3 can obtain prediction result 3 based on the feature map output by convolution set 3.

[0078] In some specific implementations, stage 1 can perform feature extraction on the input image to obtain a first intermediate feature map. Then, the first intermediate feature map can be decoded to obtain a higher-scale first intermediate feature map. Stage 2 can perform feature extraction on the higher-scale first intermediate feature map to obtain a second intermediate feature map. Then, the second intermediate feature map can be decoded to obtain a higher-scale second intermediate feature map. The scale of the first intermediate feature map is larger than that of the second intermediate feature map; specifically, the height and width of the first intermediate feature map can both be twice that of the second intermediate feature map. Similarly, stages 3, 4, and 5 can also perform the aforementioned feature map scale reduction processing on the intermediate feature maps; to avoid repetition, these will not be elaborated further here.

[0079] For example, an input image can be represented as Where 'I' can represent the input image. It can represent the set of real numbers, that is, the pixel value of each pixel in the input image is a real number; h can represent the height of the input image, that is, the number of pixels in the vertical direction of the input image; w can represent the width of the input image, that is, the number of pixels in the horizontal direction of the input image; and 3 can represent the number of channels of the portrait image.

[0080] In some alternative implementations, the intermediate feature map output at the i-th stage (i∈[1,5]) can take the form shown in expression (1):

[0081]

[0082] Among them, f i This can represent the intermediate feature map output by stage i. `h` can represent the set of real numbers, meaning that the pixel value of each pixel in the intermediate feature map is a real number. `h` can represent the height of the intermediate feature map, i.e., the number of pixels in the vertical direction. `w` can represent the width of the intermediate feature map, i.e., the number of pixels in the horizontal direction. `c`... i This can represent the number of channels in the intermediate feature map output at the i-th stage. And, [c1, c2, c3, c4, c5] = [64, 65, 128, 256, 512].

[0083] It is understood that the backbone network can also be a mobile convolutional neural network (MobileNet), an extremely efficient convolutional neural network for mobile devices (SuffleNet), etc., and this application does not specifically limit it. When the backbone network is MobileNet, [c1, c2, c3, c4, c5] = [64, 128, 256, 1024, 2048]; when the backbone network is SuffleNet, [c1, c2, c3, c4, c5] = [24, 24, 144, 288, 576]. In this way, by continuously increasing the number of channels in the feature map, the richness and diversity of features can be improved.

[0084] It can be understood that each convolution set in convolution set 1, convolution set 2, and convolution set 3 can be a fully convolutional structure, and each convolution set can include multiple convolutional activation modules (CRs). Each CR includes concatenated convolutional layers (C) and rectified linear units (ReLU). Furthermore, each CR can perform convolution and ReLU nonlinear operations on the input feature map to input the nonlinearly transformed feature map to the next CR.

[0085] In some alternative implementations, such as Figure 5 As shown, each convolutional set may include a first convolutional activation module, a second convolutional activation module, a third convolutional activation module, a fourth convolutional activation module, and a fifth convolutional activation module. For example, the number of channels in the input feature map and the output feature map of each convolutional activation module may be the same.

[0086] Specifically, the convolutional layers in the first convolutional activation module may include 1×1 convolutional kernels, therefore, the first convolutional activation module can be abbreviated as 1×1CR; the convolutional layers in the second convolutional activation module may include 3×3 convolutional kernels, therefore, the second convolutional activation module can be abbreviated as 3×3CR; the convolutional layers in the third convolutional activation module may include 1×1 convolutional kernels, therefore, the third convolutional activation module can be abbreviated as 1×1CR; the convolutional layers in the fourth convolutional activation module may include 1×1 convolutional kernels, therefore, the fourth convolutional activation module can be abbreviated as 1×1CR; and the convolutional layers in the fifth convolutional activation module may include 3×3 convolutional kernels, therefore, the fifth convolutional activation module can be abbreviated as 3×3CR.

[0087] Furthermore, the nonlinear transformation feature map output by the first convolutional activation module can be used as the input to the second convolutional activation module, the nonlinear transformation feature map output by the second convolutional activation module can be used as the input to the third convolutional activation module, the nonlinear transformation feature map output by the third convolutional activation module can be used as the input to the fourth convolutional activation module, the nonlinear transformation feature map output by the fourth convolutional activation module can be used as the input to the fifth convolutional activation module, and the nonlinear transformation feature map output by the fifth convolutional activation module can be used as the input to the upsampling module or the prediction head.

[0088] In some specific implementations, the feature map of the input convolution set 1 can take the form shown in expression (2):

[0089] i1=f5(2)

[0090] Where f5 can represent the smaller-scale intermediate feature map obtained by further feature extraction processing of the intermediate feature map output by stage 4 in stage 5, and i1 can represent the feature map of input convolution set 1.

[0091] For convolutional sets other than convolutional set 1, their input feature maps can include intermediate feature maps output from the corresponding stage in the backbone network and fused feature maps output from the previous sampling module. For example, the feature map of the input convolutional set j (j≠1) can be i j The feature map output by convolution set j can be o j Then the feature map of the input convolution set j can be expressed in the form of the following expression (3):

[0092] i j =cat(u j-1 ,f 6-j (3)

[0093] Among them, i j It can represent the feature map of the input convolution set j, u j-1 f can represent the fused feature map output by the previous sampling module. 6-j This represents the intermediate feature map output by stage 6-j in the backbone network. "cat" can represent a concatenation operation. For example, suppose there are two feature maps C and D with shapes [batch_size, height, width, channels1] and [batch_size, height, width, channels2], respectively. If a concatenation (cat) operation is performed on the channel dimension, the resulting feature map will be [batch_size, height, width, channels1+channels2].

[0094] Furthermore, assuming that the size of the feature map input to each convolutional set is a×b, the feature map output by each convolutional set can take the form shown in the following expression (4):

[0095]

[0096] Among them, i j It can represent the feature map of the input convolution set j, o j It can represent the feature map of the input prediction head of the convolution set j. It can represent the set of real numbers, that is, the pixel value of each pixel in the feature map is a real number. 'a' can represent the width of the feature map, 'b' can represent the height of the feature map, and 'c' can represent the height of the feature map. inj c can represent the number of channels in the feature map of the input convolution set j. outj It can represent the number of channels in the feature map of the input prediction head.

[0097] In some alternative implementations, when the backbone network is ResNet-18, [c out1 c out2 c out3 ] = [c3, c4, c5].

[0098] It is understandable that each upsampling module in Upsampling 1 and Upsampling 2 can increase the scale of the feature map output by the convolution set, so as to fuse it with the large-scale feature map obtained from the previous convolution set to obtain a fused feature map. For example... Figure 6 As shown, each upsampling module in Upsampling 1 and Upsampling 2 can include a convolutional activation submodule and an upsampling submodule. The convolutional activation submodule can include cascaded convolutional layers and activation units. The convolutional layer can include a 1×1 convolutional kernel, and the activation unit can be a ReLU activation function; therefore, the convolutional activation submodule can be abbreviated as 1×1 CR. The upsampling submodule can be a 2× bilinear upsampling layer. In some optional implementations, the number of channels in the input and output feature maps of the convolutional layer can be the same.

[0099] It is understood that in some optional implementations, each upsampling module in Upsampling 1 and Upsampling 2 can also be an upsampling module that performs transposed convolutional upsampling operations and nearest neighbor upsampling operations on the feature maps output by the convolutional set. This application embodiment does not impose specific limitations on this. Furthermore, the convolutional kernels in the convolutional layers of each upsampling module can also be convolutional kernels of other sizes, or stacks of convolutional kernels of various sizes. This application embodiment does not impose specific limitations on this either. In addition, the activation unit can also be the tanh activation function. This application embodiment does not impose specific limitations on this either.

[0100] It is understandable that in some optional implementation methods, such as Figure 7 As shown, each prediction head in prediction head 1, prediction head 2, and prediction head 3 can include a convolutional activation submodule and a convolutional layer (conv) submodule. The convolutional activation submodule can include cascaded convolutional layers and activation units. The convolutional layers in the convolutional activation submodule can include 3×3 convolutional kernels; therefore, the convolutional activation submodule can be abbreviated as 3×3CR. The convolutional layer submodule can include 1×1 convolutional kernels; therefore, the convolutional layer submodule can be abbreviated as 1×1conv. Furthermore, the non-linear feature map output by the convolutional activation submodule can be used as the input to the convolutional layer submodule. In some optional implementations, the input and output feature maps of the 3×3CR can have the same number of channels, while the output feature map of the 1×1conv can have 5 channels.

[0101] Furthermore, in the prediction results output by the prediction head, that is, in the output feature map of the convolutional layer submodule in the prediction head, each pixel has a corresponding pixel in the input image, and this corresponding pixel can be called the point corresponding to the index coordinate.

[0102] In some alternative implementations, the prediction result output by the prediction head k can take the form shown in expression (5):

[0103]

[0104] Where, p k This can represent the index coordinates of the face region predicted by the prediction head k. The five channels of these index coordinates can be mapped to the input image (index coordinates multiplied by 2). 6-k The corresponding information includes, for example, the distance from the left boundary of the bounding box centered at the point corresponding to the index coordinate to the point corresponding to the index coordinate, the distance from the right boundary of the bounding box centered at the point corresponding to the index coordinate to the point corresponding to the index coordinate, the distance from the top boundary of the bounding box centered at the point corresponding to the index coordinate to the point corresponding to the index coordinate, the distance from the bottom boundary of the bounding box centered at the point corresponding to the index coordinate to the point corresponding to the index coordinate, and the probability that the object in the bounding box is a human face.

[0105] In this way, the electronic device can crop out the face region from the input image based on the maximum probability that the object in the bounding box of the prediction head 1, prediction head 2 and prediction head 3 is a face, as well as the information corresponding to the remaining four channels of the bounding box, and obtain a face image.

[0106] In some specific implementations, during the training process of the face recognition model, the L2 loss function can be used to predict the top, bottom, left, and right information of the bounding box, while the binary cross-entropy loss function can be used to predict the face probability.

[0107] Therefore, using a fully convolutional neural network model for face detection can reduce the computational requirements of electronic devices, since at least some of the convolutional layers in the fully convolutional neural network model are grouped convolutional layers and depthwise separable convolutional layers. Thus, it can be applied to electronic devices with relatively weak computing power, such as mobile phones and tablets.

[0108] The following is about Figure 2 and Figure 3 The training process of the reconstruction parameter prediction model mentioned is described in detail.

[0109] In some alternative implementations, such as Figure 8 As shown, the sample input image is input into the reconstruction parameter prediction model to be trained, and the reconstruction parameters corresponding to the sample input image are obtained. These reconstruction parameters may include shape parameters, camera position, illumination parameters, albedo, expression parameters, and pose parameters, among others.

[0110] Image rendering is performed based on the reconstructed parameters to obtain a cartoonish face model. This cartoonish face model is then mapped to the image coordinate system corresponding to the input sample image, resulting in a mapped image. The total loss value of the reconstruction parameter prediction model to be trained is determined based on the overlap rates of facial keypoints, eye keypoints, face regions, and facial features between the mapped image and the input sample image. The overlap rate of facial keypoints can include the overlap rates of 68 facial keypoints.

[0111] When the total loss value is greater than the preset loss threshold, adjust the parameters of the reconstruction parameter prediction model to be trained, and repeat the above steps of inputting the sample input image into the reconstruction parameter prediction model to be trained, rendering the image based on the reconstruction parameters, and mapping the cartoon face model to the image coordinate system corresponding to the sample input image, until the total loss value of the reconstruction parameter prediction model to be trained is less than or equal to the preset loss threshold, and the trained reconstruction parameter prediction model is obtained.

[0112] It is understandable that shape parameters can include facial contour parameters, facial feature position and shape parameters, and expression parameters. Among these, facial contour parameters can include the length-to-width ratio of the face, the pointedness of the chin, and the contour curve of the face. Facial feature position and shape parameters can include the position of the eyes, the size of the eyes, the length-to-width ratio of the eyes, the tilt of the eye corners, the position of the nose, the height of the nose, the width of the nose, the curvature of the bridge of the nose, the position of the mouth, the size of the mouth, and the thickness of the lips.

[0113] The camera position is used to determine the shooting angle of the portrait image, so that in the process of face reconstruction, the shooting angle of the portrait image can be combined to accurately determine the three-dimensional shape of the cartoon face model.

[0114] Different camera positions result in varying lighting effects on a user's face, thus affecting the brightness, shadows, and colors of the portrait image. Therefore, during face reconstruction, both camera position and lighting parameters need to be considered to improve the realism of the reconstructed cartoon-like face model.

[0115] Albedo represents the ability of a user's face surface to reflect light. For example, albedo can represent the color and brightness characteristics of parts of a user's face such as skin, hair, and eyes. Thus, using albedo for face reconstruction can improve the accuracy of reproducing the true colors and gloss of a user's face in cartoon face models.

[0116] Facial expression parameters can include the degree to which the corners of the mouth turn up when smiling, the depth of the wrinkles on the forehead when frowning, etc.

[0117] Attitude parameters can include head rotation parameters, such as yaw angle, pitch angle, and roll angle. Yaw angle represents the angle at which the head turns left and right, pitch angle represents the angle at which the head swings up and down, and roll angle represents the angle at which the head tilts left and right.

[0118] In some specific implementations, the reconstruction parameter prediction model to be trained may include a skeletal skinning mesh model, a facial landmark animation and editing model (FLAME), a 50-layer residual network (Res-Net50), MobileNet, ShuffleNet, a regularized network (RegNet), etc.

[0119] The skeleton skin mesh model can be represented in the form shown in expression (6):

[0120]

[0121] in, J(β), θ, Ω) represent the skeletal skinning mesh model, where J(β) represents a bone node, such as the left corner of the eye, the right corner of the eye, etc. β represents a skin vertex, for example, a pixel corresponding to the left cheek, and β can represent a shape parameter. θ can represent attitude parameters. It can represent facial expression parameters. Ω can represent the blend weight. n represents the number of grid cells.

[0122] For skinned vertices The electronic device can be calculated using the following formula (7):

[0123]

[0124] Where T represents the initial pose template, BS(β) represents the shape representation function, and BP(θ) represents the pose representation function. This represents the facial expression function, where β can represent the shape parameter. θ can represent attitude parameters. It can represent facial expression parameters.

[0125] The reconstruction parameter prediction model to be trained may also include an illumination model.

[0126] In some alternative implementations, the lighting model can be represented in the form shown in expression (8):

[0127] B(α,l,N)=A(α)⊙∑l×H(N) Formula (8)

[0128] Where A() can represent the FLAME UV access albedo function, H() can represent the spherical harmonics (SH) function, and ⊙ can represent the dot product calculation. It can represent facial expression parameters. l can represent lighting parameters.

[0129] In some alternative implementations, the electronic device can render an image based on reconstruction parameters and illumination parameters to obtain a cartoonish face model. The cartoonish face model can then be mapped to the image coordinate system corresponding to the input image, for example, using a projection function. This allows the total loss value of the reconstruction parameter prediction model to be determined based on at least one of the following: the overlap of facial keypoints, the overlap of eye keypoints, the overlap of facial regions, and the overlap of facial features.

[0130] It also determines the coordinates of multiple facial landmarks in the image coordinate system corresponding to the input image, and determines the coordinates of these facial landmarks in the image coordinate system corresponding to the input image from the input image.

[0131] When rendering an image based on reconstruction parameters, a rendering function as shown in formula (9) can be used:

[0132] l r =R(M, B, c) Formula (9)

[0133] Among them, l r R can represent a cartoonish human face model, R() can represent the rendering function, M represents the reconstruction parameters, B represents the lighting parameters, and c represents the camera position.

[0134] Loss = L landmark +L eye +L photo +L reg Formula (10)

[0135] Where Loss can represent the total loss value, L landmark It can represent the degree of overlap of facial key points, L eye It can indicate the degree of overlap of key points around the eye, Lphoto L can represent the degree of overlap of facial regions. reg It can indicate the degree of overlap in the positions of facial features.

[0136] Specifically, the degree of overlap of the facial key points can be calculated using the following formula (11):

[0137]

[0138] Among them, L landmark k can represent the degree of overlap of facial key points. i It can represent the x-coordinate of facial key points in the image coordinate system corresponding to the input image, project(M i The x-coordinate of a key facial point can be represented when a cartoonish face model is mapped to the image coordinate system corresponding to the input image.

[0139] Specifically, the degree of overlap of key points around the eye can be calculated using the following formula (12):

[0140]

[0141] Among them, L eye It can indicate the degree of overlap of key points around the eye.

[0142] Assume, I r If there is a mask label for the user's face (a pixel-level classification 0-1 image of the image, where the part containing the user's face is 1, otherwise it is 0), then the overlap of the face regions can be calculated using the following formula (13):

[0143] L pho =||mask(II) r )||1 Formula (13)

[0144] Among them, L pho It can represent the degree of overlap of facial regions.

[0145] In some alternative implementations, facial features may include shape parameters, expression parameters, and albedo. Therefore, the positional overlap of facial features can be determined based on the regularization loss of each facial feature. Specifically, the positional overlap of facial features can be calculated using the following formula (14):

[0146]

[0147] Among them, L reg β can represent the degree of overlap in facial regions, and β can represent a shape parameter. It can represent facial expression parameters, and α can represent albedo.

[0148] After training the reconstruction parameter prediction model, when reconstructing a face based on the input image or face image, only the shape parameters can be output as face parameters. Based on the face parameters, image rendering can be performed to obtain a cartoon-style face model.

[0149] The effects of the cartoon-style digital human generation method mentioned in the embodiments of this application will be explained below in conjunction with some methods for generating cartoon-style digital humans.

[0150] In some methods for generating cartoon-like digital humans, electronic devices can acquire images of target objects and use generative facial prior-generative adversarial networks (GFP-GANs) to reconstruct the 3D face of the target object in the image, generating a 3D face model. Then, the electronic device can use voice-driven methods to facialize the 3D face model, obtaining the final 3D face model. However, this method only involves 3D face reconstruction and cannot customize a cartoon-like digital human corresponding to any given user's face in an input image.

[0151] In other methods of generating cartoon-like digital humans, electronic devices can acquire textual descriptions of the 3D digital human from the user, and then use natural language processing (NLP) techniques to parse the descriptions to obtain the target's key attribute information. Next, the electronic device can determine the values ​​of control parameters in the 3D digital human's parametric model based on these key attribute information, and input these values ​​into a stable diffusion model to generate the cartoon-like digital human. However, this method uses a stable diffusion model, making it difficult to deploy on mobile devices with relatively weak computing power, such as smartphones and tablets. Furthermore, the cartoon-like digital humans generated using this method are unresponsive.

[0152] Therefore, compared with the above-mentioned cartoon-style digital human generation method, the cartoon-style digital human generation method mentioned in the embodiments of this application can reduce the computing power requirements of electronic devices, and thus can be applied to electronic devices with relatively weak computing power, such as mobile phones and tablets.

[0153] It is understood that the cartoon-style digital human generation method provided in this application embodiment can be applied to electronic devices. The hardware structure of the electronic device to which the cartoon-style digital human generation method provided in this application embodiment is applicable will be described exemplarily below.

[0154] like Figure 9As shown, the electronic device 900 may include a processor 910, an external memory interface 920, an internal memory 921, a universal serial bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna, a wireless communication module 950, an audio module 960, a speaker 960A, a receiver 960B, a microphone 960C, a headphone jack 960D, a camera 970, a display screen 980, etc.

[0155] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 900. In other embodiments of this application, the electronic device 900 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0156] Processor 910 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0157] The controller can generate operation control signals based on the instruction opcode and timing signals to control instruction fetching and execution. The processor 910 can control instruction fetching and execution via the controller to implement the cartoon-style digital human generation method provided in this application embodiment. For example, the processor 910 can control instruction fetching and execution via the controller to implement the above-mentioned... Figure 2 or Figure 3 The steps to be implemented in the process shown are as follows.

[0158] The processor 910 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 910 is a cache memory. This memory can store instructions or data that the processor 910 has just used or that are used repeatedly. If the processor 910 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 910, and thus improves the efficiency of the system.

[0159] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules 950, modem processors, and baseband processors.

[0160] Antennas are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antennas can be reused as diversity antennas for a wireless local area network. In some other embodiments, antennas can be used in conjunction with tuning switches.

[0161] The wireless communication module 950 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 950 can be one or more devices integrating at least one communication processing module. The wireless communication module 950 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 910. The wireless communication module 950 can also receive signals to be transmitted from the processor 910, perform frequency modulation and amplification, and then convert them into electromagnetic waves for radiation via the antenna.

[0162] Electronic devices utilize a GPU, a display screen 980, and an application processor to achieve display functionality. The GPU is a microprocessor for image processing, connecting the display screen 980 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 910 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0163] Display screen 980 is used to display images, videos, etc. Display screen 980 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, MicroLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device may include one or N displays 980, where N is a positive integer greater than 1.

[0164] In some cases, the embodiments disclosed in this application may be implemented in hardware, firmware, software, or any combination thereof.

[0165] The embodiments disclosed in this application can also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, the instructions can be distributed via a network or via other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, magnetic disks, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0166] Embodiments of this application can be implemented as computer programs or program code that execute on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0167] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0168] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0169] The above describes the possible hardware structures of electronic devices. It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of both.

[0170] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0171] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0172] Although this application has been illustrated and described with reference to certain embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A method for generating cartoon-style digital humans, applied to electronic devices, characterized in that, include: Acquire a first image, the first image including at least the user's face; The first image is input into a face recognition model to identify the region corresponding to the user's face from the first image; A cartoon-style face model is generated based on the region corresponding to the user's face; The cartoon-style face model and the first cartoon-style torso model in the torso material library are combined to obtain a cartoon-style digital human. The face recognition model includes a backbone network, a first convolutional set, a second convolutional set, a third convolutional set, a first upsampling module, a second upsampling module, a first prediction head, a second prediction head, and a third prediction head; The backbone network includes a first sub-network, a second sub-network, a third sub-network, a fourth sub-network, and a fifth sub-network. The feature map output by the first sub-network is the input of the second sub-network, the feature map output by the second sub-network is the input of the third sub-network, the feature map output by the third sub-network is the feature map output by the fourth sub-network, and the feature map output by the fourth sub-network is the input of the fifth sub-network. The feature map output by the fifth sub-network is the input of the first convolution set, and the feature map output by the first convolution set is the input of the first prediction head. The first prediction head is used to determine a first prediction result based on the feature map output by the first convolution set. The first prediction result includes a first region in the first image and the probability that the first region is the region corresponding to the user's face in the first image. The feature map output by the first convolutional set is the input of the first upsampling module, and the feature map output by the first upsampling module is the input of the second convolutional set. The feature map output by the fourth sub-network is the input of the second convolution set, and the feature map output by the second convolution set is the input of the second prediction head. The second prediction head is used to determine a second prediction result based on the feature map output by the second convolution set. The second prediction result includes a second region in the first image and the probability that the second region is the region corresponding to the user's face in the first image. The feature map output by the second convolutional set is the input of the second upsampling module, and the feature map output by the second upsampling module is the input of the third convolutional set; The feature map output by the third sub-network is the input of the third convolution set, and the feature map output by the third convolution set is the input of the third prediction head. The third prediction head is used to determine a third prediction result based on the feature map output by the third convolution set. The third prediction result includes a third region in the first image and the probability that the third region is the region corresponding to the user's face in the first image. Among the first region, the second region, and the third region, the region with the highest probability of being the region corresponding to the user's face in the first image is taken as the region corresponding to the user's face.

2. The method according to claim 1, characterized in that, The step of generating a cartoon-style face model based on the region corresponding to the user's face includes: Based on the region corresponding to the user's face in the first image, the facial parameters of the user's face are determined; The cartoon-style face model is obtained by rendering the image based on the facial parameters.

3. The method according to claim 2, characterized in that, The step of determining the facial parameters of the user's face based on the region corresponding to the user's face in the first image includes: The first image is input into the reconstruction parameter prediction model to determine the facial parameters of the user's face.

4. The method according to claim 3, characterized in that, The reconstruction parameter prediction model was trained in the following way: At least one sample image is acquired, and the sample image is input into the reconstruction parameter prediction model to be trained to predict the reconstruction parameters corresponding to the user's face in each sample image. Based on the reconstruction parameters corresponding to the user's face in each of the sample images, image rendering is performed to obtain the first cartoon-style face model corresponding to each of the sample images; Each of the first cartoon-style face models is mapped to the coordinate system of the corresponding sample image to obtain the mapped image corresponding to each sample image; Based on the overlap of the user's face in each sample image and the corresponding mapped image, the model parameters of the reconstruction parameter prediction model to be trained are adjusted until the overlap meets the termination condition.

5. The method according to claim 4, characterized in that, The overlap of the user's face in each of the sample images and the corresponding mapped images includes one or more of the following: the overlap of facial key points, the overlap of eye key points, the overlap of face regions, and the overlap of facial features.

6. An electronic device, characterized in that, include: A memory for storing instructions executed by one or more processors of the electronic device, and a processor, being one of one or more processors of the electronic device, for performing the cartoon-style digital human generation method according to any one of claims 1-5.

7. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the cartoon-style digital human generation method according to any one of claims 1-5.

8. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by an electronic device, execute the computer program code of the cartoon-style digital human generation method as described in any one of claims 1-5.

Images