Face parameter generation method and device

Abstract

The embodiment of the invention discloses a face parameter generation method and device. In the embodiment of the invention, a real face image is acquired; performing feature point extraction on the real face image to generate standardized feature points of the real face image; inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image; and inputting the stylized face image into a pre-trained parameter decoder, and generating a face parameter corresponding to the stylized face image. Through the method, the stylized face image and the face parameter which accord with various styles and are relatively high in quality can be generated from the real face image, a user can adjust the face parameter on the basis, a stylized face which better accords with the user demand is generated, and the use experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a method and device for generating facial parameters. Background Art

[0002] As the complexity of games and the realism of graphics continue to increase, in-game characters can also achieve intelligent control, such as path planning, motion control, etc. In order to make game characters more realistic, users can create their own virtual images of game characters. Using their own exclusive character images can greatly enhance users' sense of participation and experience.

[0003] In the prior art, the DualStyleGAN method and the InsightFace method are used to generate stylized face images from input real face images. However, the DualStyleGAN method focuses on modifying the color and size of key facial features, and the stylization quality of the overall image is not high. The stylized faces generated by the InsightFace method retain more detailed features of real faces and cannot restore the style of the game. If the user wants to customize an exclusive character image that conforms to the game style based on a specific real face image, the user adjusts the face parameters based on the stylized faces generated by the DualStyleGAN method and the InsightFace method to generate a target stylized face that meets their own needs. However, since the stylized faces generated by the above two methods are quite different from the game style, even if the user adjusts the face parameters, the quality of the target face image generated still has a lot of room for improvement.

[0004] In summary, how to generate high-quality stylized face images that conform to various game styles from real face images is a problem that needs to be solved at present. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for generating facial parameters, which can generate stylized facial images and facial parameters with high quality that conform to various game styles from real facial images. Users can adjust the facial parameters on this basis to generate stylized faces that better meet user needs, thereby improving user experience.

[0006] In a first aspect, an embodiment of the present invention provides a method for generating facial parameters, the method comprising:

[0007] Get real face images;

[0008] Extracting feature points from the real face image to generate standardized feature points of the real face image;

[0009] Inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image;

[0010] The stylized face image is input into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

[0011] Optionally, the method further includes:

[0012] Output the facial parameters and / or the stylized facial image.

[0013] Optionally, extracting feature points from the real face image to generate standardized feature points of the real face image specifically includes:

[0014] Extracting feature points from the real face image according to a set method to generate face feature points;

[0015] The facial feature points are standardized to generate the standardized feature points.

[0016] Optionally, the pre-trained stylized face image generation model includes a control network and a diffusion model.

[0017] Optionally, the step of inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image specifically includes:

[0018] Inputting the standardized feature points of the real face image into the control network;

[0019] The control network guides the diffusion model to denoise the pure noise image to generate the stylized face image.

[0020] Optionally, the training process of the stylized face image generation model is as follows:

[0021] Get the face parameter set in the set scene;

[0022] Generate a plurality of historical stylized face images by using the face parameter set according to a setting engine;

[0023] Determine a historical real face image corresponding to each of the historical stylized face images;

[0024] Obtaining historical standardized feature points of the historical real face image;

[0025] The stylized face image generation model is trained according to multiple groups of historical standardized feature points of the historical real face images and historical stylized face images corresponding to the historical real face images.

[0026] Optionally, the set scene is a game scene, an animation scene, an advertising scene or a social avatar scene; in response to the set scene being a game scene, the setting engine is a game engine.

[0027] Optionally, determining the historical real face image corresponding to each of the historical stylized face images specifically includes:

[0028] Fine-tuning the style-based generative adversarial network according to the plurality of historical stylized face images to generate a fine-tuned style-based generative adversarial network;

[0029] Inverse mapping the fine-tuned style-based generative adversarial network to generate a facial feature vector using the historical stylized facial image;

[0030] Inputting the facial feature vector into the style-based generative adversarial network to generate a historical real face image;

[0031] Determine the historical real face image corresponding to each historical stylized face image.

[0032] Optionally, the obtaining of historical standardized feature points of the historical real face image specifically includes:

[0033] Extracting feature points from the historical real face image according to a set method to generate historical face feature points;

[0034] The historical facial feature points are standardized to generate the historical standardized feature points.

[0035] Optionally, the training of the stylized face image generation model according to the multiple groups of historical standardized feature points of the historical real face images and the historical stylized face images corresponding to the historical real face images specifically includes:

[0036] The multiple groups of historical standardized feature points of the historical real face images are used as inputs to the stylized face image generation model, and the historical stylized face images are used as outputs. The control network in the stylized face image generation model is trained to guide the diffusion model in the stylized face image generation model to denoise the noisy stylized face images.

[0037] Optionally, the method further includes:

[0038] A parameter decoder is trained according to the plurality of the historical stylized face images and the face parameter set, wherein the parameter decoder is used to inversely map the historical stylized face images into face parameters.

[0039] In a second aspect, an embodiment of the present invention provides a device for generating face parameters, the device comprising:

[0040] An acquisition unit, used for acquiring a real face image;

[0041] A first generating unit, configured to extract feature points from the real face image to generate standardized feature points of the real face image;

[0042] A second generating unit, configured to input the standardized feature points of the real face image into a pre-trained stylized face image generating model to generate a stylized face image;

[0043] The third generating unit is used to input the stylized face image into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

[0044] Optionally, the device further comprises:

[0045] An output unit is used to output the facial parameters and / or the stylized facial image.

[0046] Optionally, the first generating unit is specifically used for:

[0047] Extracting feature points from the real face image according to a set method to generate face feature points;

[0048] The facial feature points are standardized to generate the standardized feature points.

[0049] Optionally, the pre-trained stylized face image generation model includes a control network and a diffusion model.

[0050] Optionally, the second generating unit is specifically configured to:

[0051] Inputting the standardized feature points of the real face image into the control network;

[0052] The control network guides the diffusion model to denoise the pure noise image to generate the stylized face image.

[0053] Optionally, during the training of the stylized face image generation model:

[0054] The acquisition unit is also used to: acquire a set of face parameters in a set scene;

[0055] A fourth generating unit, configured to generate a plurality of historical stylized face images by using the face parameter set according to a setting engine;

[0056] A determination unit, used to determine a historical real face image corresponding to each of the historical stylized face images;

[0057] The acquisition unit is also used to: acquire the historical standardized feature points of the historical real face image;

[0058] A training unit is used to train the stylized face image generation model based on multiple groups of historical standardized feature points of the historical real face images and historical stylized face images corresponding to the historical real face images.

[0059] Optionally, the set scene is a game scene, an animation scene, an advertising scene or a social avatar scene; in response to the set scene being a game scene, the setting engine is a game engine.

[0060] Optionally, the determining unit is specifically configured to:

[0061] Fine-tuning the style-based generative adversarial network according to the plurality of historical stylized face images to generate a fine-tuned style-based generative adversarial network;

[0062] Inverse mapping the fine-tuned style-based generative adversarial network to generate a facial feature vector using the historical stylized facial image;

[0063] Inputting the facial feature vector into the style-based generative adversarial network to generate a historical real face image;

[0064] Determine the historical real face image corresponding to each historical stylized face image.

[0065] Optionally, the acquiring unit is specifically used for:

[0066] Extracting feature points from the historical real face image according to a set method to generate historical face feature points;

[0067] The historical facial feature points are standardized to generate the historical standardized feature points.

[0068] Optionally, the training unit is specifically used for:

[0069] The multiple groups of historical standardized feature points of the historical real face images are used as inputs to the stylized face image generation model, and the historical stylized face images are used as outputs. The control network in the stylized face image generation model is trained to guide the diffusion model in the stylized face image generation model to denoise the noisy stylized face images.

[0070] Optionally, the training unit is further used for:

[0071] A parameter decoder is trained according to the plurality of the historical stylized face images and the face parameter set, wherein the parameter decoder is used to inversely map the historical stylized face images into face parameters.

[0072] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, the memory being used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in the first aspect or any possible embodiment of the first aspect.

[0073] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement a method as described in the first aspect or any possible embodiment of the first aspect.

[0074] In an embodiment of the present invention, a real face image is obtained; feature points are extracted from the real face image to generate standardized feature points of the real face image; the standardized feature points of the real face image are input into a pre-trained stylized face image generation model to generate a stylized face image; the stylized face image is input into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image. Through the above method, a real face image can be generated into a stylized face image and face parameters that conform to various game styles and have high quality. On this basis, the user can adjust the face parameters to generate a stylized face that better meets the user's needs, thereby improving the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0076] FIG. 1( a ) is a schematic diagram of a real face image according to an embodiment of the present invention;

[0077] FIG1( b ) is a schematic diagram of another real face image in an embodiment of the present invention;

[0078] FIG1( c ) is a schematic diagram of a stylized face image according to an embodiment of the present invention;

[0079] FIG1(d) is a schematic diagram of another stylized face image in an embodiment of the present invention;

[0080] FIG2( a ) is a schematic diagram of a stylized face image according to an embodiment of the present invention;

[0081] FIG2( b ) is a schematic diagram of another stylized face image in an embodiment of the present invention;

[0082] FIG3( a ) is a schematic diagram of a real face image according to an embodiment of the present invention;

[0083] FIG. 3( b ) is a schematic diagram of another real face image in an embodiment of the present invention

[0084] FIG3( c ) is a schematic diagram of a stylized face image according to an embodiment of the present invention;

[0085] Figure 4 is a flow chart of a training stylized face image generation model in an embodiment of the present invention;

[0086] Figure 5 is a flow chart of generating a historical real face image in an embodiment of the present invention;

[0087] Figure 6 is a flowchart of another model for training stylized face image generation in an embodiment of the present invention;

[0088] Figure 7 is a structural schematic diagram of a training process in an embodiment of the present invention;

[0089] Figure 8 is a flow chart of a method for generating face parameters in an embodiment of the present invention;

[0090] Fig. 9 is a flow chart of another method for generating facial parameters in an embodiment of the present invention;

[0091] Fig.10 is a schematic diagram of a device for generating facial parameters in an embodiment of the present invention;

[0092] Fig.11 is a schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0093] The present application is described below based on embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, some specific details are described in detail. It is possible for those skilled in the art to fully understand the present application without the description of these details. In order to avoid confusing the essence of the present application, known methods, processes, flows, components and circuits are not described in detail.

[0094] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0095] Unless the context clearly requires otherwise, the words "include", "comprising" and similar words throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, the meaning is "including but not limited to".

[0096] In the description of this application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" is two or more.

[0097] In the prior art, a DualStyleGAN method, an InsightFace method, and a method that relies on the authenticity of the rendering effect in the game to directly calculate the difference with the real face image are used to generate a stylized face image from an input real face image; specifically, the above-mentioned DualStyleGAN method focuses on modifying the color and size of key facial features, and the stylization quality of the overall image is not high. As shown in Figure 1, Figure 1(a) and Figure 1(b) are real face images, Figure 1(c) is a stylized face generated by the DualStyleGAN method of Figure 1(a) or Figure 1(b), and Figure 1(d) is a stylized face image in the game. Figures 1(c) and 1(d) are The style difference is large, that is, the stylized quality of Figure 1(c) generated by the DualStyleGAN method of Figure 1(a) or Figure 1(b) is not high; the stylized face generated by the InsightFace method retains more detailed features of the real face and cannot restore the style of the game. As shown in Figure 2, Figure 2(a) is a stylized face generated by the InsightFace method based on the real face, and Figure 2(b) is a stylized face image in the game. The style difference between Figure 2(a) and Figure 2(b) is large, that is, the stylized quality of Figure 2(a) generated by the InsightFace method is not high; if the user wants to customize the stylized face image based on a specific real face image For an exclusive character image that matches the game style, the user adjusts the facial parameters based on the stylized faces generated by the above-mentioned DualStyleGAN method and InsightFace method to generate a target stylized face that meets their own needs. However, since the stylized faces generated by the above-mentioned DualStyleGAN method and InsightFace method are quite different from the game style, even if the user adjusts the facial parameters, the quality of the target facial image generated still has a lot of room for improvement. The above-mentioned method that relies on the authenticity of the in-game rendering effect and directly calculates the gap with the real facial image is to generate the in-game facial image based on the real facial image, but this method does not stylize the face, as shown in the figure. As shown in FIG. 3 , FIG. 3(a) is a real face image. After the real face image is preliminarily processed, FIG. 3(b) is generated. Then, FIG. 3(b) is calculated and FIG. 3(c) is output. FIG. 3(c) is a face image in the game. FIG. 3(b) and FIG. 3(c) have the same style, that is, the face image in the game generated by the method of directly calculating the difference with the real face image based on the authenticity of the rendering effect in the game is the same as the style of the real face image. The implementation depends on the realistic style of the face in the specific game, that is, the game face and the real face are in the same domain, and the similarity and semantic feature area can be directly calculated. However, not all games have a realistic face model system, and the above method cannot be widely used.Therefore, how to generate high-quality stylized face images that conform to various game styles from real face images is a problem that needs to be solved at present.

[0098] In the embodiments of the present invention, in addition to the need to generate stylized face images of various game styles from real face images in game scenes, animation scenes, advertising scenes, or social avatar scenes also need to generate stylized faces corresponding to various scenes from real face images; that is, in different scenes, it is necessary to generate stylized face images from real face images, and the stylized face image generation process used is the same.

[0099] In the embodiment of the present invention, in order to solve the above problems, a stylized face image generation model is trained, specifically as follows Figure 4 As shown, the specific training process includes the following steps:

[0100] Step S401: Obtain a set of face parameters in a set scene.

[0101] The set scene may be a game scene, an animation scene, an advertising scene, or a social avatar scene, or may be other scenes, which are not limited in the embodiments of the present invention.

[0102] Specifically, a facial parameter set is generated by random sampling of a normal distribution, so that most of the facial parameters in the generated facial parameter set are within a reasonable range, and it is avoided that when the facial parameters are near the maximum or minimum value, an extreme face is generated, which is greatly different from the real portrait.

[0103] In a possible implementation, the facial parameters may be eye size, eye position, eyebrow size, eyebrow shape, eyebrow position, nose position, etc., which are determined based on actual conditions.

[0104] Step S402: generating a plurality of historical stylized face images by using the face parameter set according to a setting engine.

[0105] Specifically, assuming that the setting engine is a game engine, the facial parameter collection is input into the game engine, and each set of facial parameters in the facial parameter collection can generate a historical stylized facial image.

[0106] Step S403: Determine the historical real face image corresponding to each of the historical stylized face images.

[0107] Specifically, each scene (for example, each game) has its own corresponding stylized face image, and the historical real face image corresponding to each of the historical stylized face images is determined, specifically as follows: Figure 5 As shown, the following steps are included:

[0108] Step S501: fine-tune the style-based generative adversarial network according to the plurality of historical stylized face images to generate a fine-tuned style-based generative adversarial network.

[0109] Specifically, a style-based generative adversarial network (Style-Based Generator Architecture for Generative Adversarial Networks, StyleGAN) is fine-tuned according to the multiple historical stylized face images. Before fine-tuning, the StyleGAN can only generate real faces. After fine-tuning with the multiple historical stylized face images, the styled face images can be generated.

[0110] Step S502: inverse mapping the fine-tuned style-based generative adversarial network to generate a facial feature vector using the historical stylized facial image.

[0111] Specifically, the fine-tuned StyleGAN is used in reverse, that is, the facial feature vector can be generated through the historical stylized facial image.

[0112] Step S503: input the facial feature vector into the style-based generative adversarial network to generate a historical real face image.

[0113] Specifically, due to the consistency of the feature space of the StyleGAN before and after adjustment, the facial feature vector can be input into the StyleGAN before fine-tuning, and a historical real facial image can be generated according to the StyleGAN before fine-tuning.

[0114] Step S504: Determine the historical real face image corresponding to each of the historical stylized face images.

[0115] In a possible implementation, since the facial feature vector for generating the historical real facial image is generated based on the historical style facial image, the historical real facial image corresponding to the historical stylized facial image is determined by an inverse mapping method.

[0116] In an embodiment of the present invention, the facial feature vector can be continuously optimized so that the fine-tuned StyleGAN generates a stylized facial image that is closer to a specific game, and then the optimized facial feature vector is input into the StyleGAN before fine-tuning to obtain a paired real facial image. Through the above method, the real facial image corresponding to the stylized facial image can be determined, and then a paired data set of stylized facial images and real facial images can be generated.

[0117] Step S404: Acquire historical standardized feature points of the historical real face image.

[0118] Specifically, feature points of the historical real face image are extracted according to a set method to generate historical face feature points; and the historical face feature points are standardized to generate the historical standardized feature points.

[0119] In a possible implementation, the setting method is media pipe, which is a Python library for extracting feature points from an input face. Since the angle of the face in a historical real face image may not be a straight face, but may be tilted up, down, left, or right, it is necessary to standardize the facial feature points and control the mean and variance of the distribution of all facial feature points.

[0120] Step S405: training the stylized face image generation model according to multiple groups of historical standardized feature points of the historical real face images and historical stylized face images corresponding to the historical real face images.

[0121] Specifically, the multiple groups of historical standardized feature points of the historical real face images are used as input to the stylized face image generation model, and the historical stylized face images are used as output. The control network in the stylized face image generation model is trained to guide the diffusion model in the stylized face image generation model to denoise the noisy stylized face images.

[0122] In a possible implementation, in order to prevent factors such as the background of different faces and the proportion of different faces in the image from affecting the generalization of the method, standardized feature points are used as the input of the control network Control Net in the stylized face image generation model to guide the diffusion model Stable Diffusion in the stylized face image generation model to denoise the noisy stylized face image, wherein the Stable Diffusion can select a pre-trained model of a similar style or other face image models, which is not limited in the embodiment of the present invention.

[0123] In a possible implementation, after step S402, the following steps are further included. Figure 6 As shown, including:

[0124] Step S406: training a parameter decoder according to the plurality of the historical stylized face images and the face parameter set, wherein the parameter decoder is used to inversely map the historical stylized face images into face parameters.

[0125] Specifically, the plurality of stylized face images are taken as input, and the face parameter set is taken as output to train the parameter decoder Decoder in reverse.

[0126] In a possible implementation, the structural diagram of the training process is as follows: Figure 7 As shown, a facial parameter set 701 generates a historical stylized facial image 703 through a setting engine 702, and then a parameter decoder 704 is trained through the historical stylized facial image 703 to reversely generate the facial parameters; the historical stylized facial image 703 generates a historical real face 706 through StyleGAN705, and the historical real face 706 is subjected to feature extraction 707 to generate historical standardized feature points 708, and the historical standardized feature points 708 are input into a control network 709, and the control network 709 is used to guide a diffusion model 710 to denoise a noise image 711 to generate a historical style facial image 712, and the stylized facial image is decoded through a parameter decoder 704 to generate facial parameters 713.

[0127] In the embodiment of the present invention, after the training is completed, the stylized face image generation model can be used to generate a stylized face image and face parameters corresponding to the stylized face image from a real face image, as shown in the following example: Figure 8 As shown, the following steps are included:

[0128] Step S801: Acquire a real face image.

[0129] Step S802: extract feature points from the real face image to generate standardized feature points of the real face image.

[0130] Specifically, feature points are extracted from the real face image according to a set method to generate face feature points; and the face feature points are standardized to generate the standardized feature points.

[0131] Step S803: input the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image.

[0132] The pre-trained stylized face image generation model includes a control network and a diffusion model.

[0133] Specifically, the standardized feature points of the real face image are input into the control network; the control network guides the diffusion model to denoise the pure noise image to generate the stylized face image.

[0134] Step S804: input the stylized face image into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

[0135] In a possible implementation, after step S804, the method further includes other steps, such as Fig. 9As shown, including the following:

[0136] Step S805: output the facial parameters and / or the stylized facial image.

[0137] Specifically, the user can adjust the output facial parameters to generate a stylized facial image required by the user.

[0138] Through the above embodiments, the task of generating face parameters in the game is transformed into the task of face stylization, eliminating the obstacles to comparing faces of different styles. By using standardized face feature points, any input face photo can be directly stylized without restricting the photo information such as face position and background style, removing the mode restriction on the input real face image, and improving the versatility of the model. By fine-tuning the StyleGAN network and the inverse mapping method, the real face data is matched for the game face, providing reference images for subsequent training, thereby improving the quality of stylized face generation; using Stable Diffusion and Control Net networks to accurately reproduce the style, generating high-quality stylized face images.

[0139] In an embodiment of the present invention, a device for generating face parameters is provided. Fig.10 As shown, it specifically includes: an acquisition unit 1001, a first generation unit 1002, a second generation unit 1003 and a third generation unit 1004, wherein the acquisition unit 1001 is used to acquire a real face image; the first generation unit 1002 is used to extract feature points of the real face image to generate standardized feature points of the real face image; the second generation unit 1003 is used to input the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image; the third generation unit 1004 is used to input the stylized face image into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

[0140] Furthermore, the device also includes:

[0141] An output unit is used to output the facial parameters and / or the stylized facial image.

[0142] Furthermore, the first generating unit is specifically used for:

[0143] Extracting feature points from the real face image according to a set method to generate face feature points;

[0144] The facial feature points are standardized to generate the standardized feature points.

[0145] Furthermore, the pre-trained stylized face image generation model includes a control network and a diffusion model.

[0146] Furthermore, the second generating unit is specifically used to:

[0147] Inputting the standardized feature points of the real face image into the control network;

[0148] The control network guides the diffusion model to denoise the pure noise image to generate the stylized face image.

[0149] Furthermore, during the training process of the stylized face image generation model:

[0150] The acquisition unit is also used to: acquire a set of face parameters in a set scene;

[0151] A fourth generating unit, configured to generate a plurality of historical stylized face images by using the face parameter set according to a setting engine;

[0152] A determination unit, used to determine a historical real face image corresponding to each of the historical stylized face images;

[0153] The acquisition unit is also used to: acquire the historical standardized feature points of the historical real face image;

[0154] A training unit is used to train the stylized face image generation model based on multiple groups of historical standardized feature points of the historical real face images and historical stylized face images corresponding to the historical real face images.

[0155] Further, the set scene is a game scene, an animation scene, an advertisement scene or a social avatar scene; in response to the set scene being a game scene, the setting engine is a game engine.

[0156] Furthermore, the determining unit is specifically configured to:

[0157] Fine-tuning the style-based generative adversarial network according to the plurality of historical stylized face images to generate a fine-tuned style-based generative adversarial network;

[0158] Inverse mapping the fine-tuned style-based generative adversarial network to generate a facial feature vector using the historical stylized facial image;

[0159] Inputting the facial feature vector into the style-based generative adversarial network to generate a historical real face image;

[0160] Determine the historical real face image corresponding to each historical stylized face image.

[0161] Furthermore, the acquisition unit is specifically used for:

[0162] Extracting feature points from the historical real face image according to a set method to generate historical face feature points;

[0163] The historical facial feature points are standardized to generate the historical standardized feature points.

[0164] Furthermore, the training unit is specifically used for:

[0165] The multiple groups of historical standardized feature points of the historical real face images are used as inputs to the stylized face image generation model, and the historical stylized face images are used as outputs. The control network in the stylized face image generation model is trained to guide the diffusion model in the stylized face image generation model to denoise the noisy stylized face images.

[0166] Furthermore, the training unit is also used for:

[0167] A parameter decoder is trained according to the plurality of the historical stylized face images and the face parameter set, wherein the parameter decoder is used to inversely map the historical stylized face images into face parameters.

[0168] Fig.11 Schematic diagram of the structure of the electronic device in the embodiment of the present invention. Fig.11 As shown, it includes a general computer hardware structure, which at least includes a processor 1101 and a memory 1102. The processor 1101 and the memory 1102 are connected via a bus 1103. The memory 1102 is suitable for storing instructions or programs executable by the processor 1101. The processor 1101 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 1101 executes the instructions stored in the memory 1102, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 1103 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 1104 and the display device and the input / output (I / O) device 1105. The input / output (I / O) device 1105 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 1105 is connected to the system through an input / output (I / O) controller 1106.

[0169] Among them, the instructions stored in the memory 1102 are executed by at least one processor 1101 to achieve: obtaining a real face image; extracting feature points of the real face image to generate standardized feature points of the real face image; inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image; inputting the stylized face image into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

[0170] Specifically, the electronic device includes: one or more processors 1101 and a memory 1102, Fig.11 Take a processor 1101 as an example. The processor 1101 and the memory 1102 may be connected via a bus or other means. Fig.11 In the example, the bus connection is used. The memory 1102 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor 1101 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 1102, that is, the above-mentioned method for determining the generation of facial parameters is implemented.

[0171] The memory 1102 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1102 may optionally include a memory remotely arranged relative to the processor 1101, and these remote memories may be connected to an external device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] One or more modules are stored in the memory 1102, and when executed by one or more processors 1101, the method for generating facial parameters in any of the above method embodiments is executed.

[0173] As will be appreciated by those skilled in the art, various aspects of the embodiments of the present invention may be implemented as a system, method or computer program product. Therefore, various aspects of the embodiments of the present invention may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation that combines software aspects with hardware aspects, which may generally be referred to herein as a "circuit," "module," or "system." In addition, various aspects of the embodiments of the present invention may take the form of a computer program product implemented in one or more computer-readable media having a computer-readable program code implemented thereon.

[0174] Any combination of one or more computer-readable media can be utilized. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, (but not limited to) electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or apparatuses, or any suitable combination of the foregoing. More specific examples (non-exhaustive enumeration) of computer-readable storage media will include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of an embodiment of the present invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by an instruction execution system, device or apparatus or a program used in conjunction with an instruction execution system, device or apparatus.

[0175] A computer readable signal medium may include a propagated digital signal having a computer readable program code implemented therein, such as in baseband or as part of a carrier wave. Such propagated signals may take any of a variety of forms, including but not limited to electromagnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any of the following computer readable media: not a computer readable storage medium, and may communicate, propagate, or transmit a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0176] Program code embodied on a computer readable medium may be transmitted using any appropriate medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0177] The computer program code for performing operations for various aspects of the embodiments of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional process programming languages ​​such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer as a stand-alone software package; partially on the user's computer and partially on a remote computer; or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using the Internet of an Internet service provider).

[0178] The flowchart legend and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present invention described above describe various aspects of the embodiment of the present invention.It will be understood that each block of the flowchart legend and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be implemented by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instructions (executed by the processor of the computer or other programmable data processing device) create a device for implementing the function / action specified in the flowchart and / or block diagram block or block.

[0179] These computer program instructions may also be stored in a computer-readable medium that can direct a computer, other programmable data processing device, or other apparatus to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0180] The computer program instructions may also be loaded onto a computer, other programmable data processing device or other apparatus so that a series of operable steps are performed on the computer, other programmable device or other apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in the flowchart and / or block diagram block or blocks.

[0181] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and pictures involved in this application are all information, data and pictures authorized by the user or fully authorized by all parties. Among them, the pictures are mainly used to show the algorithm for generating facial parameters, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse. The user's refusal to process personal information other than the necessary information required for basic functions will not affect the user's use of basic functions.

Claims

1. A method for generating facial parameters, characterized in that: The method comprises: Get real face images; Extracting feature points from the real face image to generate standardized feature points of the real face image; Inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image; The stylized face image is input into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

2. The method according to claim 1, characterized in that The method further comprises: Output the facial parameters and / or the stylized facial image.

3. The method according to claim 1, characterized in that The step of extracting feature points from the real face image to generate standardized feature points of the real face image specifically includes: Extracting feature points from the real face image according to a set method to generate face feature points; The facial feature points are standardized to generate the standardized feature points.

4. The method according to claim 1, characterized in that: The pre-trained stylized face image generation model includes a control network and a diffusion model.

5. The method according to claim 4, characterized in that The step of inputting the standardized feature points of the real face image into a pre-trained stylized face image generation model to generate a stylized face image specifically includes: Inputting the standardized feature points of the real face image into the control network; The control network guides the diffusion model to denoise the pure noise image to generate the stylized face image.

6. The method according to claim 1, characterized in that The training process of the stylized face image generation model is as follows: Get the face parameter set in the set scene; Generate a plurality of historical stylized face images by using the face parameter set according to a setting engine; Determine a historical real face image corresponding to each of the historical stylized face images; Obtaining historical standardized feature points of the historical real face image; The stylized face image generation model is trained according to multiple groups of historical standardized feature points of the historical real face images and historical stylized face images corresponding to the historical real face images.

7. The method according to claim 6, characterized in that The set scene is a game scene, an animation scene, an advertising scene or a social avatar scene; In response to the setting scene being a game scene, the setting engine is a game engine.

8. The method according to claim 6, characterized in that The determining of the historical real face image corresponding to each of the historical stylized face images specifically includes: Fine-tuning the style-based generative adversarial network according to the plurality of historical stylized face images to generate a fine-tuned style-based generative adversarial network; Inverse mapping the fine-tuned style-based generative adversarial network to generate a facial feature vector using the historical stylized facial image; Inputting the facial feature vector into the style-based generative adversarial network to generate a historical real face image; Determine the historical real face image corresponding to each historical stylized face image.

9. The method according to claim 6, characterized in that The step of obtaining the historical standardized feature points of the historical real face image specifically includes: Extracting feature points from the historical real face image according to a set method to generate historical face feature points; The historical facial feature points are standardized to generate the historical standardized feature points.

10. The method according to claim 6, characterized in that The training of the stylized face image generation model according to the multiple groups of historical standardized feature points of the historical real face images and the historical stylized face images corresponding to the historical real face images specifically includes: The multiple groups of historical standardized feature points of the historical real face images are used as inputs to the stylized face image generation model, and the historical stylized face images are used as outputs. The control network in the stylized face image generation model is trained to guide the diffusion model in the stylized face image generation model to denoise the noisy stylized face images.

11. The method according to claim 6, characterized in that The method further comprises: A parameter decoder is trained according to the plurality of the historical stylized face images and the face parameter set, wherein the parameter decoder is used to inversely map the historical stylized face images into face parameters.

12. A device for generating facial parameters, characterized in that: The device comprises: An acquisition unit, used for acquiring a real face image; A first generating unit, configured to extract feature points from the real face image to generate standardized feature points of the real face image; A second generating unit, configured to input the standardized feature points of the real face image into a pre-trained stylized face image generating model to generate a stylized face image; The third generating unit is used to input the stylized face image into a pre-trained parameter decoder to generate face parameters corresponding to the stylized face image.

13. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images