Face generation method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, in particular to a face generation method and device, electronic equipment and a storage medium, and the device comprises a face changing module which is used for carrying out face changing operation on a face in a base image based on a face in a reference image to obtain an initial face changing image containing the generated face; and the face regeneration module is used for regenerating the generated face in the initial face change image based on the true and false face difference parameter to obtain a target face change image, and the true and false face difference parameter is used for representing the difference between the true face and the generated face. According to the embodiment of the invention, the target face changing image similar to a real face can be obtained, the slight difference between the target face changing image and the real face is reduced, and the authenticity and naturalness of the generated face are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a face re-generation module construction method, a face generation method and device, an electronic device and a storage medium. BACKGROUND

[0002] In the field of artificial intelligence content production (AIGC), face generation technology is a popular application direction. Traditional face generation methods usually rely on one or more face photos input by a user, and through simple text description or selection of options in a template library, a face image satisfying the user can be generated. These generated face images can be used in various scenarios, such as ID photos, travel photos, and realizing the user's wild imagination, such as generating classic cartoon characters or main characters in a hit drama.

[0003] At present, some face generation technology solutions have achieved remarkable results. However, although these technology solutions have made significant progress in improving the similarity between the generated face and the reference face, there is still a deficiency in the realism of the generated image. Realism is an important indicator for evaluating the quality of the generated face, especially in some scenarios that require "blurring the line between reality and fiction" and "de-AI", such as resume photos, work photos, wedding photos, etc., the demand for realism is particularly urgent.

[0004] However, existing face generation technology solutions often focus too much on improving similarity and ignore optimizing realism, resulting in poor realism of the generated face. SUMMARY

[0005] The present disclosure proposes a face generation technology solution.

[0006] According to an aspect of the present disclosure, a face generation device is provided, comprising:

[0007] a face changing module configured to perform a face changing operation on a face in a base image based on a face in a reference image to obtain an initial face changing image containing a generated face;

[0008] a face re-generation module configured to perform face re-generation on the generated face in the initial face changing image based on a real and fake face difference parameter to obtain a target face changing image, the real and fake face difference parameter being used to represent the difference between a real face and the generated face.

[0009] In a possible implementation manner, the device further comprises:

[0010] a base image generation module configured to generate the base image according to a prompt condition;

[0011] a fine drawing and magnification module configured to perform fine drawing and magnification on the base image to obtain a fine drawing and magnification base image.

[0012] In a possible implementation, the face-changing module includes:

[0013] A reference feature mapping module is used to map the facial features in the reference image to the face area in the base image to obtain an intermediate feature tensor that integrates the reference facial features;

[0014] A face posture maintaining module, configured to generate face posture control data based on key facial points in the base image, wherein the generated face posture control data is used to maintain the posture of the generated face consistent with the facial posture in the base image;

[0015] A character posture maintaining module, configured to generate character posture control data based on edge data in the base image, wherein the character posture control data is used to maintain the overall consistency between the posture of the generated face and the posture of the character;

[0016] The basic image generation module is used to redraw the base image using the facial posture control data, the character posture control data and the intermediate feature tensor as control signals to obtain an initial face-changing image containing the generated face.

[0017] In one possible implementation, the basic image generation module includes a conditional generation model and an unconditional generation model. When generating an initial face-changing image, the basic image generation module inputs the facial posture control data, the character posture control data and the intermediate feature tensor as control signals into the unconditional generation model, redraws the base image, generates an initial face-changing image, and disables the conditional generation model.

[0018] In a possible implementation, the face regeneration module includes:

[0019] a cropping module, configured to crop the generated face in the initial face-swapped image to obtain a generated face image;

[0020] A real-false face transformation module, configured to transform the generated face image based on the real-false face difference parameters to obtain a real-like face image;

[0021] The splicing module is used to splice the real-looking face image back into the initial face-swapped image to obtain a target face-swapped image.

[0022] In a possible implementation, the true and false face difference parameter is obtained by training a face generation model based on true and false face samples;

[0023] The face sample pair includes a real face sample and a generated face sample, and the generated face sample is obtained by performing a face generation operation using the real face sample as a reference face.

[0024] In a possible implementation, the real-fake face difference parameter is constructed based on a difference between the first face generation model and the second face generation model trained based on the same sample pair.

[0025] The first face generation model is trained based on a reference condition as input and the real face sample as expected output; and the first face generation model outputs a face consistent with the real face sample when any reference condition is input.

[0026] The second face generation model is trained based on the generated face sample as expected output; and the second face generation model outputs a face consistent with the generated face sample when any reference condition is input.

[0027] In a possible implementation, the real-fake face difference parameter is obtained by averaging a plurality of parameter difference values corresponding to sample pairs, and the parameter difference value is a difference between parameters of the first face generation model and the second face generation model trained based on the same sample pair.

[0028] According to an aspect of the present disclosure, a face re-generation module construction method is provided for generating a face re-generation module in the above device, comprising:

[0029] A data set composed of real-fake face sample pairs is constructed, wherein the real-fake face sample pairs include real face samples and generated face samples.

[0030] A real-fake face difference parameter in the face re-generation module is constructed based on a difference between the real face sample and the generated face sample, and a face re-generation module constructed based on the real-fake face difference parameter is obtained.

[0031] In a possible implementation, the data set composed of real-fake face sample pairs is constructed, comprising:

[0032] A generated face is obtained by performing a face generation operation with a face in the real face sample as a reference face.

[0033] The real face in the real face sample is replaced by the generated face to obtain a generated face sample.

[0034] In a possible implementation, the real-fake face difference parameter in the face re-generation module is constructed based on a difference between the real face sample and the generated face sample, comprising:

[0035] The first face generation model is trained by taking the reference condition as input and the real face sample as expected output, so that the first face generation model outputs a face consistent with the real face sample when any reference condition is input;

[0036] The second face generation model is trained by taking the reference condition as input and the generated face sample as expected output, so that the second face generation model outputs a face consistent with the generated face sample when any reference condition is input;

[0037] The difference between the first face generation model and the second face generation model trained based on the same sample pair is used to construct a real-fake face difference parameter in a face re-generation module.

[0038] In a possible implementation, the difference between the first face generation model and the second face generation model trained based on the same sample pair is used to construct a real-fake face difference parameter in a face re-generation module, including:

[0039] The parameter difference between the first face generation model and the second face generation model trained based on the same sample pair is determined;

[0040] The parameter difference values corresponding to the plurality of sample pairs are averaged to obtain a real-fake face difference parameter.

[0041] According to an aspect of the present disclosure, a face generation method is provided, including:

[0042] Performing face replacement on a face in a base image based on a face in a reference image to obtain an initial face replacement image containing a generated face;

[0043] A real-fake face difference parameter is used to re-generate the generated face in the initial face replacement image to obtain a target face replacement image, and the real-fake face difference parameter is used to represent the difference between a real face and a generated face.

[0044] According to an aspect of the present disclosure, a face re-generation module construction device is provided for generating a face re-generation module in the above device, including:

[0045] A data set construction module is configured to construct a data set composed of real-fake face sample pairs, wherein the real-fake face sample pairs contain real face samples and generated face samples;

[0046] A difference parameter construction module is configured to construct a real-fake face difference parameter in a face re-generation module according to the difference between the real face samples and the generated face samples, and obtain a face re-generation module constructed based on the real-fake face difference parameter.

[0047] According to an aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

[0048] According to an aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions, which, when executed by a processor, implement the above method.

[0049] In the embodiments of the present disclosure, the face changing module performs face changing operation on the face in the bottom image based on the face in the reference image to obtain an initial face changing image containing a generated face; the face re-generation module performs re-generation on the generated face in the initial face changing image based on the real and fake face difference parameter to obtain a target face changing image, the real and fake face difference parameter being obtained based on the difference between the real face and the generated face. Thus, since the real and fake face difference parameter is obtained based on the difference between the real face and the generated face, re-generation on the generated face in the initial face changing image based on the real and fake face difference parameter can obtain a target face changing image similar to the real face, reducing the subtle difference from the real face and improving the authenticity and naturalness of the generated face.

[0050] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0052] Figure 1 A block diagram of a face generation apparatus according to an embodiment of the present disclosure is shown.

[0053] Figure 2 A block diagram of a specific implementation of a face generation apparatus according to an embodiment of the present disclosure is shown.

[0054] Figure 3 A flowchart of a face re-generation module construction method according to an embodiment of the present disclosure is shown.

[0055] Figure 4 A schematic diagram of a set of sample pairs according to an embodiment of the present disclosure is shown.

[0056] Figure 5 A flowchart of a face generation method according to an embodiment of the present disclosure is shown.

[0057] Figure 6 A block diagram of a face re-generation module construction apparatus is shown.

[0058] Figure 7 A block diagram of an electronic device is shown. DETAILED DESCRIPTION

[0059] Various exemplary embodiments, features, and aspects of the present disclosure will be explained in greater detail below with reference to the accompanying drawings. Like reference numerals may be used to refer to like elements throughout the several embodiments. Unless explicitly described otherwise, the drawings shown in the several embodiments are not necessarily drawn to scale.

[0060] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0061] The term "and / or" used herein only means an association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0062] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail in order to highlight the main idea of the present disclosure.

[0063] Figure 1 A block diagram of a face generation apparatus is shown. Figure 1 As shown, the apparatus comprises:

[0064] The face changing module 11 is configured to perform a face changing operation on the face in the bottom image based on the face in the reference image, to obtain an initial face changing image containing a generated face.

[0065] The face re-generation module 12 is configured to perform face re-generation on the generated face in the initial face changing image based on a real and fake face difference parameter, to obtain a target face changing image, wherein the real and fake face difference parameter is used to represent the difference between the real face and the generated face.

[0066] The face changing module can apply the facial features in the reference image to the face in another base image. The reference image is used as a reference for generating a new face image, and the face changing module can extract the facial features (such as facial feature shape, skin color, expression, etc.) in the reference image and replace the facial features in the base image. For example, a user uploads a photo of himself or an image containing the facial features of a target person, which will be used as a reference for generating a new face image.

[0067] The face changing module can accurately identify the facial features in the reference image through face recognition and replacement technology, and match and replace them with the face in the base image. And in the process of replacement, the generated face is consistent with the base image in overall structure and posture. For example, a user can provide a photo of himself as a reference image and select a template base image (such as an ID photo template). The face changing module can replace the facial features in the user's photo with the template base image to generate an initial face changing image containing the user's facial features.

[0068] The face regeneration module uses real and fake face difference parameters to further optimize the initial face changing image. Real and fake face difference parameters are used to measure and represent the subtle differences between real faces and generated faces during face generation, so that the face regeneration model generates a target face changing image that closely resembles a real face. These parameters are obtained by comparing and analyzing the differences between real face samples and generated face samples and training. These differences may include skin texture, gloss, color, and other subtle features. The face regeneration module uses these parameters to fine-tune the generated face in the initial face changing image to make it more closely resemble a real face.

[0069] When constructing real and fake face difference parameters, a set of data sets composed of real face samples and generated face sample pairs can be constructed. Then, based on machine learning or deep learning techniques, these sample pairs are used to train the face generation model and extract model parameters that can represent the differences between real faces and generated faces. For the specific construction process, please refer to the face regeneration module construction method provided by the present disclosure, which will not be repeated here.

[0070] The face regeneration module uses real and fake face difference parameters to generate a new face based on the generated face in the initial face changing image by adding noise and denoising to obtain a target face changing image similar to a real face, thereby reducing the subtle differences between the generated face and the real face and improving the realism and naturalness of the generated face.

[0071] In the embodiments of the present disclosure, the face changing module performs face changing operation on the face in the bottom image based on the face in the reference image to obtain an initial face changing image containing a generated face; and the face re-generation module performs re-generation on the generated face in the initial face changing image based on the real and fake face difference parameters to obtain a target face changing image, wherein the real and fake face difference parameters are obtained based on the difference between the real face and the generated face. Thus, since the real and fake face difference parameters are obtained based on the difference between the real face and the generated face, the re-generation on the generated face in the initial face changing image based on the real and fake face difference parameters can obtain a target face changing image similar to the real face, reduce the subtle differences with the real face, and improve the authenticity and naturalness of the generated face.

[0072] In a possible implementation, the device further includes a base image generation module configured to generate the bottom image according to a prompt condition; and a fine drawing magnification module configured to perform fine drawing magnification on the bottom image to obtain a fine-drawing-magnified bottom image.

[0073] The base image generation module can quickly generate a bottom image according to a given prompt condition (such as a text description or an image example). The bottom image does not need to be particularly exquisite, but only needs to contain the general outline and basic features, so the generation time can be greatly shortened.

[0074] The base image generation module can be a trained text-to-image model or an image-to-image model. The model can be the same as the base model of the face generation model used in the subsequent training of the real and fake face difference parameters, so as to ensure that the parameters of the model are one-to-one corresponding to the real and fake face difference parameters in the subsequent steps.

[0075] In order to generate images more in line with the aesthetics of Asians, a Low-Rank Adaptation (LoRA) adjustment method for Asian facial features can be selectively applied to fine-tune the base image generation module. LoRA is a lightweight model adjustment method that can adapt to new tasks or styles by adjusting a small number of parameters without changing the original model structure.

[0076] In addition, if the user wants to generate the style of a known picture, the base image generation module can also input a style image. The module can contain an adapter for image style transfer, which can integrate specific style features into the generated image without changing the overall structure of the model.

[0077] The fine-drawing upscaling module can perform fine-drawing upscaling on the base image generated by the base image generation module to obtain a high-resolution and high-quality image. The fine-drawing upscaling model can use a diffusion model to perform fine-drawing upscaling on the base image. The diffusion model is a generative model that can simulate the generation process of data by gradually adding noise, and then generate a high-quality image through a reverse process (i.e., denoising).

[0078] To achieve fine-drawing upscaling, the fine-drawing upscaling module can perform fine-drawing upscaling based on a tile control network (Tile ControlNet). The Tile ControlNet can guide the model to perform tile processing on the image and add details while maintaining the overall layout, thereby achieving higher-quality image generation. Through the diffusion model and the Tile ControlNet in the fine-drawing upscaling model, an image with realistic skin texture and high resolution can be generated.

[0079] Then, the face swapping module can perform face swapping on the face in the fine-drawing upscaled base image based on the face in the reference image to obtain an initial face-swapped image containing a generated face. Then, the face re-generation module can perform face re-generation on the generated face in the initial face-swapped image based on the real and fake face difference parameters to obtain a target face-swapped image.

[0080] Figure 2 A block diagram illustrating one specific implementation of a face generation apparatus according to an embodiment of the present disclosure is shown. In this implementation, the base image generation module is a text-to-image model that can generate a base image from text description and style generated by a style model, and the text-to-image model is a model fine-tuned by Asian face LoRA. The base image is upscaled and fine-drawn by the fine-drawing upscaling model to obtain an upscaled and fine-drawn image. The face swapping module includes an instant ID control network (instantIDControlnet) and an edge control network (edge Controlnet) module, which are used to perform face swapping on the base image based on a reference face and face key points obtained by face key point detection on the base image to obtain an initial face-swapped image containing a generated face. Then, the face in the initial face-swapped image is cropped out by face detection. Then, the face re-generation module performs face re-generation on the cropped face based on real and fake face difference parameters (real and fake face difference LoRA) to paste the re-generated real-like face back into the base image to obtain a target face-swapped image. Finally, a beauty model is used to perform beauty on the target face-swapped image to obtain a final face-swapped image.

[0081] In a possible implementation, the face changing module comprises: a reference feature mapping module, configured to map a face feature in a reference image to a face region in a base image to obtain an intermediate feature tensor of a fused reference face feature; a face posture maintaining module, configured to generate face posture control data based on a face key point in the base image, the face posture control data being used to maintain a posture of a generated face consistent with a face posture in the base image; a character posture maintaining module, configured to generate character posture control data based on edge data in the base image, the character posture control data being used to maintain the posture of the generated face consistent with a posture of a character as a whole; and a base image generating module, configured to redraw the base image by taking the face posture control data, the character posture control data, and the intermediate feature tensor as control signals to obtain an initial face changing image containing the generated face.

[0082] The reference feature mapping module can accurately map the face feature in the reference image to the face region in the base image. Then, the features of the reference face (the face that the user wants to change) provided by the user, such as the shape of the facial features, the color of the eyes, and the like, will be extracted and applied to the corresponding positions of the base image (that is, the original image containing the target region of the face that the user wants to replace).

[0083] The reference feature mapping module can be a lightweight adapter network that can effectively integrate the feature information of a specific field into the pre-trained model without changing the overall structure of the base graphics generation model (pre-trained model), that is, aligning and fusing the features of the reference face with the base image.

[0084] The reference feature mapping module can extract high-level semantic features (such as facial feature contours and skin texture) of the reference face through a pre-trained network, and then inject these features into the intermediate layer of the base diffusion model through a cross-attention mechanism. In this process, the features of the reference face are encoded into a feature vector, and matrix multiplication is performed with the face region features of the base image to generate a fused intermediate feature tensor. In this way, the fusion of the reference face features and the features in the base image is not a simple pixel replacement, but an identity information migration in the feature space, ensuring that the generated face is highly close to the reference face in vision while maintaining coordination with the lighting environment of the base image.

[0085] The face posture maintaining module can ensure that the posture of the generated face is consistent with the face posture in the base image during the generation of the image. That is, no matter whether the face in the base image is front-facing, side-facing, or any other angle, the generated face will maintain the same posture. The face posture maintaining module guides the image generation process by introducing spatial condition information (such as face key points and posture), and uses this information to adjust the local features of the generated image to ensure that the generated face is consistent with the base image in posture.

[0086] The face posture maintaining module first pre-processes the input base image, extracts face key points using a face key point detection algorithm, for example, can extract face key point coordinates of positions such as eye corners, nose tips, and mouth corners. Then, after converting these coordinate data into a heat map, input the heat map into the encoder of the face posture control network, the face posture control network will generate a spatial control signal (i.e. face posture control data), which will act on each denoising iteration process of the diffusion model of the base image generation module through a residual connection method. At each step of the diffusion model generating an image, the face posture control data will adjust the face generation direction to ensure that the final output head rotation angle and expression state are completely matched with the original face of the base image.

[0087] The character posture maintaining module can keep the posture of the generated face consistent with the overall posture of the whole character. That is, when generating an image, not only the posture of the face, but also the overall posture of the character (such as body posture, arm position, etc.) should be consistent. Figure 1

[0088] The character posture maintaining module extracts the posture information of the character in the base image by using edge detection technology, and uses this information to guide the generation process of the generated image. Specifically, a binary image representing the posture information of the character can be input into the encoder of the edge control network to generate multi-scale geometric control signals (i.e. character posture control data). The character posture control data corrects the geometric structure of the generated image layer by layer through feature superposition in the decoding stage of the diffusion model of the base image generation module. By ensuring that the posture of the character in the generated image is consistent with the posture of the character in the base image, the realism and coordination of the generated image can be further improved. Figure 1

[0089] The base image generation module is a basic image generation module for generating images according to control signals or prompt conditions. This module can be used to generate a base image, and in face swapping, this module is used to generate an image in the face swapping module and keep the global transition of the generated initial face-swapped image natural.

[0090] The base image generation module provides an initial image generation framework, and other modules (such as the reference feature mapping module, the face posture maintaining module, and the character posture maintaining module) modify and enhance the base image generation module in the form of intermediate layers or branches based on this framework to obtain the face swapping module.

[0091] ​​The base image generation module can be a pre-trained diffusion model or a generative adversarial network (GAN) that has learned how to generate high-quality images. Then, in the face swapping process of the present implementation, the base image generation module generates an initial image similar to the base image, and then gradually redraws the initial image through denoising iterations. At each denoising iteration, the face pose control data, the character pose control data, and the intermediate feature tensor provided by the prequel module are fused, and the image is gradually redrawn through multiple iterations to obtain an initial face swapping image containing a generated face.

[0092] In the embodiments of the present disclosure, the reference feature mapping module can accurately map the facial features in the reference image to the face region in the base image. The face pose maintaining module ensures that the pose of the generated face is consistent with the pose of the face in the base image during the generation of the image. This module can identify and replicate subtle changes in the pose of the face in the base image, such as the head tilt angle and facial expressions, making the generated image appear very natural in terms of face pose. The character pose maintaining module can maintain the overall consistency of the pose of the generated face with the pose of the character, ensuring that the generated face is coordinated with the pose, action, and background environment of the body, avoiding the appearance of "patchiness" or "incoordination". The base image generation module is responsible for the generation of the overall image and maintains the natural global transition of the generated face swapping image. That is, after mapping the reference facial features to the base image, the generated image can be highly consistent with the base image in terms of overall color, brightness, texture, etc., making the face swapping effect appear more natural and realistic.

[0093] In summary, in the present implementation, by integrating multiple key modules, precise face feature mapping, face pose consistency, overall coordination of character pose, and natural global transition can be achieved. These effects collectively act on the generated image, making the final face swapping effect very realistic and natural, and meeting the needs of users in various scenarios.

[0094] In a possible implementation, the base image generation module includes a conditional generative model and an unconditional generative model. When generating an initial face swapping image, the base image generation module inputs the face pose control data, the character pose control data, and the intermediate feature tensor as control signals into the unconditional generative model, redraws the base image, generates an initial face swapping image, and disables the conditional generative model.

[0095] The base image generation module includes a conditional generative model (Conditional Generative Model) and an unconditional generative model (Unconditional Generative Model), wherein:

[0096] The conditional generation model considers additional conditional information when generating images. These conditional information can be labels, text descriptions, images or other forms of data, which are used to guide the image generation process so that the generated image meets certain requirements or styles.

[0097] The non-conditional generation model does not require any additional conditional information, and the model generates images from random noise. This type of model is usually used to generate images with certain statistical properties, such as style transfer, image completion, etc. When the face changing module performs face changing operation, i.e. the base image generation module generates the initial face changing image, a non-conditional generation model is used to generate images, and the conditional generation model is disabled.

[0098] The base image generation module receives the human face pose control data, the character pose control data and the intermediate feature tensor, which are uniformly input to the non-conditional generation model. Since the conditional generation model is disabled, these data are not input to the conditional generation model. The non-conditional generation model performs iterative denoising and redrawing on the base image according to the input data, integrates the input data into the base image through residual connection and feature superposition, redraws the base image in a denoising manner, and gradually reconstructs the initial face changing image through multiple iterations.

[0099] At this time, since the conditional generation model is disabled, the system does not need to process additional conditional information, nor does it need to generate images according to the conditional information. The system does not need to calculate and store tensors related to the conditional generation model, thereby simplifying the generation process and reducing the consumption of video memory.

[0100] In the embodiments of the present disclosure, the base image generation module can be used to generate the base image, can be included in the fine drawing upscaling module to generate the fine drawing upscaled base image by diffusion image generation, can be included in the face changing module to generate images and keep the global transition of the generated initial face changing image natural, and can be included in the face re-generation module to obtain a real face-like image.

[0101] Since the generated base image is already obtained based on the prompt condition, it only needs to replace the face in the base image with the reference face, so when the face changing module performs face changing, it does not need to generate a face changing image based on the prompt condition. Therefore, the conditional generation model in the base image generation module can be disabled, thereby simplifying the generation process and reducing the consumption of video memory.

[0102] The base image generation model also includes a noise intensity parameter, which is used to control the intensity of adding noise to the image in image processing. A smaller noise intensity parameter value means a lower noise intensity, and the generated image will be closer to the original image. A larger noise intensity parameter value will result in a significant difference between the generated image and the original image, and even the generated image may completely change into another style of image.

[0103] Since the main task of the face changing module is to change faces, the image information of the original image needs to be preserved as much as possible, therefore, a smaller noise addition intensity parameter value can be used to reduce the data processing amount of the subsequent denoising generated image, thereby accelerating the overall face changing speed.

[0104] In a possible implementation, the face re-generation module comprises: a cutting module configured to cut the generated face in the initial face changing image to obtain a generated face image; a real and fake face transformation module configured to transform the generated face image based on the real and fake face difference parameters to obtain a real-like face image; and a splicing module configured to splice the real-like face image back into the initial face changing image to obtain a target face changing image.

[0105] The cutting module can cut the generated face in the initial face changing image. The cutting module can accurately identify the face region in the initial face changing image by using a face detection algorithm, and cut it out to form a generated face image. The size of the generated face image cut out can be the same as the size of the face sample used when constructing the real and fake face difference parameters later.

[0106] The real and fake face transformation module transforms the generated face image cut out based on the real and fake face difference parameters. The real and fake face transformation module can be constructed by adding the real and fake face difference parameters to the basic image generation module, that is, adding the real and fake face difference parameters to the parameters in the basic image generation module to obtain new parameters as the parameters of the real and fake face transformation module.

[0107] Then, the generated face image is input into the real and fake face transformation model, and after transformation, the generated face image will present a higher degree of realism. For ease of description, the face image output by the real and fake face transformation model is referred to as a real-like face image.

[0108] Then, the splicing module splices the real-like face image obtained after transformation back into the initial face changing image to form a target face changing image. The splicing module can seamlessly splice the real-like face image and the background part of the initial face changing image based on an image splicing algorithm or related technology. For example, a face mask can be used to map the face back into the image, and the face mask will block the non-face area in the real-like face image, realizing the splicing of only the face part back into the initial face changing image, and ensuring that the spliced image is visually consistent. After splicing, the target face changing image obtained will have a higher degree of realism and better visual effect.

[0109] Finally, the face beautification module can be used to perform face beautification operations on the target face changing image to slightly adjust and beautify the generated face image, and obtain the final face changing image.

[0110] In a possible implementation, the real and fake face difference parameter is obtained based on training of a face generation model by using a real and fake face sample pair, the real and fake face sample pair includes a real face sample and a generated face sample, and the generated face sample is obtained by performing face generation on the real face sample as a reference face.

[0111] In a possible implementation, the real and fake face difference parameter is constructed based on a difference between a first face generation model and a second face generation model trained by using the same sample pair.

[0112] The first face generation model is obtained by training the face generation model by using a reference condition as input and the real face sample as expected output, and the first face generation model outputs a face consistent with the real face sample when any reference condition is input.

[0113] The second face generation model is obtained by training the face generation model by using the generated face sample as expected output, and the second face model outputs a face consistent with the generated face sample when any reference condition is input.

[0114] In a possible implementation, the real and fake face difference parameter is obtained by averaging a plurality of sample pair corresponding parameter differences, and the parameter difference is a difference between parameters of the first face generation model and the second face generation model trained by using the same sample pair.

[0115] In addition, the present disclosure also provides a face re-generation module construction method for generating the face re-generation module in any device in the present disclosure. Figure 3 A flowchart of the face re-generation module construction method according to an embodiment of the present disclosure is shown in FIG. 2. Figure 3 As shown in FIG. 2, the method includes the following steps.

[0116] In step S21, a data set composed of real and fake face sample pairs is constructed, and the real and fake face sample pair includes a real face sample and a generated face sample.

[0117] In this process, real face samples can be collected, and the real face samples can be derived from public face data sets, and in addition, a certain number of real high-definition face photos can also be collected. The faces in the real face samples can cover different ages, genders and poses to ensure the diversity and comprehensiveness of the data set.

[0118] In a possible implementation, the constructing the data set composed of the real and fake face sample pairs includes: taking a face in the real face sample as a reference face, performing a face generation operation to obtain a generated face; and replacing the real face in the real face sample with the generated face to obtain a generated face sample.

[0119] In this implementation, for any real face sample, the face in the real face sample can be taken as the reference face to replace the face in the real face sample itself. In this process, the face generation operation can generate a generated face that is consistent with the reference face in overall facial structure, layout of facial features, and the like, but has slight differences in skin texture, light and shadow effect, expression details, and the like.

[0120] After the generated face is obtained, the generated face is replaced into a corresponding position in the original real face sample by using image synthesis technology such as image fusion, image replacement, and the like, to realize "self-replacement of self" of the face in the real face sample, and to obtain a generated face sample containing the generated face. Thus, a sample pair containing a real face and a generated face, that is, a real and fake face sample pair, can be obtained, in which the real face comes from the original real face sample, and the generated face is obtained by the face generation operation.

[0121] Figure 4 A schematic diagram of a set of sample pairs provided according to an embodiment of the present disclosure is shown, in which the left image is a real face picture, and the right image is a generated face picture. The two pictures are almost consistent in overall structure, but the difference between the real face picture and the generated face picture (the skin of the AI-generated face is too smooth) can be obviously seen in details. It should be noted that, to protect privacy, key parts of the face image are coded, but the image details in the uncoded area are not affected.

[0122] The above process is repeated to perform face generation and replacement operations on multiple real face samples, to construct a data set containing multiple real and fake face sample pairs. The data set will serve as a basis for subsequent training of a face generation model and construction of real and fake face difference parameters.

[0123] In the embodiment of the present disclosure, by taking a face in a real face sample as a reference face, a face generation operation is performed to obtain a generated face; and the generated face is replaced into the real face sample to obtain a generated face sample. Since the generated face is consistent with the original face in facial features, posture, and the like, but has slight differences in skin texture, light and shadow effect, expression details, and the like, the generated face sample is a real and fake face sample pair. Figure 1Therefore, the real face sample and the generated face sample in the obtained sample pair are visually very close, forming a "pixel-level-like" match. That is, the real face sample and the generated face sample are close to each other at the same position, and only have slight differences. Then, the sample pair with the "pixel-level-like" match can be used to construct the real-fake face difference parameters in the face re-generation module, so that the face generation model can only add real details to the generated face, without changing the overall framework and pose of the face, and without adding real details to other areas outside the face, thereby improving the accuracy and authenticity of the subsequent generated real face.

[0124] In step S22, the real-fake face difference parameters in the face re-generation module are constructed according to the difference between the real face sample and the generated face sample, and the face re-generation module constructed based on the real-fake face difference parameters is obtained.

[0125] In this step, the key differences between the real face sample and the generated face sample obtained by the face generation model are identified and analyzed. These differences can be reflected in skin texture, light and shadow effect, expression details, naturalness of facial features, and other aspects. By accurately capturing these differences, real-fake face difference parameters can be further constructed, which can reflect the essential difference between the real face and the generated face.

[0126] Specifically, a first face generation model capable of outputting a real face can be trained using the real face sample in the same sample pair, and a second face generation model capable of outputting a generated face can be trained using the generated face sample. The face re-generation module can be constructed by comparing the difference between the parameters of the first face generation model and the second face generation model.

[0127] The initial model during training can be a face generation model, which can be a basic image generation model in the embodiments of the present disclosure, or other face generation models.

[0128] In a possible implementation, the constructing the real and fake face difference parameter in the face re-generation module according to the difference between the real face sample and the generated face sample comprises: training the face generation model to obtain a first face generation model, taking a reference condition as input and the real face sample as expected output; the first face generation model outputs a face image consistent with the real face sample when any reference condition is input; training the face generation model to obtain a second face generation model, taking the reference condition as input and the generated face sample as expected output; the second face generation model outputs a face image consistent with the generated face sample when any reference condition is input; and constructing the real and fake face difference parameter in the face re-generation module based on the difference between the first face generation model and the second face generation model trained based on the same sample pair.

[0129] When the face generation model is trained to obtain the first face generation model, a sample pair can be selected from the data set, and the real face sample in the sample pair is taken as the expected output (label) of the face generation model. Meanwhile, one or more reference conditions are set as the input of the face generation model, which can be random noise, text description, or other feature vectors capable of guiding the face generation model to generate a face.

[0130] The training target of the training is to make the model reach a "overfitting" state, that is, the face image output by the model is consistent with the real face sample regardless of the input reference condition. After the training is completed, the first face generation model is obtained, which can output the same face image as the real face sample when any reference condition is input.

[0131] When the face generation model is trained to obtain the second face generation model, the generated face sample corresponding to the real face sample in the selected sample pair is taken as the expected output (label). Meanwhile, one or more reference conditions are set as the input of the face generation model.

[0132] The training target of the training is to make the model reach a "overfitting" state, that is, the face image output by the model is consistent with the generated face sample regardless of the input reference condition. After the training is completed, the second face generation model is obtained, which can output the same face image as the generated face sample when any reference condition is input.

[0133] Then, based on one sample pair, two models can be obtained: one outputs real face samples when any reference condition is input (the first face generation model), and the other outputs generated face samples when any reference condition is input (the second face generation model). By comparing the differences in the parameters of the two models, the real-fake face difference parameters can be obtained. These parameters can include the weights, biases, etc. of the models.

[0134] Using the constructed real-fake face difference parameters, a face re-generation module can be further constructed. This module can receive an input face image (whether a real face or a generated face) and perform fine adjustment and optimization based on the real-fake face difference parameters, thereby outputting a face image that is more realistic, natural, and consistent with the characteristics of real faces.

[0135] In the embodiments of the present disclosure, since the real face samples and the generated face samples in the generated sample pairs only differ in the face part, and there is no difference in other areas except the face, by comparing the differences in the parameters of the first face generation model and the second face generation model, the real-fake face difference parameters obtained only reflect the difference in the face part between the real face samples and the generated face samples, and do not reflect the difference in the background and clothing areas in the image. Therefore, the face re-generation module obtained based on the real-fake face difference parameters only adds real features to the face part, and does not modify other areas except the face, thereby better preserving the information of the original image.

[0136] In a possible implementation, the real-fake face difference parameters in the face re-generation module are constructed based on the differences between the first face generation model and the second face generation model trained based on the same sample pairs, and include: determining the parameter difference value of the first face generation model and the second face generation model trained based on the same sample pairs; and averaging the parameter difference values corresponding to multiple sample pairs to obtain the real-fake face difference parameters.

[0137] For each group of sample pairs, the parameter difference values of the first face generation model and the second face generation model are calculated respectively to obtain the parameter difference values of each group of sample pairs. Then, the parameter difference values corresponding to multiple sample pairs are averaged to obtain the real-fake face difference parameters.

[0138] In the embodiments of the present disclosure, by averaging multiple groups of parameter difference values, a unique real-fake face difference parameter can be obtained. This parameter can reflect the difference between real faces and generated faces in key features such as facial features and skin.

[0139] In a possible implementation, the face generation apparatus can be implemented by electronic devices such as terminal devices and servers, and the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. The method can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0140] According to an aspect of the present disclosure, a face generation method is provided, Figure 5 A flowchart of a face generation method according to an embodiment of the present disclosure is shown in Figure 5 As shown, the method comprises:

[0141] In step S31, a face replacement operation is performed on the face in the base image based on the face in the reference image, to obtain an initial face replacement image containing a generated face.

[0142] In step S32, the generated face in the initial face replacement image is re-generated based on a real and fake face difference parameter, to obtain a target face replacement image. The real and fake face difference parameter is used to represent the difference between a real face and a generated face.

[0143] In a possible implementation, the method further comprises:

[0144] Generating the base image according to a prompt condition.

[0145] Performing fine drawing and enlargement on the base image to obtain a fine-drawing and enlarged base image.

[0146] In a possible implementation, the face replacement operation performed on the face in the base image based on the face in the reference image to obtain an initial face replacement image containing a generated face comprises:

[0147] Mapping the face features in the reference image to the face region in the base image to obtain an intermediate feature tensor fused with the reference face features;

[0148] Generating face pose control data based on the face key points in the base image, the face pose control data being used to keep the pose of the generated face consistent with the pose of the face in the base image;

[0149] Generating character pose control data based on the edge data in the base image, the character pose control data being used to keep the pose of the generated face consistent with the overall pose of the character;

[0150] The base image is redrawn using the face posture control data, the character posture control data and the intermediate feature tensor as control signals to obtain an initial face-changing image containing the generated face.

[0151] In one possible implementation, redrawing the base image using the facial posture control data, the character posture control data, and the intermediate feature tensor as control signals to obtain an initial face-swapped image containing a generated face includes:

[0152] The facial posture control data, the character posture control data and the intermediate feature tensor are input into the unconditional generation model as control signals, the base image is redrawn to generate an initial face-changing image, and the conditional generation model is disabled.

[0153] In one possible implementation, regenerating the generated face in the initial face-swapped image based on the real-face-and-fake-face difference parameter to obtain the target face-swapped image includes:

[0154] a cropping module, configured to crop the generated face in the initial face-swapped image to obtain a generated face image;

[0155] A real-false face transformation module, configured to transform the generated face image based on the real-false face difference parameters to obtain a real-like face image;

[0156] The splicing module is used to splice the real-looking face image back into the initial face-swapped image to obtain a target face-swapped image.

[0157] In a possible implementation, the true and false face difference parameter is obtained by training a face generation model based on true and false face samples;

[0158] The face sample pair includes a real face sample and a generated face sample, and the generated face sample is obtained by performing a face generation operation using the real face sample as a reference face.

[0159] In one possible implementation, the true and false face difference parameter is constructed based on the difference between a first face generation model and a second face generation model trained on the same sample pair;

[0160] The first face generation model is obtained by training the face generation model using the reference condition as input and the real face sample as the expected output; the face output by the first face generation model is consistent with the real face sample when any reference condition is input;

[0161] The second face generation model is obtained by training the face generation model with the generated face sample as expected output. The face output by the second face generation model under any reference condition is consistent with the generated face sample.

[0162] In a possible implementation, the real-fake face difference parameter is obtained by averaging parameter difference values of a plurality of sample pairs, and the parameter difference value is a difference between parameters of the first face generation model and the second face generation model trained based on the same sample pair.

[0163] In addition, the present disclosure also provides a face re-generation module construction device, an electronic device, a computer readable storage medium, and a program, which can be used to implement any face re-generation module construction method provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and are not repeated here.

[0164] According to an aspect of the present disclosure, a face re-generation module construction device is provided for generating a face re-generation module in the device, Figure 6 A block diagram of a face re-generation module construction device according to an embodiment of the present disclosure is shown in FIG. 1. Figure 6 As shown in FIG. 1, the device includes:

[0165] A data set construction module 41 is configured to construct a data set composed of real-fake face sample pairs, wherein the real-fake face sample pairs include real face samples and generated face samples.

[0166] A difference parameter construction module 42 is configured to construct a real-fake face difference parameter in the face re-generation module according to a difference between the real face samples and the generated face samples, and obtain the face re-generation module constructed based on the real-fake face difference parameter.

[0167] In a possible implementation, the data set construction module is configured to:

[0168] perform a face generation operation with a face in the real face sample as a reference face to obtain a generated face;

[0169] replace the real face in the real face sample with the generated face to obtain a generated face sample.

[0170] In a possible implementation, the difference parameter construction module is configured to:

[0171] train a face generation model with a reference condition as input and the real face sample as expected output to obtain a first face generation model, wherein the first face generation model outputs a face consistent with the real face sample under any reference condition;

[0172] training the face generation model to obtain a second face generation model, wherein the second face generation model outputs a face consistent with the generated face sample when any reference condition is input;

[0173] constructing a true-false face difference parameter in the face re-generation module based on a difference between the first face generation model and the second face generation model trained based on the same sample pair.

[0174] In a possible implementation, the difference parameter construction module is configured to:

[0175] determine a parameter difference value of the first face generation model and the second face generation model trained based on the same sample pair;

[0176] average the parameter difference values corresponding to the plurality of sample pairs to obtain the true-false face difference parameter.

[0177] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0178] The embodiments of the present disclosure also provide a computer-readable storage medium having computer program instructions stored therein, and the computer program instructions are executed by a processor to implement the above method. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0179] The embodiments of the present disclosure also provide an electronic device, including a processor, and a memory for storing processor-executable instructions, wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

[0180] The embodiments of the present disclosure also provide a computer program product, including computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code, when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above method.

[0181] The electronic device can be provided as a terminal, a server or other forms of devices.

[0182] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 can be provided as a server or a terminal device. Referring to Figure 7The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by the memory 1932, for storing instructions executable by the processing component 1922, such as an application program. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0183] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Microsoft Windows Server operating system (Windows Server TM ), Apple's graphical user interface-based operating system (Mac OSX TM ), multi-user multi-process computer operating system (Unix TM ), free and open source Unix-like operating system (Linux TM ), open source Unix-like operating system (FreeBSD TM ) or the like.

[0184] In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932 including computer program instructions executable by the processing component 1922 of the electronic device 1900 to perform the above method is also provided.

[0185] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0186] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0187] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0188] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0189] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0190] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0191] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0192] The flow diagrams and the block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0193] The computer program product can be embodied in a tangible medium of

[0194] The above description of the various embodiments is intended to be illustrative in all aspects, rather than being restrictive. Those skilled in the art can refer to the description of the various embodiments to make modifications and / or improvements.

[0195] Those skilled in the art can understand that, in the above-described method of the specific embodiments, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible inherent logic.

[0196] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0197] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A face generation device, characterized in that: include: A face-swapping module is used to perform a face-swapping operation on the face in the base image based on the face in the reference image, and obtain an initial face-swapping image containing the generated face; The face regeneration module is used to regenerate the generated face in the initial face-swapped image based on the true and false face difference parameters to obtain the target face-swapped image. The true and false face difference parameters are used to characterize the difference between the real face and the generated face.

2. The device according to claim 1, characterized in that The device further comprises: The basic image generation module is used to generate the base map according to the prompt conditions; The precision drawing and enlarging module is used to precisely draw and enlarge the base map to obtain the precisely drawn and enlarged base map.

3. The device according to claim 1, characterized in that The face-changing module includes: A reference feature mapping module is used to map the facial features in the reference image to the face area in the base image to obtain an intermediate feature tensor that integrates the reference facial features; A face posture maintaining module, configured to generate face posture control data based on key facial points in the base image, wherein the generated face posture control data is used to maintain the posture of the generated face consistent with the facial posture in the base image; A character posture maintaining module, configured to generate character posture control data based on edge data in the base image, wherein the character posture control data is used to maintain the overall consistency between the posture of the generated face and the posture of the character; The basic image generation module is used to redraw the base image using the facial posture control data, the character posture control data and the intermediate feature tensor as control signals to obtain an initial face-changing image containing the generated face.

4. The device according to claim 3, characterized in that The basic image generation module includes a conditional generation model and an unconditional generation model. When generating an initial face-changing image, the basic image generation module inputs the facial posture control data, the character posture control data and the intermediate feature tensor as control signals into the unconditional generation model, redraws the base image, generates an initial face-changing image, and disables the conditional generation model.

5. The device according to claim 1, characterized in that The face regeneration module includes: a cropping module, configured to crop the generated face in the initial face-swapped image to obtain a generated face image; A real-false face transformation module, configured to transform the generated face image based on the real-false face difference parameters to obtain a real-like face image; The splicing module is used to splice the real-looking face image back into the initial face-swapped image to obtain a target face-swapped image.

6. The device according to claim 1, characterized in that The true and false face difference parameters are obtained by training the face generation model based on true and false face samples; The face sample pair includes a real face sample and a generated face sample, and the generated face sample is obtained by performing a face generation operation using the real face sample as a reference face.

7. The device according to claim 6, characterized in that The true and false face difference parameter is constructed based on the difference between the first face generation model and the second face generation model obtained by training the same sample pair; The first face generation model is obtained by training the face generation model using the reference condition as input and the real face sample as the expected output; The first face generation model outputs a face that is consistent with the real face sample when any reference conditions are input; The second face generation model is obtained by training the face generation model with the generated face sample as the expected output; when any reference conditions are input, the output face of the second face generation model is consistent with the generated face sample.

8. The device according to claim 7, characterized in that The true and false face difference parameters are obtained by averaging the parameter differences corresponding to multiple sample pairs, and the parameter difference is the difference between the parameters of the first face generation model and the second face generation model obtained by training based on the same sample pair.

9. A method for constructing a face regeneration module, characterized in that: A face regeneration module for generating the apparatus according to any one of claims 1 to 8, comprising: Constructing a dataset consisting of pairs of real and fake face samples, wherein the pairs of real and fake face samples include real face samples and generated face samples; According to the difference between the real face sample and the generated face sample, the true and false face difference parameters in the face regeneration module are constructed, and the face regeneration module constructed based on the true and false face difference parameters is obtained.

10. The method according to claim 9, characterized in that The construction of a data set consisting of real and fake face sample pairs includes: Performing a face generation operation using a face in the real face sample as a reference face to obtain a generated face; The generated face replaces the real face in the real face sample to obtain a generated face sample.

11. The method according to claim 9, characterized in that The step of constructing a true-false face difference parameter in a face regeneration module based on the difference between the real face sample and the generated face sample includes: The face generation model is trained using the reference condition as input and the real face sample as the expected output to obtain a first face generation model; the first face generation model outputs a face that is consistent with the real face sample when any reference condition is input; The face generation model is trained using the reference condition as input and the generated face sample as the desired output to obtain a second face generation model; the second face generation model outputs a face that is consistent with the generated face sample when any reference condition is input; Based on the difference between the first face generation model and the second face generation model obtained by training the same sample pair, a true and false face difference parameter in the face regeneration module is constructed.

12. The method according to claim 11, characterized in that The difference between the first face generation model and the second face generation model obtained by training based on the same sample pair is used to construct the true and false face difference parameters in the face regeneration module, including: Determining a parameter difference between a first face generation model and a second face generation model trained based on the same sample pair; The parameter differences corresponding to multiple sample pairs are averaged to obtain the difference parameters between real and fake faces.

13. A face generation method, characterized in that: include: Based on the face in the reference image, perform a face swap operation on the face in the base image to obtain an initial face swap image containing the generated face; The real and fake face difference parameters are used to regenerate the generated face in the initial face-swapped image to obtain the target face-swapped image. The real and fake face difference parameters are used to characterize the difference between the real face and the generated face.

14. A device for constructing a face regeneration module, characterized in that: A face regeneration module for generating the apparatus according to any one of claims 1 to 8, comprising: A data set construction module is used to construct a data set consisting of real and fake face sample pairs, wherein the real and fake face sample pairs include real face samples and generated face samples; The difference parameter construction module is used to construct the true and false face difference parameters in the face reconstruction module according to the difference between the real face sample and the generated face sample, and obtain the face reconstruction module constructed based on the true and false face difference parameters.

15. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to implement the method according to any one of claims 9 to 13.

16. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 9 to 13 is implemented.

Images