Face image synthesis method and device, electronic equipment and storage medium

Through the combination of face image coding and difference recognition model, the problem of low naturalness in face image synthesis is solved, and a higher quality image synthesis effect is achieved.

CN118982600APending Publication Date: 2024-11-19PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411183429.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Existing face image synthesis technology is difficult to accurately capture and reproduce subtle differences between face images, resulting in a low naturalness of the face image obtained by the synthesis.

Method used

By acquiring the original face image and the target face image, the preset face image encoder and style encoder are used for encoding, and the difference calculation and feature stitching are performed by combining the pre-trained target face difference recognition model to generate the target face features data, and then perform facial image synthesis.

Benefits of technology

The naturalness and quality of face image synthesis are improved, and the face characteristics of the target face image and the identity characteristics and style elements of the original face image are preserved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982600A_ABST
    Figure CN118982600A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a face image synthesis method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence and finance. The method comprises the following steps: encoding an original face image through a face image encoder to obtain original face encoding data, and encoding the original face image through a style image encoder to obtain original face style data; encoding the target face image through a face image encoder to obtain target face encoding data; performing face difference calculation on the original face coding data, the original face style data and the target face coding data to obtain target image difference feature data; performing feature splicing on the target image difference feature data and the original face style data to obtain target face feature data; and performing face image synthesis based on the target face feature data. According to the embodiment of the invention, the synthesized image can be accurately adjusted by using the difference feature data of the image, and the naturalness and quality of face image synthesis are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and financial technology, and in particular to a method and device for synthesizing facial images, an electronic device, and a storage medium. Background Art

[0002] In the financial field, facial image synthesis technology has been widely used in the production of advertisements, promotional videos and instructional videos. Facial image synthesis technology can personalize the expression and even the overall face of the original face according to the characteristics of the target face to synthesize a new facial image.

[0003] However, current facial image synthesis technology has difficulty in accurately capturing and reproducing the subtle differences between facial images, resulting in the synthesized facial images being less natural.

[0004] Therefore, how to improve the naturalness of face image synthesis has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to propose a facial image synthesis method and device, an electronic device and a storage medium, aiming to improve the naturalness of facial image synthesis.

[0006] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a method for synthesizing a face image, the method comprising:

[0007] Acquire an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data;

[0008] Acquire a target face image, and perform face encoding on the target face image through the face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image;

[0009] Performing face difference calculation on the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data;

[0010] Perform feature splicing on the target image difference feature data and the original face style data to obtain target face feature data;

[0011] A face image is synthesized based on the target face feature data to obtain a target face synthesized image.

[0012] In some embodiments, the method further includes training the target face difference recognition model, specifically including:

[0013] Acquire face sample data, the face sample data comprising a first face sample image and a second face sample image; wherein the first face sample image and the second face sample image are different;

[0014] Performing face encoding on the first face sample image by the face image encoder to obtain first face encoding data, and performing style encoding on the first face sample image by the style image encoder to obtain first face style data;

[0015] Performing face encoding on the second face sample image by the face image encoder to obtain second face encoding data, and performing style encoding on the second face sample image by the style image encoder to obtain second face style data;

[0016] Performing face difference calculation on the first face encoding data, the second face encoding data and the first face style data through a preset original face difference recognition model to obtain sample image difference feature data;

[0017] The original face difference recognition model is adjusted based on the first face style data, the second face style data and the sample image difference feature data to obtain the target face difference recognition model.

[0018] In some embodiments, the adjusting the original face difference recognition model based on the first face style data, the second face style data and the sample image difference feature data to obtain the target face difference recognition model includes:

[0019] Calculating feature differences between the first face style data and the second face style data to obtain face style feature difference data;

[0020] Perform loss calculation based on the sample image difference feature data and the face style feature difference data to obtain feature difference loss data;

[0021] The original face difference recognition model is optimized based on the feature difference loss data to obtain the target face difference recognition model.

[0022] In some embodiments, the performing face difference calculation on the original face encoding data, the original face style data, and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data includes:

[0023] Calculating feature differences between the original face coding data and the target face coding data to obtain face coding feature difference data;

[0024] Perform feature splicing on the target face coding data, the face coding feature difference data and the original face style data to obtain target splicing feature data;

[0025] The target face difference recognition model is used to perform difference feature calculation on the target splicing feature data to obtain the target image difference feature data.

[0026] In some embodiments, synthesizing a face image based on the target face feature data to obtain a target face synthesized image includes:

[0027] Performing noise processing on the target facial feature data to obtain facial noise feature data;

[0028] Performing feature mapping on the facial noise feature data to obtain facial feature map data;

[0029] Sampling the face feature map data to obtain face sampling feature data;

[0030] Image generation is performed on the face sampling feature data to obtain the target face composite image.

[0031] In some embodiments, obtaining the original face image includes:

[0032] Get the original video;

[0033] Extracting video frames from the original video to obtain original video frame images;

[0034] Perform face recognition on the original video frame image to obtain the original face image.

[0035] In some embodiments, after synthesizing a face image based on the target face feature data to obtain a target face synthesized image, the method further includes:

[0036] Filtering a face replacement image from the original video frame image based on the target face composite image and the original face image;

[0037] Replacing the face replacement image with the target face composite image to obtain a target video frame image;

[0038] Obtaining the video frame rate of the original video;

[0039] Video generation is performed based on the video frame rate and the target video frame image.

[0040] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a face image synthesis device, the device comprising:

[0041] The original face image processing module is used to obtain an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data;

[0042] A target face image processing module, used for acquiring a target face image, and performing face encoding on the target face image through the face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image;

[0043] A face image difference calculation module is used to calculate the face difference of the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data;

[0044] A face feature splicing module, used for performing feature splicing on the target image difference feature data and the original face style data to obtain target face feature data;

[0045] The image synthesis module is used to synthesize a face image based on the target face feature data to obtain a target face synthesized image.

[0046] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0047] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0048] The face image synthesis method and device, electronic device and storage medium proposed in the present application obtain an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, perform style encoding on the original face image through a preset style image encoder to obtain original face style data; obtain a target face image, perform face encoding on the target face image through a face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image; perform face difference calculation on the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data; perform feature splicing on the target image difference feature data and the original face style data to obtain target face feature data; perform face image synthesis based on the target face feature data to obtain a target face synthetic image. The method can make full use of the learned difference feature data between images to accurately adjust the synthesized image, retain the face features of the target face image and the identity features and style elements of the original face image, and improve the naturalness and quality of face image synthesis. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a facial image synthesis method provided in an embodiment of the present application;

[0050] Figure 2 This is a flowchart of obtaining an original face image provided by an embodiment of the present application;

[0051] Figure 3 is a flow chart of a training target face difference recognition model provided in an embodiment of the present application;

[0052] Figure 4 yes Figure 3 Flow chart of step S305 in FIG.

[0053] Figure 5 yes Figure 1 Flow chart of step S103 in FIG.

[0054] Figure 6 yes Figure 1 Flow chart of step S105 in FIG.

[0055] Figure 7 is a flowchart of a facial image synthesis method provided by another embodiment of the present application;

[0056] Figure 8 is a structural schematic diagram of a face image synthesis device provided in an embodiment of the present application;

[0057] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] First, some nouns involved in this application are analyzed:

[0062] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] Face image migration refers to the process of migrating certain facial features (such as expression, skin color, posture, etc.) from a target face image to the face of another original face image, with the goal of making the original face image visually show the facial features of the target face image. It is widely used in image processing, computer vision, artificial intelligence, virtual reality (VR) and augmented reality (AR) and other fields. After completing the face image migration, these features need to be fused or recombined to eventually generate a face image that contains both the facial features of the target face and the specific features of the original face.

[0064] CLIP (Contrastive Language-Image Pre-training) is a multimodal pre-training model based on contrastive learning. It aims to enable the model to understand and establish the matching relationship between images and texts through contrastive learning of large-scale image-text pairs. The core idea of ​​the CLIP model is to use the correspondence between images and texts to optimize model parameters through contrastive learning, thereby improving the performance of the model on a variety of visual tasks.

[0065] The CLIP model implements its functionality through two main components: an image encoder and a text encoder.

[0066] In the CLIP model, the image encoder is the key component used to convert the input image data into a feature vector. The image encoder can be a model based on a convolutional neural network (such as ResNet) or a Transformer-based model (such as ViT). These models convert the pixel information in the image into a high-level feature representation through a series of convolution operations or self-attention mechanisms. The role of the image encoder is to convert the raw image data into a form that can be understood and processed by the model so that it can be matched with the feature vector generated by the text encoder in the subsequent contrastive learning process.

[0067] StyleGAN, StyleGAN is a generative adversarial network (GAN) architecture used to generate high-quality, high-resolution facial images. By introducing the concept of style, it achieves precise control of different levels of features (such as global features, mid-level features, and detail features) in the image generation process. StyleGAN adopts a hierarchical generator structure, combined with adaptive instance normalization (AdaIN) technology, which can inject different style information at different resolution stages to generate realistic and detailed facial images. In addition, StyleGAN also introduces techniques such as style mixing to further enhance the flexibility and diversity of image generation. This architecture has not only achieved remarkable results in the field of facial image generation, but also provided new ideas and methods for other image generation and editing tasks.

[0068] In the financial field, facial image synthesis technology has been widely used in the production of advertisements, promotional videos and instructional videos. Facial image synthesis technology can personalize the expression and even the overall face of the original face according to the characteristics of the target face to synthesize a new facial image.

[0069] However, current facial image synthesis technology has difficulty in accurately capturing and reproducing the subtle differences between facial images, resulting in the synthesized facial images being less natural.

[0070] Based on this, the embodiments of the present application provide a facial image synthesis method and device, an electronic device and a storage medium, aiming to improve the naturalness of facial image synthesis.

[0071] The face image synthesis method and device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the face image synthesis method in the embodiments of the present application is described.

[0072] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0073] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0074] The facial image synthesis method provided in the embodiment of the present application relates to the field of artificial intelligence and financial technology. The facial image synthesis method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the facial image synthesis method, etc., but is not limited to the above forms.

[0075] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0076] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0077] Figure 1 is an optional flowchart of the face image synthesis method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0078] Step S101, obtaining an original face image, and performing face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and performing style encoding on the original face image through a preset style image encoder to obtain original face style data;

[0079] Step S102, obtaining a target face image, and performing face encoding on the target face image through a face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image;

[0080] Step S103, performing face difference calculation on the original face coding data, the original face style data and the target face coding data by using a pre-trained target face difference recognition model to obtain target image difference feature data;

[0081] Step S104, performing feature splicing on the target image difference feature data and the original face style data to obtain target face feature data;

[0082] Step S105, performing facial image synthesis based on the target facial feature data to obtain a target facial synthetic image.

[0083] Steps S101 to S105 shown in the embodiment of the present application are as follows: by acquiring an original face image and a target face image, face encoding the original face image through a preset face image encoder to obtain original face encoding data, style encoding the original face image through a preset style image encoder to obtain original face style data, and face encoding the target face image through a face image encoder to obtain target face encoding data; then face difference calculation is performed on the original face encoding data, the original face style data, and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data, feature splicing is performed on the target image difference feature data and the original face style data to obtain target face feature data, and face image synthesis is performed based on the target face feature data to obtain a target face synthetic image. The learned difference feature data between images can be fully utilized to accurately adjust the synthesized image, retain the face features of the target face image and the identity features and style elements of the original face image, and improve the naturalness and quality of face image synthesis.

[0084] It should be noted that the original face image is the image that needs to be migrated, and the target face image is the image used for face migration, that is, the facial features in the target face image are migrated to the original face image.

[0085] Therefore, it can be understood that the original face image and the target face image are not the same.

[0086] It should be noted that the preset face image encoder and the preset style image encoder are both pre-trained encoders; among them, the face image encoder can be a CLIP image encoder, a ResNet image encoder, a VisionTransformer (ViT) image encoder, etc.; the style image encoder can be a StyleGAN encoder, a BigGAN encoder, a ProGAN encoder, etc.; which specific model to use needs to be selected based on the actual application scenario, but is not limited to this.

[0087] In one embodiment, the face image encoder is a CLIP image encoder, and the style image encoder is a StyleGAN image encoder; wherein the CLIP image encoder can accurately obtain facial features in a face image, and the StyleGAN encoder can extract facial features with facial style.

[0088] See also Figure 2 In some embodiments, the "obtaining the original face image" in step S101 may include but is not limited to steps S201 to S203:

[0089] Step S201, obtaining the original video;

[0090] Step S202, extracting video frames from the original video to obtain original video frame images;

[0091] Step S203, performing face recognition on the original video frame image to obtain an original face image.

[0092] Steps S201 to S203 shown in the embodiment of the present application extract video frames from the original video to obtain original video frame images, and perform face recognition on the original video frame images to find video frame images with faces, thereby facilitating face migration. This can meet the needs of entertainment, creative production, etc., and can be extended to the production of advertisements, promotional videos, and instructional videos in the financial field.

[0093] In step S202 of some embodiments, the video frames of the original video may be extracted by methods provided in the OpenCV library, video editing software, FFmpeg command line tools, etc., but are not limited thereto.

[0094] It should be noted that the essence of the original face image is the video frame image containing the face in the original video frame image. For example: 500 original video frame images are extracted from the original video, of which only 250 video frame images contain faces. Therefore, the number of original face images is 250.

[0095] In some embodiments, the facial image synthesis method also includes pre-training a target facial difference recognition model, which is used to learn and extract difference features between the original facial image and the target facial image, and the difference features can assist in synthesizing more realistic and natural facial images.

[0096] Specifically, see Figure 3 The training target face difference recognition model may include but is not limited to steps S301 to S305:

[0097] Step S301, obtaining face sample data, the face sample data including a first face sample image and a second face sample image; wherein the first face sample image and the second face sample image are different;

[0098] Step S302, performing face encoding on the first face sample image by using a face image encoder to obtain first face encoding data, and performing style encoding on the first face sample image by using a style image encoder to obtain first face style data;

[0099] Step S303, performing face encoding on the second face sample image by means of a face image encoder to obtain second face encoding data, and performing style encoding on the second face sample image by means of a style image encoder to obtain second face style data;

[0100] Step S304, performing face difference calculation on the first face encoding data, the second face encoding data and the first face style data by using a preset original face difference recognition model to obtain sample image difference feature data;

[0101] Step S305 , adjusting the original face difference recognition model based on the first face style data, the second face style data and the sample image difference feature data to obtain a target face difference recognition model.

[0102] In steps S301 to S305 shown in the embodiment of the present application, face sample data is obtained, and the first face sample image is processed respectively to obtain the first face coding data and the first face style data, and the second face sample image is processed to obtain the second face coding data and the second face style data. Based on the first face coding data, the first face style data, the second face coding data and the second face style data, the original face difference recognition model is trained to obtain the target face difference recognition model, so that the target face difference recognition model can more accurately capture the subtle differences between faces, and enhance the recognition and learning capabilities of the target face difference recognition model for face change features, thereby achieving a more natural, accurate and personalized effect in face migration and synthesis.

[0103] It should be noted that, in the process of training the target face difference recognition model, the face image encoder and style image encoder used in the previous encoding process are consistent with the face image encoder and style image encoder used in the face image synthesis process, and are all pre-trained.

[0104] In some embodiments, step S304 may include but is not limited to the following steps:

[0105] Calculate the feature difference between the first face coding data and the second face coding data to obtain sample coding feature difference data;

[0106] Perform feature splicing on the second face coding data, the sample coding feature difference data and the first face style data to obtain sample splicing feature data;

[0107] The difference feature calculation is performed on the sample splicing feature data through the original face difference recognition model to obtain the sample image difference feature data.

[0108] Among them, feature difference calculation can be achieved by calculating Hamming distance, Euclidean distance, Manhattan distance, cosine similarity and other methods. The specific selection needs to be made according to the actual application scenario, but is not limited to this.

[0109] In some embodiments, feature splicing is to connect multiple data. For example, assuming that the feature dimensions of the second face encoding data, the sample encoding feature difference data and the first face style data are all 2*2*3, the dimension of the sample splicing feature data obtained after splicing is 2*2*9.

[0110] In addition, the original face difference recognition model includes the original convolution layer and the original U-Net layer;

[0111] Specifically, the sample splicing feature data is convolved through the original convolution layer in the original face difference recognition model to obtain sample convolution feature data; the sample convolution feature data is processed through the original U-Net layer in the original face difference recognition model to obtain sample image difference feature data, thereby completing the difference feature calculation process.

[0112] See also Figure 4 In some embodiments, step S305 may include but is not limited to steps S401 to S403:

[0113] Step S401, calculating a feature difference between the first face style data and the second face style data to obtain face style feature difference data;

[0114] Step S402, performing loss calculation on the original face difference recognition model based on the feature difference loss data to obtain feature difference loss data;

[0115] Step S403, optimizing the original face difference recognition model based on the feature difference loss data to obtain a target face difference recognition model.

[0116] In the steps S401 to S403 shown in the embodiment of the present application, the facial style feature difference data is obtained by performing feature difference calculation on the first facial style data and the second facial style data, and loss calculation is performed based on the facial style feature difference data and the sample image difference feature data, thereby guiding the optimization of the original facial difference recognition model and obtaining the target facial difference recognition model. The difference between the first facial style data and the second facial style data can be comprehensively considered, and through refined loss calculation, the original facial difference recognition model can accurately learn the difference between the first facial sample image and the second facial sample image, and transfer this ability to the application scenario, thereby enhancing the generalization ability and robustness of the model, so that the target facial difference recognition model finally obtained can show higher accuracy and stability when processing complex and changeable facial images, providing more reliable technical support for subsequent facial migration, recognition and other applications.

[0117] In step S401 of some embodiments, feature difference calculation can be achieved by calculating Hamming distance, Euclidean distance, Manhattan distance, cosine similarity, etc. The specific selection needs to be made according to the actual application scenario, but is not limited to this.

[0118] In step S402 of some embodiments, a loss function can be calculated for the original face difference recognition model based on the feature difference loss data to obtain the feature difference loss data, wherein the loss function can be a mean square error loss function, a cross entropy loss function, a likelihood loss function, an adversarial loss function, etc., but is not limited to this.

[0119] In step S403 of some embodiments, back propagation may be performed based on the feature difference loss data to adjust the model parameters of the original face difference recognition model.

[0120] In one embodiment, the original face difference recognition model includes an original convolution layer and an original U-Net layer. The parameters of the original convolution layer and the original U-Net layer are adjusted based on the feature difference loss data, and the original face difference recognition model is iteratively trained until the feature difference loss data meets the preset training criteria.

[0121] See also Figure 5 In some embodiments, step S103 may also include but is not limited to steps S501 to S503:

[0122] Step S501, calculating feature difference between original face coding data and target face coding data to obtain face coding feature difference data;

[0123] Step S502, performing feature splicing on the target face coding data, the face coding feature difference data and the original face style data to obtain target splicing feature data;

[0124] Step S503, performing difference feature calculation on the target splicing feature data through the target face difference recognition model to obtain target image difference feature data.

[0125] In steps S501 to S503 shown in the embodiment of the present application, feature differences are calculated between the original face coding data and the target face coding data to obtain face coding feature difference data, and feature splicing is performed on the face coding feature difference data, the target face coding data and the original face style data. Subsequently, difference features are calculated on the spliced ​​target spliced ​​feature data through a target face difference recognition model, which can accurately capture the subtle differences between the original face image and the target face image, thereby achieving a more natural, accurate and personalized effect in face migration and synthesis, and improving the naturalness and quality of face image synthesis.

[0126] In step S501 of some embodiments, feature difference calculation can be achieved by calculating Hamming distance, Euclidean distance, Manhattan distance, cosine similarity, etc. The specific selection needs to be made according to the actual application scenario, but is not limited to this.

[0127] In step S502 of some embodiments, feature concatenation is to connect multiple data, for example:

[0128] Assuming that the feature dimensions of the target face encoding data, face encoding feature difference data and original face style data are all 3*3*3, the dimension of the target spliced ​​feature data obtained after splicing is 3*3*9.

[0129] In step S503 of some embodiments, the target face difference recognition model includes a target convolution layer and a target U-Net layer;

[0130] Specifically, the target splicing feature data is convolved through the target convolution layer in the target face difference recognition model to obtain face convolution feature data; the face convolution feature data is processed through the target U-Net layer in the target face difference recognition model to obtain target image difference feature data, thereby completing the difference feature calculation process.

[0131] For example: the convolution kernel of the target convolution layer is 3*3, and the step size is 1, then the dimension of the processed face convolution feature data is 3*3*32;

[0132] After processing through the target U-Net layer, the target image difference feature data with a dimension of 3*3*3 is obtained.

[0133] In step S104 of some embodiments, feature stitching is to connect multiple data. For example, assuming that the dimensions of the target image difference feature data and the original facial style data are both 3*3*3, the dimension of the target facial feature data obtained after stitching is 3*3*6.

[0134] By performing feature splicing of the target image difference feature data and the original face style data, the learned differences can be transferred to the original face, thereby synthesizing a more natural and accurate face image.

[0135] In step S105 of some embodiments, a pre-trained StyleGAN model is used to perform facial image synthesis on the target facial feature data to obtain a target facial synthesized image.

[0136] Among them, the StyleGAN model includes a noise addition layer, a generator, and an image output layer;

[0137] See also Figure 6 In some embodiments, step S105 includes but is not limited to steps S601 to S604:

[0138] Step S601, performing noise processing on the target face feature data to obtain face noise feature data;

[0139] Step S602, performing feature mapping on the facial noise feature data to obtain facial feature map data;

[0140] Step S603, sampling the face feature map data to obtain face sampling feature data;

[0141] Step S604, performing image generation on the face sampling feature data to obtain a target face composite image.

[0142] In steps S601 to S604 shown in the embodiment of the present application, a series of operations such as processing, feature mapping, and sampling processing are performed on the target facial feature data, so that the finally generated target face composite image not only transfers the facial features of the target facial image, but also retains the identity features and style elements of the original facial image, and can present more realistic and delicate textures and details, thereby improving the naturalness and quality of facial image synthesis.

[0143] In step S601 of some embodiments, the target facial feature data is subjected to noise addition processing through a noise adding layer, thereby adjusting details in the facial image such as hair, wrinkles, freckles, etc., to prevent blurring of these details and improve the naturalness of the synthesis.

[0144] The noise may be random Gaussian noise, uniform noise, etc. Specifically, the Adaptive Instance Normalization (AdaIN) technology may be used to implement the addition of noise, but is not limited thereto.

[0145] In some embodiments, a generator performs feature mapping on the facial noise feature data to obtain facial feature map data, and performs sampling processing on the facial feature map data to obtain facial sampling feature data;

[0146] The sampling process may include up-sampling process and down-sampling process, which needs to be selected in combination with the actual application scenario, but is not limited thereto.

[0147] In some embodiments, after the sampling process, the face sampling feature data is image generated through the image output layer to obtain a target face composite image.

[0148] See also Figure 7 In some embodiments, after step S105, the face image synthesis may also include but is not limited to steps S701 to S704:

[0149] Step S701, selecting a face replacement image from the original video frame image based on the target face composite image and the original face image;

[0150] Step S702, replacing the face replacement image with the target face composite image to obtain a target video frame image;

[0151] Step S703, obtaining the video frame rate of the original video;

[0152] Step S704: generate video based on the video frame rate and the target video frame image.

[0153] In steps S701 to S704 shown in the embodiment of the present application, after completing the migration of the face image, the face replacement image to be replaced is screened out from the original video frame image based on the target face composite image and the original face image, and the face replacement image is replaced with the target face composite image to obtain the target video frame image; then, video generation is performed according to the video frame rate of the original video and the target video frame image, thereby realizing the migration and synthesis of the faces in the video, and while improving the naturalness and quality of the face image synthesis, the naturalness and quality of the migration of the video face image can be improved, thereby maximizing the overall quality of the video and making the migrated faces look more realistic and natural.

[0154] It can be understood that the target face composite image corresponds to the original face image, that is, after the original face image is subjected to face migration, the target face composite image is obtained; and the face replacement image corresponds to the original face image.

[0155] Therefore, replacing the face replacement image with the target face synthesis image can be understood as: replacing the original face image in the original video frame image with the target face synthesis image to obtain the target video frame image;

[0156] For example: assuming that the original video frame images include video frame image 1, video frame image 2, video frame image 3, video frame image 4 and video frame image 5;

[0157] Wherein, video frame image 2, video frame image 3 and video frame image 5 contain human faces, then the original human face image contains video frame image 2, video frame image 3 and video frame image 5;

[0158] After completing the face migration, a target face composite image is obtained, wherein the target face composite image includes video frame image 21 (corresponding to video frame image 2), video frame image 31 (corresponding to video frame image 3) and video frame image 51 (corresponding to video frame image 5);

[0159] Correspondingly, the screened face replacement images include video frame image 2, video frame image 3 and video frame image 5;

[0160] After the replacement operation is performed, the target video frame images obtained include: video frame image 1, video frame image 21, video frame image 31, video frame image 4 and video frame image 51.

[0161] It should be noted that when performing video face migration, the synthesis of the migrated video frame images is performed based on the parameters of the original video (such as video frames), which can retain key information such as the frame rate, motion trajectory, and lighting changes of the original video, so that the migrated faces can be naturally integrated into the original video scene, avoiding obvious abruptness or artifacts.

[0162] The facial image synthesis method provided in the embodiment of the present application can be applied to advertising posters, promotional videos or instructional videos.

[0163] In one embodiment, the face image synthesis method is applied to scenes such as promotional videos or instructional videos, and may include but is not limited to the following steps:

[0164] Step 1.1: Get the original video;

[0165] Extract video frames from the original video to obtain original video frame images;

[0166] Perform face recognition on the original video frame image to obtain the original face image.

[0167] Step 1.2: Perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data.

[0168] Step 1.3: Obtain a target face image, wherein the target face image is different from the original face image.

[0169] Step 1.4: Perform face encoding on the target face image through a face image encoder to obtain target face encoding data.

[0170] Step 1.5: Calculate the feature difference between the original face coding data and the target face coding data to obtain face coding feature difference data.

[0171] The target face encoding data, the face encoding feature difference data and the original face style data are feature spliced ​​to obtain the target spliced ​​feature data.

[0172] The target face difference recognition model trained in advance is used to calculate the difference features of the target splicing feature data to obtain the target image difference feature data.

[0173] Step 1.6: Perform feature concatenation on the target image difference feature data and the original face style data to obtain the target face feature data.

[0174] Step 1.7: The pre-trained StyleGAN model performs face image synthesis on the target face feature data to obtain a target face synthetic image, wherein the StyleGAN model includes a noise addition layer, a generator, and an image output layer; the specific face image synthesis includes the following steps:

[0175] The target face feature data is subjected to noise adding processing through the noise adding layer to obtain face noise feature data;

[0176] Performing feature mapping on the facial noise feature data through a generator to obtain facial feature map data, and performing sampling processing on the facial feature map data to obtain facial sampling feature data;

[0177] The image output layer is used to generate the image of the face sampling feature data to obtain the target face composite image.

[0178] Step 1.8: Filtering out a face replacement image from the original video frame image based on the target face composite image and the original face image;

[0179] Replace the face replacement image with the target face synthetic image to obtain the target video frame image;

[0180] Get the video frame rate of the original video;

[0181] Video generation is performed based on video frame rate and target video frame image.

[0182] The embodiments of the present application realize the migration and synthesis of faces in videos, and utilize the learned feature data of image differences to accurately adjust the synthesized image, thereby retaining the facial features of the target facial image and the identity features and style elements of the original facial image. While improving the naturalness and quality of facial image synthesis, the naturalness and quality of video facial image migration can be improved, thereby maximizing the overall quality of the video and making the migrated faces look more realistic and natural.

[0183] In one embodiment, the face image synthesis method is applied to scenes such as advertising posters, and may include but is not limited to the following steps:

[0184] Step 2.1: Get the original face image.

[0185] Step 2.2: Perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data.

[0186] Step 2.3: Obtain a target face image, wherein the target face image is different from the original face image.

[0187] Step 2.4: Perform face encoding on the target face image through a face image encoder to obtain target face encoding data.

[0188] Step 2.5: Calculate the feature difference between the original face coding data and the target face coding data to obtain face coding feature difference data;

[0189] Perform feature splicing on the target face coding data, the face coding feature difference data and the original face style data to obtain target splicing feature data;

[0190] The target face difference recognition model trained in advance is used to calculate the difference features of the target splicing feature data to obtain the target image difference feature data.

[0191] Step 2.6: Perform feature concatenation on the target image difference feature data and the original face style data to obtain the target face feature data.

[0192] Step 2.7: The pre-trained StyleGAN model performs face image synthesis on the target face feature data to obtain a target face synthetic image, wherein the StyleGAN model includes a noise addition layer, a generator, and an image output layer; the specific face image synthesis includes the following steps:

[0193] The target face feature data is subjected to noise adding processing through the noise adding layer to obtain face noise feature data;

[0194] Performing feature mapping on the facial noise feature data through a generator to obtain facial feature map data, and performing sampling processing on the facial feature map data to obtain facial sampling feature data;

[0195] The image output layer is used to generate the image of the face sampling feature data to obtain the target face composite image.

[0196] The embodiments of the present application realize the migration and synthesis of faces in videos, and utilize the learned feature data of the differences between images to accurately adjust the synthesized image, thereby retaining the facial features of the target facial image and the identity features and style elements of the original facial image, and improving the naturalness and quality of facial image synthesis.

[0197] In one embodiment, the training process of the target face difference recognition model includes but is not limited to the following steps:

[0198] Step 3.1: Acquire face sample data, the face sample data including a first face sample image and a second face sample image; wherein the first face sample image and the second face sample image are different.

[0199] Step 3.2: Perform face encoding on the first face sample image through a face image encoder to obtain first face encoding data, and perform style encoding on the first face sample image through a style image encoder to obtain first face style data.

[0200] Step 3.3: Perform face encoding on the second face sample image through a face image encoder to obtain second face encoding data, and perform style encoding on the second face sample image through a style image encoder to obtain second face style data.

[0201] Step 3.4: Calculate the feature difference between the first face coding data and the second face coding data to obtain sample coding feature difference data;

[0202] Perform feature splicing on the second face coding data, the sample coding feature difference data and the first face style data to obtain sample splicing feature data;

[0203] The difference feature calculation is performed on the sample splicing feature data through the original face difference recognition model to obtain the sample image difference feature data.

[0204] Step 3.5: Calculate the feature difference between the first face style data and the second face style data to obtain face style feature difference data;

[0205] Based on the feature difference loss data, the original face difference recognition model is subjected to loss calculation to obtain the feature difference loss data;

[0206] The original face difference recognition model is optimized based on the feature difference loss data to obtain the target face difference recognition model.

[0207] By training the original face difference recognition model, the target face difference recognition model is obtained, so that the target face difference recognition model can more accurately capture the subtle differences between faces, and enhance the target face difference recognition model's ability to recognize and learn face change features, thereby achieving a more natural, accurate and personalized effect in face migration and synthesis.

[0208] See also Figure 8 The embodiment of the present application also provides a face image synthesis device, which can implement the above face image synthesis method, and the device includes:

[0209] The original face image processing module 801 is used to obtain an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data;

[0210] The target face image processing module 802 is used to obtain a target face image and perform face encoding on the target face image through a face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image;

[0211] The face image difference calculation module 803 is used to calculate the face difference of the original face encoding data, the original face style data and the target face encoding data through the pre-trained target face difference recognition model to obtain the target image difference feature data;

[0212] The face feature splicing module 804 is used to perform feature splicing on the target image difference feature data and the original face style data to obtain the target face feature data;

[0213] The image synthesis module 805 is used to synthesize a face image based on the target face feature data to obtain a target face synthesized image.

[0214] The specific implementation of the face image synthesis device is basically the same as the specific implementation of the above-mentioned face image synthesis method, and will not be repeated here.

[0215] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned face image synthesis method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0216] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0217] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0218] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the face image synthesis method of the embodiment of this application;

[0219] Input / output interface 903, used to implement information input and output;

[0220] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0221] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0222] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0223] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned facial image synthesis method is implemented.

[0224] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0225] The face image synthesis method and device, electronic device and storage medium provided in the embodiment of the present application are as follows: obtaining an original face image, performing face encoding on the original face image through a preset face image encoder to obtain original face encoding data, performing style encoding on the original face image through a preset style image encoder to obtain original face style data; obtaining a target face image, and performing face encoding on the target face image through a face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image; performing face difference calculation on the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data; performing feature splicing on the target image difference feature data and the original face style data to obtain target face feature data; performing face image synthesis based on the target face feature data to obtain a target face synthetic image. The method can make full use of the learned difference feature data between images to accurately adjust the synthesized image, retain the face features of the target face image and the identity features and style elements of the original face image, and improve the naturalness and quality of face image synthesis.

[0226] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0227] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0228] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0230] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0236] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A method for synthesizing a face image, characterized in that: The method comprises: Acquire an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data; Acquire a target face image, and perform face encoding on the target face image through the face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image; Performing face difference calculation on the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data; Perform feature splicing on the target image difference feature data and the original face style data to obtain target face feature data; A face image is synthesized based on the target face feature data to obtain a target face synthesized image.

2. The method according to claim 1, characterized in that The method further includes training the target face difference recognition model, specifically including: Acquire face sample data, the face sample data comprising a first face sample image and a second face sample image; wherein the first face sample image and the second face sample image are different; Performing face encoding on the first face sample image by the face image encoder to obtain first face encoding data, and performing style encoding on the first face sample image by the style image encoder to obtain first face style data; Performing face encoding on the second face sample image by the face image encoder to obtain second face encoding data, and performing style encoding on the second face sample image by the style image encoder to obtain second face style data; Performing face difference calculation on the first face encoding data, the second face encoding data and the first face style data through a preset original face difference recognition model to obtain sample image difference feature data; The original face difference recognition model is adjusted based on the first face style data, the second face style data and the sample image difference feature data to obtain the target face difference recognition model.

3. The method according to claim 2, characterized in that The adjusting the original face difference recognition model based on the first face style data, the second face style data and the sample image difference feature data to obtain the target face difference recognition model includes: Calculating feature differences between the first face style data and the second face style data to obtain face style feature difference data; Perform loss calculation based on the sample image difference feature data and the face style feature difference data to obtain feature difference loss data; The original face difference recognition model is optimized based on the feature difference loss data to obtain the target face difference recognition model.

4. The method according to claim 1, characterized in that The method of performing face difference calculation on the original face encoding data, the original face style data and the target face encoding data through the pre-trained target face difference recognition model to obtain target image difference feature data includes: Calculating feature differences between the original face coding data and the target face coding data to obtain face coding feature difference data; Perform feature splicing on the target face coding data, the face coding feature difference data and the original face style data to obtain target splicing feature data; The target face difference recognition model is used to perform difference feature calculation on the target splicing feature data to obtain the target image difference feature data.

5. The method according to any one of claims 1 to 4, characterized in that: The step of synthesizing a face image based on the target face feature data to obtain a target face synthesized image includes: Performing noise processing on the target facial feature data to obtain facial noise feature data; Performing feature mapping on the facial noise feature data to obtain facial feature map data; Sampling the face feature map data to obtain face sampling feature data; Image generation is performed on the face sampling feature data to obtain the target face composite image.

6. The method according to claim 5, characterized in that The obtaining of the original face image comprises: Get the original video; Extracting video frames from the original video to obtain original video frame images; Perform face recognition on the original video frame image to obtain the original face image.

7. The method according to claim 6, characterized in that After synthesizing the face image based on the target face feature data to obtain the target face synthesized image, the method further includes: Filtering a face replacement image from the original video frame image based on the target face composite image and the original face image; Replacing the face replacement image with the target face composite image to obtain a target video frame image; Obtaining the video frame rate of the original video; Video generation is performed based on the video frame rate and the target video frame image.

8. A facial image synthesis device, characterized in that: The device comprises: The original face image processing module is used to obtain an original face image, perform face encoding on the original face image through a preset face image encoder to obtain original face encoding data, and perform style encoding on the original face image through a preset style image encoder to obtain original face style data; A target face image processing module, used for acquiring a target face image, and performing face encoding on the target face image through the face image encoder to obtain target face encoding data; wherein the target face image is different from the original face image; A face image difference calculation module is used to calculate the face difference of the original face encoding data, the original face style data and the target face encoding data through a pre-trained target face difference recognition model to obtain target image difference feature data; A face feature splicing module, used for performing feature splicing on the target image difference feature data and the original face style data to obtain target face feature data; The image synthesis module is used to synthesize a face image based on the target face feature data to obtain a target face synthesized image.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Face generation method and device, electronic equipment and storage medium

    CN120808416A