A face swapping method and system based on StyleGAN2 architecture

Abstract

This invention discloses a face swapping method and system based on the StyleGAN2 architecture, belonging to the field of face swapping technology. The method includes: decoupling the source identity from the target attributes through feature space transformation, extracting identity and attribute features separately using independent networks, and fusing the features through a dynamic balancing mechanism to eliminate artifacts; processing the fused features using an unmodified StyleGAN2 generator, and combining soft masking technology to achieve seamless fusion of the face and background, simplifying the generation process; and improving face alignment accuracy and enhancing adaptability to complex scenarios based on a multi-level detection mechanism. This method systematically addresses the problems of identity attribute conflict, artifact generation, and insufficient scene robustness, achieving efficient and high-fidelity face swapping.

Description

Technical Field

[0001] The present invention belongs to the field of face swapping technology, and in particular relates to a face swapping method and system based on the StyleGAN2 architecture. Background Art

[0002] Currently, face swapping methods based on generative adversarial networks (GANs) are relatively common. These methods typically use a generator network to transform facial feature maps and facial images between the source and target images, and a discriminator network to determine whether the input image is real or generated, assigning a corresponding probability. By optimizing the adversarial loss function between the generator and discriminator networks, they achieve high-quality and realistic face swapping. For example, the design proposed in patent application number CN202410010020.0 inputs facial ID information and several feature maps into the MobileSwap model to generate a generated face. The generated face image is then combined with a transparency map to produce the final face swap image. While these methods can achieve good face swapping results, they also have drawbacks, such as relying on complex network and loss design, struggling to balance information between the source and target faces, and often producing visible artifacts.

[0003] In addition, face swapping methods based on the StyleGAN2 latent space have been proposed. These methods often first encode the image into StyleGAN2's editable latent space through GAN inversion, and then perform operations in the latent space to achieve the purpose of face swapping. For example, the design proposal StyleSwap from ECCV 2022 made minor modifications to the StyleGAN2 generator, enabling it to successfully process the required information from the source and target, thereby performing face swapping. This type of method is based on the inherent prior knowledge of the pre-trained GAN model and relies on the powerful generation capabilities of StyleGAN2 to produce high-resolution and high-fidelity face swapping results. However, these methods are limited in their ability to decouple the latent code in the W / W+ space, and the face swapping results they generate often fail to well preserve the target face's expression, posture, and other attribute characteristics. Therefore, a new face swapping method is urgently needed. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a face swapping method and system based on the StyleGAN2 architecture to solve the problems existing in the above-mentioned prior art.

[0005] In a first aspect, to achieve the above-mentioned objectives, the present invention provides a face swapping method based on the StyleGAN2 architecture, comprising the following steps:

[0006] Perform face detection and alignment on the source image and target image using a face detection algorithm;

[0007] Extracting feature maps of facial information of the source image and the target image in the W+ space, and converting the feature maps to the S space through an affine transformation;

[0008] Extracting the identity features of the source image and the attribute features of the target image in the S space respectively, and fusing the identity features with the attribute features;

[0009] The fused features are input into the StyleGAN2 generator to generate face swap images;

[0010] The generated face swapped image is smoothly fused with the target image.

[0011] Optionally, the face detection and alignment process includes:

[0012] The MTCNN algorithm is used to construct an image pyramid for the source and target images, and the P-Net is used to generate candidate face windows and calibrate the bounding boxes.

[0013] Further refine the candidate window through R-Net and reject wrong candidates;

[0014] The final bounding box and facial landmark locations are output through O-Net.

[0015] Optionally, the process of converting the feature map into S space includes:

[0016] Extract the feature map of W+ space through the pre-trained pSp encoder;

[0017] The feature map of the W+ space is converted to the S space using affine transformation to achieve the decoupling of identity features and attribute features.

[0018] Optionally, the process of extracting the identity features and attribute features includes:

[0019] Extract identity information from the source feature map in S space through the identity feature extraction network;

[0020] Extract attribute information from the target feature map in S space through the attribute feature extraction network;

[0021] The identity feature extraction network and the attribute feature extraction network share a backbone structure but have independent parameters.

[0022] Optionally, the fusion process includes:

[0023] Dynamically balance the contribution of identity features and attribute features at each layer in the S space through the weight matrix;

[0024] The identity features and attribute features are weighted and summed according to a weight matrix to generate a fused S-space feature map.

[0025] Optionally, the smooth fusion process includes:

[0026] The generated face swapped image is fused element-wise with the target image through soft mask;

[0027] The soft mask is expanded channel by channel to match the RGB channels of the target image.

[0028] In a second aspect, the present invention further provides a face swapping system based on the StyleGAN2 architecture, for implementing a face swapping method based on the StyleGAN2 architecture, the system comprising:

[0029] Face processing module, used to detect and align faces on source and target images;

[0030] A feature extraction module is used to extract feature maps of the W+ space from the source image and the target image, and convert the feature maps into the S space;

[0031] Feature fusion module, used to extract the identity features of the source image and the attribute features of the target image in the S space and fuse the two;

[0032] The image generation module is used to generate face-swapped images based on the fused features using the StyleGAN2 generator;

[0033] The image fusion module is used to smoothly fuse the generated face swap image with the target image.

[0034] Optionally, the image fusion module includes:

[0035] A soft mask generation unit, configured to generate a soft mask for fusion;

[0036] a weighted fusion unit for performing element-wise weighted fusion of the generated face-swapped image with the target image through a soft mask;

[0037] Channel matching unit, which is used to expand the channels of the soft mask to match the RGB channels of the target image to achieve smooth fusion.

[0038] In a third aspect, the present invention further provides a computer terminal device, comprising:

[0039] one or more processors;

[0040] a memory, coupled to the processor, for storing one or more programs;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement a face swapping method based on the StyleGAN2 architecture.

[0042] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements a face swapping method based on the StyleGAN2 architecture.

[0043] Compared with the prior art, the present invention has the following advantages and technical effects:

[0044] The present invention provides a face swapping method and system based on the StyleGAN2 architecture. The present invention converts the feature map of the W+ space into the S space through affine transformation, and uses its hierarchical decoupling characteristics to accurately separate the source identity features and the target attribute features. This solves the attribute loss problem caused by insufficient decoupling ability in the W / W+ space in traditional methods, and significantly improves the naturalness of the generated image. The identity feature extraction network and attribute feature extraction network with independent parameters are used to extract identity information and attribute information from the S space respectively, and the contribution of each layer feature is dynamically balanced in combination with the weight matrix, effectively avoiding facial blur or edge breakage caused by the conflict between identity and attribute. Shadow, ensuring high-fidelity details of the generated image; by directly using the unmodified StyleGAN2 generator to process the fused S-space features, the limitations of existing solutions that require adjustment of the generator structure are avoided. At the same time, soft mask fusion technology is used to achieve seamless connection between the generated face and the target background, reducing dependence on complex post-processing algorithms, significantly reducing implementation complexity and improving processing efficiency; based on the MTCNN multi-level detection mechanism to achieve high-precision face alignment, combined with the adaptability of the S-space to complex scenes such as occlusion and extreme lighting, the method can still output stable and high-quality face swap results under non-ideal conditions, expanding the coverage of practical application scenarios. Through the systematic improvement of the above-mentioned technical means, the present invention has overcome the difficult problems of difficult balance between identity and attributes, artifact generation and insufficient robustness in complex scenes while simplifying the process and reducing structural dependence, providing a solution for high-precision face swapping that is both efficient and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0046] Figure 1 This is a flow chart of a face swapping method based on the StyleGAN2 architecture in a specific implementation of the first embodiment of the present invention;

[0047] Figure 2Schematic diagram of a feature map extraction structure according to an embodiment of the present invention;

[0048] Figure 3 Schematic diagram of conversion from W+ space to S space according to an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a network for extracting identity features according to an embodiment of the present invention;

[0050] Figure 5 A schematic diagram of an attribute feature extraction network according to an embodiment of the present invention;

[0051] Figure 6 The image after face swapping is generated by the StyleGAN2 generator in an embodiment of the present invention;

[0052] Figure 7 This is a flow chart of an embodiment of the present invention, in which: is the identity loss, ArcFace is a pre-trained state-of-the-art face recognition model, which acts as an identity encoder to extract identity features. is the feature matching loss, represents the m-th layer output of the discriminator, is the layer that starts calculating feature matching loss, is the total number of layers. DETAILED DESCRIPTION

[0053] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0054] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0055] First, the technical terms involved in the following embodiments are explained.

[0056] This paper uses the powerful generation capability of StyleGAN2 to perform face swapping, ensuring high resolution and high fidelity of the generated images.

[0057] This paper exploits the higher-level decoupling properties of S-space to more effectively separate source identity from target attributes, enhancing the flexibility of latent encoding. The designed mechanism effectively balances the transfer of source identity with the preservation of target facial features (such as expression and posture), avoiding information loss and artifacts.

[0058] This method reduces the need for complex network architectures and loss functions, simplifies the implementation process, and improves ease of application. It demonstrates improved robustness under various conditions and can handle different types of source and target images.

[0059] The key points of the present invention include:

[0060] 1. A technical method for decoupling the w latent code using S space to ensure the effective separation of source identity and target attributes.

[0061] 2. Reduce dependence on complex network architecture and loss functions, simplify the implementation process, and improve application convenience.

[0062] 3. Steps and methods for extracting and fusing features through identity feature extraction network and attribute feature extraction network.

[0063] Example 1

[0064] like Figure 7 As shown, this embodiment provides a face swapping method based on the StyleGAN2 architecture, including:

[0065] Perform face detection and alignment on the source image and target image using a face detection algorithm;

[0066] Extracting feature maps of facial information of the source image and the target image in the W+ space, and converting the feature maps to the S space through an affine transformation;

[0067] Extracting the identity features of the source image and the attribute features of the target image in the S space respectively, and fusing the identity features with the attribute features;

[0068] The fused features are input into the StyleGAN2 generator to generate face swap images;

[0069] The generated face swapped image is smoothly fused with the target image.

[0070] Specifically, based on exchanging faces to retain the source image Identity information and target images The goal of attribute information, the process of the method is as follows Figure 1 As shown, there are four main steps:

[0071] Obtain the source image and target image to be exchanged, and perform face detection and alignment on the source image and target image using the face detection algorithm MTCNN.

[0072] The feature maps of the source image face information and the target image face information in the W+ space are extracted through the pre-trained pSp encoder, and then the feature maps of the W+ space are converted to the S space through the affine transformation A to achieve better information decoupling.

[0073] The identity information of the source image in the S space is extracted through the identity feature extraction network, and the attribute information of the target image in the S space is extracted through the attribute information extraction network, and then the two are fused.

[0074] The fusion information of the S space obtained in the previous step is sent to the StyleGAN2 generator to generate the face-swapped image.

[0075] As an implementation method of this embodiment, the face detection and alignment process includes:

[0076] The MTCNN algorithm is used to construct an image pyramid for the source and target images, and the P-Net is used to generate candidate face windows and calibrate the bounding boxes.

[0077] Further refine the candidate window through R-Net and reject wrong candidates;

[0078] The final bounding box and facial landmark locations are output through O-Net.

[0079] Specifically, extract the source image and the target image The process of facial information in includes:

[0080] When performing face swapping, the first step is to obtain a source image and a target image. However, these images often contain more than just facial information, including background information and other information. To achieve a better face swap effect, the corresponding facial information should be extracted from the source and target images to reduce interference from non-face areas during the face swap process. Therefore, the present invention uses the MTCNN face detection algorithm to perform face detection and alignment on the source and target images.

[0081] Here are the steps:

[0082] Step 1-1: If Figure 2 As shown in the figure, given an input image, it is initially resized to different scales to construct an image pyramid. A fully convolutional network, called a proposal network (P-Net), is then used to obtain candidate face windows and their bounding box regression vectors. The candidates are then aligned based on the estimated bounding box regression vectors. Afterwards, non-maximum suppression (NMS) is used to merge highly overlapping candidates.

[0083] Step 1-2: All candidates are fed into another CNN, called the Refinement Network (R-Net). These candidates are then refined by the Refinement Network (R-Net), which further rejects a large number of false candidates, calibrates using bounding box regression, and performs NMS.

[0084] Steps 1-3: With more supervision to identify the face region, the output network (O-Net) produces the final bounding box and facial landmark locations.

[0085] As an implementation method in this embodiment, the process of converting the feature map into the S space includes:

[0086] Extract the feature map of W+ space through the pre-trained pSp encoder;

[0087] The feature map of the W+ space is converted to the S space using affine transformation to achieve the decoupling of identity features and attribute features.

[0088] Specifically, the source image and the target image The process of decoupling feature information includes:

[0089] Because face swapping requires the source image to provide identity features and the target image to provide attribute features, after performing face detection and alignment on the source and target images, it is necessary to extract the identity features of the source image and the attribute features of the target image. Before feature extraction, it is necessary to decouple the feature information of the source and target images. The steps are as follows:

[0090] Step 2-1: Extract source image face information through pre-trained pSp encoder and target image face information Feature map in W+ space 、 , first extract feature maps using a standard feature pyramid on the ResNet backbone. For each of the 18 target styles, a small mapping network is trained to extract the learned style from the corresponding feature map, where styles (0-2) are generated from small feature maps, (3-6) from medium feature maps, and (7-18) from the largest feature map. The mapping network map2style is a small fully convolutional network that gradually reduces the spatial size using a set of 2-stride convolutions and LeakyReLU activations. The structural framework is as follows Figure 2 shown.

[0091] The "style" mentioned above generally refers to the feature vectors injected into different layers (at different resolutions) of the generator. These style vectors influence different scales of image generation, controlling everything from overall shape to local texture. For example, early style layers (injected into low-resolution layers) often determine "coarse" features such as the image's overall outline, large color blocks, and composition; whereas later style layers (injected into high-resolution layers) determine more "fine" features such as local texture, details, and color.

[0092] Step 2-2: W+ has limited ability to decouple latent codes in space, and the face swapping results it generates often cannot well preserve the target face’s expression, posture and other attribute features. Figure 3 As shown, the present invention is 、 The matching affine transformation A transforms the feature map of W+ space into S space to obtain 、 , to achieve better information decoupling, thereby better extracting the source image identity features and target image attribute features.

[0093] As an implementation method of this embodiment, the process of extracting the identity features and attribute features includes:

[0094] Extract identity information from the source feature map in S space through the identity feature extraction network;

[0095] Extract attribute information from the target feature map in S space through the attribute feature extraction network;

[0096] The identity feature extraction network and the attribute feature extraction network share a backbone structure but have independent parameters.

[0097] Specifically, after information decoupling, the identity features of the source image in the S space are extracted through the identity feature extraction network, and the attribute features of the target image in the S space are extracted through the attribute feature extraction network, and then the two are fused.

[0098] Step 3-1: Feature mapping of the source image face information in S space Input the identity feature extraction network to obtain the identity feature mapping of the source image face information in S space ,like Figure 4 shown.

[0099] Step 3-2: Feature mapping of the target image face information in S space Input the attribute feature extraction network to obtain the attribute feature mapping of the target image face information in S space ,like Figure 5 As shown in Figure 2, the identity feature extraction network and the attribute feature extraction network share a set of network structures with different parameters.

[0100] Step 3-3: Map identity features and attribute feature maps Fusion to obtain fusion information , as shown in formula (1). Where M is the weight parameter matrix and 1 is the all-1 matrix. and attribute feature maps Both have 18 layers, so M is used to balance the contribution of source image identity information and target image attribute information in each layer of S space.

[0101]

[0102] As an implementation method in this embodiment, the fusion process includes:

[0103] Dynamically balance the contribution of identity features and attribute features at each layer in the S space through the weight matrix;

[0104] The identity features and attribute features are weighted and summed according to a weight matrix to generate a fused S-space feature map.

[0105] Specifically, the information will be integrated Send it to the StyleGAN2 generator to generate the face-swapped image. Because the face-swapped image only includes face information, it needs to be compared with the target image. Smooth blending.

[0106] Step 4-1: Fusion information Feed it into each layer of the StyleGAN2 generator to generate the face-swapped image. Already in S space, so here we use the StyleGAN2 generator with the affine transformation A removed, such as Figure 6 shown.

[0107] As an implementation method of this embodiment, the smooth fusion process includes:

[0108] The generated face swapped image is fused element-wise with the target image through soft mask;

[0109] The soft mask is expanded channel by channel to match the RGB channels of the target image.

[0110] Specifically, step 4-2: because the image after face swap Only includes face information, which needs to be compared with the original target image Smooth fusion to get the final image The present invention uses a soft mask Mask for fusion, as shown in formula (2). Mask is a soft mask with each value between 0 and 1, where * indicates element-wise multiplication with broadcasting and 1 is a tensor of all 1s. The mask repeats itself three times per channel to match the RGB channels.

[0111]

[0112] Based on this, embodiments of the present invention provide a face swapping method based on the StyleGAN2 architecture. This method leverages the higher-level decoupling properties of S-space to more effectively separate source identity from target attributes, enhancing the flexible manipulation of latent codes. This method achieves a more natural face swapping effect while preserving the characteristics of both the source and target faces.

[0113] This method effectively balances the relationship between source identity transfer and target attribute preservation, avoiding the information loss and artifacts common in traditional methods. Through improved encoding decoupling, it reduces the loss of latent structural information and improves the quality and detail preservation of the generated image.

[0114] The present invention demonstrates improved robustness under various conditions and can process different types of source and target images. Compared to traditional methods that rely on complex networks and loss functions, the new architecture can reduce design complexity.

[0115] Example 2

[0116] In this embodiment, a computer terminal device is provided, including:

[0117] one or more processors;

[0118] a memory, coupled to the processor, for storing one or more programs;

[0119] When the one or more programs are executed by the one or more processors, the one or more processors implement the methods in the above embodiments.

[0120] In this embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method in the above embodiment is implemented.

[0121] In this embodiment, an electronic device is further provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0122] The above program can be executed in a processor or stored in a memory (or computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0123] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0124] This embodiment provides such a device or system. The system is called a face swapping system based on the StyleGAN2 architecture, and includes:

[0125] Face processing module, used to detect and align faces on source and target images;

[0126] A feature extraction module is used to extract feature maps of the W+ space from the source image and the target image, and convert the feature maps into the S space;

[0127] Feature fusion module, used to extract the identity features of the source image and the attribute features of the target image in the S space and fuse the two;

[0128] The image generation module is used to generate face-swapped images based on the fused features using the StyleGAN2 generator;

[0129] The image fusion module is used to smoothly fuse the generated face swap image with the target image.

[0130] As an implementation method of this embodiment, the face processing module includes:

[0131] An image pyramid construction unit, used to construct an image pyramid for the source image and the target image using the MTCNN algorithm;

[0132] The candidate window generation unit is used to generate candidate face windows and calibrate the bounding box using P-Net;

[0133] The candidate window refinement unit is used to further refine the candidate window and eliminate incorrect candidates through R-Net;

[0134] Bounding box detection unit, which is used to output the final bounding box and facial landmark locations through O-Net.

[0135] As an implementation method of this embodiment, the feature extraction module includes:

[0136] A W+ space feature extraction unit is used to extract feature maps of the W+ space from the source image and the target image respectively using the pre-trained pSp encoder;

[0137] The affine transformation unit is used to transform the feature map of the W+ space into the S space using affine transformation to achieve the decoupling of identity features and attribute features.

[0138] As an implementation method of this embodiment, the feature fusion module includes:

[0139] an identity feature extraction unit, configured to extract identity information from a source image feature map in the S space;

[0140] An attribute feature extraction unit, configured to extract attribute information from a target image feature map in the S space;

[0141] The dynamic weighted fusion unit is used to perform weighted summation on the identity features and attribute features of each layer in the S space according to a preset weight matrix to generate a fused S space feature map.

[0142] As an implementation manner in this embodiment, the image generation module includes:

[0143] Feature input unit, used to receive the fused S-space feature map;

[0144] StyleGAN2 generation unit, used to generate face-swapped images based on the StyleGAN2 generator.

[0145] As an implementation in this embodiment, the image fusion module includes:

[0146] A soft mask generation unit, configured to generate a soft mask for fusion;

[0147] a weighted fusion unit for performing element-wise weighted fusion of the generated face-swapped image with the target image through a soft mask;

[0148] Channel matching unit, which is used to expand the channels of the soft mask to match the RGB channels of the target image to achieve smooth fusion.

[0149] The system or device is used to implement the functions of the method in the above-mentioned embodiment. Each module in the system or device corresponds to each step in the method, which has been explained in the method and will not be repeated here.

[0150] Through the above implementation, the problem of face swapping based on the StyleGAN2 architecture in the related art is solved. The present invention uses the powerful generation capability of StyleGAN2 to perform face swapping, ensuring high resolution and high fidelity of the generated images.

[0151] This paper exploits the higher-level decoupling properties of S-space to more effectively separate source identity from target attributes, enhancing the flexibility of latent encoding. The designed mechanism effectively balances the transfer of source identity with the preservation of target facial features (such as expression and posture), avoiding information loss and artifacts.

[0152] This method reduces the need for complex network architectures and loss functions, simplifies the implementation process, and improves ease of application. It demonstrates improved robustness under various conditions and can handle different types of source and target images.

[0153] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A face swapping method based on the StyleGAN2 architecture, characterized in that: The following steps are involved: Perform face detection and alignment on the source image and target image using a face detection algorithm; The face detection and alignment process includes: The MTCNN algorithm is used to construct an image pyramid for the source and target images, and the P-Net is used to generate candidate face windows and calibrate the bounding boxes. Further refine the candidate window through R-Net and reject wrong candidates; The final bounding box and facial landmark positions are output through O-Net. The feature maps of the facial information of the source and target images in W+ space are extracted through the pre-trained pSp encoder, and the feature maps are converted to S space through affine transformation to achieve decoupling of identity features and attribute features. Extracting the identity features of the source image and the attribute features of the target image in the S space respectively, and fusing the identity features with the attribute features; The process of extracting the identity features and attribute features includes: The feature map of the source image face information in S space Input the identity feature extraction network to obtain the identity feature mapping of the source image face information in S space ; The feature map of the target image face information in S space Input the attribute feature extraction network to obtain the attribute feature mapping of the target image face information in S space ,The identity feature extraction network and the attribute feature extraction network share a set of network structures with different parameters; Mapping identity features and attribute feature maps Fusion to obtain fusion information : Where M is the weight parameter matrix, 1 is the all-1 matrix, and the identity feature map and attribute feature maps Both have 18 layers, and M is used to balance the contribution of source image identity information and target image attribute information in each layer of S space; The fusion process includes: Dynamically balance the contribution of identity features and attribute features at each layer in the S space through the weight matrix; Performing weighted summation of the identity features and attribute features according to a weight matrix to generate a fused S-space feature map; The fused features are input into the StyleGAN2 generator to generate face swap images; Smoothly fuse the generated face swap image with the target image; The smooth fusion process includes: The generated face swap image is weightedly fused with the target image element by element through a soft mask. The soft mask is expanded channel by channel to match the RGB channels of the target image; Where Mask represents a soft mask with each value between 0 and 1, where * indicates element-wise multiplication with broadcasting, and 1 is a tensor of all 1s, and the mask repeats itself three times per channel to match the RGB channels. For the image after face swap, is the original target image.

2. A face swapping system based on the StyleGAN2 architecture, characterized by: The system is used to implement the various steps of the face swapping method based on the StyleGAN2 architecture as claimed in claim 1, and the system includes: Face processing module, used to detect and align faces on source and target images; A feature extraction module is used to extract feature maps of the W+ space from the source image and the target image, and convert the feature maps into the S space; Feature fusion module, used to extract the identity features of the source image and the attribute features of the target image in the S space and fuse the two; The image generation module is used to generate face-swapped images based on the fused features using the StyleGAN2 generator; The image fusion module is used to smoothly fuse the generated face swap image with the target image.

3. The system according to claim 2, characterized in that The image fusion module includes: A soft mask generation unit, configured to generate a soft mask for fusion; a weighted fusion unit for performing element-wise weighted fusion of the generated face-swapped image with the target image through a soft mask; Channel matching unit, which is used to expand the channels of the soft mask to match the RGB channels of the target image to achieve smooth fusion.

4. A computer terminal device, characterized in that: include: one or more processors; a memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the face swapping method based on the StyleGAN2 architecture as claimed in claim 1.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the face swapping method based on the StyleGAN2 architecture as claimed in claim 1 is implemented.

Images