Generative adversarial network training method, image face swapping, video face swapping method and device

Abstract

The embodiment of the present invention discloses a generative adversarial network training method, image face-changing, video face-changing method and device, including: obtaining an original image and a target image; initializing a generator and a discriminator of a generative adversarial network; inputting the original image and the target image into the generator for training to obtain a face-changing image, wherein the generator extracts an attribute feature map of a first face from the original image, extracts an identity feature of a second face from the target image, injects the identity feature into the attribute feature map to generate a mixed feature map, decodes the mixed feature map according to the identity feature and the attribute feature map to obtain a face-changing image after the second face replaces the first face; inputs the original image and the face-changing image into the discriminator for training to obtain a judgment value; and adjusts the generator and the discriminator according to the judgment value, the face-changing image, the original image and the target image. The trained generator is used to generate the face-changing image, and the face-changing effect is real and natural, and the attribute features of the original image and the identity features of the target image can be maintained.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a generative adversarial network training method, an image face-swapping method, a video face-swapping method and a device. Background Art

[0002] With the popularization of video applications such as short videos and live broadcasts, face changing is an important technology in the field of computer vision. Face changing is widely used in content production, film production, entertainment video production, etc.

[0003] Face swapping means that given an original image and a target image, the identity features in the target image are transferred to the original image to obtain a face swapped image, so that the face swapped image not only maintains the identity features of the target image, but also has the attribute features such as facial posture and facial expression of the original image. In addition, the face swapped image is required to be real and natural. The existing technology includes the following three face swapping methods:

[0004] 1) Face fusion face swapping based on facial key points. This method first obtains the facial key points of the original image and the target image, then extracts the facial area of ​​the original image through the key points of the original image, and then fuses the facial area of ​​the original image into the target image based on the key points of the target image. This method can easily cause the face of the face-swapped image to be unrealistic and unnatural.

[0005] 2) Face-changing based on 3D facial modeling. This method reconstructs 3D models for the original image and the target image respectively, then extracts identity features from the 3D model of the target image, and combines the attribute features of the 3D model of the original image to generate a face-changing image. The face-changing image generated by this method is also unrealistic and unnatural.

[0006] 3) Face-changing based on generative adversarial networks. This method uses a neural network to extract attribute features from the original image and identity features from the target image. The two features are then combined and decoded through a decoder to obtain a replaced face image. The face-changing image generated by this method is real and natural, but it is difficult to simultaneously maintain the attribute features of the original image and the identity features of the target image.

[0007] In summary, it is difficult for existing face-changing technologies to obtain real and natural face-changing images, and it is impossible to simultaneously maintain the attribute characteristics of the original image and the identity characteristics of the target image in the face-changing images. Summary of the invention

[0008] The embodiments of the present invention provide a generative adversarial network training method, an image face-changing method, a video face-changing method, an apparatus, an electronic device and a storage medium to solve the problem that a real and natural face-changing image cannot be obtained in the existing face-changing technology, and the attribute characteristics of the original image and the identity characteristics of the target image cannot be simultaneously maintained in the face-changing image.

[0009] In a first aspect, an embodiment of the present invention provides a method for training a generative adversarial network, comprising:

[0010] Obtaining an original image containing a first face and a target image containing a second face;

[0011] Initialize the generator and discriminator of the generative adversarial network;

[0012] The original image and the target image are input into the generator for training to obtain a face-swapped image, wherein the generator is used to extract an attribute feature map of the first face from the original image, extract an identity feature of the second face from the target image, inject the identity feature into the attribute feature map to generate a mixed feature map, and decode the mixed feature map according to the identity feature and the attribute feature map to obtain a face-swapped image in which the second face replaces the first face;

[0013] Inputting the original image and the face-swapped image into the discriminator for training to obtain a judgment value;

[0014] The parameters of the generator and the discriminator are adjusted according to the judgment value, the face-changing image, the original image and the target image.

[0015] In a second aspect, an embodiment of the present invention provides an image face-swapping method, comprising:

[0016] Obtaining an original image containing a first face and a target image containing a second face;

[0017] Inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face;

[0018] Wherein, the generator is trained by the generative adversarial network training method described in any embodiment of the present invention.

[0019] In a third aspect, an embodiment of the present invention provides a video face swapping method, comprising:

[0020] Get the video data to be replaced;

[0021] Extracting a video image containing the first face from the video data as an original image;

[0022] Obtain a target image containing a second face;

[0023] Inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face;

[0024] generating face-swapped video data based on the face-swapped image;

[0025] Wherein, the generator is trained by the generative adversarial network training method described in any embodiment of the present invention.

[0026] In a fourth aspect, an embodiment of the present invention provides a generative adversarial network training device, comprising:

[0027] The original image and target image acquisition module is used to acquire the original image containing the first face and the target image containing the second face;

[0028] Generative adversarial network initialization module, used to initialize the generator and discriminator of the generative adversarial network;

[0029] a generator training module, configured to input the original image and the target image into the generator for training to obtain a face-swapped image, wherein the generator is configured to extract an attribute feature map of the first face from the original image, extract an identity feature of the second face from the target image, inject the identity feature into the attribute feature map to generate a mixed feature map, and decode the mixed feature map according to the identity feature and the attribute feature map to obtain a face-swapped image in which the second face replaces the first face;

[0030] A discriminator training module, used for training the discriminator using the original image and the face-swapped image to obtain a judgment value;

[0031] A parameter adjustment module is used to adjust the parameters of the generator and the discriminator according to the judgment value, the face-changing image, the original image and the target image.

[0032] In a fifth aspect, an embodiment of the present invention provides an image face-changing device, comprising:

[0033] The original image and target image acquisition module is used to acquire the original image containing the first face and the target image containing the second face;

[0034] An image face-swapping module, used for inputting the original image and the target image into a generator of a generative adversarial network, and obtaining a face-swapping image of the original image after replacing the first face with the second face;

[0035] Wherein, the generator is trained by the generative adversarial network training method described in any embodiment of the present invention.

[0036] In a sixth aspect, an embodiment of the present invention provides a video face-changing device, comprising:

[0037] The module for acquiring the video data to be replaced is used to acquire the video data to be replaced;

[0038] An original image extraction module, used for extracting a video image containing a first face from the video data as an original image;

[0039] A target image acquisition module, used to acquire a target image containing a second face;

[0040] A video face-swapping module, used for inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapping image of the original image after replacing the first face with the second face;

[0041] A face-swap video data generation module, used to generate face-swap video data based on the face-swap image;

[0042] Wherein, the generator is trained by the generative adversarial network training method described in any embodiment of the present invention.

[0043] In a seventh aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0044] one or more processors;

[0045] a storage device for storing one or more programs,

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the generative adversarial network training method, and / or the image face-changing method, and / or the video face-changing method described in any embodiment of the present invention.

[0047] In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the generative adversarial network training method, and / or the image face-changing method, and / or the video face-changing method described in any embodiment of the present invention.

[0048] In the embodiment of the present invention, when training a generator, the generator extracts an attribute feature map of a first face from an original image, extracts an identity feature of a second face from a target image, and injects the identity feature into the attribute feature map to generate a mixed feature map, and decodes the mixed feature map according to the identity feature and the attribute feature map to obtain a face-changing image after the second face replaces the first face, and inputs the original image and the face-changing image into a discriminator for training to obtain a judgment value; according to the judgment value, the face-changing image, the original image and the target image, the parameters of the generator and the discriminator are adjusted to obtain a trained generative adversarial network, and the mixed feature map is decoded in combination with the attribute feature and the identity feature during the decoding process, so that the face-changing image better maintains the attribute features of the original image such as facial posture and facial expression, and the identity features in the target image are also better integrated into the face-changing image, thereby enhancing the migration ability of the identity features of the target image. When the trained generator is used to face-change an image or video, the obtained face-changing image or video is real and natural, and can maintain the attribute features of the original image and the identity features of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flowchart of the steps of a generative adversarial network training method provided in Embodiment 1 of the present invention;

[0050] Figure 2A is a flowchart of a method for training a generative adversarial network provided in Embodiment 2 of the present invention;

[0051] Figure 2B is a schematic diagram of a generator according to an embodiment of the present invention;

[0052] Figure 3 is a flowchart of the steps of an image face-swapping method provided by Embodiment 3 of the present invention;

[0053] Figure 4 This is a flowchart of the steps of a video face swapping method provided by Embodiment 4 of the present invention;

[0054] Figure 5 It is a structural block diagram of a generative adversarial network training device provided in Embodiment 5 of the present invention;

[0055] Figure 6 is a structural block diagram of an image face-swapping device provided by Embodiment 6 of the present invention;

[0056] Figure 7 This is a structural block diagram of a video face-changing device provided by Embodiment 7 of the present invention;

[0057] Figure 8 It is a structural schematic diagram of an electronic device provided in Embodiment 8 of the present invention. DETAILED DESCRIPTION

[0058] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0059] Embodiment 1

[0060] Figure 1 A flowchart of a method for training a generative adversarial network provided in the first embodiment of the present invention is provided. The embodiment of the present invention can be applied to training a generative adversarial network to perform face-changing on an image or video through a generator of a trained generative adversarial network. The method can be performed by a generative adversarial network training device of the embodiment of the present invention. The generative adversarial network training device can be implemented by hardware or software and integrated in the electronic device provided in the embodiment of the present invention. Specifically, Figure 1 As shown, the generative adversarial network training method of the embodiment of the present invention may include the following steps:

[0061] S101: Acquire an original image containing a first face and a target image containing a second face.

[0062] In an embodiment of the present invention, the original image and the target image are images including faces, wherein the original image is an image whose face needs to be replaced, and the target image is an image used to replace the face of the original image. In an example of the present invention, the original image may be an image containing a first face extracted from video data, and the target image may be an image containing a second face, wherein the first face and the second face are different faces. In practical applications, a large number of images of different faces may be obtained, from which two images are randomly selected as the original image and the target image, and the original image and the target image constitute an image pair as a training sample.

[0063] S102: Initialize the generator and discriminator of the generative adversarial network.

[0064] In the embodiment of the present invention, the Generative Adversarial Networks (GANs) includes a generator and a discriminator, the generator is used to generate a new image after inputting data, and the discriminator is used to determine the probability of whether the new image is real. In the embodiment of the present invention, the generator and the discriminator can be neural networks, and the network parameters of the generator and the discriminator of the generative adversarial network can be initialized.

[0065] In an optional embodiment of the present invention, the generator may include an encoding network, a decoding network, an identity extraction network and a residual network, wherein the identity extraction network and the residual network may be pre-trained networks, and the initialization generator may be the network parameters of the encoding network and the decoding network of the initialization generator.

[0066] S103, inputting the original image and the target image into the generator for training to obtain a face-swapped image.

[0067] In an embodiment of the present invention, a generator may include an encoding network, a decoding network, an identity extraction network and a residual network. In the generator, an attribute feature map of a first face may be extracted from an original image through the encoding network, identity features of a second face may be extracted from a target image through the identity extraction network, and the identity features may be injected into the attribute feature map using a residual network to generate a mixed feature map, and the mixed feature map may be decoded in a decoding network according to the identity features and the attribute feature map to obtain a face-changed image after the second face replaces the first face, wherein the attribute feature map may be a feature map expressing attributes such as facial posture and facial expression of the first face, and the identity feature may be information that can identify the identity of the second face.

[0068] S104, inputting the original image and the face-swapped image into the discriminator for training to obtain a judgment value.

[0069] In the training process of the adversarial generative network, the purpose of the generator is to generate realistic images as much as possible to deceive the discriminator, and the purpose of the discriminator is to distinguish the images generated by the generator from the real images. By alternately training the generator and the discriminator, the training of the generator and the discriminator constitutes a dynamic "game process", and the images generated by the trained generator are finally "indistinguishable from the real", that is, infinitely close to the real image. Specifically in the embodiment of the present invention, the original image and the face-swapped image are input into the discriminator to train the discriminator, and the judgment value of the face-swapped image is obtained after each input, wherein the judgment value can be the probability that the face-swapped image belongs to the real image.

[0070] S105. Adjust the parameters of the generator and the discriminator according to the judgment value, the face-changing image, the original image and the target image.

[0071] Specifically, the total loss can be calculated based on the judgment value, the face-swapped image, the original image and the target image. When the total loss is less than a preset value, the training of the generator and the discriminator is stopped. Otherwise, the network parameters of the discriminator and the generator are adjusted according to the total loss, and then a new round of training iterative updates is started until the total loss is less than the preset value. After stopping the training, a trained adversarial generative network is obtained. After the original image and the target image are input into the generator of the adversarial generative network, the generator automatically outputs the face-swapped image.

[0072] In an optional embodiment of the present invention, the adversarial loss and key point loss can be calculated based on the judgment value, the original image and the face-swapped image, the identity feature loss can be calculated based on the target image and the face-swapped image, and after the two original images are input into the generator to obtain the self-swapped image of the original image, the reconstruction loss is calculated based on the original image and the self-swapped image, the sum of the adversarial loss, reconstruction loss, key point loss and identity feature loss is calculated to obtain the total loss, and the gradient is calculated based on the total loss to adjust the parameters of the encoding network and the decoding network of the generator, and the gradient is calculated based on the adversarial loss to adjust the parameters of the discriminator.

[0073] The generative adversarial network of the embodiment of the present invention includes a generator and a discriminator. The generator extracts an attribute feature map of a first face from an original image, extracts an identity feature of a second face from a target image, and injects the identity feature into the attribute feature map to generate a mixed feature map, and decodes the mixed feature map according to the identity feature and the attribute feature map to obtain a face-changing image after the second face replaces the first face, and inputs the original image and the face-changing image into the discriminator for training to obtain a judgment value; according to the judgment value, the face-changing image, the original image and the target image, the parameters of the generator and the discriminator are adjusted until a trained generative adversarial network is obtained, and the mixed feature map is decoded in combination with the attribute features of the original image and the identity features of the target image during the decoding process, so that the face-changing image better maintains the attribute features of the original image such as facial posture and facial expression, and the identity features in the target image are also better integrated into the face-changing image, thereby enhancing the migration ability of the identity features of the target image. When the trained generative adversarial network generator is used to change the face of an image or video, the obtained face-changing image or video is real and natural, and can maintain the attribute features of the original image and the identity features of the target image.

[0074] Embodiment 2

[0075] Figure 2A A flowchart of a method for training a generative adversarial network is provided in the second embodiment of the present invention. The embodiment of the present invention is optimized on the basis of the first embodiment. Specifically, Figure 2A As shown, the generative adversarial network training method of the embodiment of the present invention may include the following steps:

[0076] S201: Acquire an original image containing a first face and a target image containing a second face.

[0077] S202, initializing the parameters of the discriminator of the generative adversarial network, the parameters of the encoding network and the decoding network of the generator, and obtaining the trained residual network and identity extraction network used in the generator.

[0078] In an embodiment of the present invention, the generative adversarial network includes a discriminator and a generator, and the generator may include an encoding network, a decoding network, a residual network, and an identity extraction network, wherein the residual network and the identity extraction network may be various pre-trained neural networks. The initialization referred to in the embodiment of the present invention may be to initialize the parameters of the discriminator, the encoding network, and the decoding network. Specifically, it may be to construct a network structure of the discriminator, the encoding network, and the decoding network, and set the network parameters of the network structure. In an embodiment of the present invention, the discriminator, the encoding network, and the decoding network may be various neural networks.

[0079] like Figure 2B The following is a schematic diagram of the generator. Figure 2B In the figure, the generator 30 includes an encoding network 301, a decoding network 302, an identity extraction network 303, and a residual module 304, wherein the encoding network 301 and the decoding network 302 can be symmetrical convolutional neural networks and deconvolutional neural networks, and the residual module 304 is connected between the encoding network 301 and the decoding network 302. After the original image 10 and the target image 20 are input into the generator 30, a face-changing image 40 can be obtained.

[0080] S203: Use the encoding network to encode the original image to obtain an attribute feature map of the first face.

[0081] In an embodiment of the present invention, the original image may be preprocessed to obtain a preprocessed original image, and then the preprocessed original image may be input into the encoding network to obtain a downsampled feature map output by each downsampled convolution layer. The preprocessing includes adjusting the image size, and the downsampled feature map output by the last downsampled convolution layer of the encoding network is the attribute feature map of the first face.

[0082] For example, Figure 2B As shown, the encoding network 301 can be a network including multiple down-sampling convolution layers. After the original image 10 is cropped into an image of a specified size, the cropped original image is input into the down-sampling convolution layer. Each down-sampling convolution layer performs sampling and encoding processing on the cropped original image to output a down-sampling feature map, and the down-sampling feature map is input into the next down-sampling convolution layer. The down-sampling feature map output by the last down-sampling convolution layer of the encoding network is the attribute feature map F of the first face. H×W×D , H and W are the height and width of the attribute feature map, respectively, D is the number of channels, and each downsampling convolutional layer outputs a downsampling feature map like Figure 2B As shown, the encoding network 301 finally outputs the attribute feature map 50 of the first face.

[0083] S204: Input the target image into the identity extraction network to extract the identity features of the second face.

[0084] In the embodiment of the present invention, the identity feature may refer to information that can distinguish the identities of two faces belonging to different persons, and the identity extraction network may be a pre-trained network, for example, a pre-trained convolutional neural network CNN, a recurrent neural network RNN, a deep neural network DNN, etc. After the target image is input into the identity extraction network, the identity feature F of the second face can be extracted. ID , identity feature F ID It can be a one-dimensional vector that contains the identity information of the face, such as Figure 2B As shown, the target image 20 is input into the identity extraction network 303 to obtain the identity feature 60.

[0085] S205: Use the residual network to inject the identity feature into the attribute feature map to obtain a mixed feature map.

[0086] In an optional embodiment of the present invention, the identity feature can be converted first to obtain the identity feature mean and identity feature variance of the identity feature, and the identity feature mean, identity feature variance and attribute feature map are input into the residual network to migrate the identity feature to the attribute feature map through the residual network to obtain a mixed feature map.

[0087] like Figure 2B As shown, the identity feature 60 can output the identity feature mean μ and the identity feature variance σ after passing through a fully connected layer 305. The identity feature mean μ, the identity feature variance σ and the attribute feature 50 are input into the residual network 304 together to obtain a mixed feature map 70.

[0088] In an optional embodiment of the present invention, the residual network may be an adaptive instance normalized residual module (AdaINResBlk). The residual network may describe a style image as the mean and variance of a feature map, and implement style injection by changing the mean and variance of the feature map of the content image. With x representing the content feature map and y representing the style image, the formula of the residual network is:

[0089]

[0090] In the above formula, μ_y1, σ_y1 are the mean and variance of the identity feature to be injected. Specifically in the embodiment of the present invention, x is the identity feature, y is the attribute feature map, and AdaIN(x, y) is the mixed feature map.

[0091] The embodiment of the present invention can inject the identity features of the second face into the attribute feature map of the first face through the residual network, thereby replacing the identity features of the first face with the identity features of the second face, and can retain the posture, expression and other information of the first face in the original image, thereby realizing the combination of the attribute features of the first face in the original image and the identity features of the second face in the target image.

[0092] S206. Based on the attribute features and the identity features, the mixed feature map is decoded using the decoding network to obtain a face-changing image in which the first face is replaced by the second face.

[0093] In an optional embodiment of the present invention, after obtaining a mixed feature map, the mixed feature map and the identity feature are spliced ​​to obtain a spliced ​​feature, and the spliced ​​feature is input into a decoding network to obtain a face-changing image through sampling processing through multiple upsampling convolution layers, wherein, for each upsampling convolution layer in the decoding network, the downsampling convolution layer corresponding to the upsampling convolution layer in the encoding network is determined, the downsampling feature map output by the downsampling convolution layer is obtained, and the upsampling feature output by the previous upsampling convolution layer of the upsampling convolution layer is obtained, and the downsampling feature map and the upsampling feature are spliced ​​as the decoding object of the upsampling convolution layer.

[0094] Specifically, Figure 2B As shown, after the residual network 304 outputs the mixed feature map 70, the identity features extracted by the identity extraction network 303 are spliced ​​into the mixed feature map 70 to obtain the spliced ​​features, thereby improving the migration ability of the identity features of the second face. After the decoding network 302 decodes the spliced ​​features, it can better maintain the identity information of the second face.

[0095] More preferably, Figure 2B As shown in the figure, the intermediate features of the decoding network are connected to the feature layer of the decoding network through cross-connection. Specifically, the decoding network and the encoding network are symmetrical up-sampling convolutional neural network and down-sampling convolutional neural network. In the encoding network, each down-sampling convolutional layer outputs down-sampled features. In the decoding network, each upsampling convolutional layer inputs the upsampled features For a certain upsampling convolution layer i in the decoding network, the corresponding downsampling convolution layer of the upsampling convolution layer i in the encoding network can be determined, and the downsampling features of the output of the downsampling convolution layer can be obtained. Will and After adding, upsample and output upsampled features As the input of the next upsampling convolutional layer.

[0096] During the decoding process, the embodiment of the present invention inputs the intermediate features output by the downsampling convolution layer of the encoding network into the upsampling convolution layer of the decoding network through a cross-connection operation, so that the attribute features of the first face in the original image are better integrated into the face-swapped image, and the face-swapped image is more realistic and natural.

[0097] S207, inputting the original image and the face-swapped image into the discriminator for training to obtain a judgment value.

[0098] The embodiment of the present invention alternately trains the generator and the discriminator. The generator is first trained to obtain a face-swapped image. The face-swapped image and the original image are then used to train the discriminator. Then, the generator is trained alternately. The generative adversarial network is trained alternately in this way. Training the generator once and the discriminator once is one round of training. After each round of training, the generator generates a face-swapped image. The discriminator judges the face-swapped image to obtain a judgment value. The judgment value can be the probability that the face-swapped image belongs to a real image.

[0099] S208. Calculate a total loss according to the judgment value, the face-swapped image, the original image and the target image.

[0100] In the embodiment of the present invention, the total loss may be the sum of the adversarial loss, the reconstruction loss, the key point loss and the identity feature loss. The adversarial loss, the reconstruction loss, the key point loss and the identity feature loss may be calculated respectively, and then the adversarial loss, the reconstruction loss, the key point loss and the identity feature loss are summed to obtain the total loss. Specifically, the following sub-steps may be included:

[0101] S2081. Calculate adversarial loss and key point loss based on the judgment value, the original image and the face-swapped image.

[0102] Specifically, the adversarial loss can be calculated according to the judgment value and the preset adversarial loss function, the key points of the face in the original image and the face-swapped image are obtained, and the distance between the key points of the face in the original image and the face-swapped image is calculated to obtain the key point loss.

[0103] For example, the adversarial loss gan_loss is:

[0104] gan_loss = ∑ - log D(G(X i ))

[0105] In the above formula, G(X i ) is the face-changing image generated by the generator, D(G(X i )) is the discriminator for distinguishing the face-swapped image G(X i ) is the judgment value of the real image.

[0106] Exemplarily, for the key point loss, the facial key points of the original image and the face-swapped image can be extracted through a pre-trained facial posture evaluation network, and then the facial key points of the face-swapped image are constrained to be similar to the facial key points of the original image. Specifically, the facial key point coordinates lmks_gen of the face-swapped image and the facial key point coordinates lmks_src of the original image can be obtained, and the key point loss lmks_loss is:

[0107] lmks_loss=||lmks_gen-lmks_target||2

[0108] The embodiment of the present invention constrains the face-swapped image by calculating the key point loss, so that the facial key points of the face-swapped image are similar to the facial key points of the original image, thereby allowing the face-swapped image to better maintain the facial expression, facial posture and other attribute features of the original image.

[0109] S2082. Calculate identity feature loss based on the target image and the face-swapped image.

[0110] In an optional embodiment of the present invention, the identity extraction network is a pre-trained network, and the target image and the face-swapped image can be respectively input into the identity extraction network to extract the identity features of the face in the target image and the identity features of the face in the face-swapped image through the identity extraction network, and calculate the distance between the identity features of the face in the target image and the identity features of the face in the face-swapped image to obtain the identity feature loss. For example, the identity feature of the face-swapped image can be recorded as FeatID gen , the identity feature of the target image is recorded as FeatID target , then the identity feature loss ID_loss is:

[0111] ID_loss=||FeatID gen -FeatID target ||2

[0112] In an embodiment of the present invention, by calculating the identity feature loss of the target image and the face-swapped image, the identity features of the face-swapped image can be constrained to be more similar to the identity features of the target image, so that the face-swapped image can better maintain the identity features of the target image.

[0113] S2083. Input the two original images into the generator to obtain a self-swapped image of the original image, and calculate the reconstruction loss based on the original image and the self-swapped image.

[0114] Specifically, the original image can be simultaneously input into the encoding network and the identity extraction network of the generator, and the self-face-swapped image of the original image is generated by the generator, that is, the face-swapped image in which the face of one original image replaces the face of another original image is generated by the generator, and then the reconstruction loss of the reconstructed original image is calculated. For example, the two original images are taken as original_img, and the generated self-face-swapped image is taken as src_img, then the reconstruction loss recon_loss is:

[0115] recon_loss=||src img -original img ||2

[0116] The above formula is the difference between the pixel values ​​of the pixels at the same position in the original image and the self-face-swapped image. By calculating the reconstruction loss of the generator, the parameters of the generator are constrained and adjusted, so that the face-swapped image generated by the generator can better maintain the attribute characteristics of the original image, and the face-swapped image is more realistic and natural.

[0117] S2084. Calculate the sum of the adversarial loss, the reconstruction loss, the key point loss and the identity feature loss to obtain a total loss.

[0118] Specifically, the total loss is the sum of adversarial loss, reconstruction loss, key point loss and identity feature loss, that is, the total loss total_loss is:

[0119] total_loss=recon_loss+ID_loss+gan_loss+lmks_loss

[0120] Of course, in practical applications, weights can also be set for adversarial loss, reconstruction loss, key point loss and identity feature loss, and the product of the weight and loss is calculated to obtain the weight of each loss, and the sum of the weights is taken as the total loss. Technical personnel in this field can also calculate the weighted average of each loss as the total loss. The embodiment of the present invention does not limit the method of calculating the total loss.

[0121] S209. Adjust parameters of the generator and the discriminator according to the total loss.

[0122] In an embodiment of the present invention, a generative adversarial network includes a generator and a discriminator. The generative adversarial network can be trained by alternately training the generator and the discriminator, and finally the parameters of the generator and the discriminator are adjusted by calculating the total loss. Optionally, the total loss is calculated after a round of alternating training, and it is determined whether the total loss is less than a preset threshold. If the total loss is less than the preset threshold, it means that the accuracy of the generator is high enough, and the face-swapped image generated by the generator is sufficient to deceive the discriminator. The training of the generator and the discriminator can be stopped. If the total loss is greater than the preset threshold, it means that the accuracy of the generator is insufficient, and the discriminator can still identify the authenticity of the face-swapped image generated by the generator. The parameters of the discriminator can be adjusted according to the adversarial loss, and the parameters of the encoder and decoder in the generator can be adjusted according to the total loss, and return to S203 to alternately train the generator and the discriminator until the condition for stopping iterative training is met.

[0123] Specifically, the parameters of the discriminator, encoder or decoder are adjusted by updating the parameters through a gradient descent algorithm, wherein the gradient descent algorithm may be a stochastic gradient descent method SGD or other gradient descent methods. The embodiments of the present invention do not limit the gradient algorithm and the method for adjusting and updating the parameters.

[0124] When training a generator in an embodiment of the present invention, an encoding network is used to encode the original image to obtain an attribute feature map of the first face, the target image is input into the identity extraction network to extract the identity features of the second face, a residual network is used to inject the identity features into the attribute feature map to obtain a mixed feature map, and based on the attribute features and the identity features, a decoding network is used to decode the mixed feature map to obtain a face-changing image, the original image and the face-changing image are input into the discriminator for training to obtain a judgment value, the total loss is calculated according to the judgment value, the face-changing image, the original image and the target image, and the parameters of the generator and the discriminator are adjusted according to the total loss, so that the mixed feature map is decoded in combination with the attribute features of the original image and the identity features of the target image during the decoding process, so that the face-changing image better maintains the attribute features of the original image such as facial posture and facial expression, and the identity features in the target image are also better integrated into the face-changing image, thereby enhancing the migration ability of the identity features of the target image, and when the generator of the trained generative adversarial network is used to change the face of an image or video, the obtained face-changing image or video is real and natural, and can maintain the attribute features of the original image and the identity features of the target image.

[0125] Furthermore, the total loss includes key point loss, which constrains the facial key points of the face-swapped image to be similar to the facial key points of the original image, so that the face-swapped image can better maintain the facial expression, facial posture and other attribute features of the original image.

[0126] Furthermore, the total loss includes identity feature loss, which constrains the identity features of the face-swapped image to be more similar to the identity features of the target image, so that the face-swapped image can better maintain the identity features of the target image.

[0127] Embodiment 3

[0128] Figure 3 This is a flowchart of a method for face-swapping an image provided in Embodiment 3 of the present invention. The embodiment of the present invention is applicable to the case of replacing a face in an image. The method can be performed by an image face-swapping device according to the embodiment of the present invention. The image face-swapping device can be implemented by hardware or software and integrated in an electronic device provided in the embodiment of the present invention. Specifically, Figure 3 As shown, the image face-changing method of the embodiment of the present invention may include the following steps:

[0129] S301: Acquire an original image containing a first face and a target image containing a second face.

[0130] In one example of the present invention, a user uses a second face in a target image to replace a first face in an original image, so that the image after the face-changing can maintain the identity features of the second face and the posture, expression and other attributes of the first face. In an application scenario, the original image is the image that the user needs to change the face, and the target image can be an image containing the user's face.

[0131] In an embodiment of the present invention, an interactive interface may be provided, in which operations for users to determine the original image and the target image are provided. The user can specify the original image and the target image in the interactive interface. For example, the interactive interface may provide an image upload operation, and the user can upload the original image and the target image in the interactive interface. For example, the interactive interface first prompts the user to upload the original image, and then prompts the user to upload the target image. Of course, the original image and target image specified by the user may also be obtained through other interactive operations, and the embodiment of the present invention is not limited to this.

[0132] S302: Input the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face.

[0133] In an embodiment of the present invention, the generator may be a neural network that replaces a first face in an original image with a second face in a target image. The generator may be obtained by training a generative adversarial network, wherein the generator may be trained by the generative adversarial network training method provided in Example 1 or Example 2. For specific training details, please refer to Example 1 or Example 2 and will not be described in detail here.

[0134] For a trained generator, the original image can be input into the encoding network of the generator to extract the attribute feature map of the first face, the target image can be input into the identity extraction network of the generator to extract the identity features of the second face, and the identity features can be injected into the attribute feature map through the residual network of the generator to generate a mixed feature map. The encoding network of the generator decodes the mixed feature map according to the identity features and the attribute feature map to obtain the face-changed image after the second face replaces the first face.

[0135] After the embodiment of the present invention obtains an original image containing a first face and a target image containing a second face, the original image and the target image are input into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face. The generator of the embodiment of the present invention realizes decoding of the mixed feature map by combining the attribute features of the original image and the identity features of the target image during the decoding process, so that the face-swapped image better maintains the attribute features of the original image such as facial posture and facial expression. At the same time, the identity features in the target image are also better integrated into the face-swapped image, thereby enhancing the migration ability of the identity features of the target image. When the trained generator of the generative adversarial network is used to swap the image, the obtained face-swapped image is real and natural, and can maintain the attribute features of the original image and the identity features of the target image.

[0136] Embodiment 4

[0137] Figure 4 This is a flowchart of a video face-changing method provided in Embodiment 4 of the present invention. The embodiment of the present invention is applicable to the case of replacing a face in a video. The method can be performed by a video face-changing device in the embodiment of the present invention. The video face-changing device can be implemented by hardware or software and integrated in the electronic device provided in the embodiment of the present invention. Specifically, Figure 4 As shown, the video face swapping method of the embodiment of the present invention may include the following steps:

[0138] S401, obtaining video data to be replaced with a face.

[0139] In an embodiment of the present invention, the video data to be replaced with a face may be short video data, live video data, movie video data, etc. containing human faces. The video data to be replaced with a face may include one or more faces. In actual applications, the user may specify the video data to be replaced with a face in the provided face-changing editing interactive interface, for example, upload the video data to be replaced with a face or input the address of the video data to be replaced with a face, which may be a local storage address of the video data to be replaced with a face, or a network address of the video data to be replaced with a face.

[0140] S402: extracting a video image including a first face from the video data as an original image.

[0141] Specifically, during the process of decoding the video data, face detection can be performed on the faces in the video data. When the first face is detected, the video image is extracted as the original image, wherein the first face can be a face specified by the user. For example, when acquiring the video data, the user can be prompted to specify a face in the video data as the first face.

[0142] S403: Acquire a target image containing a second face.

[0143] In an embodiment of the present invention, the target image is an image used to replace the face in the original image, and the target image includes a second face. Exemplarily, the target image can be a selfie of the user, and of course it can also be other images specified by the user that include the second face. Preferably, the first face and the second face are different faces.

[0144] S404: Input the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face.

[0145] In an embodiment of the present invention, the generator may be a neural network that replaces a first face in an original image with a second face in a target image. The generator may be obtained by training a generative adversarial network, wherein the generator may be trained by the generative adversarial network training method provided in Example 1 or Example 2. For specific training details, please refer to Example 1 or Example 2 and will not be described in detail here.

[0146] For a trained generator, the original image can be input into the encoding network of the generator to extract the attribute feature map of the first face, the target image can be input into the identity extraction network of the generator to extract the identity features of the second face, and the identity features can be injected into the attribute feature map through the residual network of the generator to generate a mixed feature map. The encoding network of the generator decodes the mixed feature map according to the identity features and the attribute feature map to obtain the face-changed image after the second face replaces the first face.

[0147] S405: Generate face-swapped video data based on the face-swapped image.

[0148] After obtaining the face-swapped image of the original image, video encoding can be performed on the face-swapped images of each original image according to the preset frame rate and bit rate to obtain the video data after the face-swapped face. The face in the video data after the face-swapped face retains the identity characteristics of the second face and the attribute characteristics such as posture and expression of the first face.

[0149] After obtaining the video data to be face-changed, the embodiment of the present invention extracts the original image containing the first face from the video data, obtains the target image containing the second face, inputs the original image and the target image into the generator of the generative adversarial network, obtains the face-changed image of the original image after replacing the first face with the second face, and generates the face-changed video data based on the face-changed image. The generator of the embodiment of the present invention realizes decoding of the mixed feature map in combination with the attribute features of the original image and the identity features of the target image during the decoding process, so that the face-changed image better maintains the attribute features of the original image such as the face posture and facial expression, and at the same time, the identity features in the target image are better integrated into the face-changed image, enhancing the migration ability of the identity features of the target image. When the trained generator of the generative adversarial network is used to change the face of the video data, the obtained face-changed video data is real and natural, and can maintain the attribute features of the face in the video data and the identity features of the face in the target image.

[0150] Embodiment 5

[0151] Figure 5 is a structural block diagram of a generative adversarial network training device provided in Embodiment 5 of the present invention, such as Figure 5 As shown, the generative adversarial network training device of the embodiment of the present invention may specifically include the following modules:

[0152] The original image and target image acquisition module 501 is used to acquire the original image containing the first face and the target image containing the second face;

[0153] Generative adversarial network initialization module 502, used to initialize the generator and discriminator of the generative adversarial network;

[0154] A generator training module 503 is used to input the original image and the target image into the generator for training to obtain a face-swapped image, wherein the generator is used to extract an attribute feature map of the first face from the original image, extract an identity feature of the second face from the target image, inject the identity feature into the attribute feature map to generate a mixed feature map, and decode the mixed feature map according to the identity feature and the attribute feature map to obtain a face-swapped image in which the second face replaces the first face;

[0155] A discriminator training module 504 is used to train the discriminator using the original image and the face-swapped image to obtain a determination value;

[0156] The parameter adjustment module 505 is used to adjust the parameters of the generator and the discriminator according to the judgment value, the face-changing image, the original image and the target image.

[0157] The generative adversarial network training device provided in the embodiment of the present invention can execute the generative adversarial network training method provided in the first and second embodiments of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0158] Embodiment 6

[0159] Figure 6 is a structural block diagram of an image face-changing device provided by Embodiment 6 of the present invention. Figure 6 As shown, the image face-changing device of the embodiment of the present invention may specifically include the following modules:

[0160] The original image and target image acquisition module 601 is used to acquire the original image containing the first face and the target image containing the second face;

[0161] An image face-swapping module 602 is used to input the original image and the target image into a generator of a generative adversarial network to obtain a face-swapping image of the original image after replacing the first face with the second face;

[0162] Wherein, the generator is trained by the generative adversarial network training method described in Example 1 or Example 2 of the present invention.

[0163] The image face-changing device provided in the embodiment of the present invention can execute the image face-changing method provided in the third embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0164] Embodiment 7

[0165] Figure 7 is a structural block diagram of a video face-changing device provided by Embodiment 7 of the present invention, such as Figure 7 As shown, the video face-changing device of the embodiment of the present invention may specifically include the following modules:

[0166] The module 701 for acquiring the video data to be replaced with a face is used to acquire the video data to be replaced with a face;

[0167] An original image extraction module 702 is used to extract a video image containing a first face from the video data as an original image;

[0168] The target image acquisition module 703 is used to acquire a target image containing a second face;

[0169] A video face-swapping module 704 is used to input the original image and the target image into a generator of a generative adversarial network to obtain a face-swapping image of the original image after replacing the first face with the second face;

[0170] A face-swapped video data generating module 705 is used to generate face-swapped video data based on the face-swapped image;

[0171] Wherein, the generator is trained by the generative adversarial network training method described in Example 1 or Example 2 of the present invention.

[0172] The video face-changing device provided in the embodiment of the present invention can execute the video face-changing method provided in the fourth embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0173] Embodiment 8

[0174] Reference Figure 8 , shows a schematic diagram of the structure of an electronic device in an example of the present invention. Figure 8 As shown, the electronic device may specifically include: a processor 801, a storage device 802, a display screen 803 with a touch function, an input device 804, an output device 805, and a communication device 806. The number of processors 801 in the device may be one or more. Figure 8 A processor 801 is taken as an example. The processor 801, storage device 802, display screen 803, input device 804, output device 805 and communication device 806 of the device can be connected through a bus or other means. Figure 8 The device is used to execute the generative adversarial network training method, and / or the image face-changing method, and / or the video face-changing method as provided in any embodiment of the present invention.

[0175] An embodiment of the present invention also provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the generative adversarial network training method, and / or the image face-changing method, and / or the video face-changing method as described in the above method embodiments.

[0176] It should be noted that, for the device, electronic device, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0177] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0178] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A generative adversarial network training method, characterized in that: include: Obtaining an original image containing a first face and a target image containing a second face; Initialize the parameters of the discriminator of the generative adversarial network, the parameters of the encoding network and the decoding network of the generator, and obtain the trained residual network and identity extraction network used in the generator; The original image and the target image are input into the generator for training to obtain a face-swapped image, wherein the generator is used to extract an attribute feature map of the first face from the original image, extract an identity feature of the second face from the target image, inject the identity feature into the attribute feature map to generate a mixed feature map, and decode the mixed feature map using the decoding network according to the identity feature and the attribute feature map to obtain a face-swapped image in which the second face replaces the first face; Inputting the original image and the face-swapped image into the discriminator for training to obtain a judgment value; Adjusting parameters of the generator and the discriminator according to the determination value, the face-changing image, the original image, and the target image; The generator is used to extract the attribute feature map of the first face from the original image, including: using the encoding network to encode the original image to obtain the attribute feature map of the first face; The encoding network includes multiple layers of down-sampling convolutional layers, and the decoding network includes multiple layers of up-sampling convolutional layers. The mixed feature map is decoded by the decoding network according to the identity feature and the attribute feature map to obtain a face-changing image after the first face is replaced by the second face, including: splicing the mixed feature map and the identity feature to obtain a spliced ​​feature; inputting the spliced ​​feature into the decoding network to obtain a face-changing image through sampling processing through multiple layers of up-sampling convolutional layers; for each up-sampling convolutional layer in the decoding network, the down-sampling convolutional layer corresponding to the up-sampling convolutional layer in the encoding network is determined, the down-sampling feature map output by the down-sampling convolutional layer is obtained, and the up-sampling feature output by the previous up-sampling convolutional layer of the up-sampling convolutional layer is obtained, and the down-sampling feature map and the up-sampling feature are spliced ​​as the decoding object of the up-sampling convolutional layer.

2. The method according to claim 1, characterized in that: The original image and the target image are input into the generator for training to obtain a face-swapped image, including: Inputting the target image into the identity extraction network to extract the identity features of the second face; Using the residual network to inject the identity feature into the attribute feature map to obtain a mixed feature map; Based on the attribute feature map and the identity feature, the mixed feature map is decoded using the decoding network to obtain a face-changing image after the first face is replaced by the second face.

3. The method according to claim 1, characterized in that: The encoding network includes multiple layers of downsampling convolutional layers, and the encoding network is used to encode the original image to obtain the attribute feature map of the first face, including: Preprocessing the original image to obtain a preprocessed original image; Inputting the preprocessed original image into the encoding network to obtain a downsampled feature map output by each downsampled convolutional layer; The preprocessing includes adjusting the image size, and the down-sampled feature map output by the last down-sampling convolutional layer of the encoding network is the attribute feature map of the first face.

4. The method according to claim 2, characterized in that: The adopting the residual network to inject the identity feature into the attribute feature map to obtain a mixed feature map includes: Transforming the identity feature to obtain an identity feature mean and an identity feature variance of the identity feature; The identity feature mean, the identity feature variance and the attribute feature map are input into the residual network, so as to transfer the identity feature to the attribute feature map through the residual network to obtain a mixed feature map.

5. The method according to claim 1, characterized in that The adjusting the parameters of the generator and the discriminator according to the determination value, the face-changing image, the original image and the target image comprises: Calculate the total loss according to the determination value, the face-swapped image, the original image and the target image; Parameters of the generator and the discriminator are adjusted according to the total loss.

6. The method according to claim 5, characterized in that The calculating the total loss according to the determination value, the face-changing image, the original image and the target image comprises: Calculate adversarial loss and key point loss according to the judgment value, the original image and the face-swapped image; Calculating identity feature loss based on the target image and the face-swapped image; Inputting the two original images into the generator to obtain a self-swapping image of the original image; Calculating a reconstruction loss based on the original image and the self-face-swapped image; The total loss is obtained by calculating the sum of the adversarial loss, the reconstruction loss, the key point loss and the identity feature loss.

7. The method according to claim 6, characterized in that The calculating the adversarial loss and the key point loss according to the determination value, the original image and the face-changing image comprises: Calculate the adversarial loss according to the determination value and a preset adversarial loss function; Obtaining key points of the human face in the original image and the face-swapped image; The distance between the key points of the face in the original image and the face-swapped image is calculated to obtain the key point loss.

8. The method according to claim 6, characterized in that The calculating the identity feature loss according to the target image and the face-swapped image includes: Inputting the target image and the face-swapped image into the identity extraction network respectively to obtain the identity features of the face in the target image and the identity features of the face in the face-swapped image; The distance between the identity feature of the face in the target image and the identity feature of the face in the face-swapped image is calculated to obtain the identity feature loss.

9. The method according to claim 6, characterized in that The adjusting the parameters of the generator and the discriminator according to the total loss includes: Determining whether the total loss is less than a preset threshold; If yes, stop training the generator and the discriminator; If not, adjust the parameters of the discriminator according to the adversarial loss, and adjust the parameters of the encoder and decoder in the generator according to the total loss, and return to the step of using the encoding network to encode the original image to obtain the attribute feature map of the first face.

10. An image face-swapping method, characterized in that: include: Obtaining an original image containing a first face and a target image containing a second face; Inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face; Wherein, the generator is trained by the generative adversarial network training method described in any one of claims 1-9.

11. A video face-changing method, characterized in that: include: Get the video data to be replaced; Extracting a video image containing the first face from the video data as an original image; Obtain a target image containing a second face; Inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapped image of the original image after replacing the first face with the second face; generating face-swapped video data based on the face-swapped image; Wherein, the generator is trained by the generative adversarial network training method described in any one of claims 1-9.

12. A generative adversarial network training device, characterized in that: include: The original image and target image acquisition module is used to acquire the original image containing the first face and the target image containing the second face; A generative adversarial network initialization module, used to initialize the parameters of the discriminator of the generative adversarial network, the parameters of the encoding network and the decoding network of the generator, and to obtain the trained residual network and identity extraction network used in the generator; a generator training module, configured to input the original image and the target image into the generator for training to obtain a face-swapped image, wherein the generator is configured to extract an attribute feature map of the first face from the original image, extract an identity feature of the second face from the target image, inject the identity feature into the attribute feature map to generate a mixed feature map, and decode the mixed feature map using the decoding network according to the identity feature and the attribute feature map to obtain a face-swapped image in which the second face replaces the first face; A discriminator training module, used for training the discriminator using the original image and the face-swapped image to obtain a judgment value; A parameter adjustment module, used for adjusting the parameters of the generator and the discriminator according to the determination value, the face-changing image, the original image and the target image; The generator is used to extract the attribute feature map of the first face from the original image, including: using the encoding network to encode the original image to obtain the attribute feature map of the first face; The encoding network includes multiple layers of down-sampling convolutional layers, and the decoding network includes multiple layers of up-sampling convolutional layers. The mixed feature map is decoded by the decoding network according to the identity feature and the attribute feature map to obtain a face-changing image after the first face is replaced by the second face, including: splicing the mixed feature map and the identity feature to obtain a spliced ​​feature; inputting the spliced ​​feature into the decoding network to obtain a face-changing image through sampling processing through multiple layers of up-sampling convolutional layers; for each up-sampling convolutional layer in the decoding network, the down-sampling convolutional layer corresponding to the up-sampling convolutional layer in the encoding network is determined, the down-sampling feature map output by the down-sampling convolutional layer is obtained, and the up-sampling feature output by the previous up-sampling convolutional layer of the up-sampling convolutional layer is obtained, and the down-sampling feature map and the up-sampling feature are spliced ​​as the decoding object of the up-sampling convolutional layer.

13. An image face-changing device, characterized in that: include: The original image and target image acquisition module is used to acquire the original image containing the first face and the target image containing the second face; An image face-swapping module, used for inputting the original image and the target image into a generator of a generative adversarial network, and obtaining a face-swapping image of the original image after replacing the first face with the second face; Wherein, the generator is trained by the generative adversarial network training method described in any one of claims 1-9.

14. A video face-changing device, characterized in that: include: The module for acquiring the video data to be replaced is used to acquire the video data to be replaced; An original image extraction module, used for extracting a video image containing a first face from the video data as an original image; A target image acquisition module, used to acquire a target image containing a second face; A video face-swapping module, used for inputting the original image and the target image into a generator of a generative adversarial network to obtain a face-swapping image of the original image after replacing the first face with the second face; A face-swap video data generation module, used to generate face-swap video data based on the face-swap image; Wherein, the generator is trained by the generative adversarial network training method described in any one of claims 1-9.

15. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the generative adversarial network training method as described in any one of claims 1 to 9, and / or the image face-changing method as described in claim 10, and / or the video face-changing method as described in claim 11.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the generative adversarial network training method as described in any one of claims 1 to 9, and / or the image face-changing method as described in claim 10, and / or the video face-changing method as described in claim 11.

Images