Identity-preserving face reenactment method, apparatus, device and storage medium

Abstract

The present application relates to a kind of identity keeping face replay method, device, equipment and storage medium, the method comprises: initial video data is sampled to obtain data training set and data test set;Based on preset training method, according to the data training set, initial face replay model is trained to obtain transition face replay model;According to the data test set, the transition face replay model is tested, and test result is evaluated, and target face replay model is screened;Face image to be replayed is input to the target face replay model and is replayed, and identity keeping face replay image is output to obtain.The present application provides a kind of identity keeping face replay method, device, equipment and storage medium, by data training set and data test set, face replay model is trained and optimized, so that face replay model can obtain identity keeping face replay image, avoid the problem of identity leakage in image generation process.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and image processing technology, and in particular to a method, apparatus, device and storage medium for face reconstruction with identity preservation. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, significant progress has been made in the field of image generation. Face re-enactment is an important branch of face image generation. Its main task is to generate a new image given a source image and a driving image. This new image should retain motion information such as facial expressions and head pose from the driving image, while preserving content information such as the identity of the person and the background of the source image. Face re-enactment technology has broad application prospects and can be applied to scenarios such as image editing, entertainment, human-computer interaction, and cultural education.

[0003] Currently, face reconstruction models often employ network structures based on flow field deformation. These network structures learn the pixel motion of the face by learning the optical flow or displacement field of the face. The motion flow field is extracted through the optical flow or displacement field to reconstruct the face.

[0004] However, existing technologies do not consider the identity information of people when extracting the moving flow field through the network structure of flow field deformation, which leads to the problem of identity leakage in the image generation process, resulting in a certain loss of the identity information of people in the generated image. Summary of the Invention

[0005] In view of this, it is necessary to provide a face re-enactment method, apparatus, device and storage medium that preserves identity, in order to solve the problem that the existing technology does not consider the identity information of the person in the process of extracting the motion flow field, resulting in identity leakage in the image generation process and causing a certain loss of the identity information of the person in the generated image.

[0006] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a face reenactment method that preserves identity, comprising:

[0008] The initial video data is sampled to obtain the training set and the test set;

[0009] Based on a pre-defined training method, an intermediate face replay model is obtained by training the initial face replay model using the data training set.

[0010] The transitional face replay model was tested based on the test data set, and the test results were evaluated to select the target face replay model.

[0011] The face image to be recreated is input into the target face recreating model to perform face recreating, and the output is an identity-preserving face recreated image.

[0012] In some possible implementations, an intermediate face replay model is obtained by training an initial face replay model on a training data set based on a preset training method, including:

[0013] The first stage of training of the initial face replay model is carried out by selecting feature images of the same person from the data training set.

[0014] Based on the data training set, feature images of different people are selected to conduct a second stage of training on the initial face replay model after the first stage of training, resulting in a transitional face replay model.

[0015] In some possible implementations, the initial face reconstruction model is trained in the first stage by selecting feature images of the same person from the training data set, including:

[0016] Two different facial images of the same person are randomly selected from the training data set as the first image;

[0017] The first image is input into the initial face reconstruction model, and the first loss function is calculated.

[0018] Repeat the above steps until the first loss function converges.

[0019] In some possible implementations, feature images of different individuals are selected from the training data set to perform a second-stage training on the initial face reconstruction model trained in the first stage, resulting in a transitional face reconstruction model, including:

[0020] From the training dataset, one frame of facial image is randomly selected from each of two different individuals as the second image;

[0021] The second image is input into the initial face replay model trained in the first stage, and the second loss function is calculated.

[0022] Repeat the above steps until the second loss function converges.

[0023] In some possible implementations, the transitional face replay model is tested against a data test set, and the test results are evaluated to select the target face replay model, including:

[0024] Two frames of images from different data sets were randomly selected as test images.

[0025] The test image is input into the transitional face replay model to obtain the test replay image;

[0026] Calculate the preset evaluation index based on the test image and the test replay image;

[0027] Based on preset evaluation indicators, the test results are evaluated and selected to obtain the target face reconstruction model.

[0028] In some possible implementations, the test image includes a driving image and a source image; preset evaluation metrics are calculated based on the test image and the test replay image, including:

[0029] Calculate the similarity of facial expressions between the source image and the test replay image;

[0030] Calculate face pose similarity based on the driving image and the test replay image;

[0031] The identity information retention status is calculated based on the source image and the test replay image.

[0032] In some possible implementations, the initial video data is sampled to obtain a training set and a test set, including:

[0033] The initial video data is preprocessed, and the preprocessed initial video data is divided into a training set and a test set according to a preset ratio.

[0034] Secondly, the present invention also provides an identity-preserving face reenactment device, comprising:

[0035] The sampling module is used to sample the initial video data to obtain the training set and the test set.

[0036] The training module is used to train the initial face replay model based on a preset training method and a data training set to obtain a transitional face replay model.

[0037] The testing module is used to test the transitional face replay model based on the data test set, evaluate the test results, and select the target face replay model.

[0038] The replay module is used to input the face image to be replayed into the target face replay model for face replay and output the identity-preserving face replay image.

[0039] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein,

[0040] Memory, used to store programs;

[0041] The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the identity-preserving face reenactment method in any of the above implementations.

[0042] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in the identity-preserving face reenactment method in any of the above implementations.

[0043] The beneficial effects of the above embodiments are as follows: This invention relates to an identity-preserving face replay method, apparatus, device, and storage medium. The method includes: sampling initial video data to obtain a data training set and a data test set; training an initial face replay model based on a preset training method and the data training set to obtain a transitional face replay model; testing the transitional face replay model based on the data test set, evaluating the test results, and selecting a target face replay model; inputting the face image to be replayed into the target face replay model for face replay, and outputting an identity-preserving face replay image. The identity-preserving face replay method, apparatus, device, and storage medium provided by this invention trains and optimizes the face replay model using a data training set and a data test set, enabling the face replay model to obtain identity-preserving face replay images and avoiding identity leakage during image generation. Attached Figure Description

[0044] Figure 1 A flowchart illustrating an embodiment of the face replay method for identity preservation provided by the present invention;

[0045] Figure 2 A flowchart illustrating an embodiment of the first-stage training provided by the present invention;

[0046] Figure 3 A flowchart illustrating an embodiment of the second-stage training provided by the present invention;

[0047] Figure 4 for Figure 1 A schematic flowchart of an embodiment of step S103;

[0048] Figure 5 A schematic diagram of a structure of an embodiment of the face replay device for identity preservation provided by the present invention;

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0051] In the description of this application, "multiple" means two or more, unless otherwise expressly and specifically defined.

[0052] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] This invention provides a method, apparatus, device, and storage medium for face reenactment with identity preservation, which will be described below.

[0054] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the identity-preserving face replay method provided by the present invention. A specific embodiment of the present invention discloses an identity-preserving face replay method, comprising:

[0055] S101. Sample the initial video data to obtain a training set and a test set;

[0056] S102. Based on the preset training method, the initial face replay model is trained according to the data training set to obtain the transition face replay model.

[0057] S103. Test the transitional face replay model based on the data test set, evaluate the test results, and select the target face replay model.

[0058] S104. Input the face image to be recreated into the target face recreating model to perform face recreating, and output the identity-preserving face recreated image.

[0059] In the above steps, the initial video data can be obtained through samples. In order to ensure that the data training set and data test set are rich enough, the content of the initial video data should be sufficient. In this invention, a long video can be selected as the initial video data, or a combination of multiple video segments can be selected as the initial video data. This invention does not impose further restrictions on this, only requiring that a sufficient number of data training sets and data test sets can be obtained.

[0060] The preset training method is a course-based learning method, which consists of two stages of training. Different loss functions are used in the two stages of training. Through the two-stage training, a transitional face replay model that can identify the same person and different people is obtained.

[0061] The transitional face replay model was tested using a data test set. The replay images obtained from the test were evaluated, and evaluation indicators were calculated. The transitional face replay model was then selected based on the evaluation indicators to obtain models that can accurately replay faces and ensure that the identity information of the person is not leaked.

[0062] The face image to be replayed includes the source image and the driving image. The input of the face replay model is the source image and the driving image, and the output is the face replay image. The output face replay image has motion information such as facial expressions and head posture of the driving image, while retaining content information such as the identity of the person and the background of the image in the source image.

[0063] It should be noted that the face reconstruction model is a neural network model. It should be understood that the model structure of the face reconstruction model includes, but is not limited to, deep neural networks (DNN), recurrent neural networks (RNN), convolutional neural networks (CNN), deep generative models (DGM), generative adversarial networks (GAN), long short-term memory networks (LSTM), support vector machines (SVM), and deep cross-models.

[0064] Compared with existing technologies, this embodiment provides an identity-preserving face replay method. The method includes: sampling initial video data to obtain a training set and a test set; training an initial face replay model using the training set based on a preset training method to obtain a transitional face replay model; testing the transitional face replay model using the test set, evaluating the test results, and selecting a target face replay model; inputting the face image to be replayed into the target face replay model for face replay, and outputting an identity-preserving face replay image. This invention provides an identity-preserving face replay method, apparatus, device, and storage medium that trains and optimizes the face replay model using a training set and a test set, enabling the face replay model to obtain identity-preserving face replay images and avoiding identity leakage during image generation.

[0065] In some embodiments of the present invention, an initial face replay model is trained based on a preset training method and a data training set to obtain a transitional face replay model, including:

[0066] The first stage of training of the initial face replay model is carried out by selecting feature images of the same person from the data training set.

[0067] Based on the data training set, feature images of different people are selected to conduct a second stage of training on the initial face replay model after the first stage of training, resulting in a transitional face replay model.

[0068] In the above embodiment, the first stage of training is to enable the initial face replay model to recognize the same person. Therefore, two feature images of the same person need to be selected from the data training set as input to train the initial face replay model so that the face replay model can replay the face of the same person from two images.

[0069] The second stage of training is to enable the face replay model trained in the first stage to recognize different people. Therefore, two feature images of different people need to be selected from the training data set as input to train the face replay model trained in the first stage, so as to avoid the face replay model performing face replay based on the feature images of different people.

[0070] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the first-stage training provided by the present invention. In some embodiments of the present invention, the first-stage training of the initial face reconstruction model is performed by selecting feature images of the same person based on the data training set, including:

[0071] S201. Randomly select two different facial images of the same person from the training data set as the first image;

[0072] S202. Input the first image into the initial face reconstruction model and calculate the first loss function;

[0073] S203. Repeat the above steps until the first loss function converges.

[0074] In the above embodiment, a video segment is randomly selected from the data training set, the selected video segment is divided into frames, and then two frames are randomly selected as the input of the face replay network. One frame is used as the source image and the other frame is used as the driving image, ensuring that the source image and the driving image come from the same person. At this time, the source image and the driving image are collectively referred to as the first image.

[0075] The first image is input into the initial face replay network, and the first loss function is calculated. The network parameters are updated using the Adam optimization algorithm. The loss function during training is determined by the facial expression flow field loss L. e Attitude flow field loss L p Perceptual loss function L recand cycle-consistent loss function L cycle Together they form the following:

[0076]

[0077] Where, λ rec , λ cycle , λ e , λ p These are used to balance the weight parameters between different loss functions.

[0078] It should be noted that λ rec , λ cycle , λ e , λ p The settings can be configured according to actual needs, and this invention does not impose further limitations on them.

[0079] Using perceptual loss L rec As a measure of the quality of the generated image, the reconstruction loss is as follows:

[0080]

[0081] in Representing the reconstructed image and the target image respectively; N i (·) represents the feature of the i-th layer extracted by VGG-19, |·| represents the L1 norm; n indicates which layers of VGG features are used to calculate the perceptual loss. This invention uses features from five layers of the VGG network: layers 2, 7, 12, 21, and 30, for the calculation of the perceptual loss. Using this loss function, the features of the reconstructed image can be better measured, making the reconstructed image more realistic.

[0082] Repeat steps S201 and S202 until the first loss function is achieved. If convergence occurs, the first phase of training is complete.

[0083] Please see Figure 3 , Figure 3 This is a flowchart illustrating an embodiment of the second-stage training provided by the present invention. In some embodiments of the present invention, feature images of different individuals are selected from the training data set to perform a second-stage training on the initial face replay model after the first-stage training, resulting in a transitional face replay model, including:

[0084] S301. Randomly select one frame of face image from each of two different people in the data training set as the second image.

[0085] S302. Input the second image into the initial face replay model trained in the first stage, and calculate the second loss function;

[0086] S303. Repeat the above steps until the second loss function converges.

[0087] In the above embodiment, two video segments are randomly selected from the data training set, the two video segments are divided into frames, and then one frame is randomly selected from each of the two videos as the input of the initial face replay model after the first stage of training. One frame is used as the source image and the other frame is used as the driving image, ensuring that the source image and the driving image come from different people. At this time, the source image and the driving image are collectively referred to as the second image.

[0088] The second image is input into the face replay network, and the second loss function is calculated. The Adam optimization algorithm is used to update network parameters. The loss function during training is determined by the facial expression flow field loss L. e Attitude flow field loss L p and cycle-consistent loss function L cycle Together they form the following:

[0089]

[0090] Repeat steps S301 and S302 until the second loss function is achieved. If convergence occurs, the second phase of training is complete.

[0091] Please see Figure 4 , Figure 4 for Figure 1 A flowchart illustrating an embodiment of step S103. In some embodiments of the present invention, the transitional face replay model is tested based on a data test set, the test results are evaluated, and a target face replay model is selected, including:

[0092] S401. Randomly select two frames of images with different data from the data test set as test images;

[0093] S402. Input the test image into the transitional face reconstruction model to obtain the test reconstruction image;

[0094] S403. Calculate the preset evaluation index based on the test image and the test replay image;

[0095] S404. Based on preset evaluation indicators, evaluate the test results and select the target face reconstruction model.

[0096] In the above embodiment, all data test sets are combined in pairs, and any two data sets are combined into a group of data. Each group of combined data includes two video segments. One frame is randomly extracted from each of the two video segments in the combined data. One frame is used as the source image and the other frame is used as the driving image. The source image and the driving image obtained at this time are collectively referred to as the test image.

[0097] The test image is input into the transitional face replay model for face replay testing, and the transitional face replay model outputs the test replay image. Evaluation metrics are calculated using the test image and the test replay image, and the test results are then evaluated using these metrics to select the target face replay model.

[0098] It should be noted that the preset evaluation indicators include Expression Difference (ED), Pose Cosine Similarity between Image (PSIM), and Cosine Similarity between Image Embeddings (CSIM).

[0099] In some embodiments of the present invention, the test image includes a driving image and a source image; a preset evaluation index is calculated based on the test image and the test replay image, including:

[0100] Calculate the similarity of facial expressions between the source image and the test replay image;

[0101] Calculate face pose similarity based on the driving image and the test replay image;

[0102] The identity information retention status is calculated based on the source image and the test replay image.

[0103] In the above embodiments, the expression difference (ED) of facial expressions is calculated using the following formula.

[0104] ED(g,s)=||E p (g)-R p (s)||2;

[0105] Where g and s represent the test replay image and the source image, respectively, E p (·) represents the facial motion unit obtained using OpenFace. The distance between the AU information obtained from the two images is calculated using the L2 norm to measure the difference between facial expressions in the two face images. The smaller the ED value, the smaller the difference between the facial expressions in the two face images, the more similar the expressions are, and the better the replay effect.

[0106] Pose Cosine Similarity between Image (PSIM): The calculation formula is shown below.

[0107] PSIM(g,s)=1-cos<OF(g),OF(t)> ;

[0108] Where t represents the driving image, and OF(·) represents the face pose information obtained using the pose estimation method in OpenFace. That is, the offset angles of the face along the X, Y, and Z axes. Then, the similarity between the poses of the test replay image and the driving image is calculated using cosine distance. The smaller the value, the better the test replay image preserves the pose information of the driving image, meaning it achieves a better pose replay effect.

[0109] The identity preservation performance (CSIM) of the test replay image is calculated as follows:

[0110] CSIM(g,s)=1-cos<FR(g),FR(s)> ;

[0111] Wherein, FR(·) represents the face recognition network, and the CSIM metric describes the distance between the feature vector of the test replay image and the feature vector of the given source person image. The smaller the CSIM value, the closer the feature vectors of the test replay image and the source person image are, indicating that the model retains the person's identity information more completely.

[0112] In some embodiments of the present invention, initial video data is sampled to obtain a data training set and a data test set, including:

[0113] The initial video data is preprocessed, and the preprocessed initial video data is divided into a training set and a test set according to a preset ratio.

[0114] In the above embodiments, the preprocessing of the initial video data includes: filtering out video sequences with low resolution from the initial video data, then adjusting the size of the filtered initial video data to 256×256, and then enhancing the data by randomly flipping, adjusting the brightness, saturation, contrast, etc.

[0115] The preprocessed video data is divided into training and testing datasets according to a certain ratio. It should be noted that the specific ratio can be set according to the actual situation, and this invention does not impose further restrictions on it.

[0116] To better implement the identity-preserving face reenactment method in the embodiments of the present invention, based on the identity-preserving face reenactment method, please refer to the corresponding... Figure 5 , Figure 5 This is a schematic diagram of a structure of an embodiment of the identity-preserving face replay device provided by the present invention. The embodiment of the present invention provides an identity-preserving face replay device 500, comprising:

[0117] The sampling module 510 is used to sample the initial video data to obtain a training set and a test set.

[0118] Training module 520 is used to train the initial face replay model based on a preset training method and a data training set to obtain a transitional face replay model.

[0119] The test module 530 is used to test the transitional face replay model based on the data test set, evaluate the test results, and select the target face replay model.

[0120] The replay module 540 is used to input the face image to be replayed into the target face replay model for face replay and output the identity-preserving face replay image.

[0121] It should be noted that the device 500 provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding content in the above method embodiments, and will not be repeated here.

[0122] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Based on the above-described identity-preserving face replay method, the present invention also provides an identity-preserving face replay device, which can be a computing device such as a mobile terminal, desktop computer, laptop, handheld computer, or server. The identity-preserving face replay device includes a processor 610, a memory 620, and a display 630. Figure 6 Only some components of the electronic device are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0123] In some embodiments, memory 620 may be an internal storage unit of the identity-preserving face replay device, such as a hard drive or memory of the identity-preserving face replay device. In other embodiments, memory 620 may be an external storage device of the identity-preserving face replay device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the identity-preserving face replay device. Further, memory 620 may include both internal and external storage units of the identity-preserving face replay device. Memory 620 is used to store application software and various types of data installed on the identity-preserving face replay device, such as program code for the identity-preserving face replay device. Memory 620 may also be used to temporarily store data that has been output or will be output. In one embodiment, memory 620 stores an identity-preserving face replay program 640, which can be executed by processor 610 to implement the identity-preserving face replay method of the various embodiments of this application.

[0124] In some embodiments, processor 610 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 620 or process data, such as performing an identity-preserving face replay method.

[0125] In some embodiments, display 630 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 630 is used to display information from the identity-preserving face replay device and to display a visual user interface. Components 610-630 of the identity-preserving face replay device communicate with each other via a system bus.

[0126] In one embodiment, the steps of the identity-preserving face replay method described above are implemented when the processor 610 executes the identity-preserving face replay program 640 in the memory 620.

[0127] This embodiment also provides a computer-readable storage medium storing an identity-preserving face replay program, which, when executed by a processor, performs the following steps:

[0128] The initial video data is sampled to obtain the training set and the test set;

[0129] Based on a pre-defined training method, an intermediate face replay model is obtained by training the initial face replay model using the data training set.

[0130] The transitional face replay model was tested based on the test data set, and the test results were evaluated to select the target face replay model.

[0131] The face image to be recreated is input into the target face recreating model to perform face recreating, and the output is an identity-preserving face recreated image.

[0132] In summary, this embodiment provides an identity-preserving face replay method, apparatus, device, and storage medium. The method includes: sampling initial video data to obtain a data training set and a data test set; training an initial face replay model based on a preset training method using the data training set to obtain a transitional face replay model; testing the transitional face replay model using the data test set, evaluating the test results, and selecting a target face replay model; inputting the face image to be replayed into the target face replay model for face replay, and outputting an identity-preserving face replay image. This invention provides an identity-preserving face replay method, apparatus, device, and storage medium that trains and optimizes the face replay model using a data training set and a data test set, enabling the face replay model to obtain identity-preserving face replay images and avoiding identity leakage during image generation.

[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A face reenactment method that preserves identity, characterized in that, include: The initial video data is sampled to obtain the training set and the test set; Based on a preset training method, an initial face replay model is trained using the training data set to obtain a transitional face replay model. This includes: selecting feature images of the same person from the training data set to perform a first-stage training on the initial face replay model; and selecting feature images of different people from the training data set to perform a second-stage training on the initial face replay model after the first-stage training to obtain the transitional face replay model. The preset training method is a course-based learning method, which consists of two stages of training, each using a different loss function. The transitional face replay model is tested based on the data test set, and the test results are evaluated to select the target face replay model. The face image to be recreated is input into the target face recreating model to perform face recreating, and the output is an identity-preserving face recreated image.

2. The face re-enactment method for maintaining identity according to claim 1, characterized in that, The step of selecting feature images of the same person from the training data set to perform the first stage training of the initial face reconstruction model includes: Two different facial images of the same person are randomly selected from the training data set as the first image; The first image is input into the initial face reconstruction model to calculate the first loss function; Repeat the above steps until the first loss function converges.

3. The face replay method for maintaining identity according to claim 1, characterized in that, The step of selecting feature images of different people from the training data set to perform a second stage of training on the initial face replay model after the first stage of training, to obtain a transitional face replay model, includes: From the training data set, one frame of facial image is randomly selected from each of two different people as the second image; The second image is input into the initial face replay model trained in the first stage, and the second loss function is calculated. Repeat the above steps until the second loss function converges.

4. The face replay method for maintaining identity according to claim 1, characterized in that, The step of testing the transitional face replay model based on the data test set, evaluating the test results, and selecting the target face replay model includes: Two frames of images with different data are randomly selected from the data test set as test images; The test image is input into the transitional face replay model to obtain the test replay image; Calculate the preset evaluation index based on the test image and the test replay image; Based on the preset evaluation indicators, the test results are evaluated and the target face reconstruction model is obtained.

5. The face replay method for maintaining identity according to claim 4, characterized in that, The test image includes a driving image and a source image; the calculation of preset evaluation indicators based on the test image and the test replay image includes: Calculate the facial expression similarity of the person based on the source image and the test replay image; Calculate face pose similarity based on the driving image and the test replay image; The identity information retention status is calculated based on the source image and the test replay image.

6. The face replay method for maintaining identity according to claim 1, characterized in that, The process of sampling the initial video data to obtain the training set and test set includes: The initial video data is preprocessed, and the preprocessed initial video data is divided into a training set and a test set according to a preset ratio.

7. A face replay device for identity preservation, characterized in that, include: The sampling module is used to sample the initial video data to obtain the training set and the test set. The training module is used to train an initial face replay model to obtain a transitional face replay model based on the training data set using a preset training method. This includes: performing a first-stage training on the initial face replay model using feature images of the same person selected from the training data set; and performing a second-stage training on the initial face replay model after the first-stage training using feature images of different people selected from the training data set, to obtain the transitional face replay model. The preset training method is a course-based learning method, which consists of two stages of training, each using a different loss function. The testing module is used to test the transitional face replay model according to the data test set, evaluate the test results, and select the target face replay model. The replay module is used to input the face image to be replayed into the target face replay model for face replay and output an identity-preserving face replay image.

8. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the identity-preserving face reenactment method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, are capable of implementing the steps in the identity-preserving face reenactment method according to any one of claims 1 to 6.

Images