Face-swapping Identification Model Training Method and Face-swapping Video Identification Method

Abstract

The present application discloses a method for training a face-swapping discrimination model and a method for discriminating face-swapping videos. The training method includes: obtaining real videos and face-swapping videos; magnifying the physiological features in the real videos and the face-swapping videos; generating a first training sample set according to the real videos and the magnified real videos; generating a second training sample set according to the face-swapping videos and the magnified face-swapping videos; selecting anchor samples, positive samples, and negative samples from the first training sample set and the second training sample set and inputting them into the face-swapping discrimination model to obtain the representation vectors corresponding to each sample; calculating the positive cosine distance and the negative cosine distance; adjusting the parameters of the face-swapping discrimination model according to the positive cosine distance and the negative cosine distance until the positive cosine distance is lower than the first threshold and the negative cosine distance is higher than the second threshold, thereby obtaining the trained face-swapping discrimination model. It can be seen that the present application can use the representation vectors output by the face-swapping discrimination model to discriminate the authenticity of videos.

Description

Technical Field

[0001] This application relates to the field of Internet technologies, and more specifically, to a method for training a face-swapping discrimination model and a method for discriminating face-swapped videos. Background Art

[0002] The abuse of Deepfake technology has brought harm to society. Many lawbreakers use Deepfake technology to swap the faces of people in videos and create incidents with the face-swapped videos, leading to widespread rumors. In this situation, proving that the video causing the incident is a face-swapped video, that is, proving that the video causing the incident is a fake video, has become the only option for rumor refutation. Therefore, how to prove or discriminate face-swapped videos has received extensive attention.

[0003] Based on this, to solve the above problems, a discrimination model capable of discriminating face-swapped videos can be introduced to identify videos. Among them, how to introduce a discrimination model capable of discriminating face-swapped videos has become the key to solving the above problems. Therefore, there is an urgent need for a method for training a face-swapping discrimination model, which can be trained on a video training data set and can realize the discrimination of the authenticity of videos. Summary of the Invention

[0004] In view of this, this application provides a method for training a face-swapping discrimination model and a method for discriminating face-swapped videos, which are used to realize the discrimination of the authenticity of videos.

[0005] To achieve the above object, the following solutions are proposed:

[0006] A method for training a face-swapping discrimination model includes:

[0007] Obtain real videos and face-swapped videos;

[0008] Amplify the physiological features in the real videos and the face-swapped videos to obtain amplified real videos and amplified face-swapped videos;

[0009] Generate a first facial pixel change map according to each real video and the amplified real video corresponding to the real video, and multiple first facial pixel change maps form a first training sample set;

[0010] Generate a second facial pixel change map according to each face-swapped video and the amplified face-swapped video corresponding to the face-swapped video, and multiple second facial pixel change maps form a second training sample set;

[0011] Select three samples from the first training sample set and the second training sample set. Among them, select one sample as the anchor sample and the other sample as the positive example sample from two samples from the same training sample set, and the sample from the other training sample set among the three samples is used as the negative example sample;

[0012] Input the anchor sample, the positive example sample, and the negative example sample into the face swapping discrimination model to obtain the representation vector corresponding to each sample output by the face swapping discrimination model;

[0013] Calculate the positive cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample;

[0014] Determine the loss of the face swapping discrimination model according to the positive cosine distance and the negative cosine distance, and adjust the parameters of the face swapping discrimination model based on the loss until the model training end condition is reached, so as to obtain the trained face swapping discrimination model.

[0015] Preferably, the generating the first facial pixel change map according to each real video and the magnified real video corresponding to the real video includes:

[0016] Extract the first facial feature points on each real video and the magnified real video corresponding to the real video;

[0017] Divide the first facial area according to the first facial feature points;

[0018] Generate the first facial pixel change map recording the pixel change of each first facial area according to each real video and the magnified real video corresponding to the real video;

[0019] The generating the second facial pixel change map according to each face swapped video and the magnified face swapped video corresponding to the face swapped video includes:

[0020] Extract the second facial feature points on each face swapped video and the magnified face swapped video corresponding to the face swapped video;

[0021] Divide the second facial area according to the second facial feature points;

[0022] Generate the second facial pixel change map recording the pixel change of each second facial area according to each face swapped video and the magnified face swapped video corresponding to the face swapped video.

[0023] Preferably, the generating the first facial pixel change map recording the pixel change of each first facial area according to each real video and the magnified real video corresponding to the real video includes:

[0024] Based on each real video and the magnified real video corresponding to the real video, generate a first brightness change map recording the brightness change of each first facial area, a first color change map recording the color change of each first facial area, and a first saturation change map recording the saturation change of each first facial area. Among them, the first brightness change map, the first color change map, and the first saturation change map corresponding to one real video form a first facial pixel change map.

[0025] Preferably, the generating a second facial pixel change map recording the pixel change of each second facial area according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video includes:

[0026] Based on each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, generate a second brightness change map recording the brightness change of each second facial area, a second color change map recording the color change of each second facial area, and a second saturation change map recording the saturation change of each second facial area. Among them, the second brightness change map, the second color change map, and the second saturation change map corresponding to one face-swapped video form a second facial pixel change map.

[0027] Preferably, the face-swapping discrimination model is a triplet model;

[0028] Input the anchor sample, the positive sample, and the negative sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model, including:

[0029] Input the anchor sample, the positive sample, and the negative sample into the triplet model at the same time to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model.

[0030] Preferably, the face-swapping discrimination model is a siamese model with shared parameters;

[0031] Input the anchor sample, the positive sample, and the negative sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model, including:

[0032] Input the anchor sample and the positive sample into the siamese model with shared parameters;

[0033] Input the anchor sample and the negative sample into the siamese model with shared parameters.

[0034] Preferably, magnifying the physiological features in the real video and the face-swapped video includes:

[0035] Performing three Euler video magnifications on the real video and the face-swapped video;

[0036] Among them, when performing the first Euler video magnification, two Gaussian downsamplings are first performed, and the video obtained after the two Gaussian downsamplings is band-pass filtered in the time dimension to obtain the first pixel information within the human heart rate frequency range, and the first pixel information is magnified by a factor of two;

[0037] When performing the second Euler video magnification, three Gaussian downsamplings are first performed, and the video obtained after the three Gaussian downsamplings is band-pass filtered in the time dimension to obtain the second pixel information within the human heart rate frequency range, and the second pixel information is magnified by a factor of three;

[0038] When performing the third Euler video magnification, four Gaussian downsamplings are first performed, and the video obtained after the four Gaussian downsamplings is band-pass filtered in the time dimension to obtain the third pixel information within the human heart rate frequency range, and the third pixel information is magnified by a factor of five.

[0039] A method for training a face swapping discrimination model includes:

[0040] Obtain a real video and a face-swapped video;

[0041] Magnify the physiological features in the real video and the face-swapped video to obtain the magnified real video and the magnified face-swapped video;

[0042] Generate a first facial pixel change map according to each real video and the magnified real video corresponding to the real video;

[0043] Label the first facial pixel change map as true, and multiple labeled first facial pixel change maps form a first training sample set;

[0044] Generate a second facial pixel change map according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video;

[0045] Label the second facial pixel change map as false, and multiple labeled second facial pixel change maps form a second training sample set;

[0046] Select three samples from the first training sample set and the second training sample set. Among them, select one sample as the anchor sample and another sample as the positive example sample from two samples from the same training sample set, and the sample from the other training sample set among the three samples is used as the negative example sample;

[0047] Input the anchor samples, the positive samples, and the negative samples into the face-swapping discrimination model to obtain the representation vectors corresponding to each sample determined by the face-swapping discrimination model, and the discrimination results determined by the face-swapping discrimination model using the representation vectors corresponding to the samples, where the discrimination results include the authenticity of the video corresponding to the samples;

[0048] Calculate the positive cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive sample, and the negative cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative sample;

[0049] Determine the vector loss of the face-swapping discrimination model according to the positive cosine distance and the negative cosine distance;

[0050] Determine the prediction loss of the face-swapping discrimination model according to the difference between the discrimination result corresponding to each sample and the labeled tags obtained after the sample is labeled;

[0051] Based on the vector loss and the prediction loss, adjust the parameters of the face-swapping discrimination model until the set training end condition is reached to obtain the trained face-swapping discrimination model.

[0052] Preferably, the discrimination result further includes: when the video corresponding to the sample is fake, the generation source of the video;

[0053] Label the second facial pixel change map as fake, and multiple labeled second facial pixel change maps form a second training sample set, including:

[0054] Label the second facial pixel change map as fake, and based on the video generation source corresponding to the second facial pixel change map, label the generation source of the second facial pixel change map;

[0055] Classify each second facial pixel change map according to the labeled generation source, and multiple labeled second facial pixel change maps of the same class form a generation source training sample set, and multiple generation source training sample sets form a second training sample set;

[0056] Select three samples from the first training sample set and the second training sample set, including:

[0057] Select two target training sample sets from each generation source training sample set of the first training sample set and the second training sample set;

[0058] Select three samples from the two target training sample sets.

[0059] A face-swapping video discrimination method, including:

[0060] Obtain the video to be identified;

[0061] Enlarge the physiological features in the video to be identified to obtain an enlarged video to be identified;

[0062] Generate a facial pixel change map according to the video to be identified and the enlarged video to be identified;

[0063] Input the facial pixel change map into a face swapping identification model trained by a face swapping identification model training method, and obtain an identification result based on the representation vector output by the face swapping identification model, or input the facial pixel change map into a face swapping identification model trained by a face swapping identification model training method to obtain the identification result output by the face swapping identification model.

[0064] As can be seen from the above technical solutions, for the face swap discrimination model training method provided in this application, in order to train a face swap discrimination model that can identify whether a video has been face-swapped using Deepfake technology, before training, it is necessary to obtain real videos and face-swapped videos, and it is not possible to obtain only real videos or only face-swapped videos; since the spatial coherence and temporal consistency of physiological features in face-swapped videos will be destroyed, therefore, in this application, the physiological features in the real videos and the face-swapped videos are magnified to obtain magnified real videos and magnified face-swapped videos, so as to better observe the spatial coherence and temporal consistency of the physiological features in the videos; and since facial pixels can reflect the changes in physiological features, therefore, in this application, according to each real video and the magnified real video corresponding to the real video, a first facial pixel change map is generated, and multiple first facial pixel change maps form a first training sample set, and according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, a second facial pixel change map is generated, and multiple second facial pixel change maps form a second training sample set; to observe the changes in the facial pixels of the people in the videos; when the training set is obtained, the face swap discrimination model can be trained. Three samples are selected from the first training sample set and the second training sample set. Among them, one sample is selected as the anchor sample from two samples from the same training sample set, and the other sample is used as the positive example sample, and the sample from the other training sample set among the three samples is used as the negative example sample; the anchor sample, the positive example sample, and the negative example sample are input into the face swap discrimination model to obtain the representation vector corresponding to each sample output by the face swap discrimination model; calculate the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample; according to the positive example cosine distance and the negative example cosine distance, determine the loss of the face swap discrimination model, and adjust the parameters of the face swap discrimination model based on the loss until the model training end condition is reached, and obtain the trained face swap discrimination model. Among them, during the training process, there are anchor samples, negative example samples, and positive example samples, and the cosine distance between the representation vectors of the anchor sample and the negative example sample can be made farther and farther by adjusting the parameters, and the cosine distance between the representation vectors of the anchor sample and the positive example sample can be made closer and closer. In this way, the authenticity of the video can be discriminated using the representation vectors output by the face swap discrimination model.

[0065] In addition, in this application, the physiological features of the video are magnified. Through the magnification of the physiological features, the changes in the physiological features in the video can be better recognized, small changes will not be missed, and the discrimination accuracy of the face swap discrimination model can be improved.

[0066] In addition, this application generates a pixel change map and uses the pixel change map to train a face swap discrimination model to distinguish between real and fake videos through changes in physiological features. The artifacts in face swap videos will cause abnormal pixel changes, and the pixel changes in real videos follow certain rules. Therefore, this application generates a pixel change map to utilize the differences between the above-mentioned face swap videos and real videos, further improving the discrimination accuracy of the face swap discrimination model.

[0067] Furthermore, in this application, the loss of the face swap discrimination model is determined based on the positive example cosine distance and negative example cosine distance of the samples. When the representation vectors of the positive example samples and the anchor samples are in the same direction, the cosine distance is zero. In this way, the correlation between the representation vectors can be better determined, and thus the loss of the face swap discrimination model can be better determined based on the positive example cosine distance and negative example cosine distance to complete the training of the face swap discrimination model. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative efforts.

[0069] Figure 1 It is a flowchart of the first method for training a face swap discrimination model disclosed in this application;

[0070] Figure 2 It is a flowchart of the first method for discriminating face swap videos disclosed in this application;

[0071] Figure 3 It is a flowchart of the second method for training a face swap discrimination model disclosed in this application;

[0072] Figure 4 It is a flowchart of the second method for discriminating face swap videos disclosed in this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0073] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0074] The abuse of Deepfake technology has brought harm to society. Lawbreakers use Deepfake technology to fabricate rumors, which has an impact on people's happy lives.

[0075] To solve the above problems, this application attempts to introduce a discrimination model capable of identifying face-swapped videos to authenticate videos. To obtain such a discrimination model capable of identifying face-swapped videos, this application provides a method for training a face-swapping discrimination model, using real videos and face-swapped videos forged using Deepfake technology to train the model to ensure the ability to distinguish between real videos and face-swapped videos.

[0076] Next, in combination with Figure 1 A detailed introduction to the first method for training a face-swapping discrimination model of this application is as follows, including the following steps:

[0077] Step S1: Obtain real videos and face-swapped videos.

[0078] Specifically, real videos and face-swapped videos can be obtained through various ways that do not violate morality and laws. For example, the deepfake video dataset released by Google can be obtained and used to train the face-swapping discrimination model.

[0079] Furthermore, those skilled in the art of this application can also collect real videos of themselves or authorized users and perform face swapping on the real videos to obtain real videos and face-swapped videos.

[0080] Step S2: Amplify the physiological features in the real videos and the face-swapped videos to obtain the amplified real videos and amplified face-swapped videos.

[0081] Specifically, since the DeepFakes technology performs frame-by-frame face swapping operations in the pipeline, the physiological features extracted from the facial sequence are discontinuous in the time dimension. And there is a certain distribution pattern of capillaries in the human face, which will lead to phase differences in the periodic color signals of different facial regions during the cardiac cycle, while the faces generated by the DeepFake technology do not have such phase differences. Therefore, there are also spatial position differences in the facial pixel sequences of real videos and face-swapped videos. To more effectively highlight the physiological features of the face and use the temporal and spatial continuity of the physiological features to distinguish the authenticity of the videos. This application amplifies the physiological features in the real videos and face-swapped videos. In this way, it can also amplify the abnormal pixel changes caused by face-swapped video artifacts.

[0082] To further amplify the physiological features so that the face-swapping discrimination model can better output the representation vectors, this application can perform multiple amplifications of the physiological features on the same video.

[0083] Step S3: Generate a first facial pixel change map based on each real video and the amplified real video corresponding to the real video, and multiple first facial pixel change maps form a first training sample set.

[0084] Specifically, using a real video and its magnified video, a first facial pixel change map corresponding to the real video and its magnified video is generated. Among them, the pixel changes of the real video and its magnified video are recorded in the first facial pixel change map.

[0085] Among them, the pixel average value of each frame of the video can be recorded. The abscissa of the first facial pixel change map is each frame in the video, and the ordinate of the first facial pixel change map is the pixel average value corresponding to each frame in the video.

[0086] Step S4: According to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, a second facial pixel change map is generated, and multiple second facial pixel change maps form a second training sample set.

[0087] Specifically, using a face-swapped video and its magnified video, a second facial pixel change map corresponding to the face-swapped video and its magnified video is generated. Among them, the pixel changes of the face-swapped video and its magnified video are recorded in the second facial pixel change map.

[0088] Among them, the pixel average value of each frame of the video can be recorded. The abscissa of the second facial pixel change map is each frame in the video, and the ordinate of the second facial pixel change map is the pixel average value corresponding to each frame in the video.

[0089] It should be noted that in this application, the first facial pixel change map refers to the facial pixel change map corresponding to the real video, and the second facial pixel change map refers to the facial pixel change map corresponding to the face-swapped video.

[0090] The first training sample set refers to the training sample set obtained using the real video, and the second training sample set refers to the training sample set obtained using the face-swapped video.

[0091] Step S5: Select three samples from the first training sample set and the second training sample set. Among them, select one sample as the anchor sample and another sample as the positive example sample from two samples from the same training sample set, and the sample from the other training sample set among the three samples is used as the negative example sample.

[0092] Specifically, there are two ways to select three samples from the first training sample set and the second training sample set. For example, two samples can be selected from the first training sample set and one sample can be selected from the second training sample set; or one sample can be selected from the first training sample set and two samples can be selected from the second training sample set.

[0093] In this application, two samples from the same training sample set are used. One is used as the anchor sample, and the other is used as the positive example sample. The samples other than the anchor sample and the positive example sample among the three samples are used as negative example samples, and these negative example samples are from a training sample set different from the anchor sample and the positive example sample.

[0094] Among them, the anchor sample and the positive example sample do not necessarily refer to the samples from the first training sample set, that is, the anchor sample and the positive example sample are not necessarily the samples corresponding to the real video, and the negative example sample does not necessarily refer to the sample corresponding to the face-swapped video.

[0095] Step S6: Input the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vectors corresponding to each sample output by the face-swapping discrimination model.

[0096] Specifically, after inputting the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model, the representation vectors corresponding to the anchor sample, the positive example sample, and the negative example sample output by the face-swapping discrimination model can be obtained.

[0097] Among them, the face-swapping discrimination model outputs the representation vectors corresponding to the input facial pixel change map based on the input facial pixel change map.

[0098] The representation vector can be a 128-dimensional vector.

[0099] Step S7: Calculate the positive cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample.

[0100] Specifically, considering that if two representation vectors are in the same direction, the cosine distance is 0. If the common Euclidean distance is used to determine the difference between two representation vectors, it is possible that two representation vectors are in the same direction, but their Euclidean distance is large, resulting in a large loss. When using the Euclidean distance to determine the loss to adjust the parameters of the face-swapping discrimination model, the requirements for the representation vectors output by the face-swapping discrimination model are relatively high, and the specific values of the representation vectors output by it need to be controlled. However, when using the cosine distance to determine the loss, the requirements for the face-swapping discrimination model are not too high, and only the direction of the representation vectors output by it needs to be controlled.

[0101] The calculation formula for the cosine distance between two representation vectors is as follows:

[0102]

[0103] Among them, a can be the representation vector corresponding to the anchor sample, and b can be the representation vector corresponding to the positive example sample or the negative example sample. Substituting the specific representation vectors into the above formula, the positive example cosine distance and the negative example cosine distance can be obtained.

[0104] The value range of the cosine distance between two representation vectors is [0, 2]. When the distance between a and b is smaller, that is, a and b are more similar, the cosine distance is closer to 0, and vice versa, it is closer to 2.

[0105] Step S8: Determine the loss of the face swap discrimination model according to the positive example cosine distance and the negative example cosine distance, and adjust the parameters of the face swap discrimination model based on the loss until the model training end condition is reached, and obtain the trained face swap discrimination model.

[0106] Specifically, the following formula can be used to calculate the loss of the face swap discrimination model:

[0107] Loss = max(0, Cosin(f(a), f(b1)) - Cosin(f(a), f(b2)) + α)

[0108] Among them, a can be the representation vector corresponding to the anchor sample, b1 can be the representation vector corresponding to the positive example sample, and b2 can be the representation vector corresponding to the negative example sample. Cosin(f(a), f(b1)) represents the positive example cosine distance, and Cosin(f(a), f(b2)) represents the negative example cosine distance. α represents the tolerance, and is used to define the direction range of the representation vector output by the face swap discrimination model when inputting the facial pixel change map corresponding to the real video.

[0109] After training the face swap discrimination model, the facial pixel change map of the video can be input into the face swap discrimination model to obtain the representation vector corresponding to the video. A representation vector corresponding to a real video can be stored in advance, and the cosine distance between the representation vector corresponding to this video and the pre-stored representation vector can be calculated to distinguish the authenticity of this video; or a new output layer can be added after the trained face swap discrimination model, and this output layer directly outputs the discrimination result.

[0110] The model training end condition can be that the positive example cosine distance between samples from the same training set is lower than the first threshold multiple times, and at the same time, the negative example cosine distance between samples from different training sets is higher than the second threshold.

[0111] As can be seen from the above formula, a loss is incurred only when the negative cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample minus the positive cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample is less than α; otherwise, the loss is 0. In this way, the similarity of the representation vectors of the anchor sample and the positive example sample is reduced, and the similarity of the representation vectors of the anchor sample and the negative example sample is increased, so as to achieve the purpose of reducing the intra-class distance and increasing the inter-class distance, thereby clarifying the boundary between the real video and the face-swapped video and improving the discrimination accuracy.

[0112] As can be seen from the above technical solution, the face-swapping discrimination model training method provided by the embodiments of the present application can make the cosine distance between the representation vectors of the anchor sample and the negative example sample farther and farther through parameter adjustment, and make the cosine distance between the representation vectors of the anchor sample and the positive example sample closer and closer. In this way, the authenticity of the video can be discriminated by using the representation vectors output by the face-swapping discrimination model.

[0113] In addition, the physiological features of the video are amplified in the present application. Through the amplification of the physiological features, the changes in the physiological features in the video can be better recognized, and no tiny changes will be missed, which can improve the discrimination accuracy of the face-swapping discrimination model.

[0114] In addition, the present application generates a pixel change map and uses the pixel change map to train the face-swapping discrimination model to distinguish between real and fake videos through the changes in physiological features. The artifacts in the face-swapped video will cause abnormal pixel changes, and the pixel changes in the real video follow certain rules. Therefore, the present application generates a pixel change map to utilize the differences between the above-mentioned face-swapped video and the real video to further improve the discrimination accuracy of the face-swapping discrimination model. Moreover, even if the face-swapped video is a compressed face-swapped video, there will still be abnormal pixel changes, and the present application can still maintain a high accuracy. In this way, the anti-compression and anti-motion robustness of the discrimination are also improved.

[0115] Furthermore, in the present application, the loss of the face-swapping discrimination model is determined according to the positive cosine distance and negative cosine distance of the samples. When the representation vectors of the positive example sample and the anchor sample are in the same direction, the cosine distance is zero. In this way, the correlation between the representation vectors can be better determined, and thus the loss of the face-swapping discrimination model can be better determined based on the positive cosine distance and negative cosine distance to complete the training of the face-swapping discrimination model.

[0116] In some embodiments of the present application, considering that the lengths of the training samples are consistent, it is easier to train the face swapping discrimination model. However, it is very likely that the lengths of the real video and the face swapped video obtained in step S1 are inconsistent. Therefore, it can be considered to make the samples have consistent lengths. However, directly cutting the video may make the training process cumbersome. Therefore, it can be tried to directly cut the generated first facial pixel change map and the second facial pixel change map. Further, according to the frame rate of the video and the sliding time of the cutting video, a first facial pixel change map can be cut into multiple first facial pixel change maps with a fixed number of frames.

[0117] Cut a second facial pixel change map into multiple second facial pixel change maps with a fixed number of frames.

[0118] Next, the cutting process will be introduced by taking a 250-frame first facial pixel change map as an example and cutting it into multiple 196-frame first facial pixel change maps.

[0119] The sliding time of the cutting video can be set to 0.5 s, and the frame rate of the real video corresponding to the first facial pixel change map is determined. The frame rate can be 30 frames per second. Then, the number of sliding frames in the first facial pixel change map can be obtained as 15 frames.

[0120] In this way, the first 196 frames of the first facial pixel change map can be copied to obtain the first cut map; the 16th to 211th frames of the first facial pixel change map can be copied to obtain the second cut map, which contains the 16th to 211th frames of the first facial pixel change map; the 31st to 226th frames of the first facial pixel change map can be copied to obtain the third cut map, which contains the 31st to 226th frames of the first facial pixel change map; the 46th to 241st frames of the first facial pixel change map can be copied to obtain the fourth cut map, which contains the 46th to 241st frames of the first facial pixel change map. At this time, the remaining first facial pixel change map has less than 196 frames and will not be copied anymore. The four newly obtained cut maps are used as the new first facial pixel change maps.

[0121] In some embodiments of the present application, the process of step S2, which magnifies the physiological features in the real video and the face swapped video to obtain the magnified real video and the magnified face swapped video, will be described in detail as follows:

[0122] S20. Perform three Euler video magnifications on the real video and the face swapped video.

[0123] Specifically, during the first Euler video magnification, two Gaussian downsamplings are first performed. Then, the video obtained after the two Gaussian downsamplings is band-pass filtered in the time dimension to obtain the first pixel information within the human heart rate frequency range, and the first pixel information is magnified by a factor of two.

[0124] During the second Euler video magnification, three Gaussian downsamplings are first performed. Then, the video obtained after the three Gaussian downsamplings is band-pass filtered in the time dimension to obtain the second pixel information within the human heart rate frequency range, and the second pixel information is magnified by a factor of three.

[0125] During the third Euler video magnification, four Gaussian downsamplings are first performed. Then, the video obtained after the four Gaussian downsamplings is band-pass filtered in the time dimension to obtain the third pixel information within the human heart rate frequency range, and the third pixel information is magnified by a factor of five.

[0126] Among them, for the human heart rate frequency range of 0.75 Hz to 3 Hz, time band-pass filtering is used to retain the data within 0.75 Hz to 3 Hz and filter out the data outside 0.75 Hz to 3 Hz. In this way, pixel information highly correlated with physiological characteristics can be obtained. In subsequent steps, based on this data, the real video, and the face-swapped video, pixel changes highly correlated with physiological characteristics are generated.

[0127] After obtaining the magnified pixel information, a video can be synthesized to obtain the magnified real video and the magnified face-swapped video.

[0128] It can be seen from the above technical solution that through the above technical solution, the physiological characteristics in the video can be magnified to obtain a sample that is easier to distinguish between authenticity, thereby better completing the training of the face-swapping discrimination model.

[0129] In some embodiments of the present application, the process of step S3, generating a first facial pixel change map according to each real video and the magnified real video corresponding to the real video, and a plurality of first facial pixel change maps forming a first training sample set, is described in detail as follows:

[0130] S31: Extract the first facial feature points on each real video and the magnified real video corresponding to the real video.

[0131] Specifically, the Dlib face recognition technology can be used to extract the first facial feature points on each real video and the magnified real video corresponding to the real video.

[0132] Among them, 68 facial feature points can be extracted from one face of the real video, and these 68 facial feature points are used as the first facial feature points.

[0133] S32. Divide the first facial region according to the first facial feature points.

[0134] Specifically, the first facial region can be divided according to the extracted first facial feature points.

[0135] Among them, the first facial region refers to the facial region corresponding to the real video or the magnified real video.

[0136] Some facial feature points can be symmetrically connected to divide the facial region. Since the forehead is often heavily blocked by items such as hair, hats, or headgear, the extraction of pixel changes in the forehead is often affected, and the blockers can also cause redundant noise in training, which is not conducive to training. Therefore, the forehead can be excluded from the facial region and does not participate in the subsequent steps.

[0137] Among them, if 68 facial feature points are extracted, 15 facial regions can be obtained. Based on this, if 68 facial feature points are extracted from the real video or the magnified real video, 15 facial regions can be obtained as the first facial region.

[0138] The first facial regions of the same real video and the magnified real video corresponding to this real video are the same.

[0139] S33. Generate a first facial pixel change map recording the pixel changes of each first facial region according to each real video and the magnified real video corresponding to the real video.

[0140] Specifically, a first facial pixel change map recording the pixel changes of each first facial region can be generated according to each real video and the magnified real video corresponding to the real video.

[0141] Among them, the first facial pixel change map records the pixel average value of each first facial region in each frame of the real video and the magnified real video.

[0142] Based on this, the process of step S4: Generate a second facial pixel change map according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, and a plurality of second facial pixel change maps form a second training sample set will be described in detail. The specific steps are as follows:

[0143] S41. Extract the second facial feature points on each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video.

[0144] Specifically, the Dlib face recognition technology can be used to extract the second facial feature points on each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video.

[0145] Among them, 68 facial feature points can be extracted from one face in the face-swapped video, and these 68 facial feature points are used as the second facial feature points.

[0146] S42. Divide the second facial region according to the second facial feature points.

[0147] Specifically, the second facial region can be divided according to the extracted second facial feature points.

[0148] Among them, the first facial region refers to the facial region corresponding to the real video or the enlarged real video. The second facial region refers to the facial region corresponding to the face-swapped video or the enlarged face-swapped video.

[0149] The second facial regions of the same face-swapped video and the enlarged face-swapped video corresponding to this face-swapped video are the same.

[0150] S43. Generate a second facial pixel change map recording the pixel changes of each second facial region according to each face-swapped video and the enlarged face-swapped video corresponding to this face-swapped video.

[0151] Specifically, a second facial pixel change map recording the pixel changes of each second facial region can be generated according to each face-swapped video and the enlarged face-swapped video corresponding to this face-swapped video.

[0152] Among them, the second facial pixel change map records the pixel average value of each second facial region in each frame of this face-swapped video and the enlarged face-swapped video.

[0153] It can be seen from the above technical solutions that this embodiment provides an optional solution of generating a first facial pixel change map according to each real video and the enlarged real video corresponding to this real video, and a plurality of first facial pixel change maps form a first training sample set, and generating a second facial pixel change map according to each face-swapped video and the enlarged face-swapped video corresponding to this face-swapped video, and a plurality of second facial pixel change maps form a second training sample set. It can be seen from the above technical solutions that the pixel changes in the forehead region are not included in the facial pixel change maps generated in this embodiment, considering that redundant noise will be generated when the forehead is often blocked. Through the above technical solutions, the influence of the forehead occlusion on the output representation vector of the face-swapping discrimination model is avoided, and further, the accuracy of the representation vector output by the face discrimination model is improved.

[0154] In some embodiments of the present application, the process of step S33, generating a first facial pixel change map recording the pixel changes of each first facial region according to each real video and the enlarged real video corresponding to this real video, is described in detail, and the steps are as follows:

[0155] S330. Generate a first brightness change diagram that records the brightness change of each first facial area based on each real video and the magnified real video corresponding to the real video.

[0156] Specifically, the first brightness change diagram records the average brightness of each first facial area in each frame of the real video and the magnified real video.

[0157] The change in the average brightness of each frame in the video forms a brightness change.

[0158] Among them, the average value of the Y channel of each first facial area in each frame can be calculated to obtain the average brightness value.

[0159] S331. Generate a first color change diagram that records the color change of each first facial area based on each real video and the magnified real video corresponding to the real video.

[0160] Specifically, the first color change diagram records the average color of each first facial area in each frame of the real video and the magnified real video.

[0161] The change in the average color of each frame in the video forms a color change.

[0162] Among them, the average value of the U channel of each first facial area in each frame can be calculated to obtain the average color value.

[0163] S332. Generate a first saturation change diagram that records the saturation change of each first facial area based on each real video and the magnified real video corresponding to the real video.

[0164] Specifically, the first saturation change diagram records the average saturation of each first facial area in each frame of the real video and the magnified real video.

[0165] The change in the average saturation of each frame in the video forms a saturation change.

[0166] Among them, the average value of the V channel of each first facial area in each frame can be calculated to obtain the average saturation value.

[0167] The first brightness change diagram, the first color change diagram, and the first saturation change diagram corresponding to one real video form a first facial pixel change diagram.

[0168] In some embodiments of the present application, the process of step S43: Generate a second facial pixel change diagram that records the pixel change of each second facial area based on each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video is described in detail as follows:

[0169] S430. Generate a second brightness change diagram that records the brightness change of each second facial region based on each face-swapping video and the magnified face-swapping video corresponding to the face-swapping video.

[0170] Specifically, the average brightness value of each second facial region in each frame of the face-swapping video and the magnified face-swapping video is recorded in the second brightness change diagram.

[0171] S431. Generate a second color change diagram that records the color change of each second facial region based on each face-swapping video and the magnified face-swapping video corresponding to the face-swapping video.

[0172] Specifically, the average color value of each second facial region in each frame of the face-swapping video and the magnified face-swapping video is recorded in the second color change diagram.

[0173] S432. Generate a second saturation change diagram that records the saturation change of each second facial region based on each face-swapping video and the magnified face-swapping video corresponding to the face-swapping video.

[0174] Specifically, the average saturation value of each second facial region in each frame of the face-swapping video and the magnified face-swapping video is recorded in the second saturation change diagram.

[0175] The second brightness change diagram, the second color change diagram, and the second saturation change diagram corresponding to a face-swapping video form a second facial pixel change diagram.

[0176] As can be seen from the above technical solutions, an optional method for generating the first facial pixel change diagram and the second facial pixel change diagram is provided in this embodiment. Through the above technical means, the generation of the brightness change diagram, the color change diagram, and the saturation change diagram is realized, thereby generating the facial pixel change diagram. Through the above generation method, the change of facial pixels can be considered from multiple dimensions, so as to more accurately output the representation vector and obtain the discrimination result based on the representation vector.

[0177] In some embodiments of the present application, multiple models can be used as the face-swapping discrimination model. Among them, the triplet model can be used as the face-swapping discrimination model, or the siamese model with shared parameters can be used as the face-swapping discrimination model. Among them, different models will result in different ways of inputting samples.

[0178] When the triplet model is used as the face-swapping discrimination model, the process of step S6: inputting the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model is described in detail, and the steps are as follows:

[0179] S60. At the same time, input the anchor sample, the positive example sample, and the negative example sample into the triplet model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model.

[0180] Specifically, in one input, simultaneously input the anchor sample, the positive example sample, and the negative example sample into the triplet model. The triplet model simultaneously learns the differences and similarities between the anchor sample, the positive example sample, and the negative example sample to complete the training.

[0181] When using the siamese model with shared parameters as the face-swapping discrimination model, the process of step S6 of inputting the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model is described in detail as follows:

[0182] S61. Input the anchor sample and the positive example sample into the siamese model with shared parameters.

[0183] Specifically, simultaneously input the anchor sample and the positive example sample into the siamese model with shared parameters, so that the face-swapping discrimination model learns the similarities between the anchor sample and the positive example sample.

[0184] S62. Input the anchor sample and the negative example sample into the siamese model with shared parameters.

[0185] Specifically, simultaneously input the anchor sample and the negative example sample into the siamese model with shared parameters, so that the face-swapping discrimination model learns the differences between the anchor sample and the negative example sample.

[0186] It can be seen from the above technical solutions that this embodiment provides two optional types of face-swapping discrimination models. It can be seen from the above technical solutions that this embodiment can obtain a face-swapping discrimination model with better training effect by adopting different training methods for different types of face-swapping discrimination models.

[0187] In some embodiments of the present application, considering that the face-swapping discrimination model trained by the above first face-swapping discrimination model training method can be used to distinguish the authenticity of a video, on this basis, it can be combined with Figure 2 , and the discrimination process of the first face-swapping video discrimination method is described in detail as follows:

[0188] Step S9. Obtain the video to be discriminated.

[0189] Specifically, a video for which it is necessary to know whether it has been face-swapped can be obtained.

[0190] Step S10. Amplify the physiological features in the video to be discriminated to obtain the amplified video to be discriminated.

[0191] Specifically, the physiological features in the acquired video are magnified. Herein, the magnification method adopted may be the same as that in step S2 or step S20 in the foregoing embodiment. For details, please refer to the foregoing introduction and will not be elaborated herein.

[0192] Step S11: Generate a facial pixel change map according to the video to be discriminated and the magnified video to be discriminated.

[0193] Specifically, the method for generating the facial pixel change map adopted herein is the same as that for generating the first facial pixel change map and the second facial pixel change map in the foregoing embodiment. For details, please refer to the foregoing introduction and will not be elaborated herein.

[0194] Step S12: Input the facial pixel change map into a face swapping discrimination model trained by the face swapping discrimination model training method, and obtain a discrimination result based on the representation vector output by the face swapping discrimination model.

[0195] Specifically, the face swapping discrimination model trained herein cannot directly output a discrimination result but can only directly output a representation vector. Therefore, after obtaining the representation vector corresponding to the video to be discriminated output by the face swapping discrimination model, the authenticity of the video to be discriminated can be obtained by judging the direction of the representation vector.

[0196] As can be seen from the above technical solution, this embodiment provides a method for discriminating the authenticity of a video. Through the above method, a representation vector corresponding to the video can be determined by using the face swapping discrimination model, and the authenticity of the video can be accurately discriminated based on the representation vector.

[0197] Next, a first face swapping discrimination model training device provided by an embodiment of the present application will be described. The first face swapping discrimination model training device described below can be correspondingly referred to with the first face swapping discrimination model training method described above.

[0198] The first face swapping discrimination model training device may include:

[0199] A video acquisition unit, configured to acquire a real video and a face swapped video;

[0200] A feature magnification unit, configured to magnify the physiological features in the real video and the face swapped video to obtain a magnified real video and a magnified face swapped video;

[0201] A change map generation unit, configured to generate a first facial pixel change map according to each real video and the magnified real video corresponding to the real video, and a plurality of first facial pixel change maps form a first training sample set;

[0202] A pixel map generation unit, configured to generate a second facial pixel change map according to each face-swapping video and the magnified face-swapping video corresponding to the face-swapping video, and multiple second facial pixel change maps form a second training sample set;

[0203] A sample selection unit, configured to select three samples from the first training sample set and the second training sample set. Among them, one sample is selected as the anchor sample from two samples from the same training sample set, and the other sample is used as the positive example sample, and the sample from the other training sample set among the three samples is used as the negative example sample;

[0204] A sample input unit, configured to input the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model;

[0205] A distance calculation unit, configured to calculate the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample;

[0206] A parameter adjustment unit, configured to determine the loss of the face-swapping discrimination model according to the positive example cosine distance and the negative example cosine distance, and adjust the parameters of the face-swapping discrimination model based on the loss until the model training end condition is reached, so as to obtain the trained face-swapping discrimination model.

[0207] Optionally, the refinement function and the extension function of the program can be referred to the above description.

[0208] Next, a first face-swapping video discrimination device provided by an embodiment of the present application will be described. The first face-swapping video discrimination device described below can be correspondingly referred to the first face-swapping video discrimination method described above.

[0209] A video to be discriminated acquisition unit, configured to acquire a video to be discriminated;

[0210] A physiological feature magnification unit, configured to magnify the physiological features in the video to be discriminated to obtain a magnified video to be discriminated;

[0211] A facial pixel change map generation unit, configured to generate a facial pixel change map according to the video to be discriminated and the magnified video to be discriminated;

[0212] A discrimination result acquisition unit, configured to input the facial pixel change map into the face-swapping discrimination model trained by the face-swapping discrimination model training method, and obtain a discrimination result based on the representation vector output by the face-swapping discrimination model.

[0213] In some embodiments of the present application, considering that directly outputting the discrimination result is more time-saving and convenient than outputting the representation vector. Therefore, the present application also provides a second method for training a face-swapping discrimination model, and the face-swapping discrimination model trained by the second method for training a face-swapping discrimination model can directly output the discrimination result. Next, it will be combined with Figure 3 to introduce the second method for training a face-swapping discrimination model in detail. The specific steps are as follows:

[0214] S1. Obtain a real video and a face-swapped video.

[0215] S2. Amplify the physiological features in the real video and the face-swapped video to obtain an amplified real video and an amplified face-swapped video.

[0216] S3. Generate a first facial pixel change map according to each real video and the amplified real video corresponding to the real video.

[0217] The above steps S1-S3 correspond one by one to S1-S3 in the foregoing embodiments. For details, please refer to the foregoing introduction and will not be elaborated here.

[0218] S4. Label the first facial pixel change map as true, and multiple labeled first facial pixel change maps form a first training sample set.

[0219] Specifically, since the videos corresponding to the first facial pixel change map are the real video and the amplified real video, the first facial pixel change map can be directly labeled as true, indicating that the videos corresponding to the first facial pixel change map are real.

[0220] S5. Generate a second facial pixel change map according to each face-swapped video and the amplified face-swapped video corresponding to the face-swapped video.

[0221] Specifically, step S5 corresponds to step S4 in the foregoing embodiments. For details, please refer to the foregoing introduction and will not be elaborated here.

[0222] S6. Label the second facial pixel change map as false, and multiple labeled second facial pixel change maps form a second training sample set.

[0223] Specifically, since the videos corresponding to the second facial pixel change map are the face-swapped video and the amplified face-swapped video, the second facial pixel change map can be directly labeled as false, indicating that the videos corresponding to the second facial pixel change map are false.

[0224] S7. Select three samples from the first training sample set and the second training sample set. Among them, select one sample as the anchor sample from two samples originating from the same training sample set, and the other sample as the positive example sample. The sample originating from the other training sample set among the three samples is used as the negative example sample.

[0225] Specifically, step S7 corresponds to step S5 in the foregoing embodiment. For details, please refer to the foregoing introduction and will not be elaborated herein.

[0226] S8. Input the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample determined by the face-swapping discrimination model, and the discrimination result determined by using the representation vector corresponding to the sample. The discrimination result includes the authenticity of the video corresponding to the sample.

[0227] Specifically, after inputting the sample into the face-swapping discrimination model, the representation vector corresponding to each sample determined by the face-swapping discrimination model can be obtained.

[0228] Among them, the representation vector can be a 128-dimensional vector. The last layer of the face-swapping discrimination model can be a 128-dimensional fully connected layer. Then the structure of the fully connected layer should be (128, 2).

[0229] This fully connected layer can output the discrimination result by using the representation vector determined by the face-swapping discrimination model.

[0230] The face-swapping discrimination model can be a ResNet50 network with shared weights.

[0231] S9. Calculate the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample.

[0232] Specifically, step S9 corresponds to step S7 in the foregoing embodiment. For details, please refer to the foregoing introduction and will not be elaborated herein.

[0233] S10. Determine the vector loss of the face-swapping discrimination model according to the positive example cosine distance and the negative example cosine distance.

[0234] Specifically, the method for determining the vector loss in step S10 corresponds to the method for determining the loss in step S8 in the foregoing embodiment. For details, please refer to the foregoing introduction and will not be elaborated herein.

[0235] S11. Determine the prediction loss of the face-swapping discrimination model according to the difference between the discrimination result corresponding to each sample and the labeled label obtained after labeling the sample.

[0236] Specifically, the difference between the discrimination result corresponding to each sample and the labeled tag corresponding to the sample can be analyzed. Based on the differences corresponding to the anchor samples, positive samples, and negative samples, the prediction loss of the face swap discrimination model is determined.

[0237] S12. Based on the vector loss and the prediction loss, adjust the parameters of the face swap discrimination model until the set training end condition is reached, and obtain the trained face swap discrimination model.

[0238] Specifically, the parameters of all layers of the face swap discrimination model can be adjusted based on the vector loss and the prediction loss.

[0239] The set training end condition can be that the discrimination result of each sample output by the face swap discrimination model is consistent with the labeled tag obtained after labeling the sample multiple times.

[0240] It can be seen from the above technical solutions that this embodiment provides an optional way to obtain a new face swap discrimination model. Through the above solution, the new face swap discrimination model can directly obtain the discrimination result and can more conveniently confirm the authenticity of the video.

[0241] In some embodiments of the present application, the process of step S6, labeling the second facial pixel change map as false, and multiple labeled second facial pixel change maps form a second training sample set is described in detail as follows:

[0242] S60. Label the second facial pixel change map as false, and based on the video generation source corresponding to the second facial pixel change map, label the generation source of the second facial pixel change map.

[0243] Specifically, since the second facial pixel change map corresponds to a face swap video, and each face swap video has a corresponding generation source, it is possible to try to label the second facial pixel change map.

[0244] After labeling, the labeled tag of the second facial pixel change map indicates that the video is false and indicates the generation source of the video.

[0245] Among them, the video generation source is the face swap model, face swap program, face swap means, or face swap technology used for the face swap video. The video generation source can include: Face Swap, Face2Face, and Neural Textures, etc.

[0246] S61. Classify each second facial pixel change map according to the labeled generation source, and multiple labeled second facial pixel change maps of the same class form a generation source training sample set, and multiple generation source training sample sets form a second training sample set.

[0247] Specifically, classify each second facial pixel change map according to the generation source of the video corresponding to each second facial pixel change map, and regard the second facial pixel change maps corresponding to the videos with the same generation source as the same class.

[0248] For example, the generation sources marked on each second facial pixel change map can be Face2Face and NeuralTextures. Regard the second facial pixel change maps corresponding to the face-swapping videos with the generation source of Face2Face as the same class. Regard the second facial pixel change maps corresponding to the face-swapping videos with the generation source of Neural Textures as the same class. In this way, two classes of second facial pixel change maps are obtained.

[0249] After that, the labeled second facial pixel change maps of the same class can be grouped into a generation source training sample set. For example, multiple second facial pixel change maps marked with the video generation source of Face2Face form a generation source training sample set.

[0250] The second training sample set contains multiple different generation source training sample sets. The labels of the second facial pixel change maps in each generation source training sample set are the same, and the labels indicating the video generation source corresponding to the second facial pixel change maps from different generation source training sample sets are different.

[0251] The second training sample set contains second facial pixel change maps corresponding to multiple video generation sources.

[0252] Based on this, the process of step S7 of selecting three samples from the first training sample set and the second training sample set is described in detail. Among them, select one sample as the anchor sample and another sample as the positive example sample from two samples from the same training sample set, and the sample from another training sample set among the three samples is used as the negative example sample. The steps are as follows:

[0253] S70. Select two target training sample sets from each generation source training sample set of the first training sample set and the second training sample set.

[0254] Specifically, select two training sample sets from the first training sample set and each generation source training sample set as the target training sample sets.

[0255] Among them, it is possible that the two selected target training sample sets are both generation source training sample sets, and it is possible that one of the two target training sample sets is the first training sample set and the other is a generation source training sample set.

[0256] S71. Select three samples from the two target training sample sets.

[0257] Specifically, one target training sample set can be selected from two target training sample sets, two samples can be selected from this target training sample set, and one sample can be selected from the other target training sample set.

[0258] At this time, the discrimination results output by the face-swapping discrimination model can include: the authenticity of the video and the source of generation of the video when the video is fake.

[0259] It can be seen from the above technical solutions that this embodiment provides an optional solution of adding a fully connected layer to be trained after the face-swapping discrimination model is trained. Through the above technical solutions, a new face-swapping discrimination model can be obtained. This new face-swapping discrimination model can not only output discrimination results indicating the authenticity of the video, but also, when the video is fake, discriminate the source of generation of the video. By using three samples for discrimination training, the boundaries of the representation vectors corresponding to face-swapped videos of different sources of generation are implicitly restricted, and at the same time, the inter-class margin is expanded to prevent the phenomenon of boundary fusion, achieving more efficient discrimination.

[0260] In some embodiments of the present application, considering that the face-swapping discrimination model trained by the above second face-swapping discrimination model training method can be used to discriminate the authenticity of the video and directly output the discrimination result, on this basis, it can be combined with Figure 4 to elaborate on the second face-swapped video discrimination method in detail. The specific steps are as follows:

[0261] Step S13: Obtain the video to be discriminated.

[0262] Specifically, a video of unknown authenticity can be obtained.

[0263] Step S14: Amplify the physiological features in the video to be discriminated to obtain the amplified video to be discriminated.

[0264] Specifically, the physiological features in the obtained video are amplified. Here, the amplification method used can be the same as the amplification method in step S2 or step S20 in the foregoing embodiment. For detailed information, please refer to the foregoing introduction and will not be elaborated here.

[0265] Step S15: Generate a facial pixel change map based on the video to be discriminated and the amplified video to be discriminated.

[0266] Specifically, the method for generating the facial pixel change map used here is the same as the method for generating the first facial pixel change map and the second facial pixel change map in the foregoing embodiment. For detailed information, please refer to the foregoing introduction and will not be elaborated here.

[0267] Step S16: Input the facial pixel change map into the face-swapping discrimination model trained by the face-swapping discrimination model training method to obtain the discrimination result output by the face-swapping discrimination model.

[0268] Specifically, the face-swapping discrimination model trained herein can directly obtain the discrimination result corresponding to the video to be discriminated output by the face-swapping discrimination model, and directly confirm the authenticity of the video to be discriminated through the discrimination result.

[0269] It can be seen from the above technical solution that this embodiment provides an optional way to discriminate the authenticity of a video. Through the above method, the authenticity of the video can be discriminated by using the pixel changes caused by the physiological features in the video.

[0270] Next, a second face-swapping discrimination model training device provided by an embodiment of the present application will be described. The second face-swapping discrimination model training device described below can be correspondingly referred to the second face-swapping discrimination method described above.

[0271] The second face-swapping discrimination model training device may include:

[0272] A video acquisition unit, configured to acquire a real video and a face-swapped video;

[0273] A feature amplification unit, configured to amplify the physiological features in the real video and the face-swapped video to obtain an amplified real video and an amplified face-swapped video;

[0274] A change map generation unit, configured to generate a first facial pixel change map according to each real video and the amplified real video corresponding to the real video;

[0275] A change map annotation unit, configured to annotate the first facial pixel change map as true, and multiple annotated first facial pixel change maps form a first training sample set;

[0276] A pixel map generation unit, configured to generate a second facial pixel change map according to each face-swapped video and the amplified face-swapped video corresponding to the face-swapped video;

[0277] A pixel map annotation unit, configured to annotate the second facial pixel change map as false, and multiple annotated second facial pixel change maps form a second training sample set;

[0278] A sample selection unit, configured to select three samples from the first training sample set and the second training sample set. Among them, one sample is selected as an anchor sample from two samples from the same training sample set, and the other sample is used as a positive example sample. The sample from the other training sample set among the three samples is used as a negative example sample;

[0279] A sample input unit for inputting the anchor sample, the positive example sample, and the negative example sample into a face swap discrimination model to obtain a representation vector corresponding to each sample determined by the face swap discrimination model, and a discrimination result determined by the face swap discrimination model using the representation vector corresponding to the sample, where the discrimination result includes the authenticity of the video corresponding to the sample;

[0280] A distance calculation unit for calculating the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample;

[0281] A vector loss determination unit for determining the vector loss of the face swap discrimination model according to the positive example cosine distance and the negative example cosine distance;

[0282] A prediction loss determination unit for determining the prediction loss of the face swap discrimination model according to the difference between the discrimination result corresponding to each sample and the labeled label obtained after the sample is labeled;

[0283] A model adjustment unit for adjusting the parameters of the face swap discrimination model based on the vector loss and the prediction loss until a set training end condition is reached, and obtaining a trained face swap discrimination model.

[0284] Next, a second face swap video discrimination device provided by an embodiment of the present application will be described. The second face swap video discrimination device described below can be mutually corresponding and referred to the second face swap video discrimination method described above.

[0285] A video to be discriminated acquisition unit for acquiring a video to be discriminated;

[0286] A physiological feature amplification unit for amplifying the physiological features in the video to be discriminated to obtain an amplified video to be discriminated;

[0287] A facial pixel change map generation unit for generating a facial pixel change map according to the video to be discriminated and the amplified video to be discriminated;

[0288] A discrimination result obtaining unit for inputting the facial pixel change map into a face swap discrimination model trained by a face swap discrimination model training method to obtain a discrimination result obtained from the representation vector output by the face swap discrimination model.

[0289] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0290] The various embodiments in this specification are described in a progressive manner, and the key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the various embodiments, reference may be made to each other.

[0291] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. The various embodiments of the present application can be combined with each other. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a face-swapping discrimination model, characterized in that, Including: Obtaining a real video and a face-swapped video; Magnifying the physiological features in the real video and the face-swapped video to obtain a magnified real video and a magnified face-swapped video; Generating a first facial pixel change map according to each real video and the magnified real video corresponding to the real video, and a plurality of first facial pixel change maps form a first training sample set; Generating a second facial pixel change map according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, and a plurality of second facial pixel change maps form a second training sample set; Selecting three samples from the first training sample set and the second training sample set. Among them, one sample is selected as the anchor sample from two samples from the same training sample set, and the other sample is used as the positive example sample, and the sample from the other training sample set among the three samples is used as the negative example sample; Inputting the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model; Calculating the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample; Determining the loss of the face-swapping discrimination model according to the positive example cosine distance and the negative example cosine distance, and adjusting the parameters of the face-swapping discrimination model based on the loss until the model training end condition is reached, to obtain the trained face-swapping discrimination model.

2. The method according to claim 1, wherein The generating a first facial pixel change map according to each real video and the magnified real video corresponding to the real video includes: Extracting the first facial feature points on each real video and the magnified real video corresponding to the real video; Dividing the first facial area according to the first facial feature points; Generating a first facial pixel change map recording the pixel change of each first facial area according to each real video and the magnified real video corresponding to the real video; The generating a second facial pixel change map according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video includes: Extracting the second facial feature points on each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video; Dividing the second facial area according to the second facial feature points; Generating a second facial pixel change map recording the pixel change of each second facial area according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video.

3. The method according to claim 2, wherein The generating a first facial pixel change map recording the pixel change of each first facial area according to each real video and the magnified real video corresponding to the real video includes: Generate a first brightness change map recording the brightness change of each first face region, a first color change map recording the color change of each first face region, and a first saturation change map recording the saturation change of each first face region according to each real video and the magnified real video corresponding to the real video. Among them, the first brightness change map, the first color change map, and the first saturation change map corresponding to one real video form a first face pixel change map.

4. The method according to claim 2, wherein Generate a second face pixel change map recording the pixel change of each second face region according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video, including: Generate a second brightness change map recording the brightness change of each second face region, a second color change map recording the color change of each second face region, and a second saturation change map recording the saturation change of each second face region according to each face-swapped video and the magnified face-swapped video corresponding to the face-swapped video. Among them, the second brightness change map, the second color change map, and the second saturation change map corresponding to one face-swapped video form a second face pixel change map.

5. The method according to claim 1, wherein The face-swapping discrimination model is a triplet model; Input the anchor sample, the positive sample, and the negative sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model, including: Input the anchor sample, the positive sample, and the negative sample into the triplet model at the same time to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model.

6. The method according to claim 1, wherein The face-swapping discrimination model is a siamese model with shared parameters; Input the anchor sample, the positive sample, and the negative sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample output by the face-swapping discrimination model, including: Input the anchor sample and the positive sample into the siamese model with shared parameters; Input the anchor sample and the negative sample into the siamese model with shared parameters.

7. The method according to claim 1, characterized in that Magnify the physiological features in the real video and the face-swapped video, including: Perform three Euler video magnifications on the real video and the face-swapped video; Among them, during the first Euler video magnification, first perform two Gaussian downsamplings, perform band-pass filtering on the video obtained after the two Gaussian downsamplings in the time dimension to obtain the first pixel information within the human heart rate frequency range, and magnify the first pixel information by two times; During the second Euler video magnification, first perform three Gaussian downsamplings, perform band-pass filtering on the video obtained after the three Gaussian downsamplings in the time dimension to obtain the second pixel information within the human heart rate frequency range, and magnify the second pixel information by three times; During the third Euler video magnification, first perform four Gaussian downsamplings, perform band-pass filtering on the video obtained after the four Gaussian downsamplings in the time dimension to obtain the third pixel information within the human heart rate frequency range, and magnify the third pixel information by five times.

8. A method for training a face-swapping discrimination model, characterized in that, Include: Obtain the real video and the face-swapped video; Enlarge the physiological features in the real video and the face-swapped video to obtain an enlarged real video and an enlarged face-swapped video; Generate a first facial pixel change map based on each real video and the enlarged real video corresponding to the real video; Label the first facial pixel change map as true, and multiple labeled first facial pixel change maps form a first training sample set; Generate a second facial pixel change map based on each face-swapped video and the enlarged face-swapped video corresponding to the face-swapped video; Label the second facial pixel change map as false, and multiple labeled second facial pixel change maps form a second training sample set; Select three samples from the first training sample set and the second training sample set. Among them, select one sample as the anchor sample and the other sample as the positive example sample from two samples from the same training sample set, and the sample from the other training sample set among the three samples is used as the negative example sample; Input the anchor sample, the positive example sample, and the negative example sample into the face-swapping discrimination model to obtain the representation vector corresponding to each sample determined by the face-swapping discrimination model, and the discrimination result determined by the face-swapping discrimination model using the representation vector corresponding to the sample. The discrimination result includes the authenticity of the video corresponding to the sample; Calculate the positive example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the positive example sample, and the negative example cosine distance between the representation vector corresponding to the anchor sample and the representation vector corresponding to the negative example sample; Determine the vector loss of the face-swapping discrimination model according to the positive example cosine distance and the negative example cosine distance; Determine the prediction loss of the face-swapping discrimination model according to the difference between the discrimination result corresponding to each sample and the labeled label obtained after the sample is labeled; Based on the vector loss and the prediction loss, adjust the parameters of the face-swapping discrimination model until the set training end condition is reached to obtain a trained face-swapping discrimination model.

9. The method according to claim 8, characterized in that, The discrimination result further includes: when the video corresponding to the sample is false, label the video generation source; Label the second facial pixel change map as false, and multiple labeled second facial pixel change maps form a second training sample set, including: Label the second facial pixel change map as false and label the video generation source of the second facial pixel change map; Classify each second facial pixel change map according to the labeled video generation source of each second facial pixel change map. Multiple labeled second facial pixel change maps of the same class form a generation source training sample set, and multiple generation source training sample sets form a second training sample set; Select three samples from the first training sample set and the second training sample set, including: Select two target training sample sets from each generation source training sample set of the first training sample set and the second training sample set; Select three samples from the two target training sample sets.

10. A method for identifying face-swapped videos, characterized in that, Including: Obtain the video to be discriminated; Enlarge the physiological features in the video to be discriminated to obtain an enlarged video to be discriminated; Generate a facial pixel change map based on the video to be identified and the magnified video to be identified; Input the facial pixel change map into the face swapping identification model trained by using the face swapping identification model training method according to any one of claims 1-7, and obtain an identification result based on the representation vector output by the face swapping identification model, or input the facial pixel change map into the face swapping identification model trained by using the face swapping identification model training method according to claim 8 or 9, and obtain the identification result output by the face swapping identification model.

Images