Deep pseudo video identification model training method, identification method and electronic equipment

Abstract

The invention provides a deep pseudo video identification model training method, an identification method and electronic equipment, authenticity of a to-be-identified video can be identified through a target deep pseudo video identification model obtained through training, and in the model training process, adversarial sample video feature data is introduced into training data, so that the authenticity of the to-be-identified video can be identified. The model not only can identify a real sample video and a deep pseudo sample video in the learning process, but also can correctly identify an adversarial sample video, and the robustness and generalization ability of the model to various inputs are improved.

Description

Technical Field

[0001] The present application relates to the technical field of deep fake video identification, and in particular to a training method, an identification method and an electronic device for a deep fake video identification model. Background Art

[0002] Deepfake technology has made significant progress with advanced methods such as generative adversarial networks (GANs). By using large amounts of data for training, these technologies can generate highly realistic fake videos in which facial expressions, movements, and voices of people can be accurately forged. The widespread dissemination of such deepfake videos may lead to serious consequences. Therefore, how to achieve the authenticity identification of videos has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a training method, an identification method and an electronic device for a deep fake video identification model to solve the above-mentioned technical problems.

[0004] On the one hand, a training method for a deep fake video identification model is provided, comprising: Acquire training sample feature data and an initial deep fake video identification model; the training sample feature data includes real sample video feature data, deep fake sample video feature data and adversarial sample video feature data; the adversarial sample video feature data is video feature data generated by inputting the real sample video feature data and / or the deep fake sample video feature data into a generator; In each training process, the training sample feature data is input into the initial deep fake video identification model, and the initial deep fake video identification model outputs a corresponding video type discrimination result for the training sample feature data; Adjusting model parameters in the initial deep fake video identification model according to a difference between the video type discrimination result and a true label corresponding to the training sample feature data; When the preset training completion conditions are reached, the training is stopped to obtain the target deep fake video identification model.

[0005] In one embodiment, the real sample video feature data includes real sample video visual feature data, and the deep fake sample video feature data includes deep fake sample video visual feature data; and obtaining the training sample feature data includes: Obtain real sample videos and deep fake sample videos; Performing image preprocessing on the real sample video and the deep fake sample video; Image feature extraction is performed on the real sample video and the deep fake sample video after image preprocessing to obtain visual feature data of the real sample video and visual feature data of the deep fake sample video.

[0006] In one of the embodiments, the initial deep fake video identification model includes a convolutional layer, a pooling layer, a fully connected layer and an output layer; The convolution layer performs convolution operations on the input real sample video visual feature data and the deep fake sample video visual feature data respectively to obtain first real sample visual feature data and first deep fake sample visual feature data; The pooling layer performs pooling processing on the first real sample visual feature data and the first deep fake sample visual feature data respectively to obtain second real sample visual feature data and second deep fake sample visual feature data; The fully connected layer outputs corresponding third real sample visual feature data based on the second real sample visual feature data, and outputs corresponding third deep fake sample visual feature data based on the second deep fake sample visual feature data; The output layer is used to output the corresponding video type discrimination result based on the output of the fully connected layer.

[0007] In one of the embodiments, the initial deep fake video identification model further includes an attention module, the attention module being configured to output corresponding real sample weighted visual feature data according to the second real sample visual feature data, and output corresponding deep fake sample weighted visual feature data according to the second deep fake sample visual feature data; The fully connected layer outputs the third real sample visual feature data according to the real sample weighted visual feature data, and outputs the third deep fake sample visual feature data according to the deep fake sample weighted visual feature data.

[0008] In one of the embodiments, the real sample video feature data also includes real sample video audio feature data, and the deep fake sample video feature data also includes deep fake sample video audio feature data; the initial deep fake video identification model also includes a feature fusion module; The step of obtaining training sample feature data includes: Extracting audio features from the real sample video and the deep fake sample video to obtain audio feature data of the real sample video and audio feature data of the deep fake sample video; The feature fusion module is used to fuse the real sample video and audio feature data with the third real sample visual feature data to obtain real sample fusion feature data, and use the real sample fusion feature data as the input of the output layer; the feature fusion module is also used to fuse the deep fake sample video and audio feature data with the third deep fake sample visual feature data to obtain deep fake sample fusion feature data, and use the deep fake sample fusion feature data as the input of the output layer.

[0009] In one embodiment, the method further comprises: During the training process, according to the difference between the video type discrimination result and the real label corresponding to the training sample feature data, the parameters of the generator are adjusted to obtain an updated generator; Generate new adversarial sample video feature data based on the updated generator; The initial deep fake video identification model is trained using the new adversarial sample video feature data.

[0010] In one embodiment, generating new adversarial sample video feature data based on the updated generator includes: The real sample video feature data and / or the deep fake sample video feature data are input into the updated generator to generate new adversarial sample video feature data.

[0011] Furthermore, a deep fake video identification method is also provided, comprising: Get the video to be identified; Extracting video feature data of the video to be identified; The video feature data is input into any of the above-mentioned target deep fake video identification models to obtain the identification result of the video to be identified.

[0012] In one embodiment, after obtaining the identification result of the video to be identified, the method includes: When the video to be identified is displayed, prompt information of the identification result of the video to be identified is superimposed and displayed on the video to be identified.

[0013] Furthermore, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above methods.

[0014] The training method, identification method and electronic device of the deep fake video identification model provided by the present application can identify the authenticity of the video to be identified by the target deep fake video identification model obtained through training. During the training process of the model, by introducing adversarial sample video feature data into the training data, the model can not only identify real sample videos and deep fake sample videos during the learning process, but also correctly identify adversarial sample videos, thereby improving the robustness and generalization ability of the model to various inputs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a method for training a deep fake video identification model provided in Example 1 of the present application; Figure 2A schematic diagram of the process of identifying a deep fake video provided in Example 1 of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in Example 2 of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0017] The rapid development of deep fake technology has led to the emergence of a large number of fake videos in the online environment, posing a serious challenge to the authenticity of information and social order. The embodiments of the present application aim to provide an efficient and accurate solution for identifying deep fake videos and ensure the credibility of video content.

[0018] This application embodiment provides a training method for a deep fake video identification model, see Figure 1 As shown, including: S11: Obtain training sample feature data and an initial deep fake video identification model; the training sample feature data includes real sample video feature data, deep fake sample video feature data and adversarial sample video feature data; the adversarial sample video feature data is the video feature data generated by inputting the real sample video feature data and / or the deep fake sample video feature data into a generator.

[0019] S12: During each training process, the training sample feature data is input into the initial deep fake video identification model, and the initial deep fake video identification model outputs a corresponding video type discrimination result based on the training sample feature data.

[0020] S13: Adjusting model parameters in the initial deep fake video identification model according to the difference between the video type discrimination result and the real label corresponding to the training sample feature data.

[0021] S14: Stop training when the preset training completion condition is reached to obtain the target deep fake video identification model.

[0022] The above steps are described in detail below.

[0023] In the above step S11, an adversarial training method can be used to introduce the adversarial idea in the Generative Adversarial Network (GAN). Constructing adversarial samples , by minimizing the adversarial loss To train the network's ability to recognize adversarial samples. The adversarial loss can be expressed as ,in It is the discriminator network, that is, the initial deepfake video identification model in the embodiment of this application. In this way, the model is more robust to different types of deepfake attacks, and can effectively deal with videos generated by various complex deepfake generation algorithms, and even in the face of new deepfake technologies, it can maintain a high detection accuracy.

[0024] The function of the generator in the embodiment of the present application is to generate adversarial samples. It can take the real sample video feature data and / or the deep fake sample video feature data as input, and generate adversarial samples by adding tiny, carefully designed perturbations. These perturbations are usually calculated based on gradient information, with the aim of making the generated adversarial samples visually indistinguishable from the original samples, but being able to produce a large difference in the output of the model. For example, algorithms such as the fast gradient sign method (FGSM) can be used to generate adversarial samples. Specifically, for an input sample x and a corresponding label y, the gradient of the model's loss function L on x with respect to x is calculated. , and then add disturbances according to the direction of the gradient to generate adversarial samples , where ε is a small constant that controls the amplitude of the perturbation.

[0025] The real sample video feature data in the embodiment of the present application includes real sample video visual feature data, and the deep fake sample video feature data includes deep fake sample video visual feature data; correspondingly, the adversarial sample video feature data at this time includes adversarial sample video visual feature data.

[0026] At this time, obtaining training sample feature data in step S11 includes: Step 1: Obtain real sample videos and deep fake sample videos; Step 2: performing image preprocessing on the real sample video and the deep fake sample video; Step three: extract image features from the real sample video and the deep fake sample video after image preprocessing to obtain visual feature data of the real sample video and visual feature data of the deep fake sample video.

[0027] In the above step 1, real sample videos can be collected through the camera, that is, the sample videos collected through the camera can be used as the real sample videos here. The real sample videos come from a wide range of sources, covering different scenes (such as indoors, outdoors, different light conditions, etc.), shooting equipment (including various models of cameras, mobile phones, etc.) and types of people (different ages, genders, skin colors, etc.).

[0028] Deep fake sample videos can be generated or synthesized through forgery technology. Deep fake sample videos include sample videos generated by various mainstream deep fake technologies (such as GAN-based Deepfake, FaceSwap, etc.), ensuring the diversity and representativeness of the data.

[0029] The image preprocessing process in the above step 2 is introduced as follows: when performing image preprocessing on the real sample video, a standardization operation is performed on each frame of the real sample image in the real sample video. The standardization operation performed on each frame of the real sample image may include the following contents: Assume that the real sample image is ,in is the image pixel coordinate.

[0030] First, the size normalization operation is performed, using the formula ,in and They are x and y The scaling factor in the direction adjusts the image size to a fixed size for subsequent processing.

[0031] Then perform color space conversion, for example, converting from the common RGB color space to the YUV color space that is more suitable for feature extraction. The conversion formula is: .

[0032] Thus, for each frame of the real sample image, its corresponding preprocessed image can be obtained. When performing image preprocessing on the deep fake sample video, the above-mentioned method for performing image preprocessing on the real sample video can be referred to, and no further description is given here. For each frame of the deep fake sample image, its corresponding preprocessed image can also be obtained.

[0033] In the above step 3, for each frame of the preprocessed real sample image, image feature extraction is performed to obtain the corresponding visual feature data, and the visual feature data corresponding to each frame of the preprocessed real sample image is combined to obtain the real sample video visual feature data of the real sample video. Similarly, the deep fake sample video visual feature data of the deep fake sample video can be obtained.

[0034] The following is an introduction to the process of extracting image feature data: For example, for a facial image, a facial feature point detection algorithm (such as an active shape model ASM or an active appearance model AAM) can be used to extract facial key feature points. ,in Indicates At the same time, by calculating the texture features of the image, such as the gray-level co-occurrence matrix (GLCM), Indicates the distance ,direction The gray value of the upper pixel is and The occurrence probability of is used to quantify the image texture information, and the extracted image texture information and facial key feature points can be used as image feature data of the image.

[0035] In an embodiment of the present application, real sample video visual feature data and / or deep fake sample video visual feature data may be input into a generator to generate adversarial sample video visual feature data.

[0036] In some other embodiments, real sample videos and / or deep fake sample videos may be input into a generator to generate adversarial sample videos, and then visual feature extraction may be performed on the adversarial sample videos with reference to the above-mentioned method of extracting visual feature data from real sample videos, thereby obtaining the visual feature data of the adversarial sample videos.

[0037] It should be noted that the real sample video feature data in the embodiments of the present application may also include real sample video and audio feature data, and the deep fake sample video feature data may also include deep fake sample video and audio feature data; correspondingly, the adversarial sample video feature data may also include adversarial sample video and audio feature data.

[0038] At this time, obtaining training sample feature data in step S11 includes: Step 1: Obtain real sample videos and deep fake sample videos; Step 2: extracting audio features from the real sample video and the deep fake sample video to obtain audio feature data of the real sample video and audio feature data of the deep fake sample video; In one example, the real sample video and audio feature data and / or the deep fake sample video and audio feature data may be input into the generator to obtain anti-sample video and audio feature data.

[0039] In another example, the real sample video and / or the deep fake sample video can be input into the generator to obtain an adversarial sample video, and then audio features can be extracted from the adversarial sample video to obtain corresponding adversarial sample video audio feature data.

[0040] In the embodiment of the present application, the model architecture of the initial deep fake video identification model can be flexibly selected by the developer according to the needs. For example, the initial deep fake video identification model can be a CNN (convolutional neural network) model, including a convolution layer, a pooling layer, a fully connected layer and an output layer.

[0041] The convolution layer performs convolution operations on the input real sample video visual feature data and the deep fake sample video visual feature data respectively to obtain first real sample visual feature data and first deep fake sample visual feature data; similarly, the convolution layer can also perform convolution operations on the input adversarial sample video visual feature data to obtain first adversarial sample visual feature data.

[0042] The convolution layer is the key part of feature extraction. Let the real sample video visual feature data of the input convolution layer be the feature map. , the convolution kernel is , the bias is , then the convolution operation can be expressed as ,in The coordinates of the output feature map. Through multiple convolution kernels of different sizes and parameters, the different levels of features of the image from shallow to deep layers are automatically extracted.

[0043] The pooling layer performs pooling processing on the first real sample visual feature data and the first deep fake sample visual feature data respectively to obtain second real sample visual feature data and second deep fake sample visual feature data; similarly, the pooling layer can also perform pooling processing on the first adversarial sample visual feature data to obtain second adversarial sample visual feature data.

[0044] The pooling layer performs dimensionality reduction on the feature maps output by the convolutional layer to reduce the amount of computation and model parameters. For example, the maximum pooling operation can be expressed as ,in R is the pooling window area, s is the pooling step size. This operation can reduce the data dimension while retaining the main features.

[0045] The fully connected layer outputs corresponding third real sample visual feature data based on the second real sample visual feature data, and outputs corresponding third deep fake sample visual feature data based on the second deep fake sample visual feature data; and can output corresponding third adversarial sample visual feature data based on the second adversarial sample visual feature data.

[0046] The fully connected layer is used to classify the extracted features. Suppose the input vector of the fully connected layer is , the weight matrix is , the bias vector is , the output is = + During the training process, the Adam optimizer is used to update the model parameters, and the parameter update formula is: ; Among them and are the first-order moment estimate and the second-order moment estimate, respectively. , is the decay rate, is the gradient, is the learning rate, is a small constant used to prevent the denominator from being zero. At the same time, the cross entropy loss function is used To measure the difference between the model prediction result and the true label, is the true label, is the model predicted probability.

[0047] The output layer is used to output the corresponding video type discrimination result based on the output of the fully connected layer.

[0048] It can be understood that the initial deep fake video identification model can include multiple convolutional layers, multiple pooling layers, and multiple fully connected layers. The specific number of layers can be flexibly set by the developer according to actual conditions, and the connection relationship between the layers can also be flexibly set by the developer.

[0049] At the beginning of training, you can set appropriate training parameters, such as the learning rate and the number of training rounds. During the training process, adjust the model's hyperparameters by monitoring the accuracy and loss value of the model on the validation set. When it is found that the accuracy is no longer improving or the loss value is no longer decreasing, use techniques such as early stopping to avoid overfitting. At the same time, according to the requirements of adversarial training, adversarial samples are regularly generated and added to the training set. The generation of adversarial samples can be achieved by adding small perturbations to the original samples. The calculation of the perturbation is based on the adversarial training algorithm to ensure that the perturbation can interfere with the model without affecting the visual perception of the human eye, thereby enhancing the robustness of the model.

[0050] Exemplarily, n convolutional layers, n pooling layers, and n fully connected layers are set, the output of the i-th convolutional layer is used as the input of the i-th pooling layer, and the output of the i-th pooling layer is used as the input of the i+1-th convolutional layer, that is, one convolutional layer is connected to one pooling layer, and the output of the n-th pooling layer is used as the input of the first fully connected layer, and each fully connected layer is connected in sequence.

[0051] Each layer can process the data as follows: Convolutional layer 1: The input is the visual feature data of real sample videos, deep fake sample videos, and adversarial sample videos. Convolutional layer 1 performs convolution operations on these input data through multiple convolution kernels to extract shallow features of the image, such as basic visual information such as edges and textures, and outputs feature maps of convolutional layer 1, which contain local feature representations of the input data under different convolution kernels.

[0052] Convolutional layer 2: The input is the output of pooling layer 1. Pooling layer 1 performs dimensionality reduction on the output feature map of convolutional layer 1, retaining the main features while reducing the amount of data. On this basis, convolutional layer 2 further extracts deeper features. These features are more abstract and representative than the features of convolutional layer 1, and can capture more complex patterns and structures in the image. The output is the feature map of convolutional layer 2.

[0053] Convolutional layer 3: The input is the output of pooling layer 2. Similar to convolutional layer 2, convolutional layer 3 continues to deeply mine features and extract higher-level features. Due to the two pooling operations, the input feature map is further reduced in spatial dimension, but the feature abstraction is higher. On this basis, convolutional layer 3 can capture more global and complex feature information, and the output is the feature map of convolutional layer 3.

[0054] Convolutional layer n: The input is the output of pooling layer n-1. Convolutional layer n is the last convolutional layer in the neural network. Its input feature map has undergone multiple convolution and pooling operations and has very rich high-level feature information. The task of convolutional layer n is to perform the final feature refinement and enhancement based on these high-level features, and the output feature map will serve as the input of pooling layer n.

[0055] Pooling layer 1: The input is the output feature map of convolution layer 1. Pooling layer 1 performs dimensionality reduction processing on the shallow features extracted by convolution layer 1 through operations such as maximum pooling, retains the main feature information in the feature map, and reduces the amount of data and calculation. The output is the feature map of pooling layer 1. These feature maps are smaller in spatial dimension than the output feature map of convolution layer 1, but have stronger feature expression ability.

[0056] Pooling layer 2: The input is the output feature map of convolution layer 2. Pooling layer 2 reduces the dimensionality of the deeper features extracted by convolution layer 2, further reducing the spatial size of the feature map while retaining the key features. The output is the feature map of pooling layer 2, providing a more compact and abstract feature representation for convolution layer 3.

[0057] Pooling layer n: The input is the output feature map of convolution layer n. As the last pooling layer, pooling layer n performs the final dimensionality reduction on the high-level features output by convolution layer n, and the output feature map is input to the fully connected layer 1 to provide more refined feature information for the final classification.

[0058] Fully connected layer 1: The input is the weighted feature map output by the last pooling layer (pooling layer n) or the attention module (if the attention module is used after the pooling layer). These feature maps are flattened and converted into one-dimensional vectors. The fully connected layer 1 performs linear transformation and nonlinear activation on these feature vectors to further extract the high-level semantic information of the features and outputs the feature vectors of the fully connected layer 1.

[0059] Fully connected layer 2: The input is the output feature vector of fully connected layer 1. Fully connected layer 2 continues to process and abstract the features, and the output is the feature vector of fully connected layer 2, which is closer to the final classification result.

[0060] The fully connected layer n can be used as the output layer, and the input is the output feature vector of the fully connected layer n-1. The output layer maps the feature vector to the final output category through activation functions such as softmax, that is, it determines whether the corresponding training sample video is a real sample video, a deep fake sample video, or an adversarial sample video. The output is the final detection result of the model, which is usually a probability value, for example, it can represent the probability that the input video is a deep fake video.

[0061] Optionally, the initial deep fake video identification model also includes an attention module, which can be set between the pooling layer and the fully connected layer, and the attention module is used to output the corresponding real sample weighted visual feature data according to the second real sample visual feature data, and output the corresponding deep fake sample weighted visual feature data according to the second deep fake sample visual feature data; similarly, the attention module is also used to output the corresponding adversarial sample weighted visual feature data according to the second adversarial sample visual feature data.

[0062] Assume that the feature map of the input attention module (the second real sample visual feature data) is A, and by calculating the attention weight w, the weighted feature map (the real sample weighted visual feature data) is output. =w·A. The attention mechanism can automatically focus on the most critical areas and features in the video frame for deepfake judgment, such as areas with abnormal facial expression changes and image synthesis boundaries. The process of calculating attention weights can be based on the self-attention mechanism or other advanced attention algorithms, such as determining weights by calculating the correlation between features. This mechanism significantly improves the model's ability to detect complex deepfake patterns and prevents the model from being disturbed by irrelevant information.

[0063] The fully connected layer outputs the third real sample visual feature data according to the real sample weighted visual feature data, and outputs the third deep fake sample visual feature data according to the deep fake sample weighted visual feature data. Similarly, the fully connected layer is also used to output the third adversarial sample visual feature data according to the adversarial sample weighted visual feature data. At this time, the output layer can output the corresponding video type discrimination results based on the third real sample visual feature data, the third deep fake sample visual feature data, and the third adversarial sample visual feature data.

[0064] Optionally, the initial deep fake video identification model also includes a feature fusion module, which is used to fuse the real sample video and audio feature data with the third real sample visual feature data to obtain real sample fusion feature data, and use the real sample fusion feature data as the input of the output layer; the feature fusion module is also used to fuse the deep fake sample video and audio feature data with the third deep fake sample visual feature data to obtain deep fake sample fusion feature data, and use the deep fake sample fusion feature data as the input of the output layer; the feature fusion module is also used to fuse the adversarial sample video and audio feature data with the third adversarial sample visual feature data to obtain adversarial sample fusion feature data, and use the adversarial sample fusion feature data as the input of the output layer. The output layer can output the video type discrimination result corresponding to each fusion feature data, that is, it can output whether the corresponding sample video is a real video, a deep fake video, or a video generated by an adversarial method.

[0065] Assume that the visual feature data (vector) is , the audio feature data (vector) is , the feature fusion module can use a splicing operation The two are fused in the appropriate dimension. The fused feature vector can more comprehensively capture the abnormal performance of deep fake videos in multimodal aspects, because deep fake videos often have flaws in the synchronization of audio and video, the authenticity of audio content, etc. Through the splicing operation, the visual feature vector and the audio feature vector are fused in the appropriate dimension, and the fused feature vector is output. This fused feature vector can more comprehensively capture the abnormal performance of deep fake videos in multimodal aspects, providing richer information for the final classification judgment.

[0066] During the training process of the embodiment of the present application, the parameters of the generator can be adjusted according to the difference between the video type discrimination result and the real label corresponding to the training sample feature data to obtain an updated generator; new adversarial sample video feature data is generated based on the updated generator; and the initial deep fake video identification model is trained using the new adversarial sample video feature data. Although the main purpose of the generator is to generate adversarial samples, in adversarial training, it is also necessary to update the parameters of the generator according to the feedback from the discriminator. The goal of the update is to enable the generator to generate adversarial samples that are more realistic and more difficult for the discriminator to recognize, thereby further improving the robustness of the discriminator.

[0067] Among them, the real sample video feature data and / or the deep fake sample video feature data in step S11 can be input into the updated generator to generate new adversarial sample video feature data. Of course, in some other embodiments, new real sample video feature data and / or new deep fake sample video features can also be re-acquired, and then the new real sample video feature data and / or the new deep fake sample video features can be input into the updated generator to generate new adversarial sample video feature data. During the training process, the generator is regularly used to generate new adversarial samples and added to the training set. These adversarial samples constantly challenge the recognition ability of the discriminator, prompting the discriminator to continuously learn and adjust to cope with more complex input situations.

[0068] It can be understood that in adversarial training, the discriminator, that is, the initial deep fake video identification model mentioned above, has the task of distinguishing between real sample videos, deep fake sample videos, and generated adversarial sample videos.

[0069] During the training process, the model needs to forward pass real data: the real sample video feature data is input into the discriminator, and the discriminator outputs the prediction result of the real sample video feature data. At the same time, the loss function, such as cross entropy loss, is calculated to measure the difference between the prediction result and the real label.

[0070] Forward pass of deep fake data: The deep fake sample video feature data is input into the discriminator, and the discriminator outputs the prediction result of the deep fake sample video feature data. At the same time, the loss function, such as cross entropy loss, is calculated to measure the difference between the prediction result and the deep fake label.

[0071] Forward pass to generate data: The adversarial sample video feature data generated by the generator is input into the discriminator, and the discriminator outputs the prediction result of the adversarial sample video feature data. The loss function is also calculated to measure the discriminator's ability to recognize adversarial samples.

[0072] Update the discriminator: Based on the loss functions of real data, deep fake data, and generated data, use an optimization algorithm (such as the Adam optimizer) to update the network parameters of the discriminator. The goal of the update is to enable the discriminator to better identify real video frames, deep fake video frames, and adversarial samples, and improve its robustness under various inputs.

[0073] Through adversarial training, the neural network model can be exposed to more diverse input samples during the training process, including normal samples, deep fake samples, and adversarial samples. This requires the model to not only focus on the characteristics of normal samples during the learning process, but also be able to identify small perturbations and abnormal features in adversarial samples. Therefore, when faced with videos generated by new deep fake technology or videos that have been slightly modified, the model can maintain a high detection accuracy rate, effectively cope with various complex input situations, and significantly improve the robustness and generalization ability of the model.

[0074] It is understandable that the preset training completion condition in step S14 can be flexibly set by the developer, for example, the number of training times reaches a preset iteration threshold, or the value of the target loss function is less than or equal to a preset loss threshold.

[0075] The present application also provides a method for identifying deep fake videos. Figure 2 As shown, including: S21: Obtain the video to be identified.

[0076] S22: Extracting video feature data of the video to be identified.

[0077] S23: Input the video feature data into a target deep fake video identification model to obtain an identification result of the video to be identified.

[0078] The video feature data in step S22 includes video visual feature data and / or video audio feature data.

[0079] The target deep fake video identification model in step S23 is a model obtained by the model training method introduced above.

[0080] After obtaining the identification result of the video to be identified, the method includes: When the video to be identified is displayed, prompt information of the identification result of the video to be identified is superimposed and displayed on the video to be identified.

[0081] It can be understood that the target deep fake video identification model in the embodiment of the present application can output an identification result for the video to be identified, for example, it can output an identification result that the video to be identified is a real video, or output an identification result that the video to be identified is a non-real video.

[0082] In some videos, some video clips may be real video clips, and other parts may be video clips synthesized using deep fake technology. In order to accurately identify such videos, in some other embodiments, an identification result can be output for each frame image of the video to be identified.

[0083] Specifically, each frame in the video to be identified can be labeled. If the model determines that a frame is a deep fake video frame, a corresponding prompt is given on the video playback interface, such as superimposing a warning mark on the video frame. Suppose the video frame is , warning sign is M, superimposed image = +M. At the same time, record the relevant information of the frame, including timestamp t, confidence c, etc., and store this information in the data structure R= In addition, the detection results can be output as log files to facilitate subsequent analysis and auditing. The content of the log file can be recorded in a certain format, such as recording one frame of information per line: t, c, label, where label indicates whether it is a real video frame or a deep fake video frame.

[0084] It should be understood that, although the various steps in the above-mentioned flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above-mentioned flow chart may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0085] Based on the same inventive concept, the present application embodiment provides an electronic device, see Figure 3 As shown, it includes a processor 301 and a memory 302, wherein the memory 302 stores a computer program, and the processor 301 executes the computer program to implement the steps of the method introduced above, which will not be repeated here.

[0086] The processor 301 may be an integrated circuit chip with signal processing capabilities. The processor 301 may be a general-purpose processor, including a CPU (central processing unit), an NP (network processor), etc.; it may also be a DSP (digital signal processor), an ASIC (application-specific integrated circuit), an FPGA (off-the-shelf programmable gate array), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It may implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0087] The memory 302 may include, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read Only Memory), EPROM (Erasable Read Only Memory), EEPROM (Electrically Erasable Read Only Memory), and the like.

[0088] This embodiment also provides a computer-readable storage medium, such as a floppy disk, a CD, a hard disk, a flash memory, a USB flash disk, an SD card, an MMC card, etc., in which one or more programs for implementing the above steps are stored. These one or more programs can be executed by one or more processors to implement the steps of the method in the above embodiment 1, which will not be repeated here.

[0089] It should be noted that the diagram provided in the present embodiment only illustrates the basic concept of the present invention in a schematic manner, so the diagram only shows the components related to the present invention rather than drawing according to the number, shape and size of the components during actual implementation. The type, quantity and ratio of each component during actual implementation can be a random change, and the component layout type may also be more complicated. The structure, ratio, size, etc. illustrated in the drawings of the present specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions that the present invention can implement, so they have no technical substantive significance. Any modification of the structure, change of the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the effect that the present invention can produce and the purpose that can be achieved. At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" quoted in this specification are only for the convenience of narration, and are not used to limit the scope of the present invention. The change or adjustment of its relative relationship should also be regarded as the scope of the present invention without substantially changing the technical content.

[0090] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A training method for a deep fake video identification model, characterized in that: include: Acquire training sample feature data and an initial deep fake video identification model; the training sample feature data includes real sample video feature data, deep fake sample video feature data and adversarial sample video feature data; the adversarial sample video feature data is video feature data generated by inputting the real sample video feature data and / or the deep fake sample video feature data into a generator; In each training process, the training sample feature data is input into the initial deep fake video identification model, and the initial deep fake video identification model outputs a corresponding video type discrimination result for the training sample feature data; Adjusting model parameters in the initial deep fake video identification model according to a difference between the video type discrimination result and a true label corresponding to the training sample feature data; When the preset training completion conditions are reached, the training is stopped to obtain the target deep fake video identification model.

2. The method for training a deep fake video identification model according to claim 1, wherein: The real sample video feature data includes real sample video visual feature data, and the deep fake sample video feature data includes deep fake sample video visual feature data; The step of obtaining training sample feature data includes: Obtain real sample videos and deep fake sample videos; Performing image preprocessing on the real sample video and the deep fake sample video; Image feature extraction is performed on the real sample video and the deep fake sample video after image preprocessing to obtain visual feature data of the real sample video and visual feature data of the deep fake sample video.

3. The method for training a deep fake video identification model as claimed in claim 2, characterized in that: The initial deep fake video identification model includes a convolutional layer, a pooling layer, a fully connected layer and an output layer; The convolution layer performs convolution operations on the input real sample video visual feature data and the deep fake sample video visual feature data respectively to obtain first real sample visual feature data and first deep fake sample visual feature data; The pooling layer performs pooling processing on the first real sample visual feature data and the first deep fake sample visual feature data respectively to obtain second real sample visual feature data and second deep fake sample visual feature data; The fully connected layer outputs corresponding third real sample visual feature data based on the second real sample visual feature data, and outputs corresponding third deep fake sample visual feature data based on the second deep fake sample visual feature data; The output layer is used to output a corresponding video type discrimination result based on the output of the fully connected layer.

4. The method for training a deep fake video identification model as claimed in claim 3, characterized in that: The initial deep fake video identification model further includes an attention module, the attention module being configured to output corresponding real sample weighted visual feature data according to the second real sample visual feature data, and to output corresponding deep fake sample weighted visual feature data according to the second deep fake sample visual feature data; The fully connected layer outputs the third real sample visual feature data according to the real sample weighted visual feature data, and outputs the third deep fake sample visual feature data according to the deep fake sample weighted visual feature data.

5. The method for training a deep fake video identification model as claimed in claim 3, characterized in that: The real sample video feature data also includes real sample video audio feature data, and the deep fake sample video feature data also includes deep fake sample video audio feature data; The initial deep fake video identification model also includes a feature fusion module; The step of obtaining training sample feature data includes: Extracting audio features from the real sample video and the deep fake sample video to obtain audio feature data of the real sample video and audio feature data of the deep fake sample video; The feature fusion module is used to fuse the real sample video and audio feature data with the third real sample visual feature data to obtain real sample fusion feature data, and use the real sample fusion feature data as the input of the output layer; the feature fusion module is also used to fuse the deep fake sample video and audio feature data with the third deep fake sample visual feature data to obtain deep fake sample fusion feature data, and use the deep fake sample fusion feature data as the input of the output layer.

6. The method for training a deep fake video identification model according to claim 1, wherein: The method further comprises: During the training process, according to the difference between the video type discrimination result and the real label corresponding to the training sample feature data, the parameters of the generator are adjusted to obtain an updated generator; Generate new adversarial sample video feature data based on the updated generator; The initial deep fake video identification model is trained using the new adversarial sample video feature data.

7. The method for training a deep fake video identification model according to claim 1, wherein: The generating new adversarial sample video feature data based on the updated generator includes: The real sample video feature data and / or the deep fake sample video feature data are input into the updated generator to generate new adversarial sample video feature data.

8. A method for identifying deep fake videos, characterized in that: include: Get the video to be identified; Extracting video feature data of the video to be identified; The video feature data is input into the training method of the deep fake video identification model as described in any one of claims 1-7 to obtain the identification result of the video to be identified.

9. The deep fake video identification method according to claim 8, characterized in that: After obtaining the identification result of the video to be identified, the method includes: When the video to be identified is displayed, prompt information of the identification result of the video to be identified is superimposed and displayed on the video to be identified.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1 to 9.

Images