Generative video authenticity identification method and device, equipment, medium and product

Abstract

The invention discloses a generative video authenticity identification method, apparatus and device, a medium and a product. The method comprises the steps of constructing a generative video authenticity identification model and a training set based on triple metric learning; performing end-to-end training optimization on the generative video authenticity identification model by using the training set to obtain a trained generative video authenticity identification model; obtaining a target video; and inputting the target video into the trained generative video authenticity identification model to obtain a authenticity detection result of the target video. According to the method, the feature difference between the real video and the generative video can be accurately captured through the constructed generative video authenticity identification model based on triple metric learning, and the identification precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video authenticity identification, and in particular to a generative video authenticity identification method, device, equipment, medium and product. Background Art

[0002] In recent years, the rapid development of generative models, particularly the technological advancements of generative adversarial networks (GANs), variational autoencoders (VAEs), and the more recently emerging diffusion models, has not only led to a surge in high-quality synthetic media content, but has also reached levels of authenticity so convincing that even humans struggle to discern the authenticity of many videos. With the widespread adoption of these technologies, a vast amount of synthetically generated video content has emerged online. If this content is maliciously used to create fake news, political propaganda, fraudulent activities, or manipulate social media, it could trigger serious economic and social crises. Therefore, research on generative video authentication technology is crucial.

[0003] Currently, authentication technologies for generative videos primarily include frame-based analysis models, models based on multimodal feature fusion, and approaches that learn holistic video features. Frame-based analysis models decompose videos into individual frames, then apply a specially designed deep learning model to each frame to identify authenticity. By analyzing each frame independently, this information is combined to reach a comprehensive decision. However, frame-based analysis models cannot fully detect subtle differences between samples, and many forged videos that require identification involve highly granular features such as faces, resulting in limited detection accuracy.

[0004] Among them, models based on multimodal feature fusion consider both video frames and audio signals, designing corresponding neural networks for feature extraction and recognition. This leverages the complementarity between audio and video data, providing a comprehensive basis for judgment after integrating multiple signal sources. However, these models struggle to integrate data from different modalities.

[0005] Holistic video feature learning uses neural networks to extract the overall features of a video and make judgments based on them. This approach can understand dynamic changes and temporal information in video streams, enabling identification to encompass dynamic behavior rather than relying solely on static information. However, this approach struggles to capture semantic information, resulting in poor model generalization. Summary of the Invention

[0006] In order to solve the technical problems existing in the background technology, the present invention proposes a generative video authenticity identification method, device, equipment, medium and product.

[0007] In a first aspect, the present invention proposes a generative video authenticity identification method based on positive and negative sample verification, comprising:

[0008] Construct a generative video authenticity verification model and training set based on triplet metric learning;

[0009] The generative video authenticity identification model is trained and optimized end-to-end using the training set to obtain a trained generative video authenticity identification model.

[0010] Get the target video;

[0011] Input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

[0012] Preferably, the generative video authenticity identification model includes a video semantic feature learning sub-model and a video authenticity detection sub-model; wherein, the video semantic feature learning sub-model includes a video frame extraction module, a pre-trained base module and a spacing feature recognition module; the video frame extraction module is used to extract video frames from the input video respectively; the pre-trained base module is used to perform feature encoding on each extracted video frame respectively; the spacing feature recognition module is used to perform semantic feature learning on each feature obtained by encoding and construct a high-discriminative feature space; the video authenticity detection sub-model is used to receive the feature vector output by the spacing feature recognition module, and judge the authenticity of the input video through the feature vector and the distribution relationship in the learned high-discriminative feature space.

[0013] Preferably, the pre-training base module is a 3D convolution module, the spacing feature recognition module is a feedforward neural network layer; and the video authenticity detection sub-model adopts a lightweight feedforward neural network structure.

[0014] Preferably, the training set includes a first training subset and a second training subset; wherein the first training subset includes multiple triplets, each triplet includes an anchor sample video clip, a positive sample video clip and a negative sample video clip; the second training subset includes training video clips with true and false labels.

[0015] Preferably, the generative video authenticity identification model is end-to-end trained and optimized using the training set to obtain a trained generative video authenticity identification model, specifically including:

[0016] Using the first training subset to train the video semantic feature learning sub-model to obtain a trained video semantic feature learning sub-model;

[0017] Using the trained video semantic feature learning sub-model, video semantic features are extracted from the second training subset to obtain a video semantic feature set;

[0018] The video semantic feature set is used to train the video authenticity detection sub-model to obtain a trained video authenticity detection sub-model;

[0019] The trained video semantic feature learning sub-model and the video authenticity detection sub-model are combined as the trained generative video authenticity identification model.

[0020] Preferably, the video semantic feature learning sub-model is trained using the first training subset to obtain a trained video semantic feature learning sub-model, specifically including:

[0021] The video frame extraction module extracts video frames from the anchor sample video clips, the positive sample video clips and the negative sample video clips respectively to obtain the anchor sample video frames, the positive sample video frames and the negative sample video frames;

[0022] The anchor sample video frames, positive sample video frames and negative sample video frames are feature-encoded through the pre-trained base module to obtain the anchor sample features, positive sample features and negative sample features;

[0023] The spacing feature recognition module performs feature learning on the anchor sample features, positive sample features, and negative sample features respectively to obtain the anchor sample feature vector, positive sample feature vector, and negative sample feature vector;

[0024] Calculate the Euclidean distance between the anchor point sample feature vector and the positive sample feature vector, as well as the Euclidean distance between the anchor point sample feature vector and the negative sample feature vector;

[0025] A triplet loss function is constructed, and the stochastic gradient descent algorithm is used to optimize the parameters of the video semantic feature learning sub-model until the triplet loss function converges to obtain a trained video semantic feature learning sub-model.

[0026] Preferably, the video semantic feature set is used to train the video authenticity detection sub-model to obtain a trained video authenticity detection sub-model, specifically including:

[0027] Input the video semantic features in the video semantic feature set into the video true / false detection sub-model to obtain the true / false detection result;

[0028] According to the true and false detection results and the corresponding true and false labels, a binary cross entropy loss function is established;

[0029] The video authenticity detection sub-model is optimized according to the binary cross entropy loss function until the video authenticity detection sub-model converges to obtain a trained video authenticity detection sub-model.

[0030] Preferably, the video authenticity detection sub-model is expressed as

[0031] f(y) = σ(ReLU(yWp + bp)Wc + bc);

[0032] In the formula, y represents the feature vector output by the spacing feature recognition module; f(y) represents the probability that the video in the input video true / false detection sub-model is a real video; σ(·) represents the Sigmoid activation function, which is used to map the output to the interval [0, 1]; ReLU represents the ReLU non-linear activation function; Wp and bp respectively represent the weight matrix and bias vector of the feature projection layer; Wc and bc respectively represent the weight matrix and bias vector of the classification layer.

[0033] Preferably, the binary cross-entropy loss function of the video true / false detection sub-model is

[0034] L ce = -[y label · log(f(y)) + (1 - y label )· log(1 - f(y))];

[0035] In the formula, y label ∈ {0, 1} represents the true / false label of the video in the input video true / false detection sub-model, 1 represents a real video, and 0 represents a generative forged video; f(y) represents the probability that the video output by the video true / false detection sub-model is a real video.

[0036] Preferably, the triplet loss function of the video semantic feature learning sub-model is

[0037] L(anchor, pos, neg) = max(0, m + d1 - d2);

[0038] In the formula, L(anchor, pos, neg) represents the triplet loss function, max() represents taking the maximum value, d1 represents the Euclidean distance between the anchor sample feature vector and the positive sample feature vector, d2 represents the Euclidean distance between the anchor sample feature vector and the negative sample feature vector, m is the threshold, anchor represents the anchor sample feature vector, pos represents the positive sample feature vector, and neg represents the negative sample feature vector;

[0039] Among them,

[0040] In the formula, represents the square of the L2 norm.

[0041] Preferably, according to the feature distance relationship, the triplet that meets the condition of d2 > d1 + m is recorded as a simple sample, the triplet that meets the condition of d1 < d2 < d1 + m is recorded as a semi-difficult sample, and the triplet that meets the condition of d2 < d1 is recorded as a difficult sample; the first training subset includes difficult samples or difficult samples and semi-difficult samples.

[0042] Preferably, the target video is input into a trained generative video authenticity identification model to obtain the authenticity detection result of the target video, specifically including:

[0043] Extract the target video frame through the video frame extraction module to obtain the target video frame;

[0044] The target video frame is feature-encoded through the pre-trained base module to obtain the target sample features;

[0045] The target sample features are learned through the spacing feature recognition module to obtain the target sample feature vector;

[0046] The target sample feature vector is input into the video authenticity detection sub-model to obtain the authenticity detection result of the target video.

[0047] In a second aspect, the present invention further proposes a generative video authenticity identification device based on positive and negative sample verification, comprising:

[0048] A construction unit for building a generative video authenticity identification model and training set based on triplet metric learning;

[0049] A training unit, configured to perform end-to-end training and optimization on the generative video authenticity identification model using the training set to obtain a trained generative video authenticity identification model;

[0050] An acquisition unit, used for acquiring a target video;

[0051] The authenticity identification unit is used to input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

[0052] In a third aspect, the present invention further proposes an electronic device comprising: a processor and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects is implemented.

[0053] In a fourth aspect, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects.

[0054] In a fifth aspect, the present invention further proposes a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects.

[0055] The proposed generative video authentication method, apparatus, device, medium, and product utilize a generative video authentication model based on triplet metric learning to introduce positive samples (real videos) and negative samples (generated videos) as reference standards, thereby constructing a comprehensive video authentication method. This method accurately captures the feature differences between real and generated videos by comparing the features of the three samples. This addresses the difficulty of existing technologies in identifying high-quality forged videos, providing a more reliable technical foundation for the authentication of generative videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a generative video authenticity identification method based on positive and negative sample verification in one embodiment of the present invention.

[0057] Figure 2 Schematic diagram of a generative video authenticity verification model in one embodiment of the present invention.

[0058] Figure 3 Schematic diagram of a video semantic feature learning sub-model in one embodiment of the present invention. DETAILED DESCRIPTION

[0059] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0060] First, refer to Figure 1 The present invention proposes a generative video authenticity identification method based on positive and negative sample verification, comprising:

[0061] Build a generative video authenticity verification model and training set based on triplet metric learning;

[0062] The generative video authenticity identification model is trained and optimized end-to-end using the training set to obtain a trained generative video authenticity identification model.

[0063] Get the target video;

[0064] Input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

[0065] This paper uses a generative video authentication model based on triplet metric learning to introduce positive samples (real videos) and negative samples (generated videos) as reference standards, thereby constructing a complete video authentication method. By comparing the features of the three samples, this video authentication method can accurately capture the feature differences between real videos and generated videos. This solves the problem that existing technologies have difficulty in identifying high-quality forged videos, and provides a more reliable technical guarantee for the authenticity authentication of generated videos.

[0066] like Figure 2 and Figure 3 As shown, in this embodiment, the generative video authenticity identification model includes a video semantic feature learning sub-model and a video authenticity detection sub-model; wherein, the video semantic feature learning sub-model includes a video frame extraction module, a pre-trained base module and a spacing feature recognition module; the video frame extraction module is used to extract video frames from the input video respectively; the pre-trained base module is used to perform feature encoding on each extracted video frame respectively; the spacing feature recognition module is used to perform semantic feature learning on each feature obtained by encoding, and construct a high-discriminative feature space; the video authenticity detection sub-model is used to receive the feature vector output by the spacing feature recognition module, and judge the authenticity of the input video through the feature vector and the distribution relationship in the learned high-discriminative feature space.

[0067] The generative video authenticity identification model constructed in this embodiment includes a video semantic feature learning sub-model and a video authenticity detection sub-model. The video semantic feature learning sub-model constructs a highly discriminative feature space based on a contrastive learning strategy to capture video feature differences. Compared with traditional methods, it is not limited to the binary decision of true or false video content, but deeply learns the essential distinguishing features between true and false videos. While maintaining high identification accuracy, it significantly improves the generalization ability of the generative video authenticity identification model.

[0068] Among them, the pre-training base module is a 3D convolution module (3D ConvNet), the spacing feature recognition module is a feedforward neural network layer (Feed Forward Neural Layer); the video authenticity detection sub-model adopts a lightweight feedforward neural network structure.

[0069] It is important to know that the structure of the video semantic feature learning sub-model is as follows Figure 2 shown.

[0070] Among them, the training set in this embodiment includes a first training subset and a second training subset; wherein, the first training subset includes multiple triplets, each triplet includes an anchor sample video clip, a positive sample video clip and a negative sample video clip; the second training subset includes training video clips with true and false labels.

[0071] It's important to note that in this example, the positive sample video clips are real video clips, the negative sample video clips are generated videos, and the anchor sample video clips are real video clips, serving as reference points for model learning. These three types of samples form a triplet. The training goal is to minimize the feature distance between the anchor sample and the positive sample, while maximizing the feature distance between the anchor sample and the negative sample, thereby learning to distinguish between real and generated videos.

[0072] The training set is used to perform end-to-end training optimization on the generative video authenticity identification model to obtain a trained generative video authenticity identification model, specifically including:

[0073] Using the first training subset to train the video semantic feature learning sub-model to obtain a trained video semantic feature learning sub-model;

[0074] Using the trained video semantic feature learning sub-model, video semantic features are extracted from the second training subset to obtain a video semantic feature set;

[0075] The video semantic feature set is used to train the video authenticity detection sub-model to obtain a trained video authenticity detection sub-model;

[0076] The trained video semantic feature learning sub-model and the video authenticity detection sub-model are combined as the trained generative video authenticity identification model.

[0077] This embodiment introduces positive samples (real videos) and negative samples (generated videos) as reference standards during the training phase of the video semantic feature learning sub-model, thereby constructing a complete video semantic feature learning method. By comparing the features of the three samples, this video semantic feature learning method can accurately capture the feature differences between real videos and generated videos. This solves the problem that existing technologies have difficulty identifying high-quality forged videos, and provides more reliable technical support for the subsequent authenticity verification of the video authenticity detection sub-model.

[0078] In a further embodiment, the video semantic feature learning sub-model is trained using the first training subset to obtain a trained video semantic feature learning sub-model, specifically including:

[0079] The video frame extraction module extracts video frames from the anchor sample video clips, the positive sample video clips and the negative sample video clips respectively to obtain the anchor sample video frames, the positive sample video frames and the negative sample video frames;

[0080] The anchor sample video frames, positive sample video frames and negative sample video frames are feature-encoded through the pre-trained base module to obtain the anchor sample features, positive sample features and negative sample features;

[0081] The spacing feature recognition module performs feature learning on the anchor sample features, positive sample features, and negative sample features respectively to obtain the anchor sample feature vector, positive sample feature vector, and negative sample feature vector;

[0082] Calculate the Euclidean distance between the anchor point sample feature vector and the positive sample feature vector, as well as the Euclidean distance between the anchor point sample feature vector and the negative sample feature vector;

[0083] A triplet loss function is constructed, and the stochastic gradient descent algorithm is used to optimize the parameters of the video semantic feature learning sub-model until the triplet loss function converges to obtain a trained video semantic feature learning sub-model.

[0084] This embodiment uses a triplet loss function during training to enable the video semantic feature learning sub-model to learn the differences and similarities between the anchor sample and the two differential inputs, making the distance d1 between the anchor sample and the positive sample closer and the distance d2 between the anchor sample and the negative sample farther, thereby achieving a precise partitioning of the video feature space. This innovative architecture enables the video semantic feature learning sub-model to learn the subtle differences between real and generated videos while retaining temporal information. It is particularly suitable for videos containing fine-grained features such as faces, overcoming the difficulties of multimodal feature fusion and the poor generalization ability of overall video feature learning, making it simpler and more efficient.

[0085] This embodiment uses the L2 norm to calculate the distance between different sample feature variables; wherein: the distance d1 between the anchor sample feature vector and the positive sample feature vector is expressed as: The distance d2 between the anchor sample feature vector and the negative sample feature vector is expressed as: Among them, anchor represents the anchor sample feature vector, pos represents the positive sample feature vector, and neg represents the negative sample feature vector. Represents the square of the L2 norm.

[0086] During the actual training process, in order to prevent the model from taking shortcuts and training the positive and negative sample features to be close enough to meet the constraints of the loss function, the objective function of the video semantic feature learning sub-model in this embodiment is a triplet loss function, and the triplet loss function is set to make the difference between the distance d1 between the anchor point sample feature vector and the positive sample feature vector and the distance d2 between the anchor point sample feature vector and the negative sample feature vector greater than the threshold m.

[0087] Among them, the triplet loss function is \(L(anchor,pos,neg)=\max(0,m + d1 - d2)\); in the formula, \(L(anchor,pos,neg)\) represents the triplet loss function, \(\max()\) represents taking the maximum value, \(d1\) represents the Euclidean distance between the anchor sample feature vector and the positive sample feature vector, \(d2\) represents the Euclidean distance between the anchor sample feature vector and the negative sample feature vector, \(m\) is the threshold, \(anchor\) represents the anchor sample feature vector, \(pos\) represents the positive sample feature vector, and \(neg\) represents the negative sample feature vector. Among them, \(m\) is set to 0.1 during the training process of this model.

[0088] In this embodiment, the triplets that meet the condition \(d2 > d1 + m\) are denoted as simple samples, the triplets that meet the condition \(d1 < d2 < d1 + m\) are denoted as semi-difficult samples, and the triplets that meet the condition \(d2 < d1\) are denoted as difficult samples. The triplets in the first training subset in this embodiment preferentially select difficult samples or combinations of difficult samples and semi-difficult samples to improve the sensitivity of the video semantic feature learning sub-model to visual detail features, and can gradually establish sensitivity to the subtle feature differences between real videos and generated videos, especially performing more excellently in the task of video authenticity identification containing high-detail features such as faces.

[0089] To train the true and false detection ability of the trained video true and false detection sub-model, in a further embodiment, the video true and false detection sub-model is trained using the video semantic feature set to obtain the trained video true and false detection sub-model, specifically including:

[0090] Input the video semantic features in the video semantic feature set into the video true and false detection sub-model to obtain the true and false detection result;

[0091] According to the true and false detection result and the corresponding true and false label, establish a binary cross-entropy loss function;

[0092] Optimize the video true and false detection sub-model according to the binary cross-entropy loss function until the video true and false detection sub-model converges to obtain the trained video true and false detection sub-model.

[0093] The video true and false detection sub-model in this embodiment is expressed as

[0094] \(f(y)=\sigma(ReLU(yW_p + b_p)W_c + b_c)\);

[0095] Where y represents the feature vector output by the spacing feature recognition module; f(y) represents the probability that the video in the input video true / false detection sub-model is a real video; σ(·) represents the Sigmoid activation function, which is used to map the output to the interval [0, 1]; ReLU represents the ReLU nonlinear activation function; Wp and bp represent the weight matrix and bias vector of the feature projection layer, respectively; Wc and bc represent the weight matrix and bias vector of the classification layer, respectively.

[0096] Specifically, the binary cross entropy loss function of the video authenticity detection sub-model is

[0097] L ce =-[y label ·log(f(y))+(1-y label )·log(1-f(y))];

[0098] Where y label ∈{0,1} represents the true or false label of the video in the input video true or false detection sub-model, 1 represents a real video, and 0 represents a generated forged video; f(y) represents the probability that the video output by the video true or false detection sub-model is a real video.

[0099] Therefore, in this embodiment, the target video is input into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video, which specifically includes:

[0100] Extract the target video frame through the video frame extraction module to obtain the target video frame;

[0101] The target video frame is feature-encoded through the pre-trained base module to obtain the target sample features;

[0102] The target sample features are learned through the spacing feature recognition module to obtain the target sample feature vector;

[0103] The target sample feature vector is input into the video authenticity detection sub-model to obtain the authenticity detection result of the target video.

[0104] In a second aspect, the present invention further proposes a generative video authenticity identification device based on positive and negative sample verification, comprising:

[0105] A construction unit for building a generative video authenticity identification model and training set based on triplet metric learning;

[0106] A training unit, configured to perform end-to-end training and optimization on the generative video authenticity identification model using the training set to obtain a trained generative video authenticity identification model;

[0107] An acquisition unit, used for acquiring a target video;

[0108] The authenticity identification unit is used to input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

[0109] Among them, the generative video authenticity identification model includes a video semantic feature learning sub-model and a video authenticity detection sub-model; among them, the video semantic feature learning sub-model includes a video frame extraction module, a pre-trained base module and a spacing feature recognition module; the video frame extraction module is used to extract video frames from the input video respectively; the pre-trained base module is used to perform feature encoding on each extracted video frame respectively; the spacing feature recognition module is used to learn semantic features of each encoded feature and construct a high-discriminative feature space; the video authenticity detection sub-model is used to receive the feature vector output by the spacing feature recognition module, and judge the authenticity of the input video through the feature vector and the distribution relationship in the learned high-discriminative feature space.

[0110] The training set in this embodiment includes a first training subset and a second training subset; wherein the first training subset includes multiple triplets, each triplet includes an anchor sample video clip, a positive sample video clip and a negative sample video clip; the second training subset includes training video clips with true and false labels.

[0111] Among them, the training process of the generative video authenticity identification model in this embodiment specifically includes: using the first training subset to train the video semantic feature learning sub-model to obtain a trained video semantic feature learning sub-model; using the trained video semantic feature learning sub-model to extract video semantic features from the second training subset to obtain a video semantic feature set; using the video semantic feature set to train the video authenticity detection sub-model to obtain a trained video authenticity detection sub-model; combining the trained video semantic feature learning sub-model and the video authenticity detection sub-model as a trained generative video authenticity identification model.

[0112] In a further embodiment, the video semantic feature learning sub-model is trained using the first training subset to obtain a trained video semantic feature learning sub-model, specifically including:

[0113] The video frame extraction module extracts video frames from the anchor sample video clips, the positive sample video clips and the negative sample video clips respectively to obtain the anchor sample video frames, the positive sample video frames and the negative sample video frames;

[0114] The anchor sample video frames, positive sample video frames and negative sample video frames are feature-encoded through the pre-trained base module to obtain the anchor sample features, positive sample features and negative sample features;

[0115] The spacing feature recognition module performs feature learning on the anchor sample features, positive sample features, and negative sample features respectively to obtain the anchor sample feature vector, positive sample feature vector, and negative sample feature vector;

[0116] Calculate the Euclidean distance between the anchor point sample feature vector and the positive sample feature vector, as well as the Euclidean distance between the anchor point sample feature vector and the negative sample feature vector;

[0117] A triplet loss function is constructed, and the stochastic gradient descent algorithm is used to optimize the parameters of the video semantic feature learning sub-model until the triplet loss function converges to obtain a trained video semantic feature learning sub-model.

[0118] In a further embodiment, a video authenticity detection sub-model is trained using a video semantic feature set to obtain a trained video authenticity detection sub-model, specifically comprising: inputting the video semantic features in the video semantic feature set into the video authenticity detection sub-model to obtain a authenticity detection result; establishing a binary cross entropy loss function based on the authenticity detection result and the corresponding authenticity label; optimizing the video authenticity detection sub-model based on the binary cross entropy loss function until the video authenticity detection sub-model converges to obtain a trained video authenticity detection sub-model.

[0119] The video authenticity detection sub-model in this embodiment is expressed as

[0120] f(y)=σ(ReLU(yWp+bp)Wc+bc);

[0121] Where y represents the feature vector output by the spacing feature recognition module; f(y) represents the probability that the video in the input video true / false detection sub-model is a real video; σ(·) represents the Sigmoid activation function, which is used to map the output to the interval [0, 1]; ReLU represents the ReLU nonlinear activation function; Wp and bp represent the weight matrix and bias vector of the feature projection layer, respectively; Wc and bc represent the weight matrix and bias vector of the classification layer, respectively.

[0122] Among them, the binary cross entropy loss function of the video authenticity detection sub-model in this embodiment is

[0123] L ce =-[y label ·log(f(y))+(1-y label )·log(1-f(y))];

[0124] Where y label ∈{0,1} represents the true or false label of the video in the input video true or false detection sub-model, 1 represents a real video, and 0 represents a generated forged video; f(y) represents the probability that the video output by the video true or false detection sub-model is a real video.

[0125] Among them, the triplet loss function of the video semantic feature learning sub-model is

[0126] L(anchor,pos,neg)=max(0,m+d1-d2);

[0127] Where L(anchor,pos,neg) represents the triplet loss function, max() represents the maximum value, d1 represents the distance between the anchor point sample feature vector and the positive sample feature vector, d2 represents the distance between the anchor point sample feature vector and the negative sample feature vector, and m is the threshold;

[0128] in,

[0129] Where d1 represents the distance between the anchor sample feature vector and the positive sample feature vector, d2 represents the distance between the anchor sample feature vector and the negative sample feature vector, anchor represents the anchor sample feature vector, pos represents the positive sample feature vector, and neg represents the negative sample feature vector. Represents the square of the L2 norm.

[0130] Among them, the detection process of the target video in this embodiment includes: extracting the video frame of the target video through the video frame extraction module to obtain the target video frame; encoding the feature of the target video frame through the pre-training base module to obtain the target sample feature; learning the feature of the target sample through the spacing feature recognition module to obtain the target sample feature vector; inputting the target sample feature vector into the video true and false detection sub-model to obtain the true and false detection result of the target video.

[0131] In a third aspect, the present invention further proposes an electronic device comprising: a processor and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects is implemented.

[0132] In a fourth aspect, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects.

[0133] In a fifth aspect, the present invention further proposes a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the generative video authenticity identification method based on positive and negative sample verification as described in any one of the first aspects.

[0134] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A generative video authenticity identification method based on positive and negative sample verification, characterized in that: This includes building a generative video authenticity verification model and training set based on triplet metric learning; The generative video authenticity identification model is trained and optimized end-to-end using the training set to obtain a trained generative video authenticity identification model. Get the target video; Input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

2. The generative video authenticity identification method based on positive and negative sample verification according to claim 1 is characterized in that: The generative video authenticity assessment model includes a video semantic feature learning sub-model and a video authenticity detection sub-model; wherein the video semantic feature learning sub-model includes a video frame extraction module, a pre-trained base module and a spacing feature recognition module; the video frame extraction module is used to extract video frames from the input video; the pre-trained base module is used to perform feature encoding on each extracted video frame; the spacing feature recognition module is used to learn semantic features of each encoded feature and construct a high-discriminative feature space; the video authenticity detection sub-model is used to receive the feature vector output by the spacing feature recognition module and judge the authenticity of the input video through the feature vector and the distribution relationship in the learned high-discriminative feature space; Preferably, the pre-training base module is a 3D convolution module, the spacing feature recognition module is a feedforward neural network layer; and the video authenticity detection sub-model adopts a lightweight feedforward neural network structure.

3. The generative video authenticity identification method based on positive and negative sample verification according to claim 2 is characterized in that: The training set includes a first training subset and a second training subset; wherein the first training subset includes a plurality of triplets, each triplet includes an anchor sample video clip, a positive sample video clip, and a negative sample video clip; the second training subset includes training video clips with true and false labels; Preferably, the generative video authenticity identification model is end-to-end trained and optimized using the training set to obtain a trained generative video authenticity identification model, specifically including: Using the first training subset to train the video semantic feature learning sub-model to obtain a trained video semantic feature learning sub-model; Using the trained video semantic feature learning sub-model, video semantic features are extracted from the second training subset to obtain a video semantic feature set; The video semantic feature set is used to train the video authenticity detection sub-model to obtain a trained video authenticity detection sub-model; The trained video semantic feature learning sub-model and the video authenticity detection sub-model are combined as the trained generative video authenticity identification model; Preferably, the video semantic feature learning sub-model is trained using the first training subset to obtain a trained video semantic feature learning sub-model, specifically including: The video frame extraction module extracts video frames from the anchor sample video clips, the positive sample video clips and the negative sample video clips respectively to obtain the anchor sample video frames, the positive sample video frames and the negative sample video frames; The anchor sample video frames, positive sample video frames and negative sample video frames are feature-encoded through the pre-trained base module to obtain the anchor sample features, positive sample features and negative sample features; The spacing feature recognition module performs feature learning on the anchor sample features, positive sample features, and negative sample features respectively to obtain the anchor sample feature vector, positive sample feature vector, and negative sample feature vector; Calculate the Euclidean distance between the feature vector of the anchor sample and the feature vector of the positive sample, and the Euclidean distance between the feature vector of the anchor sample and the feature vector of the negative sample; Construct a triplet loss function, and use the stochastic gradient descent algorithm to optimize the parameters of the video semantic feature learning sub-model until the triplet loss function converges, and obtain the trained video semantic feature learning sub-model; Preferably, use the video semantic feature set to train the video authenticity detection sub-model to obtain the trained video authenticity detection sub-model, specifically including: Input the video semantic features in the video semantic feature set into the video authenticity detection sub-model to obtain the authenticity detection result; According to the authenticity detection result and the corresponding authenticity label, establish a binary cross-entropy loss function; Optimize the video authenticity detection sub-model according to the binary cross-entropy loss function until the video authenticity detection sub-model converges, and obtain the trained video authenticity detection sub-model.

4. The generative video authenticity identification method based on positive and negative sample verification according to claim 2 or 3, characterized in that: The video authenticity detection sub-model is expressed as f(y) = σ(ReLU(yWp + bp)Wc + bc); In the formula, y represents the feature vector output by the spacing feature recognition module; f(y) represents the probability that the video input into the video authenticity detection sub-model is a real video; σ(·) represents the Sigmoid activation function, which is used to map the output to the interval [0,1]; ReLU represents the ReLU non-linear activation function; Wp and bp respectively represent the weight matrix and bias vector of the feature projection layer; Wc and bc respectively represent the weight matrix and bias vector of the classification layer; Preferably, the binary cross-entropy loss function of the video authenticity detection sub-model is L ce =-[y label ·log(f(y))+(1-y label )·log(1-f(y))]; Where y label ∈{0,1} represents the true or false label of the video in the input video true or false detection sub-model, 1 represents the real video, and 0 represents the generated fake video; f(y) represents the probability that the video output by the video authenticity detection sub-model is a real video.

5. The generative video authenticity identification method based on positive and negative sample verification according to claim 2 or 3, characterized in that: The triplet loss function of the video semantic feature learning sub-model is L(anchor, pos, neg) = max(0, m + d1 - d2); In the formula, L(anchor, pos, neg) represents the triplet loss function, max() represents taking the maximum value, d1 represents the Euclidean distance between the feature vector of the anchor sample and the feature vector of the positive sample, d2 represents the Euclidean distance between the feature vector of the anchor sample and the feature vector of the negative sample, m is the threshold, anchor represents the feature vector of the anchor sample, pos represents the feature vector of the positive sample, and neg represents the feature vector of the negative sample; in, Where, represents the square of L2 norm; Preferably, according to the feature distance relationship, the triplets that meet the condition of d2 > d1 + m are recorded as simple samples, the triplets that meet the condition of d1 < d2 < d1 + m are recorded as semi-difficult samples, and the triplets that meet the condition of d2 < d1 are recorded as difficult samples; The first training subset includes difficult samples or difficult samples and semi-difficult samples.

6. The generative video authenticity identification method based on positive and negative sample verification according to claim 2 is characterized in that: Input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video, specifically including: Extract video frames from the target video through the video frame extraction module to obtain target video frames; Perform feature encoding on the target video frames through the pre-trained basis module to obtain target sample features; Perform feature learning on the target sample features through the spacing feature recognition module to obtain target sample feature vectors; The target sample feature vector is input into the video authenticity detection sub-model to obtain the authenticity detection result of the target video.

7. A generative video authenticity identification device based on positive and negative sample verification, characterized in that: include: A construction unit for building a generative video authenticity identification model and training set based on triplet metric learning; A training unit, configured to perform end-to-end training and optimization on the generative video authenticity identification model using the training set to obtain a trained generative video authenticity identification model; An acquisition unit, used for acquiring a target video; The authenticity identification unit is used to input the target video into the trained generative video authenticity identification model to obtain the authenticity detection result of the target video.

8. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store one or more programs; When one or more programs are executed by a processor, the generative video authenticity identification method based on positive and negative sample verification as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the generative video authenticity identification method based on positive and negative sample verification described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the generative video authenticity identification method based on positive and negative sample verification described in any one of claims 1 to 6 are implemented.

Images