Video authenticity detection method based on GAAE model
By applying the GAAE model in the authenticity of the video, combined with positive and reverse video playback, the difficulty of detecting the difficulty of detection in existing technology has achieved more efficient and accurate false video detection.
Patent Information
- Application Number
- CN202411331696.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2025-01-28
AI Technical Summary
Existing video authenticity detection methods are difficult to effectively detect state-of-the-art fake videos, and deep learning-based methods require a large amount of labeled data and are susceptible to overfitting, and fail to fully consider the consistency of videos.
Put forward a video authentic detection method based on the GAAE model. By collecting real videos and AI generating videos as training data, the GAAE model is used for training. Forgery data generation and identification with the discriminator, and eventually the authenticity identification is performed through positive and reverse video playback.
Through the positive and reverse playback of the video, combining the characteristics of AI generating video, it can more comprehensively analyze the authenticity of the video, improve the accuracy and efficiency of false video detection, and avoid overfitting and data labeling problems.
Smart Images
Figure CN119380233A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of artificial intelligence and deep learning, and mainly to a video authenticity detection method based on the GAAE model. Background Art
[0002] With the popularity of ChatGPT, AI has penetrated deeper into the lives of most people. AIGC has also expanded from AI articles and AI pictures to AI videos and AI digital people. At the same time, AI can be used to create false video content, leading to the spread of false information and misleading the public. Using AI video technology, someone's face can be replaced with someone else's face to create a false video. This may lead to problems such as privacy leakage and identity theft.
[0003] The existing methods for detecting the authenticity of videos include: methods based on visual features and methods based on deep learning, but both methods have inevitable defects: in the method based on visual features, high-quality forged videos may not leave obvious quality traces, making it difficult for this method to detect the most advanced forgeries. In the method based on deep learning, a large amount of labeled data is required to train the model, and this data may not be easy to obtain or require a lot of manual labeling. The model may be affected by overfitting, resulting in poor performance on unseen data. In addition, the detection methods of the existing technology are more focused on the authenticity detection of video frames, without considering the coherence of the entire video; therefore, this paper proposes a method of reversing the video, which can better distinguish the authenticity of the video through the changes before and after the video frame, and effectively improve the detection of fake videos. Summary of the invention
[0004] In response to the problems existing in the current prior art, this application proposes a video authenticity detection method based on the GAAE model.
[0005] According to one aspect of the present invention, a video authenticity detection method based on the GAAE model is proposed, comprising:
[0006] S1. Collect real video samples and AI-generated video samples as training data sets, wherein the forward playback video and the reverse playback video of the real video samples are used as positive samples, and the forward playback video and the reverse playback video of the AI-generated video samples are used as negative samples;
[0007] S2. Using the training data set as input data of the GAAE model, further performing GAAE model training. The model training is specifically as follows:
[0008] S2.1, the GAAE model maps the input data to a latent space through an encoder, samples latent variables from the latent space, and reconstructs the input data using the latent variables through a decoder to obtain a reconstruction result, wherein the reconstruction loss and KL divergence are used for optimization training;
[0009] S2.2, using the real video sample and the reconstruction result in combination with a generator to generate forged data, using the forged data to train a discriminator, the discriminator maximizes the prediction probability of the real video sample and minimizes the prediction probability of the forged data, and outputs the discriminator result, wherein the discriminator loss function is continuously optimized and converged;
[0010] S2.3, using the discriminator result as feedback, adjusting the parameters of the generator to maximize the probability that the generated forged data is judged by the discriminator as a real video sample, wherein the generator loss function is continuously optimized and converged;
[0011] S3. Process the video to be detected into a forward playback video and a reverse playback video, and input them into the trained GAAE model for authenticity identification. If the similarity between the two output results is greater than 99%, the video is a real video, otherwise, the video is a fake video.
[0012] Furthermore, the encoder operation step of mapping the input data to the latent space specifically includes: the input data is processed by the encoder neural network to output a mean vector and a variance vector, which define a Gaussian latent variable distribution for generating latent variables.
[0013] The specific formula for processing the encoder neural network is as follows:
[0014]
[0015] in, represents the forward propagation function of the encoder neural network, represents the parameters of the encoder, and x represents the input data.
[0016] The specific formula of Gaussian latent variable distribution is as follows:
[0017]
[0018] Among them, z represents the latent variable, I represents the unit matrix of the same dimension as μ, μ represents the mean vector, σ 2 Represents the variance vector, N represents the mean μ and the variance σ 2 The multivariate normal distribution of .
[0019] Furthermore, the specific formula for obtaining hidden variables by sampling from the latent space is expressed as follows:
[0020] z=μ+σ×ε
[0021] Among them, z represents the latent variable, μ represents the mean vector, σ represents the standard deviation vector, and ε represents the random noise sampled from the standard normal distribution.
[0022] Furthermore, the reconstruction loss uses the average error as a metric, and the specific formula for optimizing the training using the reconstruction loss and the KL divergence is as follows:
[0023] L=L rec +L KL
[0024]
[0025] Among them, D represents the dimension of the latent variable, x represents the input data, and x i represents the i-th dimension in the input data, Reconstruction data. represents the i-th dimension in the reconstructed data, σ 2 represents the variance vector, μ represents the mean vector, L rec represents the reconstruction loss function, L KL represents the KL divergence function.
[0026] During the training process, the network parameters are updated by minimizing the ELBO objective function. Common optimization algorithms include stochastic gradient descent (SGD) and Adam. After the training is completed, the latent variables are obtained by sampling from the latent space, and the decoder is used to restore the new data samples.
[0027] Furthermore, the generator generates forged data specifically including: the generator is composed of a convolution layer and an activation function, maps the data vector in the reconstruction result to the real space under the real video sample, and generates a realistic sample, thereby generating forged data.
[0028] Furthermore, the discriminator is composed of a convolutional layer and an activation function, which extracts features from the input sample and outputs a probability, wherein the probability represents the probability that the input sample is a true sample.
[0029] Furthermore, the adjustment of the parameters of the generator is updated by an optimization algorithm in the optimizer, and the optimization algorithm is stochastic gradient descent. The specific formula for updating the parameters in the model is as follows:
[0030]
[0031] Among them, θ trepresents the parameter vector of the model at the tth iteration, η represents the learning rate, It means that the objective function J has parameter θ t The gradient of θ at θ, J represents the objective function, m j represents the input features of the jth training sample, n j represents the target value of the jth training sample.
[0032] The above steps are repeated alternately, and training is repeated in the process of continuous convergence. GAAE model training is a dynamic process in which the generator and discriminator continuously optimize their respective loss functions to improve the quality of generated data and the accuracy of discrimination.
[0033] According to a second aspect of the present invention, a computer-readable storage medium is provided, on which one or more computer programs are stored. When the one or more computer programs are executed by a computer processor, the above method is implemented.
[0034] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:
[0035] 1) Through forward and reverse reasoning of the video, combined with AI-generated video, more consideration is given to generation, and the abnormal situations that may appear in the reverse video are compared. The authenticity of the video is discovered from both positive and negative angles, which enables a more comprehensive analysis of the authenticity of the video.
[0036] 2) For the original video, whether it is played forward or reverse, it can be in a continuous state. However, for the AI-synthesized video, the next frame is predicted based on the previous frame, which means that only the normal forward playback is considered, and the reverse playback will be ignored, resulting in abnormalities in the reverse predicted picture frame. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate the embodiments and are used together with the description to explain the principles of the present invention. It will be easy to recognize other embodiments and many expected advantages of the embodiments because they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale with each other. The same reference numerals refer to corresponding similar parts.
[0038] Figure 1 A schematic diagram of a process flow of a reverse video authenticity detection method based on a GAAE model according to an embodiment of the present invention is shown;
[0039] Figure 2 A schematic diagram of the GAAE model training process according to a specific embodiment of the present invention is shown;
[0040] Figure 3 A schematic diagram of a video identification process according to a specific embodiment of the present invention is shown. DETAILED DESCRIPTION
[0041] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0043] Figure 1 FIG. 4 shows a flow chart of a reverse video authenticity detection method based on a GAAE model according to an embodiment of the present invention. Figure 1 As shown:
[0044] S1. Collect real video samples and AI-generated video samples as training data sets, wherein the forward playback video and the reverse playback video of the real video samples are used as positive samples, and the forward playback video and the reverse playback video of the AI-generated video samples are used as negative samples;
[0045] S2. Using the training data set as the input data of the GAAE model, further training the GAAE model. The model training is as follows: Figure 2 As shown, the specific steps are as follows:
[0046] S2.1, the GAAE model maps the input data to a latent space through an encoder, samples latent variables from the latent space, and reconstructs the input data using the latent variables through a decoder to obtain a reconstruction result, wherein the reconstruction loss and KL divergence are used for optimization training;
[0047] The encoder operation step of mapping the input data to the latent space specifically includes: the input data is processed by the encoder neural network, and a mean vector and a variance vector are output, which define a Gaussian latent variable distribution for generating latent variables.
[0048] The specific formula for processing the encoder neural network is as follows:
[0049]
[0050] in, represents the forward propagation function of the encoder neural network, represents the parameters of the encoder, and x represents the input data.
[0051] The specific formula of Gaussian latent variable distribution is as follows:
[0052]
[0053] Among them, z represents the latent variable, I represents the unit matrix of the same dimension as μ, μ represents the mean vector, σ 2 Represents the variance vector, N represents the mean μ and the variance σ 2 The multivariate normal distribution of .
[0054] The specific formula for obtaining hidden variables by sampling from the latent space is as follows:
[0055] z=μ+σ×ε
[0056] Among them, z represents the latent variable, μ represents the mean vector, σ represents the standard deviation vector, and ε represents the random noise sampled from the standard normal distribution.
[0057] The encoder compresses the input data into a latent variable vector, which contains the potential representation of the input data. The decoder recovers the approximation of the original data based on this latent variable vector.
[0058] The decoder neural network takes the latent variables as input and outputs a probability distribution of the reconstructed original data. By maximizing the reconstruction probability, the model learns an efficient way to sample from the data distribution.
[0059] The reconstruction loss uses the average error as a metric, and the specific formula for optimizing the training using the reconstruction loss and KL divergence is as follows:
[0060] L=L rec +L KL
[0061]
[0062] Among them, D represents the dimension of the latent variable, x represents the input data, and x i represents the i-th dimension in the input data, Reconstruction data. represents the i-th dimension in the reconstructed data, σ 2 represents the variance vector, μ represents the mean vector, L rec represents the reconstruction loss function, L KL represents the KL divergence function.
[0063] The reconstruction loss measures the difference between the data output by the decoder and the original data, and the mean square error (MSE) is used as a measure of the reconstruction loss. The KL divergence measures the difference between the distribution of latent variables and the Gaussian distribution of the specified latent space. By optimizing the KL divergence, the model attempts to make the latent variables close to the Gaussian distribution, thereby ensuring the representation ability of the latent space.
[0064] During the training process, VAE updates the network parameters by minimizing the ELBO objective function. Common optimization algorithms include stochastic gradient descent (SGD) and Adam. After the training is completed, we can obtain the hidden variables by sampling from the latent space and use the decoder to restore the new data samples.
[0065] S2.2, using the real video sample and the reconstruction result in combination with a generator to generate forged data, using the forged data to train a discriminator, the discriminator maximizes the prediction probability of the real video sample and minimizes the prediction probability of the forged data, and outputs the discriminator result, wherein the discriminator loss function is continuously optimized and converged;
[0066] The generator generates forged data specifically including: the generator is composed of a convolution layer and an activation function, maps the data vector in the reconstruction result to the real space under the real video sample, and generates a realistic sample, thereby generating forged data.
[0067] The discriminator is composed of a convolutional layer and an activation function, extracts features from the input sample and outputs a probability, which represents the probability that the input sample is a true sample.
[0068] S2.3, using the discriminator result as feedback, adjusting the parameters of the generator to maximize the probability that the generated forged data is judged by the discriminator as a real video sample, wherein the generator loss function is continuously optimized and converged;
[0069] The adjustment of the parameters of the generator is updated by an optimization algorithm in the optimizer, and the optimization algorithm is stochastic gradient descent. The specific formula for updating the parameters in the model is as follows:
[0070]
[0071] Among them, represents the parameter vector of the model at the t-th iteration, represents the learning rate, represents the gradient of the objective function J at the parameter, J represents the objective function, represents the input feature of the j-th training sample, and represents the target value of the j-th training sample.
[0072] S3. Process the video to be detected into a forward playback video and a reverse playback video, and input them into the trained GAAE model for authenticity identification. If the similarity between the two output results is greater than 99%, the video is a real video, otherwise, the video is a fake video.
[0073] like Figure 3 As shown, when detecting a video, the forward playback video frame of the video is used as input to the GAAE model to obtain the generated intermediate frame; in fact, the reverse playback video frame of the video is used as input to the GAAE model to obtain the generated intermediate frame, and the images obtained in these two different directions are verified and compared. If the similarity is close to 100%, it means that it is a real video. If there is an obvious difference, it may indicate that the video is fake.
[0074] The divided training set is processed into symbols needed by the machine and input into the GAAE model for deep learning training to obtain the model results. The model results are then verified and the verification set is used to verify the model effect and identify the final results.
[0075] Combining the characteristics of the samples and utilizing the features of the original video, smooth and coherent intermediate frames can be obtained regardless of whether the intermediate frames are obtained from the forward input sample picture frames or the reverse input sample picture frames. In essence, the intermediate frames generated forward and the intermediate frames generated reversely correspond to the same picture, and the similarity is close to 100%. However, if it is an AI-synthesized video, only the continuity of the forward playback is considered, and it is difficult to consider the reasonable continuity of the reverse playback. Therefore, the intermediate frames obtained from the forward input samples and the intermediate frames obtained from the reverse input are difficult to match, resulting in low similarity.
[0076] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0077] The modules involved in the embodiments of the present application may be implemented by software or by hardware.
[0078] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.
Claims
1. A video authenticity detection method based on the GAAE model, characterized in that: The following steps are involved: S1. Collect real video samples and AI-generated video samples as training data sets, wherein the forward playback video and the reverse playback video of the real video samples are used as positive samples, and the forward playback video and the reverse playback video of the AI-generated video samples are used as negative samples; S2. Using the training data set as input data of the GAAE model, further performing GAAE model training. The model training is specifically as follows: S2.1, the GAAE model maps the input data to a latent space through an encoder, samples latent variables from the latent space, and reconstructs the input data using the latent variables through a decoder to obtain a reconstruction result, wherein the reconstruction loss and KL divergence are used for optimization training; S2.2, using the real video sample and the reconstruction result in combination with a generator to generate forged data, using the forged data to train a discriminator, the discriminator maximizes the prediction probability of the real video sample and minimizes the prediction probability of the forged data, and outputs the discriminator result, wherein the discriminator loss function is continuously optimized and converged; S2.3, using the discriminator result as feedback, adjusting the parameters of the generator to maximize the probability that the generated forged data is judged by the discriminator as a real video sample, wherein the generator loss function is continuously optimized and converged; S3. Process the video to be detected into a forward playback video and a reverse playback video, and input them into the trained GAAE model for authenticity identification. If the similarity between the two output results is greater than 99%, the video is a real video, otherwise, the video is a fake video.
2. The video authenticity detection method according to claim 1, characterized in that: The encoder operation step of mapping the input data to the latent space specifically includes: the input data is processed by the encoder neural network, and a mean vector and a variance vector are output, which define a Gaussian latent variable distribution for generating latent variables.
3. The video authenticity detection method according to claim 2, characterized in that: The specific formula for processing the encoder neural network is as follows: in, represents the forward propagation function of the encoder neural network, represents the parameters of the encoder, and x represents the input data.
4. The video authenticity detection method according to claim 2, characterized in that: The specific formula of Gaussian latent variable distribution is as follows: Among them, z represents the latent variable, I represents the unit matrix of the same dimension as μ, μ represents the mean vector, σ 2 Represents the variance vector, N represents the mean μ and the variance σ 2 The multivariate normal distribution of .
5. The video authenticity detection method according to claim 1, characterized in that: The specific formula for obtaining hidden variables by sampling from the latent space is as follows: z=μ+σ×ε Among them, z represents the latent variable, μ represents the mean vector, σ represents the standard deviation vector, and ε represents the random noise sampled from the standard normal distribution.
6. The video authenticity detection method according to claim 1, characterized in that: The reconstruction loss uses the average error as a metric, and the specific formula for optimizing the training using the reconstruction loss and KL divergence is as follows: L=L rec +L KL Among them, D represents the dimension of the latent variable, x represents the input data, and x i represents the i-th dimension in the input data, Reconstruction data. represents the i-th dimension in the reconstructed data, σ 2 represents the variance vector, μ represents the mean vector, L rec represents the reconstruction loss function, L KL represents the KL divergence function.
7. The video authenticity detection method according to claim 1, characterized in that: The generator generates forged data specifically including: the generator is composed of a convolution layer and an activation function, maps the data vector in the reconstruction result to the real space under the real video sample, and generates a realistic sample, thereby generating forged data.
8. The video authenticity detection method according to claim 1, characterized in that: The discriminator is composed of a convolutional layer and an activation function, extracts features from the input sample and outputs a probability, which represents the probability that the input sample is a true sample.
9. The video authenticity detection method according to claim 1, characterized in that: The adjustment of the parameters of the generator is updated by an optimization algorithm in the optimizer, and the optimization algorithm is stochastic gradient descent. The specific formula for updating the parameters in the model is as follows: Among them, θ t represents the parameter vector of the model at the tth iteration, η represents the learning rate, It means that the objective function J has parameter θ t The gradient of θ at θ, J represents the objective function, m j represents the input features of the jth training sample, n j represents the target value of the jth training sample.
10. A computer program product, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.