Forged Video Detection Method Based on Similarity Value of Facial Action Unit Intensity Curves

Abstract

The present invention belongs to the technical field of forged video face detection, specifically a forged video detection method based on facial action unit (FAU) intensity curve similarity values. The method comprises steps 1: obtaining AU intensity values; 2: preprocessing the AU intensity curves; 3: calculating AU intensity curve similarity values; and 4: classification and detection using an SVM classifier. The forged video detection method based on FAU intensity curve similarity values provided by the present invention does not reduce detection effectiveness due to reduced forgery traces caused by technical flaws in forgery. Furthermore, it can detect and judge forged face videos using FAU intensity curve similarity even when the original dataset size is insufficient to provide a classifier with a range of individual features.

Description

Technical Field

[0001] The invention belongs to the technical field of forged video face detection, and in particular is a forged face detection method based on similarity values of facial action unit intensity curves. Background Art

[0002] With the in-depth study of deepfake technology, a growing number of open-source forgery software and commercial applications have appeared online. These applications fall into two main categories: open-source forgery projects on GitHub, such as Faceswap; and commercial apps that offer features like face swapping, expression exchange, and attribute manipulation. These one-click commercial apps significantly lower the barrier to entry for creating fake facial videos. Users only need to upload an image or video to create a fake face video, and the resulting fakes are so convincing that they are difficult to distinguish with the human eye. Due to the highly deceptive nature of fake facial videos and the range of social issues they can cause, in-depth research on deepfake detection technology has become increasingly important and urgent.

[0003] Current mainstream deepfake video detection and judgment technologies fall into four categories: visual deepfake detection based on specific artifacts, data-driven visual deepfake detection, information-inconsistency-based visual deepfake detection, and other types of visual deepfake detection. While these detection methods are effective in detecting fake facial videos that contain artifacts due to flaws in the forgery process, as deepfake detection technology advances, deepfake generation technology will also improve accordingly. In the future, fake facial videos will only visually differ from the original videos in the face, while features such as facial movements, head posture, and eye contact will remain largely consistent. Furthermore, artifacts caused by flaws in the forgery process will be reduced, reducing the effectiveness of previous detection methods. Therefore, detecting and judging fake facial videos based on biometric information such as facial movements will be a future research direction.

[0004] Hu Xiaorui et al. proposed an innovative facial expression synthesis method in the article “Generative Adversarial Network Based on Facial Action Coding System” (article [1]). By controlling the one-dimensional target condition vector representing the activation degree of different facial action units, for a single input face image, only the texture of the part corresponding to the target condition vector is changed while keeping the character information and other parts in the image unchanged, and a face image consistent with the target expression coding is generated.

[0005] Yuming Gu et al. mentioned in the article "Agarwal Protecting World Leaders Against DeepFakes CVPRW 2019 paper" (article [2]) that different individuals will show relatively different facial and head movement patterns when speaking. Therefore, the facial action coding system is used to extract the intensity and frequency characteristics of facial action units of different individuals when speaking. These characteristics are imported into the SVM, and the classifier is allowed to learn the feature ranges of different people. When detecting the video to be tested, it is checked which person's feature range such features of the video to be tested fall within, and then it is judged whether the person is the same person as the person in the video to be tested, and then it is judged whether the face video is forged. However, this method requires the original video data set to be large enough to divide the personal feature range and detect forged videos. That is, the limitation of this method is that it is not applicable to the case of a small original data set. In real life, if ordinary people are harmed by face-changing videos, the above method cannot well detect whether the video is forged, because it is difficult for ordinary people to construct a large enough original video data set for the classifier to learn the personal feature range. Summary of the Invention

[0006] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a forged video detection method based on the similarity value of the facial action unit intensity curve. This method can detect and judge forged face videos through the similarity of the facial action unit (abbreviated as "AU") intensity curve when the size of the original data set is not large enough to provide the classifier with the range of personal features.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A forged video detection method based on facial action unit intensity curve similarity values is characterized by comprising the following steps:

[0009] Step 1: Obtain AU intensity values: Preprocess the test video and sample video of the same corpus through the facial action unit acquisition module to obtain the AU intensity values of the time period when only the speaker is speaking in the video;

[0010] Step 2: AU intensity curve preprocessing: This preprocessing includes selecting an AU type and fitting and denoising the AU intensity curve. The AU type selection is used to extract AUs that are highly correlated with speech behavior, and the AU intensity curve fitting and denoising is used to reduce the interference of AU intensity curve noise on subsequent calculations.

[0011] Step 3: Calculate the similarity value of the AU intensity curve: Under the premise of the same AU type, calculate the similarity value of the intensity curve of each AU in the test video and the sample video after the above processing;

[0012] Step 4: Classifier training and testing: Input the similarity values of each AU curve of the test video and the sample video into the trained classifier for comparison, and use the classifier to detect whether the test video is a fake face video;

[0013] The training of the classifier includes calculating the similarity values of the facial action unit intensity curves of the same speaker and different speakers under the premise of the same corpus and AU type, obtaining the distribution intervals of the curve similarity values of the same person and the curve similarity values of different people of each AU type as training data sets, and providing the classifier with supervised learning.

[0014] As a preferred technical solution, the acquisition of the AU intensity value in the video in step 1 includes the following steps:

[0015] Step 1.1: Use the facial action unit acquisition module to analyze and process the video to obtain a quantitative table of the speaker's AU intensity in the entire video;

[0016] Step 1.2: Based on the AU intensity quantization table obtained in step 1.1, set AU25 as the basis for judging the start and end points of the speech time; the AU intensity quantization table is tested in positive order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the first frame is used as the start time point of the speech; the AU intensity quantization table is tested in reverse order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the last frame is used as the end time point of the speech; according to the start and end time points of the speech, the AU intensity quantization table obtained in step 1.1 is extracted and simplified.

[0017] As a preferred technical solution, the AU intensity curve preprocessing in step 2 includes the following steps:

[0018] Step 2.1: Select AUs that are highly correlated with speech behavior and perform another extraction and simplification on the AU intensity quantization table obtained in step 1.2;

[0019] Step 2.2: Use the Savitzky-Golay filter to fit and denoise the facial action unit intensity curve in the AU intensity quantization table obtained in step 2.1.

[0020] As a preferred technical solution, the calculation of the similarity value of the intensity curve of the facial action unit in step 3 includes calculating the similarity value of the intensity curve of each AU in the test video and the sample video based on the Fast-DTW algorithm in the dynamic regularization algorithm, which serves as the detection basis of the subsequent classifier.

[0021] As a preferred technical solution, the classifier in step 4 is an SVM classifier, and the training of the classifier includes the following steps:

[0022] Step 4.1: Using the Fast-DTW algorithm in the dynamic warping algorithm, calculate the similarity values of the facial action unit intensity curves of the same speaker and different speakers based on the same corpus and AU type. The distribution ranges of the similarity values of the curves of the same person and different people for each AU type are obtained as the training dataset.

[0023] Step 4.2: In Matlab, use the training dataset obtained above to provide SVM classifier supervised learning.

[0024] Compared with the existing technology, the forged video detection method based on the similarity value of the facial action unit intensity curve provided by the present invention will not reduce the detection effect due to the reduction of forgery traces caused by defects in the forgery technology. In addition, it can realize the detection and judgment of forged face videos through the similarity of the facial action unit intensity curve when the scale of the original data set is not sufficient to provide the classifier with the range of personal features.

[0025] The concept, working process and effects of the present invention will be further described below with reference to the accompanying drawings to fully understand the purpose, features and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 4 is a flow chart of a method for detecting forged videos based on similarity values of facial action unit intensity curves in an embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] like Figure 1 As shown in FIG, the forged video detection method based on the similarity value of the facial action unit intensity curve includes the following steps:

[0029] S1. Obtaining AU intensity values: Preprocess the video to be tested and the sample video (the original video database of the test object) of the same corpus using the facial action unit acquisition module (program) to obtain the AU intensity values of the video during the time period containing only the speaker's speech;

[0030] Specifically, the facial action unit acquisition module is used to analyze and process the video to obtain a quantitative table of the speaker's AU intensity in the entire video;

[0031] Based on the obtained AU intensity quantization table, AU25 is set as the basis for judging the start and end points of the speech time; the AU intensity quantization table is checked in positive order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the first frame is used as the start time point of the speech; the AU intensity quantization table is checked in reverse order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the last frame is used as the end time point of the speech; after determining the start and end time points of the speech in the video, the intensity values of various facial action units within the speech time period are extracted from the AU intensity quantization table, and the AU intensity quantization table for the speech time period is compiled.

[0032] S2. AU intensity curve preprocessing: This preprocessing includes AU type selection and AU intensity curve fitting and denoising. The AU type selection is used to extract AUs with a high correlation with speech behavior, and the AU intensity curve fitting and denoising is used to reduce the interference of AU intensity curve noise on subsequent calculations.

[0033] Specifically, according to the articles [1] and [2] in the background technology, AUs with a high correlation with speech behavior are selected, and the intensity values of AUs with a high correlation with speech behavior are retained in the AU intensity quantization table within the above-mentioned speech time period;

[0034] The Savitzky-Golay filter is used to fit and denoise the facial action unit intensity curve after the above processing to obtain the intensity curve of each AU in the test video and the sample video.

[0035] S3. Calculate the similarity of AU intensity curves: Under the premise of the same AU type, use the Fast-DTW algorithm in the dynamic regularization algorithm to calculate the similarity of the intensity curves of each facial action unit of the same speaker (test video) and different speakers (sample videos). This serves as the basis for subsequent SVM classifier detection.

[0036] S4. Training and detection of SW classifier: The similarity value of the intensity curve of each AU of the test video and the sample video calculated above is used as the detection basis of the classifier, and input into the trained SVM classifier for comparison. The classifier is used for classification detection to determine whether the test video is a fake face video.

[0037] The training of the classifier includes calculating the similarity values of the facial action unit intensity curves of the same speaker and different speakers under the premise of the same corpus and AU type, obtaining the distribution intervals of the curve similarity values of the same person and the curve similarity values of different people of each AU type as training data sets, and providing an SVM classifier for supervised learning.

[0038] Specifically, using the Fast-DTW algorithm in the dynamic warping algorithm, we calculated the similarity values of the facial action unit intensity curves of the same speaker and different speakers based on the same corpus and AU types. We then sorted out the distribution ranges of the similarity values of the curves of the same person and different people for each AU type as the training dataset.

[0039] In Matlab, the training dataset obtained by the above calculation is used to provide SVM classifier supervised learning. The template of each sub-data in the training set is: AU26 intensity curve similarity value (feature 1), AU25 intensity curve similarity value (feature 2), ... same person or different people (label).

[0040] The forged video detection method based on the similarity value of the facial action unit intensity curve provided by the present invention will not reduce the detection effect due to the reduction of forgery traces caused by defects in the forgery technology, and can realize the detection and judgment of forged face videos through the similarity of the facial action unit intensity curve when the scale of the original data set is insufficient to provide the classifier with a range of personal features.

[0041] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A forged video detection method based on similarity values of facial action unit intensity curves, characterized in that: The steps include: Step 1: Obtain AU intensity values: Preprocess the test video and sample video of the same corpus through the facial action unit acquisition module to obtain the AU intensity values of the time period when only the speaker is speaking in the video; Step 2: AU intensity curve preprocessing: This preprocessing includes selecting an AU type and fitting and denoising the AU intensity curve. The AU type selection is used to extract AUs that are highly correlated with speech behavior, and the AU intensity curve fitting and denoising is used to reduce the interference of AU intensity curve noise on subsequent calculations. Step 3: Calculate the similarity value of the AU intensity curve: Under the premise of the same AU type, calculate the similarity value of the intensity curve of each AU in the test video and the sample video after the above processing; Step 4: Classifier training and testing: Input the similarity values of the AU intensity curves of the test video and the sample video into the trained classifier for comparison, and use the classifier to detect whether the test video is a fake face video; The training of the classifier includes calculating the similarity values of the facial action unit intensity curves of the same speaker and different speakers under the premise of the same corpus and AU type, obtaining the distribution intervals of the curve similarity values of the same person and the curve similarity values of different people of each AU type as training data sets, and providing the classifier with supervised learning.

2. The forged video detection method based on facial action unit intensity curve similarity value according to claim 1, characterized in that: The acquisition of the AU intensity value in the video in step 1 is implemented by the following steps: Step 1.1: Use the facial action unit acquisition module to analyze and process the video to obtain a quantitative table of the speaker's AU intensity in the entire video; Step 1.2: Based on the AU intensity quantization table obtained in step 1.1, set AU25 as the basis for judging the start and end points of the speech time; the AU intensity quantization table is tested in positive order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the first frame is used as the start time point of the speech; the AU intensity quantization table is tested in reverse order with 5 frames as a group. When the intensity difference between the first and last frames is ≥0.5, the last frame is used as the end time point of the speech; according to the start and end time points of the speech, the AU intensity quantization table obtained in step 1.1 is extracted and simplified.

3. The forged video detection method based on facial action unit intensity curve similarity value according to claim 2, characterized in that: The AU intensity curve preprocessing in step 2 includes the following steps: Step 2.1: Select AUs that are highly correlated with speech behavior and perform another extraction and simplification on the AU intensity quantization table obtained in step 1.2; Step 2.2: Use the Savitzky-Golay filter to fit and denoise the facial action unit intensity curve in the AU intensity quantization table obtained in step 2.

1.

4. The forged video detection method based on facial action unit intensity curve similarity value according to claim 1, characterized in that: The calculation of the similarity value of the intensity curve of the facial action unit in step 3 includes calculating the similarity value of the intensity curve of each AU of the test video and the sample video according to the Fast-DTW algorithm in the dynamic warping algorithm, which serves as the detection basis of the subsequent classifier.

5. The forged video detection method based on facial action unit intensity curve similarity value according to claim 1, characterized in that: The classifier in step 4 is an SVM classifier, and the training of the classifier includes the following steps: Step 4.1: Using the Fast-DTW algorithm in the dynamic warping algorithm, calculate the similarity values of the facial action unit intensity curves of the same speaker and different speakers based on the same corpus and AU type. The distribution ranges of the similarity values of the curves of the same person and different people for each AU type are obtained as the training dataset. Step 4.2: In Matlab, use the training dataset obtained above to provide SVM classifier supervised learning.

Images