Deep counterfeit compressed face image identification method based on deep learning

Abstract

The invention discloses a deep learning-based deep forged compressed face image identification method, which belongs to the field of artificial intelligence image identification, and comprises the following steps of: 1, carrying out frequency domain modal extraction on a deep forged compressed face image by adopting dynamic adaptive discrete cosine transform; 2, a multi-head self-attention mechanism and a multi-dimensional entropy state load mutual feedback mechanism are adopted to extract frequency domain high-dimensional load features; step 3, extracting spatial domain features based on the pre-trained deep residual network; 4, performing feature fusion on the frequency domain high-dimensional load features and the spatial domain features based on an attention mechanism; and 5, discriminating and classifying the fusion features by using a classifier comprising a full connection layer. According to the method, high-accuracy identification can be carried out on the compressed face image manufactured by adopting a deep counterfeiting means.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition of artificial intelligence, and specifically relates to a method for identifying deep forged compressed facial images based on deep learning. Background Art

[0002] Deep fake refers to the use of deep learning and computer vision technology to synthesize realistic fake facial images and videos. Deep fake facial image identification is the process of identifying and detecting fake facial images generated using artificial intelligence technology.

[0003] At present, the dual-branch deep fake face image identification network that combines the spatial domain and the frequency domain uses the traditional discrete cosine transform to extract the frequency domain modal information of the fixed spectrum, and uses the combination of convolutional neural network and Transformer to extract local and global features to identify the fake face image. The main problems of this are: (1) The detection accuracy of compressed fake face images is very low. The images and videos uploaded on social media are often compressed. The compression process will blur many fake features in the face image and add new confusing features to disturb the fake identification results. The current detection method is difficult to deal with compression. (2) The traditional discrete cosine transform uses a fixed frequency decomposition and will be more biased towards low-frequency spectrum information when extracting frequency domain modalities, but high-frequency spectrum information often retains more fake traces. The traditional discrete cosine transform method extracts fewer high-frequency fake features contained in the frequency domain modal. (3) When using convolutional neural networks and Transformers to extract local and global features, convolutional neural networks will focus on extracting local features, while Transformers will focus on extracting global features. However, there is little interaction between global and local features, which makes the identification network less capable of analyzing nonlinear boundaries and makes it difficult to identify nonlinear forgery traces. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a method for identifying deep fake compressed facial images based on deep learning, adopts a dual-branch network architecture combining spatial domain and frequency domain, designs a dynamic adaptive discrete cosine transform method to flexibly extract frequency domain modes that are more suitable for deep fake compressed facial images and contain dynamic spectrum information, and designs a multi-dimensional entropy load mutual feedback mechanism to allow high-dimensional global and local information to fully interact and feedback in both directions, and deeply mine nonlinear forgery trace features. In the process of deep learning model training, it learns to effectively extract, fuse, and identify dual-modal forgery traces in the spatial domain and frequency domain of the image. The present invention can identify compressed facial images produced by deep fake means with high accuracy.

[0005] The technical solution of the present invention is as follows:

[0006] A deep learning-based method for identifying deep fake compressed face images comprises the following steps:

[0007] Step 1: Use dynamic adaptive discrete cosine transform to extract frequency domain modality of deep fake compressed face image;

[0008] Step 2: Use multi-head self-attention mechanism and multi-dimensional entropy state load mutual feedback mechanism to extract frequency domain high-dimensional load features;

[0009] Step 3: Extract spatial features based on the pre-trained deep residual network;

[0010] Step 4: Based on the attention mechanism, feature fusion is performed on the frequency domain high-dimensional load features and the spatial domain features;

[0011] Step 5: Use a classifier containing a fully connected layer to identify and classify the fused features.

[0012] Furthermore, the specific process of step 1 is as follows:

[0013] Step 1.1: Deep fake compressed face image X∈R N×N After unified preprocessing, it is projected into the universal domain space and recorded as the face image matrix X N , and for X N Apply a learnable dynamic discrete cosine transform matrix M D , M D The loss function is continuously updated through feedback during training, and the calculation formula for generating matrix elements is:

[0014]

[0015] Where N represents the size of the dynamic discrete cosine transform matrix; Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-th round of training; express The update amount for the t-1th round of training according to the loss function;

[0016] Step 1.2: Use M D X N Perform discrete cosine transform to obtain high-dimensional frequency domain response result X freq , the calculation formula is:

[0017]

[0018] in, Represents the high-dimensional frequency domain response result of the tth round of training; represents the dynamic discrete cosine transform matrix of the tth round of training; T represents transpose;

[0019] Step 1.3: Update the adaptive multi-frequency filter F A ∈{F h ,F m ,F l ,F a}, F h 、F m 、F l 、F a They correspond to the high-frequency adaptive filter, the intermediate-frequency adaptive filter, the low-frequency adaptive filter, and the full-frequency adaptive filter respectively, and the calculation formula is:

[0020]

[0021] in, is the adaptive multi-frequency filter for training the tth round; They represent the high-frequency adaptive filter, medium-frequency adaptive filter, low-frequency adaptive filter, and full-frequency adaptive filter of the t-th round of training respectively; σ(·) represents the Sigmoid activation function; ε represents the adaptive parameter; Respectively The update amount of the training round t-1 according to the loss function; B h , B m , B l , B a They represent high-frequency basic filter, medium-frequency basic filter, low-frequency basic filter, and full-frequency basic filter respectively, and the calculation formula is:

[0022]

[0023] in, Represents the matrix row number of the basic filter; Represents the matrix column number of the basic filter;

[0024] Step 1.4: Use F A X freq Decompose and fuse to obtain the final frequency domain modal image Y freq , the calculation formula is:

[0025]

[0026] in, represents the frequency domain modal image of the tth round of training; F conv (·) represents the convolution mapping operation of the frequency domain modal feature channel; Indicates cascade operation; * indicates matrix multiplication;

[0027] Step 1.5: During the training process, the loss function is used to feedback M D and F APerform dynamic adaptive update adjustment, M D and F A The update rule calculation formula is:

[0028]

[0029] in, express The update amount for the t-1th round of training according to the loss function, Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-1th round of training; express The update amount for the t-1th round of training according to the loss function, represents the adaptive multi-frequency filter of the t-1th round of training; η represents the learning rate; L(t-1) represents the loss function of the t-1th round;

[0030] Step 1.6: Extract the frequency domain modal image Y freq A 5×5 convolutional neural network is used to extract shallow frequency domain modal forgery features, and then the shallow frequency domain modal forgery features are segmented into individual loads. The calculation formulas for the number of loads Q and the load dimension D are:

[0031]

[0032] Among them, Ω h ,Ω w ,Ω c They represent the vertical dimension, horizontal dimension, and number of characteristic channels of the load in the entropy space respectively; Represents the boundary control factor of the perception window.

[0033] Furthermore, the specific process of step 2 is as follows:

[0034] Step 2.1: Capture the long-term dependencies of the entropy space context of the payload through the multi-head self-attention mechanism. After the Q payloads pass through the multi-head self-attention mechanism, they form Q high-dimensional payloads. All high-dimensional payloads constitute a high-dimensional payload set S = {s1, s2, …, s Q},s Q is the Qth high-dimensional load;

[0035] Step 2.2: Use the multi-dimensional entropy load feedback mechanism to perform multi-dimensional fine-grained feature interaction and feedback attention calculation.

[0036] Furthermore, the specific process of step 2.2 is as follows:

[0037] Step 2.2.1: The entropy state of each high-dimensional load in the high-dimensional load set is an asymmetric non-normal distribution. The entropy state of each high-dimensional load is measured by the standard deviation statistical functional, and the high-dimensional loads are globally sorted according to the entropy state size. The calculation formula is:

[0038]

[0039] Where R(S) represents the sorted high-dimensional load set; E[·] represents the expected value of the entropy state; F Rank (·) indicates a sorting algorithm;

[0040] Step 2.2.2: Decompose the sorted high-dimensional load set R(S) into two subsets, which are mapped to different entropy dimension regions. The two subsets are low-entropy high-dimensional load subsets R(S) a and high-entropy high-dimensional load subset R(S) b , where a and b represent different entropy dimensions; the high entropy state load is included in R(S) b middle;

[0041] Step 2.2.3: Adjust the size and offset of the perception window To adaptively reconstruct R(S) b ,During reconstruction, the boundary control factor of the perception window is set to twice the original;

[0042] The reconstructed high-entropy high-dimensional load subset is R(S) b ′,R(S) b The calculation formulas for the number of high-dimensional loads Q′ and the dimension of high-dimensional loads D′ are:

[0043]

[0044] Step 2.2.4: R(S) a and R(S) b ′ performs two-way cross-dimensional interaction; the specific calculation formula is:

[0045]

[0046] Among them, cross_atten a 、cross_atten b Respectively represent R(S) a 、R(S) b ′High-dimensional load characteristics after bidirectional cross-dimensional interaction; g cross_attention (·) indicates the cross-attention fusion method;

[0047] Step 2.2.5: cross_atten bPerform shape correction and set cross_atten b Revert to cross_atten a Same shape, after changing to the same shape, cross_atten a and cross_atten b The final high-dimensional load characteristics in the frequency domain are obtained by cascading in the entropy state dimension.

[0048] Furthermore, in step 3, a deep residual network ResNet50 is pre-trained on the InmageNet dataset. ResNet50 extracts features by stacking residual blocks. The calculation formula for each residual block is:

[0049]

[0050] in, represents the residual convolution operation performed by the kth residual block; represents the spatial features extracted from the kth residual block; represents the spatial features extracted from the k-1th residual block; Represents the total number of residual blocks.

[0051] Furthermore, the specific process of step 4 is as follows:

[0052] Step 4.1: First, concatenate the spatial domain features and the frequency domain high-dimensional load features, and then calculate the attention weight W of the concatenated features. atten , the calculation formula is:

[0053]

[0054] Among them, Features spa Represents spatial features; Features freq represents the high-dimensional load feature in the frequency domain; Conv(·) represents a 1×1 convolution operation; Φ(·) represents the RELU activation function;

[0055] Step 4.2: Multiply the attention weight of the cascade feature by the cascade feature element by element to obtain the fused feature.

[0056] Furthermore, the specific process of step 5 is as follows:

[0057] Step 5.1. Use the fully connected layer to calculate the unnormalized score of the fused feature. The fully connected layer projects the dimension of the fused feature to the dimension of the category to be classified. The formula for the fully connected layer projection is as follows:

[0058]

[0059] in, Indicates the classification category, Indicates the forged category, represents the real category; Indicates category Normalized score of Features fus Indicates fusion features; Indicates category The corresponding weight matrix of the fully connected layer; Indicates category The corresponding fully connected layer bias;

[0060] Step 5.2: Use the Softmax activation function to convert the unnormalized scores into corresponding probabilities, and select the category with the largest probability as the output category determined by the model. The probability calculation formula of the Softmax activation function is as follows:

[0061]

[0062] Among them, P represents the probability of different categories; are the normalized scores of the fake and real categories respectively.

[0063] The beneficial technical effects brought about by the present invention are as follows.

[0064] (1) Achieve efficient and comprehensive frequency domain modality capture. Dynamic adaptive discrete cosine transform can adaptively adjust the weights of different frequency domain extractions according to the characteristics of the input compressed face image, dynamically enhance the sensitivity to the forged area, and thus more accurately capture the frequency domain modalities related to deepfakes.

[0065] (2) Improve the ability to capture high-dimensional local and global features in the frequency domain. The multi-dimensional entropy load feedback mechanism uses standard deviation statistical functional sorting to prioritize high-entropy high-dimensional loads containing high-critical information, reconstructs high-entropy high-dimensional loads through perception window adaptation, and conducts complex information flow through bidirectional cross-dimensional interaction, thereby improving sensitivity to subtle forgery traces in the frequency domain.

[0066] (3) Improve the accuracy of identification of compressed images. The present invention uses dynamic adaptive discrete cosine transform to extract the frequency domain mode of facial images, effectively retaining the forged features blurred during the compression process, and through the multi-dimensional entropy load feedback mechanism, it can clearly distinguish the confusion features and effectively extract the subtle forged features of the frequency domain mode, greatly improving the accuracy of identification of compressed images. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a process diagram of the deep fake compressed face image identification method based on deep learning of the present invention.

[0068] Figure 2 It is a process diagram of using a perception window to change the high-dimensional load scale in the multi-dimensional entropy state load mutual feedback mechanism of the present invention.

[0069] Figure 3 It is a ROC curve diagram of the present invention on the FaceForensics++ dataset C23 version.

[0070] Figure 4 It is a ROC curve diagram of the present invention on the FaceForensics++ dataset C40 version. DETAILED DESCRIPTION

[0071] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0072] like Figure 1 As shown, a deep learning-based deep forged compressed face image identification method includes the following steps:

[0073] Step 1: Use dynamic adaptive discrete cosine transform to extract frequency domain modality from deep fake compressed face image. The present invention first uses dynamic adaptive discrete cosine transform to extract frequency domain modality image from a deep fake compressed face image. The specific process is as follows:

[0074] Step 1.1: Input deep fake compressed face image X∈R N×N After unified preprocessing, it is projected into the universal domain space and recorded as the face image matrix X N , and for X N Apply a learnable dynamic discrete cosine transform matrix M D , M D The loss function is continuously updated through feedback during training, and the calculation formula for generating matrix elements is:

[0075]

[0076] Where t represents the round of training; i represents the matrix row subscript; j represents the matrix column subscript; N represents the size of the dynamic discrete cosine transform matrix; Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-th round of training; express The update amount for the t-1th round of training according to the loss function.

[0077] Step 1.2: Use M D x N Perform discrete cosine transform to obtain high-dimensional frequency domain response result X freq , the calculation formula is:

[0078]

[0079] in, Represents the high-dimensional frequency domain response result of the tth round of training; represents the dynamic discrete cosine transform matrix of the tth round of training; T represents the transpose.

[0080] Step 1.3: Update the adaptive multi-frequency filter F A ∈{F h ,F m ,F l ,F a}, F h 、F m 、F l 、F a They correspond to the high-frequency adaptive filter, the intermediate-frequency adaptive filter, the low-frequency adaptive filter, and the full-frequency adaptive filter respectively, and the calculation formula is:

[0081]

[0082] in, is the adaptive multi-frequency filter for training the tth round; They represent the high-frequency adaptive filter, medium-frequency adaptive filter, low-frequency adaptive filter, and full-frequency adaptive filter of the t-th round of training respectively; σ(·) represents the Sigmoid activation function; ε represents the adaptive parameter; Respectively The update amount of the training round t-1 according to the loss function; B h , B m , B l , B a They represent high-frequency basic filter, medium-frequency basic filter, low-frequency basic filter, and full-frequency basic filter respectively. The basic filter is used to decompose the high-dimensional frequency domain response results into high-frequency band, medium-frequency band, low-frequency band, and full-frequency band, corresponding to the first 1 / 16, 1 / 16 to 1 / 8, the last 1 / 8, and all of the entire spectrum respectively. The calculation formulas for each basic filter are:

[0083]

[0084] in, Represents the matrix row number of the basic filter; Represents the matrix column number of the basic filter;

[0085] Step 1.4: Use F A X freq Decompose and fuse to obtain the final frequency domain modal image Y freq , the calculation formula is:

[0086]

[0087] in, represents the frequency domain modal image of the tth round of training; F conv (·) represents the convolution mapping operation of the frequency domain modal feature channel; represents a cascade operation; * represents matrix multiplication.

[0088] Step 1.5: M in the dynamic adaptive discrete cosine transform method D and F A It is learnable and can be dynamically updated and adjusted through feedback from the loss function during training to better capture frequency domain data. D and F A The update rule calculation formula is:

[0089]

[0090] in, express The update amount for the t-1th round of training according to the loss function, Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-1th round of training; express The update amount for the t-1th round of training according to the loss function, represents the adaptive multi-frequency filter of the t-1th round of training; η represents the learning rate; L(t-1) represents the loss function of the t-1th round.

[0091] Step 1.6: Extract the frequency domain modal image Y freq A 5×5 convolutional neural network is used to extract shallow frequency domain modal forgery features, and then the shallow frequency domain modal forgery features are segmented into individual loads. The calculation formulas for the number of loads Q and the load dimension D are:

[0092]

[0093] Among them, Ω h ,Ω w ,Ω c They represent the vertical dimension, horizontal dimension, and number of characteristic channels of the load in the entropy space respectively; represents the boundary control factor of the perception window, The size can be determined according to the image resolution, hardware conditions and other specific circumstances.

[0094] Step 2: Use multi-head self-attention mechanism and multi-dimensional entropy state load mutual feedback mechanism to extract frequency domain high-dimensional load features; the present invention realizes the deep extraction of global and local frequency domain high-dimensional load features. The specific process is as follows:

[0095] Step 2.1: The multi-head self-attention mechanism first captures the long-term dependencies of the entropy space context of the payload. After the multi-head self-attention mechanism, Q payloads are converted into Q high-dimensional payloads. All high-dimensional payloads constitute a high-dimensional payload set S = {s1, s2, …, s Q},s Q is the Qth high-dimensional load; the multi-head self-attention mechanism ignores the local detail forgery features, and the ignored local detail forgery features may often be the important features that affect the identification results.

[0096] Step 2.2: In order to further capture the local detail forged features, a multi-dimensional entropy load mutual feedback mechanism is then used to perform multi-dimensional fine-grained feature interaction and feedback attention calculation. The specific implementation steps of the multi-dimensional entropy load mutual feedback mechanism are:

[0097] Step 2.2.1: The entropy state of each high-dimensional load in the high-dimensional load set is an asymmetric non-normal distribution. The entropy state of each high-dimensional load is measured through the standard deviation statistical functional, and the high-dimensional loads are globally sorted according to the entropy state. High entropy means high criticality, ensuring that high-entropy high-dimensional loads are processed first in the multi-scale expression of information, thereby achieving deeper correlation extraction in the high-dimensional feature space. The calculation formula for this process is:

[0098]

[0099] Where R(S) represents the sorted high-dimensional load set; E[·] represents the expected value of the entropy state; F Rank (·) indicates a sorting algorithm.

[0100] Step 2.2.2: Decompose the sorted high-dimensional load set R(S) into two subsets, which are mapped to different entropy dimension regions. The two subsets are low-entropy high-dimensional load subsets R(S) a , high entropy state high dimensional load subset R(S) b , where a and b represent different entropy dimensions; the high entropy state load is included in R(S) b middle.

[0101] Step 2.2.3: Perform multi-scale entropy state reconstruction based on deformable perception window to reconstruct R(S) b Adaptive reconstruction is performed through deformable perception windows to achieve dynamic adjustment of the scale and distribution of high-dimensional loads. This reconstruction is achieved by adjusting the size of the perception window (during reconstruction, the boundary control factor of the perception window is set to twice the original, i.e. ) and the offset of the perception window To adaptively reconstruct R(S) b . Figure 2 As an example of the reconstruction process, Figure 2It is assumed that there are 9 high-dimensional loads in total initially. These 9 high-dimensional loads form a 3×3 matrix and are numbered in sequence from 1 to 9. The dotted box represents the perception window. The size of the perception window is assumed to be 2×2, and the offset of the perception window is assumed to be 1. The perception window moves according to the offset each time. Each time the perception window will select 2×2, that is, 4 high-dimensional loads. These 4 high-dimensional loads will re-form a new load of a larger scale. From the process in the figure, we can see that the perception window first selects high-dimensional loads 1, 2, 4, and 5 to form a new first high-dimensional load. The perception window moves and selects high-dimensional loads 2, 3, 4, and 5 to form a new second high-dimensional load. The perception window continues to move and selects high-dimensional loads 4, 5, 7, and 8 to form a new third high-dimensional load. The perception window moves again and selects high-dimensional loads 5, 6, 8, and 9 to form a new fourth high-dimensional load. The perception window moves and the high-dimensional load scale change is completed. Finally, the four high-dimensional loads are combined into a new high-dimensional load of a larger scale. The new high-dimensional load is in the form of a 4×4 matrix.

[0102] The reconstructed high-entropy high-dimensional load subset is R(S) b ′, the number and dimension of high-dimensional loads have changed, and load overlaps have been formed. These overlaps realize the repeated interaction of local information, thereby improving the capture of high-order microscopic features and their complex associations, especially enhancing the ability to analyze nonlinear boundaries, R(S) b The calculation formulas for the number of high-dimensional loads Q′ and the dimension of high-dimensional loads D′ are:

[0103]

[0104] Step 2.2.4: In order to establish the complex global and local intercommunication between high-dimensional information, R(S) a and R(S) b ′ performs two-way cross-dimensional interaction. The specific calculation formula is:

[0105]

[0106] Among them, cross_atten a 、cross_atten b Respectively represent R(S) a 、R(S) b ′High-dimensional load characteristics after bidirectional cross-dimensional interaction; g cross_attention (·) indicates the cross-attention fusion method.

[0107] Step 2.2.5: cross_atten a and cross_atten bCascade, that is, reconstruct high-dimensional load features across scales, because Cross_atten a and cross_atten b It is a high-dimensional load feature of different scales, so cross_atten b Perform shape correction and set Cross_atten b Restore to Cross_atten a The same shape, after becoming the same shape, cascading these two high-dimensional load features in the entropy dimension will obtain the final frequency domain high-dimensional load feature.

[0108] Step 3: Extract spatial features based on a pre-trained deep residual network (ResNet50 network for short); use a dual-branch parallel approach to extract spatial features from the input compressed face image while extracting high-dimensional load features in the frequency domain.

[0109] While implementing frequency domain modal extraction and frequency domain high-dimensional load feature extraction, the input face image also enters the spatial domain branch. In the spatial domain branch, the deep residual network ResNet50 pre-trained on the InmageNet dataset is used to extract spatial domain features. ResNet50 is a residual network that extracts features by stacking residual blocks. The calculation formula for each residual block is:

[0110]

[0111] in, represents the residual convolution operation performed by the kth residual block; represents the spatial features extracted from the kth residual block; represents the spatial features extracted in the k-1th residual block; Represents the total number of residual blocks.

[0112] Step 4: Based on the attention mechanism, the frequency domain high-dimensional load features and the spatial domain features are fused. Based on the splicing of the spatial domain features and the frequency domain high-dimensional load features, the present invention uses the attention mechanism to calculate the weight of the spliced ​​features, and uses the attention mechanism to enhance important features and suppress unimportant features. The specific process is:

[0113] Step 4.1: First, concatenate the spatial domain features and the frequency domain high-dimensional load features, and then calculate the attention weight of the concatenated features. The attention weight can learn which forged features are more important, thereby enhancing these forged features. The attention weight W atten The calculation formula is:

[0114]

[0115] Among them, Features spa Represents spatial features; Features freq represents the high-dimensional load feature in the frequency domain; Conv(·) represents a 1×1 convolution operation; Φ(·) represents the RELU activation function.

[0116] Step 4.2: Multiply the attention weight of the cascade feature by the cascade feature element by element to obtain the fused feature. In this way, the important forged features in the spliced ​​features will be enhanced, and the unimportant forged features will be suppressed, resulting in a better feature fusion effect.

[0117] Step 5: Use a classifier including a fully connected layer to identify and classify the fused features. The present invention sends the fused features to the classifier to generate an identification result, which is true or false.

[0118] The present invention constructs a binary classifier for mapping the fused features to the target category, that is, obtaining the results of the two categories of forged or real. This binary classifier uses a fully connected layer and a Softmax activation function for category judgment; the specific process is as follows:

[0119] Step 5.1: Use the fully connected layer to calculate the unnormalized score of the fused feature. The fully connected layer will project the dimension of the fused feature to the category dimension to be classified. The category dimension of the classification in the present invention is 2. The formula for the fully connected layer projection is as follows:

[0120]

[0121] in, Indicates the classification category, Indicates the forged category, represents the real category; Indicates category Normalized score of Features fus Indicates fusion features; Indicates category The corresponding weight matrix of the fully connected layer; Indicates category The corresponding fully connected layer bias.

[0122] Step 5.2: Determine the category based on the unnormalized score to obtain the classification result of the model of the present invention. Use the Softmax activation function to convert the unnormalized score into the corresponding probability, and select the category with a larger probability as the output category determined by the model of the present invention. The probability calculation formula of the Softmax activation function is as follows:

[0123]

[0124] Among them, P represents the probability of different categories; are the normalized scores of the fake and real categories respectively.

[0125] The deep learning-based deep fake compressed face image identification method provided by the present invention realizes high-accuracy identification of compressed face images by designing a dynamic adaptive discrete cosine transform method and a multi-dimensional entropy load mutual feedback mechanism. The F3-Net, which first proposed the use of the traditional discrete cosine transform method for frequency domain modal extraction, was used as the baseline for comparison of experimental results. The evaluation indicators include: accuracy (ACC), sensitivity curve (ROC curve) and area under the sensitivity curve (AUC).

[0126] ACC is used to measure the proportion of samples correctly classified by the model on the test set. It is the most intuitive evaluation indicator. The specific calculation formula is:

[0127]

[0128] Among them, ACC is the accuracy value.

[0129] The ROC curve (also known as the sensitivity curve) takes the true positive rate (TPR) as the vertical axis and the false positive rate (FPR) as the horizontal axis. It can intuitively evaluate the classification performance of the compressed face image identification classifier. The specific calculation formula is:

[0130]

[0131] Among them, TPR is the true positive rate; FPR is the false positive rate; TP stands for true positive, that is, the number of samples whose actual label is positive and the predicted result is also positive; FP stands for false positive, that is, the number of samples whose actual label is negative and the wrong prediction is positive; TN stands for true negative, that is, the number of samples whose actual label is negative and the predicted result is also negative; FN stands for false negative, that is, the number of samples whose actual label is positive and the wrong prediction is negative.

[0132] AUC represents the area under the ROC curve, which is a numerical value used to evaluate the classification ability.

[0133] Performance evaluation is performed on the FaceForensics++ dataset, which contains 1,000 real videos and 4,000 fake videos. The real videos are original videos from YouTube, and the fake videos are generated based on the original videos using 4 deep fake methods. There are two compressed versions: C23 version and C40 version, where C23 is a high-quality compressed video dataset and C40 is a low-quality compressed video dataset. From this dataset, face images are collected by the data balancing method to perform performance evaluation. The comparison results of the accuracy and area under the sensitivity curve between the method of the present invention and F3-Net are shown in Table 1 below:

[0134] Table 1 Comparison results of C23 version and C40 version in different methods

[0135]

[0136]

[0137] From the comparison results, it can be seen that the higher the compression rate and the lower the image quality, the better the performance improvement of the present invention. On the C23 high-quality compressed version, ACC increased by 0.64% and AUC increased by 1.35%; on the C40 low-quality compressed version, ACC increased by 7.75% and AUC increased by 6.51%, showing very good identification performance for compressed face images.

[0138] The ROC curves of the present invention on the FaceForensics++ dataset C23 and C40 versions are shown as follows: Figure 3 , Figure 4 As shown, through Figure 3 , Figure 4 It can be seen intuitively that the present invention has good classification performance.

[0139] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A deep learning-based method for identifying deep fake compressed face images, characterized in that: The steps include: Step 1: Use dynamic adaptive discrete cosine transform to extract frequency domain modality of deep fake compressed face image; Step 2: Use multi-head self-attention mechanism and multi-dimensional entropy state load mutual feedback mechanism to extract frequency domain high-dimensional load features; Step 3: Extract spatial features based on the pre-trained deep residual network; Step 4: Based on the attention mechanism, feature fusion is performed on the frequency domain high-dimensional load features and the spatial domain features; Step 5: Use a classifier containing a fully connected layer to identify and classify the fused features.

2. According to the deep learning-based deep forged compressed face image identification method of claim 1, it is characterized in that: The specific process of step 1 is as follows: Step 1.1: Deep fake compressed face image X∈R N×N After unified preprocessing, it is projected into the universal domain space and recorded as the face image matrix X N , and for X N Apply a learnable dynamic discrete cosine transform matrix M D , M D The loss function is continuously updated through feedback during training, and the calculation formula for generating matrix elements is: Where N represents the size of the dynamic discrete cosine transform matrix; Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-th round of training; express The update amount for the t-1th round of training according to the loss function; Step 1.2: Use M D X N Perform discrete cosine transform to obtain high-dimensional frequency domain response result X freq , the calculation formula is: in, Represents the high-dimensional frequency domain response result of the tth round of training; represents the dynamic discrete cosine transform matrix of the tth round of training; T represents transpose; Step 1.3: Update the adaptive multi-frequency filter F A ∈{F h ,F m ,F l ,F a }, F h 、F m 、F l 、F a They correspond to the high-frequency adaptive filter, the intermediate-frequency adaptive filter, the low-frequency adaptive filter, and the full-frequency adaptive filter respectively, and the calculation formula is: in, is the adaptive multi-frequency filter for training the tth round; They represent the high-frequency adaptive filter, medium-frequency adaptive filter, low-frequency adaptive filter, and full-frequency adaptive filter of the t-th round of training respectively; σ(·) represents the Sigmoid activation function; ε represents the adaptive parameter; Respectively The update amount of the training round t-1 according to the loss function; B h , B m , B l , B a They represent high-frequency basic filter, medium-frequency basic filter, low-frequency basic filter, and full-frequency basic filter respectively, and the calculation formula is: in, Represents the matrix row number of the basic filter; Represents the matrix column number of the basic filter; Step 1.4: Use F A X freq Decompose and fuse to obtain the final frequency domain modal image Y freq , the calculation formula is: in, represents the frequency domain modal image of the tth round of training; F conv (·) represents the convolution mapping operation of the frequency domain modal feature channel; Indicates cascade operation; * indicates matrix multiplication; Step 1.5: During the training process, the loss function is used to feedback M D and F A Perform dynamic adaptive update adjustment, M D and F A The update rule calculation formula is: in, express The update amount for the t-1th round of training according to the loss function, Represents the value of the element in the i-th row and j-th column of the dynamic discrete cosine transform matrix during the t-1th round of training; express The update amount for the t-1th round of training according to the loss function, represents the adaptive multi-frequency filter of the t-1th round of training; η represents the learning rate; L(t-1) represents the loss function of the t-1th round; Step 1.6: Extract the frequency domain modal image Y freq A 5×5 convolutional neural network is used to extract shallow frequency domain modal forgery features, and then the shallow frequency domain modal forgery features are segmented into individual loads. The calculation formulas for the number of loads Q and the load dimension D are: Among them, Ω h ,Ω w ,Ω c They represent the vertical dimension, horizontal dimension, and number of characteristic channels of the load in the entropy space respectively; Represents the boundary control factor of the perception window.

3. According to claim 2, the deep learning-based deep forged compressed face image identification method is characterized in that: The specific process of step 2 is: Step 2.1: Capture the long-term dependencies of the entropy space context of the payload through the multi-head self-attention mechanism. After the Q payloads pass through the multi-head self-attention mechanism, they form Q high-dimensional payloads. All high-dimensional payloads constitute a high-dimensional payload set S = {s1, s2, …, s Q },s Q is the Qth high-dimensional load; Step 2.2: Use the multi-dimensional entropy load feedback mechanism to perform multi-dimensional fine-grained feature interaction and feedback attention calculation.

4. According to claim 3, the deep learning-based deep forged compressed face image identification method is characterized in that: The specific process of step 2.2 is as follows: Step 2.2.1: The entropy state of each high-dimensional load in the high-dimensional load set is an asymmetric non-normal distribution. The entropy state of each high-dimensional load is measured by the standard deviation statistical functional, and the high-dimensional loads are globally sorted according to the entropy state size. The calculation formula is: Where R(S) represents the sorted high-dimensional load set; E[·] represents the expected value of the entropy state; F Rank (·) indicates a sorting algorithm; Step 2.2.2: Decompose the sorted high-dimensional load set R(S) into two subsets, which are mapped to different entropy dimension regions. The two subsets are low-entropy high-dimensional load subsets R(S) a and high entropy state high dimensional load subset R(S) b , where a and b represent different entropy dimensions; the high entropy state load is included in R(S) b middle; Step 2.2.3: Adjust the size and offset of the perception window To adaptively reconstruct R(S) b ,During reconstruction, the boundary control factor of the perception window is set to twice the original; The reconstructed high-entropy high-dimensional load subset is R(S) b ′,R(S) b The calculation formulas for the number of high-dimensional loads Q′ and the dimension of high-dimensional loads D′ are: Step 2.2.4: R(S) a and R(S) b ′ performs two-way cross-dimensional interaction; the specific calculation formula is: Among them, cross_atten a 、cross_atten b Respectively represent R(S) a 、R(S) b ′High-dimensional load characteristics after bidirectional cross-dimensional interaction; g cross _ attention (·) indicates the cross-attention fusion method; Step 2.2.5: cross_atten b Perform shape correction and set cross_atten b Revert to cross_atten a Same shape, after changing to the same shape, cross_atten a and cross_atten b The final high-dimensional load characteristics in the frequency domain are obtained by cascading in the entropy state dimension.

5. According to claim 4, the deep learning-based deep forged compressed face image identification method is characterized in that: In step 3, the deep residual network ResNet50 is pre-trained on the InmageNet dataset. ResNet50 extracts features by stacking residual blocks. The calculation formula for each residual block is: in, represents the residual convolution operation performed by the kth residual block; represents the spatial features extracted from the kth residual block; represents the spatial features extracted from the k-1th residual block; Represents the total number of residual blocks.

6. The deep learning-based deep forged compressed face image identification method according to claim 5, characterized in that: The specific process of step 4 is as follows: Step 4.1: First, concatenate the spatial domain features and the frequency domain high-dimensional load features, and then calculate the attention weight W of the concatenated features. atten , the calculation formula is: Among them, Features spa Represents spatial features; Features freq represents the high-dimensional load feature in the frequency domain; Conv(·) represents a 1×1 convolution operation; Φ(·) represents the RELU activation function; Step 4.2: Multiply the attention weight of the cascade feature by the cascade feature element by element to obtain the fused feature.

7. The deep learning-based deep forged compressed face image identification method according to claim 6, characterized in that: The specific process of step 5 is as follows: Step 5.

1. Use the fully connected layer to calculate the unnormalized score of the fused feature. The fully connected layer projects the dimension of the fused feature to the dimension of the category to be classified. The formula for the fully connected layer projection is as follows: in, Indicates the classification category, Indicates the forged category, represents the real category; Indicates category Normalized score of Features fus Indicates fusion features; Indicates category The corresponding weight matrix of the fully connected layer; Indicates category The corresponding fully connected layer bias; Step 5.2: Use the Softmax activation function to convert the unnormalized scores into corresponding probabilities, and select the category with the largest probability as the output category determined by the model. The probability calculation formula of the Softmax activation function is as follows: Among them, P represents the probability of different categories; are the normalized scores of the fake and real categories respectively.

Images