A deep fake image detection method integrating deep and wide learning

By combining depth and width learning methods, combined with frequency domain feature extraction and channel convolution autoencoding modules, the problems of poor generalization ability and high computational cost of deep forgery image detection in the prior art are solved, and efficient and accurate Detection effect.

CN114549481BActive Publication Date: 2024-11-29HEBEI UNIV OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210177248.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-25
Publication Date
2024-11-29
Estimated Expiration
2042-02-25

AI Technical Summary

Technical Problem

The existing deep forged image detection methods have poor generalization capabilities when processing forged images generated by unknown generators, and have high training time and calculation costs.

Method used

Using a method of fusion depth and width learning, image features are extracted through the frequency domain feature extraction module and the channel convolution auto-encoding module, and a lightweight classifier is built using the width learning system to reduce training time and calculation costs.

Benefits of technology

Improve the accuracy and efficiency of detection results, and can detect tamper traces in depth forged images faster, reducing the need for training data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549481B_ABST
    Figure CN114549481B_ABST
Patent Text Reader

Abstract

The present invention is a deep forgery image detection method that integrates deep and wide learning. First, the image to be detected is converted into a grayscale image and divided into two groups of image blocks. The centralized frequency domain amplitude spectrum of each image block is extracted and an attention mechanism is applied to it. Then, the two groups of image blocks are channel-connected to obtain primary features. Secondly, a channel convolution autoencoder module is constructed, and the encoder of the pre-trained channel convolution autoencoder module is used to perform feature fusion on the primary features to obtain two intermediate features, which are respectively used as inputs of the feature mapping stream and the feature enhancement stream to obtain two mapping features and enhanced features. Finally, three classifiers are constructed according to the principle of the wide learning system, and the output results of the three classifiers are weighted averaged to obtain the final detection result. This method applies an attention mechanism to the image blocks, and can focus on the areas with obvious tampering traces from the global information. The data and time required for model training are small, and both accuracy and efficiency are achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of forged image detection, and specifically is a deep forged image detection method integrating depth and width learning. Background Art

[0002] With the development of computer technology, it is becoming easier to tamper with or synthesize images through artificial intelligence (AI), ProGAN, AttGAN and other technologies. The images obtained by these methods are called deep fake images. At present, deep fake images have reached the level of being indistinguishable from the real ones, posing a great threat to the security field.

[0003] Currently, classic deep learning models such as VGG, DenseNet, and Xception are commonly used to detect deep fake images. Although deep learning models have achieved good detection results on a single dataset, the improvement of detection performance has also brought about a sharp increase in the number of parameters and data requirements. The model needs to be retrained for different types of image detection, and the time and computational cost of retraining are high.

[0004] The shortcomings of deep learning models have prompted scholars to use non-deep methods for image detection. In the paper "DeepFake Detection by Analyzing Convolutional Traces" published by Luca et al. in the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops in 2020, based on the impact of the transposed convolutional layer of the GAN generator on the local pixel distribution, a clustering algorithm was used to solve a filter with sufficient discrimination. The filter is used to extract the relationship between local pixels, namely the convolution traces, and use this as a feature to detect deep fake images. In the paper "DefakeHop: A Light-Weight High-Performance Deepfake Detector" published by Chen et al. in the IEEE International Conference on Multimedia and Expo in 2021, the face image is divided into blocks and multiple Pixelhop++ modules are used to extract features and reduce the dimension of local face areas respectively, and then the areas are integrated and classified.

[0005] In terms of feature extraction, most of the current detection methods use the artifacts of the tampered area in the image as features, and enhance the feature extraction capability by suppressing the image content, extracting local features by region, and converting the image to the frequency domain. With the continuous development of deep fake technology, deep fake images are becoming more and more refined, and the traces of tampering left in the pixel content are becoming less and less, and the artifact clues can be eliminated through post-processing, such as Gaussian blur. In addition, Yu et al. published a paper titled "Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints" in the IEEE / CVF International Conference on Computer Vision in 2019, which analyzed the possibility of using GAN fingerprints for deep fake image detection. Existing feature extraction methods based on GAN fingerprints include detecting the impact of the transposed convolution layer of GAN on local pixels, clustering after extracting GAN fingerprints, etc. Most feature extraction methods based on GAN fingerprints mainly rely on the structure of the GAN generator, which makes the detection model overfit on the specific fake images involved in the training, and cannot handle fake images generated by unknown generators, and the feature representation ability and method generalization are poor. Summary of the invention

[0006] In view of the deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a deep fake image detection method that integrates depth and width learning.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] A deep fake image detection method integrating depth and width learning, characterized in that the method comprises the following steps:

[0009] The first step is to obtain the image to be detected and convert it into a grayscale image with a size of 256*256 pixels;

[0010] The second step is to use the frequency domain feature extraction module to extract the primary features of the image to be detected;

[0011] 2-1 The grayscale image is divided twice to obtain two image block groups A and A′, where the image block group A = {a1, a2, …, a 256}Contains 256 image blocks of size 16*16 pixels a i ,i=1,2,…,256; image block group A′={a′1,a′2,…,a′ 64} contains 64 image blocks a′ of size 32*32 pixels j, j = 1, 2, ..., 64; for each image block, a frequency domain amplitude spectrum is obtained by fast Fourier transform, and then the frequency domain amplitude spectrum is centralized to obtain a centralized frequency domain amplitude spectrum of the image block;

[0012] 2-2 Perform global fast Fourier transform on the grayscale image, and then center the frequency domain amplitude spectrum to obtain the centralized frequency domain amplitude spectrum of the grayscale image; perform high-pass filtering on the centralized frequency domain amplitude spectrum of the grayscale image, perform inverse fast Fourier transform and de-centering on the centralized frequency domain amplitude spectrum after high-pass filtering, map the centralized frequency domain amplitude spectrum to the spatial domain, and finally normalize the centralized frequency domain amplitude spectrum mapped to the spatial domain through a nonlinear function to obtain a global high-frequency attention map;

[0013] 2-3 Segment the global high-frequency attention map to obtain two local attention block groups Att and Att′, where the local attention block group Att = {Att1, Att2, …, Att 256}Contains 256 local attention blocks Att i , local attention block group Att′={Att′1,Att′2,…,Att′ 64}Contains 64 local attention blocks Att′ j , the local attention blocks correspond one-to-one to the image blocks in step 2-1; for all local attention blocks Att i and Att′ j Perform median pooling operation to obtain the attention coefficient of each image block; then multiply the attention coefficient with the centralized frequency domain amplitude spectrum of the corresponding image block to obtain the image block after attention is applied;

[0014] Through the maximum pooling operation, the size of all the image blocks that are applied to attention is adjusted to 16*16 pixels to obtain the normalized image blocks; all the normalized image blocks are connected with all the image blocks that are applied to attention through channels to obtain the primary features;

[0015] The third step is to build a channel convolution self-encoding module, which includes an encoder and a decoder. The encoder of the pre-trained channel convolution self-encoding module is used to fuse the primary features to obtain two intermediate features. and

[0016] Step 4: According to formulas (12) and (13), the intermediate features and Perform vector flattening operation to obtain two one-dimensional vectors f m and f e ; According to equations (14) and (15), the one-dimensional vector f m Perform two linear transformations to obtain two mapping features and According to the Sigmoid function of equations (16) and (17), the one-dimensional vector f e Perform two processes to obtain enhanced features and

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023] In the formula, flatten(·) represents the vector flattening operation, Respectively represent the r1-th column of the weight matrix W1 and the r1-th component of the bias vector β1, r1 = 1, 2, ..., R1; Respectively represent the r2-th column of the weight matrix W2 and the r2-th component of the bias vector β2, r2 = 1, 2, ..., R2; Respectively represent the r3th column of the weight matrix W3 and the r3th component of the bias vector β3, r3 = 1, 2, ..., R3; They represent the r4th column of the weight matrix W4 and the r4th component of the bias vector β4, r4 = 1, 2, ..., R4; W1, W2, W3 and W4 are all randomly generated weight matrices, β1, β2, β3 and β4 are all randomly generated bias vectors, R1 and R2 are mapping features respectively. and The dimension of R3 and R4 are enhanced features. and Dimensions;

[0024] Step 5: Build three classifiers based on the principle of width learning system; Choose one mapping feature and two enhancement features Select any one of the enhanced features for splicing to obtain the splicing feature; use the splicing feature, the remaining mapping features and the enhanced features as the input of the three classifiers, and calculate the weight matrix of the output layer of each classifier; multiply the weight matrix of each classifier output layer by the input of the classifier to obtain the output result of the classifier; finally, perform weighted average on the output results of the three classifiers to obtain the final detection result;

[0025] This completes the detection of deep fake images.

[0026] Compared with the prior art, the present invention has the following outstanding substantive features and significant improvements:

[0027] (1) In order to make the model easier to detect tampering traces, the present invention first divides the image into multiple image blocks. Since the texture features of deep fake images are obvious in the high-frequency area, the frequency domain feature extraction module obtains a global high-frequency attention map through a high-pass filter attention mechanism. The high-pass filter attention mechanism can highlight or suppress local features in low-level frequency domain features. In effect, it enhances the features of high-frequency areas and suppresses the features of low-frequency areas, so that the method can focus on the areas with obvious tampering traces from the global information, making the detection results more accurate. Since the encoder of the pre-trained channel convolution self-encoder module is used as a feature extractor to fuse the primary features, the channel convolution self-encoder module only needs to be trained once with a real image and can be used for the detection of any type of deep fake images, reducing the training time of the model. The convolution kernel size of the channel convolution self-encoder module is 1*1, which is essentially a feature fusion of the primary features, which can significantly reduce the feature dimension and the number of weight matrices, and to a certain extent overcome the defect that the traditional width learning system needs to randomly generate a large number of weight matrices for complex pattern recognition tasks. Three classifiers are constructed using the principle of the width learning system to achieve lightweight classifiers, greatly saving time and computing costs, and improving detection speed. A comprehensive classifier is added on the basis of the width learning system to avoid the unstable results caused by a small random weight matrix. The traditional deep learning model requires a large number of images to participate in the training, while this method only needs to train the channel convolution autoencoder module, and the number of images required for training is small. In addition, the classifier is based on the principle of width learning and can complete the learning through matrix operations without lengthy training. Therefore, the method of the present invention achieves a balance between accuracy and efficiency and significantly reduces data requirements.

[0028] (2) A Chinese patent with publication number CN112183501A discloses a method and device for detecting deep fake images. The method extracts traditional image features including facial texture features through a deep convolutional neural network, and further processes the traditional image features including facial texture features to achieve image detection. Facial texture features are shallow features. With the advancement of deep fake technology, deep fake face images have become very fine, so the texture information contained in the image is very close to the real face. Therefore, the detection ability of this method is relatively weak. Compared with CN112183501A, the present invention uses real images to train a channel convolution self-encoder module, so that the channel convolution self-encoder module fits the restoration process of the real image. Therefore, the encoder of the channel convolution self-encoder module is used to extract features from real images and deep fake images. The extracted features have sufficient distinguishability, which improves the accuracy of the detection results.

[0029] (3) A Chinese patent with publication number CN112528969A discloses a method for detecting the authenticity of a face image. The method extracts the texture features, the gradient-based clarity index, and the frequency domain features of the face image, and combines them with the depth features to achieve multi-feature-based face image detection. Compared with CN112528969A, the present invention guides the feature extraction of the original image by applying attention to the original image. The feature extraction process is simple and efficient, and the region with obvious tampering traces can be focused on from the global information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the overall flow chart of the present invention;

[0031] Figure 2 It is a structural schematic diagram of the frequency domain feature extraction module of the present invention;

[0032] Figure 3 It is a structural diagram of the channel convolution self-encoding module of the present invention. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is described in detail below in conjunction with embodiments, but the protection scope of the present application is not limited thereto.

[0034] The present invention is a deep fake image detection method (hereinafter referred to as method) integrating depth and width learning, comprising the following steps:

[0035] The first step is data preparation. The image to be detected is obtained and converted into a grayscale image with a size of 256*256 pixels.

[0036] The second step is to use the frequency domain feature extraction module (FFE) to extract the primary features of the image to be detected;

[0037] 2-1 As Figure 2 As shown, the grayscale image is segmented twice to obtain two image block groups A and A′, where the image block group A = {a1, a2, …, a 256}Contains 256 image blocks of size 16*16 pixels a i , image block group A′={a′1,a′2,…,a′ 64} contains 64 image blocks a′ of size 32*32 pixels j ; For each image block, the frequency domain amplitude spectrum is obtained by fast Fourier transform, and then the frequency domain amplitude spectrum is centralized to obtain the centralized frequency domain amplitude spectrum of the image block. The specific expressions are shown in equations (1) and (2):

[0038] A i = shift(FFT(a i )) i=1,2,…,256 (1)

[0039] A′ j =shift(FFT(a′ j )) j=1,2,…,64 (2)

[0040] Where FFT(·) represents fast Fourier transform, shift(·) represents the central processing operation; A i , A′ j Represents image block a respectively i and a′ j The centered frequency domain amplitude spectrum of;

[0041] 2-2 According to formula (3), the grayscale image is globally fast Fourier transformed, the grayscale image is mapped to the frequency domain, and the frequency domain amplitude spectrum is centralized to obtain the centralized frequency domain amplitude spectrum of the grayscale image; the centralized frequency domain amplitude spectrum of the grayscale image is high-pass filtered by formula (4) to obtain the centralized frequency domain amplitude spectrum after high-pass filtering; then according to formula (5), the centralized frequency domain amplitude spectrum after high-pass filtering is inversely fast Fourier transformed and decentralized to map the centralized frequency domain amplitude spectrum to the spatial domain, and finally the centralized frequency domain amplitude spectrum mapped to the spatial domain is normalized by the Sigmoid function to obtain the global high-frequency attention map;

[0042] A global =shift(FFT(I gray )) (3)

[0043]

[0044] I att =Sigmoid(IFFT(Ishift(A filtered ))) (5)

[0045] Among them, A global Represents the input image grayscale image I gray The centered frequency domain amplitude spectrum, T high_pass represents the high-pass filter template, represents the multiplication operation, A filtered represents the centralized frequency domain amplitude spectrum after high-pass filtering, IFFT(·) and Ishift(·) represent the inverse fast Fourier transform and decentralization operations respectively, and I att represents the global high-frequency attention map;

[0046] 2-3 Similarly, for the global high-frequency attention map I att Perform the same segmentation as in step 2-1 to obtain two local attention block groups Att and Att′, where the local attention block group Att = {Att1, Att2, …, Att 256}Contains 256 local attention blocks Att i , local attention block group Att′={Att′1,Att′2,…,Att′ 64}Contains 64 local attention blocks Att′ j , the local attention blocks correspond one-to-one to the image blocks in step 2-1; for all local attention blocks Att i and Att′ j Perform median pooling to obtain the attention coefficient of each image block; then multiply the attention coefficient by the centered frequency domain amplitude spectrum of the corresponding image block to obtain the image block after attention is applied, see equations (6) and (7);

[0047] M i =MidPool(Att i , 16)·A i i=1,2,…,256 (6)

[0048] M′ j =MidPool(Att′ j , 32)·A′ j j = 1, 2, ..., 64 (7)

[0049] Among them, MidPool(·, n) means median pooling operation according to n*n size, n=16, 32; M i , M′ j Represents the image block P i and P′ j Image patch after attention is applied;

[0050] Through the maximum pooling operation of formula (8), all the image blocks M′ that are given attention are jThe size is adjusted to 16*16 pixels to obtain the normalized image block N j , the convolution kernel size of the maximum pooling is 2*2; finally, according to formula (9), all normalized image blocks and all image blocks M that apply attention are combined. i Perform channel connection to obtain the primary feature F1 with a size of 320*320*16 pixels. F1 is the primary feature extracted by the frequency domain feature extraction module.

[0051] N j =MaxPool(M′ j , 2) j = 1, 2, ..., 64 (8)

[0052] F1={M1,M2,…,M 256 , N1, N2, …, N 64} (9)

[0053] Where MaxPool(·, n) means the maximum value pooling operation is performed according to the size of n*n, n=2;

[0054] The third step is to build a channel convolutional autoencoder module (CAE), such as Figure 3 As shown in Figure 1, the channel convolution self-encoding module includes an encoder and a decoder. The encoder is used for channel fusion and contains four consecutive convolutional layers. The convolution kernel size of each convolutional layer is 1*1, and the number of convolution kernels is 256, 128, 64, and 32 respectively. The decoder and the encoder are symmetrical structures. The decoder is used to restore the number of channels and contains four consecutive convolutional layers. The convolution kernel size is 1*1, and the number of convolution kernels is 32, 64, 128, and 256 respectively. The encoder of the pre-trained channel convolution self-encoding module is used to fuse the primary feature F1 to obtain two intermediate features. and The intermediate features and As input to the feature mapping stream and feature enhancement stream respectively;

[0055] The expression of the encoder is:

[0056]

[0057] In the formula, Indicates that g convolution kernels of size 1*1 are used to perform convolution operation on feature X, g = 256, 128, 64, 32;

[0058] According to formula (11), the intermediate features Perform convolution operation to obtain another intermediate feature

[0059]

[0060] Step 4: According to formulas (12) and (13), the intermediate features and Perform vector flattening operation to obtain two one-dimensional vectors f m and f e ; According to equations (14) and (15), the one-dimensional vector f m Perform two linear transformations to obtain two mapping features and According to the Sigmoid function of equations (16) and (17), the one-dimensional vector f e Perform two processes to obtain enhanced features and

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067] In the formula, flatten(·) represents the vector flattening operation, Respectively represent the r1-th column of the weight matrix W1 and the r1-th component of the bias vector β1, r1 = 1, 2, ..., R1; Respectively represent the r2-th column of the weight matrix W2 and the r2-th component of the bias vector β2, r2 = 1, 2, ..., R2; Respectively represent the r3th column of the weight matrix W3 and the r3th component of the bias vector β3, r3 = 1, 2, ..., R3; They represent the r4th column of the weight matrix W4 and the r4th component of the bias vector β4, r4 = 1, 2, ..., R4; W1, W2, W3 and W4 are all randomly generated weight matrices, β1, β2, β3 and β4 are all randomly generated bias vectors, R1 and R2 are mapping features respectively. and The dimension of R3 and R4 are enhanced features. and Dimensions;

[0068] Step 5: Build three classifiers based on the principle of width learning system; Choose one mapping feature and two enhancement features Select any one of the enhanced features for splicing to obtain the spliced ​​features; use the spliced ​​features, the remaining mapped features and the enhanced features as the inputs of the three classifiers, and calculate the weight matrix of the classifier output layer according to the width learning system principle of equations (18) and (19);

[0069]

[0070] P=Q + Y (19)

[0071] Among them, Q represents the concatenated features, the remaining mapped features or the remaining enhanced features, I represents the identity matrix, λ represents the regularization coefficient, T represents the matrix transpose, Y represents the sample label corresponding to the input image during the classifier learning process, and P represents the weight matrix of the classifier output layer;

[0072] The weight matrix of each classifier output layer is multiplied by the input of the classifier to obtain the output result of the classifier; finally, the output results of the three classifiers are weighted averaged to obtain the final detection result, thus completing the detection of deep fake images.

[0073] Example 1

[0074] This embodiment takes a deep fake face image as an example to illustrate a deep fake image detection method integrating depth and width learning, including the following steps:

[0075] Step 1. Data preparation. This embodiment uses real face images from the CelebA-HQ and CelebA datasets as the basis, and generates deep fake face images through StarGAN, StyleGAN, StyleGAN2, AttGAN, and FaceForensics++ methods, respectively, to obtain five training sets and five test sets, each of which contains 1,500 and 500 deep fake face images, respectively. FaceForensics++ is open source fake face video data. This embodiment uses a single-frame face image of its fake video as a deep fake face image to demonstrate that the method of the present invention is also applicable to deep fake face videos. All images are converted into grayscale images of 256*256 pixels in size.

[0076] The second step is to use the frequency domain feature extraction module to extract the primary features of the deep fake face image;

[0077] 2-1 Segment the grayscale image twice to obtain two image block groups A and A′, where the image block group A = {a1, a2, …, a 256}Contains 256 image blocks of size 16*16 pixels a i , image block group A′={a′1,a′2,…,a′ 64} contains 64 image blocks a′ of size 32*32 pixels j ; For each image block, the frequency domain amplitude spectrum is obtained by fast Fourier transform, and then the frequency domain amplitude spectrum is centralized to obtain the centralized frequency domain amplitude spectrum of the image block. The specific expressions are shown in equations (1) and (2):

[0078] A i = shift(FFT(a i )) i=1,2,…,256 (1)

[0079] A′ j =shift(FFT(a′ j )) j=1,2,…,64 (2)

[0080] Where FFT(·) represents fast Fourier transform, shift(·) represents the central processing operation; A i , A′ j Represents image block a respectively i and a′ j The centered frequency domain amplitude spectrum of;

[0081] 2-2 According to formula (3), the grayscale image is globally fast Fourier transformed, the grayscale image is mapped to the frequency domain, and the frequency domain amplitude spectrum is centralized to obtain the centralized frequency domain amplitude spectrum of the grayscale image; the centralized frequency domain amplitude spectrum of the grayscale image is high-pass filtered by formula (4) to remove the low-frequency components in the centralized frequency domain amplitude spectrum, and the centralized frequency domain amplitude spectrum after high-pass filtering is obtained; then according to formula (5), the centralized frequency domain amplitude spectrum after high-pass filtering is inversely fast Fourier transformed and de-centered, and the centralized frequency domain amplitude spectrum is mapped to the spatial domain. Finally, the centralized frequency domain amplitude spectrum is mapped to the spatial domain by the Sigmoid function for normalization to obtain the global high-frequency attention map;

[0082] A global =shift(FFT(I gray )) (3)

[0083]

[0084] I att =Sigmoid(IFFT(Ishift(A filtered ))) (5)

[0085] Among them, A global Represents the grayscale image I gray The centered frequency domain amplitude spectrum, T high_pass represents the high-pass filter template, represents the multiplication operation, A filteredrepresents the centralized frequency domain amplitude spectrum after high-pass filtering, IFFT(·) and Ishift(·) represent the inverse fast Fourier transform and decentralization operations respectively, and I att represents the global high-frequency attention map;

[0086] 2-3 Similarly, for the global high-frequency attention map I att Perform the same segmentation as in step 2-1 to obtain two local attention block groups Att and Att′, where the local attention block group Att = {Att1, Att2, …, Att 256}Contains 256 local attention blocks Att i , local attention block group Att′={Att′1,Att′2,…,Att′ 64}Contains 64 local attention blocks Att′ j , the local attention blocks correspond one-to-one to the image blocks in step 2-1; for all local attention blocks Att i and Att′ j Perform median pooling to obtain the attention coefficient of each image block, and then multiply the attention coefficient with the centered frequency domain amplitude spectrum of the corresponding image block to obtain the image block to which attention is applied, see equations (6) and (7);

[0087] M i =MidPool(Att i , 16)·A i i=1,2,…,256 (6)

[0088] M′ j =MidPool(Att′ j , 32)·A′ j j = 1, 2, ..., 64 (7)

[0089] Among them, MidPool(·, n) means median pooling operation according to n*n size, n=16, 32; M i , M′ j Represents the image block P i and P′ j Image patch after attention is applied;

[0090] Through the maximum pooling operation of formula (8), all the image blocks M′ that are paid attention to are j The size is adjusted to 16*16 pixels to obtain the normalized image block N j ; Finally, all normalized image blocks and all image blocks M that apply attention i Channel connection is performed to obtain the primary feature F1 with a size of 320*320*16 pixels. F1 is the primary feature extracted by the frequency domain feature extraction module. Its expression is shown in formula (9);

[0091] N j =MaxPool(M′ j ,2) j=1,2,…,64 (8)

[0092] F1={M1,M2,…,M 256 , N1, N2, …, N 64} (9)

[0093] Where MaxPool(·, n) means the maximum value pooling operation is performed according to the size of n*n, n=2;

[0094] The third step is to build a channel convolution self-encoding module, such as Figure 3 As shown in the figure, the channel convolution self-encoding module includes an encoder and a decoder. The encoder contains four consecutive convolutional layers. The convolution kernel size of each convolutional layer is 1*1, and the number of convolution kernels is 256, 128, 64, and 32 respectively. The decoder and the encoder have a symmetrical structure. The decoder contains four consecutive convolutional layers. The convolution kernel size is 1*1, and the number of convolution kernels is 32, 64, 128, and 256 respectively.

[0095] The expression of the encoder is:

[0096]

[0097] In the formula, Indicates that g convolution kernels of size 1*1 are used to perform convolution operation on feature X, g = 256, 128, 64, 32;

[0098] According to formula (11), the intermediate features Perform convolution operation to obtain another intermediate feature

[0099]

[0100] The primary features of the 2000 real face images extracted in the second step are used to train the channel convolutional autoencoder module. The Adam optimizer is used to calculate the loss using the mean square loss function. After the channel convolutional autoencoder module converges, the training of the channel convolutional autoencoder module is completed.

[0101] The encoder of the pre-trained channel convolutional autoencoder module is used to fuse the primary feature F1 to obtain two intermediate features. and The intermediate features and As input to the feature mapping stream and feature enhancement stream respectively;

[0102] Step 4: According to formulas (12) and (13), the intermediate features and Perform vector flattening operation to obtain two one-dimensional vectors f m and f e ; According to equations (14) and (15), the one-dimensional vector f m Perform two linear transformations to obtain two mapping features and According to the Sigmoid function of equations (16) and (17), the one-dimensional vector f e Perform two processes to obtain two enhanced features and

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] In the formula, flatten(·) represents the vector flattening operation, Respectively represent the r1-th column of the weight matrix W1 and the r1-th component of the bias vector β1, r1 = 1, 2, ..., R1; Respectively represent the r2-th column of the weight matrix W2 and the r2-th component of the bias vector β2, r2 = 1, 2, ..., R2; Respectively represent the r3th column of the weight matrix W3 and the r3th component of the bias vector β3, r3 = 1, 2, ..., R3; They represent the r4th column of the weight matrix W4 and the r4th component of the bias vector β4, r4 = 1, 2, ..., R4; W1, W2, W3 and W4 are all randomly generated weight matrices, β1, β2, β3 and β4 are all randomly generated bias vectors, R1 and R2 are mapping features respectively. and The dimension of R3 and R4 are enhanced features. and Dimensions;

[0110] Step 5: Build three classifiers based on the principle of width learning system; Choose one mapping feature and two enhancement features And any one of the enhancement features is spliced. In this embodiment, the mapping feature is selected and Enhanced Features Get splicing features Join features The remaining mapped features and Enhanced Features These three features are used as the input of three classifiers, and the weight matrix of the classifier output layer is calculated according to the principle of width learning system in equations (18) and (19), which are P1, P2, and P3 respectively;

[0111]

[0112] P=Q + Y (19)

[0113] Among them, Q represents the concatenated features, the remaining mapped features or the remaining enhanced features, I represents the identity matrix, λ represents the regularization coefficient, Y represents the sample label corresponding to the input image during the classifier learning process, and P represents the weight matrix of the classifier output layer;

[0114] According to equations (20) to (22), the weight matrices of the output layers of the three classifiers are multiplied by the inputs of the corresponding classifiers to obtain the output results Y'1, Y'2 and Y'3 of the three classifiers;

[0115]

[0116]

[0117]

[0118] The output results of the three classifiers are weighted averaged by formula (23) to obtain the final detection result;

[0119] Y'=αY'1+βY'2+γY'3 (23)

[0120] Wherein, α, β and γ are weight coefficients of three classifiers respectively, α+β+γ=1, in this embodiment, α=β=0.2, γ=0.4;

[0121] At this point, the deep fake image detection integrating depth and width learning of this embodiment is completed.

[0122] Table 1 Data details and training time of each training set

[0123]

[0124] As can be seen from Table 1, the total construction time of the method of the present invention is short. In this embodiment, 3000 training images are used to participate in the method construction process, and the total time is only about 60 minutes, of which the weight matrix calculation time of the classifier output layer is only about 2 minutes. It is worth noting that the total time for method construction in Table 1 only represents the time consumed when the method is initially constructed. When the method is reconstructed, there is no need to train the channel convolution autoencoder module. The total time for method reconstruction is the time for calculating the weight matrix of the classifier output layer.

[0125] Table 2 Data details and test results of each test set

[0126]

[0127] As can be seen from Table 2, the method of the present invention can achieve good detection results on the five test sets, with an accuracy rate of more than 98%. It has strong adaptability to deep fake face images synthesized by different technologies, while taking into account both detection accuracy and detection efficiency.

[0128] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A deep fake image detection method integrating deep and wide learning, characterized in that: The method comprises the following steps: The first step is to obtain the image to be detected and convert it into a grayscale image with a size of 256*256 pixels; The second step is to use the frequency domain feature extraction module to extract the primary features of the image to be detected; 2-1 The grayscale image is divided twice to obtain two image block groups A and A′, where the image block group A = {a1, a2, …, a 256 }Contains 256 image blocks of size 16*16 pixels a i ,i=1,2,…,256; image block group A′={a′1,a′2,…,a′ 64 } contains 64 image blocks a′ of size 32*32 pixels j , j = 1, 2, ..., 64; for each image block, a frequency domain amplitude spectrum is obtained by fast Fourier transform, and then the frequency domain amplitude spectrum is centralized to obtain a centralized frequency domain amplitude spectrum of the image block; 2-2 Perform global fast Fourier transform on the grayscale image, and then center the frequency domain amplitude spectrum to obtain the centralized frequency domain amplitude spectrum of the grayscale image; perform high-pass filtering on the centralized frequency domain amplitude spectrum of the grayscale image, perform inverse fast Fourier transform and de-centering on the centralized frequency domain amplitude spectrum after high-pass filtering, map the centralized frequency domain amplitude spectrum to the spatial domain, and finally normalize the centralized frequency domain amplitude spectrum mapped to the spatial domain through a nonlinear function to obtain a global high-frequency attention map; 2-3 Segment the global high-frequency attention map to obtain two local attention block groups Att and Att′, where the local attention block group Att = {Att1, Att2, …, Att 256 }Contains 256 local attention blocks Att i , local attention block group Att′={Att′1,Att′2,…,Att′ 64 }Contains 64 local attention blocks Att′ j , the local attention blocks correspond one-to-one to the image blocks in step 2-1; for all local attention blocks Att i and Att′ j Perform median pooling operation to obtain the attention coefficient of each image block; then multiply the attention coefficient with the centralized frequency domain amplitude spectrum of the corresponding image block to obtain the image block after attention is applied; Through the maximum pooling operation, the size of all the image blocks that are applied to attention is adjusted to 16*16 pixels to obtain the normalized image blocks; all the normalized image blocks are connected with all the image blocks that are applied to attention through channels to obtain the primary features; The third step is to build a channel convolution self-encoding module, which includes an encoder and a decoder. The encoder of the pre-trained channel convolution self-encoding module is used to fuse the primary features to obtain two intermediate features. and Step 4: According to formulas (12) and (13), the intermediate features and Perform vector flattening operation to obtain two one-dimensional vectors f m and f e ; According to equations (14) and (15), the one-dimensional vector f m Perform two linear transformations to obtain two mapping features and According to the Sigmoid function of equations (16) and (17), the one-dimensional vector f e Perform two processes to obtain enhanced features and In the formula, flatten(·) represents the vector flattening operation, Respectively represent the r1-th column of the weight matrix W1 and the r1-th component of the bias vector β1, r1 = 1, 2, ..., R1; Respectively represent the r2-th column of the weight matrix W2 and the r2-th component of the bias vector β2, r2 = 1, 2, ..., R2; Respectively represent the r3th column of the weight matrix W3 and the r3th component of the bias vector β3, r3 = 1, 2, ..., R3; They represent the r4th column of the weight matrix W4 and the r4th component of the bias vector β4, r4 = 1, 2, ..., R4; W1, W2, W3 and W4 are all randomly generated weight matrices, β1, β2, β3 and β4 are all randomly generated bias vectors, R1 and R2 are mapping features respectively. and The dimension of R3 and R4 are enhanced features. and Dimensions; Step 5: Build three classifiers based on the principle of width learning system; Choose one mapping feature and two enhancement features Select any one of the enhanced features for splicing to obtain the splicing feature; use the splicing feature, the remaining mapping features and the enhanced features as the input of the three classifiers, and calculate the weight matrix of the output layer of each classifier; multiply the weight matrix of each classifier output layer by the input of the classifier to obtain the output result of the classifier; finally, perform weighted average on the output results of the three classifiers to obtain the final detection result; This completes the detection of deep fake images.

2. The deep fake image detection method integrating depth and width learning according to claim 1 is characterized in that: The encoder and decoder of the channel convolutional autoencoder module are symmetrical in structure. Both the encoder and the decoder contain four consecutive convolutional layers, and the convolution kernel size of each convolutional layer is 1*1; the number of convolution kernels of the four convolutional layers of the encoder are 256, 128, 64, and 32 respectively; the number of convolution kernels of the four convolutional layers of the encoder are 32, 64, 128, and 256 respectively.

Citation Information

Patent Citations

  • Deep forged image detection method and device

    CN112183501A

  • Face image authenticity detection method and system, computer equipment and storage medium

    CN112528969A

  • Face-changing video tampering detection method and system based on multi-domain feature fusion

    CN112734696A

  • Novel multi-scale depth-width combined pathological picture classification method and system, and medium

    CN113657479A