Depth counterfeit image detection method fusing content-independent features and content-related features

Abstract

The invention discloses a deep counterfeit image detection method fusing content-independent features and content-related features. The method comprises the following steps: firstly, obtaining a smooth image, a texture image and a detail image through a Gaussian filter, a Gabor filter and a wavelet filter respectively, carrying out feature pyramid network processing after fusion, and obtaining an authenticity probability based on frequency domain features; secondly, a difference image and a noise image are obtained through an unsharpened mask and a DnCNN denoising model, after fusion, the difference image and the noise image are processed through the feature pyramid network, and the authenticity probability based on the spatial domain features is obtained; and the frequency domain and spatial domain features jointly form content-independent features of the image. In addition, content-related features of the image are extracted through a ViT model, and the authenticity probability based on the content-related features is obtained. And finally, carrying out weighted fusion on the three authenticity probabilities, and outputting an authenticity judgment result of the image. The method is excellent in performance in cross-generation-model and different-data-source forged image detection, and has high generalization and universality.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and forgery detection technology, and in particular to a deep forgery image detection method that integrates content-independent features with content-related features. Background Art

[0002] Artificial Intelligence Generated Content (AIGC) is a technology that uses complex algorithms, models, and rules to learn from large-scale data sets to create new original content. This technology has a wide range of applications in areas such as text and image generation, game development, and film and television production, bringing convenience to all walks of life. However, improper use of this technology has also caused some problems and brought security risks.

[0003] Early image generation techniques, such as Generative Adversarial Network (GAN), generate high-quality fake images through adversarial training of generators and discriminators. Such techniques have problems such as high training difficulty, weak scalability, and insufficient diversity of generated images. At present, some progress has been made in the detection methods for GAN-generated images. These methods include detection based on spatial or frequency domain feature differences, such as color space differences, gradient texture features, high-frequency patterns, and noise residuals. The Diffusion Model (DM), which has emerged in recent years, generates images from noise samples by gradually removing prediction noise, showing better stability, training controllability, and generated image quality. Since this technology is still an emerging field, there are few related studies. Most of the existing detection methods are directly migrated from the GAN detector, but the effect is not good, and there are problems such as low detection accuracy, poor generalization and robustness.

[0004] In addition, existing models often have difficulty dealing with forged images from unknown data sources and unknown generation models, and the versatility and generalization of the models are generally poor. Some studies have improved the generalization of models through data augmentation and dataset expansion, but the improvement effect of this method is limited, and it is difficult to fully utilize the various effective features of the generated images. There are also studies that directly migrate existing detectors to perform cross-model image detection, but the classification accuracy of the relevant models cannot meet practical requirements.

[0005] With the rapid advancement of generative artificial intelligence technology, models such as generative adversarial networks and diffusion models have been able to create realistic fake images, namely deep fake images. Although image generation technology has achieved remarkable results, the corresponding deep fake image detection technology is relatively lagging behind, especially for images generated by the currently widely used diffusion model, the existing detection methods have obvious deficiencies in accuracy. In addition, most existing detection methods only target specific image generation models and lack the ability to detect a wide range of fake images from different sources. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a deep fake image detection method that integrates content-independent features and content-related features, which can perform high-precision detection on images generated by generative adversarial networks and diffusion models.

[0007] Technical solution: A deep fake image detection method of the present invention that integrates content-independent features and content-related features comprises the following steps:

[0008] S1: Input the original image I into the Gaussian filter, Gabor filter and Wavelet filter to obtain the smoothed image I smooth , Texture Image I Gabor and detail image I detail I smooth ,I Gabor and I detail It is called the frequency domain feature of the image. These three features are spliced ​​in the channel dimension to obtain the fused frequency domain feature I freq . Will I freq Input the Feature Pyramid Network (FPN), and then process it through the classification head to obtain the true and false probability p based on the frequency domain features. freq ;

[0009] S2: Input the original image I into the Unsharp Masking model and the DnCNN (Denoising Convolutional Neural Network) denoising model to obtain the difference image I sharp and the noisy image I noise I sharp and I noise It is called the spatial domain feature of the image. These two features are spliced ​​in the channel dimension to obtain the fused spatial domain feature I spatial . Will I spatial Input the feature pyramid network, and then process it through the classification head to obtain the true or false probability p based on the spatial domain features.spatial The three frequency domain features obtained by S1 and the two spatial domain features obtained by S2 are collectively referred to as content-independent features of the image;

[0010] S3: Input the original image I into the pre-trained Vision Transformer (ViT) model to extract the content-related features of the image, replace the original classification head of the ViT model with a fully connected layer, and obtain the true or false probability p based on the content-related features. content ;

[0011] S4: The three true and false probabilities p obtained by S1, S2, and S3 respectively freq 、p spatial 、p content Weighted addition to obtain the final true or false probability p final , according to p final Get the true and false classification result of the image I result .

[0012] Furthermore, the step S1 specifically includes the following steps:

[0013] S1.1: Input the original image I into the Gaussian filter, use the Gaussian filter to perform weighted averaging on the image to reduce high-frequency noise and obtain a smoothed image I smooth ;

[0014] S1.2: Input the original image I into the Gabor filter to capture the texture information in different directions in the image and obtain the texture image I Gabor ;

[0015] S1.3: Input the original image I into the wavelet filter to perform image decomposition and reconstruction to obtain the detail image I detail ;

[0016] S1.4: The I obtained by S1.1, S1.2, and S1.3 smooth ,I Gabor ,I detail The fused frequency domain features I are obtained by splicing in the channel dimension freq , will I freq The input feature pyramid network is then processed by the classification head including the pooling layer, the fully connected layer and the softmax activation function to obtain the true or false probability p based on the frequency domain features. freq ;

[0017] Furthermore, the step S2 specifically includes the following steps:

[0018] S2.1: The original image I is processed using the unsharp masking technique. The original image I is first subjected to Gaussian blur processing to obtain a smoothed image I smooth, then subtract the smoothed image I from the original image I smooth Get the difference image I sharp ;

[0019] S2.2: Input the original image I into the DnCNN denoising model, and use multi-layer convolution and activation functions to obtain the noisy image I noise ;

[0020] S2.3: The I obtained by S2.1 and S2.2 sharp with I noise The fused spatial domain features I are obtained by splicing in the channel dimension spatial , will I spatial Input the feature pyramid network, and then go through the same classification head processing as S1.4 to obtain the true or false probability p based on spatial domain features. spatial ;

[0021] Furthermore, in step S3, the pre-trained ViT model is used to extract content-related features of the image. The original classification head of the ViT model is replaced with a fully connected layer with an output dimension of 2, which directly generates a binary classification prediction result, that is, the true or false probability p based on the image content-related features. content .

[0022] Further, in step S4, the true and false probability p based on the frequency domain feature obtained in step S1 is freq , the true or false probability p based on spatial domain features obtained in step S2 spatial , the true or false probability p based on content-related features obtained in step S3 content Perform weighted addition to obtain the final true or false probability p final According to p final Get the final image authenticity classification result I result .

[0023] The present invention also discloses a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention extracts content-independent features of images by combining multiple filters (Gaussian, Gabor, wavelet filters), unsharp masking technology, and DnCNN denoising model; and uses a fine-tuned pre-trained Vision Transformer (ViT) model to extract content-related features of images. Among them, the content-independent features include frequency domain features and spatial domain features, thereby constructing a complete image feature representation. Experimental results show that the model shows higher detection accuracy than existing methods for both GAN-generated images and diffusion model-generated images. In particular, for forged images across generation models and different data sources, the model has good detection performance, demonstrating its excellent generalization and versatility. The design of the model enables it to effectively deal with the problem of deep forged image detection from unknown sources, which is an important demand in the current field of forged image detection. The introduction of this model represents a technological innovation in the field of deep forged image detection and provides a new solution for this field. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the two-dimensional wavelet decomposition process in the specific implementation scheme of the present invention.

[0026] Figure 2 It is a characteristic pyramid network structure diagram in a specific implementation scheme of the present invention.

[0027] Figure 3 This is a structural diagram of DnCNN in a specific implementation scheme of the present invention.

[0028] Figure 4 It is a schematic diagram of the ViT model in a specific implementation scheme of the present invention.

[0029] Figure 5 This is a structural diagram of a detection network that integrates content-independent features and content-related features in a specific implementation scheme of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0031] The present invention is achieved by the following measures: a deep fake image detection model that integrates content-independent features and content-dependent features, which includes the following steps:

[0032] S1: Input the original image I into the Gaussian filter, Gabor filter and Wavelet filter to obtain the smoothed image I smooth , Texture Image I Gabor and detail image I detail Ismooth ,I Gabor and I detail It is called the frequency domain feature of the image. These three features are spliced ​​in the channel dimension to obtain the fused frequency domain feature I freq . Will I freq Input the Feature Pyramid Network (FPN), and then process it through the classification head to obtain the true and false probability p based on the frequency domain features. freq ;

[0033] S2: Input the original image I into the Unsharp Masking model and the DnCNN (Denoising Convolutional Neural Network) denoising model to obtain the difference image I sharp and the noisy image I noise I sharp and I noise It is called the spatial domain feature of the image. These two features are spliced ​​in the channel dimension to obtain the fused spatial domain feature I spatial . Will I spatial Input the feature pyramid network, and then process it through the classification head to obtain the true or false probability p based on the spatial domain features. spatial The three frequency domain features obtained by S1 and the two spatial domain features obtained by S2 are collectively referred to as content-independent features of the image;

[0034] S3: Input the original image I into the pre-trained Vision Transformer (ViT) model to extract the content-related features of the image, replace the original classification head of the ViT model with a fully connected layer, and obtain the true or false probability p based on the content-related features. content ;

[0035] S4: The three true and false probabilities p obtained by S1, S2, and S3 respectively freq 、p spatial 、p content Weighted addition to obtain the final true or false probability p final , according to p final Get the true and false classification result of the image I result ;

[0036] Furthermore, the step S1 specifically includes the following steps:

[0037] S1.1: Input the original image I into the Gaussian filter to obtain the smoothed image I smooth .

[0038] Specifically, the Gaussian filter is used to smooth the image and reduce high-frequency noise. The two-dimensional Gaussian distribution density function is shown as follows:

[0039]

[0040] In the formula, G(x, y) represents the value of the Gaussian function at the position (x, y), that is, the weight of the position. σ is the standard deviation of the Gaussian distribution, and μ represents the mean of the Gaussian distribution. The present invention uses a 3×3 convolution kernel, with the standard deviation and mean both set to 1. The shape of the Gaussian convolution kernel is shown in the following formula:

[0041]

[0042] The Gaussian convolution kernel is normalized, and the sum of the weights is 1, so that the total brightness of the image remains unchanged. The image is weighted averaged by applying the above Gaussian convolution kernel. The pixels in the central area have a larger weight and contribute more significantly to the final result, while the pixels in the edge area have a smaller weight, thus achieving image smoothing and extraction of low-frequency features.

[0043] S1.2: Input the original image I into the Gabor filter to obtain the texture image I Gabor ;

[0044] The Gabor filter detects texture features in a specific direction in an image by combining a Gaussian function and a sine wave function. The mathematical expression of the Gabor filter is shown below:

[0045]

[0046] Where x' and y' are the coordinates after rotation transformation, λ represents the wavelength of the filter, θ represents the direction of the parallel stripes of the Gabor function, ψ represents the phase shift of the sine wave, σ represents the standard deviation of the Gaussian function, and γ represents the spatial aspect ratio.

[0047] The present invention sets the Gabor filter kernel size to 31×31, the wavelength λ to 10, the phase offset ψ to 0, the standard deviation σ to 4, and the spatial aspect ratio γ to 0.5. In order to comprehensively consider the features in different directions, the present invention adopts the maximum merging strategy of multi-directional responses, uses 16 Gabor filter kernels in different directions, and sets their stripe directions θ to be uniformly distributed between 0° and 360°. Finally, the Gabor kernel is normalized as shown in the following formula to ensure that the filter has consistent response strength in different directions. Through the above settings, the Gabor filter can capture multi-directional texture information in the image.

[0048]

[0049] S1.3: Input the original image I into the wavelet filter to obtain the detail image I detail ;

[0050] In the present invention, Haar is used as the basis function of wavelet filtering, and two-dimensional wavelet transform is used to perform wavelet decomposition and reconstruction on the image. The two-dimensional wavelet decomposition process is as follows: Figure 1 The process of wavelet decomposition and reconstruction is shown in the following formula:

[0051] DWT(I)→(LL,LH,HL,HH)

[0052] I detail =∑(|LH|+|HL|+|HH|)

[0053] In the formula, I represents the input image, LL represents the low-frequency part of the image, that is, the overall structure and large-scale features of the image; LH, HL, and HH represent the high-frequency components of the image in the horizontal, vertical, and diagonal directions, respectively. The present invention first performs a two-dimensional wavelet transform decomposition on the original input image to obtain a low-frequency vector LL and high-frequency vectors LH, HL, and HH in three directions. Then the high-frequency components in the three directions are merged to obtain a complete high-frequency image, that is, the detail image I detail Before applying the wavelet transform, the image pixel values ​​are normalized to the range [0,255] to ensure the stability and consistency of the image data during the transformation process. The results after the wavelet transform are also denormalized to ensure that the image data is suitable for further processing and analysis.

[0054] S1.4: The I obtained by S1.1, S1.2 and S1.3 smooth ,I Gabor ,I detail The fused frequency domain features I are obtained by splicing in the channel dimension freq , will I freq The input feature pyramid network is then processed by the classification head including the pooling layer, the fully connected layer and the softmax activation function to obtain the true or false probability p based on the frequency domain features. freq , whose value range is between [0,1]. The structure of the feature pyramid network is as follows Figure 2 shown.

[0055] Furthermore, the step S2 specifically includes the following steps:

[0056] S2.1: The original image I is passed through the unsharp mask model to obtain the difference image I sharp ;

[0057] The present invention first divides the original image I into three color channels I R ,I G ,I B , and to I R ,I G ,I B Gaussian blur processing is performed respectively to generate a smooth image Ismooth_R ,I smooth_G ,I smooth_B For each color channel, the smoothed image is subtracted from the original image, and the results are recorded as I diff_R ,I diff_G ,I diff_B , that is I diff_x =I x -I smooth_x Then the i of the three color channels diff_R ,I diff_G ,I diff_B Add together to get the complete I diff Here we set a threshold value T = 50, and the final difference image I sharp Only I is retained diff The pixel values ​​in the image are greater than or equal to T, and the other pixel values ​​are set to 0. The above process can be expressed as:

[0058]

[0059] S2.2: Input the original image I into the DnCNN denoising model, and obtain the noisy image I through multiple layers of convolution and activation functions. noise ;

[0060] The DnCNN structure of the present invention is as follows Figure 3 As shown in Figure 2, a blind denoising model (DnCNN with Blind NoiseLevel) is selected to adapt to noises of different types and intensities, where the number of layers of the model and the noise level of the test set are set to 20 and 25 respectively. For each original image I, it is separated into I R ,I G and I B Three color channels. Apply DnCNN to each channel to get the predicted noise image I noise_R ,I noise_G and I noise_B The noise images of each color channel are concatenated in the channel dimension to obtain the complete noise image I noise .

[0061] S2.3: The I obtained by S2.1 and S2.2 sharp with I noise The fused spatial domain features I are obtained by splicing in the channel dimension spatial , will I spatial Input the feature pyramid network, and then go through the same classification head processing as S1.4 to obtain the true or false probability p based on spatial domain features. spatial , its value range is [0,1].

[0062] Furthermore, in step S3, a Vision Transformer (ViT) model is used to extract content-related features of the image. The structure of the ViT model used is as follows: Figure 4 Select the ViT "vit_b_16" pre-trained model, and replace the original classification head of the model with a fully connected layer with an output dimension of 2, and output the true and false probability p based on content-related features. content , its value range is [0,1].

[0063] Further, in step S4, the three true and false probabilities p obtained by S1, S2, and S3 are freq 、p spatial 、p content The final probability of authenticity p is obtained by adding them together through the weighted voting mechanism. final . freq 、p spatial 、p content The weights are recorded as weight1, weight2, and weight3, and the initial values ​​of the three weights are set to 0.4, 0.2, and 0.4 respectively. final =weight1*p freq +weight2*p spatial +weight3*p content The overall structure of the model is as follows Figure 5 As shown in Figure 2. During the training process, the model can automatically adjust the weights according to the contribution of different features to better balance the influence of each branch. The weighted voting mechanism can comprehensively consider the feature information from different branches and reduce possible conflicts, thereby maintaining the high performance of the model when processing complex and diverse forged images. Finally, the true and false probability p final The output of the model is obtained, and the image is classified into a category with a larger probability of authenticity, thereby determining its authenticity classification result.

Claims

1. A deep fake image detection method that integrates content-independent features and content-dependent features, characterized in that: The steps include: Step S1: Input the original image I into the Gaussian filter, Gabor filter and wavelet filter respectively to obtain smoothed images I smooth , Texture Image I Gabor and detail image I detail ;I smooth ,I Gabor and I detail It is called the frequency domain feature of the image. These three features are spliced ​​in the channel dimension to obtain the fused frequency domain feature I freq ; will I freq The input is fed into the feature pyramid network FPN, and then processed by the classification head to obtain the true or false probability p based on the frequency domain features. freq ; Step S2: Input the original image I into the unsharp mask model and the DnCNN denoising model to obtain the difference image I sharp and the noisy image I nois e;I sharp and I noise It is called the spatial domain feature of the image. These two features are spliced ​​in the channel dimension to obtain the fused spatial domain feature I spatial ; will I spatial Input into the feature pyramid network FPN, and then processed by the classification head to obtain the true or false probability p based on spatial domain features spatial The three frequency domain features obtained in step S1 and the two spatial domain features obtained in step S2 are collectively referred to as content-independent features of the image; Step S3: Input the original image I into the pre-trained visual converter ViT model to extract the content-related features of the image, replace the original classification head of the visual converter ViT model with a fully connected layer, and obtain the true or false probability p based on the content-related features content ; Step S4: The three true and false probabilities p obtained in steps S1, S2, and S3 are freq 、p spatial 、p content Weighted addition to obtain the final true or false probability p final , according to p final Get the true and false classification result of the image I result .

2. According to claim 1, a deep fake image detection method integrating content-independent features and content-dependent features is characterized in that: Step S1 specifically includes the following steps: S1.1: Input the original image I into the Gaussian filter, use the Gaussian filter to perform weighted averaging on the image to reduce high-frequency noise and obtain a smoothed image I smooth ; S1.2: Input the original image I into the Gabor filter to capture the texture information in different directions in the image and obtain the texture image I Gabor ; S1.3: Input the original image I into the wavelet filter to perform image decomposition and reconstruction to obtain the detail image I detail ; S1.4: The I obtained by S1.1, S1.2, and S1.3 smooth ,I Gabor ,I detail The fused frequency domain features I are obtained by splicing in the channel dimension freq , will I freq The input feature pyramid network is then processed by the classification head including the pooling layer, the fully connected layer and the softmax activation function to obtain the true or false probability p based on the frequency domain features. freq .

3. According to claim 2, a deep fake image detection method integrating content-independent features and content-dependent features is characterized in that: Step S1.1 is as follows: Gaussian filter is used to smooth the image and reduce high-frequency noise. The two-dimensional Gaussian distribution density function is shown as follows: In the formula, G(x,y) represents the value of the Gaussian function at the position (x,y), that is, the weight of the position, σ is the standard deviation of the Gaussian distribution, and μ represents the mean of the Gaussian distribution; The Gaussian convolution kernel is normalized and the sum of the weights is 1, so that the total brightness of the image remains unchanged. The image is weighted averaged by applying the Gaussian convolution kernel.

4. According to claim 2, a deep fake image detection method integrating content-independent features and content-dependent features is characterized in that: Step S1.2 is as follows: Gabor filter detects texture features in a specific direction in the image by combining Gaussian function and sine wave function. The mathematical expression of Gabor filter is as follows: Where x' and y' are the coordinates after rotation transformation, λ represents the wavelength of the filter, θ represents the direction of the parallel stripes of the Gabor function, ψ represents the phase shift of the sine wave, σ represents the standard deviation of the Gaussian function, and γ represents the spatial aspect ratio; The maximum merging strategy of multi-directional responses is adopted, 16 Gabor filter kernels with different directions are used, and their stripe directions θ are set to be uniformly distributed between 0° and 360°; finally, the Gabor kernel is normalized as shown in the following formula: In the formula, kernel represents the original weight matrix of the filter, ∑kernel represents the sum of all elements in the matrix; the weight matrix normalized_kernel obtained by the normalization operation maintains consistent response strength in different directions.

5. According to claim 2, a deep fake image detection method integrating content-independent features and content-dependent features is characterized in that: Step S1.3 is as follows: using Haar as the basis function of wavelet filtering, using two-dimensional wavelet transform to perform wavelet decomposition and reconstruction on the image. The process of wavelet decomposition and reconstruction is shown in the following formula: DWT(I)→(LL,LH,HL,HH) <h2 style=";text-align:left;direction:ltr">I<h2 style=";text-align:left;direction:ltr"> detail <h2 style=";text-align:left;direction:ltr"> (LH|+|HL|+|HH|) Where I represents the input image, LL represents the low-frequency part of the image, that is, the overall structure and large-scale features of the image; LH, HL, and HH represent the high-frequency components of the image in the horizontal, vertical, and diagonal directions, respectively; before applying the wavelet transform, the image pixel values ​​are normalized to the range of [0, 255]; the result after the wavelet transform is denormalized.

6. A deep fake image detection method integrating content-independent features and content-dependent features according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1: The original image I is processed using the unsharp masking technique. The original image I is first subjected to Gaussian blur processing to obtain a smoothed image I smooth , then subtract the smoothed image I from the original image I smooth Get the difference image I sharp ; S2.2: Input the original image I into the DnCNN denoising model, and use multi-layer convolution and activation functions to obtain the noisy image I noise ; S2.3: The I obtained by S2.1 and S2.2 sharp with I noise The fused spatial domain features I are obtained by splicing in the channel dimension spatial , will I spatial Input the feature pyramid network, and then go through the same classification head processing as S1.4 to obtain the true or false probability p based on spatial domain features. spatial .

7. A deep fake image detection method integrating content-independent features and content-dependent features according to claim 6, characterized in that: Step S2.1 is as follows: the original image I is divided into three color channels I R ,I G ,I B , and to I R ,I G ,I B Gaussian blur processing is performed respectively to generate a smooth image I smooth_R ,I smooth_G ,I smooth_B For each color channel, the smoothed image is subtracted from the original image, and the results are recorded as I diff_R ,I diff_G ,I diff_B , that is I diff_x =I x -I smooth_x , and then the I of the three color channels diff_R ,I diff_G ,I diff_B Add together to get the complete I diff ; Set a threshold T = 50, and the final difference image I sharp Only I is retained diff The pixel values ​​in the matrix are greater than or equal to T, and the other pixel values ​​are set to 0. The above process is expressed as:

8. The deep fake image detection method integrating content-independent features and content-dependent features according to claim 1, characterized in that: In step S3, the pre-trained ViT model is used to extract the content-related features of the image. The original classification head of the ViT model is replaced by a fully connected layer with an output dimension of 2 to directly generate the prediction result of the binary classification, that is, the true and false probability p based on the content-related features of the image content .

9. The method for detecting deep fake images by integrating content-independent features and content-dependent features according to claim 1, characterized in that: Step S4 specifically includes: freq 、p spatial 、p content The weights are recorded as weight1, weight2, and weight3, and the initial values ​​of the three weights are set; the final true or false probability p final =weight1*p freq +weight2*p spatial +weight3*p content .

10. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

Images