Method and system for detecting forged images based on frequency domain information and generative adversarial network

Abstract

The application discloses a kind of based on frequency domain information and generation confrontation network's fake image detection method and system, it is related to image processing and deep learning technical field, the method includes: obtaining target image;The target image is face image or natural scene image;The target image is converted from image space to frequency domain space, to obtain target spectrum diagram;The target spectrum diagram is input into fake image detection model, to determine the discrimination result of the target image;The discrimination result includes real image and fake image;Wherein, the network structure of the fake image detection model is generation confrontation network;The generation confrontation network includes generator and discriminator;The discriminator adopts U-Net structure.The present application can not only detect fake face image, but also detect natural scene fake image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and deep learning, in particular to a forged image detection method and system based on frequency domain information and a generative adversarial network. BACKGROUND

[0002] With the development of image processing technology and deep learning technology, the quality of forged images is getting higher and higher and the cost is getting lower and lower, which poses a growing threat to social life and production activities.

[0003] In recent years, with the popularization of multimedia collection tools, especially the promotion of image editing software such as Photoshop and Meitu Xiu Xiu, the production and dissemination of digital images have entered an era of explosion. Existing forged image detection methods can be roughly divided into two categories: one is to use specific clues in the image, such as pixel-level inconsistency, to detect forged images; the other is to use deep learning models to capture forged features in images to improve detection performance. In particular, with the continuous in-depth study of deep learning, deep convolutional neural networks (Deep CNNs, DCNNs) have been gradually applied to the field of forged image detection and have shown great potential for development. However, existing forged image detection techniques are mostly limited to the detection of forged face images, and there are few methods for detecting forged natural images. In fact, the detection of natural scene forged images is also crucial, such as in the field of military reconnaissance. When planning a march route, if a river or a mountain is "created" on a topographic map that originally has no river or mountain, this approach will greatly affect the planning of the march route, and thus have serious consequences. Therefore, it is urgent to develop a method for detecting forged natural images. SUMMARY

[0004] The purpose of the present application is to provide a forged image detection method and system based on frequency domain information and a generative adversarial network, which can not only detect forged face images, but also detect natural scene forged images.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a forged image detection method based on frequency domain information and a generative adversarial network, comprising:

[0007] obtaining a target image; the target image is a face image or a natural scene image;

[0008] converting the target image from image space to frequency domain space to obtain a target spectrum image;

[0009] inputting the target spectrum diagram into a fake image detection model to determine a discrimination result of the target image; the discrimination result includes a real image and a fake image;

[0010] The network structure of the fake image detection model is a generative adversarial network; the generative adversarial network includes a generator and a discriminator; the discriminator adopts a U-Net structure.

[0011] Optionally, the conversion of the target image from an image space to a frequency domain space to obtain a target spectrum diagram specifically includes:

[0012] Optionally, the conversion of the target image from an image space to a frequency domain space to obtain a target spectrum diagram specifically includes:

[0013] Optionally, the conversion of the target image from an image space to a frequency domain space to obtain a target spectrum diagram specifically includes:

[0014] Optionally, the generator includes a first encoder and a first decoder; the discriminator includes a second encoder and a second decoder.

[0015] The output end of the first encoder is connected to the input end of the first decoder; the output end of the first decoder is connected to the input end of the second encoder; the first output end of the second encoder is used to output a sample image category in a model training process; and the second output end of the second encoder is connected to the input end of the second decoder.

[0016] Optionally, the first encoder is used to extract a latent code of the target spectrum diagram; and the first decoder is used to reconstruct the target spectrum diagram based on the latent code of the target spectrum diagram to obtain a reconstructed image.

[0017] The first encoder includes five first convolution modules, wherein the first four first convolution modules include a convolution layer, a batch processing layer and an LReLU activation layer, and the last first convolution module includes a convolution layer, a batch processing layer and an ReLU activation layer.

[0018] The first decoder includes five first deconvolution modules which are the same; each of the first deconvolution modules includes a deconvolution layer and an ReLU activation layer.

[0019] Optionally, the second output end of the second encoder is used to output a latent code of the reconstructed image.

[0020] The output end of the second decoder is used to output an image pixel-by-pixel discrimination result value; and the image pixel-by-pixel discrimination result value is the discrimination result of the target image.

[0021] The second encoder comprises at least five second convolution modules, wherein the first second convolution module comprises a two-dimensional convolution layer and a spectral normalization layer, and the last four second convolution modules comprise a two-dimensional convolution layer, a spectral normalization layer and a ReLU activation layer.

[0022] The second decoder comprises at least five second deconvolution modules which are the same; each second deconvolution module comprises a deconvolution layer and a ReLU activation layer.

[0023] Optionally, the loss function of the fake image detection model is a composite loss function.

[0024] The composite loss function comprises an adversarial loss subfunction, a first label loss subfunction, a second label loss subfunction and a reconstruction subfunction.

[0025] The adversarial loss subfunction is a cross-entropy loss function.

[0026] The first label loss subfunction is used to represent the difference value between the latent encoding of the sample spectral graph and the real label of the sample image; the real label of the sample image is the label corresponding to the sample spectral graph; the sample spectral graph is obtained by converting the sample image from the image space to the frequency domain space;

[0027] The second label loss subfunction is used to represent the difference value between the latent encoding of the sample reconstructed image and the real label of the sample image; the sample reconstructed image is an image reconstructed based on the latent encoding of the sample spectral graph.

[0028] The reconstruction loss subfunction is used to represent the pixel-level loss value between the sample reconstructed image and the sample spectral graph.

[0029] In a second aspect, the present application provides a fake image detection system based on frequency domain information and a generative adversarial network, comprising:

[0030] A data acquisition module is configured to acquire a target image; the target image is a face image or a natural scene image.

[0031] A processing module is configured to convert the target image from the image space to the frequency domain space to obtain a target spectral graph.

[0032] A category result determination module is configured to input the target spectral graph into a fake image detection model to determine a discrimination result of the target image; the discrimination result comprises a real image and a fake image.

[0033] The network structure of the fake image detection model is a generative adversarial network; the generative adversarial network comprises a generator and a discriminator; the discriminator adopts a U-Net structure.

[0034] Optionally, the processing module specifically comprises:

[0035] a transformation unit, configured to perform a discrete Fourier transform on the target image to obtain an initial frequency spectrum image;

[0036] a centering processing unit, configured to perform a centering processing on the initial frequency spectrum image to obtain a target frequency spectrum image.

[0037] Optionally, the generator comprises a first encoder and a first decoder; and the discriminator comprises a second encoder and a second decoder;

[0038] an output terminal of the first encoder is connected to an input terminal of the first decoder; an output terminal of the first decoder is connected to an input terminal of the second encoder; a first output terminal of the second encoder is configured to output a sample image category in a model training process; and a second output terminal of the second encoder is connected to an input terminal of the second decoder.

[0039] Optionally, the first encoder is configured to extract a latent code of the target frequency spectrum image; and the first decoder is configured to reconstruct the target frequency spectrum image based on the latent code of the target frequency spectrum image to obtain a reconstructed image.

[0040] the first encoder comprises five first convolution modules, wherein the first four first convolution modules each comprise a convolution layer, a batch processing layer and an LReLU activation layer, and the last first convolution module comprises a convolution layer, a batch processing layer and an ReLU activation layer.

[0041] the first decoder comprises five first deconvolution modules which are identical to each other; and each of the first deconvolution modules comprises a deconvolution layer and an ReLU activation layer.

[0042] the second output terminal of the second encoder is configured to output a latent code of the reconstructed image.

[0043] an output terminal of the second decoder is configured to output an image pixel-by-pixel discrimination result value; and the image pixel-by-pixel discrimination result value is a discrimination result of the target image.

[0044] wherein the second encoder comprises at least five second convolution modules, wherein the first second convolution module comprises a two-dimensional convolution layer and a spectral normalization layer, and the last four second convolution modules each comprise a two-dimensional convolution layer, a spectral normalization layer and an ReLU activation layer.

[0045] the second decoder comprises at least five second deconvolution modules which are identical to each other; and each of the second deconvolution modules comprises a deconvolution layer and an ReLU activation layer.

[0046] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0047] The present application uses the game idea of the generative adversarial network, introduces the generative adversarial network into the field of fake image detection, and proposes a new fake image detection method and system based on frequency domain information and the generative adversarial network. First, the target image is converted from the image space to the frequency domain space, and then the processed target image is input into the generator of the generative adversarial network; then the U-Net structure, i.e. the encoder-decoder, is introduced into the discriminator, thereby enhancing the discrimination ability of the discriminator; finally, the output result of the generator is input into the discriminator. Since the target image is converted from the image space to the frequency domain space and the U-Net structure is introduced into the discriminator, the discrimination ability is enhanced. Obviously, the fake image detection method and system based on frequency domain information and the generative adversarial network provided by the present application can not only detect fake face images, but also detect natural scene fake images (images that are not easy to identify). BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The frequency spectrum corresponding to the real image and the fake image generated by different models in the embodiments of the present application; Figure 1 (a) (b) (c) (d) (e) (f) (g) are different real images in the embodiments of the present application; Figure 1 (h) (i) (j) (k) (m) (n) (o) are the frequency spectrum corresponding to the fake image generated by different models in the embodiments of the present application;

[0050] Figure 2 The generator structure diagram in the embodiments of the present application;

[0051] Figure 3 The discriminator structure diagram in the embodiments of the present application;

[0052] Figure 4 The spectral normalization algorithm flowchart in the embodiments of the present application;

[0053] Figure 5 The flowchart of the fake image detection method based on frequency domain information and the generative adversarial network in the embodiments of the present application;

[0054] Figure 6A structure schematic diagram of a forged image detection system based on frequency domain information and a generative adversarial network according to an embodiment of the present application;

[0055] Figure 7 A general architecture diagram of a forged image detection method based on frequency domain information and a generative adversarial network according to an embodiment of the present application;

[0056] Figure 8 A data set display diagram according to an embodiment of the present application;

[0057] Figure 9 A detection accuracy rate diagram of whether a discriminator adopts a U-Net structure according to an embodiment of the present application;

[0058] Figure 10 A detection precision rate diagram of whether a discriminator adopts a U-Net structure according to an embodiment of the present application;

[0059] Figure 11 A detection precision rate diagram of whether a discriminator adopts a U-Net structure according to an embodiment of the present application;

[0060] Figure 12 A detection F1 value diagram of whether a discriminator adopts a U-Net structure according to an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0062] Recently, generative adversarial networks (GANs) have shown great application potential in the field of computer vision and image processing, including image translation, image inpainting, image synthesis and other applications, due to their powerful image generation capabilities. Also because of the development of generative adversarial network technology, a large number of fake images that are similar to real images and difficult for human eyes to distinguish have begun to appear in the public's field of vision. A large number of fake images in various forms may have a huge negative impact on personal privacy and even social stability, so there is an urgent need for a forged image detection technology to detect false images generated by convolutional neural networks (CNNs), especially GANs. The research of this technology is not only beneficial to protect personal privacy and maintain social harmony, but also has long-term application prospects and great research value.

[0063] With the development of deep learning methods, deepfake technology is becoming more and more mature. A large number of natural images close to reality have poured into people's lives, while meeting personal entertainment interests, the abuse of deepfake technology poses a potential threat to personal privacy, economic markets and even national security. Therefore, the detection method for forged images needs to be studied. The existing forged image detection technology mostly has the problems of low accuracy and poor generalization. In view of this, the present application provides a forged image detection method and system based on frequency domain information and a generative adversarial network. The present application starts from the image forgery mechanism of deepfake technology, converts the image from the image domain to the frequency domain, and introduces a discriminator based on the U-Net structure, and proposes a forged image detection model. The technical scheme provided by the present application is verified by experiments on 7 separate data sets and mixed data sets. Compared with some advanced methods, the present application can reach 100% accuracy on separate data sets, and the minimum accuracy can reach 94.93%; the model detection recall rate, precision and F1 score can reach 98.17%, 98.25% and 98.19% on average, and the accuracy on the mixed data set can reach 92.96%. Experiments show that the present application provides an effective and good generalization performance of the forged image detection method and system.

[0064] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0065] Embodiment one

[0066] In view of the above problems, the present application utilizes the game idea of GANs, introduces GANs into the field of forged image detection, and proposes a new detection method for forged natural scene images or forged face images. In the training stage, the forged image is first converted from the image space to the frequency domain space, and then the processed forged image is input into the generator of GANs to extract the latent code and image reconstruction; then the U-Net structure, i.e. the encoder-decoder, is introduced into the discriminator, wherein the encoder is used for image classification, and the decoder is used for image pixel discrimination, so as to enhance the discrimination ability of the discriminator. In the test stage, the trained discriminator is used for forged image detection.

[0067] The present application also designs a composite loss function to better optimize the model. In order to comprehensively evaluate the effectiveness of the method of the present application, the performance of the model is analyzed by using various evaluation indexes, which shows that the method of the present application can achieve good forged image detection performance.

[0068] The forged image detection method provided by the present application combines frequency domain information and an adversarial network, which includes the following steps:

[0069] S1: A new method for detecting forged natural scene images is proposed by introducing GANs into the field of forged image detection based on the game idea of GANs.

[0070] S2: First, the forged image is converted from the image space to the frequency domain.

[0071] S2-1: The frequency domain conversion module, Figure 1 The frequency spectrum corresponding to the forged image generated by different generation models is shown. The first and second rows represent real images and forged images generated by different generation models, respectively, and each column is a real image and the corresponding forged image of the real image. First, the real image and the forged image obtained based on different generation models are subjected to discrete Fourier transform (DFT) to obtain the frequency spectrum, and then the obtained frequency spectrum is centered. The centered frequency spectrum is then input into the forged image detection model.

[0072] The most important reason for converting the forged image from the image space to the frequency domain is that the forged image generated by the deep learning model is often difficult to find traces of forgery from the image domain, especially for models such as StyleGAN, StyleGAN2, etc. proposed in recent years, which can often generate high-quality, high-resolution images. By converting the forged image from the image domain to the frequency domain, it can be found that there is a big difference between the real image and the forged image. Observation Figure 1 It can be found that the high-frequency information of the real image is significantly higher than that of the forged image. This is because the forged image obtained by the generation model can be well generated for the area with slow gray scale change, while it is difficult to generate for the area with rapid gray scale change, i.e. the edge area.

[0073] S3: The obtained centered frequency spectrum is input into the encoder of the generator to obtain the latent encoding of the image; then the latent encoding is input into the decoder of the generator for image reconstruction. Figure 2 The structure of the generator in the proposed forged image detection model is shown in FIG. 4. The generator consists of an encoder and a decoder.

[0074] S3-1: The encoder part is used to extract the latent code of the input image, and the encoder in the generator is composed of 5 convolution modules, wherein the first four convolution modules are composed of a convolution layer (convolution, Conv) and a batch normalization layer (Batchnormalization, BN) followed by a Leaky-Rectified Linear Units (LReLU) activation layer, and the last convolution module replaces the LReLU activation layer with a Rectified Linear Units (ReLU) activation layer.

[0075] S3-2: The decoder is used for image reconstruction. The decoder structure corresponds to the encoder structure and is composed of 5 deconvolution modules, each of which has the same structure, including a deconvolution layer and a ReLU activation layer.

[0076] S4: The reconstructed image and the input image (i.e. the centralized spectrum graph) are input into the discriminator to identify the authenticity. The discriminator proposed in the embodiment of the present application adopts a U-Net structure and is composed of an encoder and a decoder. Figure 3 The structure of the discriminator in the proposed fake image detection model is shown in FIG. 2. Among them, Conv2D represents two-dimensional convolution, SpectralNorm represents batch normalization, k represents the size of the convolution kernel, s represents the step, d represents the number of channels of the output feature map, LReLU represents the leakyReLU activation function, ReLU represents the ReLU activation function, and deConv2D represents two-dimensional deconvolution.

[0077] S4-1: The encoder classifies the input image, that is, classifies the input image as a real image or a fake image.

[0078] S4-2: The decoder discriminates the authenticity of the image pixels, so the discriminator has two outputs corresponding to the image detection result and the image pixel-by-pixel discrimination result.

[0079] S4-3: The present application applies spectral normalization (SpectralNormalization, SN) to the discriminator to make the discriminator D satisfy the Lipschitz continuity in a more elegant way, which limits the degree of function change, so that the model is more stable. Before implementing SN, the singular value of the convolution kernel (weight matrix) W in the convolution network needs to be solved first, so as to obtain the spectral norm of each layer parameter matrix. In this process, the "power iteration method" is used to approximate the solution, and the iteration process is shown in steps 2.2 and 2.3 of FIG. 3. After obtaining the spectral norm, the parameters on each parameter matrix are divided by it to achieve the purpose of normalization. The specific process of the algorithm is shown in FIG. 4. Figure 4 Figure 4

[0080] ​​Spectrum normalization layer is added in the discriminator, so that the discriminator satisfies Lipschitz continuity. The spectrum normalization used in the embodiment of the present application is consistent with the existing spectrum normalization technology.

[0081] S5: In addition to using the adversarial loss to optimize the generator and the discriminator, the label loss is also used to constrain the consistency between the generated latent code and the image label. In order to ensure the similarity between the reconstructed image and the input image, the embodiment of the present application avoids the pixel-level loss in the process of reconstructing the image by using the reconstruction loss. The generator is trained by the adversarial loss, the label loss and the reconstruction loss; the discriminator is trained by the adversarial loss and the label loss.

[0082] S5-1: Adversarial loss; since the present application proposes a detection model based on GANs, the cross-entropy loss in the original GANs is continued to be used in the present application, which can maximize the discrimination of the discriminator and minimize the difference between the output of the generator and the real data. The calculation formula of the adversarial loss L GAN is as follows.

[0083]

[0084] Where x represents the input spectrum, D and G represent the discriminator and the generator respectively.

[0085] S5-2: Label loss; in order to constrain the difference between the latent code obtained by the encoder in the generator and the real label of the image, so that the reconstructed image can be more consistent with the input image. In addition, the cross-entropy loss between the output of the encoder in the discriminator and the label is also calculated, so as to assist the discriminator to better identify the authenticity of the input image. The calculation formula of the label loss is as follows.

[0086] L label_G = |l-G E (x) |;

[0087] L label_D = -[l log(D E (x))+(1-l)log(1-D E (x)) ];

[0088] Where l represents the label of the input image, G E and D E represent the encoder part of the generator and the encoder part of the discriminator respectively.

[0089] S5-3: In order to make the reconstructed image of the generator highly restore the input image, the present application also proposes a reconstruction loss, which calculates the pixel-level loss between the reconstructed image G(x) and the input image x, and uses Lreconstruction The loss can be calculated by the following formula:

[0090]

[0091] Where m, n represent the size of the image, here m = n = 256. The total loss function is the weighting of the above loss functions, denoted as:

[0092] L total = L GAN + λ label (L label_G + L label_D ) + λ rec L reconstruction ;

[0093] Where λ label , λ rec respectively represent the weights of the label loss and the reconstruction loss.

[0094] Example two

[0095] As Figure 5 shown, a method for detecting fake images based on frequency domain information and a generative adversarial network, comprising:

[0096] Step 100: obtaining a target image; the target image is a face image or a natural scene image.

[0097] Step 200: converting the target image from image space to frequency domain space to obtain a target spectrum image.

[0098] Step 300: inputting the target spectrum image into a fake image detection model to determine the discrimination result of the target image; the discrimination result includes real images and fake images.

[0099] Wherein, the network structure of the fake image detection model is a generative adversarial network; the generative adversarial network includes a generator and a discriminator; the discriminator adopts a U-Net structure.

[0100] Step 200 specifically includes:

[0101] Discrete Fourier transform is performed on the target image to obtain an initial spectrum image; the initial spectrum image is centrally processed to obtain a target spectrum image.

[0102] The generator comprises a first encoder and a first decoder; the discriminator comprises a second encoder and a second decoder; an output end of the first encoder is connected to an input end of the first decoder; an output end of the first decoder is connected to an input end of the second encoder; a first output end of the second encoder is used for outputting a sample image category in a model training process; and a second output end of the second encoder is connected to an input end of the second decoder.

[0103] Further, the first encoder is used for extracting latent encoding of the target spectrum graph; and the first decoder is used for reconstructing the target spectrum graph based on the latent encoding of the target spectrum graph to obtain a reconstructed image. The first encoder comprises five first convolution modules, wherein the first four first convolution modules comprise a convolution layer, a batch layer and an LReLU activation layer, and the last first convolution module comprises a convolution layer, a batch layer and an ReLU activation layer; the first decoder comprises five and same first deconvolution modules; and the first deconvolution module comprises a deconvolution layer and an ReLU activation layer.

[0104] The second output end of the second encoder is used for outputting latent encoding of the reconstructed image.

[0105] An output end of the second decoder is used for outputting an image pixel-by-pixel discrimination result value; the image pixel-by-pixel discrimination result value is a discrimination result of the target image; wherein the second encoder comprises at least five second convolution modules, wherein the first second convolution module comprises a two-dimensional convolution layer and a spectral normalization layer, and the last four second convolution modules comprise a two-dimensional convolution layer, a spectral normalization layer and an ReLU activation layer; the second decoder comprises at least five and same second deconvolution modules; and the second deconvolution module comprises a deconvolution layer and an ReLU activation layer.

[0106] In one example, the loss function of the fake image detection model is a composite loss function.

[0107] The composite loss function comprises an adversarial loss sub-function, a first label loss sub-function, a second label loss sub-function and a reconstruction sub-function.

[0108] The adversarial loss function is the cross-entropy loss function; the first label loss function is used to represent the difference between the latent encoding of the sample spectrogram and the true label of the sample image; the true label of the sample image is the label corresponding to the sample spectrogram; the sample spectrogram is obtained by converting the sample image from image space to frequency domain space; the second label loss function is used to represent the difference between the latent encoding of the reconstructed sample image and the true label of the sample image; the reconstructed sample image is an image reconstructed based on the latent encoding of the sample spectrogram; the reconstruction loss function is used to represent the pixel-level loss value between the reconstructed sample image and the sample spectrogram.

[0109] Example 3

[0110] like Figure 6 As shown, this embodiment provides a forged image detection system based on frequency domain information and generative adversarial networks, including:

[0111] The data acquisition module 400 is used to acquire a target image; the target image is a face image or a natural scene image.

[0112] The processing module 500 is used to convert the target image from the image space to the frequency domain space to obtain the target spectrum map.

[0113] The category result determination module 600 is used to input the target spectrogram into the fake image detection model to determine the discrimination result of the target image; the discrimination result includes real images and fake images.

[0114] The network structure of the forged image detection model is a generative adversarial network (GAN); the GAN includes a generator and a discriminator; the discriminator adopts a U-Net structure.

[0115] The processing module 500 specifically includes:

[0116] The transformation unit is used to perform a discrete Fourier transform on the target image to obtain an initial spectrum.

[0117] Centralization processing is used to centralize the initial spectrogram to obtain the target spectrogram.

[0118] The generator includes a first encoder and a first decoder; the discriminator includes a second encoder and a second decoder; the output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the second encoder; the first output of the second encoder is used to output the sample image category during model training; the second output of the second encoder is connected to the input of the second decoder.

[0119] The first encoder is configured to extract latent encoding of the target spectral map; the first decoder is configured to reconstruct the target spectral map based on the latent encoding of the target spectral map to obtain a reconstructed image; the first encoder comprises five first convolution modules, wherein the first four first convolution modules comprise a convolution layer, a batch layer and an LReLU activation layer, and the last first convolution module comprises a convolution layer, a batch layer and an ReLU activation layer; the first decoder comprises five and same first deconvolution modules; each first deconvolution module comprises a deconvolution layer and an ReLU activation layer.

[0120] The second output end of the second encoder is configured to output latent encoding of the reconstructed image; the output end of the second decoder is configured to output an image pixel-by-pixel discrimination result value; the image pixel-by-pixel discrimination result value is a discrimination result of the target image; wherein the second encoder comprises at least five second convolution modules, wherein the first second convolution module comprises a two-dimensional convolution layer and a spectral normalization layer, and the last four second convolution modules comprise a two-dimensional convolution layer, a spectral normalization layer and an ReLU activation layer; the second decoder comprises at least five and same second deconvolution modules; each second deconvolution module comprises a deconvolution layer and an ReLU activation layer.

[0121] Embodiment four

[0122] Referring to the architecture of Figure 7 , the embodiment provides a method for detecting fake images based on frequency domain information and a production adversarial network, which comprises the following settings:

[0123] S1: Use the fake image dataset provided by Wang et al. to train and verify the model, as shown in Figure 8 . The dataset contains 11 sets of fake images obtained by different CNN-based generation models. In this embodiment, only 8 sets of fake image datasets are used for model verification, including StyleGAN, BigGAN, CycleGAN, Star-GAN, CRN, IMLE, StyleGAN2, and GauGAN.

[0124] S2: The first 7 datasets in S1 are not only used to verify the detection performance of the detection method on individual datasets, but also applied to the performance verification on mixed datasets. The ratio of real images and fake images in each dataset is 1:1. In the selected dataset, 80% of the data is used for training, and the remaining 20% is used to verify the effectiveness of the proposed model.

[0125] S3: The key parameters in the verification are set as follows: the image size for training is 256*256, the learning rate is 0.0002, the training times are 100 epochs, and the batch size is 1.

[0126] Example Five

[0127] In order to fully verify the effectiveness of the proposed method, the proposed model is tested on different fake image datasets. According to the settings of Example One, ablation experiments are performed on whether the frequency domain conversion module is present or not.

[0128] S1: Table 1 to Table 1 shows the experimental results of the proposed method with or without frequency domain conversion and SN operation.

[0129] Table 1 Comparison of model accuracy (ACC) with or without frequency domain conversion and SN operation

[0130]

[0131] Table 2 Comparison of model precision (P) with or without frequency domain conversion and SN operation

[0132]

[0133] Table 3 Comparison of model recall (R) with or without frequency domain conversion and SN operation

[0134]

[0135] Table 4 Comparison of model F1 value with or without frequency domain conversion and SN operation

[0136]

[0137] S2: After adding the frequency domain conversion and SN operation, the model detection accuracy ACC, precision P, recall R and F1 value are generally improved. Especially for StyleGAN, BigGAN and StarGAN, the accuracy ACC and precision P are extremely low without frequency domain conversion and SN operation, indicating that the detection model has almost no effect on the fake images generated by these models.

[0138] S3: After adding the frequency domain conversion and SN operation, the detection accuracy ACC and precision P are greatly improved, and the recall R also remains at a relatively optimal level.

[0139] S4: This embodiment shows that in the frequency domain, the defects present in the fake images obtained by the generation model are more easily detected.

[0140] Example Six

[0141] In order to fully verify the effectiveness of the method proposed in the present application, the present embodiment carries out an ablation experiment on whether the discriminator structure in the network structure adopts the U-Net structure according to the setting of embodiment one.

[0142] S1: The experimental results are intuitively displayed in the form of a chart in Figures 9-12 .

[0143] S2: The adoption of the U-Net structure improves the performance of model detection to some extent, wherein the model detection accuracy ACC, precision P, recall R and F1 value are respectively improved by an average of 3.698%, 2.039%, 6.064% and 4.241%.

[0144] S3: The present embodiment shows that the encoder-decoder structure adopted in the proposed model is more suitable for fake image detection than the ordinary discriminator structure.

[0145] Embodiment Seven

[0146] In order to fully verify the effectiveness of the method proposed in the present application, the present embodiment carries out model training on a mixed data set, and then carries out experiments on separate training on different data sets.

[0147] S1: There are a total of 4000 images in the mixed data set, wherein the proportion of real images and fake images is 1:1. The mixed data set contains a total of 8000 fake images generated by five different generation models: IMLE, StyleGAN, StyleGAN2, CRN and GauGAN, wherein real images and fake images each account for 50%

[0148] S2: In order to fully demonstrate the generalization performance of the detection model proposed in the present paper, a comprehensive analysis of the experimental results is carried out, and the results are shown in Table 5 below.

[0149] Table 5 result display table

[0150]

[0151]

[0152] S3: The present embodiment shows that the method of the present application can also achieve better detection performance on the mixed data set, and the average detection accuracy can reach 92.96%, thereby showing the detection generalization of the method proposed in the present application.

[0153] Embodiment Eight

[0154] In order to fully verify the effectiveness of the method proposed in the present application, the present embodiment compares the method of the present application with some advanced existing methods.

[0155] The following table shows the experimental comparison and analysis of the method of the present application and some advanced methods currently available:

[0156] Table 6 Comparison and analysis table

[0157]

[0158] The bold font part represents the optimal value of each index. The comparison methods include Inception, Resnet50, Xception, Mesonet, Mesonet-Inception and EfficientNet six comparison algorithms. The data in the table show that the method of the present application can exhibit excellent detection performance on most data sets, and compared with the fake image detection model used for comparison, although the method of the present application is slightly lower than the Inception and EfficientNet models on the CRN and StyleGAN data sets, it also achieves a better detection accuracy. In general, the method of the present application has good detection performance.

[0159] The present application has the following beneficial effects:

[0160] Firstly, the present application uses the game idea of GANs, introduces GANs into the field of fake image detection, and proposes a new method for detecting fake natural scene images.

[0161] Secondly, the U-Net structure, i.e. the encoder-decoder, is introduced into the discriminator, wherein the encoder is used for image classification, and the decoder is used for discriminating image pixels, so as to enhance the discrimination ability of the discriminator.

[0162] Thirdly, the present application designs a composite loss function to better optimize the model.

[0163] Fourthly, the performance of the model is analyzed by using various evaluation indexes, which shows that the method of the present application can achieve good fake image detection performance.

[0164] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0165] In this paper, specific examples are used to illustrate the principles and implementation modes of the present application. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for detecting forged images based on frequency domain information and generative adversarial networks, characterized in that, include: Acquire the target image; The target image is a face image or a natural scene image; The target image is converted from the image space to the frequency domain space to obtain the target spectrum map; The target spectrogram is input into the forgery image detection model to determine the discrimination result of the target image; The discrimination results include real images and fake images; The network structure of the forged image detection model is a generative adversarial network (GAN); the GAN includes a generator and a discriminator; the discriminator adopts a U-Net structure. The generator includes a first encoder and a first decoder; the discriminator includes a second encoder and a second decoder. The output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the second encoder; the first output of the second encoder is used to output the sample image category during model training; the second output of the second encoder is connected to the input of the second decoder. The first encoder is used to extract the latent code of the target spectrogram; the first decoder is used to reconstruct the target spectrogram based on the latent code of the target spectrogram to obtain a reconstructed image. The first encoder includes five first convolutional modules, wherein the first four first convolutional modules include a convolutional layer, a batch processing layer and an LReLU activation layer, and the last first convolutional module includes a convolutional layer, a batch processing layer and a ReLU activation layer. The first decoder includes five identical first deconvolution modules; each first deconvolution module includes a deconvolution layer and a ReLU activation layer. The second output terminal of the second encoder is used to output the latent code of the reconstructed image; The output of the second decoder is used to output the pixel-by-pixel discrimination result value of the image; the pixel-by-pixel discrimination result value of the image is the discrimination result of the target image; The second encoder includes at least five second convolutional modules, wherein the first second convolutional module includes a two-dimensional convolutional layer and a spectral normalization layer, and the latter four second convolutional modules include a two-dimensional convolutional layer, a spectral normalization layer and a ReLU activation layer. The second decoder includes at least five identical second deconvolution modules; each second deconvolution module includes a deconvolution layer and a ReLU activation layer. The loss function of the forged image detection model is a composite loss function; The composite loss function includes an adversarial loss subfunction, a first label loss subfunction, a second label loss subfunction, and a reconstruction subfunction; The adversarial loss function is the cross-entropy loss function; The first label loss function is used to represent the difference between the latent encoding of the sample spectrogram and the true label of the sample image; the true label of the sample image is the label corresponding to the sample spectrogram; the sample spectrogram is obtained by converting the sample image from the image space to the frequency domain space. The second label loss function is used to represent the difference between the latent encoding of the reconstructed sample image and the true label of the sample image; the reconstructed sample image is an image reconstructed based on the latent encoding of the sample spectrogram; The reconstruction loss function is used to represent the pixel-level loss value between the reconstructed image of the sample and the spectrogram of the sample.

2. The method for detecting forged images based on frequency domain information and generative adversarial networks according to claim 1, characterized in that, The step of converting the target image from image space to frequency domain space to obtain the target spectrum map specifically includes: Perform a discrete Fourier transform on the target image to obtain an initial spectrum. The initial spectrogram is centered to obtain the target spectrogram.

3. A forged image detection system based on frequency domain information and generative adversarial networks, characterized in that, include: The data acquisition module is used to acquire the target image; The target image is a face image or a natural scene image; The processing module is used to convert the target image from the image space to the frequency domain space to obtain the target spectrum map; The category result determination module is used to input the target spectrogram into the forgery image detection model to determine the discrimination result of the target image; The discrimination results include real images and fake images; The network structure of the forged image detection model is a generative adversarial network (GAN); the GAN includes a generator and a discriminator; the discriminator adopts a U-Net structure. The generator includes a first encoder and a first decoder; the discriminator includes a second encoder and a second decoder. The output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the second encoder; the first output of the second encoder is used to output the sample image category during model training; the second output of the second encoder is connected to the input of the second decoder. The first encoder is used to extract the latent code of the target spectrogram; the first decoder is used to reconstruct the target spectrogram based on the latent code of the target spectrogram to obtain a reconstructed image. The first encoder includes five first convolutional modules, wherein the first four first convolutional modules include a convolutional layer, a batch processing layer and an LReLU activation layer, and the last first convolutional module includes a convolutional layer, a batch processing layer and a ReLU activation layer. The first decoder includes five identical first deconvolution modules; each first deconvolution module includes a deconvolution layer and a ReLU activation layer. The second output terminal of the second encoder is used to output the latent code of the reconstructed image; The output of the second decoder is used to output the pixel-by-pixel discrimination result value of the image; the pixel-by-pixel discrimination result value of the image is the discrimination result of the target image; The second encoder includes at least five second convolutional modules, wherein the first second convolutional module includes a two-dimensional convolutional layer and a spectral normalization layer, and the latter four second convolutional modules include a two-dimensional convolutional layer, a spectral normalization layer and a ReLU activation layer. The second decoder includes at least five identical second deconvolution modules; each second deconvolution module includes a deconvolution layer and a ReLU activation layer. The loss function of the forged image detection model is a composite loss function; The composite loss function includes an adversarial loss subfunction, a first label loss subfunction, a second label loss subfunction, and a reconstruction subfunction; The adversarial loss function is the cross-entropy loss function; The first label loss function is used to represent the difference between the latent encoding of the sample spectrogram and the true label of the sample image; the true label of the sample image is the label corresponding to the sample spectrogram; the sample spectrogram is obtained by converting the sample image from the image space to the frequency domain space. The second label loss function is used to represent the difference between the latent encoding of the reconstructed sample image and the true label of the sample image; the reconstructed sample image is an image reconstructed based on the latent encoding of the sample spectrogram; The reconstruction loss function is used to represent the pixel-level loss value between the reconstructed image of the sample and the spectrogram of the sample.

4. The forged image detection system based on frequency domain information and generative adversarial networks according to claim 3, characterized in that, The processing module specifically includes: The transformation unit is used to perform a discrete Fourier transform on the target image to obtain an initial spectrum. Centralization processing is used to centralize the initial spectrogram to obtain the target spectrogram.

Images