Method and system for detecting robustness of diffusion model generated image based on frequency domain guided reconstruction error

Abstract

The invention discloses a robustness detection method for a diffusion model generated image based on a frequency domain guide reconstruction error, belongs to the field of artificial intelligence security and computer vision, and utilizes the inherent defect that the diffusion model is difficult to accurately reconstruct specific intermediate frequency information in a real image. The frequency spectrum of the input image is analyzed through an adaptive frequency mask refining module, and an intermediate frequency part which is difficult to reconstruct is filtered out to construct a pseudo-generated image; respectively reconstructing an original image and a pseudo-generated image by using a lightweight reconstruction network, and calculating respective reconstruction error graphs; the features of the two reconstruction error graphs are spliced and then input into the classifier, the authenticity of the image is judged by comparing the change of the reconstruction error before and after information filtering, the artifact feature with the most distinguishing property can be accurately extracted, and the efficiency, accuracy and robustness of detection and the generalization ability of an unknown generation model are improved.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence security and computer vision, and particularly relates to a robust detection method for images generated by a diffusion model based on frequency domain guided reconstruction error. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, image generation technologies, represented by diffusion models (DMs), have made significant progress. Diffusion models can generate high-quality, high-resolution images that are highly similar to real images, making them difficult for the human eye to distinguish from the real thing. While these generated images have shown great potential in fields such as artistic creation, digital entertainment, and virtual reality, they have also brought serious social challenges. Deepfake technology can be used to create fake news, malicious defamation, and online fraud, posing a serious threat to personal privacy, social trust, and even national security. Therefore, developing efficient and reliable detection methods to identify images generated by diffusion models has become an urgent technical problem to be solved.

[0003] Currently, detection methods for AI-generated images can be mainly divided into several categories. One category is detection methods based on spatial domain features, which typically use convolutional neural networks (CNNs) to directly identify forgery traces from the pixel information of the image. However, as the capabilities of generative models improve, these subtle forgery traces are becoming increasingly difficult to detect. Another category is methods based on frequency domain analysis, which discover patterns in the frequency domain that may exist in the generated image that differ from those in the real image.

[0004] Building upon this, a new class of detection methods is based on the reconstruction error of diffusion models. The core idea is that images generated by diffusion models are conceptually closer to the model's latent space. Therefore, after denoising using a pre-trained diffusion model, the reconstruction error (the difference between the original and reconstructed images) is significantly lower than that of the real image. Diffusion Reconstruction Error (DIRE) methods have achieved good detection results in specific scenarios, and the models are typically lightweight, sometimes even requiring no training.

[0005] However, existing detection methods based on reconstruction error still have significant technical bottlenecks: 1. Insufficient generalization ability: Existing methods show a significant drop in detection performance when faced with entirely new diffusion models. This is because different diffusion models have different structures and training data, resulting in variations in the fake features they generate, making it difficult for the detection model to generalize.

[0006] 2. Insufficient Information Utilization: Traditional methods typically input the entire reconstruction error map indiscriminately into the backend classifier. However, the reconstruction error map contains a large amount of content-related bias information and noise introduced by the diffusion process. This redundant information interferes with the classifier's judgment, while the truly discriminative key artifact information is not effectively extracted and utilized. Some studies have observed that reconstruction errors are mainly reflected in the high-frequency components of the image, but there is still a lack of precise extraction of this difficult-to-reconstruct information.

[0007] Therefore, how to extract content-related interference information from reconstruction errors and accurately focus on stable and generalized discriminative features that can essentially distinguish between real and generated images has become the key to breaking through the current technical bottleneck. Summary of the Invention

[0008] The purpose of this invention is to address the aforementioned problems by proposing a method for generating images based on frequency-guided reconstruction error detection (AIGC). This method introduces an adaptive frequency mask refinement module (AdaptiveFMRE) and a lightweight reconstruction network (LightweightVAE), which significantly reduces computational costs while improving detection accuracy, robustness, and generalization ability, making it suitable for real-time and edge computing applications.

[0009] The technical solution of this invention is: In a first aspect, the present invention provides a robustness detection method for images generated by a diffusion model based on frequency domain guided reconstruction error, comprising the following steps: Step 1: Input the image to be detected, analyze the spectrum of the input image using the adaptive frequency mask refinement module, generate a complementary mask, and filter out preset frequency band information by applying the complementary mask in the frequency domain, thereby constructing a pseudo-generated image; Step 2: Using a lightweight reconstruction network, reconstruct the input image and the pseudo-generated image respectively to obtain the original reconstructed image and the pseudo-reconstructed image; Step 3: Calculate the difference between the input image and its original reconstructed image to obtain the original reconstruction error map, and calculate the difference between the pseudo-generated image and its pseudo-reconstructed image to obtain the pseudo reconstruction error map; Step 4: Perform feature fusion between the original reconstruction error map and the pseudo reconstruction error map, and input the fused features into a classifier for classification and discrimination, and output the detection result of whether the input image is a real image or a generated image.

[0010] Furthermore, the specific steps of step 1 are as follows: Step 1.1: Perform a Fast Fourier Transform on the input image to obtain the spectrum, and use an adaptive frequency masking refinement module to analyze the spectral characteristics of the input image; Step 1.2: Generate an intermediate frequency mask that adapts to the spectral features of the input image; Step 1.3: Generate a complementary mask based on the intermediate frequency mask, wherein the complementary mask and the intermediate frequency mask are complementary in the frequency domain; Step 1.4: Apply the complementary mask in the frequency domain to filter out the preset frequency band information and construct a pseudo-generated image.

[0011] Furthermore, the adaptive frequency mask refining module adopts an encoder-decoder architecture and performs the following steps: The encoder extracts the depth features of the spectrum of the input image, the first decoder generates an intermediate frequency mask based on the depth features, and the second decoder generates a complementary mask based on the depth features. The adaptive frequency mask refining module introduces an adaptive attention mechanism to dynamically adjust the mask generation strategy based on spectral characteristics.

[0012] Further, step 1.4 includes: multiplying the spectrum with the complementary mask to obtain the filtered spectrum; performing an inverse fast Fourier transform on the filtered spectrum to obtain the pseudo-generated image; wherein the pseudo-generated image retains the low-frequency and high-frequency information of the input image and filters out the mid-frequency information.

[0013] Furthermore, in step 2, the lightweight reconstruction network employs a lightweight variational autoencoder that does not involve a diffusion model denoising step during reconstruction, and performs the following steps: Step 2.1: Reconstruct the input image using a lightweight reconstruction network to obtain the original reconstructed image; Step 2.2: Reconstruct the pseudo-generated image using the same lightweight reconstruction network to obtain the pseudo-reconstructed image.

[0014] Furthermore, the specific steps of step 3 are as follows: Step 3.1: Calculate the difference between the input image and its original reconstructed image to obtain the original reconstruction error map; Step 3.2: Calculate the difference between the pseudo-generated image and the pseudo-reconstructed image to obtain the pseudo-reconstruction error map.

[0015] Furthermore, the specific steps of step 4 are as follows: Step 4.1: Stitch the original reconstruction error map and the pseudo reconstruction error map along the channel dimension to form a multi-channel fusion feature; Step 4.2: Input the fused features into the classifier, which performs convolution and pooling on the fused features to obtain classification features; Step 4.3: Output the detection result based on the classification features to determine whether the input image is a real image or a generated image.

[0016] Furthermore, this method performs end-to-end training using a combined loss function, which includes: Intermediate frequency reconstruction alignment loss This is used to align the intermediate frequency image region extracted by the intermediate frequency mask with the original reconstruction error map; Cross-entropy loss , used to train the classifier.

[0017] Furthermore, the combined loss function also includes an adaptive masking refinement loss. This is used to guide the mask generated by the frequency mask refining module to approximate a predefined ideal mask.

[0018] Secondly, the present invention provides an AIGC-generated image detection system based on frequency reconstruction error, comprising: The frequency mask module is used to perform spectral analysis on the input image to be detected and generate a complementary mask, and filter out preset frequency band information based on the complementary mask to generate a pseudo image; The reconstruction network module is used to reconstruct the image to be detected and the pseudo image respectively, and generate the corresponding original reconstructed image and pseudo reconstructed image; An error calculation module is used to calculate the difference between the image to be detected and the original reconstructed image to generate an original reconstruction error map, and to calculate the difference between the pseudo image and the pseudo reconstructed image to generate a pseudo reconstruction error map; The fusion classification module is used to fuse the original reconstruction error map and the pseudo reconstruction error map, and classify the fused features to output the authenticity detection result of the image to be detected.

[0019] Compared with the prior art, the technical solution provided by the present invention can achieve the following beneficial effects: (1) The present invention introduces an adaptive frequency mask refining mechanism, which can analyze the spectral characteristics of each input image, dynamically locate and extract the intermediate frequency information that is difficult to reconstruct from the diffusion model, and then realize the active extraction and enhancement of discriminative information by constructing a "pseudo-generated image" and calculating the comparison error.

[0020] (2) This invention amplifies the inherent information of the real image that is difficult for the generative model to accurately reconstruct, namely the error signal corresponding to the "hard reconstruction" information; making the features received by the subsequent classifier more discriminative and essential, thus exhibiting stronger generalization ability and more stable high detection accuracy when facing different diffusion models and different image content.

[0021] (3) The present invention uses a lightweight variational autoencoder as the reconstruction network, which significantly reduces the computational overhead while maintaining the reconstruction quality. The VAE model size is reduced and the inference speed is increased by approximately 100%. It is suitable for real-time and edge deployment.

[0022] (4) This invention integrates frequency mask generation, dual-path reconstruction, error comparison calculation and final classification into an end-to-end training framework. Each module can work together, which enhances the alignment of feature representation with task and improves the generalization ability and robustness of the method.

[0023] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0024] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the invention.

[0025] Figure 1 This is the overall architecture of the AIGC-based image generation method based on frequency-guided reconstruction error detection proposed in this invention.

[0026] Figure 2 This is a schematic diagram of the core processing of the AIGC-based image generation method based on frequency-guided reconstruction error detection proposed in this invention during the model training stage. Detailed Implementation

[0027] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0028] Example 1: like Figure 1 , 2 As shown, this invention proposes a method for generating images based on adaptive frequency-guided reconstruction error detection (AIGC), comprising the following steps: Step 1: Input the image to be detected The frequency spectrum is analyzed using the Adaptive FMRE module to generate an intermediate frequency mask. With complementary mask By applying complementary masks in the frequency domain to filter out preset intermediate frequency information, a pseudo-generated image is constructed. ; In one example, the input image The spectrum was obtained by Fast Fourier Transform (FFT). The input is fed into the Adaptive Frequency Mask Refinement (AdaptiveFMRE) module. This module extracts deep spectral features through the encoder, dynamically adjusts the feature response through the Adaptive Attention Unit (AAU), and the two decoders generate intermediate frequency masks respectively. and complementary mask By using complementary masks to filter out intermediate frequency information, a pseudo-generated image is obtained:

[0029] Step 2: Use the LightweightVAE network to reconstruct the input images respectively. and the pseudo-generated image Reconstruction is performed to obtain the original reconstructed image. and pseudo-reconstructed images ; In one example, a pre-trained lightweight variational autoencoder (LightweightVAE) is used to reconstruct the original image and the pseudo-generated image through depthwise separable convolutions, channel attention mechanisms, and bottleneck structures.

[0030] in, and These are the encoder and decoder for a lightweight VAE, respectively.

[0031] Step 3: Calculate the input images respectively. Compared with its original reconstructed image The difference between them yields the original reconstruction error image. and the pseudo-generated image Rather than pseudo-reconstructed images The difference between them yields a pseudo-reconstruction error image. ;

[0032] Step 4: Convert the original reconstruction error image With the pseudo-reconstruction error image Feature fusion is performed along the channel dimension, and the fused features are input into the back-end classifier for classification to obtain the detection result of the input image.

[0033] In one example, two error images are stitched together along the channel dimension and then input into the lightweight classification network MobileNetV3 for real / false classification: To fully utilize these two types of error information, this invention will use the original reconstructed error image. and pseudo-reconstruction error image By stitching along the channel dimension, a 6-channel feature is formed:

[0034]

[0035] in, This indicates a splicing operation. This represents the backend classifier.

[0036] In this example, a pre-trained MobileNetV3 network is used as the classifier. This classifier receives the concatenated feature maps and outputs probability values. , indicating the input image The probability of generating the image.

[0037] Specifically, in step 1, the adaptive frequency mask refining module adopts an encoder-decoder architecture and integrates an adaptive attention unit (AAU). This unit dynamically generates attention weights based on the global features of the input spectrum, guiding two independent decoders to generate an intermediate frequency mask adapted to the image content. and complementary mask The formula is expressed as:

[0038] in, For encoder, and There are two independent decoders. This represents the Fast Fourier Transform.

[0039] In one example, the method performs end-to-end training by combining loss functions, which include: 1. Intermediate frequency reconstruction alignment loss Used to extract image content from intermediate frequency masks Error map compared with the original reconstruction Align in space:

[0040] in, This refers to the extracted intermediate frequency information.

[0041] 2. Adaptive mask refinement loss This is used to guide mask generation and introduce regularization based on image content complexity, using a predefined ideal intermediate frequency mask. As a weak supervisory signal, guiding and The learning directions should be complementary:

[0042] in, and For a predefined ideal mask, For regularization terms, This is the balance coefficient.

[0043] Cross-entropy loss The standard loss function used to train the backend classifier minimizes the gap between the predicted label and the true label:

[0044] in, These are the real labels; the real image is 0, and the generated image is 1. It is the predicted probability of the model.

[0045] Minimize the total loss using backpropagation and gradient descent (Adam optimizer) It can simultaneously optimize the parameters of the AdaptiveFMRE module and the backend classifier to achieve the best detection performance.

[0046] .

[0047] Example 2: This invention provides an AIGC-generated image detection system based on frequency reconstruction error, comprising: The frequency mask module is used to perform spectral analysis on the input image to be detected and generate a complementary mask, and filter out preset frequency band information based on the complementary mask to generate a pseudo image; The reconstruction network module is used to reconstruct the image to be detected and the pseudo image respectively, and generate the corresponding original reconstructed image and pseudo reconstructed image; An error calculation module is used to calculate the difference between the image to be detected and the original reconstructed image to generate an original reconstruction error map, and to calculate the difference between the pseudo image and the pseudo reconstructed image to generate a pseudo reconstruction error map; The fusion classification module is used to fuse the original reconstruction error map and the pseudo reconstruction error map, and classify the fused features to output the authenticity detection result of the image to be detected.

[0048] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A robust detection method for images generated using a diffusion model based on frequency domain-guided reconstruction error, characterized in that, Includes the following steps: Step 1: Input the image to be detected, analyze the spectrum of the input image using the adaptive frequency mask refinement module, generate a complementary mask, and filter out preset frequency band information by applying the complementary mask in the frequency domain, thereby constructing a pseudo-generated image; Step 2: Using a lightweight reconstruction network, reconstruct the input image and the pseudo-generated image respectively to obtain the original reconstructed image and the pseudo-reconstructed image; Step 3: Calculate the difference between the input image and its original reconstructed image to obtain the original reconstruction error map, and calculate the difference between the pseudo-generated image and its pseudo-reconstructed image to obtain the pseudo reconstruction error map; Step 4: Perform feature fusion between the original reconstruction error map and the pseudo reconstruction error map, and input the fused features into a classifier for classification and discrimination, and output the detection result of whether the input image is a real image or a generated image.

2. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, The specific steps of step 1 are as follows: Step 1.1: Perform a Fast Fourier Transform on the input image to obtain the spectrum, and use an adaptive frequency masking refinement module to analyze the spectral characteristics of the input image; Step 1.2: Generate an intermediate frequency mask that adapts to the spectral features of the input image; Step 1.3: Generate a complementary mask based on the intermediate frequency mask, wherein the complementary mask and the intermediate frequency mask are complementary in the frequency domain; Step 1.4: Apply the complementary mask in the frequency domain to filter out the preset frequency band information and construct a pseudo-generated image.

3. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 2, characterized in that, The adaptive frequency mask refining module adopts an encoder-decoder architecture and performs the following steps: The encoder extracts the depth features of the spectrum of the input image, the first decoder generates an intermediate frequency mask based on the depth features, and the second decoder generates a complementary mask based on the depth features. The adaptive frequency mask refining module introduces an adaptive attention mechanism to dynamically adjust the mask generation strategy based on spectral characteristics.

4. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, Step 1.4 includes: multiplying the spectrum with the complementary mask to obtain the filtered spectrum; performing an inverse fast Fourier transform on the filtered spectrum to obtain the pseudo-generated image; wherein the pseudo-generated image retains the low-frequency and high-frequency information of the input image and filters out the mid-frequency information.

5. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, In step 2, the lightweight reconstruction network employs a lightweight variational autoencoder and performs the following steps: Step 2.1: Reconstruct the input image using a lightweight reconstruction network to obtain the original reconstructed image; Step 2.2: Reconstruct the pseudo-generated image using the same lightweight reconstruction network to obtain the pseudo-reconstructed image.

6. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, The specific steps of step 3 are as follows: Step 3.1: Calculate the difference between the input image and its original reconstructed image to obtain the original reconstruction error map; Step 3.2: Calculate the difference between the pseudo-generated image and the pseudo-reconstructed image to obtain the pseudo-reconstruction error map.

7. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, The specific steps of step 4 are as follows: Step 4.1: Stitch the original reconstruction error map and the pseudo reconstruction error map along the channel dimension to form a multi-channel fusion feature; Step 4.2: Input the fused features into the classifier, which performs convolution and pooling on the fused features to obtain classification features; Step 4.3: Output the detection result based on the classification features to determine whether the input image is a real image or a generated image.

8. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 1, characterized in that, This method performs end-to-end training using a combined loss function, which includes: Intermediate frequency reconstruction alignment loss This is used to align the intermediate frequency image region extracted by the intermediate frequency mask with the original reconstruction error map; Cross-entropy loss , used to train the classifier.

9. The robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error according to claim 8, characterized in that, The combined loss function also includes adaptive masking refinement loss. This is used to guide the mask generated by the frequency mask refining module to approximate a predefined ideal mask.

10. A system used in the robustness detection method for image generation based on a diffusion model with frequency domain guided reconstruction error as described in any one of claims 1-9, characterized in that, include: The frequency mask module is used to perform spectral analysis on the input image to be detected and generate a complementary mask, and filter out preset frequency band information based on the complementary mask to generate a pseudo image; The reconstruction network module is used to reconstruct the image to be detected and the pseudo image respectively, and generate the corresponding original reconstructed image and pseudo reconstructed image; An error calculation module is used to calculate the difference between the image to be detected and the original reconstructed image to generate an original reconstruction error map, and to calculate the difference between the pseudo image and the pseudo reconstructed image to generate a pseudo reconstruction error map; The fusion classification module is used to fuse the original reconstruction error map and the pseudo reconstruction error map, and classify the fused features to output the authenticity detection result of the image to be detected.

Images