Diffusion model generated image detection method and device and storage medium

Abstract

The invention relates to a diffusion model generated image detection method and device, and a storage medium. The method comprises the following steps: S1, obtaining an original input image and a label value thereof; s2, reconstructing the original input image for N times to obtain N generated images, and forming a basic image sequence by the original input image and all the generated images; s3, deleting a first image of the basic image sequence to obtain a first enhanced image sequence; s4, performing linear combination on every two adjacent images in the basic image sequence to obtain second generated images, and arranging all the second generated images in sequence to obtain a second enhanced image sequence; s5, deleting the last image in the basic image sequence to obtain a reference image sequence; s6, performing feature extraction on each image sequence to obtain a respective feature vector; s7, training the detection model by using the feature vectors and label values of the first enhanced image sequence, the second enhanced image sequence and the reference image sequence; and S8, carrying out authenticity detection on a to-be-detected image by using the trained detection model. Compared with the prior art, the method has the advantages that higher detection precision can be realized under the condition of less training data, the average accuracy on multiple reference sets is remarkably improved, and the method has robustness on JPEG compression and zooming.

Description

Technical Field

[0001] This invention relates to the fields of digital image forensics and AI content security technology, and in particular to a diffusion model-generated image detection method, apparatus and storage medium. Background Technology

[0002] With the rapid development of technology, diffusion models have become capable of generating highly realistic images. These images are often indistinguishable from real images by the naked eye, leading to an increasing risk of model misuse. Therefore, reliable detection techniques are urgently needed to identify such images. To address this issue, some research has attempted to utilize the knowledge of large-scale pre-trained models (such as CLIP) to assist image detection. However, these methods have failed to fully consider the inherent characteristics of images generated by diffusion models, resulting in limited detection effectiveness. In recent years, some research has begun to focus on detection by comparing the differences between images before and after processing with diffusion models.

[0003] For example, Chinese patent CN120997641A discloses an image detection method based on a diffusion model for image reconstruction using a convolutional neural network. This method includes: acquiring a target dataset, which includes a real image, a first image, and a second image; the first image is determined by diffusion of the real image; the second image is an enhanced version of the real image and the first image; reconstructing the target dataset to obtain a training dataset; training a pre-defined original image detection model based on the training dataset to obtain a target image detection model; and inputting the image to be detected into the target image detection model for detection to determine whether the image to be detected is a real image.

[0004] However, most existing technologies, including the detection methods mentioned above, rely on a single reconstruction, failing to fully capture the key features of the images generated by the diffusion model. To address this issue, this invention proposes to construct multiple reconstruction sequences through iterative image reconstruction for learning, thereby more comprehensively capturing the representative features of the diffusion model and significantly improving detection performance. Summary of the Invention

[0005] The purpose of this invention is to provide a diffusion model-generated image detection method, apparatus, and storage medium.

[0006] The objective of this invention can be achieved through the following technical solutions: A diffusion model-generated image detection method, comprising: Step S1: Obtain the original input image and its label value, wherein the label value is a floating-point number between 0 and 1, where 1 indicates a real image and 0 indicates a generated image, and the label value of the original input image is 1 or 0; Step S2: Reconstruct the original input image N times to obtain N generated images, and combine the original input image and all generated images to form a basic image sequence; Step S3: Delete the first image of the base image sequence to obtain the first enhanced image sequence; Step S4: Linearly combine every two adjacent images in the base image sequence to obtain the second generated image, and arrange all the second generated images in order to obtain the second enhanced image sequence; Step S5: Delete the last image in the base image sequence to obtain the reference image sequence; Step S6: Extract features from the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence to obtain their respective feature vectors. Set the label value of the first enhanced image sequence to the generated image, set the label value of the reference image sequence to the label value of its corresponding original input image, and set the label value of the second enhanced image sequence according to the difference between the first image in the second enhanced image sequence and the original input image and the label value of the original input image. Step S7: Train the detection model using the feature vectors and label values ​​of the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence; Step S8: Use the trained detection model to perform authenticity detection on the image to be detected.

[0007] The reconstruction process in step S2 is implemented by using a diffusion model to perform DDIM inversion and generation, and the Nth reconstruction is based on the image after the N-1th reconstruction.

[0008] The second enhanced image sequence is: in: For the second enhanced image sequence, It is a random floating-point number between 0 and 1. The original input image, This is the generated image after the first reconstruction. The generated image after the (N-1)th reconstruction. For the generated image after the Nth reconstruction, when the label value of the original input image is 1, the label value of the second enhanced image sequence is set to... When the label value of the original input image is 0, the label value of the second enhanced image sequence is set to 0.

[0009] Step S6 includes: Step S6-1: Extract features from the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence respectively to obtain their respective feature sequences; Step S6-2: Perform feature fusion on the obtained feature sequence using self-attention to obtain the feature vector; Step S6-3: Set the label values ​​of the first enhanced image sequence and the second enhanced image sequence to the generated image, and set the label values ​​of the reference image sequence to the label values ​​of its corresponding original input image.

[0010] In step S6-1, multi-scale feature extraction is used, and each image is split to obtain multiple feature sequences, each of which corresponds to a different scale.

[0011] In step S6-2, feature fusion is first performed within the sequence at a single scale using self-attention, and then cross-scale fusion is performed between different scales using self-attention to obtain feature vectors that enhance the global representation.

[0012] The loss function of the detection model is: in: For loss function, For label values, This represents the model's prediction results.

[0013] The detection model is a binary classification model.

[0014] A diffusion model-generated image detection apparatus includes a memory, a processor, and a program stored in the memory, characterized in that the processor executes the program to implement the method described above.

[0015] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0016] Compared with the prior art, the present invention has the following advantages: by performing multiple iterations of image reconstruction, the characteristics of the diffusion model itself are fully explored, high detection accuracy can be achieved with less training data, the accuracy on multiple benchmark datasets is significantly improved, and it has strong robustness to JPEG compression and downsampling. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the main steps of the method of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0019] A diffusion model-based image detection method, such as Figure 1As shown, it includes: Step S1: Obtain the original input image and its label value, where the label value is a floating-point number between 0 and 1. When it is 1, it represents the real image, and when it is 0, it represents the generated image. The label value of the original input image is 1 or 0. The original input images may be real or fake, and the specific label values ​​are set according to the actual situation. In this embodiment, in order to ensure the reliability of the sample label values, real images are used as the original input images, so the label values ​​are real images.

[0020] Step S2: Reconstruct the original input image N times to obtain N generated images, and combine the original input image and all generated images to form a basic image sequence; The reconstruction process employs a diffusion model to perform DDIM inversion and generation, and the Nth reconstruction is based on the image after the N-1th reconstruction.

[0021] In this embodiment, the number of diffusion steps is 20 and N is 5. Of course, in other embodiments, it is not limited to this.

[0022] Step S3: Delete the first image from the base image sequence to obtain the first enhanced image sequence: in: This is the first enhanced image sequence. This is the generated image after the first reconstruction. The generated image after the Nth reconstruction Step S4: Linearly combine every two adjacent images in the base image sequence to obtain the second generated image, and arrange all the second generated images in order to obtain the second enhanced image sequence: in: For the second enhanced image sequence, The value is a random floating-point number between 0 and 1. In this embodiment, it is obtained by random sampling from the beta distribution. The original input image, For the generated image after the (N-1)th reconstruction, when the label value of the original input image is 1, the label value of the second enhanced image sequence is set to... When the label value of the original input image is 0, the label value of the second enhanced image sequence is set to 0.

[0023] The linear combination method used in this embodiment has the core advantage of shifting the model's focus from the static and volatile appearance of images to the dynamic process characteristics that reflect the essence of the diffusion model's generation. This method, by mining the continuity and inherent patterns within the generated image sequences, provides the model with richer, more fundamental, and more stable learning signals. Therefore, it outperforms strategies that directly mix real and fake images in terms of data efficiency, generalization ability, and anti-interference capability, as detailed below: 1. The core of the above methods lies in capturing the "trajectory" of the generation process rather than a single "state": directly mixing the original real image and the final generated image can only provide a static and atypical intermediate state. However, the method in the patent mixes images from adjacent steps in the multiple reconstruction processes of the diffusion model. This combination simulates the continuous and gradual evolution trajectory of the image in the latent space during the generation or inversion of the diffusion model.

[0024] Enhancing the model's ability to identify the intrinsic features of generated images: By learning and comparing this continuous evolutionary sequence generated by the model itself, the detection model can gain a deeper understanding of the "behavioral patterns" or "fingerprint features" of images generated by the diffusion model, rather than simply learning to distinguish a static, possibly special, mixed image. This is equivalent to providing the model with richer meta-information about "how images are generated," thereby enabling it to learn the essential features of generated images more effectively with limited data, achieving higher detection accuracy.

[0025] 2. Focusing on relative relationships can improve feature stability: Post-processing such as JPEG compression and scaling can destroy the absolute pixel information of an image. A direct linear combination of real and fake images results in a mixture that heavily depends on the absolute pixel values ​​of the original and generated images. Post-processing can easily distort these values, leading to feature failure.

[0026] Leveraging the inherent consistency of the generation process—the linear combination of adjacent images—the key information lies in the relative patterns of change between adjacent reconstruction steps. During reconstruction, the diffusion model exhibits a certain inherent consistency and regularity in the evolution of image structure. While post-processing may destroy details in individual images, it often struggles to completely erase the relative relationships of change between images in a sequence. By learning these sequence-based relative features that reflect the inherent generational regularities, the model can better resist the interference of post-processing on individual image features, thus demonstrating stronger robustness.

[0027] 3. Learn more generalizable essential features: Different diffusion models or their variants may generate very different specific images. Direct real-fake image features may overfit a particular model or data.

[0028] Focusing on common generative dynamics: the changes between adjacent reconstruction steps are closer to the underlying "dynamic" features or decision boundaries that different diffusion models may share when generating images. By forcing the model to learn these more general, generative process-related features, rather than appearance features specific to a particular image class, the trained detection model can better generalize to datasets generated by different models and with diverse distributions, thus achieving stable and higher average accuracy across multiple benchmarks.

[0029] Step S5: Delete the last image in the base image sequence to obtain the reference image sequence; Since the length of the two enhanced image sequences is N, and the length of the base image sequence S is N + 1, in order to keep the lengths of the three sequences consistent before feature extraction, the base image sequence S is used to obtain the reference image sequence by taking only the first N images during training.

[0030] Step S6: Extract feature vectors from the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence respectively; set the label value of the first enhanced image sequence to the generated image; set the label value of the reference image sequence to the label value of its corresponding original input image; and set the label value of the second enhanced image sequence based on the difference between the first image in the second enhanced image sequence and the original input image, and the label value of the original input image, including: Step S6-1: Perform feature extraction on the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence to obtain their respective feature sequences. Multi-scale feature extraction is used, and multiple feature sequences are obtained from each image split, with each feature sequence corresponding to a different scale.

[0031] In this embodiment, a pre-trained feature extractor (e.g., CLIP) is used to extract multi-scale features from all images in the three sequences. Specifically, the network is divided into four stages according to the number of layers. For any image, its multi-stage features... F It can be represented as: in: v i Indicates the first Features extracted in stages. For features of length... N The image sequence can yield four sets of feature sequences corresponding to different semantic levels. F s Its formula is: in: Indicates the first position in the sequence N The image in the first iThe characteristics of each stage. This design can simultaneously capture low-level texture information (extracted from early stages) and high-level semantic information (extracted from later stages) of the image, thus more comprehensively representing the traces generated by the diffusion model.

[0032] Step S6-2: The obtained feature sequence is fused using self-attention to obtain a feature vector. First, feature fusion is performed within the sequence at a single scale using self-attention, and then cross-scale fusion is performed between different scales using self-attention to obtain a feature vector that enhances the global representation.

[0033] After feature extraction, a specially designed multi-scale self-attention mechanism is used for fusion. Specifically, the features within the sequence at each scale are first modeled using the self-attention mechanism and then channel-level average pooling is performed to obtain the aggregated features for that stage. F i The formula is as follows: in: This refers to a multi-head self-attention mechanism, where Avg represents channel-level average pooling and concat represents the concatenation operation. The aggregated features from the four stages are then passed through another parameter-independent self-attention layer for interactive fusion, forming the final representation F', as shown in the following formula: This representation is fed into a fully connected layer to obtain the final discrimination result. This mechanism can not only learn the evolutionary dynamics within the sequence, but also effectively integrate semantic features at different scales, thereby forming a more robust and discriminative feature representation for diffuse-generated images.

[0034] Step S6-3: Set the label value of the first enhanced image sequence to the generated image, set the label value of the reference image sequence to the label value of its corresponding original input image, and set the label value of the second enhanced image sequence according to the difference between the first image in the second enhanced image sequence and the original input image.

[0035] Step S7: Train the detection model using the feature vectors and label values ​​of the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence; The detection model is a binary classification model, and the loss function of the detection model is: in: For loss function, For label values, This represents the model's prediction results.

[0036] During training, loss functions are constructed for three image sequences, where the label of each sequence depends on the first image. If the first image is real, the sequence is labeled as real; otherwise, it is labeled as false. If it is a weighted sum of real and false images, the label is also the corresponding weighted value. The network is jointly optimized using a three-branch binary cross-entropy total loss. During the testing phase, only the first N images of the unenhanced sequence are used for judgment.

[0037] Step S8: Use the trained detection model to perform authenticity detection on the image to be detected.

[0038] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A diffusion model-generated image detection method, characterized in that, include: Step S1: Obtain the original input image and its label value, wherein the label value is a floating-point number between 0 and 1, where 1 indicates a real image and 0 indicates a generated image, and the label value of the original input image is 1 or 0; Step S2: Reconstruct the original input image N times to obtain N generated images, and combine the original input image and all generated images to form a basic image sequence; Step S3: Delete the first image of the base image sequence to obtain the first enhanced image sequence; Step S4: Linearly combine every two adjacent images in the base image sequence to obtain the second generated image, and arrange all the second generated images in order to obtain the second enhanced image sequence; Step S5: Delete the last image in the base image sequence to obtain the reference image sequence; Step S6: Extract features from the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence to obtain their respective feature vectors. Set the label value of the first enhanced image sequence to the generated image, set the label value of the reference image sequence to the label value of its corresponding original input image, and set the label value of the second enhanced image sequence according to the difference between the first image in the second enhanced image sequence and the original input image and the label value of the original input image. Step S7: Train the detection model using the feature vectors and label values ​​of the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence; Step S8: Use the trained detection model to perform authenticity detection on the image to be detected.

2. The diffusion model-generated image detection method according to claim 1, characterized in that, The reconstruction process in step S2 is implemented by using a diffusion model to perform DDIM inversion and generation, and the Nth reconstruction is based on the image after the N-1th reconstruction.

3. The diffusion model-generated image detection method according to claim 1, characterized in that, The second enhanced image sequence is: in: For the second enhanced image sequence, It is a random floating-point number between 0 and 1. The original input image, This is the generated image after the first reconstruction. The generated image after the (N-1)th reconstruction. For the generated image after the Nth reconstruction, when the label value of the original input image is 1, the label value of the second enhanced image sequence is set to... When the label value of the original input image is 0, the label value of the second enhanced image sequence is set to 0.

4. The diffusion model-generated image detection method according to claim 1, characterized in that, Step S6 includes: Step S6-1: Extract features from the first enhanced image sequence, the second enhanced image sequence, and the reference image sequence respectively to obtain their respective feature sequences; Step S6-2: Perform feature fusion on the obtained feature sequence using self-attention to obtain the feature vector; Step S6-3: Set the label values ​​of the first enhanced image sequence and the second enhanced image sequence to the generated image, and set the label values ​​of the reference image sequence to the label values ​​of its corresponding original input image.

5. The diffusion model-generated image detection method according to claim 4, characterized in that, In step S6-1, multi-scale feature extraction is used, and each image is split to obtain multiple feature sequences, each of which corresponds to a different scale.

6. The diffusion model-generated image detection method according to claim 4, characterized in that, In step S6-2, feature fusion is first performed within the sequence at a single scale using self-attention, and then cross-scale fusion is performed between different scales using self-attention to obtain feature vectors that enhance the global representation.

7. The diffusion model-generated image detection method according to claim 1, characterized in that, The loss function of the detection model is: in: For loss function, For label values, This represents the model's prediction results.

8. The diffusion model-generated image detection method according to claim 1, characterized in that, The detection model is a binary classification model.

9. A diffusion model-generated image detection apparatus, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-8.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-8.

Images