Layered artifact-aware generated image detection method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-02-09
- Publication Date
- 2026-06-02
Smart Images
Figure CN122134651A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of generated image detection technology, and more specifically, relates to a generated image detection method based on layered artifact perception. Background Technology
[0002] Generative image detection technology aims to effectively distinguish between natural and synthetic images in the context of rapid development of generative models, thereby ensuring information security and application reliability. Existing generative image detection methods mainly include pixel-based spatial analysis, frequency domain analysis, and pre-trained model representation-based methods. Pixel-based detection methods directly model the local texture, structure, or statistical features of the image in the spatial domain to identify generation traces in synthetic images. Related research characterizes the structural and distributional differences caused by the synthesis process through non-projective reconstruction, image transformation, or color distribution statistics. Frequency-based detection methods analyze spectral artifacts introduced by upsampling or reconstruction operations in the image from a frequency domain perspective. Existing methods extract local or global frequency features through frequency decomposition methods such as discrete cosine transform and model high-frequency amplitude spectrum, phase spectrum, or frequency reconstruction errors to achieve detection of various generation types. Pre-trained model-based detection methods utilize visual or visual language models trained on large-scale datasets for feature extraction. A series of works directly classify or discriminate based on pre-trained representations, while other studies guide the model to extract discriminative features related to image generation through mechanisms such as cue learning or information bottlenecks.
[0003] Furthermore, existing synthetic image detection methods based on pixel space or frequency domain typically rely on data from limited sources for training. The learned discrimination boundaries are biased towards the distribution of known generators. When faced with unseen generators, false samples easily fall into the ambiguous region between the real and generated categories, resulting in scattered detection results and insufficient generalization ability. Meanwhile, existing detection methods based on pre-trained models mainly rely on content- or category-related semantic representations, making it difficult to effectively characterize consistent generation traces across generation sources. Moreover, a large amount of task-irrelevant information in the representations can obscure key discrimination clues, affecting the establishment of reliable decision boundaries.
[0004] Overall, existing methods have achieved certain results in scenarios with known generators, but their training data distribution and feature representation are still affected by specific generator sources and content semantics, and the discrimination boundary learned by the model is difficult to cover the distribution of unknown generators. At the same time, the over-reliance on content and category-related semantics makes it difficult for the model to stably extract generation trace features that are consistent in different generator sources, thus limiting the overall generalization ability. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a generated image detection method based on hierarchical artifact perception. This method adds hierarchical artifact perception representation to image feature extraction, aggregates artifact information at the global and local levels, enhances the robust representation ability of generated images, and thus improves the accuracy of generated image detection.
[0006] To achieve the above-mentioned objectives, the generative image detection method based on layered artifact perception of the present invention includes the following steps:
[0007] S1: Collect several images according to actual needs and label them with real-world tags to form a training sample set;
[0008] S2: Construct a generative image detection model based on hierarchical artifact perception, including an image feature extraction module, a hierarchical artifact perception representation module, and a detection module, wherein:
[0009] The image feature extraction module is used to extract image features from the input image. It adopts a multi-level feature extraction structure, where each feature extraction layer extracts global labels and local labels of image patches from its input. Let the first layer be denoted as . Layer output features are ,in Indicates the first The global marker obtained by the layer Indicates the first The first layer obtained Local features of an image patch , This indicates the hierarchy of the image feature extraction module. , Indicates the number of image patches. Indicates the size of the image patch matrix. Indicate the dimension to be marked; then Layer features Send to the layered artifact perception representation module;
[0010] The layered artifact perception representation module is used for... Layer features Artifact-aware fusion is performed to obtain fused features. And send it to the detection module; the hierarchical artifact perception representation module includes a global label fusion module, The system comprises a hierarchical region pooling module, a pooling label fusion module, and a cross-granularity fusion module, among which:
[0011] The global marker fusion module is used for... Global markers Perform fusion to obtain a fusion global marker. And send it to the cross-granularity fusion module;
[0012] The hierarchical region pooling module is used to respectively process the first... Local markers in layer features Perform hierarchical region pooling to obtain local pooling labels. Send to the pooling tag fusion module;
[0013] The pooling tag fusion module is used for... Local pooling features Perform fusion to obtain fused local markers. And send it to the cross-granularity fusion module;
[0014] The cross-granularity fusion module is used for global fusion marking. and fusion local markers By performing fusion, fusion characteristics are obtained. ;
[0015] The detection module is used to detect based on fused features. Predict the probability of obtaining the authenticity of the input image;
[0016] S3: Use the training sample set from step S1 to train the generated image detection model to obtain the trained generated image detection model;
[0017] S4: Input the image to be detected into the trained generated image detection model to obtain the detection result of whether it is a generated image.
[0018] This invention relates to a generated image detection method based on hierarchical artifact perception. It constructs a generated image detection model based on hierarchical artifact perception. The image feature extraction module extracts global and local image patch labels from the input image layer by layer. The hierarchical artifact perception representation module fuses the global and local image patch labels of all layers. The detection module obtains the generated image detection result based on the fused features. After training the generated image detection model with a pre-acquired training sample set, it can be used for the generation and detection of actual images.
[0019] The present invention has the following beneficial effects:
[0020] 1) This invention designs a hierarchical artifact perception representation module, which suppresses the dominant role of content and category-related semantics, retains key structural clues, aggregates global and local artifact information, and extracts discriminative invariant features that are consistent across different generation sources, thereby enhancing the model's robust representation ability of generated images and improving the detection accuracy of generated images.
[0021] 2) In terms of constructing the training sample set, the present invention can also expand the distribution range of training data by constructing continuous transition samples between real images and generated images, so as to simulate the characteristics of potential unknown generators, guide the model to learn more transferable and inclusive discrimination boundaries, and achieve stable detection of images synthesized by unknown generators. Attached Figure Description
[0022] Figure 1 This is a flowchart illustrating a specific implementation of the generative image detection method based on layered artifact perception according to the present invention.
[0023] Figure 2 This is a schematic diagram of the hybrid drive distribution enhancement method in this embodiment;
[0024] Figure 3 This is a structural diagram of the image detection model generated in this invention. Detailed Implementation
[0025] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.
[0026] Example
[0027] Figure 1 This is a flowchart illustrating a specific implementation of the generative image detection method based on strong layered artifact perception according to the present invention. Figure 1 As shown, the generative image detection method based on layered artifact perception of the present invention includes the following steps:
[0028] S101: Obtain the training sample set:
[0029] Collect several images according to actual needs and label them with authenticity to form a training sample set.
[0030] In practical applications, training data often comes from limited generative models, which makes it difficult to cover potential unknown generative distributions. This embodiment proposes a hybrid driven distribution enhancement method to construct continuous transition samples between real and generated images, thereby expanding the training data distribution and guiding the model to learn more transferable discrimination boundaries. Figure 2 This is a schematic diagram of the hybrid drive distribution enhancement method in this embodiment. For example... Figure 2 As shown, the specific method of the hybrid drive distribution enhancement method in this embodiment is as follows:
[0031] First, a set of real images and a set of generated images are collected. Then, several mixed samples are generated to form a mixed image set. The method for generating the mixed samples is as follows:
[0032] Randomly select real image samples from the real image set and the generated image set respectively. and generate image samples Then generate mixed image samples. And set its label to generate image samples Labels, mixed image samples The generation method is as follows:
[0033] ,
[0034] in, This represents the mixing coefficient, which is the mixing coefficient in this embodiment. Obtained by random sampling from the Beta distribution.
[0035] The training sample set is obtained by merging the set of real images and the set of mixed images.
[0036] S102: Constructing a generative image detection model:
[0037] To improve the accuracy of generated image detection, this invention constructs a generated image detection model based on hierarchical artifact perception. Figure 3 This is a structural diagram of the image detection model generated in this invention. For example... Figure 3 As shown, the image detection model generated in this invention includes an image feature extraction module, a hierarchical artifact perception representation module, and a detection module, wherein:
[0038] The image feature extraction module is used to extract image features from the input image. It adopts a multi-level feature extraction structure. Each feature extraction layer extracts a global token (CLS Token) and a local token (Local Token) for its input image patch. Layer output features are ,in Indicates the first The global marker obtained by the layer Indicates the first The first layer obtained Local features of an image patch , This indicates the hierarchy of the image feature extraction module. , Indicates the number of image patches. Indicates the size of the image patch matrix. This indicates the labeled dimension. Then... Layer features The image is sent to the layered artifact perception representation module. In this embodiment, the image feature extraction module uses a CLIP model image encoder.
[0039] The layered artifact perception representation module is used for... Layer features Artifact-aware fusion is performed to obtain fused features. And send it to the detection module. For example... Figure 3 As shown, the hierarchical artifact perception representation module in this invention includes a global label fusion module. The system comprises a Hierarchical Region Pooling (HiRP) module, a Pooling Tag Fusion module, and a Cross-Granularity Fusion (CFG) module, among which:
[0040] The global marker fusion module is used for... Global markers Perform fusion to obtain a fusion global marker. And send it to the cross-granularity fusion module.
[0041] The hierarchical region pooling module is used to respectively process the first... Local markers in layer features Perform hierarchical region pooling to obtain local pooling labels. Send to the pooling tag fusion module.
[0042] Considering that the generated traces have different manifestations at different spatial scales, the specific method of hierarchical region pooling in this embodiment is as follows:
[0043] Set according to actual needs Individual scale , For local markings of each layer Using different scales Divide it into Identify non-overlapping regions, and then perform pooling operations on each region to obtain pooling labels. , Finally, the following formula is used for multi-scale aggregation to obtain the local pooling markers. :
[0044] .
[0045] The pooling tag fusion module is used for... Local pooling flags Perform fusion to obtain fused local markers. And send it to the cross-granularity fusion module.
[0046] Since generated clues may be distributed across different network depths and semantic granularities, this invention designs a cross-layer fusion mechanism to fuse multi-layer global and multi-layer local tags, thereby fully utilizing shallow texture information and deep structural information. In this embodiment, both the global tag fusion module and the pooling tag fusion module employ adaptive weighted fusion, the specific method of which is as follows:
[0047] Set learnable weight vector The normalized weights are generated using the following formula. :
[0048] .
[0049] Then, the fusion global marker is calculated using the following formula. :
[0050] ,
[0051] in, , Represents normalized weights The Middle Each weight value.
[0052] The cross-granularity fusion module is used for global fusion marking. and fusion local markers By performing fusion, fusion characteristics are obtained. .
[0053] In this embodiment, to achieve better fusion results, a cross-granularity fusion module based on adaptive weights is employed. For example... Figure 3 As shown, the cross-granularity fusion module in this embodiment includes a first splicing layer, a first linear layer, a GELU function layer, a second linear layer, a weighted layer, and a second splicing layer, wherein:
[0054] The first splicing layer is used to merge global markers. and fusion local markers By splicing the pieces together, we obtain the splicing markers. And send it to the first linear layer.
[0055] The first linear layer is used for splicing markers. Perform a linear mapping and send the resulting features to the GELU function layer.
[0056] The GELU function layer is used to process the received features using the GELU activation function and then send the resulting features to the second linear layer.
[0057] The second linear layer is used to linearly map the received features to generate a 2D weight vector. And send it to the weighted layer.
[0058] Weighted layers are used to calculate the weights based on a 2D weight vector. For fusion global markers and fusion local markers The weighted values are processed separately to obtain the weighted global labels. and weighted local labels And send it to the second splicing layer.
[0059] The second concatenation layer is used for weighted global marking. and weighted local labels By splicing the data, the fused features are obtained. .
[0060] The detection module is used to detect based on fused features. Predict the probability of the input image being true.
[0061] S103: Training the generated image detection model:
[0062] The generated image detection model is trained using the training sample set in step S1 to obtain the trained generated image detection model.
[0063] In this embodiment, the cross-entropy loss function is used during the training of the generated image detection model to form a compact intra-class and separated inter-class discriminative distribution, thereby significantly improving the generalization ability in the task of synthesizing images with unknown generators.
[0064] Furthermore, in order to enhance the ability to perceive generated cues without compromising the pre-trained representation capabilities, this embodiment employs a low-rank adaptive method in the CLIP model image encoder during the training process of the generated image detection model. Low-rank adaptive parameters are introduced into the attention mechanism of the transformer to perform lightweight adjustments on query, key, and value projections, so that the model can gradually enhance its attention to generated features while maintaining the original semantic structure.
[0065] S104: Image Detection
[0066] The image to be detected is input into the trained generated image detection model to obtain the detection result of whether it is a generated image.
[0067] To better illustrate the technical effects of the present invention, specific examples are used to experimentally verify the present invention.
[0068] This embodiment systematically evaluates the present invention on three representative synthetic image detection benchmark datasets to verify its generalization performance under different generation mechanisms and data distributions. Specifically, these include: the GenImage dataset, which contains real and synthetic images from eight different generation models, boasting a large scale and diverse generator types; the Ojha dataset, covering generated images based on diffusion and autoregressive models, sourced from ImageNet and LAION data distributions; and the ForenSynths dataset, primarily used to evaluate the model's cross-generator generalization ability across various CNN-based generation models (including deep generation models). To ensure the comparability and objectivity of the experimental results, this invention strictly adheres to evaluation protocols widely used in existing work. Model performance is measured by accuracy (Accuracy) and average precision (Average Precision, AP), with a uniform classification decision threshold set to 0.5.
[0069] For each dataset, this embodiment selects the best-performing existing method as the comparison method, totaling 10 comparison methods:
[0070] CNNDect: See the document "Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. Cnn-generated images are surprisingly easy to spot... for now. In CVPR, pages 8695–8704, 2020.";
[0071] LGrad: See the literature "Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on gradients: Generalized artifacts representation for gan-generated images detection. In CVPR, pages 12105–12114, 2023.";
[0072] UniFD: See the literature "Utkarsh Ojha, Yuheng Li, and Yong Jae Lee. Towards universal fake image detectors that generalize across generative models. In CVPR, pages 24480–24489, 2023.";
[0073] NPR: See the literature "Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Rethinking the up-sampling operations in cnn-based generative network for generalizable deepfake detection. In CVPR, pages 28130–28139, 2024.";
[0074] FreqNet: See the literature "Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, Ping Liu, and Yunchao Wei. Frequency-aware deepfake detection: Improving generalizability through frequency space domain learning. In AAAI, pages 5052–5060, 2024.";
[0075] SAFE: See the literature "Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Fuli Feng. Improving synthetic image detection towards generalization: An image transformation perspective. In KDD, pages 2405–2414, 2025.";
[0076] C2P-CLIP: See the literature "Chuangchuang Tan, Renshuai Tao, Huan Liu, GuanghuaGu, Baoyuan Wu, Yao Zhao, and Yunchao Wei. C2p-clip: Injecting categorycommon prompt in clip to enhance generalization in deepfake detection. InAAAI, pages 7184–7192, 2025.";
[0077] VIB: See the document "Haifeng Zhang, Qinghui He, Xiuli Bi, Weisheng Li, BoLiu, and Bin Xiao. Towards universal ai-generated image detection by variational information bottleneck network. In CVPR, pages 23828–23837, 2025.";
[0078] CoD: See the literature "Zexi Jia, Chuanwei Huang, Yeshuang Zhu, Hongyan Fei, Xiaoyue Duan, Zhiqiang Yuan, Ying Deng, Jiapei Zhang, Jinchao Zhang, and JieZhou. Secret lies in color: Enhancing ai-generated images detection with color distribution analysis. In CVPR, pages 13445–13454, 2025.";
[0079] AIDE: See the literature "Shilin Yan, Ouxiang Li, Jiayin Cai, Yanbin Hao, Xiaolong Jiang, Yao Hu, and Weidi Xie. A sanity check for ai-generated image detection. In ICLR."
[0080] Table 1 is a performance comparison table of the present invention and the comparison method on the GenImage dataset in this embodiment.
[0081]
[0082] Table 1
[0083] As shown in Table 1, the present invention achieves superior average detection performance compared to existing methods in settings that include multiple GANs and diffusion generators, with an average accuracy of 97.6%, indicating that it has stable and reliable discrimination capabilities under complex generation distributions.
[0084] Table 2 is a performance comparison table of the present invention and the comparison method on the Ojha dataset in this embodiment.
[0085]
[0086] Table 2
[0087] As shown in Table 2, when trained using only a single diffusion model, this invention can effectively generalize to a variety of different diffusion architectures, with an average accuracy of 98.3%.
[0088] Table 3 is a performance comparison table of the present invention and the comparison method on the ForenSynths dataset in this embodiment.
[0089]
[0090] Table 3
[0091] As shown in Table 3, on the traditional GAN-based ForenSynths dataset, the average accuracy of the present invention can surpass the state-of-the-art methods, and its performance is particularly outstanding on the semantically complex subset deepfake.
[0092] The results above demonstrate that the present invention exhibits stable and transferable detection performance under different generation mechanisms, different data distributions, and high-difficulty generation scenarios, verifying its effectiveness and versatility in practical applications.
[0093] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.
Claims
1. A method for detecting generated images based on hierarchical artifact perception, characterized in that, Includes the following steps: S1: Collect several images according to actual needs and label them as generated images to form a training sample set; S2: Construct a generative image detection model based on hierarchical artifact perception, including an image feature extraction module, a hierarchical artifact perception representation module, and a detection module, wherein: The image feature extraction module is used to extract image features from the input image. It adopts a multi-level feature extraction structure, where each feature extraction layer extracts global labels and local labels of image patches from its input. Let the first layer be denoted as . Layer output features are ,in Indicates the first The global marker obtained by the layer Indicates the first The first layer obtained Local features of an image patch , This indicates the hierarchy of the image feature extraction module. , Indicates the number of image patches. Indicates the size of the image patch matrix. Indicate the dimension to be marked; then Layer features Send to the layered artifact perception representation module; The layered artifact perception representation module is used for... Layer features Artifact-aware fusion is performed to obtain fused features. And send it to the detection module; the hierarchical artifact perception representation module includes a global label fusion module, The system comprises a hierarchical region pooling module, a pooling label fusion module, and a cross-granularity fusion module, among which: The global marker fusion module is used for... Global markers Perform fusion to obtain a fusion global marker. And send it to the cross-granularity fusion module; The hierarchical region pooling module is used to respectively process the first... Local markers in layer features Perform hierarchical region pooling to obtain local pooling labels. Send to the pooling tag fusion module; The pooling tag fusion module is used for... Local pooling features Perform fusion to obtain fused local markers. And send it to the cross-granularity fusion module; The cross-granularity fusion module is used for global fusion marking. and fusion local markers By performing fusion, fusion characteristics are obtained. ; The detection module is used to detect based on fused features. Predict the probability of obtaining the authenticity of the input image; S3: Use the training sample set from step S1 to train the generated image detection model to obtain the trained generated image detection model; S4: Input the image to be detected into the trained generated image detection model to obtain the detection result of whether it is a generated image.
2. The generated image detection method according to claim 1, characterized in that, The method for constructing the training sample set in step S1 is as follows: First, a set of real images and a set of generated images are collected. Then, several mixed samples are generated to form a mixed image set. The method for generating the mixed samples is as follows: Randomly select real image samples from the real image set and the generated image set respectively. and generate image samples Then generate mixed image samples. And set its label to generate image samples Labels, mixed image samples The generation method is as follows: ， in, Indicates the mixing coefficient; The training sample set is obtained by merging the set of real images and the set of mixed images.
3. The generated image detection method according to claim 2, characterized in that, The mixing coefficient Obtained by random sampling from the Beta distribution.
4. The generated image detection method according to claim 1, characterized in that, The image feature extraction module uses a CLIP model image encoder.
5. The generated image detection method according to claim 1, characterized in that, The specific method of hierarchical region pooling in the hierarchical region pooling module in step S2 is as follows: Set according to actual needs Individual scale , For local markings of each layer Using different scales Divide it into Identify non-overlapping regions, and then perform pooling operations on each region to obtain pooling labels. , Finally, the following formula is used to perform multi-scale aggregation to obtain the local pooling markers. : 。 6. The generated image detection method according to claim 1, characterized in that, In step S2, both the global label fusion module and the pooling label fusion module employ adaptive weighted fusion, and the specific method is as follows: Set learnable weight vector The normalized weights are generated using the following formula. : ； Then, the fusion global marker is calculated using the following formula. : ， in, , Represents normalized weights The Middle Each weight value.
7. The generated image detection method according to claim 1, characterized in that, The cross-granularity fusion module in step S2 includes a first splicing layer, a first linear layer, a GELU function layer, a second linear layer, a weighted layer, and a second splicing layer, wherein: The first splicing layer is used to merge global markers. and fusion local markers By splicing the pieces together, we obtain the splicing markers. And send it to the first linear layer; The first linear layer is used for splicing markers. Perform a linear mapping and send the resulting features to the GELU function layer; The GELU function layer is used to process the received features using the GELU activation function and then send the resulting features to the second linear layer. The second linear layer is used to linearly map the received features to generate a 2D weight vector. And send it to the weighted layer; Weighted layers are used to calculate the weights based on a 2D weight vector. For fusion global markers and fusion local markers The weighted values are processed separately to obtain the weighted global labels. and weighted local labels And send it to the second splicing layer; The second concatenation layer is used for weighted global marking. and weighted local labels By splicing the data, the fused features are obtained. .
8. The generated image detection method according to claim 1, characterized in that, In step S3, the loss function used during the training of the image detection model is the cross-entropy loss function.