AI-generated image detection method based on tile consistency

Abstract

The application discloses an AI generated image detection method based on tile consistency, constructs a mixed training data set including a labeled training data set and an unlabeled mixed image set, constructs an AI generated image detection model including an image blocking module, an image feature extraction module and a prediction module, trains the AI generated image detection model by using the mixed training data set, wherein a loss term of a loss function includes a cross entropy loss calculated based on the labeled training data set, and a tile consistency based regularization term and a tile based entropy regularization term calculated based on the mixed training data set, and detects images by using the trained AI generated image detection model. The application uses the mixed training data set, introduces the tile consistency based regularization term and the tile based entropy regularization term in the loss function of the model training, and improves the accuracy, robustness and generalization ability of the AI generated image detection.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically, relates to an AI-generated image detection method based on tile consistency. Background Technology

[0002] Existing AI-generated image detection technologies are mainly divided into two categories: one is detection methods based on transfer learning, and the other is training-free detection methods.

[0003] The core approach of transfer learning-based detection methods (such as DRCT, NPR, and Fatformer) is as follows: A training set is constructed using a limited number of labeled synthetic images generated by a specific generative model and labeled real images. The detector is then trained to learn the image features of that specific generative model, with the aim of generalizing the learned feature representations to image detection tasks generated by unseen generative models. Some methods extract frequency domain artifacts, physical consistency anomalies, or tile-level discriminative features as detection criteria. For example, the Fatformer model integrates CLIP image tile encoding technology to achieve language-guided discrimination, while the BSA model directly extracts discriminative features from tiles using a simple network.

[0004] The core approach of training-free detection methods (such as AEROBLADE and Brokman2025) is that there is no need to build a training set for model training. The theory is based on the assumption that "the distribution of AI-generated content is not within the range of real data distribution". The difference between real images and fake images is quantified by pre-trained models, or the difference in coding cost in lossless image compression is used as the basis for anomaly detection. The optimal threshold is calculated on labeled real images, and then the detection of unknown images is achieved based on this threshold.

[0005] The two methods above have shortcomings in different aspects:

[0006] (1) Detection methods based on transfer learning have poor generalization ability: These methods rely on labeled synthetic data of specific generative models for training. Due to the distribution difference between the training domain and the deployment domain, the detection performance will drop sharply when faced with novel generative models that have not been seen before. For example, the DRCT and NPR methods perform well on images generated by the StableDiffusion series models, but the accuracy drops significantly on images generated by generative models with large differences such as BigGAN and VAR. Among them, the accuracy of DRCT on images generated by VAR models is only 57%. At the same time, these methods require a lot of continuous annotation work to adapt to emerging generative models, resulting in high costs in practical applications.

[0007] (2) Untrained detection methods are unstable and threshold sensitive: These methods lack explicit modeling of the boundary between real and fake images, and their performance is highly sensitive to the discrimination criteria used. Their performance fluctuates greatly on images generated by different generative models. For example, AEROBLADE achieves approximately 80% accuracy on images generated by the Glide model, but its accuracy drops sharply to 51% on images generated by the VQDM model; Brokman2025 achieves nearly 87% accuracy on images generated by the BigGAN model, but its accuracy is only 43% on images generated by the Midjourney model. Furthermore, when encountering real images that deviate from the training distribution, false positives are easily generated, mistakenly identifying real images as fake images.

[0008] (3) Limited self-supervised capability of unlabeled data: Faced with existing unlabeled data, consistency constraints are applied to images under different augmentations. These methods rely on the design of the augmentation method, lack class constraints at the tile level, and are difficult to amplify subtle differences in generated image artifacts, thus limiting the model's generalization ability and detection performance. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide an AI-generated image detection method based on tile consistency. This method uses a mixed training dataset and introduces a tile consistency-based regularization term and a tile entropy-based regularization term into the loss function of the model training, thereby improving the accuracy, robustness, and generalization ability of AI-generated image detection.

[0010] To achieve the above-mentioned objectives, the AI-generated image detection method based on tile consistency of the present invention includes the following steps:

[0011] S1: Construct a hybrid training dataset, including a labeled training dataset. and unlabeled mixed image sets Among them, the labeled training dataset It contains several training datasets, each containing a natural image and its corresponding ground truth label; and an unlabeled mixed image set. It contains several real images and AI-generated images;

[0012] S2: Construct an AI-generated image detection model, including an image segmentation module, an image feature extraction module, and a prediction module, where:

[0013] The image segmentation module normalizes the input image to a preset size, then divides it into several non-overlapping patches, from which samples are obtained. Each image patch is then sent to the image feature extraction module;

[0014] The image feature extraction module is used for... Feature extraction is performed on each image patch, and the obtained features are... Patch features Send to the prediction module, ;

[0015] The prediction module is used to predict based on the features of each patch. The log odds of the real image and the AI-generated image were calculated separately. , Based on the preset temperature coefficient The probability that each patch belongs to the real image is calculated. And the probability of belonging to AI-generated images ;

[0016] S3: The AI-generated image detection model is trained using a mixed training dataset to obtain a trained AI-generated image detection model. The loss function for each batch of training data is calculated as follows:

[0017] ,

[0018] in, Represents the normalized coefficients of the unsupervised and supervised terms. The coefficient representing the average entropy term, The cross-entropy loss represents the cross-entropy loss of the labeled training data in the current batch, and is calculated using the following formula:

[0019] ,

[0020] in, This represents the set of labeled training data in the current batch. This indicates the number of labeled training data sets in the current batch. Represents the labeled training data set Training data The Middle The probability that a tile belongs to a real image;

[0021] The regularization term based on tile consistency is calculated as follows:

[0022] ,

[0023] in, This represents the set of training data in the current batch. Indicates an indicator function, Indicates training data based on the current batch The Middle Logarithmic odds of a tile Use another temperature coefficient The calculated probability that the obtained patch belongs to the real image;

[0024] The entropy regularization term based on the graph tiles is calculated using the following formula:

[0025] ,

[0026] in, The average probability of the corresponding category is expressed by the following formula:

[0027] ;

[0028] S4: Input the image to be detected into the trained AI-generated image detection model to obtain its... The probability that a patch belongs to a real image. Then to Probability By performing integration, the probability that the image to be detected belongs to a real image can be obtained. When the probability If the value is greater than a preset threshold, the image to be detected is a real image; otherwise, the image to be detected is an AI-generated image.

[0029] This invention discloses an AI-generated image detection method based on tile consistency. It constructs a hybrid training dataset comprising a labeled training dataset and an unlabeled mixed image set. It also constructs an AI-generated image detection model including an image segmentation module, an image feature extraction module, and a prediction module. The AI-generated image detection model is trained using the hybrid training dataset. The loss function includes a cross-entropy loss calculated based on the labeled training dataset, and regularization terms based on tile consistency and tile entropy calculated based on the hybrid training dataset. The trained AI-generated image detection model is then used to detect images.

[0030] The present invention has the following beneficial effects:

[0031] 1) In this invention, the image is segmented and sampled, which destroys the global semantic structure of the image and forces the AI-generated image detection model to focus on the texture statistical features and local generation traces inside the image patch, thereby achieving semantically independent feature learning and improving the accuracy of the AI-generated image detection model.

[0032] 2) This invention introduces a regularization term based on patch consistency into the loss function calculation. For unlabeled data, it forces multiple patches from the same image to output similar predicted probability distributions. By utilizing the fact that different parts of the same image have the same real / fake attributes, a novel consistency constraint is achieved. This provides an implicit supervision signal for the model in the absence of labels, guiding the model to discover statistical anomalies unique to generated images and improving the performance of AI-generated image detection models.

[0033] 3) This invention introduces a tile-based entropy regularization term in the calculation of the loss function, which can prevent the model from being biased into predicting all unlabeled data as the real category due to only contacting labeled real samples. This forces the AI-generated image detection model to maintain a statistical balance in the prediction between real and fake categories, thereby explicitly exploring the decision boundary between the two types of images and further improving the performance of the AI-generated image detection model.

[0034] 4) This invention employs a mixed training dataset that includes labeled training datasets and unlabeled mixed image datasets. This setup allows AI-generated image detection models to utilize massive amounts of unlabeled data and adapt to emerging new generative models, thereby alleviating the difficulty of collecting data for new generative models and improving the generalization ability of AI-generated image detection models. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a specific implementation of the AI-generated image detection method based on tile consistency according to the present invention.

[0036] Figure 2 This is a structural diagram of the AI-generated image detection model in this invention;

[0037] Figure 3 This is a schematic diagram of image segmentation in this embodiment. Detailed Implementation

[0038] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0039] Example

[0040] Figure 1 This is a flowchart illustrating a specific implementation of the AI-generated image detection method based on tile consistency according to the present invention. Figure 1 As shown, the AI-generated image detection method based on tile consistency of the present invention includes the following steps:

[0041] S101: Constructing a hybrid training dataset:

[0042] Unlike traditional fully supervised learning, which relies on pairs of "real-fake" labeled data, this invention constructs a hybrid training dataset containing two subsets: a labeled training dataset and a hybrid training dataset. and unlabeled mixed image sets Among them, the labeled training dataset It contains several training datasets, each containing a natural image and its corresponding ground truth label; and an unlabeled mixed image set. It contains several real images and AI-generated images, which can be generated by various unknown generation models.

[0043] In practical applications, to improve the training effect of AI-generated image detection models on mixed training datasets, the number of real images and AI-generated images should be balanced in the mixed training dataset, with a preferred setting for the number of real images. And the number of AI-generated images The following conditions must be met:

[0044] .

[0045] S102: Constructing an AI-generated image detection model:

[0046] In order to make better use of the mixed training dataset, this invention constructs a tile-based AI-generated image detection model. Figure 2 This is a structural diagram of the AI-generated image detection model in this invention. (Example) Figure 2 As shown, the AI-generated image detection model in this invention includes an image segmentation module, an image feature extraction module, and a prediction module. Each module will be described in detail below.

[0047] The image segmentation module normalizes the input image to a preset size (e.g., 256×256 pixels), then divides it into several non-overlapping patches, from which samples are obtained. Each image patch is then sent to the image feature extraction module.

[0048] Figure 3 This is a schematic diagram of image segmentation in this embodiment. For example... Figure 2 As shown, image segmentation and patch sampling disrupt the global semantic structure of the image, forcing the model to focus on the texture statistics and local generation traces within each patch. This enables semantically independent feature learning, effectively suppressing the interference of high-level semantic content (such as object category and scene layout) on the anti-spoofing features. Furthermore, this method can also extract implicit label information from labeled training datasets, because patches belonging to the same category or the same image share similarities. Figure 2Image pairs marked with a "+" sign need stronger constraints, while patches belonging to different categories exhibit differences ( Figure 2 Image pairs marked with "-" need to be distinguished by their features. When sampling patches, to ensure the sparsity of patch distribution in the image, the entire image is first divided into a patch array. The map is divided into several partitions, and then a tile is randomly sampled from each partition.

[0049] The feature extraction module is used for... Feature extraction is performed on each image patch, and the obtained features are... Patch features Send to the prediction module, In this embodiment, the feature extraction module adopts the DINO model with the ViT-B / 16 architecture. In order to retain common visual features and adapt to the anti-spoofing task, during the training process of the AI-generated image detection model, it is preferable to set the parameters of the last sub-module of the DINO model to adjustable parameters and freeze other parameters.

[0050] The prediction module is used to predict based on the features of each patch. The logits of the real image and the AI-generated image were calculated separately. , Based on the preset temperature coefficient The probability that each patch belongs to the real image is calculated. And the probability of belonging to AI-generated images In this embodiment, probability The softmax function is used for calculation, and the formula is as follows:

[0051] .

[0052] S103: Training the AI-generated image detection model:

[0053] An AI-generated image detection model is trained using a mixed training dataset to obtain a trained AI-generated image detection model. This invention addresses scenarios with limited labeled data, requiring the extraction and utilization of consistency information from unlabeled data, combined with labeled data to construct model training constraints. To this end, this invention designs a training loss function, whose loss terms include patch-based cross-entropy loss, entropy regularization, and patch consistency regularization terms.

[0054] 1) Patch-based cross-entropy loss

[0055] In this invention, the primary supervisory signal for the AI-generated image detection model comes from the label information in the labeled training dataset. Previous models used labels associated with each image as supervisory signals, which limited the utilization of these labels and made it difficult to leverage patch-based supervisory signals at a fine-grained level. This invention addresses this issue by designing a patch-based cross-entropy loss. The labeled training dataset used in this invention provides patch-based supervision, obtaining supervisory anchors, that is, selecting from each image... The image consists of non-overlapping patches, following the constraint that "all patches within the same image are assigned the same label". Based on this, the patch-based cross-entropy loss is calculated using the following formula. :

[0056] ,

[0057] in, This represents the set of labeled training data in the current batch. This indicates the number of labeled training data sets in the current batch. Represents the labeled training data set Training data The Middle The probability that a patch belongs to a real image can be calculated using the following formula:

[0058] ,

[0059] in, Representing training data The Middle Each image patch is represented by the logits of the real image obtained by the prediction module and the AI-generated image.

[0060] 2) Entropy regularization term based on graph tiles:

[0061] Due to the imbalance of labeled training data in existing scenarios, where the labeled data in the training dataset only contains real images, these real labels provide a stable anchoring effect. Under this premise, this invention employs a batch-level averaging strategy based on image patches, thereby designing an average entropy regularization term based on image patches to achieve statistical flatness in batch prediction. This mechanism can prevent unlabeled data from being over-classified into the real label category, thus forcing some unlabeled data to be assigned to the fake label category. Entropy regularization term. The calculation formula is as follows:

[0062] ,

[0063] in, The average probability of the corresponding category is expressed by the following formula:

[0064] .

[0065] 3) Patch-based consistency regularization term

[0066] The aforementioned average entropy regularization term encourages the model to balance the predicted mean, rather than excessively converging towards the true class. However, inter-patch supervision is equally crucial—this prevents AI-generated image detection models from making erroneous predictions where some patch outputs deviate too much from the overall trend. Therefore, this invention also introduces a patch-based consistency regularization term. To provide additional supervision, the formula is as follows:

[0067] ,

[0068] in, This represents the set of training data in the current batch. Indicates the indicator function. Indicates training data based on the current batch The Middle Logarithmic odds of a tile Use another temperature coefficient The calculated probability that the obtained patch belongs to the real image, temperature coefficient. The value is usually set based on experience. In this embodiment, the probability... The calculation formula can be expressed as follows:

[0069]

[0070] Because the image has been magnified, the differential features are highlighted. Meanwhile, tile segmentation significantly reduces semantic similarity, prompting the model to focus on common representations of the same labels within tiles, as well as invariant differences between binary categories. This consistency regularization term, through latent supervision, guides the model to focus on differential features between tiles. Therefore, regardless of semantic variations, the differences between real and fake images are more easily distinguished.

[0071] Finally, the loss function is calculated using the following formula. :

[0072] ,

[0073] in, This represents the normalization coefficients of the unsupervised and supervised terms, used to balance the weights of the two types of losses in the overall objective function, preventing one type of loss from having an excessively dominant influence on model training. The coefficient of the average entropy term.

[0074] S104: AI-generated image detection:

[0075] The image to be detected is input into the trained AI-generated image detection model to obtain its... The probability that a patch belongs to a real image. Then to Probability By performing integration, the probability that the image to be detected belongs to a real image can be obtained. When the probability If the value is greater than a preset threshold, the image to be detected is a real image; otherwise, the image to be detected is an AI-generated image.

[0076] In this embodiment, probability The calculation formula is as follows:

[0077] .

[0078] To better illustrate the technical solution of the present invention, specific examples are used to experimentally verify the present invention.

[0079] For comparison with the present invention, 10 comparison methods were selected in this embodiment. Table 1 is a comparison method information table in this embodiment.

[0080]

[0081] Table 1

[0082] In this embodiment, the detection performance of the present invention and 10 comparative methods were compared on 9 different generative models, including detection accuracy (ACC), generalization ability, and inference performance. Table 2 is a comparison table of the detection accuracy of the present invention and the comparative methods in this embodiment.

[0083]

[0084] Table 2

[0085] As shown in Table 2, across all image categories, the present invention achieves a performance improvement of at least 3.7% compared to all comparison methods. Furthermore, the DRCT method exhibits similar performance to the NPR method, particularly on images generated by the StableDiffusionV1.4, StableDiffusionV1.5, and Wukong models. This is because both methods were trained on data generated by StableDiffusionV1.4, enabling them to capture the discriminative features corresponding to similar generation techniques. However, their performance drops sharply when tested on generation models that differ significantly from the training data. For example, on images generated by BigGAN, the detection accuracy (Acc) of both methods drops to 78%; while on images generated by the VAR model, the accuracy of the DRCT method further plummets to 57%, reflecting the method's poor transferability. Other transfer learning-based methods also exhibit similar limitations: Fatformer and UniFD are both trained on data generated by ProGAN models, thus performing well on images generated by BigGAN, but poorly on images generated by diffusion models (such as StableDiffusionV1.4). This reveals a key drawback of transfer learning-based methods: although these methods aim to construct general decision boundaries from training data, the variety of generative models in the real world is extremely diverse, and limited labeled training data cannot cover all types of generative models, thus limiting their generalization ability.

[0086] As two training-free methods, AEROBLADE and Brokman2025 exhibit significant detection bias. For example, AEROBLADE achieves approximately 80% accuracy (Acc) on images generated by the Glide model, but its accuracy drops sharply to 51% on images generated by the VQDM model. Meanwhile, Brokman2025 achieves nearly 87% accuracy on images generated by the BigGAN model, but its accuracy plummets to 43% on images generated by the Midjourney model. This significant performance fluctuation indicates that although training-free methods can construct decision boundaries, the distance between real and fake data varies greatly across different generative models. This stems from the inherent limitations of training-free methods—the lack of a unified, explicitly modeled boundary, making them unable to adapt to the diversity of generative models. Therefore, these methods may perform exceptionally well on one generative model but poorly on another, highlighting their performance instability.

[0087] This invention demonstrates strong generalization ability. To further verify this, experiments were conducted on unseen data generated by the selected model. Table 3 compares the detection accuracy of this invention and the comparative method on unseen data in this embodiment.

[0088]

[0089] Table 3

[0090] In Table 3, "Fake / Real (AI-generated / Real): 5 / 2" and "Fake / Real: 2 / 5" represent different ratios of real data and fake data in the training set. "Unseen data" refers to data from the corresponding generative model that has not been previously encountered during the testing phase. As shown in Table 3, the present invention still exhibits excellent performance on unseen data. Furthermore, even with imbalanced data, the regularization constraints of the loss design can still be effective in general data distribution scenarios.

[0091] To promote practical applications, this embodiment also compares the inference performance differences between the proposed method and other methods. Four metrics are used to quantify performance: floating-point operations (FLOPs), average inference time (AIT), maximum memory usage (MMU), and number of parameters (Param). Table 4 is a comparison table of the inference performance of the proposed method and the comparative method in this embodiment.

[0092]

[0093] Table 4

[0094] As shown in Table 4, this invention is competitive with other transfer learning-based methods in both efficiency and effectiveness. Although NPR's average inference time (AIT) is 3 milliseconds shorter than this invention, the invention has a greater overall performance advantage. Notably, compared to the high-performance DRCT, the computation time and memory load of this invention are only one-third. Therefore, this invention achieves a good balance between efficiency and effectiveness.

[0095] In summary, compared with existing technical solutions, this invention has significant substantive features and advancements. It outperforms other methods of the same period in terms of classification accuracy (ACC), robustness, generalization ability, and inference cost.

[0096] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. An AI-generated image detection method based on tile consistency, characterized in that, Includes the following steps: S1: Construct a hybrid training dataset, including a labeled training dataset. and unlabeled mixed image sets Among them, the labeled training dataset It contains several training datasets, each containing a natural image and its corresponding ground truth label; and an unlabeled mixed image set. It contains several real images and AI-generated images; S2: Construct an AI-generated image detection model, including an image segmentation module, an image feature extraction module, and a prediction module, where: The image segmentation module normalizes the input image to a preset size, then divides it into several non-overlapping patches, from which samples are obtained. Each image patch is then sent to the image feature extraction module; The image feature extraction module is used for... Feature extraction is performed on each image patch, and the obtained features are... Patch features Send to the prediction module, ; The prediction module is used to predict based on the features of each patch. The log odds of the real image and the AI-generated image were calculated separately. , Based on the preset temperature coefficient The probability that each patch belongs to the real image is calculated. And the probability of belonging to AI-generated images ; S3: The AI-generated image detection model is trained using a mixed training dataset to obtain a trained AI-generated image detection model. The loss function for each batch of training data is calculated as follows: ， in, Represents the normalized coefficients of the unsupervised and supervised terms. The coefficient representing the average entropy term, The cross-entropy loss of the labeled training data in the current batch is calculated using the following formula: ， in, This represents the set of labeled training data in the current batch. This indicates the number of labeled training data sets in the current batch. Represents the labeled training data set Training data The Middle The probability that a tile belongs to a real image; The regularization term based on tile consistency is calculated as follows: ， in, This represents the set of training data in the current batch. Indicates an indicator function, Indicates training data based on the current batch The Middle Logarithmic odds of a tile Use another temperature coefficient The calculated probability that the obtained patch belongs to the real image; The entropy regularization term based on the graph tiles is calculated using the following formula: ， in, The average probability of the corresponding category is expressed by the following formula: ； S4: Input the image to be detected into the trained AI-generated image detection model to obtain its... The probability that a patch belongs to a real image. Then to Probability By performing integration, the probability that the image to be detected belongs to a real image can be obtained. When the probability If the value is greater than a preset threshold, the image to be detected is a real image; otherwise, the image to be detected is an AI-generated image.

2. The AI-generated image detection method according to claim 1, characterized in that, The number of real images in the hybrid training dataset And the number of AI-generated images The following conditions must be met: 。 3. The AI-generated image detection method according to claim 1, characterized in that, The image segmentation module uses a patch sampling method that divides the entire image into patch arrays. The map is divided into several partitions, and then a tile is randomly sampled from each partition.

4. The AI-generated image detection method according to claim 1, characterized in that, The feature extraction module adopts the DINO model with the ViT-B / 16 architecture.

5. The AI-generated image detection method according to claim 4, characterized in that, During the training process of the AI-generated image detection model, the parameters of the last sub-module of the DINO model are set to adjustable parameters, while other parameters are frozen.

6. The AI-generated image detection method according to claim 1, characterized in that, The features in step S4 The calculation formula is as follows: 。

Images