AI-generated image detection method and system based on model affinity and task difficulty calibration

Abstract

The application discloses an AI generated image detection method and system based on model affinity and task difficulty calibration. The method comprises: receiving an image to be detected; extracting model affinity features, which include image restoration accuracy and semantic segmentation confidence; extracting task difficulty features, which include pixel naturalness and image complexity; calculating a task difficulty calibration factor based on the task difficulty features; combining the image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor to generate a comprehensive feature vector; and inputting the comprehensive feature vector into a pre-trained machine learning classifier to obtain a detection result. By introducing a task difficulty calibration mechanism, the application overcomes the defects of poor generalization ability, high false positive rate, and the like caused by excessive reliance on visual artifacts or model fingerprints in existing methods, and significantly improves the accuracy and robustness of AI generated image detection.

Description

Technical Field

[0001] This invention belongs to the fields of AI-generated content detection and computer vision technology, and specifically proposes an AI-generated image detection method that integrates the model affinity between the image and the visual processing model, as well as the task difficulty of the image itself. Background Technology

[0002] The relevant technical background is as follows: 1. AI-generated image detection: In recent years, generative adversarial networks, diffusion models, and text-to-image generation models have developed rapidly, enabling the generation of highly realistic images that are difficult to distinguish from the naked eye. While such images have value in content creation, they can also be used in scenarios such as fake news, identity impersonation, and fabrication of fraudulent materials, thus posing new challenges to content authenticity verification, cybersecurity, and the detection of AI-generated content. Therefore, how to automatically and reliably distinguish between real images and AI-generated images has become an important technical problem in the fields of computer vision and AI-generated content detection.

[0003] Currently, several methods have been proposed for the automated detection of AI-generated images. The most direct approach involves manually analyzing the network structure, upsampling method, generation process, or post-processing mechanism of the generative model to extract potential visual artifacts, frequency domain anomalies, or model fingerprints from the generated images. Based on these features, detection methods are designed to determine the image's origin. Visual artifacts refer to abnormal textures, boundary anomalies, or spectral anomalies left by the generative model. Model fingerprints refer to the relatively stable features formed by a certain type of generative model due to its network structure, training method, or generation mechanism. Another approach relies on the automated feature engineering capabilities of deep neural networks: collecting a large number of real and AI-generated images, using this data to train a deep neural network-based classifier, and then using the resulting classifier to detect AI-generated images.

[0004] The methods described above improve the detection performance of AI-generated images to some extent, but limitations remain. On one hand, with the continuous development of generative models and post-processing techniques, the low-level artifacts, frequency domain anomalies, or specific model fingerprints relied upon by earlier methods may be weakened, resulting in insufficient generalization ability of the detection methods on novel generative models or high-quality generated images. On the other hand, some methods primarily rely on classifier output, model confidence, or single statistical features for judgment, without fully considering the impact of the image's inherent content complexity on the detection results. In reality, simple, clear-boundary, and low-texture real-world images are easily processed by visual models and may be misclassified as AI-generated images. Conversely, AI-generated images with complex scenes, dense textures, or irregular local structures may be missed as real images due to their higher processing difficulty.

[0005] Therefore, the industry needs an AI-generated image detection method that can simultaneously consider the image's performance in external visual models, as well as the image's naturalness and complexity. This method should not rely solely on image artifacts, model fingerprints, or absolute classification outputs, but should combine the image's model performance with the task difficulty, so that the detection results better reflect whether the image was synthesized by a generative model, rather than being determined solely by the simplicity or complexity of the image content. Summary of the Invention

[0006] This invention aims to address the technical problems of existing detection methods, which suffer from insufficient generalization ability, low accuracy in recognizing high-quality generated content, high false positive rates, and severe false negatives in complex texture scenes due to their reliance on subtle visual artifacts or model-specific fingerprints. To this end, this invention proposes an AI-generated image detection method and system based on model affinity and task difficulty calibration. It simultaneously extracts and fuses model affinity features and task difficulty features to construct a two-dimensional calibration system for semantic segmentation confidence and image restoration accuracy. Furthermore, it introduces pixel naturalness analysis and image dynamic complexity assessment to quantify the essential differences in visual task performance consistency between real images and AI-generated images, thereby significantly improving the detection's generalization ability and accuracy, and effectively reducing false positives.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, this invention proposes an AI-generated image detection method based on model affinity and task difficulty calibration, comprising the following steps: Receive the image to be detected; The image to be detected is processed to extract model affinity features, which include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected. The image to be detected is processed to extract task difficulty features, which include pixel naturalness and image complexity obtained based on the image to be detected. Based on the task difficulty characteristics, calculate the task difficulty calibration factor used to calibrate the model affinity characteristics; The image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor are combined to generate a comprehensive feature vector characterizing the image to be detected. The integrated feature vector is input into a pre-trained machine learning classifier to obtain a detection result indicating whether the image to be detected is a real image or an AI-generated image.

[0008] Furthermore, the image restoration accuracy is obtained through the following method: An occlusion region is set in the image to be detected to obtain an occlusion image; The occluded image is input into the image restoration model to obtain the restored image of the occluded area; Within the occluded area, a structural similarity index between the image to be detected and the image to be repaired is calculated as the image repair accuracy.

[0009] Furthermore, the semantic segmentation confidence score is obtained in the following way: The image to be detected is input into the semantic segmentation model to obtain a probability vector of each pixel in the image belonging to multiple semantic categories; For each pixel, the maximum class probability in its probability vector is taken as the prediction confidence of that pixel; The semantic segmentation confidence is obtained by averaging the prediction confidence of all pixels in the image.

[0010] Furthermore, the pixel naturalness is obtained in the following way: For each pixel in the image to be detected, the conditional prediction probability of that pixel is calculated based on its global context and local context; The naturalness of a pixel is calculated based on the conditional prediction probabilities of all pixels.

[0011] Furthermore, the image complexity is calculated by integrating one or more of the following features: edge density, gradient magnitude, Laplacian variance, Shannon entropy, color richness, and JPEG peak signal-to-noise ratio.

[0012] Furthermore, the task difficulty calibration factor Based on the pixel naturalness and the image complexity The calculation is as follows: , where α and β are preset weighting coefficients.

[0013] Furthermore, the combination of the image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor can be achieved in one of the following two ways: Method 1: Improve the accuracy of image restoration The semantic segmentation confidence Each with the task difficulty calibration factor The weighted combination is performed to generate the comprehensive feature vector. T ; Method 2: Improve the accuracy of image restoration The semantic segmentation confidence The pixel naturalness and the image complexity Concatenate into a four-dimensional feature vector , which serves as the comprehensive feature vector.

[0014] Furthermore, after obtaining the detection results, an anti-forensic enhancement step is also included: When the image to be detected is detected as an AI-generated image, a dynamic task detection mechanism is activated. The dynamic task detection mechanism includes replacing or supplementing the semantic segmentation task with a monocular depth estimation or surface normal prediction task, and / or replacing or supplementing the image restoration task with an image denoising or super-resolution reconstruction task, so as to update the extraction method of the model affinity features.

[0015] Secondly, an AI-generated image detection system based on model affinity and task difficulty calibration implements the method described in any one of the above embodiments, including: The feature extraction module is used to receive the image to be detected and process the image to be detected to extract model affinity features and task difficulty features; wherein, the model affinity features include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected, and the task difficulty features include pixel naturalness and image complexity obtained based on the image to be detected. The calibration factor calculation module is used to calculate the task difficulty calibration factor for calibrating the model affinity feature based on the task difficulty feature. The feature combination module is used to combine the image restoration accuracy, the semantic segmentation confidence and the task difficulty calibration factor to generate a comprehensive feature vector that characterizes the image to be detected. The image detection module is used to input the comprehensive feature vector into a pre-trained machine learning classifier to obtain the detection result of whether the image to be detected is a real image or an AI-generated image.

[0016] Compared with existing technologies, it has the following beneficial technical effects: 1. By extracting model affinity features (image restoration accuracy and semantic segmentation confidence) and task difficulty features (pixel naturalness and image complexity) in parallel, a two-dimensional feature representation system was constructed, which overcomes the limitation of existing technologies that rely solely on single visual artifacts or model fingerprints for detection, and significantly improves the generalization ability of the detection method.

[0017] 2. An innovative task difficulty calibration mechanism is introduced. By calculating a calibration factor based on task difficulty features, the model affinity features are calibrated, eliminating the interference of the complexity of the image content itself on the detection results. This effectively avoids the problem of simple real images being misjudged as AI-generated images and complex AI-generated images being missed as real images, and significantly reduces the false alarm rate and false negative rate.

[0018] 3. By inputting the calibrated feature vectors into a machine learning classifier for discrimination, automated and high-precision differentiation between AI-generated images and real images is achieved. Experimental data shows that the detection accuracy of the method of this invention reaches 90.22%, the AUC reaches 96.24%, and the false alarm rate is only 11.05%, which is superior to the existing mainstream detection methods in terms of overall performance.

[0019] In summary, this invention significantly reduces the false positive rate while maintaining a high recall rate, and possesses excellent detection accuracy and robustness. It is suitable for robust AI-generated image authenticity identification tasks in large-scale, multi-scenario environments. Attached Figure Description

[0020] Figure 1 This is an overall flowchart of the AI-generated image detection method based on model affinity and task difficulty calibration according to Embodiment 1 of the present invention.

[0021] Figure 2 This is a block diagram of the AI-generated image detection system based on model affinity and task difficulty calibration according to Embodiment 2 of the present invention.

[0022] Figure 3 This is the overall technical roadmap for the AI-generated image detection method based on model affinity and task difficulty calibration described in this invention. Detailed Implementation

[0023] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0024] Example 1: As Figure 1 As shown, the AI-generated image detection method based on model affinity and task difficulty calibration of the present invention includes the following steps: Step 1: Receive the image to be detected, as described below: Image input and initialization: The system receives the image to be detected, I. In this embodiment, the image to be detected may originate from social media, online photo albums, or specific acquisition devices. Its sources are wide-ranging and its content diverse, and may include simple landscape photos, complex urban street scenes, or portraits with fine textures.

[0025] Step 2: Process the image to be detected to extract model affinity features. The model affinity features include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected, as described in detail below: Model affinity feature extraction: To quantify the interaction characteristics between AI-generated images and existing visual models, the system processes the image to be detected and extracts model affinity features. Specifically, this step includes two parallel sub-processes: Semantic segmentation confidence extraction: The image to be detected is input into a pre-trained semantic segmentation model (such as DeepLabV3+ or HRNet). The pixel-level classification labels and their corresponding confidence scores are extracted from the model output. Since AI-generated images often have logical ambiguity or physical inconsistencies at semantic boundaries, real images usually achieve higher and more stable average confidence scores.

[0026] Image inpainting accuracy evaluation: A portion of the image to be detected is randomly occluded (e.g., a 10% rectangular mask) manually or using algorithms. Then, a pre-trained image inpainting network (e.g., LaMa or a contextual attention model) is used to fill in the occluded area. The image inpainting accuracy is determined by calculating the pixel-level difference (e.g., L1 norm or SSIM) between the inpainted image and the original unoccluded image. Due to the continuity of their latent space, AI-generated images often exhibit statistical patterns different from those of real images during local inpainting.

[0027] Step 3: Process the image to be detected to extract task difficulty features. The task difficulty features include pixel naturalness and image complexity obtained based on the image to be detected, as described in detail below: Task difficulty feature extraction: Considering that the visual complexity of the image itself can affect the reliability of the aforementioned model affinity features, this embodiment also extracts task difficulty features to characterize the intrinsic properties of the image. Specifically, these include: Pixel naturalness calculation: Using a pre-trained anomaly detection network or autoencoder, the naturalness of the image's pixel distribution is evaluated. This feature reflects whether the image has unnatural smooth transitions or anomalous noise distribution.

[0028] Image complexity calculation: The complexity of an image is quantified by analyzing its edge density, texture richness, or frequency domain energy distribution. For example, a forest image containing a lot of detail has higher complexity than a solid color block image.

[0029] Step 4: Based on the task difficulty features, calculate the task difficulty calibration factor used to calibrate the model affinity features, as described below: Generation of Task Difficulty Calibration Factor: Based on the task difficulty features (pixel naturalness N(I) and image complexity C(I)) extracted in step 3, the system calculates the task difficulty calibration factor η. In practical applications, to prevent a single feature from dominating the calibration process, a multivariate linear combination approach is used for fusion. The specific calculation formula is as follows:

[0030] in, and These are preset weighting coefficients. In this embodiment, the weighting coefficients can be optimized and determined through grid search based on the performance on the validation set, with the aim of enabling the calibration factor to distinguish feature fluctuations caused by image difficulty to the greatest extent.

[0031] Step 5: Combine the image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor to generate a comprehensive feature vector characterizing the image to be detected, as described below: Construction of the multidimensional comprehensive feature vector: This step is crucial for feature fusion. The system combines the objective metrics obtained in step 2 (image restoration accuracy and semantic segmentation confidence) with the dynamic calibration factors generated in step 4. Specifically, the values ​​of these three dimensions are concatenated or weighted to generate a high-dimensional comprehensive feature vector F. In this way, the feature vector not only contains the image's response to the model but also embeds difficulty information after eliminating content interference.

[0032] Step 6: Input the comprehensive feature vector into a pre-trained machine learning classifier to obtain the detection result of whether the image to be detected is a real image or an AI-generated image, as described in detail below: Classification and Output: Finally, the generated comprehensive feature vector F is input into a machine learning classifier (such as Support Vector Machine (SVM), Random Forest, or Fully Connected Neural Network) pre-trained on a large-scale dataset. The classifier performs a binary classification task by learning the distribution differences between real images and AI-generated images in the joint space of visual model performance and image task difficulty. The classifier outputs the final detection result, clearly identifying whether the image to be detected is a real image or an AI-generated image, and can output the confidence score in the form of a probability value.

[0033] Example 2: As Figure 2 The figure illustrates the AI-generated image detection system based on model affinity and task difficulty calibration of the present invention. The system includes a feature extraction module 100, a calibration factor calculation module 200, a feature combination module 300, and an image detection module 400. This system is used to implement the AI-generated image detection method based on model affinity and task difficulty calibration of Embodiment 1 described above. Wherein: The feature extraction module 100, connected to the calibration factor calculation module 200, is used to receive the image to be detected and process the image to be detected to extract model affinity features and task difficulty features; wherein, the model affinity features include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected, and the task difficulty features include pixel naturalness and image complexity obtained based on the image to be detected. The calibration factor calculation module 200, connected to the feature combination module 300, is used to calculate the task difficulty calibration factor for calibrating the model affinity feature based on the task difficulty feature. The feature combination module 300, connected to the image detection module 400, is used to combine the image restoration accuracy, the semantic segmentation confidence and the task difficulty calibration factor to generate a comprehensive feature vector characterizing the image to be detected. The image detection module 400 is used to input the comprehensive feature vector into a pre-trained machine learning classifier to obtain a detection result indicating whether the image to be detected is a real image or an AI-generated image.

[0034] Figure 3 shows the overall technical roadmap of the AI-generated image detection method based on model affinity and task difficulty calibration described in this invention. This technical roadmap adopts a dual-path detection framework, extracting model affinity features and task difficulty features from the same input image, and then fusing the two and inputting them into a classifier to determine whether the image is a real image or an AI-generated image.

[0035] Specifically, the model affinity path characterizes the ease with which the image to be detected is processed in an external visual model. This method calculates image restoration accuracy through an image restoration task and segmentation confidence through a semantic segmentation task to reflect whether the image exhibits predictable structural features. The task difficulty path characterizes the inherent difficulty the image itself poses to the visual task. This method reflects the predictability of image pixels through pixel naturalness and the complexity of image structure, texture, and semantic content through image complexity. During the training phase, this method extracts the above four types of features from real images with known labels and AI-generated images, concatenates them into a feature vector, and uses this vector to train the classifier. During the detection phase, the same feature extraction and fusion process is performed on unknown images, and the trained classifier outputs the detection results.

[0036] When the image to be detected is detected as an AI-generated image, a dynamic task detection mechanism is activated. The dynamic task detection mechanism includes replacing or supplementing the semantic segmentation task with a monocular depth estimation or surface normal prediction task, and / or replacing or supplementing the image restoration task with an image denoising or super-resolution reconstruction task, so as to update the extraction method of the model affinity features.

[0037] The innovation of this method lies in its departure from relying solely on image artifacts, specific model fingerprints, or single model outputs for judgment. Instead, it considers the relationship between the image's performance in the visual model and the task difficulty of the image itself. Therefore, this method reduces the risk of simple real-world images being misclassified as AI-generated images and complex AI-generated images being missed as real images, thereby improving the accuracy and robustness of AI-generated image detection.

[0038] This invention mainly includes seven technical steps: (1) image restoration accuracy calculation, (2) semantic segmentation confidence calculation, (3) pixel naturalness calculation, (4) image complexity calculation, (5) classifier training, (6) model-based image detection, and (7) adaptive attack and anti-forensics enhancement mechanism. Each step is described in detail below.

[0039] (1) Image restoration accuracy calculation: Image restoration accuracy is the first type of metric feature in the model affinity path. It quantifies the structural consistency of the image to be detected by measuring the accuracy of the image restoration model in reconstructing the occluded region. The inputs to this step are the image to be detected I and the occluded region M. First, the occluded region M is set in image I to obtain the occluded image. Then, the occluded image is input into the image restoration model, which generates the restored image Î based on the visible content around the occluded region. Subsequently, the difference between the original image I and the restored image Î is compared only within the occluded region M, and the structural similarity index SSIM is used to calculate the image restoration accuracy. Specifically, it is expressed as:

[0040]

[0041] in, Indicates the accuracy of image restoration. This represents the actual content of the original image I within the occluded region M. This indicates the reconstructed content of the repaired image Î within the occluded region M. and These represent the original pixel matrix. and reconstructing pixel matrix The average pixel value, and Let the variances of the two matrices above be represented respectively. Represents the original pixel matrix and reconstructing pixel matrix Covariance between and This represents a preset constant term to prevent the denominator from being zero and to maintain computational stability.

[0042] The method and system of this invention (FakeImageScan) employs a pluralistic inpainting model to generate repair results for occluded regions. This model can generate multiple reasonable repair results with consistent structure based on context, making it suitable for examining whether the local structure of an image under test is easily repairable. This step outputs the image repair accuracy. . The higher the value, the higher the model affinity of the image to be detected in the image inpainting task.

[0043] (2) Semantic segmentation confidence calculation: Semantic segmentation confidence is the second type of metric feature in the model affinity path, which is used to quantify the degree of certainty of the semantic segmentation model in predicting the pixel-level category in the image to be detected.

[0044] In this step, after inputting image I into the semantic segmentation model, the model outputs a semantic category probability vector for each pixel in the image. For example, the probability vector of the i-th pixel can be represented as: .

[0045] For each pixel, this method takes the maximum class probability in its probability vector as the prediction confidence of that pixel, and then averages the maximum class probabilities of all N pixels in the image to obtain the semantic segmentation confidence of the entire image. The calculation formula is as follows:

[0046] in, This represents the semantic segmentation confidence of image I. This represents the total number of pixels involved in the calculation. Indicates the total number of semantic categories. This represents the probability that the i-th pixel belongs to the c-th semantic category.

[0047] FakeImageScan uses the PSPNet model to extract semantic segmentation features and leverages its pyramid pooling module to integrate global context from different sub-regions. By aggregating these representations, the model provides a comprehensive understanding of the entire image scene. This step outputs the semantic segmentation confidence score. . The higher the value, the higher the model affinity of the image in the semantic segmentation task.

[0048] (3) Pixel naturalness calculation: Pixel naturalness is the first type of metric feature in the task difficulty path, used to quantify the inherent regularity and predictability of the input image at the pixel level. This feature is used to describe how easy or difficult it is to predict the current pixel value given a visual context.

[0049] The input for this step is the image I to be detected. For the current pixel in the image... This method is based on its preceding pixels or context pixels. Calculate the conditional prediction probability for this pixel. This conditional prediction probability is obtained by weighting the global context probability and the local context probability, as shown in the following formula:

[0050] in, This represents the global context probability, which is obtained from a large receptive field or long-range contextual dependencies. λ represents the local context probability, which is obtained from the local neighborhood of the current pixel, and λ represents the weight coefficient, which is used to control the contribution ratio of global context and local context in the pixel naturalness calculation.

[0051] The method and system of this invention, FakeImageScan, uses a super-resolution-based compression model (SReC) to calculate global and local context probabilities, and outputs pixel naturalness. . It is used to characterize the statistical consistency and predictability of image pixel patterns and serves as one of the features of task difficulty.

[0052] (4) Image complexity calculation: Image complexity is the second type of metric in the task difficulty path. This metric reflects the structural richness, detail density and internal variation of the image. High-complexity images are characterized by rich textures, sharp transitions and diverse visual structures, while low-complexity images are mainly composed of uniform and homogeneous regions. Therefore, images with higher complexity often lead to increased image segmentation difficulty, thus it is used as a quantitative indicator of image segmentation difficulty.

[0053] To comprehensively capture image complexity, FakeImageScan integrates a variety of complementary features: 1) edge density, 2) gradient magnitude, 3) Laplacian variance, 4) Shannon entropy, 5) color richness, and 6) JPEG PSNR.

[0054] Before combination, all features are standardized using z-score to ensure fair contributions across different scales. The final complexity index... The calculation is performed by summing the positively correlated complexity contributors and subtracting the JPEGPSNR features that are negatively correlated with complexity:

[0055] in, This represents the complexity value of image I. This represents the set of features that are positively correlated with complexity. Represents a set The standardized value of the k-th feature. This represents the normalized value of the JPEG PSNR.

[0056] This step employs a combination of complexity features for comprehensive calculation, outputting the image complexity. . It is used to reflect the complexity of the structure and content of an image and serves as one of the features of task difficulty.

[0057] (5) Classifier training: This step is used to integrate the above four types of features into a unified image representation, and to train a machine learning classifier based on this image representation to distinguish between real images and AI-generated images.

[0058] The input for this step is a training image set with category labels. For any image I in the training set, the image restoration accuracy is calculated according to the steps described above. Semantic segmentation confidence Pixel naturalness and image complexity .in, and This represents the affinity feature of the model. and This indicates the difficulty level of the task.

[0059] Subsequently, the four types of features are concatenated in order to form a four-dimensional feature vector:

[0060] After obtaining the feature vectors of the training images, the feature vectors are... and its category tags A Support Vector Machine (SVM) classifier is input for supervised training. The SVM is used to learn the maximum margin discrimination boundary between real and AI-generated images in a four-dimensional feature space. This method implements the SVM classifier using the machine learning toolkit (scikit-learn) and optimizes the model's hyperparameters through cross-validation.

[0061] In the actual detection process, the classifier receives the feature vector of the image to be tested. And through the following decision function Calculate and output the final authenticity determination result:

[0062] in, Represents support vectors, This indicates the distinction between real and AI-generated category labels. Represents the Lagrange multipliers. This represents the square of the Euclidean distance. Indicates the parameter for adjusting the flexibility of the decision boundary. This indicates the bias term.

[0063] (6) Model-based image detection: The model-based image detection stage utilizes a dual-path architecture to process the input image. To perform the final binary classification, first, the input image... The model's affinity path and task difficulty path are input simultaneously, and the image restoration accuracy is extracted through external generative models and statistical models. Semantic segmentation confidence Pixel naturalness and image complexity Four scalar features are used to quantify the structural predictability and intrinsic complexity of the image. Next, the extracted four scalar features are combined and standardized to construct a four-dimensional feature vector. This serves as the standard input data for subsequent classifiers. Finally, the feature vectors are... The input is used in a pre-trained support vector machine classifier. The classifier is based on a decision function. According to the feature vector The final class label is output at the position relative to the maximum margin discrimination boundary in the feature space. .

[0064] (7) Adaptive attack and anti-forensic enhancement mechanism: This step is used to improve the robustness of this method against adaptive attacks. In order to reduce the probability of the image being detected, attackers may deliberately introduce structural perturbations, boundary irregularities or texture changes to reduce the segmentation confidence and image restoration accuracy of the generated image.

[0065] To address this issue, this method can employ a dynamic task detection mechanism, which involves periodically replacing or expanding downstream visual tasks. For example, semantic segmentation tasks can be replaced or supplemented with tasks such as monocular depth estimation and surface normal prediction, while image inpainting tasks can be replaced or supplemented with tasks such as image denoising and super-resolution reconstruction.

[0066] Meanwhile, the difficulty of the task path can serve as a consistency verification mechanism. For real images, model performance usually corresponds to the complexity of the image itself. If an attacker artificially reduces the model affinity for low-complexity images, it may cause the anomaly that "the image itself is relatively simple, but the model performance is abnormally low." This anomaly can also serve as a detection clue.

[0067] like Figure 3The diagram illustrates a flowchart of an AI-generated image detection method based on model affinity and task difficulty calibration provided by an exemplary embodiment of the present invention. The diagram summarizes the core processing flow of the invention, including input data, parallel dual-path feature extraction (model affinity path and task difficulty path), feature fusion and calibration, and the final classification decision process.

[0068] In summary, this invention provides an AI-generated image detection method and system based on dual-path feature extraction and calibration. First, two types of features are extracted in parallel from the image to be detected: one type is model affinity features reflecting the image's response to an external visual model (mainly including semantic segmentation confidence and image restoration accuracy); the other type is task difficulty features characterizing the image's inherent attributes (mainly including pixel naturalness and image complexity). Then, a calibration factor is calculated based on the task difficulty features to calibrate the model affinity features, eliminating interference from the complexity of the image content itself on model affinity judgment. Finally, the calibrated features are combined into multi-dimensional discriminative features, and a machine learning classifier is used to learn the difference between real images and AI-generated images in the relationship between "visual model performance and image task difficulty," achieving robust discrimination of image authenticity (AI-generated or real). This significantly enhances the ability to recognize high-quality AI-generated images and effectively reduces misjudgments of clear, simple, and real images.

[0069] To verify the effectiveness of the method proposed in this patent, specific experimental results are presented below. In the experiments, this method was tested on publicly available benchmark datasets containing both real and AI-generated images. Real images were sourced from datasets such as COCO, LSUN, ImageNet, RAISE, FFHQ, and LAION, while AI-generated images were sourced from generative models such as BigGAN, DALL·E, DeepFloyd-IF, DiT, SDXL, Stable Diffusion, and StyleGAN. During the experiments, low-resolution images were excluded from the test data, and a balanced dataset with an equal number of real and AI-generated images was constructed.

[0070] To verify the effectiveness of our proposed method in AI-generated image detection, we selected representative detection methods as comparison methods, including Universal Fake Image Detection, ZED, and LOTA. These methods represent different detection approaches, such as those based on pre-trained features, zero-shot statistical modeling, and low bit-plane noise features. For each image in the test set, our proposed method and the aforementioned baseline methods were used for detection. We recorded and compared the specific values ​​of four core evaluation metrics, including: Accuracy% (the proportion of images correctly identified by the detection method); TPR% (the proportion of AI images successfully identified by the detection method among all real AI-generated images, reflecting its ability to avoid false negatives); False Alarm Rate% (the proportion of real images incorrectly identified as AI-generated images); and Overall Detection Performance% (AUC%), which measures the probability that the detection method can accurately detect the AI ​​image when randomly presented with a real image and an AI-generated image.

[0071] Table 1 shows a performance comparison between the method and system of this invention (FakeImageScan) and different detection methods. Table 1 presents a comparison of the detection performance of this method and the comparative methods on the test dataset. As can be seen from the table, this method achieves better detection results across all four metrics. Specifically, the accuracy of this method reaches 90.22%, higher than Universal Fake Image Detection's 70.66%, ZED's 71.73%, and LOTA's 75.13%. The recall rate of this method reaches 92.03%, indicating its effective ability to identify AI-generated images. The false positive rate of this method is 11.05%, significantly lower than Universal Fake Image Detection's 29.73%, ZED's 26.74%, and LOTA's 41.62%, strongly demonstrating that our method can reduce the probability of misclassifying real images as AI-generated images. Furthermore, the AUC of this method reaches 96.24%, also higher than all the comparative methods, indicating stronger overall discriminative ability.

[0072] To further verify whether all four types of features in this method have a positive effect on the detection performance, we also conducted ablation experiments. The ablation experiments were conducted by removing image restoration accuracy, semantic segmentation confidence, pixel naturalness, image complexity, or the entire task difficulty path while keeping other experimental settings unchanged, and then retraining and testing the classifier.

[0073] Table 2 shows the experimental results of the impact of features on performance in the ablation experiment. As can be seen from Table 2, the accuracy and AUC of this method both decreased after removing any feature. Specifically, removing image restoration accuracy reduced the AUC from 96.24% to 92.17%. Removing semantic segmentation confidence reduced the AUC to 92.32%. This indicates that image restoration accuracy and semantic segmentation confidence can effectively reflect the model affinity of the image in the external visual model. Removing pixel naturalness reduced the AUC to 93.87%. Removing image complexity reduced the AUC to 94.15%. This shows that pixel naturalness and image complexity can provide task difficulty calibration for model performance. In particular, removing the entire task difficulty path reduced the AUC to 89.17%, further demonstrating that relying solely on model affinity features is insufficient for stable detection; it is necessary to combine this with the image's inherent difficulty for judgment.

[0074] Furthermore, to verify whether the four-dimensional features constructed by this method depend on a specific classifier, the experiment also compared the impact of different classifiers on detection performance. Table 3 shows a comparison of the detection performance of different classifiers. Experimental results show that classifiers such as Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Logistic Regression (LR), Random Forest (RFR), and Gradient Boosting can all achieve good detection results based on the features extracted by this method. Specifically, SVM achieved an accuracy of 90.22% and an AUC of 96.24%, while KNN achieved an accuracy of 89.91% and an AUC of 96.01%. This result indicates that the main contribution of this method lies in the feature construction method of model affinity plus task difficulty, rather than relying on a specific classifier.

[0075] In summary, the experimental results show that this method constructs four-dimensional features by considering image restoration accuracy, semantic segmentation confidence, pixel naturalness, and image complexity. It also utilizes a classifier to learn the difference between real images and AI-generated images in terms of the relationship between "model performance and task difficulty," which can effectively improve the accuracy of AI-generated image detection and reduce the misjudgment rate of real images.

[0076] Table 1

[0077] Table 2

[0078] Table 3

[0079] The above description is merely an embodiment of the present invention and is not intended to limit the method proposed in this invention. The scope of protection of this invention is defined by the claims. Without departing from the spirit and scope of this invention, those skilled in the art will make adjustments regarding form and... All obvious modifications or changes to the details should fall within the scope of protection of this invention.

Claims

1. An AI-generated image detection method based on model affinity and task difficulty calibration, characterized in that, Includes the following steps: Receive the image to be detected; The image to be detected is processed to extract model affinity features, which include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected. The image to be detected is processed to extract task difficulty features, which include pixel naturalness and image complexity obtained based on the image to be detected. Based on the task difficulty characteristics, calculate the task difficulty calibration factor used to calibrate the model affinity characteristics; The image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor are combined to generate a comprehensive feature vector characterizing the image to be detected. The integrated feature vector is input into a pre-trained machine learning classifier to obtain a detection result indicating whether the image to be detected is a real image or an AI-generated image.

2. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, The image restoration accuracy is obtained through the following methods: An occlusion region is set in the image to be detected to obtain an occlusion image; The occluded image is input into the image restoration model to obtain the restored image of the occluded area; Within the occluded area, a structural similarity index between the image to be detected and the image to be repaired is calculated as the image repair accuracy.

3. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, The semantic segmentation confidence score is obtained through the following method: The image to be detected is input into the semantic segmentation model to obtain a probability vector of each pixel in the image belonging to multiple semantic categories; For each pixel, the maximum class probability in its probability vector is taken as the prediction confidence of that pixel; The semantic segmentation confidence is obtained by averaging the prediction confidence of all pixels in the image.

4. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, The pixel naturalness is obtained through the following methods: For each pixel in the image to be detected, the conditional prediction probability of that pixel is calculated based on its global context and local context; The naturalness of a pixel is calculated based on the conditional prediction probabilities of all pixels.

5. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, The image complexity is calculated by integrating one or more of the following features: edge density, gradient magnitude, Laplacian variance, Shannon entropy, color richness, and JPEG peak signal-to-noise ratio.

6. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, The task difficulty calibration factor Based on the pixel naturalness and the image complexity The calculation is as follows: , where α and β are preset weighting coefficients.

7. The method according to claim 1, characterized in that, The combination of the image restoration accuracy, the semantic segmentation confidence, and the task difficulty calibration factor can be achieved in one of the following two ways: Method 1: Improve the accuracy of image restoration The semantic segmentation confidence Each with the task difficulty calibration factor The weighted combination is performed to generate the comprehensive feature vector. T ; Method 2: Improve the accuracy of image restoration The semantic segmentation confidence The pixel naturalness and the image complexity Concatenate into a four-dimensional feature vector , which serves as the comprehensive feature vector.

8. The AI-generated image detection method based on model affinity and task difficulty calibration according to claim 1, characterized in that, After obtaining the detection results, an anti-forensic enhancement step is also included: When the image to be detected is detected as an AI-generated image, a dynamic task detection mechanism is activated. The dynamic task detection mechanism includes replacing or supplementing the semantic segmentation task with a monocular depth estimation or surface normal prediction task, and / or replacing or supplementing the image restoration task with an image denoising or super-resolution reconstruction task, so as to update the extraction method of the model affinity features.

9. An AI-generated image detection system based on model affinity and task difficulty calibration, implementing the method according to any one of claims 1 to 8, characterized in that, include: The feature extraction module is used to receive the image to be detected and process the image to be detected to extract model affinity features and task difficulty features; wherein, the model affinity features include image restoration accuracy and semantic segmentation confidence obtained based on the image to be detected, and the task difficulty features include pixel naturalness and image complexity obtained based on the image to be detected. The calibration factor calculation module is used to calculate the task difficulty calibration factor for calibrating the model affinity feature based on the task difficulty feature. The feature combination module is used to combine the image restoration accuracy, the semantic segmentation confidence and the task difficulty calibration factor to generate a comprehensive feature vector that characterizes the image to be detected. The image detection module is used to input the comprehensive feature vector into a pre-trained machine learning classifier to obtain the detection result of whether the image to be detected is a real image or an AI-generated image.

Images