Universal artificial intelligence generated image detection method and system

Abstract

The invention belongs to the image processing technology, and particularly relates to a universal artificial intelligence generated image detection method and system, and the method comprises the steps: dividing an input image into a plurality of small-size image blocks, and disturbing the arrangement order of the image blocks; processing the images after order disruption through multi-scale sampling, then calculating local entropies for feature maps obtained through multi-scale sampling by using a sliding window, and generating a feature map of a multi-granularity local entropy mode; and inputting the feature map of the multi-granularity local entropy mode into a pre-trained classifier, and judging whether the input image is a real image or an AI generation image by the classifier according to the input feature map. On one hand, by introducing the multi-granularity local entropy mode, common features in images generated by different generation models can be effectively captured, and the method has wide applicability; on the other hand, through image block disruption and multi-scale resampling, semantic information in the image is eliminated, content deviation is reduced, and detection robustness is improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and particularly relates to a general artificial intelligence generated image detection method and system. Background Art

[0002] In recent years, generative artificial intelligence technology has made remarkable breakthroughs. Especially in the field of image generation, the wide application of models such as Generative Adversarial Networks (GAN) and Diffusion Models has brought a qualitative leap in the quality of AI-generated images. These technologies have brought innovation and convenience to fields such as art creation, advertising design, and game production. However, at the same time, the abuse of AI-generated images in aspects such as the spread of false information and identity forgery has also raised social security and ethical issues. Therefore, AI-generated image detection technology has become an important research direction. However, the existing detection methods still have deficiencies in the following aspects:

[0003] First of all, most detection methods rely on specific content features of the generative model during training and testing. For example, detectors based on convolutional neural networks usually learn local texture and contrast features of images, but when the generative model and the scenario change, the detector often has difficulty generalizing to new scenarios or generation methods. Secondly, there are significant differences in the generation strategies adopted by different generative models. For example, GAN generates images based on the feedback of the discriminator, while the diffusion model generates images based on the noise process. Due to these different generation mechanisms, the existing detection methods often can only achieve good results on a single model, but perform poorly in cross-model detection tasks. In addition, many generative models will retain the content semantic structure of the image during the generation process, resulting in the detector being interfered by the image content during the learning process and making it difficult to learn the essential differences between generated images and real images.

[0004] Currently, most detection methods rely on specific datasets for training, but these datasets are usually limited in size and may have problems with uneven data distribution. For example, the images generated by ProGAN, StyleGAN, and BigGAN may be fully learned, but the detection performance may decline due to distribution differences for the images generated by DDPM or Stable Diffusion. Therefore, how to improve the generalization ability of the detector across models, scenarios, and contents has become a key challenge in the current field of AI-generated image detection.

[0005] In the field of AI-generated image detection, the existing detection methods can be mainly divided into the following three categories:

[0006] 1) Spatial domain methods. This type of method detects and generates features of an image by analyzing pixel values, edges, local textures, or color information of the image. This type of method directly extracts features in the spatial domain of the image. Common techniques include methods based on pixel statistics, edge detection, and local texture analysis.

[0007] 2) Frequency domain methods. This type of method analyzes the differences between the generated image and the real image in the frequency domain by performing a discrete Fourier transform (DFT) or a discrete cosine transform (DCT) on the image.

[0008] 3) Methods based on pre-trained models. In recent years, methods based on large-scale pre-trained models (such as CLIP, DALL·E) have shown certain effects in the detection of AI-generated images.

[0009] However, the limitations of existing detection methods mainly lie in: poor generalization, insufficient detection effects across models and scenarios; being easily interfered by the semantic information of images and difficult to learn the essential differences between generated images and real images; the detection performance being limited by the quantity and diversity of training data, and the performance deteriorating in new models or new scenarios. Summary of the Invention

[0010] In view of the deficiencies of current AI-generated image detection methods in terms of cross-model, cross-scenario, and content agnosticism, etc., the present invention proposes a general artificial intelligence-generated image detection method, which divides the input image into several small-sized image blocks and disrupts the arrangement order of the image blocks; processes the shuffled image through multi-scale sampling, and then uses a sliding window to calculate the local entropy pattern to generate a feature map of multi-granularity local entropy patterns; inputs the feature map of multi-granularity local entropy patterns into a pre-trained classifier, and the classifier determines whether the input image is a real image or an AI-generated image according to the input feature map.

[0011] The present invention also proposes a general artificial intelligence-generated image detection system for implementing a general artificial intelligence-generated image detection method, including:

[0012] An image acquisition module for acquiring an image to be detected;

[0013] A perturbation module for splitting the original image into smaller image blocks and disrupting the original arrangement order of the image blocks;

[0014] A multi-size feature acquisition module for processing the perturbed image by using downsampling and upsampling of different scales, and then splicing the feature maps of different scales on the channels;

[0015] The local entropy calculation module is used to calculate the local entropy of feature maps of each size according to the set sliding window size and sliding compensation, map the local entropy of each size to feature maps of three channels respectively, and splice the three-channel feature maps of all sizes together to obtain a feature map of the multi-granularity local entropy pattern;

[0016] The classifier is used to classify according to the feature map of the multi-granularity local entropy pattern.

[0017] The present invention also provides a general artificial intelligence generated image detection device, and the device includes:

[0018] A memory;

[0019] A processor;

[0020] Wherein, the memory stores computer execution instructions;

[0021] The processor executes the computer execution instructions stored in the memory to implement a general artificial intelligence generated image detection method.

[0022] The present invention also provides a computer storage medium, in which computer execution instructions are stored, and when the computer execution instructions are executed by a processor, they are used to implement a general artificial intelligence generated image detection method.

[0023] The beneficial effects of the present invention include:

[0024] 1) Improve the detection performance. The local entropy pattern effectively enhances the model's perception ability of the artifacts of the generated images, and the detection accuracy is significantly improved;

[0025] 2) Enhance the cross-model detection ability. Through multi-scale resampling and local entropy feature extraction, the feature differences of different generation models at different scales are effectively captured;

[0026] 3) Improve the cross-scene generalization ability. By performing the block scrambling operation to destroy the content semantic structure of the image, the content unknowability of the model is enhanced;

[0027] 4) Be compatible with different generation models, and be compatible with mainstream generation models such as GAN and diffusion models;

[0028] 5) Have an efficient inference speed. While maintaining high detection performance, it maintains an efficient inference speed and is suitable for large-scale data sets and real-time detection. Brief Description of the Drawings

[0029] Figure 1 It is a flowchart of a general artificial intelligence generated image detection method of the present invention;

[0030] Figure 2 It is a schematic diagram of a multi-granularity entropy pattern feature processor in the present invention;

[0031] Figure 3 Schematic diagram of the calculation process of the medium entropy mode in the present invention;

[0032] Figure 4 This is a preferred embodiment for the present invention to implement AI-generated image detection. Detailed implementation manners

[0033] Next, with reference to the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] The present invention proposes a general method for detecting AI-generated images, which divides an input image into several small-sized image blocks and disturbs the arrangement order of the image blocks; processes the shuffled image through multi-scale sampling, and then uses a sliding window to calculate the local entropy mode to generate a feature map of multi-granularity local entropy mode; inputs the feature map of multi-granularity local entropy mode into a pre-trained classifier, and the classifier determines whether the input image is a real image or an AI-generated image according to the input feature map.

[0035] The present invention proposes a method for detecting AI-generated images based on multi-granularity local entropy mode (MLEP). By performing feature modeling on the local information entropy of the image and combining block scrambling and multi-scale resampling operations, the detection performance of the model in cross-model and cross-scene scenarios is enhanced. In this embodiment, as Figure 1 , the present invention is a general method for detecting AI-generated images, which specifically includes the following steps:

[0036] For an input image to be detected, first perform a blocking operation on it, divide the input image into multiple small blocks, and the size of each small block is a specific size, usually l×l;

[0037] Then perform a scrambling operation on the blocked image, and re-arrange the positions of the image blocks by random permutation to form a new scrambled image;

[0038] Generate feature representations of different scales for the scrambled image through a multi-scale resampling module. First, downsample the scrambled image to generate image representations of different resolutions; then, through an upsampling operation, restore the images of different scales to the original resolution;

[0039] Stitch all the scale images in the channel dimension to form a complete multi-scale image feature representation. Through the multi-scale resampling operation, the cross-scale perception ability of the detection model for the artifacts of the generated images is enhanced;

[0040] Based on the multi-scale image representation, local entropy pattern (MLEP) features are extracted. A sliding window is used to scan the image, with the window size being w×w. Within each window, the local entropy value is calculated based on the Shannon entropy formula to generate a local entropy feature map. The local entropy feature map is combined with the multi-scale image representation in the channel dimension to form a complete feature representation;

[0041] The local entropy feature map is fed into a convolutional neural network (CNN) model for classification. Through end-to-end training, the CNN model learns the correspondence between the local entropy features and the image generation mechanism, and outputs "real" or "generated" labels. In this embodiment, the specific architecture of the CNN model is not limited, and various pre-trained CNN models in the prior art can be used to supervise the training and inference processes, and finally the detection result is output.

[0042] The construction process of the multi-granularity entropy pattern feature processor adopted in this embodiment includes:

[0043] Step 1) Divide the input image into multiple image blocks of a fixed size, and rearrange the positions of the image blocks through a random permutation operation to generate a new scrambled image;

[0044] Step 2) Input the scrambled image into a multi-scale resampling module to generate feature representations of different scales. By downsampling and upsampling the scrambled image, image representations of different resolutions are generated, and all scale images are concatenated in the channel dimension to generate a complete multi-scale feature representation;

[0045] Step 3) Based on the multi-scale feature representation, use a sliding window to extract local entropy features, calculate the entropy value within the window based on the Shannon entropy formula, and generate a local entropy feature map.

[0046] As an alternative implementation, in the input image processing, first divide the input image into several small patches, each with a size of l×l (preferably, l is a positive integer less than 8). Assuming the input image size is H×W×C, where C is the number of channels, the set of image blocks obtained by the block operation is represented as:

[0047]

[0048] where X represents the original image, which is composed of image blocks; X i,j represents the image block in the i-th row and j-th column in the arrangement order of the original image.

[0049] After the block operation is completed, shuffle the order of the image blocks through random permutation (Shuffling) to form a new scrambled image:

[0050]

[0051] Among them, represents the scrambled image, which is composed of individual image patches, and the image differs from image X only in the arrangement of the image patches; π is the permutation mapping of the small patch indices, that is, the image patch in the i-th row and j-th column of the original image is mapped to the position π(i, j); represents the image patch at the position π(i, j) in the scrambled image. Through the block scrambling operation in this embodiment, the global and local semantic structures of the image can be destroyed, reducing the model's dependence on content features, thereby enhancing the content agnosticism of the detector under different generative models.

[0052] After completing the block scrambling, the present invention adopts a multi-scale resampling strategy to generate multi-scale image representations to capture the generative artifact features at different scales. Given a set of scale factor sets where s k ∈(0, 1] represents the k-th scale factor, and K is the number of scale factors. The scrambled image is downsampled and upsampled through an interpolation function to generate images at different scales, and the expression is as follows:

[0053]

[0054] Among them, represents the feature map obtained by downsampling with the k-th scale factor, represents the feature map that is upsampled to the original image resolution according to ; is the downsampling operation, indicating that the image is reduced to s k H × s k W; is the upsampling operation, which restores the image to the original resolution H × W.

[0055] The images that have been successively subjected to block scrambling and downsampling and upsampling with different scale factors are concatenated on the channel to form the image

[0056]

[0057] Through multi-scale resampling, the present invention can effectively capture the artifact information of different generative models at different scales, enhancing the cross-model detection ability.

[0058] The local entropy mode feature processing network adopted in this embodiment includes two parts: local entropy feature extraction and entropy feature modeling, specifically including:

[0059] (1) Local entropy feature extraction

[0060] In the part of local entropy feature extraction, first, a sliding window is used to scan the input image. The window size is w×w, and the sliding step is 1. Inside each window, the distribution probability p(x i ) of the pixel values in the window is counted, and the local entropy value of the window is calculated according to the Shannon Entropy formula:

[0061]

[0062] where n is the number of pixels in the window, and p(x i ) represents the probability that the i-th pixel value in the window appears in the window.

[0063] As a preferred implementation, the image after multi-scale resampling will next perform local entropy calculation with a sliding window size of 2×2 and a sliding step of 1, and the entropy value of each 2×2 window is in the set {0, 0.8, 1.0, 1.5, 2.0}. For example, Figure 3 When the image block size is 2×2, since there are only four pixel values, the value of n is 4, and a pixel block is represented as X 1,1 = {x 1,1 , x 1,2 , x 2,1 , x 2,2}, and a pixel value e 1,1 of a local entropy map is generated through these four pixel values. Then, according to the pixel values of {x 1,2 , x 1,3 , x 2,2 , x 2,3}, the pixel value e 1,2 of the local entropy map is calculated. And so on, according to {x 1,W-1 , x 1,W , x 2,W-1 , x 2,W}, the first row, the W-1th pixel value e 1,W-1 of the local entropy map is calculated; according to {x H-1,W-1 , x H-1,W , x H,W-1 , x H,W}, the H-1th row, the W-1th pixel value e H-1,W-1 of the local entropy map is calculated. The length and width of the obtained local entropy map are both 1 smaller than those of the original image. The pixel values of the local entropy map are calculated according to the probability distribution of the pixel values in the pixel block. This embodiment includes the following five cases:

[0064] ① The four pixel values are equal:

[0065] H(X) = -1×log21 = 0

[0066] ②Three pixel values are equal, and the other is not equal:

[0067]

[0068] ①Two pairs of pixel values are respectively equal:

[0069]

[0070] ②One pair of pixel values is equal, and the other two pixel values are not equal to each other:

[0071]

[0072] ③All four pixel values are not equal:

[0073]

[0074] In order to make the entropy value of each 2×2 window fall within the set {0, 0.8, 1.0, 1.5, 2.0}, it is necessary to round down the entropy value in the case where three pixel values are equal and the other is not equal.

[0075] As an optional embodiment, the window of the present invention can be set as needed. The optimal range of the set window is between 2×2 and 8×8. After the window becomes larger, the combination of pixel values becomes more. The pixel values can be directly calculated according to the local entropy calculation formula, or after calculating the entropy value, the entropy values within a certain range can be normalized to a specified interval.

[0076] The multi-granularity local entropy pattern is generated by fusing local entropy feature maps of different scales, and can capture the pixel relationships inside the image as well as the pixel randomness across scales and blocks. After calculation, the length and width of the image are each reduced by one pixel, and the image is expressed as:

[0077]

[0078] where X is the input image, is the image after being calculated by the multi-granularity local entropy pattern.

[0079] The shuffled image, each multi-scale image, and the feature map of the local entropy pattern are all 3-channel image data. These image data are concatenated in the channel dimension to obtain the final entropy feature map. The local entropy pattern captures the randomness and uncertainty within the local area of the image, and effectively enhances the perception ability of the generated artifacts.

[0080] By moving the sliding window over the entire image, an entropy feature map is generated. The entropy value corresponding to each pixel position reflects the complexity of the pixel distribution within that area; for the generated image, due to different generation mechanisms, the local entropy distribution often has differences from the real image.

[0081] (2) Entropy Feature Modeling

[0082] In the entropy feature modeling part, the generated local entropy feature map is fed into a Convolutional Neural Network (CNN) for learning and modeling. The CNN extracts the high-level representation of the local entropy feature by performing convolution operations on the spatial distribution of the entropy feature. Through end-to-end training, the CNN can capture the statistical differences between the local entropy features in the generated image and the real image, and finally output the classification result of "real" or "generated".

[0083] The present invention does not limit the convolutional neural network adopted. Those skilled in the art can adopt any network in the prior art. Preferably, the classifier of the present invention can adopt the ResNet architecture, such as any one of ResNet-18, ResNet-34, ResNet-50, and ResNet-101.

[0084] The classifier in this embodiment adopts the Cross-Entropy Loss function:

[0085]

[0086] Among them, represents the binary cross-entropy loss function; N represents the total number of samples; y i represents the true label of the i-th sample; represents the predicted label of the i-th sample, f(·) represents the classifier, represents the feature map of the multi-granularity local entropy pattern.

[0087] The local entropy pattern feature processing network can effectively capture the artifact features in the generated image by modeling the pixel distribution complexity of the local area of the image, and enhance the model's perception ability of different types of generated images.

[0088] The method of the present invention can be used for a wide range of AI-generated images on the Internet. And in the training stage of the present invention, at least real images and images generated by one generation model can be used to complete the training. After the training is completed, the model can not only identify the images generated by the generation model used in the training stage, but also identify the images generated by other generation models, which can reduce the annotation cost of samples in the training stage. A specific embodiment is as Figure 4 shown. This figure shows a case of detecting AI-generated images in a real scenario. In the training stage, the model has only seen the images generated by ProGAN and real images. In the real scenario, the model has the ability to detect the images generated by different generation models and can still detect the images generated by the diffusion model with high accuracy.

[0089] This embodiment also presents the specific training process of the multi-granularity entropy pattern feature processor, which specifically includes the following steps:

[0090] 1) Dataset and preprocessing

[0091] The dataset images for training and testing the present invention have two labels: real (0) and AI-generated (1).

[0092] Among them, the training images are the images generated by ProGAN from the ForenSynths dataset. The images of this dataset are generated by a single generation model, which can verify the detection accuracy of the present invention during the test phase and also verify the generalization performance.

[0093] The training dataset comes from the ForenSynth, GANGen-Detection, DiffusionForensics, and UniversalFakeDetect datasets. In addition, it also includes images from commercial models Midjourney and DALL·E2.

[0094] Due to the different resolutions of the images in different datasets, during the training phase of the present invention, the input images are randomly cropped into images of 224×224, and during the test phase, they are centrally cropped into images of 224×224 to meet the input conditions of the ResNet network.

[0095] 2) Training of the network

[0096] The training set is the ProGAN-generated images and real images in the ForenSynths dataset.

[0097] In this embodiment, a simulation experiment is conducted to quantitatively verify the proposed general AI-generated image detection model, and its purpose is to test the usability of the model. The experiment effectively proves the superiority of the method of the present invention by comparing with the methods proposed by Ojha, Zheng, and Tan.

[0098] The test experiment uses Accuracy and Average Precision to measure the gap between the anonymized images and the original images. Tables 2 and 3 respectively show the objective indicators of the privacy protection performance of the method of the present invention on the datasets based on the generative GAN and diffusion models, and Table 4 shows the comparison of the objective privacy protection performance of the method of the present invention with various methods. The results show that the detection performance of the method of the present invention after multi-granularity entropy pattern calculation is higher than other methods.

[0099] Table 2 Detection performance of the method of the present invention on the dataset based on the generative adversarial network

[0100]

[0101] Table 3 Detection performance of the method of the present invention on a diffusion model-based dataset

[0102]

[0103] Table 4 Comparison of detection performance between the method of the present invention and other methods

[0104]

[0105] In the above table, the Ojha method is from "Towards Universal Fake Image Detectors that Generalize Across Generative Models" published by Ojha et al.; the Zheng method is from "CLIPping the Deception: Adapting Vision-Language Models for Universal Deepfake Detection" published by Zheng et al.; Tan is from "Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection" published by Tan et al.

[0106] In summary, through the above simulation experiments, the feasibility of the solution in this embodiment is verified in the examples of the present invention. An AI-generated image detection method based on a multi-granularity local entropy pattern provided by the examples of the present invention effectively solves the problem of insufficient generalization ability of existing detection methods when facing diverse generative models through local entropy feature extraction and multi-scale resampling techniques, and at the same time significantly improves the accuracy and robustness of AI-generated image detection.

[0107] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A general method for detecting AI-generated images, characterized in that, Divide the input image into several small-sized image patches and disrupt the arrangement order of the image patches; process the image with the disrupted order through multi-scale sampling, and then calculate the local entropy for the feature maps obtained by multi-scale sampling respectively using a sliding window to generate a feature map of multi-granularity local entropy patterns; input the feature map of multi-granularity local entropy patterns into a pre-trained classifier, and the classifier determines whether the input image is a real image or an AI-generated image according to the input feature map.

2. The method according to claim 1, characterized in that, Performing multi-scale resampling includes: performing downsampling and upsampling operations on the image using different scale factors, that is, first scaling the image through downsampling, and then performing upsampling on the downsampled image to restore it to the original resolution.

3. The method according to claim 1, characterized in that, Concatenate the features obtained by processing the image with the disrupted order through multi-scale sampling on the channels, and then calculate the local entropy using a sliding window with a size of 2×2 and a sliding step of 1. For each 2×2 window, calculate the entropy value using the following Shannon entropy formula: Among them, p(x i ) represents the probability that the i-th pixel value appears within the 2×2 pixel block in that pixel block.

4. The method according to claim 3, characterized in that, If the pixel values of three pixel points in a 2×2 window are equal, set the local entropy value of this window to 0.

8.

5. The method according to claim 3 or 4, characterized in that, The acquisition of the feature map of multi-granularity local entropy patterns includes: Process the features of the image with the disrupted order on the color channels respectively using downsampling and upsampling of multiple sizes to obtain the feature maps corresponding to each size on each color channel; Calculate the local entropy respectively according to the features of the feature maps after sampling with multiple different sizes on the color channels to obtain the local entropy map of each feature map.

6. Stitch all local entropy maps on the color channels to obtain a feature map of the multi-granularity local entropy pattern. According to the method described in claim 1, wherein The classifier adopts the ResNet architecture, including ResNet-18, ResNet-34, ResNet-50 or ResNet-101.

7. The method according to claim 1, characterized in that Use the binary cross-entropy loss function to train the classifier, and the binary cross-entropy loss function is expressed as: Among them, represents the binary cross-entropy loss function; N represents the total number of samples; y i represents the true label of the i-th sample; represents the predicted label of the i-th sample, f(·) represents the classifier, represents the feature map of the multi-granularity local entropy pattern.

8. A general artificial intelligence-generated image detection system, characterized in that, For implementing a general artificial intelligence-generated image detection method described in claim 1, including: An image acquisition module for acquiring an image to be detected; A perturbation module for dividing the original image into smaller image patches and disrupting the original arrangement order of the image patches; A multi-size feature acquisition module for processing the perturbed image using downsampling and upsampling of different scales, and then concatenating the feature maps of different scales on the channels; A local entropy calculation module for calculating the local entropy of the feature maps of each size according to the set sliding window size and sliding compensation, and mapping the local entropy of each size to a feature map of three channels respectively, and concatenating the three-channel feature maps of all sizes together to obtain a feature map of multi-granularity local entropy patterns; A classifier for classifying according to the feature map of multi-granularity local entropy patterns.

9. A general artificial intelligence-generated image detection device, characterized in that, The device includes: A memory; A processor; Wherein, the memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement a general artificial intelligence-generated image detection method described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, Computer execution instructions are stored in the computer storage medium, and when the computer execution instructions are executed by the processor, they are used to implement a general artificial intelligence-generated image detection method described in any one of claims 1 to 7.

Images