A computer-generated image forensics method, system, and readable storage medium

By combining convolutional neural networks and transform encoders, local and high-frequency features of images are extracted, solving the robustness problem of image detection in existing technologies and achieving efficient differentiation of computer-generated images.

CN117333756BActive Publication Date: 2026-11-17GUANGDONG POLYTECHNIC NORMAL UNIV
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311331966.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-16
Publication Date
2026-11-17
Estimated Expiration
2043-10-16

AI Technical Summary

Technical Problem

Existing technologies are not robust enough in distinguishing between computer-generated images and photographic images, especially under post-processing operations and adversarial attacks, and they neglect the importance of global and local features, resulting in a decline in detection performance.

Method used

A convolutional neural network is used to extract local features of the image and combine them with fast Fourier transform and high-frequency features. A transform encoder is used to capture global features, and a visual transformer and multi-head attention mechanism are used to enhance feature representation. Finally, global average pooling and softmax function are used for classification.

Benefits of technology

It improves robustness to post-processing operations and adversarial attacks, enhancing the accuracy and robustness of image detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117333756B_ABST
    Figure CN117333756B_ABST
Patent Text Reader

Abstract

This invention discloses a computer-generated image forensics method, system, and readable storage medium. The method first obtains local feature maps of the image through a convolutional block, and then sequentially uses Fast Fourier Transform, frequency-domain Gaussian high-pass filtering, and Inverse Fast Fourier Transform to obtain high-frequency feature maps. Both feature maps are divided into multiple equal-sized blocks. These blocks are then added, planarized, and mapped into a series of tokens, which are then embedded and input into a transform encoder composed of ViT blocks. The output of the transform encoder is then subjected to global average pooling to obtain classification features. Finally, the classification features are input into a fully connected layer and the softmax function is used to calculate the CG image forensics result. This invention leverages the advantages of ViT in capturing global features for CG image forensics. It combines local image features with high-frequency features obtained from the frequency domain to improve detection performance, especially robustness against post-processing operations and adversarial attacks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image forensics, and more specifically, to a computer-generated image forensics method, system, and readable storage medium. Background Technology

[0002] Distinguishing between Computer Generated (CG) and Photographic (PG) images is currently a hot research area in image forensics. CG images are typically generated using computer graphics techniques (such as 3D rendering) or deep learning algorithms (such as autoencoders (AE) and generative adversarial networks (GANs)). The difference between these CG images and real photographic images is difficult to discern with the naked eye, and their malicious use could pose a potential risk to social stability. Furthermore, in real-world scenarios, CG images transmitted over the internet may undergo post-processing operations such as compression and resizing, which challenges the robustness of CG image detection. Therefore, developing robust methods to accurately distinguish between CG and PG images has become particularly important. In recent years, deep neural networks, especially convolutional neural networks (CNNs), have been widely applied in the field of CG image forensics due to their powerful learning capabilities. There are many existing deep neural network-based CG image forensics methods, but they generally employ similar processes: first, a neural network is designed to extract features from the input image; then, the extracted features are input into a fully connected layer for classification to distinguish the image's generation method. To obtain more accurate results, the researchers also used various feature enhancement methods to help the network better distinguish between CG and PG images.

[0003] The main drawbacks of existing computer-generated image forensics methods based on the differentiation between CG and PG images are: (1) Existing methods experience a significant performance drop when detecting CG images after post-processing, limiting their practical applicability in the real world; (2) In CNN-based methods, the inherent characteristic of CNNs' limited receptive field leads to an overemphasis on local features such as texture and edges. However, all regions of CG images are synthesized, which may result in various artifacts across the entire image during computer generation. Therefore, global features are just as crucial as local features in CG image forensics; (3) CNN-based forensics methods are vulnerable to adversarial attacks.

[0004] Chinese patent CN115410029A, published on November 29, 2022, discloses a CG image detection method based on dual-stream neural network channel joint processing and soft pooling. This method relates to the field of image detection technology. Current CG image detection methods suffer from limited performance on highly heterogeneous datasets and neglect the complementarity between noise and image semantic features. The proposed method first obtains an image dataset composed of image samples, then constructs a dual-stream neural network model. This model includes a channel-specific residual extraction module, a joint channel information extraction module, and a classifier. The channel-specific residual extraction module incorporates residual structures to enhance image sample feature extraction. The entire network model employs soft pooling for downsampling. The model is then trained to obtain a trained dual-stream neural network model, which is finally used to detect the CG image samples to be detected. However, this technical solution also suffers from vulnerability to adversarial attacks during CG image detection. Summary of the Invention

[0005] In view of the above problems, the purpose of this invention is to provide a computer-generated image forensics method, system and readable storage medium.

[0006] The first aspect of this invention provides a computer-generated image forensics method, the method comprising the following steps:

[0007] The original image is input into the CNN model, and after passing through the high-frequency feature enhancement module, it is converted into two feature mapping blocks. Then, the two feature mapping blocks are summed, flattened, and mapped into token embeddings.

[0008] The token is embedded into the transform encoder to compute global features;

[0009] Global average pooling is used to obtain classification features from the global feature map;

[0010] The classification features are input into the fully connected layer, and then the softmax function is used to calculate the classification results of CG and PG images.

[0011] Preferably, the process of inputting the original image into the CNN model, converting it into two feature mapping blocks through the high-frequency feature enhancement module, and then summing, flattening, and mapping these two feature mapping blocks into token embeddings specifically involves:

[0012] The original image is input into the CNN model, and the matrix values ​​of the original image are converted into tensors and then input into the convolutional block processing stream and the high-frequency feature extractor processing stream.

[0013] In the convolutional block processing flow, the original image is sequentially processed through convolution, normalization, and max pooling operations to obtain local feature maps;

[0014] In the high-frequency feature extractor processing stream, the original image is sequentially processed through Fast Fourier Transform, frequency domain Gaussian high-pass filter, and inverse Fast Fourier Transform to obtain the high-frequency feature mapping of the image;

[0015] The local feature map and the high-frequency feature map of the image are divided into blocks of the same size through a convolutional layer, then added together and flattened, and finally mapped into a series of token embeddings.

[0016] Preferably, the expression for the local feature mapping is:

[0017] x l =Maxpool(BN(Conv(x)))

[0018] Where, x∈R H×W×3 x l ∈R I×J×C x is the original image, x l Let (H,W) be the local features of the original image, (I,J) be the size of the original image, (I,J) be the size of the output image after convolution, normalization, and max pooling, and C be the number of channels of the image.

[0019] Preferably, the step of obtaining the high-frequency feature mapping of the image by sequentially passing the original image through Fast Fourier Transform, a frequency domain Gaussian high-pass filter, and Inverse Fast Fourier Transform in the high-frequency feature extractor processing stream is specifically as follows:

[0020] The k-th color channel f of size (H, W) in the original image k (x,y), perform a Fast Fourier Transform to get f k Transformation of (x,y) from the spatial domain to the frequency domain:

[0021]

[0022] Where (x,y) and (u,v) are the coordinates of the image in the spatial and frequency domains, respectively;

[0023] High-frequency information is highlighted using a frequency-domain Gaussian high-pass filter. The filtering result of the frequency-domain Gaussian high-pass filter is denoted as:

[0024] G k (u,v)=F k (u,v)·H(u,v)

[0025]

[0026] Where D0∈R is the cutoff frequency, and D(u,v) is the distance from the frequency point (u,v) to the center of the spectrum.

[0027] The original image is transformed from the frequency domain back to the spatial domain using the inverse fast Fourier transform, denoted as:

[0028]

[0029] in, Spatial domain image;

[0030] High-frequency features are extracted from each color channel of the original image, and the final output of the high-frequency features of the original image is x. h ∈R H×W×3 .

[0031] Preferably, the local feature map and the high-frequency feature map of the image are divided into blocks of the same size through a convolutional layer, then added together and planarized, and finally mapped into a series of token embeddings, specifically as follows:

[0032] Map local features to x l and high-frequency features x h The feature map is divided into two blocks of size (p, p) and then the two blocks are added together to obtain a new feature map block:

[0033]

[0034] The addition process is represented as follows:

[0035] x p =Conv2(x l )+Conv3(x h )

[0036] Then the feature map is divided into blocks x p Flatten and map into a series of token embeddings x t ∈R N×D ,in D = p 2 ×3 represents the number and size of the tokens embedded.

[0037] Preferably, the step of embedding the token into the transform encoder to calculate global features specifically involves:

[0038] Divide the feature map into blocks x p The input is fed into the first VisionTransformer (ViT) block in the transform encoder;

[0039] After iterating through all ViT blocks in the transform encoder, the global feature is finally output.

[0040] Preferably, the feature mapping is divided into blocks x p The input is fed into the first VisionTransformer (ViT) block in the transform encoder, specifically:

[0041] For feature mapping blocks xp Layer normalization is performed, and then the input is fed into a multi-head attention mechanism (MHA). Residual connections are then used to divide the MHA output and feature map into blocks x. p The sum is used as the input to the next layer; then it undergoes layer normalization again, and then it is input to the multi-layer perceptron (MLP). Similarly, residual connections are used to add the output of the MLP to the input of this layer normalization operation as the input to the next ViT block.

[0042] Preferably, the transform encoder includes 12 stacked ViT blocks.

[0043] A second aspect of the present invention provides a computer-generated image forensics system, including a memory and a processor. The memory includes a computer-generated image forensics method program, which, when executed by the processor, implements the steps of the computer-generated image forensics method.

[0044] A third aspect of the present invention provides a computer-readable storage medium including a computer-generated image forensics method program, wherein when the computer-generated image forensics method program is executed by a processor, it implements the steps of the computer-generated image forensics method.

[0045] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0046] This invention provides a computer-generated image forensics method, system, and readable storage medium. The method first obtains local feature maps of the image through a convolutional block. Simultaneously, it sequentially uses Fast Fourier Transform (FFT), frequency-domain Gaussian high-pass filtering, and inverse Fast Fourier Transform (IFFT) to obtain high-frequency feature maps. Both feature maps are divided into multiple equal-sized blocks. These blocks are then summed, planarized, and mapped into a series of tokens, which are then embedded and input into a transform encoder composed of 12 stacked ViT blocks. The output of the transform encoder is then subjected to global average pooling to obtain classification features. Finally, the classification features are input into a fully connected layer and the softmax function is used to calculate the CG image forensics result. This invention leverages the advantages of ViT in capturing global features for CG image forensics. It combines local image features with high-frequency features obtained from the frequency domain to improve detection performance, especially robustness against post-processing operations and adversarial attacks. Attached Figure Description

[0047] Figure 1 This is a flowchart of a computer-generated image forensics method as described in Example 1.

[0048] Figure 2 This is an architecture diagram of a computer-generated image forensics method described in Example 1. Detailed Implementation

[0049] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0051] Example 1

[0052] like Figure 1 As shown in the figure, this embodiment discloses a computer-generated image forensics method, which includes the following steps:

[0053] S1: Input the original image into the CNN model, and after passing through the high-frequency feature enhancement module, it is converted into two feature mapping blocks. Then, the two feature mapping blocks are summed, flattened, and mapped into token embeddings.

[0054] Step S1 includes the following steps:

[0055] S1.1: Input the original image into the CNN model, convert the matrix values ​​of the original image into tensors, and then input them into the convolutional block processing stream and the high-frequency feature extractor processing stream.

[0056] S1.2: In the convolutional block processing flow, the original image is sequentially processed through convolution, normalization, and max pooling operations to obtain local feature maps.

[0057] In this embodiment, for the original image x∈R H×W×3 The local features of the image are obtained by sequentially performing convolution, normalization, and max pooling operations. These local features can be represented as follows:

[0058] x l =Maxpool(BN(Conv(x)))

[0059] Where, x∈R H×W×3 x l ∈R I×J×C x is the original image, x l Let (H,W) be the local features of the original image, (I,J) be the size of the original image, (I,J) be the size of the output image after convolution, normalization, and max pooling, and C be the number of channels of the image.

[0060] S1.3: In the high-frequency feature extractor processing stream, the original image is sequentially processed through Fast Fourier Transform, frequency domain Gaussian high-pass filter, and inverse Fast Fourier Transform to obtain the high-frequency feature mapping of the image.

[0061] In this embodiment, the R, G, and B color channels of the original image are all processed. S1.3 specifically includes the following steps:

[0062] S1.3.1: Extract the k-th color channel f of size (H, W) from the original image. k (x,y), perform a Fast Fourier Transform to get f k Transformation of (x,y) from the spatial domain to the frequency domain:

[0063]

[0064] Where (x,y) and (u,v) are the coordinates of the image in the spatial and frequency domains, respectively.

[0065] S1.3.2: Use a frequency-domain Gaussian high-pass filter to highlight high-frequency information. The filtering result of the frequency-domain Gaussian high-pass filter is denoted as:

[0066] G k (u,v)=F k (u,v)·H(u,v)

[0067]

[0068] Where D0∈R is the cutoff frequency, and D(u,v) is the distance from the frequency point (u,v) to the center of the spectrum.

[0069] S1.3.3: The original image is transformed from the frequency domain back to the spatial domain using the inverse fast Fourier transform, denoted as:

[0070]

[0071] in, Spatial domain image;

[0072] S1.3.4: Extract high-frequency features from each color channel of the original image, and finally output the high-frequency features of the original image as x. h ∈R H×W×3 .

[0073] S1.4: The local feature map and the high-frequency feature map of the image are divided into blocks of the same size through a convolutional layer, then added together and flattened, and finally mapped into a series of token embeddings.

[0074] In this embodiment, the local feature map x l and high-frequency features x hThe feature map is divided into two blocks of size (p, p) and then the two blocks are added together to obtain a new feature map block:

[0075]

[0076] The addition process is represented as follows:

[0077] x p =Conv2(x l )+Conv3(x h )

[0078] Then the feature map is divided into blocks x p Flatten and map into a series of token embeddings x t ∈R N×D ,in D = p 2 ×3 represents the number and size of the tokens embedded.

[0079] S2: Embed the token into the transform encoder to compute global features.

[0080] Specifically, the feature map is first divided into blocks x p The input is fed into the first Vision Transformer (ViT) block in the transform encoder; then all ViT blocks in the transform encoder are traversed in turn, and finally the global features are output.

[0081] Furthermore, in this embodiment, the feature mapping is first divided into blocks x. p Layer normalization is performed, and then the input is fed into a multi-head attention mechanism. Residual connections are then used to divide the MHA output and feature map into blocks x. p The sum is used as the input for the next layer; then it undergoes another layer normalization operation before being input into the MLP. Similarly, a residual connection is used to sum the MLP output with the input from the current layer normalization operation as the input for the next ViT block. For MHA, the similarity between tokens can be obtained through dot product calculation, thus achieving long-distance attention and global attention. For the MLP, dimensionality changes and nonlinear transformations are applied to each token, thereby enhancing the token's representational power.

[0082] It should be noted that the transform encoder described in this embodiment includes 12 stacked ViT blocks. Since the input and output dimensions of the ViT blocks in the transform encoder are the same, the output of the previous ViT block can be directly used as the input to the next ViT block. There are a total of 12 stacked ViT blocks, so the above steps need to be repeated.

[0083] S3: Perform global average pooling on the global feature map to obtain classification features.

[0084] It should be noted that this application does not use classification tokens to obtain classification features like the basic ViT model. Instead, it uses global average pooling to integrate features from different channels to ensure translation invariance.

[0085] S4: Input the classification features into the fully connected layer and then use the softmax function to calculate the classification results of CG and PG images.

[0086] It should be noted that in this embodiment, the classification features are input into the classifier, i.e., the fully connected layer, and then the softmax function is used to obtain the class probability results of CG and PG images.

[0087] Overall, such as Figure 2 As shown, the method described in this embodiment mainly includes three parts: (1) High-frequency feature enhancement: extracting local features and high-frequency features of the image through convolutional blocks and high-frequency feature extractors respectively; (2) Transform coding: gradually extracting global features of the image through multiple stacked ViT blocks; (3) Global average pooling layer and classifier: outputting the classification results of the final CG image and PG image.

[0088] Example 2

[0089] This embodiment discloses a computer-generated image forensics system, including a memory and a processor. The memory includes a computer-generated image forensics method program, and the processor executes the steps of the computer-generated image forensics method described in Embodiment 1.

[0090] Example 3

[0091] This embodiment discloses a computer-readable storage medium, which includes a computer-generated image forensics method program. When the computer-generated image forensics method program is executed by a processor, it implements the steps of the computer-generated image forensics method described in Embodiment 1.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0093] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0095] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for computer-generated image forensics, characterized in that, The method includes the following steps: The original image is input into the CNN model, and after passing through the high-frequency feature enhancement module, it is converted into two feature mapping blocks. Then, the two feature mapping blocks are summed, flattened, and mapped into token embeddings. The token is embedded into the transform encoder to compute global features; Global average pooling is used to obtain classification features from the global feature map; The classification features are input into the fully connected layer and then the softmax function is used to calculate the classification results of CG and PG images. The process involves inputting the original image into a CNN model, converting it into two feature mapping blocks via a high-frequency feature enhancement module, and then summing, flattening, and mapping these two feature mapping blocks into token embeddings. Specifically: The original image is input into the CNN model, and the matrix values ​​of the original image are converted into tensors and then input into the convolutional block processing stream and the high-frequency feature extractor processing stream. In the convolutional block processing flow, the original image is sequentially processed through convolution, normalization, and max pooling operations to obtain local feature maps; In the high-frequency feature extractor processing stream, the original image is sequentially processed through Fast Fourier Transform, frequency domain Gaussian high-pass filter, and inverse Fast Fourier Transform to obtain the high-frequency feature mapping of the image; The local feature map and the high-frequency feature map of the image are divided into blocks of the same size through a convolutional layer, then added together and flattened, and finally mapped into a series of token embeddings; The process of obtaining high-frequency feature mapping of the image by sequentially passing the original image through Fast Fourier Transform, frequency domain Gaussian high-pass filter, and inverse Fast Fourier Transform in the high-frequency feature extractor processing stream is as follows: The original image with a size of The kth color channel Perform a Fast Fourier Transform to Transformation from the spatial domain to the frequency domain: ; in, and These are the coordinates of the image in the spatial and frequency domains, respectively. High-frequency information is highlighted using a frequency-domain Gaussian high-pass filter. The filtering result of the frequency-domain Gaussian high-pass filter is denoted as: ; ; in It is the cutoff frequency. It is a frequency point Distance to the center of the spectrum The original image is transformed from the frequency domain back to the spatial domain using the inverse fast Fourier transform, denoted as: ; in, Spatial domain image; High-frequency features are extracted from each color channel of the original image, and the final output is the high-frequency features of the original image. ; The transform encoder includes 12 stacked visual transform blocks.

2. The computer-generated image forensics method according to claim 1, characterized in that, The expression for the local feature mapping is: ; in, , , The original image. These are local features of the original image. This refers to the original image size. It is the size of the output image after the original image has undergone convolution, normalization, and max pooling operations, and C is the number of channels in the image.

3. The computer-generated image forensics method according to claim 1, characterized in that, The local feature map and the high-frequency feature map of the image are divided into blocks of the same size through a convolutional layer, then added together and flattened, and finally mapped into a series of token embeddings, specifically as follows: Map local features and high frequency characteristics Divide into sizes of The feature map is divided into blocks, and then the two blocks are added together to obtain a new feature map block: ; The addition process is represented as follows: ; Then the feature map is divided into blocks. Flatten and map into a series of token embeddings ,in , These represent the number and size of the tokens embedded, respectively.

4. The computer-generated image forensics method according to claim 3, characterized in that, The step of embedding the token into the transform encoder to calculate global features specifically involves: Divide the feature map into blocks The input is fed into the first vision transformer block in the transform encoder; After sequentially traversing all the visual transformer blocks in the transform encoder, the global features are finally output.

5. A computer-generated image forensics method according to claim 4, characterized in that, The feature mapping is divided into blocks. The input is fed into the first visual transformer block in the transform encoder, specifically as follows: Feature mapping blocks Layer normalization is performed, and then the input is fed into a multi-head attention mechanism. Residual connections are then used to divide the output of the multi-head attention mechanism and the feature map into blocks. The sum is used as the input for the next layer; Then, after another layer normalization operation, the data is input to the multilayer perceptron. Similarly, residual connections are used to add the output of the multilayer perceptron to the input of this layer normalization operation as the input of the next visual transformer block.

6. A computer-generated image forensics system, characterized in that, The device includes a memory and a processor. The memory includes a computer-generated image forensics method program. When the computer-generated image forensics method program is executed by the processor, it implements the steps of a computer-generated image forensics method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer-generated image forensics method program, which, when executed by a processor, implements the steps of a computer-generated image forensics method as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • CG image detection method based on double-flow neural network channel combination and soft pooling

    CN115410029A