System and method for detecting computer generated images

Abstract

A system and method for detecting a computer generated image. The system includes an image processing engine for analyzing an input digital image embedded with image traces created during generation and / or post-generation processing operations of the input digital image and determining whether the input digital image is a computer generated image or a natural photographic image based on the analysis of the image traces.

Description

Technical Field

[0001] The present disclosure relates to a system and method for detecting computer-generated images, and in particular, but not exclusively, to a system for identifying computer-generated images and naturally-generated photographic images. Background Art

[0002] With the advancement of digital imaging, media editing technology, and artificial intelligence (AI) generative models (e.g., generative adversarial networks (GANs), diffusion models (DMs)), it is becoming increasingly easy to synthesize compelling computer-generated (CG) images. Although CG images have broadened the boundaries of digital multimedia, they have also attracted widespread attention on the authenticity of digital media due to the fact that fake CG images may lead to false positives and pose a serious threat to the information ecosystem. A recent example is an award-winning documentary photographer who deceived the world with computer-generated images, which shows that the photojournalism industry is vulnerable to fake news pictures.

[0003] Distinguishing between computer-generated and natural photography (PG) images is very important for verifying the authenticity and originality of digital images. However, these cutting edge generation methods enable high-quality synthesis in CG images, which makes this challenging task even trickier and more challenging. Summary of the invention

[0004] According to a first aspect of the present disclosure, there is provided a system for detecting computer-generated images. The system comprises: an image processing engine for analyzing an input digital image embedded with image traces created during generation and / or post-generation processing operations of the input digital image, and determining whether the input digital image is a computer-generated image or a natural photographic image based on the analysis of the image traces.

[0005] According to a first aspect, the image traces comprise multi-scale texture patterns of image features in the input digital image.

[0006] According to a first aspect, the image processing engine comprises a machine learning based processing engine.

[0007] According to a first aspect, the image processing engine includes a global texture representation module for obtaining relationships and differences in the multi-scale texture patterns in order to determine whether the image features are generated by a computational process or by photographic means.

[0008] According to the first aspect, the global texture representation module is combined with a ResNet architecture.

[0009] According to the first aspect, the global texture representation module includes at least one convolutional layer, a Gram matrix-based activation layer, and a global pooling layer.

[0010] According to the first aspect, the image processing engine further comprises a texture enhancement module for amplifying texture differences of the multi-scale texture pattern.

[0011] According to a first aspect, the texture enhancement module is used to amplify the distinguishing traces associated with the image features, thereby facilitating the global texture representation module to obtain the relationship and difference of the multi-scale texture patterns.

[0012] According to the first aspect, an affine transformation operation guided by a semantic segmentation map and a texture restoration based on a convolutional neural network are used to amplify the difference traces.

[0013] According to a first aspect, the texture enhancement module comprises at least one convolutional layer, a semantic segmentation map guided residual block, an affine transformation module and an upsampling module.

[0014] According to the first aspect, the image processing engine further comprises a deep parsing network for generating a segmentation map for an affine transformation operation guided by the semantic segmentation map.

[0015] According to the first aspect, the deep parsing network is further used to generate an intermediate spatial feature transformation map and a feature map associated with the image features for further processing by the affine transformation module.

[0016] According to the first aspect, the image processing engine further comprises an attention-based feature perception module for facilitating trace detection in spatial and channel dimensions.

[0017] According to the first aspect, the attention-based feature perception module includes a convolutional layer, an average pooling layer, and a channel-spatial attention module.

[0018] According to the first aspect, the channel-spatial attention module comprises a channel attention submodule sequentially connected to a spatial attention submodule.

[0019] According to the first aspect, the image processing engine also includes a fully connected layer and a Softmax layer, which is used to determine the output probability indicating whether the input digital image is a computer-generated image or a natural photographic image based on the cascaded high-level features obtained by the global texture representation module, the texture enhancement module and the attention-based feature perception module.

[0020] According to a first aspect, the image traces include texture disturbances, high frequency residuals or global space traces in the input digital image.

[0021] According to a first aspect, the computer generated image is generated by geometric data modeling, photo-realistic rendering, or based on an artificial intelligence generative model.

[0022] According to a first aspect, the image processing engine is trained by providing computer-generated images and a plurality of natural photographic images as positive and negative samples so as to train the image processing engine in a machine learning process; wherein the natural photographic images are generated by a digital camera.

[0023] According to the first aspect, the negative samples and the positive samples include natural photographic images and computer-generated images with added image noise and / or compression traces, respectively.

[0024] According to a second aspect of the present disclosure, there is provided a method for detecting a computer-generated image. The method comprises the following steps: analyzing an input digital image embedded with image traces created during generation and / or post-generation processing operations of the input digital image; and determining whether the input digital image is a computer-generated image or a natural photographic image based on the analysis of the image traces.

[0025] According to a second aspect, the image traces comprise multi-scale texture patterns of image features in the input digital image.

[0026] According to a second aspect, the method is performed by an image processing engine comprising a machine learning based processing engine.

[0027] According to a second aspect, the image processing engine includes a global texture representation module for obtaining relationships and differences in the multi-scale texture patterns in order to determine whether the image features are generated by a computational process or by photographic means.

[0028] According to the second aspect, the global texture representation module is combined with a ResNet architecture.

[0029] According to the second aspect, the global texture representation module includes at least one convolutional layer, a Gram matrix-based activation layer, and a global pooling layer.

[0030] According to the second aspect, the image processing engine further comprises a texture enhancement module for amplifying texture differences of the multi-scale texture pattern.

[0031] According to a second aspect, the texture enhancement module is used to amplify the distinguishing traces associated with the image features, thereby facilitating the global texture representation module to obtain the relationship and difference of the multi-scale texture patterns.

[0032] According to the second aspect, an affine transformation operation guided by a semantic segmentation map and a texture restoration based on a convolutional neural network are used to amplify the difference traces.

[0033] According to a second aspect, the texture enhancement module comprises at least one convolutional layer, a semantic segmentation map guided residual block, an affine transformation module and an upsampling module.

[0034] According to a second aspect, the image processing engine further comprises a deep parsing network for generating a segmentation map for an affine transformation operation guided by the semantic segmentation map.

[0035] According to the second aspect, the deep parsing network is further used to generate an intermediate spatial feature transformation map and a feature map associated with the image features for further processing by the affine transformation module.

[0036] According to a second aspect, the image processing engine further comprises an attention-based feature perception module for facilitating trace detection in spatial and channel dimensions.

[0037] According to the second aspect, the attention-based feature perception module includes a convolutional layer, an average pooling layer, and a channel-spatial attention module.

[0038] According to the second aspect, the channel-spatial attention module includes a channel attention submodule sequentially connected to a spatial attention submodule.

[0039] According to the second aspect, the image processing engine also includes a fully connected layer and a Softmax layer, which is used to determine the output probability indicating whether the input digital image is a computer-generated image or a natural photographic image based on the cascaded high-level features obtained by the global texture representation module, the texture enhancement module and the attention-based feature perception module.

[0040] According to a second aspect, the image traces include texture disturbances, high frequency residuals or global space traces in the input digital image.

[0041] According to a second aspect, the computer-generated image is generated by geometric data modeling, photo-realistic rendering, or based on an artificial intelligence generative model.

[0042] According to a second aspect, the image processing engine is trained by providing computer-generated images and a plurality of natural photographic images as positive and negative samples so as to train the image processing engine in a machine learning process, wherein the natural photographic images are generated by a digital camera.

[0043] According to a second aspect, the negative samples and the positive samples include natural photographic images and computer-generated images with added image noise and / or compression traces, respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Embodiments of the present disclosure will be described below by way of examples with reference to the accompanying drawings.

[0045] Figure 1 Schematic diagram of a computer server according to an embodiment of the present disclosure, the computer server being configured to be implemented as a system for detecting computer-generated images.

[0046] Figure 2 FIG. 4 is a block diagram of a system for detecting a computer-generated image according to an embodiment of the present disclosure.

[0047] Figure 3 Figure 2 is a diagram of a typical DM image conversion.

[0048] Figure 4 The overall structure of the system for detecting computer-generated images, namely the MDTL-NET detection network.

[0049] Figure 5 is a diagram of the structure of the depth texture enhancement module.

[0050] Figure 6 is an illustration of the residual block. P1 and P2 represent the transformation parameters obtained based on the generated feature map.

[0051] Fig. 7A A set of visualizations of deep texture enhancement and high-frequency components of the original and enhanced images. From left to right, the columns represent the original image, semantic segmentation image, enhanced image, and high-frequency components of the original and enhanced images. The original image is obtained from the PG dataset. The corresponding zoomed version of the content marked by the red box is also shown in the corner of the image.

[0052] Figure 7B Shown with Fig. 7A Different alternative examples of deep texture enhancement of the original image obtained from the PG dataset and visualization of the high frequency components of the original and enhanced images.

[0053] Figure 7C Shown with Fig. 7A and 7B Alternative examples of deep texture enhancement of the original image obtained from different CG datasets and visualization of the high frequency components of the original and enhanced images.

[0054] Fig.7D Shown with Figure 7C Alternative examples of deep texture enhancement of the original image obtained from different CG datasets and visualization of the high frequency components of the original and enhanced images.

[0055] Figure 8 Illustration of the channel attention submodule.

[0056] Fig. 9 Illustration of the spatial attention submodule.

[0057] Fig. 10A is a graph showing performance evaluation using JPEG compression.

[0058] Fig. 10Bis a graph showing performance evaluation using added noise.

[0059] Fig.11A Figure 2 is a graph showing the detection accuracy for cross-modal image detection, where the model is trained on TCG and tested on GAN.

[0060] Fig. 11B FIG. 5 is a graph showing the detection accuracy of cross-modal image detection, where the model is trained on TCG and tested on DM.

[0061] Fig. 11C Figure 2 is a graph showing the detection accuracy for cross-modal image detection, where the model is trained on DM and tested on GAN.

[0062] Fig.11D Figure 2 is a graph showing the detection accuracy for cross-modal image detection, where the model is trained on GAN and tested on DM. DETAILED DESCRIPTION

[0063] Without wishing to be bound by theory, the inventors have concluded through experiments, trials and research that computer-generated (CG) images can be divided into two categories, namely traditional CG (TCG) and AI-generated (AIG) images. The generation of TCG images can rely on geometric data modeling and photo-realistic rendering, while the generation of AIG images can be based on artificial intelligence techniques such as GAN, autoregressive models, flow and DM.

[0064] Some approaches for detecting CG images may focus on active forensic techniques, such as digital watermarking, where a watermark is embedded into an image before it is transmitted. Authenticity can be verified by comparing the extracted code with the original inserted code. However, these techniques require embedding a specific code when the picture is taken, and face the challenge of balancing invisibility and robustness.

[0065] Alternatively, one can focus on passive (blind) forensics for fake image detection, which relies only on the intrinsic traces left in the digital medium due to the image generation process. In an example detection method, the process can involve exploring hand-crafted features or statistical analysis to represent different intrinsic traces in CG and PG images, such as local binary patterns (LBP), color, texture, and shape features. However, traditional features have difficulty handling complex images with heterogeneous origins.

[0066] Deep learning can be used to improve the performance in various vision applications as well as CG image detection. However, deep learning methods may also overfit when not trained with enough representative data. In addition, the deepest features are automatically learned from the original image. Due to the lack of in-depth research on the image acquisition process, how to generate different intrinsic traces and how to effectively represent these intrinsic traces have been ignored in the field of CG image detection.

[0067] refer to Figure 1 , showing an embodiment of the present disclosure. This embodiment provides a system for detecting computer-generated images. The system includes: an image processing engine, which is used to analyze an input digital image, which is embedded with image traces created during generation and / or post-generation processing operations of the input digital image, and determine whether the input digital image is a computer-generated image or a natural photographic image based on the analysis of the image traces.

[0068] In this exemplary embodiment, the image processing engine is implemented by a computer with an appropriate user interface. The computer can be implemented by any computing architecture, including: a portable computer, a tablet computer, a stand-alone personal computer (PC), a smart device, an Internet of Things (IoT) device, an edge computing device, a client / server architecture, a "dumb" terminal / host architecture, a cloud computing-based architecture, or any other appropriate architecture. The computing device can be appropriately programmed to implement the present disclosure.

[0069] The system can be used to receive a digital image, such as a JPEG file having multiple image features present on the image, such as various objects and scenery under different lighting conditions. These features can be created by recording light using a suitable image acquisition device (e.g., a digital camera), or as a computer rendering. The image processing engine can analyze all image features and all inherent traces associated with the image features to determine whether the input digital image is a computer-generated image or a natural photographic image. For example, an "ideal" image does not include traces that should appear on a natural photographic image, such as noise that cannot be avoided due to the operation of the image sensor in a digital camera or image distortion caused by the imperfect focus of the camera lens. Using this system for detecting computer-generated images, CG images or "pseudo (forged)" images can be detected, which is useful for identifying false information to prevent fraud.

[0070] The inventors have found that there are still four major drawbacks for practical applications:

[0071] 1) In some example methods, the inherently different generation mechanisms between CG and PG images are not described or fully considered.

[0072] 2) Due to the lack of comprehensive consideration of TCG and AIG images, some example methods show obvious shortcomings.

[0073] 3) Some methods directly apply data-driven mechanisms to detection without designing latent modules to drive the model to focus on distinguishing traces.

[0074] 4) The increasingly realistic CG images make detection more difficult, while there is a lack of large-scale and highly dispersed datasets containing different types of heterogeneous CG images for the evaluation of detection methods.

[0075] To compensate for these shortcomings, the acquisition differences between PG and CG images are taken into account. Instead of learning features directly from the original images, a global texture representation module can be used to capture the relationships and differences of multi-scale texture patterns, and a texture enhancement module can be used to simultaneously amplify the distinguishing features by adopting a deep texture enhancement mechanism guided by a semantic segmentation map. In this way, the network can effectively learn representative information of texture perturbations and global traces in the image. To fully evaluate the detection performance, a new dataset DSGCG is also provided, which is characterized by large data size and high diversity in terms of resolution range, lighting conditions, image sources, and image scenes.

[0076] like Figure 1, a schematic diagram of a computer system or computer server 100 is shown. The computer system or computer server 100 is used to implement an example embodiment of a system for detecting computer-generated images. In this embodiment, the system includes the server 100. The server 100 includes appropriate components required to receive, store and execute appropriate computer instructions. These components may include: a processing unit 102, which includes a central processing unit (CPU), a math co-processing unit (Math Processor), a graphics processing unit (GPU), or a tensor processing unit (TPU) for tensor or multi-dimensional array calculation or manipulation operations; a read-only memory (ROM) 104; a random access memory (RAM) 106; an input / output (I / O) device (e.g., a disk drive 108); an input device 110 (e.g., an Ethernet port, a USB port, etc.); a display 112, such as a liquid crystal display, a light-emitting display, or any other suitable display; and a communication link 114. The computing device 100 may include instructions that may be stored in the ROM 104, the RAM 106, or the disk drive 108 and executed by the processing unit 102. A plurality of communication links 114 may be provided that may be variously connected to one or more computing devices, such as servers, personal computers, terminals, wireless or handheld computing devices, IoT devices, smart devices, and edge computing devices. At least one of the plurality of communication links may be connected to an external computing network via a telephone line or other type of communication link.

[0077] The server 100 may include storage devices such as a disk drive 108, which may include a solid-state drive, a hard disk drive, an optical drive, a tape drive, or a remote or cloud-based storage device. The server 100 may use a single disk drive or multiple disk drives, or a remote storage service 120. The server 100 may also have a suitable operating system 116 resident on the disk drive or in ROM of the server 100.

[0078] The computer or computing device may also provide the necessary computing power to operate or interface with a machine learning network such as a neural network to provide various functions and outputs. The neural network may be implemented locally, or may be accessed or partially accessed through a server or cloud-based service. The machine learning network may also be untrained, partially trained, or fully trained, and / or may also be retrained, modified, or updated over time.

[0079] refer to Figure 2 , shows an embodiment of a system 200 for detecting computer-generated images. In this embodiment, the server 100 is a part of the system 200, and the system 200 is used to: receive an input digital image 202; analyze the digital image 202 by, for example, identifying image traces created during the generation of the digital image 202, such as noise signals from an image sensor or other electronic devices, or image traces created during post-generation processing operations such as image compression, noise suppression, distortion compensation, etc.; and finally provide an output 204, which indicates whether the input digital image 202 is a computer-generated image or a natural photographic image based on the analysis of the image traces, such as texture disturbances, high-frequency residuals, or global spatial traces in the input digital image 202.

[0080] For example, a CG image having "lossless" or "perfect" rendered image characteristics is provided for analysis, and the image processing engine 206 may not be able to locate any image traces typically introduced by a digital camera. This is because the digital image 202 may be generated by geometric data modeling, photo-realistic rendering, or based on an artificial intelligence generative model. The system 200 may provide an output 204 indicating that the input digital image 202 is a CG image rather than a PG image. Alternatively, a PG image having the above-mentioned image traces may be provided for analysis, and the system may alternatively provide an output 204 indicating that the input digital image 202 is a PG image.

[0081] As mentioned before, passive forensics for CG image detection can be classified into two groups according to the feature extraction strategy, namely, hand-crafted feature-based and deep learning-based.

[0082] For detection based on handcrafted features, methods in this category can include extracting unusual statistical traces left by a specific graphics generation module and using a threshold-based evaluation to detect computer-generated images, for example, by revealing certain physical differences between the two image classes, such as gamma correction in photographic images and sharp structures in computer graphics, and then designing object geometric features for detection. This method can achieve a classification accuracy of 83.5%, which is better than classification methods based on cartoon features and wavelet features (accuracies of 71.0% and 80.3%, respectively). Alternatively, several visual features derived from color, edge, saturation and texture features with Gabor filters as discriminative features can be used.

[0083] Some approaches emphasize that image acquisition in digital cameras is fundamentally different from the generative algorithms deployed for computer-generated images. Wavelet-based denoising filters are designed for residual image detection. For example, photo-realistic TCG images may be more realistic and smoother than natural images, so it is possible to exploit image perceptual properties to detect CG images.

[0084] Another branch of methods in this category develops texture-based methods for automatic CG and PG classification. For example, local binary patterns with uniform grayscale invariance can be used, and 95.1% accuracy can be achieved using support vector machines. Alternatively, 31-dimensional statistical and texture features can be combined to identify the acquisition pipeline of digital images, and performance can be improved by constructing 9-D histogram features and 9-D multifractal spectral features to represent different textures. In addition, the distribution of histograms, quaternion wavelet transform features, imaging and visual features, etc. can be used to detect CG images.

[0085] Handcrafted feature based methods provide credible theoretical explanation capabilities, but they face the following two challenges: 1) the manual design of handcrafted features is tedious and limited by the feature description capabilities; 2) these features tend to have poor robustness to data with high diversity in content, acquisition devices, and forgery operations.

[0086] On the other hand, deep learning-based detection can be employed, for example through automatic and adaptive feature learning. For example, in the CG image detection task, statistical feature extraction can be integrated into the CNN framework to find the best features for binary classification. Alternatively, a convolutional neural network trained on image patches can be constructed, and an accuracy of 98.5% can be achieved. To further improve the performance, a network with two cascaded convolutional layers at the bottom of the CNN can be designed. The network can be easily adjusted to adapt to different sizes of input image patches while maintaining a fixed depth, the stable structure of the CNN, and good forensic performance.

[0087] In an alternative example, a sensor pattern noise (SPN) with a patch-based five-layer model can be used to detect traditional CG images. The results show that a model using three high-pass filters can achieve better results than using only one filter or no filter. Alternatively, a CNN-based model with channel and pixel correlation can be adopted. The key component of the CNN architecture is the autoencoder module, which takes a color image as input to explicitly extract the correlation between color channels. Alternatively, a two-stream convolutional neural network can be used to help the network focus on different noise-based features of CG and PG images, where one stream uses a pre-trained VGG-19 network for trace learning, while the other stream pre-processes the image using three high-pass filters. In an alternative example, a network based on VGG-16 and a convolutional block attention module that has been experimentally verified to achieve 96% accuracy on the DSTok dataset can be used.

[0088] For example, to detect AIG images, a CNN-based method focusing on high-frequency components can achieve an average accuracy of over 98%. Alternatively, representation learning and representation comparison can be utilized to enhance artifact learning for GAN image detection. In addition, standard image classifiers trained only on one specific CNN generator can be generalized to unknown architectures, datasets, and training methods. Another method for checking whether an image follows the noise pattern of a real image can be adopted. Other methods based on spatial information and methods based on frequency information can perform well in specific scenarios. However, these methods can only detect CG images generated by GAN, ignoring the performance of images generated by DM or TCG images.

[0089] Preferably, the present disclosure provides a three-class classification solution for PG, TCG, and GAN images, which can provide more specific image source information, although images generated by more realistic diffusion models may not be considered in exemplary embodiments. In addition, it seems more preferred to determine whether the image was actually taken in the real world, and it is more practical to distinguish between PG and non-PG in real-world scenes.

[0090] In a typical PG image acquisition process, an optical lens first transmits the light reflected from the scene to a color filter array (CFA), which can be a specific color mosaic that allows each pixel to collect only one specific wavelength of light. The output signal is then sent to an imaging sensor (e.g., charge-coupled device, complementary metal oxide semiconductor) consisting of an array of photodetectors, each of which corresponds to a pixel of the final image. Thereafter, an analog-to-digital converter (ADC) converts the analog signal into digital form. In addition, a demosaicing method is applied to the raw data for interpolation.

[0091] For kernel-based interpolation methods, the process can be expressed as Where V c , Denote the output and original color signals, respectively. c∈{R,G,B}, h(u,v) is the linear filter kernel function, and N represents the kernel size. After recording the color values, more color processing operations (e.g., white balance, gamma correction) are performed. Finally, the image data is compressed to reduce the cost of storage or transmission.

[0092] It can be observed that two levels of traces can be introduced in the generation of PG images, namely the hardware acquisition level (traces caused by the lens, sensor) and the software processing level (traces left by CFA interpolation and other inherent image regularities). Therefore, the final trace in the PG image can be formulated as ψ PG ={ψ ha ,ψ sp}, where ψ ha and ψsp Represent the traces caused by hardware acquisition and software processing artifacts, respectively.

[0093] In fact, ha and ψ sp The traces in can have different representations. For example, for a particular camera with image ψ ha The pattern trace in can be estimated as follows: Among them, PN i =Img i -DN(Img i ), PN i represents the pattern noise of the i-th image, and DN(·) represents the denoising operation. sp The lossy compression trace in can be represented by ψ ct =RT(IDT(D×Q))-IDT(D×Q), where RT(·) represents rounding and truncation operations; Q and D represent quantization parameters and coefficients respectively; IDT(·) represents inverse discrete cosine transform. All these traces will eventually affect the values ​​of image pixels in a specific area.

[0094] Unlike the PG image acquisition process, CG images are directly generated by computer-based algorithms for synthesis and manipulation. Taking the DM-based image generation process as an example, the diffusion model is a parameterized Markov chain trained using variational inference to produce samples that match data after a finite time.

[0095] refer to Figure 3 , DM is based on a forward diffusion phase and a reverse (backward) diffusion phase. In the forward diffusion phase, the input data is gradually perturbed over several steps by adding Gaussian noise. In the reverse diffusion phase, the diffusion process is gradually reversed by learning, so that the generative model can complete the task of recovering the original input data from the diffused (noisy) data. Diffusion models can include three typical subclasses, namely stochastic differential equations (SDEs), denoised diffusion probabilistic models (DDPMs), and noisy conditional fractional networks (NCSNs). Typically, the forward SDE consists of Modeling, where s and t represent the data and diffusion step respectively. σ(t) represents the random component, Represents a proportion. Inverse SDE calculates log probability The gradient of , so that s moves to the area with high data density p(s). This process can be obtained by In the forward process, DDPM is transformed from a normal distribution The sampling process is performed, where βt<<1 and I represents the identity matrix with the same dimensions as the input image S0. This process can be represented by The inverse process of DDPM also performs iterative sampling from a normal distribution, i.e.,

[0096]

[0097] The forward processing of NCSN simply adds normal noise to the image from the previous step by the following equation,

[0098]

[0099] Its reverse process is different from SDE and DDPM. NCSN adopts annealing Langevin dynamics for sampling.

[0100] According to the above analysis, the inherent traces in the DMG image can be expressed as: ψ DMG =ψ ns +ψ cp , where ψ ns and ψ cp They represent the traces caused by noise perturbation and lossy compression of DMG images, respectively. cp It can be calculated as ψcp=RT(IDT(D×Q))-IDT(D×Q), where RT(·) represents rounding and truncation operations; Q and D represent quantization parameters and coefficients respectively; IDT(·) represents inverse discrete cosine transform.

[0101] In fact, the process of acquiring CG images can be seen as a process aimed at solving an optimization problem. std(ψ CG ,ψ PG )<ε,ε>0. * represents the optimal output CG image, ET CG ,ET PG Represent the entities of CG and PG images respectively. CG and ψ PG Represent the generation traces in CG and PG images respectively.

[0102] From the above analysis, we can see that due to different generation modules, there are inherent traces in PG images generated by hardware acquisition and software processing, while traces in CG images are mainly generated by software operations. The huge difference in image generation traces in these images will inevitably lead to multi-level perturbations of pixels, which in turn leads to texture differences in local repetitive patterns and their arrangement rules.

[0103] Based on the above analysis, the inherent differences in the acquisition process lead to inherent traces in CG and PG images, which can be reflected in texture perturbations in particular. Preferably, a novel data-driven method is provided that applies multi-scale deep texture learning to CG image detection, where the image traces include multi-scale texture patterns of image features in the input digital image.

[0104] Preferably, refer to Figure 2, the image processing engine 206 includes a processing engine based on machine learning, and the image processing engine 206 may include three main modules, namely a global texture representation module 208, a texture enhancement module 210, and an attention-based feature perception module 212. The image processing engine may be trained by providing a training data set 214 including a plurality of computer-generated images 220 and a plurality of natural photographic images as positive samples 216 and negative samples 218, so as to train the image processing engine 206 in the machine learning process, wherein the natural photographic images 222 are generated by a digital camera.

[0105] Preferably, the global texture representation module 208 is used to obtain the relationship and difference of multi-scale texture patterns so as to determine whether the image features are generated by a computational process or by photographic means; the texture enhancement module 210 is used to amplify the distinguishing traces associated with the image features, thereby facilitating the global texture representation module 208 to obtain the relationship and difference of multi-scale texture patterns; the attention-based feature perception module 212 is used to facilitate trace detection in spatial and channel dimensions.

[0106] In an example embodiment, in order to fully utilize the trace information for CG image recognition, a robust deep learning model (also referred to as "MDTL-NET" 400) composed of heterogeneous structures for this task can be constructed. Figure 4 In the example of , the process can start by utilizing a deep parsing network for semantic segmentation, and the segmentation map is used to generate a spatial feature transform and the resulting feature map, which is further forwarded to the deep texture enhancement module 210 for texture difference amplification. Synchronously, the input image 202 is also fed into the remaining branch combined with the GTRM 208 for global texture representation. Next, the high-frequency components of the original image 202 and the enhanced image are forwarded to the attention-based feature perception module (AFPM) 212, which contains a Conv-Pool block (convolutional layer followed by an average pooling layer) and a channel-spatial attention module (CSAM) for learning representative information. The output high-level features are cascaded and fed into the fully connected layer and the Softmax layer to obtain the output probability 204 indicating whether the input image is a PG image or a CG image.

[0107] The global texture representation module 208 and the deep texture enhancement module 210 are described in detail below. Specifically, convolutional layers with kernel sizes of 5×5, 1×1, and stride 1, followed by batch normalization (BN) and rectified linear unit 6 (ReLU6) activation functions are added to AFPM. Batch normalization and ReLU6 are used to prevent network overfitting and increase the nonlinearity of the network, respectively. The element-by-element function of ReLU6 can be expressed by the formula ReLU6(i)=min(max(0,i),6), where i represents the input signal. After the three components, an average pooling layer is used instead of a maximum pooling layer to focus more on local features and facilitate learning of computer-generated traces. The process can be expressed as:

[0108] FM out =AP(min(max(0,[Conv(FM in )),6)) (1)

[0109] FM out and FM in denote the output and input feature maps, respectively. [·] denotes the elements extracted after batch normalization, and AP(·) denotes the average pooling operation.

[0110] Since the inherent traces generated are not as significant as the image content information, an attention mechanism consisting of two main parts is used to help learn this trace information. The first part is a channel attention scheme (ATc(·)) that weights the input features of different channels (FMc in). The second part is a spatial attention scheme (ATs(·)), and a lightweight and flexible convolutional block attention module similar to the spatial attention module can be used to improve the representation of interest. The method can be expressed as follows:

[0111] FM s out =ATs(ATc(FM c in )) (2)

[0112] FM c in represents the input of the channel attention submodule, FM s out Represents the output of the spatial attention submodule.

[0113] After the attention module 212, another Conv-Pool block is added to the AFPM. For the convolutional layers, the receptive field in this block can be restricted by using a 1×1 kernel size with a 1×1 stride.

[0114] In terms of the output of the framework, it is shown in Formula 1. 3. The refined features are connected for classification, and the result of whether the input image is CG or PG can be reported in the form of 0 or 1 through the fully connected layer and the Softmax layer, where 1 represents a CG image and 0 represents a PG image.

[0115] Output=SoM(FC(Concat(ft1,ft2,ft3))) (3)

[0116] Among them, ft1, ft2, and ft3 represent refined high-level features. FC() represents a fully connected operation. SoM() represents a Softmax operation.

[0117] Preferably, the GTRM 208 can be constructed to enhance the texture feature representation. Figure 4 Preferably, the global texture representation module 208 is combined with a ResNet architecture. For example, a ResNet-18 equipped with 6 GTRMs 208 can be inserted into the MDTL-NET 400. The GTRM 208 includes at least one (preferably three) convolutional layers, a Gram matrix-based activation layer, and a global pooling layer. Batch normalization (BN) and rectified linear units can be applied after each of the last two convolutional layers. The global texture representation process can be summarized as:

[0118] TE(F G in )=AP(Cv(Cv(Gm(Cv(F G in ))))) (4)

[0119] Where Gm(·) represents the activation based on the Gram matrix, F G in Represents the input feature map.

[0120] The activation performs the following calculations to achieve a good description of the global texture.

[0121]

[0122] Among them, F pt (w) and F qt (w) denotes the t-th element in the h-th and q-th feature maps of layer w, respectively.

[0123] For a given feature map of layer w, the pixel-level elements can be expressed as:

[0124]

[0125] Where m ij Represents the value of the pixel at (i, j). I and J represent the row and column in the image respectively.

[0126] Then, the activation descriptor for the input feature map can be calculated as:

[0127]

[0128] The descriptor enhances texture feature learning by simulating the gray-level co-occurrence matrix-based texture analysis method.

[0129] In addition, texture difference amplification can facilitate CG image detection, and preferably, the image processing engine 206 further includes a texture enhancement module 210 for amplifying texture differences of multi-scale texture patterns. For example, the texture enhancement module 210 is used to amplify the difference traces associated with image features, so as to facilitate the global texture representation module 208 to obtain the relationship and difference of the multi-scale texture patterns.

[0130] In a preferred embodiment, reference Figure 5 The texture enhancement module 210 may include one or more convolutional layers, a residual block guided by a semantic segmentation map, an associated affine transformation and an upsampling module. Preferably, the affine transformation operation guided by the semantic segmentation map and the texture restoration based on the convolutional neural network are used to amplify the difference traces.

[0131] The input and output channels, kernel sizes and strides of the first and last two convolutional layers are (3, 64, 3, 1) and (64, 3, 3, 1), respectively, while the remaining convolutional layers are set to (64, 64, 3, 1), and the four numbers in the brackets represent the values ​​of the corresponding parameters. The 16 residual blocks using a similar structure to the spatial feature transformation network consist of two cascaded affine transformation submodules and two convolutional layers to perform feature-wise processing and spatial transformation. Figure 6 An example structure 600 is shown, the process of which can be formulated as follows:

[0132]

[0133] where AT(·) represents affine transformation, FM r in and FM r out Represent the input and output of the residual block respectively.

[0134] As previously described, the image processing engine 206 may also include a deep parsing network that may be used to generate a segmentation map for semantic segmentation map-guided affine transformation operations, as well as an intermediate spatial feature transformation map and a feature map associated with image features for further processing by an affine transformation module.

[0135] For example, these blocks take as input the feature map from the previous layer and apply two affine transformations that modulate the feature map using a set of parameters (P1, P2) obtained by applying a learnable mapping function MAP based on the segmentation map Θ, i.e., MAP:Θ→(1,2). In summary, the affine transformation process can be summarized as follows:

[0136]

[0137] in denotes element-wise multiplication, and FM denotes a feature map with the same dimension as P1 and P2.

[0138] Preferably, the semantic segmentation map in the segmentation response module can be used to obtain the generated feature map to guide the affine transformation operation. Specifically, a deep parsing network can be used for semantic segmentation. The method introduces high-order relations and mixed label contexts into Markov random fields (MRFs) and solves MRFs by proposing CNNs that produce promising segmentation accuracy on multiple large-scale datasets. The segmentation map θ can then be used as the input of the segmentation response module and then fed into five consecutive convolutional layers. The generated intermediate spatial feature transformation map is processed by two groups of convolutional modules, each group containing two convolutional layers. Then, two generated feature maps shared by the residual block are obtained. It is worth noting that the convolutional layer has a kernel size of 1×1.

[0139] Nearest neighbor upsampling is used at the back end of the enhancement module after the residual block and the additional affine transformation module and after the convolutional layer. In addition, skip connections are used to simplify the training process. In an example embodiment, adversarial learning is also used to make the enhanced image as realistic as possible. A VGG type network is used as a discriminator. The enhancement module and the discriminator are trained with the following adversarial loss L ad =∑ i log(1-D(G(x))) and learning objective joint training:

[0140]

[0141] where pr and po represent the distribution of enhanced image and original image respectively. G(·) represents the enhancement module.

[0142] For clarity of explanation, use Figures 7A to 7DThe effectiveness of deep texture enhancement is demonstrated by the example in , which shows the visualization of deep texture enhancement and the high-frequency components of the original and enhanced images. As can be seen from the third column in each sub-figure, after semantic segmentation, objects of different categories in CG and PG images are well segmented. Based on the segmentation map, the texture of the original image becomes more fine-grained after the enhancement operation. After high-pass filtering, the high-frequency components of the enhanced image and the original image show some differences, and the former presents more detailed information. In addition, it can be found that Figures 7A to 7B The intensity of the high-frequency component of the PG image in Figures 7C to 7D The intensity of the high-frequency components in the CG images in shows a slight decrease. This finding proves that the enhancement module brings considerable and complementary differences in the high-frequency components of CG and PG images, which is crucial for improving the detection ability of the model.

[0143] In addition, the image processing engine 206 also includes an attention-based feature perception module 212, which is used to facilitate trace detection in spatial and channel dimensions. The attention mechanism is very flexible, which can obtain long-range feature interactions and improve the representation ability of convolutional neural networks. Preferably, the channel-spatial attention module includes a channel attention submodule ATc(·) and a spatial attention submodule ATs(·) connected in sequence, where the output of ATc(·) will be the input of ATs(·). More precisely, the input and output feature maps of the channel attention submodule are represented as FM c in and FM c out The channel attention submodule generates a one-dimensional channel attention map Mc(·) and performs element-wise multiplication on the input features such as . Therefore, FM c out It can be formulated as in Represents element-wise multiplication. Similarly, the spatial attention submodule FM s out The output feature map can be calculated as where Ms(·) represents a two-dimensional spatial attention map.

[0144] Preferably, channel attention is added to the model to learn the weights for each channel. Figure 8 A diagram of the channel attention submodule 800 is shown. The input of the submodule is a combination of individual feature maps of size H×W, which can be expressed as: FM c in =[FM 1 in , FM 2 in ...,FM i in ...,FMc in ], where FM c in represents the input, i and c represent the index and total number of feature maps respectively. Global average pooling is used to compress these feature maps into c feature maps with a spatial dimension of size 1×1. The i-th element of the channel-wise statistic can be calculated as follows:

[0145]

[0146] where v i (h,w) represents the value at position (h,w).

[0147] Then, the channel-by-channel statistics are channel-downscaled using a convolutional layer with a kernel size of 1×1. With the set scaling ratio r, the output statistics of size 1×1×Cr can be obtained. Furthermore, the scaled-down statistics are scaled up by the second convolutional layer with a scale of r. After obtaining the restored feature map of size 1×1×C, it is fed to the Sigmoid gate, which further outputs the final channel attention map. Finally, the input features are multiplied by the final one-dimensional channel weight Mc(FM c in ) to obtain the refined feature FM. In summary, the channel attention process can be summarized as follows:

[0148]

[0149] Where Sig(·) is the Sigmoid function, which can be expressed by fsig(x)=1 / (1+e x ) represents the convolution operation. AP(·) represents the average pooling operation.

[0150] Different from the channel attention sub-module, the spatial attention sub-module focuses on where the information is, which can provide information complementary to that of the channel attention mechanism. Fig. 9 900 is shown. First, a hybrid pooling operation consisting of average pooling and maximum pooling along the channel axis can be employed to summarize the average presence of features and the most activated presence of features, respectively. Then, these two merged features can be concatenated to produce an effective feature descriptor, which is further convolved by a standard convolutional layer. This output is also fed to a sigmoid gate in order to generate a two-dimensional spatial attention map Ms(FM c out ). In some example scenarios, the spatial attention map encodes where emphasis or suppression is needed. Similarly, the process can be summarized as:

[0151] Ms(FM c out)=Sig(Conv([AP(FM c out ),MP(FM c out )])) (13)

[0152] where AP(·) and MP(·) represent average pooling and maximum pooling operations, respectively.

[0153] In addition, the image processing engine 206 also includes a fully connected layer and a Softmax layer, which is used to determine the output probability indicating whether the input digital image 202 is a computer-generated image or a natural photographic image based on the cascaded high-level features obtained by the global texture representation module 208, the texture enhancement module 210 and the attention-based feature perception module 212.

[0154] In an example embodiment, following the channel attention and spatial attention submodules, a Conv-Pool block is added to each channel. Finally, the classification result can be obtained through a fully connected layer and a Softmax layer.

[0155] The inventors conducted experiments to evaluate the performance of the system according to the embodiments of the present disclosure. The experiments were conducted on a workstation with an Intel(R) Core(TM) i7-10700 CPU (2.90GHz) processor, an NVIDIA GeForce RTX 3060 graphics card, and a 64G DDR4 2666MHz memory.

[0156] The performance of the system is evaluated on a newly constructed dataset with high data diversity (called DSGCG) and three public datasets (i.e., DSRah dataset, DSTok dataset, and DSMan dataset). The image processing engine is trained by providing computer-generated images and multiple natural photographic images as positive and negative samples in order to train the image processing engine in the machine learning process, where the natural photographic images are generated by digital cameras.

[0157] DSGCG dataset: There are 42,000 CG images and 42,000 PG images in the DSGCG dataset with sizes ranging from 256×256 to 4928×3264, with medium to good visual quality. The CG set contains traditional CG images, GAN images, and DM images. The TCG image set is constructed by collecting CG videos or game screenshots (i.e., forzahorizon, GANTZ, God of War, Red Dead Redemption, and Playerunknown's Battlegrounds) from Kaggle and websites. GAN images (total: 18,000, each GAN image contains 6,000 images from 6 categories) are generated using pre-trained StyleGAN2, ProGAN, and BigGAN models. 18,000 (=6,000+6,000+6,000) DM images are generated using stable diffusion, latent diffusion, and DALL E Mini, respectively.

[0158] For the images in the PG group, 402 images with different contents were taken using a NIKON D5200 camera, with a focal length and exposure time of 25 mm and 1 / 200 second, respectively. 1600 and 5998 images were downloaded from the Columbia and RAISE datasets, respectively. 24,000 images were downloaded from the LSUN dataset. Since grayscale images are also widely used in real-world scenes, 10,000 grayscale PG images from the BOSSbase v1.01 dataset were also collected to enhance the diversity of images. Due to inconsistent sizes, the images were cut to 256×256 and divided into training, test, and validation sets in a ratio of 5:1:1. During the cropping process, no lossy compression traces were introduced to these images. In the experiment, CG images were defined as positive samples and PG images were defined as negative samples. However, those skilled in the art will appreciate that the definitions of positive or negative samples / results can be interchanged as needed.

[0159] DSRah dataset: This dataset consists of 1800 CG images downloaded from the Level Design Reference Database 3. The authors selected 5 different video game screenshots (i.e., Witcher 3, Battlefield 4, Battlefield BadCompany 2, Grand Theft Auto 5, and Uncharted 4) to construct the dataset. The PG set consists of 1800 natural images obtained from the RAISE dataset.

[0160] DSTok dataset: This dataset contains 4850 CG images and 4850 PG images collected from the Internet. The PG images include indoor and outdoor scenes acquired by different devices, and the CG content also contains different scenes. All images are in JPEG format and the file size is between 12KB and 1.8MB.

[0161] DSMan dataset: This dataset contains 4000 TCG images, 4000 GAN and 4000 PG images. GAN images are generated using a pre-trained model. PG and TCG images are selected from the Computer Graphics and Photo dataset.

[0162] The datasets are summarized in the table below. It can be seen that the DSGCG dataset has more images and is more diverse in terms of resolution range, image sources, and scene coverage than other datasets.

[0163]

[0164] The image processing engine was built using Pytorch-1.11.04 with CUDA-11.3 and Torchvision0.12.0 packages and trained with the Adam optimizer with a learning rate of 0.00001. The random horizontal and vertical flip probabilities were set to 0.3. The batch size was set to 30 and the maximum number of epochs was fixed to 400. The network was trained using cross entropy loss.

[0165] The accuracy (ACC), true positive rate (TPR) and true negative rate (TNR) are used for performance evaluation and can be expressed as:

[0166] and In the formula, TP, TN, FP, and FN represent the number of actually positive, actually negative, incorrectly identified, and incorrectly rejected samples, respectively. P and N represent the number of positive and negative samples.

[0167] The following table summarizes the performance of the system on four CG image detection datasets, including DSGCG, DSRah, DSTok and DSMan datasets.

[0168]

[0169] The results show that the system can achieve high accuracy on the DSGCG, DSRah, DSTok, and DSMan datasets. The average accuracy of the system is 96.04%. For the performance of the DSGCG dataset, MDTL-NET achieves an average accuracy of 95.38%. It can also be observed that in most cases, the TPRs and TNRs of the system are quite promising, with values ​​of (95.44%, 95.32%), (96.78%, 97.28%), (96.15%, 97.01%), and (94.28%, 96.02%) on the DSGCG, DSRah, DSTok, and DSMan datasets, respectively, which means that the detection capabilities of positive and negative samples are relatively balanced.

[0170] Advantageously, the high performance of the system can be attributed to the enhanced image texture representation and texture difference amplification, which facilitates MDTL-NET to effectively learn discriminative trace information instead of focusing on image content information.

[0171] CG images may be misclassified as PG images, or PG images may be misclassified as CG images. It can be found that changes in lighting may affect the performance of the method. If the scene contains large lighting or color changes, it may mislead the detection network to output wrong recognition results. In summary, these results still clearly demonstrate the effectiveness of CG recognition.

[0172] To further illustrate the contribution of different modules in the detection model, the effects of different parameter or network structure settings on the detection performance of the DSGCG dataset are given. Specifically, the following settings are used to compare the detection performance: no high-pass filtering, no random horizontal / vertical flipping, different arrangements of attention submodules (channel only, spatial only, spatial+channel), different learning rates, texture enhancement disabled, and the use of the average stochastic gradient descent (ASGD) optimizer. When modifying a module, the other settings can remain the same. For the two cases of no high-pass filtering and no texture enhancement, these modules can be removed and no modifications are made to the other modules. The comparison results are summarized as follows.

[0173]

[0174]

[0175] Without high-pass filtering, a detection accuracy of 91.70% is observed. This indicates that the filtering operation is effective in acquiring high-frequency components, suppressing interference from image content, and facilitating learning of distinguishable traces. For scenes without random horizontal / vertical flipping, the detection accuracy reaches 94.58% and 94.07%, respectively. These results indicate that data augmentation can increase the size and quality of training datasets, allowing for the construction of better deep learning models. In addition, when the attention submodules are connected in other ways, the performance degrades, and using two submodules simultaneously outperforms using a single submodule. The better performance of the combination of modules in the system can be attributed to the stronger representation of the inherent trace information. By utilizing both channel and spatial attention submodules, the network has the ability to learn “what to emphasize” and “where to emphasize”, which improves the intermediate features and effectively facilitates trace detection.

[0176] It can also be observed that too high a learning rate can cause the model to converge to a suboptimal solution too quickly. The Adam optimizer can outperform the method with the ASGD optimizer to some extent. This can be attributed to the stronger ability of the Adam optimizer to accelerate the convergence towards the relevant direction and reduce the fluctuation towards the irrelevant direction in this detection task. Last but not least, the results also show a performance degradation when the deep texture enhancement is disabled. The results show that this submodule helps to improve the detection performance by providing refined and discriminative features.

[0177] In real situations, forgers can perform post-processing operations, such as adding noise and image compression, to suppress the traces of CG and deceive forensic procedures. Therefore, it is of great significance to study the robustness and sensitivity of the detection algorithm to different post-processing operations. In this experiment, two types of post-processing operations (i.e., JPEG compression and adding noise) were performed on the images in the test subset of DSGCG. The quality factor (QF) was set to {90, 70, 50} to achieve different quality levels. Considering the case of randomly setting the QF value, where the image is compressed with an arbitrary QF, the algorithm not only picks up on the compression traces for detection. Two forms of noise commonly seen on digital images, namely Gaussian white noise and salt and pepper noise, were added to the images respectively. The mean value of the Gaussian white noise and the density of the salt and pepper noise were set to 0.01 and 0.02, respectively.

[0178] from Fig. 10A and 10BFrom the results in , it can be seen that despite the different post-processing operations performed, the detection accuracy of the present system is generally higher than that of other examples. As the QF value decreases, the detection accuracy of all methods drops sharply due to the severe compression and loss of JPEG encoding. Although the method achieves the worst results at QF=50, the present system still outperforms the example in this case, with an average accuracy of 69.37%, which is 2.87% higher than the second best method. It can also be observed that when different noises are added, the present system achieves higher accuracy, averaging 91.20% on the DSGCG dataset. As the mean of Gaussian white noise and the density of salt and pepper noise increase, the accuracy of all methods decreases. This is because higher mean or density values ​​strengthen the interference of noise information.

[0179] It can also be observed that when the value of QF is randomly set, the results are still better than other examples. It is also worth noting that in the scene with added salt and pepper noise (density = 0.01), the system maintains a very reliable detection accuracy of more than 93%. The experimental results show that the system is robust and representative. In summary, the system proves strong robustness to different post-processing operations.

[0180] Optionally, the negative and positive samples include natural photographic images and computer generated images, respectively, to which image noise and / or compression traces are added, such that the image processing engine is trained with a database including samples with these image traces.

[0181] Considering that images from unknown devices can be used for testing in real-world situations, the performance of the present disclosure is evaluated in a cross-dataset case. In this experiment, images from the DSGCG dataset are trained, and the model is tested on images from the DSTok and DSMan datasets. The network is trained for 400 epochs, and the model that gives the highest accuracy is reported. The comparison results are summarized in the following table.

[0182]

[0183] It can be seen that the present disclosure still achieves relatively promising performance in this case. When tested on DSTok, the detection accuracy of the system reaches 88.95%. In terms of TPR and TNR, the accuracy is 85.75% and 92.15%, respectively. This demonstrates the good generalization ability of the system due to the rich and fine features learned by MDTL-NET. Good results can also be obtained when the system is tested on the DSMan dataset.

[0184] The cross-modal generalization ability is also evaluated to show the performance when faced with unseen cross-modal generated images. In one experiment, the system was trained on the TCG dataset and tested on the test sets generated by GAN and DM, respectively, to analyze whether the TCG-trained model can detect AIG images. In addition, based on GAN, DM images are used for training and testing to evaluate the performance when faced with different AIG images. The rest of the experimental settings and procedures are the same as described earlier in this disclosure.

[0185] refer to Figures 11A to 11D , the performance of all methods is generally degraded compared to the in-domain detection performance. This phenomenon can be attributed to the differences between TCG, GAN and DM generation models.

[0186] refer to Fig.11A and 11B , our system still achieves the best accuracy for GAN-generated image testing, achieving an average accuracy of 69.10% on BigGAN image testing, which is 18.08%, 10.23%, 9.88%, 7.80%, 3.64%, 4.09%, 10.13%, 3.38%, and 3.66% higher than the other systems. Since GAN images are generated in different ways, it is difficult for models trained on CG images to accurately distinguish GAN images from natural images. When tested with DM-generated images, our system can also achieve higher accuracy, outperforming the other systems by 12.70%, 10.25%, 4.52%, 10.54%, 7.12%, 2.87%, 6.44%, 4.32%, and 1.54% for the DALL·E Min case.

[0187] Fig. 11C and 11D Results of different AIG image tests using networks trained on different generative model-based images are shown. It can also be observed that the system of the present disclosure outperforms other systems.

[0188] The advantages of these embodiments are that a system for detecting CG and PG images based on image trace analysis is provided. The inventors have devised that a clear understanding of how CG and PG images differ in the generation process is very helpful for exploring more effective methods to enhance and represent the distinguishing traces caused by the acquisition module. Different from other example methods for detecting CG images, the present disclosure emphasizes:

[0189] (1) Clear analysis of different acquisition processes in CG and PG images. The inherent traces of CG and PG images caused by different generation and processing operations are analyzed, which provides a basis for trace learning strategies and also makes up for the lack of theoretical analysis in existing literature.

[0190] (2) An effective module for multi-scale texture representation: Preferably, a global texture representation module (GTRM) can be used to facilitate the learning of multi-scale feature patterns.

[0191] (3) Texture difference amplification. Different from other example strategies of manual or deep learning feature extraction, preferably, a deep texture enhancement module can be used to amplify the difference traces in the image based on affine transformation operations guided by semantic segmentation maps and texture restoration based on convolutional neural networks (CNNs).

[0192] (4) A robust multi-scale deep texture learning network (MDTL-net) for CG image detection. The hybrid neural network can be used for robust CG image detection. By adopting a separation-fusion detection strategy with an attention mechanism, the network can effectively learn representative information of texture perturbations, high-frequency residuals, and global spatial traces in the image.

[0193] (5) Evaluation with outstanding performance on a newly constructed dataset and 3 existing datasets. A large-scale, highly diverse and realistic dataset with 42,000 CG and 42,000 PG images can be used to address the limitations in existing datasets. Different generation techniques such as traditional CG, GAN and DM are considered. Both within-dataset and across-dataset tests verify the performance. In addition, the robustness to post-processing operations and the general ability to detect fake images generated by different generation models are also considered in some examples.

[0194] Although not required, the embodiments described with reference to the accompanying drawings may be implemented as an application programming interface (API) or a series of libraries used by developers, or may be included in another software application, such as a terminal or personal computer operating system or a portable computing device operating system. Generally, since program modules include routines, programs, objects, components, and data files that help perform specific functions, those skilled in the art will understand that the functionality of a software application may be distributed among multiple routines, objects, or components to achieve the same functionality required herein.

[0195] It should also be understood that in the case where the methods and systems of the present disclosure are implemented in whole or in part by a computing system, any suitable computing system architecture may be used. The computing system may include tablet computers, wearable devices, smart phones, IoT devices, edge computing devices, stand-alone computers, network computers, cloud-based computing devices, and dedicated hardware devices. Where the terms "computing system" and "computing device" are used, these terms are intended to cover any suitable configuration of computer hardware capable of implementing the described functionality.

[0196] It will be appreciated by those skilled in the art that various changes and / or modifications may be made to the present disclosure as shown in the specific embodiments without departing from the spirit or scope of the present disclosure as broadly described. Therefore, the present embodiments are to be considered in all aspects as illustrative and not restrictive.

[0197] Unless otherwise indicated, any reference to prior art contained herein is not to be taken as an admission that the information is common general knowledge.

Claims

1. A system for detecting computer-generated images, characterized in that include: An image processing engine for analyzing an input digital image embedded with image traces created during generation and / or post-generation processing operations of the input digital image, and determining whether the input digital image is a computer-generated image or a natural photographic image based on the analysis of the image traces.

2. The system for detecting computer-generated images according to claim 1, characterized in that in, The image traces include multi-scale texture patterns of image features in the input digital image.

3. The system for detecting computer-generated images according to claim 2, characterized in that in, The image processing engine includes a processing engine based on machine learning.

4. The system for detecting computer-generated images according to claim 3, characterized in that in, The image processing engine includes a global texture representation module for obtaining relationships and differences in the multi-scale texture patterns to determine whether the image features are generated by computational processes or photographic means.

5. The system for detecting computer-generated images according to claim 4, characterized in that in, The global texture representation module is combined with the ResNet architecture.

6. The system for detecting computer-generated images according to claim 5, characterized in that in, The global texture representation module includes at least one convolution layer, an activation layer based on a Gram matrix, and a global pooling layer.

7. The system for detecting computer-generated images according to claim 4, characterized in that in, The image processing engine also includes a texture enhancement module for amplifying texture differences of the multi-scale texture pattern.

8. The system for detecting computer-generated images according to claim 7, characterized in that in, The texture enhancement module is used to amplify the distinguishing traces associated with the image features, so as to facilitate the global texture representation module to obtain the relationship and difference of the multi-scale texture patterns.

9. The system for detecting computer-generated images according to claim 8, characterized in that in, A semantic segmentation map-guided affine transformation operation and convolutional neural network-based texture restoration are used to amplify the distinguishing traces.

10. The system for detecting computer-generated images according to claim 9, characterized in that in, The texture enhancement module includes at least one convolutional layer, a semantic segmentation map guided residual block, an affine transformation module and an upsampling module.

11. The system for detecting computer-generated images according to claim 10, characterized in that in, The image processing engine also includes a deep parsing network for generating a segmentation map for an affine transformation operation guided by the semantic segmentation map.

12. The system for detecting computer-generated images according to claim 11, characterized in that in, The deep parsing network is also used to generate an intermediate spatial feature transformation map and a feature map associated with the image features for further processing by the affine transformation module.

13. The system for detecting computer generated images according to claim 7, characterized in that: in, The image processing engine also includes an attention-based feature perception module for facilitating trace detection in spatial and channel dimensions.

14. The system for detecting computer-generated images according to claim 13, characterized in that in, The attention-based feature perception module includes a convolutional layer, an average pooling layer, and a channel-spatial attention module.

15. The system for detecting computer generated images according to claim 14, characterized in that in, The channel-spatial attention module includes a channel attention submodule sequentially connected to a spatial attention submodule.

16. The system for detecting computer generated images according to claim 14, characterized in that in, The image processing engine also includes a fully connected layer and a Softmax layer, which is used to determine an output probability indicating whether the input digital image is a computer-generated image or a natural photographic image based on the cascaded high-level features obtained by the global texture representation module, the texture enhancement module and the attention-based feature perception module.

17. The system for detecting computer-generated images according to claim 1, characterized in that in, The image traces include texture disturbances, high frequency residuals or global space traces in the input digital image.

18. The system for detecting computer generated images according to claim 1, characterized in that in, The computer-generated image is generated through geometric data modeling, photo-realistic rendering, or based on an artificial intelligence generative model.

19. The system for detecting computer generated images according to claim 18, characterized in that in, The image processing engine is trained by providing computer-generated images and a plurality of natural photographic images as positive samples and negative samples so as to train the image processing engine in a machine learning process; wherein the natural photographic images are generated by a digital camera.

20. The system for detecting computer generated images according to claim 19, characterized in that in, The negative samples and the positive samples include natural photographic images and computer-generated images with added image noise and / or compression traces, respectively.

Images