Synthetic image detection method based on cross-modal commonality interaction

Abstract

The application provides a kind of synthetic image detection method based on cross-modal commonality interaction, comprising the following steps: input image is sent into two branches in parallel: spatial domain processing branch directly processes original image and utilizes high-frequency self-enhancement module to strengthen spatial domain artifact;Frequency domain processing branch first extracts high-frequency component through discrete wavelet transform (DWT);Then utilize multi-scale frequency enhancement module to strengthen frequency domain artifact;Two branches independently enhance respective high-frequency information, then fuse through a pooling guided cross-modal high-frequency interaction module;Finally, the classifier outputs the detection result.The application solves the technical problem that in the prior art, either only high-frequency information is utilized in a single mode (spatial domain or frequency domain), or multi-modal information is simply spliced, lacking a mechanism to deeply and interactively fuse high-frequency clues from different modes.

Description

Technical Field

[0001] The present invention relates to the technical field of synthetic image detection, and in particular to a synthetic image detection method based on cross-modal commonality interaction. Background Art

[0002] Existing synthetic image detection techniques can be categorized into three main schools: spatial domain methods, frequency domain methods, and fingerprint methods. Spatial domain methods directly analyze pixel-level image information, such as texture, gradients, and local patterns, to identify forgeries. However, these methods are susceptible to interference from the image content itself, and their learned features generalize poorly to novel generative models not previously seen in the training set. Frequency domain methods utilize tools such as the discrete Fourier transform (DFT) or discrete wavelet transform (DWT) to analyze image anomalies in the frequency domain. These methods exploit the fact that generative models, especially early GANs, are prone to introducing high-frequency artifacts during upsampling. However, their technical limitations lie in the significant variation in the frequency artifact patterns introduced by different generative models. Even some advanced models can generate images with very natural spectra, resulting in poor generalization even for methods that rely solely on the frequency domain. Fingerprint methods attempt to identify unique traces left by specific generator architectures that can serve as "fingerprints." These methods, by their nature, overfit to a specific model, exhibit minimal generalization capabilities, and have limited practical value in real-world scenarios.

[0003] Observations reveal that while these three schools of thought each have their own focus, they fundamentally rely on the "high-frequency information" of images. Whether it's edge texture in the spatial domain or abnormal spectra in the frequency domain, the most discriminative information is concentrated in the high-frequency portion. However, existing technologies either utilize high-frequency information only in a single modality (spatial or frequency domain) or simply concatenate multimodal information. They lack a mechanism for deeply and interactively integrating high-frequency cues from different modalities, which limits their ability to learn universal and robust forgery-resistant features. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a synthetic image detection method based on cross-modal common interaction, so as to solve the technical problems in the existing technology that either only high-frequency information is used in a single modality (spatial domain or frequency domain) or multi-modal information is simply spliced ​​together, and there is a lack of a mechanism to deeply and interactively fuse high-frequency clues from different modalities.

[0005] To achieve the above technical objectives, in a first aspect, the technical solution of the present invention provides a synthetic image detection method based on cross-modal commonality interaction, comprising the steps of:

[0006] Get an input image, and input the input image into the spatial domain processing branch and the frequency domain processing branch in parallel:

[0007] In the spatial domain processing branch, the input image is downsampled and basic features are extracted to obtain a low-dimensional initial feature map, and the initial feature map is subjected to deep feature extraction and enhancement to obtain a complete feature map containing rich structure and high-frequency information;

[0008] Setting an attention weight to be applied to the complete feature map, and performing a residual connection with the input image to obtain a spatial artifact feature map;

[0009] In the frequency domain processing branch, the input image is subjected to discrete wavelet transform to be decomposed into multiple frequency sub-bands, and high-frequency sub-bands are selected as high-frequency features of the frequency domain processing branch;

[0010] The high-frequency features are processed by the multi-scale frequency enhancement module to achieve adaptive capture of diverse frequency artifacts and obtain enhanced frequency domain artifact features;

[0011] The spatial domain features are refined to generate an "artifact hotspot" spatial attention map, and the final spatial domain artifact features are obtained by multiplying the spatial attention map with the spatial domain artifact features;

[0012] The spatial attention map is used to guide and modulate the frequency domain artifact features, and the guided frequency domain artifact features are integrated with the spatial domain artifact features to generate a refined universal artifact feature for image detection.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] This application proposes an innovative synthetic image detection framework based on cross-modal high-frequency information interaction. This detection framework addresses the problem of insufficient single-modal information: instead of viewing the spatial and frequency domains in isolation, the method constructs a dual-branch parallel processing architecture. One branch processes spatial domain information, while the other processes frequency domain information, simultaneously extracting high-frequency artifacts from two different perspectives. This also addresses the issue of inconspicuous high-frequency features: a dedicated high-frequency enhancement module is designed in each branch. In the spatial domain, a high-frequency self-enhancement module is designed to amplify and sharpen high-frequency artifact features; in the frequency domain, a multi-scale frequency enhancement module is designed to capture frequency anomalies at different scales. This ensures that the forged signals used for detection are sufficiently clear and conspicuous. Furthermore, this approach addresses the problem of insufficient cross-modal information fusion: a pooling-guided cross-modal high-frequency interaction module is proposed. Rather than simply concatenating features, this module leverages the more informative high-frequency features from the frequency domain, dynamically and adaptively extracting and fusing the most relevant components from the spatial domain high-frequency features to form a more comprehensive, robust, and discriminative universal forged feature representation. Ultimately, this invention significantly improves the model's ability to detect unknown generative models through this deep cross-modal high-frequency information interaction, solving the key problem of poor generalization of existing technologies.

[0015] According to some embodiments of the present invention, in the spatial domain processing branch, downsampling and basic feature extraction are performed on the input image to obtain a low-dimensional initial feature map, including the steps of:

[0016] The input image passes through a standard convolution block and a maximum pooling layer for downsampling and basic feature extraction to obtain a low-dimensional initial feature map.

[0017] According to some embodiments of the present invention, performing deep feature extraction and enhancement on the initial feature map to obtain a complete feature map containing rich structure and high-frequency information includes the following steps:

[0018] The initial feature map is sent to a series of stacked processing modules for deep feature extraction and enhancement. Each processing module contains several standard residual convolution blocks and an embedded high-frequency self-enhancement module. (Dimensions are B x C x H x W) When entering the high-frequency self-enhancement module, the processing flow is as follows:

[0019] The high-frequency self-enhancement module inputs the initial feature map Along the channel dimension Divide equally into the first part and Part II , Part 1 Send it to a convolutional layer to learn and maintain the local smooth structure and content information of the image, and the output is :

[0020]

[0021] Part 2 It is used to extract and amplify high-frequency details, select the maximum value in each local area through a maximum pooling operation of a maximum pooling layer, and extract the high-frequency response of edges, corners and textures;

[0022] The convolutional layer is used to refine the features and adjust the channels, and the output is :

[0023]

[0024] The features obtained from the two pathways and features Reassemble them in the channel dimension to form a complete feature map containing rich structure and high-frequency information .

[0025] According to some embodiments of the present invention, setting an attention weight to be applied to the complete feature map and performing a residual connection with the input image to obtain a spatial artifact feature map includes the steps of:

[0026] The complete feature map is subjected to global average pooling (GAP) and passed through a convolution and Sigmoid function to generate attention weights. The attention weights are applied to the complete feature map. The enhanced complete feature map is residually connected with the initial feature map to obtain the final output spatial artifact feature map of the spatial domain processing branch. .

[0027] According to some embodiments of the present invention, setting an attention weight to be applied to the complete feature map and performing a residual connection with the input image to obtain a spatial artifact feature map includes the steps of:

[0028] The complete feature map is subjected to global average pooling (GAP) and passed through a convolution and Sigmoid function to generate attention weights. The attention weights are applied to the complete feature map. The enhanced complete feature map is residually connected with the initial feature map to obtain the final output spatial artifact feature map of the spatial domain processing branch. .

[0029] According to some embodiments of the present invention, high-frequency features are processed by a multi-scale frequency enhancement module to achieve adaptive capture of diverse frequency artifacts, thereby obtaining enhanced frequency domain artifact features, including the following steps:

[0030] High-frequency features are frequency feature maps. When a frequency feature map When entering the multi-scale frequency enhancement module, the processing flow is as follows:

[0031] The module uses a dual-path parallel structure to process the frequency characteristic graph :

[0032] In the first pass, two special pooling operations are used: global average pooling along the height direction (GAPH) and global average pooling along the width direction (GAPW); respectively capture the global context information in the horizontal and vertical directions in the feature map, and obtain the first direction vector and the second direction vector ;

[0033] The first direction vector and the second direction vector Concatenate and fuse information through convolution, and then separate into two new vectors;

[0034] After the two new vectors are activated by the Sigmoid function, the attention weights corresponding to the horizontal and vertical directions are formed. The horizontal and vertical attention weights are multiplied element by element back to the original frequency feature map. , as shown in the formula:

[0035]

[0036] In the second path, a standard convolution is used to capture the broad local context in the feature map;

[0037] In order to intelligently fuse the information of the two pathways, a cross-scale interaction mechanism is used to calculate the channel importance weights of the output features of the two pathways respectively. ;The importance weight of the first path Multiply the output feature of the second path (local context path) and weight the importance of the second path Multiply it by the output feature of the first path (direction perception path), and finally add the two together to obtain the enhanced frequency domain artifact feature, as shown in the formula:

[0038] .

[0039] According to some embodiments of the present invention, the spatial domain features are refined to generate a "artifact hotspot" spatial attention map, including the following steps:

[0040] The input spatial artifact features are subjected to a maximum pooling operation;

[0041] Along the channel dimension, the average and maximum values ​​of the spatial artifact features are calculated simultaneously. The average and maximum values ​​represent the “general response intensity” and “peak response intensity” of the features at each spatial position, respectively.

[0042] The response map corresponding to the average value and the response map corresponding to the maximum value of the spatial artifact features are concatenated and passed through a lightweight convolution layer and a Sigmoid function to generate a final spatial attention map.

[0043] According to some embodiments of the present invention, utilizing the spatial attention map to guide and modulate frequency domain artifact features comprises the steps of:

[0044] The final spatial attention map is element-wise multiplied by the frequency domain artifact features from the frequency branch.

[0045] According to some embodiments of the present invention, the guided frequency domain artifact features are integrated with the spatial domain artifact features to generate a refined universal artifact feature for image detection, including the steps of:

[0046] The frequency domain artifact features and the spatial domain artifact features are spliced ​​in the channel dimension to form a wider intermediate feature map that contains the essence of dual modal information;

[0047] Use convolution to process the spliced ​​intermediate feature maps, perform cross-channel information fusion and dimensionality reduction, blend the information of the two modalities together, and generate the final universal forged features.

[0048] In a second aspect, the technical solution of the present invention provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the synthetic image detection method based on cross-modal common interaction as described in any one of the first aspects.

[0049] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, wherein the abstract drawing is identical to one of the drawings in the specification:

[0051] Figure 1 A flowchart of a synthetic image detection method based on cross-modal commonality interaction provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] It should be noted that although the system diagrams illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the system or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0054] Reference Figure 1 , Figure 1 A flowchart of a synthetic image detection method based on cross-modal commonality interaction provided by one embodiment of the present invention.

[0055] In one embodiment, a synthetic image detection method based on cross-modal common interaction includes the following steps: obtaining an input image, inputting the input image into a spatial processing branch and a frequency domain processing branch in parallel: in the spatial processing branch, downsampling and basic feature extraction are performed on the input image to obtain a low-dimensional initial feature map, and deep feature extraction and enhancement are performed on the initial feature map to obtain a complete feature map containing rich structure and high-frequency information; setting an attention weight to be applied to the complete feature map, and performing a residual connection with the input image to obtain a spatial artifact feature map; in the frequency domain processing branch, performing a discrete wavelet transform on the input image to be decomposed into Multiple frequency sub-bands are selected, and high-frequency sub-bands are selected as high-frequency features of the frequency domain processing branch; the high-frequency features are processed by the multi-scale frequency enhancement module to achieve adaptive capture of diverse frequency artifacts and obtain enhanced frequency domain artifact features; the spatial domain features are refined to generate an "artifact hotspot" spatial attention map, and the final spatial domain artifact features are obtained by multiplying the spatial attention map with the spatial domain artifact features; the spatial attention map is used to guide and modulate the frequency domain artifact features, and the guided frequency domain artifact features are integrated with the spatial domain artifact features to generate a refined universal artifact feature for image detection.

[0056] By feeding the input image into both the spatial and frequency domain processing branches in parallel, the present invention can mine image features from different perspectives. The spatial domain processing branch focuses on features related to the image's spatial structure, while the frequency domain processing branch emphasizes features at the frequency level. The combination of the two allows for a more comprehensive and detailed capture of all types of information contained in the image, avoiding the potential omission of important features in single-modal processing, thereby laying a good foundation for accurate detection of composite images.

[0057] The present invention first performs downsampling and basic feature extraction to obtain a low-dimensional initial feature map, and then further deeply extracts and enhances features to obtain a complete feature map. This step-by-step approach helps to gradually refine key features containing rich structures and high-frequency information from the relatively rough original image information, making the features used for subsequent detection more representative and discriminative, and can better distinguish the differences in spatial structure between synthetic images and real images.

[0058] The present invention adopts the residual connection method to connect the complete feature map after applying attention weights with the input image to obtain a spatial artifact feature map. It can effectively utilize the original information contained in the input image itself, avoid the loss of some underlying details useful for detection in the multi-layer feature extraction process, and make the final spatial artifact feature map integrate high-level abstract features and beneficial parts of the original image, thereby improving feature quality.

[0059] This method uses discrete wavelet transforms to decompose the input image and selects high-frequency subbands as high-frequency features. This is then processed by a multi-scale frequency enhancement module to adaptively capture diverse frequency artifacts. This is crucial for synthetic image detection, as synthetic images often leave specific artifacts in the frequency domain during their generation. This method can keenly capture these features, enhancing synthetic image recognition capabilities.

[0060] This method generates a spatial attention map of "artifact hotspots" to extract spatial features and use them to guide and modulate frequency-domain artifact features, achieving interactive collaboration between spatial and frequency-domain features. This cross-modal guidance mechanism allows features acquired from different modalities to complement and reinforce each other, resulting in a combined universal artifact feature that incorporates the most critical and characteristic aspects of the composite image from both modalities, further facilitating accurate composite image detection.

[0061] Finally, the guided frequency domain artifact features and spatial domain artifact features are integrated to generate refined universal artifact features for image detection. This integrated feature concentrates the advantages of the two branches and removes redundant and interfering information. When faced with synthetic images of different types and generation methods, it can provide a more stable and effective detection basis, thereby improving the versatility and accuracy of the detection method.

[0062] Overall process overview: The input image is fed into two branches in parallel:

[0063] The spatial domain processing branch directly processes the original image and uses the high-frequency self-enhancement module to enhance the spatial domain artifacts;

[0064] The frequency domain processing branch first extracts high-frequency components through discrete wavelet transform (DWT);

[0065] Then the frequency domain artifacts are enhanced using a multi-scale frequency enhancement module;

[0066] The two branches independently enhance their respective high-frequency information and then fuse them through a pooling-guided cross-modal high-frequency interaction module;

[0067] Finally, the classifier outputs the detection results.

[0068] Airspace processing branch:

[0069] In order to solve the problem that the spatial domain artifact signal is weak and easily interfered by content, the present invention designs a high-frequency self-enhancement module, which is specifically used to amplify and sharpen the high-frequency artifact features in the spatial processing branch while retaining the necessary structural information.

[0070] Input image First, a standard convolutional block (e.g., 7x7 convolution with a stride of 2, batch normalization layer, and activation function) and a max pooling layer are used for downsampling and basic feature extraction to obtain a low-dimensional initial feature map. This step aims to reduce the subsequent computational effort of the detector and capture the macroscopic outlines and structures of the image to preserve the main information.

[0071] The initial feature map is then fed into a series (e.g., 3) of stacked processing modules for deep feature extraction and enhancement. Each processing module contains several standard residual convolution blocks and embeds the core innovation of this invention, the high-frequency self-enhancement module. The high-frequency self-enhancement module is the soul of the spatial branch, responsible for amplifying and sharpening high-frequency artifacts at the feature level. When a feature map (Dimensions are B x C x H x W) When entering the high-frequency self-enhancement module, the processing flow is as follows:

[0072] First, the module takes as input the feature map Along the channel dimension Divide equally into the first part and Part II (C / 2 channels per part). This separation is the basis for parallel processing and division of labor. is fed into a standard 3x3 convolutional layer. The main function of this path is to learn and maintain the local smooth structure and content information of the image, and prevent the overall understandability of the image from being destroyed when enhancing high-frequency signals. Its output is , as shown in the formula:

[0073]

[0074] Part 2 It is specifically designed to extract and amplify high-frequency details. It first passes through a 3x3 max pooling layer. The max pooling operation selects the maximum value within each local area, which is physically very close to extracting high-frequency responses such as edges, corners, and textures. Subsequently, a 1x1 convolution is used to refine the features and adjust the channels. Its output is , as shown in the formula:

[0075]

[0076] The features obtained from the two pathways and Reassemble them in the channel dimension to form a complete feature map containing rich structure and high-frequency information In order to allow the model to adaptively focus on the most important part (i.e., the artifact area), this paper introduces a lightweight channel attention mechanism. Perform global average pooling (GAP) and pass through a convolution and Sigmoid function to generate an attention weight. This weight will be applied to Finally, the enhanced features are combined with the input of the module The residual connection is performed to obtain the final output of the module. This ensures the stability of the information flow and allows the module to focus only on learning the "residual" part, that is, the high-frequency information that needs to be enhanced. Through such a series of stacking processes, the spatial processing branch can gradually and iteratively extract a high-quality spatial artifact feature map from the original pixel input. , providing a solid foundation for subsequent cross-modal interaction with the frequency branch.

[0077] Frequency domain processing branch:

[0078] The frequency processing branch operates in parallel with the spatial branch, with the core goal of specifically capturing and enhancing abnormal patterns introduced by the forgery process in the frequency domain of the image. Unlike the spatial branch, which processes pixels directly, this branch focuses on high-frequency information from the outset and designs a unique enhancement module to process the idiosyncrasies of frequency domain features.

[0079] Input image First, the image is decomposed into multiple frequency subbands through a discrete wavelet transform. The present invention selects only the high-frequency subband HH as the input for this branch. These high-frequency features then flow through a standard convolution block and into a series of cascaded processing modules. Within each processing module, the features are processed by a multi-scale frequency enhancement module to adaptively capture diverse frequency artifacts. Ultimately, the output after processing by all processing modules is the enhanced frequency domain artifact feature.

[0080] When a frequency characteristic diagram When entering the multi-scale frequency enhancement module, the processing flow is as follows:

[0081] The module also uses a dual-path parallel structure to process input features , in order to achieve information enhancement from different dimensions. In the first path, the present invention uses two special pooling operations: global average pooling along the height direction (GAPH) and global average pooling along the width direction (GAPW). These two operations can capture the global context information in the horizontal and vertical directions of the feature map respectively, and obtain two direction vectors and .

[0082] Will and The information is concatenated and fused through a 1x1 convolution, and then separated into two new vectors. After the two vectors are activated by the Sigmoid function, the attention weights corresponding to the horizontal and vertical directions are formed. The attention weights of these two directions are multiplied element by element back to the original frequency feature map. This process can adaptively amplify frequency components that exhibit anomalies (such as periodic artifacts) in a specific direction. As shown in the formula:

[0083]

[0084] The other path is relatively simple, using a standard 3x3 convolution to capture a broader local context in the feature map. This path focuses on the patterns in the local region in the frequency domain, rather than the global pattern in a specific direction. In order to intelligently fuse the information of the two paths (direction-aware and local context), the present invention designs a cross-scale interaction mechanism. The present invention calculates the channel importance weights of the output features of the two paths separately. Then, the importance weight of a path is Multiply the output feature of another path (local context path) and weight the importance of the other path Multiply it by the output feature of the first path (direction perception path), and finally add the two together. As shown in the formula:

[0085]

[0086] This "cross" fusion method enables the model to dynamically decide whether to focus more on directional artifacts or local pattern artifacts based on the characteristics of the current sample, thereby achieving more flexible and robust feature enhancement.

[0087] Pooling-guided cross-modal high-frequency interaction module:

[0088] The pooling-guided cross-modal high-frequency interaction module does not treat the features of the two branches equally. Instead, based on the prior insight that "frequency domain information is more decisive for detection tasks", it establishes a fusion strategy dominated by the frequency domain and guided by the spatial domain.

[0089] It first refines the spatial domain features and generates an "artifact hotspot" attention map;

[0090] This spatial attention map is then used to guide and modulate frequency domain features;

[0091] Finally, the guided frequency features are finally integrated with the original spatial features to generate a highly refined universal artifact feature for classification.

[0092] In order to make the guidance information in the spatial domain more accurate and representative, the module inputs the spatial artifact features The final enhancement and refinement is performed. First, a 3x3 max pooling operation is performed. This step has two purposes: first, to further expand the receptive field; second, through maximum value selection, to further enhance and extract the most responsive high-frequency artifacts in the feature map.

[0093] Then, the present invention simultaneously calculates the mean (Mean) and maximum (Max) of the spatial artifact features along the channel dimension. These two values ​​represent the "general response strength" and "peak response strength" of the feature at each spatial position, respectively. These two response maps are concatenated and passed through a lightweight convolutional layer and a Sigmoid function to generate a final spatial attention map. .

[0094] This attention map It can be understood as an “artifact saliency map”, where the higher the value, the more likely the corresponding spatial location contains key artifact traces. and spatial artifact characteristics Multiply to obtain the final spatial artifact features.

[0095] Then, the present invention converts the spatial attention map generated in the previous step into Element-wise multiplication of the enhanced features from the frequency branch .

[0096] To ensure the frequency domain characteristics It no longer exists in isolation, and the weight of each of its spatial positions is taken from the “artifact saliency map” in the spatial domain. Modulation and screening were carried out.

[0097] If a location is considered spatially significant ( If the value is high), the frequency characteristics corresponding to the position will be retained or enhanced; on the contrary, if a position is considered to be a background or an unimportant area in space ( If the value is low, the frequency characteristics of that position will be suppressed. This ensures that the frequency information ultimately used for decision making is the most effective part that is highly correlated with spatial artifacts.

[0098] The feature maps of these two different domains are spliced ​​in the channel dimension to form a wider feature map that contains the essence of both modalities. Finally, a 1x1 convolution is used to process the spliced ​​feature map. The role of this 1x1 convolution is to perform cross-channel information fusion and dimensionality reduction, organically blending the information of the two modalities together to generate the final, highly condensed universal forged feature. .this It is then fed into the subsequent classifier for final decision.

[0099] Improved results:

[0100] The experiments of this invention mainly include two groups: one is the generalization effect verification experiment on an unknown GAN model, and the other is the generalization effect verification experiment on an unknown diffusion model.

[0101] This paper uses Accuracy (Accuracy) and Average Precision (AP) as core evaluation metrics. The former measures the overall correct judgment ratio, while the latter more comprehensively reflects the model's overall performance at different confidence thresholds, and is particularly valuable when there is class imbalance.

[0102] Generalization effect verification experiment on unknown GAN model:

[0103] Generative Adversarial Networks (GANs) are the most diverse technology in the field of synthetic images. This experiment aims to evaluate the model's generalization ability across GAN models, a core benchmark for measuring the practicality of detectors. This paper adheres to the most rigorous testing paradigm: training with samples generated by a single GAN model (ProGAN) and then testing on up to eight unknown GAN models (such as StyleGAN, StyleGAN2, BigGAN, and CycleGAN) with varying architectures and generation principles. This setup is designed to simulate the real-world situation where defenders can never predict what new GANs the attacker will use.

[0104] The proposed method demonstrated excellent and stable generalization performance across all test settings. Specifically, when trained on four image classes (n=4), it achieved a mean average accuracy (mAcc) of 95.2% and a high mean average precision (mA.P.) of 98.9% on eight unseen GAN models, both ranking first among all compared methods. Compared to FreqNet, a leading method that relies solely on frequency domain information, the proposed method achieved a 2.7% higher mean accuracy and a 0.9% higher mean precision. This demonstrates that while frequency information is crucial, the proposed method's unique PCHF module intelligently incorporates enhanced spatial high-frequency information, capturing artifact cues that would be invisible to a single modality, achieving breakthrough performance. Compared to the fingerprint-based method NPR, the proposed method achieved 2.7% higher mean accuracy and 2.8% higher mean average precision, respectively. This demonstrates that the proposed asymmetric interactive fusion strategy, which prioritizes the frequency domain and guides the spatial domain, is far more effective than simple feature concatenation or parallel processing. It better extracts and integrates bimodal information, constructing a more discriminative universal forgery feature. This series of excellent data irrefutably proves that our invention does not simply "overfit" the specific artifacts of the ProGAN model used for training. Instead, it successfully learns a more universal "commonality of high-frequency artifacts" common to various GAN architectures, thereby maintaining a very high level of detection in the face of a large number of unknown GAN attacks.

[0105] Table 1 Generalization effect on unknown GAN model

[0106]

[0107] Experiments to verify the generalization effect on unknown diffusion models:

[0108] Table 2 Experimental results on the Ojha test set

[0109]

[0110] Diffusion models (DMs) are a disruptive new generation of image generation technology that has emerged in recent years. Compared to GANs, they generate images with a more realistic feel, subtler and more difficult to detect artifacts, and pose a severe dimensionality reduction threat to existing detectors. This experiment aims to evaluate the generalization ability of our model across two completely different technology systems (from GANs to DMs), a test of the model's robustness to the extreme. We set up two extremely challenging test scenarios:

[0111] 1) Scenario A (Ojha test set): Trained on GAN (ProGAN) and tested on various mainstream diffusion models such as ADM, GLIDE, and LDM.

[0112] 2) Scenario B (Diffusion 1K-Step Test Set): This is currently recognized as one of the most challenging scenarios. The images in this dataset are generated by up to 1,000 denoising steps. Their artifacts are smoothed and eliminated to the greatest extent possible, and the signals are extremely weak.

[0113] As shown in Table 2, our method achieved an average accuracy of 95.7% and an average precision of 97.8%, comprehensively outperforming all comparable methods, including NPR. This preliminary evidence demonstrates that the common characteristics of high-frequency artifacts captured by our method are not limited to the specific generation paradigm of GANs but are also effective for diffusion models.

[0114] As shown in Table 3, the method of the present invention achieved an average accuracy of 85.1% and an average precision of 94.3%. Compared with the NPR method, which is also at the top level, the average accuracy of the present invention is 5.0% ahead, and the average precision has achieved a huge advantage of 9.5%. This result is very convincing, which shows the effectiveness of the high-frequency enhancement module. When the artifact signal is extremely weak, the high-frequency self-enhancement and frequency domain multi-scale enhancement module of the present invention can successfully amplify these signals "submerged in noise" and make them detectable. The necessity of cross-modal interaction: It is precisely because the pooling-guided cross-modal interaction module can intelligently associate and complement the weak local inconsistencies in the spatial domain with the equally weak global pattern anomalies in the frequency domain, that a sufficiently powerful feature that can make correct judgments can be finally formed.

[0115] Table 3 Experimental results on the Diffusion 1K-Step test set

[0116]

[0117] Taking all of the above experiments into account, this invention demonstrates its technological sophistication and effectiveness through consistent and outstanding performance across a wide range of challenging scenarios, both known and unknown, even across technology systems. It not only sets new industry benchmarks across key performance indicators, but more importantly, it provides a rigorously validated, robust, and reliable innovative solution to the long-standing challenge of generalization in the field of synthetic image detection.

[0118] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate and may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0120] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor or controller, for example, by a processor in the above-mentioned terminal embodiment, so that the above-mentioned processor can execute the synthetic image detection method based on cross-modal common interaction in the above-mentioned embodiment.

[0121] Those skilled in the art will appreciate that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, or any suitable combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0122] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

[0123] The specific embodiments of the present invention described above do not limit the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A synthetic image detection method based on cross-modal commonality interaction, characterized in that: Including steps: Get an input image, and input the input image into the spatial domain processing branch and the frequency domain processing branch in parallel: In the spatial domain processing branch, the input image is downsampled and basic features are extracted to obtain a low-dimensional initial feature map, and the initial feature map is subjected to deep feature extraction and enhancement to obtain a complete feature map containing rich structure and high-frequency information; Setting an attention weight to be applied to the complete feature map, and performing a residual connection with the input image to obtain a spatial artifact feature map; In the frequency domain processing branch, the input image is subjected to discrete wavelet transform to be decomposed into multiple frequency sub-bands, and high-frequency sub-bands are selected as high-frequency features of the frequency domain processing branch; The high-frequency features are processed by the multi-scale frequency enhancement module to achieve adaptive capture of diverse frequency artifacts and obtain enhanced frequency domain artifact features; The spatial domain features are refined to generate an "artifact hotspot" spatial attention map. The final spatial domain artifact features are obtained by multiplying the spatial attention map with the spatial domain artifact features. utilizing the spatial attention map to guide and modulate frequency domain artifact features, integrating the guided frequency domain artifact features with the spatial domain artifact features to generate a refined universal artifact feature for image detection; Refine the spatial domain features and generate a "artifact hotspot" spatial attention map, including the following steps: The input spatial artifact features are subjected to a maximum pooling operation; Along the channel dimension, the average and maximum values ​​of the spatial artifact features are calculated simultaneously. The average and maximum values ​​represent the "general response intensity" and "peak response intensity" of the features at each spatial position, respectively. The response map corresponding to the average value and the response map corresponding to the maximum value of the spatial artifact feature are concatenated and passed through a lightweight convolution layer and a Sigmoid function to generate a final spatial attention map; The spatial attention map is used to guide and modulate frequency domain artifact features, including the steps of: Multiply the final spatial attention map element-wise by the frequency domain artifact features from the frequency branch; The guided frequency domain artifact features are integrated with the spatial domain artifact features to generate a refined universal artifact feature for image detection, including the following steps: The frequency domain artifact features and the spatial domain artifact features are spliced ​​in the channel dimension to form a wider intermediate feature map that contains the essence of dual modal information; Use convolution to process the spliced ​​intermediate feature maps, perform cross-channel information fusion and dimensionality reduction, blend the information of the two modalities together, and generate the final universal forged features.

2. The synthetic image detection method based on cross-modal commonality interaction according to claim 1, characterized in that: In the spatial domain processing branch, the input image is downsampled and basic features are extracted to obtain a low-dimensional initial feature map, including the following steps: The input image passes through a standard convolution block and a maximum pooling layer for downsampling and basic feature extraction to obtain a low-dimensional initial feature map.

3. The synthetic image detection method based on cross-modal commonality interaction according to claim 2, characterized in that: Performing deep feature extraction and enhancement on the initial feature map to obtain a complete feature map containing rich structure and high-frequency information includes the following steps: The initial feature map is sent to a series of stacked processing modules for deep feature extraction and enhancement. Each processing module contains several standard residual convolution blocks and an embedded high-frequency self-enhancement module. , the dimensions are Bx C x H x W. When entering the high-frequency self-enhancement module, the processing flow is as follows: The high-frequency self-enhancement module inputs the initial feature map Along the channel dimension Divide equally into the first part and Part II , Part 1 Send it to a convolutional layer to learn and maintain the local smooth structure and content information of the image, and the output is : Part 2 It is used to extract and amplify high-frequency details, select the maximum value in each local area through a maximum pooling operation of a maximum pooling layer, and extract the high-frequency response of edges, corners and textures; The convolutional layer is used to refine the features and adjust the channels, and the output is : The features obtained from the two pathways and features Reassemble them in the channel dimension to form a complete feature map containing rich structure and high-frequency information .

4. The synthetic image detection method based on cross-modal commonality interaction according to claim 3, characterized in that: Setting the attention weight to be applied to the complete feature map and performing a residual connection with the input image to obtain a spatial artifact feature map, including the following steps: The complete feature map is globally average pooled and passed through a convolution and Sigmoid function to generate attention weights. The attention weights are applied to the complete feature map. The enhanced complete feature map is residually connected with the initial feature map to obtain the final output spatial artifact feature map of the spatial domain processing branch. .

5. The synthetic image detection method based on cross-modal commonality interaction according to claim 4, characterized in that: The high-frequency features are processed by the multi-scale frequency enhancement module to achieve adaptive capture of diverse frequency artifacts and obtain enhanced frequency domain artifact features, including the following steps: High-frequency features are frequency feature maps. When a frequency feature map When entering the multi-scale frequency enhancement module, the processing flow is as follows: The module uses a dual-path parallel structure to process the frequency characteristic graph : In the first pass, two special pooling operations are used: global average pooling along the height direction (GAPH) and global average pooling along the width direction (GAPW); respectively capture the global context information in the horizontal and vertical directions in the feature map, and obtain the first direction vector and the second direction vector ; The first direction vector and the second direction vector Concatenate and fuse information through convolution, and then separate into two new vectors; After the two new vectors are activated by the Sigmoid function, the attention weights corresponding to the horizontal and vertical directions are formed. The horizontal and vertical attention weights are multiplied element by element back to the original frequency feature map. , as shown in the formula: In the second path, a standard convolution is used to capture the broad local context in the feature map; In order to intelligently fuse the information of the two pathways, a cross-scale interaction mechanism is used to calculate the channel importance weights of the output features of the two pathways respectively. ;The importance weight of the first path Multiply the output feature of the second path and weight the importance of the second path Multiply it by the output feature of the first path and finally add the two together to obtain the enhanced frequency domain artifact feature, as shown in the formula: 。 6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the synthetic image detection method based on cross-modal common interaction as described in any one of claims 1 to 5.

Images