Frequency-aware high-frequency enhancement network method and system for deepfake detection

Abstract

This invention belongs to the field of computer vision and deep learning technology, and discloses a frequency-aware high-frequency enhancement network method and system for deepfake detection. The method constructs a frequency-domain aware image patch clustering module to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map to extract representative features across frequency bands, enhancing the perception of high-frequency texture details and local structural anomalies. It also constructs a spatial-channel frequency interaction module, which uses dual-branch collaborative modeling of spatial and channel paths, combined with two-dimensional discrete cosine transform and multi-scale pooling mechanisms, to achieve interaction between spatial high-frequency detail information and channel response information. Finally, it constructs a neighborhood pixel relationship representation module to model the relative relationships between local pixels during the upsampling process of the generation model, thereby mining local artifact features in fine-grained regions. This invention can effectively suppress background noise interference and can be applied to generated image security detection and content authenticity identification.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, specifically relating to a frequency-aware high-frequency enhancement network method and system for deep forgery detection. Background Technology

[0002] In recent years, with the rapid development of Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and text-driven image generation models, image synthesis technology has made significant progress in resolution, detail realism, and visual expressiveness. In particular, the emergence of conditional generation models such as DALL·E, Stable Diffusion, and Imagen allows users to generate high-quality images simply through text prompts, greatly expanding the application scope of generative models in content creation, assisted design, and advertising. However, while synthetic images bring convenience, they also increase risks such as the spread of false information, identity forgery, and content tampering, placing higher demands on content security review and digital media trust authentication. Therefore, accurately distinguishing between real and generated images has become an important research direction in the field of multimedia security.

[0003] Early forgery detection efforts largely followed traditional image classification paradigms, treating forgery detection as a binary classification task and directly using the entire image as input to train the classifier. Rossler et al. built upon this by introducing additional domain priors and constructing a face forgery detection model based on the Xception network. Some studies focused on key local facial regions such as the eyes and lips to capture forgery features. Recent research has improved detection capabilities from different perspectives: Barald et al. proposed the CoDE embedding space, combining contrastive learning and global-local similarity constraints; Tan et al. proposed C2P-CLIP, enhancing the discriminative ability of the image-text encoder through category-based general cues; Chang et al. modeled forgery detection as a visual question-answering task (AntifakePrompt); Sha et al. identified forgery samples generated from text to images through image attribution analysis and classifiers; Romeo et al. based their work on binary neural networks (BNNs) and introduced FFT and LBP channels to mine texture artifacts; and Wang et al. further improved detection performance through data augmentation. These methods primarily improve the accuracy and generalization ability of the model by focusing on image-level features, key local region features, and texture features.

[0004] While existing methods have evolved from simple image-level discrimination to incorporating multiple mechanisms such as local region mapping, texture statistics, frequency domain anomalies, cross-modal supervision, and distillation learning, global classification methods based on whole-image input, although capable of learning some differences between real and fake data, often rely heavily on the overall statistical regularities or data distribution characteristics of the training samples. When the training and test data differ in generative model type, sampling mechanism, image style, resolution range, or post-processing methods, the model is prone to learning surface features unrelated to the nature of forgery, leading to a significant drop in detection performance across models or datasets. These methods focus more on global texture and semantic differences, lacking targeted modeling for stability artifacts introduced by the generation mechanism itself. Therefore, while these models can achieve high accuracy on known datasets, they often lack generalization ability under unknown distributions.

[0005] Secondly, some methods enhance forgery detection capabilities by focusing on key local regions such as the eyes and mouth, achieving good results in specific tasks. However, these methods typically rely on strong structural priors or target region localization results, making them more suitable for scenarios with relatively fixed structures, such as face forgery. For tasks like text-to-image generation and open-domain natural scene generation, forgery traces are often not concentrated in fixed locations but exhibit characteristics such as discrete distribution, large scale variations, and uncertain spatial locations. In such cases, local region methods struggle to cover all effective discriminative clues and are prone to weakening detection performance due to inaccurate region selection. Simply relying on salient local regions cannot reliably describe the essential anomalies of forged images.

[0006] In addition to spatial domain information, existing studies have also confirmed that frequency domain features play an important role in revealing generation artifacts, laying the foundation for subsequent frequency domain-based deepfake detection methods.

[0007] Generative neural networks typically rely on convolution-based upsampling methods to generate non-scalar outputs when synthesizing images or videos. However, this process often introduces regular or anomalous forgery artifacts in the frequency domain, which are difficult to detect in the spatial domain. Durall et al.'s research indicates that commonly used upsampling methods can cause generative models to fail to accurately reproduce the spectral distribution of real images, resulting in easily detectable forgery artifacts. Therefore, Liu et al. proposed FatFormer, which adaptively adjusts image features through a forgery-aware adapter and introduces a language-guided alignment mechanism. This mechanism utilizes both image and textual cues to jointly supervise the learning of forgery features, thereby identifying local forgery artifacts in images. Yan et al. proposed AIDE, which uses a multi-expert mechanism to simultaneously extract visual artifacts and noise patterns, further improving forgery detection performance. Jeong et al. proposed FrepGAN, which performs joint training for perturbation generation and detection at the frequency level, allowing the model to fully consider frequency domain artifacts and spatial domain anomalies. Le et al. proposed the ADD framework, which uses frequency attention distillation and multi-view attention distillation to enable the student network to learn discriminative features lost in compressed images, significantly improving the detection performance of low-quality compressed forged images.

[0008] While frequency domain detection methods have demonstrated the significant value of spectral anomalies in identifying generated images, most methods still utilize frequency domain information in a relatively rudimentary manner, typically remaining at the level of overall spectrum stitching, simple frequency transformation assistance, or single-band analysis, failing to fully explore the discriminative information contained in high-frequency artifacts. Although high-frequency components can reflect the unnaturalness of the generative model in edge reconstruction and detail synthesis, this information itself has weak energy and is particularly sensitive to compression, scaling, resampling, blurring, and noise perturbations, easily weakened or even masked in actual propagation paths. Existing methods lack explicit enhancement mechanisms and robust modeling mechanisms for high-frequency anomalies, thus often struggling to reliably extract transferable forgery evidence when faced with low-quality compressed images, complex post-processed images, or mixed scenarios of multiple generative models.

[0009] Current technologies are still insufficient in fusing spatial and frequency domain information. While some methods simultaneously introduce spatial and frequency features, most simply concatenate, extract in parallel, or provide shallow auxiliary supervision, failing to establish a deep coupling relationship between the two. If the correspondence and complementarity between different domains cannot be effectively characterized, the model may rely solely on one easily learned feature to make a judgment, ignoring more fundamental forgery clues, thus affecting overall robustness and generalization performance.

[0010] Furthermore, as generative model architectures continue to evolve, different models differ in upsampling methods, denoising processes, texture synthesis strategies, and detail reconstruction mechanisms, resulting in varying artifact patterns. Most existing techniques model specific types of artifacts, lacking a unified detection approach that can address multiple generation mechanisms while simultaneously considering spatial and high-frequency anomalies. This is one of the key reasons why existing methods exhibit instability in cross-generator detection tasks.

[0011] Therefore, this invention proposes a novel generated image detection method to enhance the perception, screening, and representation of high-frequency artifact information, and to achieve effective synergy between spatial domain features and frequency domain features, thereby improving the accuracy, robustness, and generalization ability of the detection model under various generation models, multiple types of image content, and complex post-processing conditions.

[0012] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0013] (1) Insufficient perception and enhancement of high-frequency artifact information: Most existing deepfake detection models focus on spatial domain texture features, overall semantic distribution differences, or only use single frequency domain anomaly features for auxiliary discrimination. Although they can achieve good detection results on some datasets, they do not pay enough attention to the weak high-frequency artifacts introduced by the generative model during the detail reconstruction process. When the generative model reconstructs the image through processes such as upsampling, interpolation, denoising, and detail completion, it often introduces hidden high-frequency anomalies in the edges, texture transition areas, and fragmented structural areas. These anomalies are weaker and more dispersed than mid- and low-frequency features, and are more easily masked by the semantic information of the backbone network during feature extraction. Because existing methods lack explicit perception and enhancement mechanisms for high-frequency regions, the model's response to high-frequency artifacts is limited, making it difficult to stably capture the more essential differences between the generated image and the real image.

[0014] (2) Insufficient cross-band feature modeling: Generated artifacts exhibit significant distribution differences across different frequency bands. For example, some artifacts show high-frequency detail distortion, while others show abnormal energy distribution in specific frequency bands or structural inconsistencies between multiple frequency bands. However, existing methods typically only perform overall spectrum analysis, simple frequency band division, or use frequency domain features as auxiliary branches input to the network. Although they can utilize frequency domain information, they have not yet established a systematic discrimination modeling method for the different forgery clues carried by different frequency bands. Therefore, it is difficult to fully cover the abnormal traces distributed across multiple frequency bands in complex generated images, and it is also difficult to highlight the key frequency band features that truly have discrimination value.

[0015] (3) Lack of characterization of statistical dependencies between local pixels: During image synthesis, generative models often disrupt the naturally formed local pixel association patterns in real images, especially in operations such as deconvolution, interpolation, copy padding, and resampling. This leads to the introduction of pixel dependency anomalies that do not conform to the laws of natural imaging in certain areas. These anomalies are usually manifested as unnatural transitions and insufficient continuity of local structures, and are more likely to appear in areas such as contour edges, fine textures, or complex backgrounds. Existing detection methods mostly focus on macroscopic texture features or overall frequency domain statistical distribution, and lack effective modeling of the relative relationships, arrangement patterns, and statistical dependencies between the aforementioned local pixels. This results in insufficient sensitivity of the model to artifact recognition, especially when the forgery traces are weak or have been partially covered by post-processing, making it even more difficult to detect potential abnormal patterns in local areas.

[0016] (4) Insufficient robustness and generalization ability: When faced with diverse generation models, complex post-processing operations (such as compression, blurring, scaling) and various interferences, the detection performance of existing methods drops significantly. They lack stable discriminative features that can adapt to different generation sources and complex scenarios, and their generalization ability needs to be improved. Summary of the Invention

[0017] To overcome the problems existing in related technologies, the present invention discloses a frequency-aware high-frequency enhancement network method and system for deepfake detection. This involves a highly generalizable deepfake detection model based on ResNet, combined with a frequency-aware patch clustering module (FPCM), a spatial-channel frequency interaction module (SCFIM), and a nearby pixel relationship mechanism (NPR) to capture local inconsistencies in the generated image. This allows the model to maintain high detection accuracy even in complex scenes. The technical solution is as follows:

[0018] This invention is implemented as follows: a frequency-aware high-frequency enhancement network method for deepfake detection, comprising the following steps:

[0019] S1. The frequency domain-aware image patch clustering module FPCM is used to divide the input feature map into local image patches. Cluster matching is performed based on the frequency domain information of each local image patch. The matched image patches within the cluster are aggregated and filtered according to the preset frequency band to obtain cross-frequency band representative features. High-frequency enhanced features are filtered according to the energy score of the cross-frequency band representative features.

[0020] S2. The Spatial-Channel Frequency Interaction Module (SCFIM) is used to input the input features into the spatial path and the channel path respectively for dual-branch processing. The spatial path extracts spatial high-frequency detail information based on two-dimensional discrete cosine transform and multi-scale pooling, while the channel path extracts channel response information. The outputs of the spatial path and the channel path are interactively fused to realize the interaction and enhancement of spatial high-frequency detail information and channel response information.

[0021] S3. The Nearest Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model, and to mine high-frequency local artifact features in fine-grained regions.

[0022] In step S1, the Frequency Domain Aware Image Patch Clustering (FPCM) module is used to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map, including:

[0023] The input feature map is divided into multiple local image patches, and each local image patch is mapped into a low-dimensional feature vector using 1×1 convolution.

[0024] Adaptive average pooling is applied to the input feature map to generate multiple cluster centers. The cluster affiliation of each local image patch is determined based on the cosine similarity between the feature vector corresponding to each local image patch and each cluster center.

[0025] Aggregate the feature vectors corresponding to local image patches within the same cluster to obtain representative cluster features;

[0026] Two-dimensional discrete cosine transform is performed on the representative features of each cluster, and inverse discrete cosine transform is performed after extracting different frequency components by combining six preset frequency band filters.

[0027] Calculate the energy score of the feature corresponding to each frequency band, and select the local image patch with the highest energy score in each frequency band as the output feature.

[0028] Furthermore, the local image patch division, cluster center construction, and cluster allocation of the input feature map include:

[0029] Input feature map The image is divided into multiple local image patches to represent the texture structure and detail information of the image at a local scale;

[0030] use Convolution maps each image patch, projecting the original high-dimensional features into a continuous low-dimensional feature space to obtain the vector representation of each image patch, so as to extract semantic information and local representation.

[0031]

[0032] In the formula, It is the feature set of all local image patches after mapping. For use in feature mapping Convolution operation, To divide the input feature map into multiple local image patches;

[0033] Adaptive average pooling is applied to the input feature map to generate multiple cluster centers. The matching degree between each image patch vector and each cluster center is calculated based on cosine similarity, and each image patch is assigned to its corresponding cluster according to the maximum similarity principle. The calculation formula is as follows:

[0034]

[0035] In the formula, For the first The normalized cluster center vectors for Normalization operation, To perform adaptive average pooling on the feature set and generate Cluster centers, The number of cluster centers;

[0036]

[0037] In the formula, For the first The image patch and the first Cosine similarity between cluster centers For the first Feature vectors of image patches For vectors Norm, symbol This is a vector dot product operation;

[0038] Cluster assignment of image patches is determined by maximum similarity, expressed as:

[0039]

[0040] In the formula, For the first The final cluster number to which each image patch belongs; To obtain similarity The cluster center index corresponding to the maximum value represents the image patch. Assigned to cluster .

[0041] Furthermore, by filtering according to preset frequency bands, representative cross-band features are obtained, including:

[0042] Image patch vectors belonging to the same cluster are aggregated using the mean aggregation method to obtain the representative features of each cluster:

[0043]

[0044] In the formula, For the first Representative characteristics of a cluster To be assigned to the A set of image patch indexes in each cluster For set The number of elements in the middle;

[0045] A two-dimensional discrete cosine transform is performed on the representative features of each cluster, and different frequency components are separated by a preset frequency band filter. The preset frequency band filter is constructed using a triangular banding method based on the anti-diagonal index of the two-dimensional discrete cosine spectrum.

[0046]

[0047] In the formula, For the first The set of frequency domain locations corresponding to each frequency band Frequency band index; The index of the spectrum in the horizontal direction; This is the index of the spectrum in the vertical direction; These are the frequency domain position coordinates; For the first The upper boundary of each frequency band; For the first The lower boundary of each frequency band; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total number of frequency bands pre-defined;

[0048] No. The boundary of each frequency band is defined as:

[0049]

[0050] In the formula, For the first Location of each frequency band boundary; This is a floor function; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total range length of the spectrum anti-diagonal index;

[0051] According to the The set of frequency domain locations corresponding to the frequency band is used to construct the first frequency band. Individual frequency band filters:

[0052]

[0053] In the formula, For the first The frequency band filter is located in the frequency domain. The value at; When this occurs, the frequency component at that frequency location is retained; When this occurs, the frequency component at that frequency position is suppressed;

[0054] The enhanced frequency domain features are restored to the spatial domain by inverse discrete cosine transform, resulting in a feature representation that combines local structural information and frequency discrimination information.

[0055]

[0056] In the formula, For the first The frequency domain representation of representative features of a cluster obtained by performing a two-dimensional discrete cosine transform. For the first Each frequency band corresponds to a frequency band filter. For element-wise multiplication, It is a two-dimensional inverse discrete cosine transform. For the first The cluster in the th Spatial domain recovery features across a frequency band.

[0057] In step S1, representative image blocks are selected based on the frequency band energy score, and the cross-frequency band high-frequency enhancement features are output, including:

[0058] Energy scores are calculated for each cluster feature. The feature amplitudes are accumulated in logarithmic form to compress the dynamic range of different image patches in the frequency band response. This makes the score difference between high-energy and low-energy image patches smoother and more reasonable, improving the stability and robustness of representative image patch selection. The calculation formula is as follows:

[0059]

[0060] In the formula, For the first The cluster in the th Energy fraction in each frequency band For the first The cluster in the th Spatial domain recovery characteristics across frequency bands; For feature map In the The first channel, the first line, number The eigenvalues ​​at the column, For the number of channels, The spatial size of the image patch. To take the absolute value, It is a very small positive constant; the energy score is used to measure the local texture intensity, detail richness, and high-frequency information salience of the image patch in the corresponding frequency band;

[0061] Within each frequency band, the image patch with the highest score is selected as the representative output for that frequency band.

[0062]

[0063] In the formula, In the first The representative features ultimately selected within each frequency band The cluster index with the largest energy fraction.

[0064] In step S2, the Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of the input features using both spatial and channel paths, including:

[0065] The spatial path performs a two-dimensional discrete cosine transform on the input feature map to obtain frequency domain features, and constructs a weighted matrix according to a preset frequency truncation ratio to retain the low-frequency region;

[0066] The high-frequency region is extracted by utilizing the difference between the original features and the low-frequency region. Then, the high-frequency region is mapped back to the spatial domain by inverse discrete cosine transform. After 1×1 convolution and sigmoid activation, it is multiplied element-wise with the original input features to obtain the spatial path output features. By separating low-frequency information and high-frequency information in the frequency domain, the local texture abrupt change region and edge discontinuity region are highlighted to enhance the spatial sensitivity to forgery traces.

[0067] The channel path performs a two-dimensional discrete cosine transform on the input feature map to extract high-frequency features for each channel. Global average pooling, global max pooling, and global median pooling are then applied to the high-frequency features of each channel to obtain multi-scale statistical responses. Global average pooling is used to characterize the overall distribution level of high-frequency responses within a channel, global max pooling is used to characterize the intensity of significant high-frequency activations within a channel, and global median pooling is used to suppress outlier interference under conditions of local anomalous noise and sparse distribution of fake boundaries, stably characterizing the central trend of high-frequency responses within a channel to improve the ability to characterize local structural abrupt changes. After 1×1 convolution mapping, channel concatenation, convolution, and Sigmoid activation, the channel path output features are obtained.

[0068] In step S2, the Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of the input features using both spatial and channel paths. Specifically, this includes:

[0069] (1) Construct a spatial path, perform frequency domain decomposition on the input features, and extract high-frequency spatial detail features;

[0070] Input feature map The input is fed into a spatial path, and the input features are mapped to the frequency domain using a two-dimensional discrete cosine transform to obtain the corresponding frequency domain representation. A frequency weighting matrix is ​​constructed to truncate and retain the low-frequency region.

[0071]

[0072]

[0073] In the formula, The weight matrix is ​​preserved for the low-frequency region; This is the frequency cutoff ratio. and These represent the low-frequency retention ratios in the height and width directions, respectively. and These represent the height and width of the input feature map in the spatial dimension, respectively; and This is the position index in the frequency domain feature map; when the frequency coordinate is located within the preset low-frequency region, the corresponding position is set to 1, otherwise it is set to 0.

[0074] These are the low-frequency features obtained after filtering using the low-frequency retention weight matrix; This represents the input features in the frequency domain. This is an element-wise multiplication operation; The weight matrix is ​​preserved for the low-frequency region;

[0075] By multiplying the frequency domain features element-wise with the weight matrix, low-frequency region information is preserved and non-low-frequency components are suppressed;

[0076] The corresponding frequency domain features are obtained based on the retained low-frequency components. The high-frequency region is then separated by subtracting the low-frequency region from the original frequency domain features. Finally, the high-frequency features are mapped back to the spatial domain using the inverse discrete cosine transform. Convolution and Sigmoid activation generate spatial attention responses, which are then multiplied element-wise with the original input features to obtain spatial path output features;

[0077]

[0078] In the formula, Output features of the spatial path; The high-frequency features are obtained by removing the low-frequency region from the original frequency domain features; It is a two-dimensional inverse discrete cosine transform used to map high-frequency features back to the spatial domain; for Convolution mapping operation; Use the Sigmoid activation function; These are the original input features;

[0079] (2) Construct channel paths and perform multi-scale statistical modeling and enhancement of the high-frequency response of each channel;

[0080] High-frequency information of the input features is extracted by two-dimensional discrete cosine transform, and global average pooling, global max pooling and global median pooling are performed on the high-frequency regions of each channel respectively.

[0081] In the median pooling branch, the input high-frequency features are first adaptively filled with reflections according to the target output size to ensure that the spatial size is uniformly divided. Then, the padded feature map is divided into non-overlapping local regions, and the median of the elements within each region is taken to obtain the median pooling result. This process can stably statistically represent the feature responses within each local region, suppress the interference of extreme values ​​on the statistical results, and stably characterize the central trend of high-frequency responses within local regions.

[0082]

[0083] The pooling results are then processed... Convolutional mapping is used to enhance features, which are then concatenated along the channel dimension and then processed... Convolution and the Sigmoid activation function generate channel path output features;

[0084]

[0085] In the formula, The output features of the channel path, These are intermediate enhancement features within the channel path;

[0086] (3) Integrate spatial path and channel path features to output spatial-channel joint enhancement results;

[0087] The spatial path output features and the channel path output features are fused by element-wise addition to achieve interactive coupling between spatial high-frequency detail information and channel response information. The fused features are then processed by convolution and normalization to obtain the final output of the spatial-channel frequency interaction module SCFIM.

[0088]

[0089] In the formula, This refers to the fusion output characteristics of the Spatial Channel Frequency Interaction Module (SCFIM).

[0090] In step S3, the nearest neighbor pixel (NPR) mechanism is used to model the relative relationships between local pixels during the upsampling process of the generative model. A neighboring pixel refers to another pixel located within the same local image patch as the reference pixel. The process includes:

[0091] Input image scaled to The intermediate feature map obtained by the upsampling operation and the output image after convolutional layer processing are used to divide the intermediate feature map and the output image into two parts of size 1. A set of local image patches;

[0092] Using pixels in each local image patch as the basic unit, any pixel is selected as a reference pixel to construct the relative difference relationship between the pixel and its neighboring pixels, so as to characterize the dependency pattern between local pixels.

[0093] The relative difference relationship is used to characterize the local generation traces introduced by the upsampling operation and retained by the convolutional layer, thereby achieving a stable representation of forged features in fine-grained regions.

[0094] In step S3, the Nearest Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model. This specifically includes:

[0095] (1) Construct a local image patch representation for the upsampling process at the generator output;

[0096] Input features are , by scale The intermediate features are obtained after the upsampling operation. The output image is then obtained through a convolutional layer. ; intermediate feature map With output image Divide into sizes of Let the set of image patches be denoted as follows: and ,definition:

[0097]

[0098] In the formula, For output image The set of pixels of a local image patch. These are the pixel values ​​within a local image patch arranged in a predetermined order.

[0099] (2) Construct the relative difference relationship between local pixels to characterize the neighborhood dependency features introduced by upsampling;

[0100] Using pixels within each image block as the basic unit, construct the relative difference relationship between pixels; using the pixel in the image block as the basic unit, construct the relative difference relationship between pixels; Using one reference pixel as a reference pixel, the difference relationship between the reference pixel and its neighboring pixels is calculated to obtain the corresponding neighborhood pixel relationship representation. ;

[0101]

[0102] In the formula, For the first The set of relative difference relationships between neighboring pixels constructed using a reference pixel; For the first local image patch The pixel value of each reference pixel; For the first The pixel and the The relative difference between each reference pixel; Let be the index position of the reference pixel within this local image patch, and satisfy . ;

[0103] (3) Represent the neighboring pixel relationships of each local image patch According to the local image patches in the output image The original spatial arrangement in the image is combined to obtain a feature map of neighboring pixel relationships. Based on the constructed feature map, local generation traces in fine-grained regions are modeled to reveal the artifact features retained during the upsampling process of the generation model.

[0104] Another object of the present invention is to provide a frequency-aware high-frequency enhancement network system for deepfake detection, the system being used to implement the aforementioned frequency-aware high-frequency enhancement network method for deepfake detection, the system comprising:

[0105] The input module is used to acquire the image to be detected and extract the initial feature map;

[0106] The Frequency Domain Aware Image Patch Clustering (FPCM) module is used to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map, and output representative features across frequency bands.

[0107] The Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of input features using both spatial and channel paths, and outputs jointly enhanced spatial-channel features.

[0108] The Neighbor Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model and output local artifact features.

[0109] The classification module is used to output the classification results of real images and fake images based on the cross-band representative features, spatial-channel joint enhancement features and local artifact features;

[0110] The input image is first processed by the Neighborhood Pixel Relationship (NPR) module to extract local pixel relative relationship features. These features are then fed into a shallow ResNet branch with an added SCFIM module for further feature extraction. Simultaneously, the input image is processed by the FPCM module and then fed into another ResNet branch for feature extraction. Finally, the features output from the two branches are fused and sent to the classification module for discrimination.

[0111] Combining all the above technical solutions, the beneficial effects of this invention are as follows:

[0112] First, this invention constructs a frequency-domain-aware image patch clustering module to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map to extract representative features across frequency bands, enhancing the perception of high-frequency texture details and local structural anomalies. It also constructs a spatial-channel frequency interaction module, which achieves effective interaction between spatial high-frequency detail information and channel response information through dual-branch collaborative modeling of spatial and channel paths, combined with two-dimensional discrete cosine transform and multi-scale pooling mechanisms. Finally, it constructs a neighborhood pixel relationship representation module to model the relative relationships between local pixels during the upsampling process of the generation model, thereby mining local artifact features in fine-grained regions. Compared with existing technologies, this invention can effectively suppress background noise interference, enhance the representation of minute texture artifacts, high-frequency distortion, and local generation traces, and improve the accuracy, robustness, and generalization ability of deepfake detection under different generation architectures, different post-processing conditions, and complex scenarios. It can be applied to generated image security detection and content authenticity identification.

[0113] Secondly, this invention proposes a Spatial Channel Frequency Interaction Module (SCFIM), which effectively enhances feature fusion capabilities by facilitating cross-domain information interaction across spatial, channel, and frequency dimensions, enabling the model to more comprehensively capture generation artifacts at different levels. This invention also designs a Frequency Aware Image Patch Clustering Module (FPCM), which clusters image patches in the frequency domain to explicitly model the distribution differences of multi-frequency band features, thereby highlighting potential abnormal patterns and improving the model's ability to detect complex forgery traces. Finally, this invention constructs a Neighborhood Pixel Relationship Mechanism (NPR), which, by modeling the statistical dependencies between adjacent pixels, can explicitly capture inconsistencies in local regions, improving the model's detection accuracy in high-resolution and complex scenes.

[0114] Third, this invention addresses the broad application demand for generated image forgery detection, improving the accuracy, robustness, and generalization ability of detecting generated images, deepfake images, and images subjected to complex post-processing such as compression, blurring, and scaling. It possesses significant industrial application value. This technical solution can be widely applied in scenarios such as internet content security review, fake image identification on social media platforms, authenticity verification of news media, and trusted authentication of AIGC content. This invention can be deployed as a standalone forgery detection engine on cloud-based review platforms, or embedded in terminal devices, security gateways, or multimedia forensics platforms, providing standardized detection services, interface services, or industry solutions for enterprise clients. By improving generated image recognition capabilities, this invention helps reduce governance costs, review costs, and compliance risks associated with the spread of fake content, enhancing the efficiency and credibility of platform content governance, and possesses high market potential and expected economic benefits. Simultaneously, with the large-scale application of generative AI content, the demand for highly robust and highly generalizable forgery detection technologies will continue to grow; therefore, this invention has excellent industrialization prospects and sustained commercial value.

[0115] Compared to existing technologies that primarily focus on spatial domain texture analysis, single frequency domain anomaly exploitation, or simple spatial-frequency domain feature fusion, this invention further proposes a more systematic detection approach by collaboratively modeling fine-grained forgery traces in generated images from the perspectives of high-frequency artifact enhancement, cross-band representative feature extraction, and neighboring pixel relationship modeling. Especially in areas such as high-frequency anomaly information enhancement, joint expression of multi-band artifact patterns, and characterization of local pixel statistical dependencies, this invention provides a relatively complete and targeted technical path.

[0116] Fourth, a key technical challenge in the field of generated image forgery detection has long been the emergence of images that, as generation models evolve, their visual quality becomes increasingly closer to that of real images. Furthermore, post-processing techniques such as compression, blurring, scaling, cropping, and noise perturbation further weaken or even mask the original forgery traces, leading to insufficient robustness and poor generalization ability of detection models in unknown generation sources and complex application scenarios. This problem has been a pressing challenge for research and industrial applications in this field. This invention addresses this long-standing technical challenge by designing multiple aspects, including high-frequency artifact perception and enhancement, cross-band feature collaborative modeling, and characterization of neighboring pixel statistical relationships. This enables the model to more stably capture subtle but crucial abnormal features in generated images and maintain good detection performance under diverse generation models and complex post-processing conditions.

[0117] Fifth, existing technologies generally focus more on using spatial domain overall texture features, semantic features, or single frequency domain anomalies for forgery detection, while the utilization of high-frequency artifact information, cross-band collaborative features, and statistical dependencies between neighboring pixels is relatively insufficient. High-frequency information is easily attenuated or even distorted under post-processing operations such as image compression, blurring, scaling, and resampling, resulting in poor stability and making it difficult to use as a reliable detection basis. At the same time, fine-grained relationships between local pixels are easily affected by image content, noise, and differences in generation models, making it difficult to extract universally applicable discriminative features. This invention shows that although high-frequency artifacts and local pixel statistical anomalies are subtle and easily interfered with, through targeted high-frequency information enhancement, cross-band feature organization, and neighboring pixel relationship modeling, it is still possible to extract stable and discriminatively valuable anomalous features from the generated image, and maintain good detection performance under diverse generation models and complex post-processing conditions. Attached Figure Description

[0118] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the disclosure of this invention and, together with the description, serve to explain the principles of the disclosure of this invention.

[0119] Figure 1 This is a flowchart of the frequency-aware high-frequency enhancement network method for deepfake detection provided in an embodiment of the present invention;

[0120] Figure 2 This is a model structure diagram provided in the embodiments of the present invention;

[0121] Figure 3 This is a diagram of the FPCM network structure provided in an embodiment of the present invention;

[0122] Figure 4 This is a structural diagram of the SCFIM provided in an embodiment of the present invention; Figure 5 These are visualization results obtained by different models from representative samples in the test set, as provided in the embodiments of the present invention. Detailed Implementation

[0123] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0124] The innovation of this invention lies in:

[0125] 1. This invention proposes a cross-domain feature interaction mechanism for collaborative modeling in the spatial, channel, and frequency domains. Addressing the problem that existing methods often involve fragmented feature modeling in the spatial, channel, and frequency domains, making it difficult to form a unified discriminative representation, this invention designs an SCFIM module. This module effectively enhances the correlation and complementarity between multi-domain features by performing cross-domain information interaction and joint modeling across the spatial, channel, and frequency dimensions, avoiding the limitation of single-domain feature representation in traditional methods. Through this module, the model can simultaneously focus on spatial texture anomalies, channel response differences, and frequency distribution distortions in the generated image, thereby more comprehensively capturing generation artifacts at different levels and scales, improving feature fusion quality and forgery detection capabilities.

[0126] 2. This invention proposes a frequency domain clustering sensing mechanism for multi-band artifact differences. Addressing the limitation of existing methods that typically rely on single-frequency domain analysis and cannot fully cover complex artifact patterns due to significant distribution differences in generated artifacts across different frequency bands, this invention proposes an FPCM module. This module performs clustering analysis on image patches in the frequency domain, explicitly modeling the distribution differences and potential correlations between features in different frequency bands, thereby highlighting anomalous response patterns across multiple frequency bands. It identifies forgery traces in different frequency bands, enhancing the model's ability to represent complex frequency domain anomalies and improving the recognition accuracy and generalization performance of artifacts generated by various generative models.

[0127] 3. This invention proposes a neighboring pixel relationship modeling mechanism to enhance local artifact detection. Addressing the shortcomings of existing detection methods in detecting unnatural local textures, discontinuous edge transitions, and artifacts in small regions, this invention introduces a neighboring pixel relationship mechanism. This mechanism, by explicitly modeling the statistical dependencies and local structural consistency between adjacent pixels, can effectively capture local anomalous patterns introduced by the generative model during upsampling, interpolation, or reconstruction. This mechanism can more accurately describe forgery features in fine-grained regions, and is particularly beneficial for improving detection accuracy in model contour edges and complex background regions.

[0128] Example 1, such as Figure 2 As shown, this embodiment of the invention provides a highly generalizable deepfake detection model. This model uses ResNet as its backbone and combines a Frequency-Aware Patch Clustering Module (FPCM), a Spatial-Channel Frequency Interaction Module (SCFIM), and a neighboring pixel relationship mechanism to capture local inconsistencies in the generated image, enabling the model to maintain high detection accuracy even in complex scenes.

[0129] like Figure 1 As shown, the frequency-aware high-frequency enhancement network method based on the deepfake detection model provided in this embodiment of the invention specifically includes the following steps:

[0130] S1. To address the difficulty of forming a unified representation in the spatial, channel, and frequency domain feature modeling processes of existing methods, a Frequency Domain Aware Image Patch Clustering (FPCM) module is designed. The FPCM module performs local image patch segmentation, cluster matching, intra-cluster aggregation, and frequency band filtering on the input features to extract representative features across frequency bands, enhancing the perception of high-frequency texture details and local structural anomalies. This module effectively enhances feature fusion capabilities through cross-domain information interaction across spatial, channel, and frequency dimensions, enabling the model to more comprehensively capture generation artifacts at different levels.

[0131] In step S1, the following is adopted: Figure 3 The Frequency Domain Aware Image Patch Clustering (FPCM) module shown performs local image patch segmentation, cluster matching, intra-cluster aggregation, and frequency band enhancement on the input features, specifically including:

[0132] The input feature map is divided into multiple local patches, and each local patch is mapped to a low-dimensional feature vector using 1×1 convolution. Adaptive average pooling is performed on the input feature map to generate multiple cluster centers, and the cluster affiliation of each local patch is determined based on the cosine similarity between the feature vector corresponding to each local patch and each cluster center. The feature vectors corresponding to local patches within the same cluster are aggregated to obtain cluster representative features. Two-dimensional discrete cosine transform is performed on each cluster representative feature, and inverse discrete cosine transform is performed after extracting different frequency components using a frequency band filter. The energy score of the feature corresponding to each frequency band is calculated, and the local patch with the highest energy score in each frequency band is selected as the output feature to enhance the characterization ability of high-frequency texture distortion, local structural anomalies, and fine-grained generation traces in deepfake images.

[0133] Preferably, step S1 includes the following steps:

[0134] S11. Perform local image block division on the input feature map and complete the construction of cluster centers and cluster allocation.

[0135] Given input feature map First, the image is divided into multiple local patches to characterize its texture structure and detail information at a local scale; then, using... Convolution maps each patch, projecting the original high-dimensional features into a continuous low-dimensional feature space to obtain the vector representation of each patch, thereby extracting its semantic information and local representation.

[0136] (1)

[0137] In the formula, It is the feature set of all local image patches after mapping. For use in feature mapping Convolution operation, To divide the input feature map into multiple local image patches;

[0138] The input feature map is subjected to adaptive average pooling to generate multiple cluster centers; then, the matching degree between each image patch vector and each cluster center is calculated based on cosine similarity, and each image patch is assigned to the corresponding cluster according to the principle of maximum similarity. The calculation formula is as follows:

[0139] (2)

[0140] In the formula, For the first The normalized cluster center vectors for Normalization operation, To perform adaptive average pooling on the feature set and generate Cluster centers, The number of cluster centers;

[0141] (3)

[0142] In the formula, For the first The image patch and the first Cosine similarity between cluster centers For the first Feature vectors of image patches For vectors Norm, symbol This is a vector dot product operation;

[0143] Cluster assignment of image patches is determined by maximum similarity, expressed as:

[0144] (4)

[0145] In the formula, For the first The final cluster number to which each image patch belongs; To obtain similarity The cluster center index corresponding to the maximum value represents the image patch. Assigned to cluster .

[0146] S12. Aggregate the image blocks in the same cluster and extract cross-band features in the frequency domain.

[0147] Image patch vectors belonging to the same cluster are aggregated using the mean aggregation method to obtain the representative features of each cluster:

[0148] (5)

[0149] In the formula, For the first Representative characteristics of a cluster To be assigned to the A set of image patch indexes in each cluster For set The number of elements in the middle;

[0150] A two-dimensional discrete cosine transform is performed on the representative features of each cluster, and different frequency components are separated by a preset frequency band filter. The preset frequency band filter is constructed using a triangular banding method based on the anti-diagonal index of the two-dimensional discrete cosine spectrum.

[0151] (6)

[0152] In the formula, For the first The set of frequency domain locations corresponding to each frequency band Frequency band index; The index of the spectrum in the horizontal direction; This is the index of the spectrum in the vertical direction; These are the frequency domain position coordinates; For the first The upper boundary of each frequency band; For the first The lower boundary of each frequency band; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total number of frequency bands pre-defined;

[0153] No. The boundary of each frequency band is defined as:

[0154] (7)

[0155] In the formula, For the first Location of each frequency band boundary; This is a floor function; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total range length of the spectrum anti-diagonal index;

[0156] According to the The set of frequency domain locations corresponding to the frequency band is used to construct the first frequency band. Individual frequency band filters:

[0157] (8)

[0158] In the formula, For the first The frequency band filter is located in the frequency domain. The value at; When this occurs, the frequency component at that frequency location is retained; When this occurs, the frequency component at that frequency position is suppressed;

[0159] The enhanced frequency domain features are restored to the spatial domain by inverse discrete cosine transform, resulting in a feature representation that combines local structural information and frequency discrimination information.

[0160] (9)

[0161] In the formula, For the first The frequency domain representation of representative features of a cluster obtained by performing a two-dimensional discrete cosine transform. For the first Each frequency band corresponds to a frequency band filter. For element-wise multiplication, It is a two-dimensional inverse discrete cosine transform. For the first The cluster in the th Spatial domain recovery features across a frequency band.

[0162] S13. Select representative image blocks based on frequency band energy score and output cross-frequency band high-frequency enhancement features.

[0163] Energy scores are calculated for each cluster feature. The feature amplitudes are accumulated in logarithmic form, which compresses the dynamic range of different image patches in the frequency band response, making the score differences between high-energy and low-energy image patches smoother and more reasonable, thus improving the stability and robustness of representative image patch selection. The calculation formula is as follows:

[0164] (10)

[0165] In the formula, For the first The cluster in the th Energy fraction in each frequency band For the first The cluster in the th Spatial domain recovery characteristics across frequency bands; For feature map In the The first channel, the first line, number The eigenvalues ​​at the column, For the number of channels, The spatial size of the image patch. To take the absolute value, It is a very small positive constant; the energy score is used to measure the local texture intensity, detail richness, and high-frequency information salience of the image patch in the corresponding frequency band;

[0166] Within each frequency band, the image patch with the highest score is selected as the representative output for that frequency band.

[0167] (11)

[0168] In the formula, In the first The representative features ultimately selected within each frequency band The cluster index with the largest energy fraction.

[0169] Through the above steps, the Frequency Domain Aware Image Patch Clustering Module (FPCM) can extract representative patches across frequency bands from the original features, enhancing the model's sensitivity to high-frequency textures, fine-grained distortions, and local structural anomalies, and providing more robust local discriminative features for deepfake image detection.

[0170] In some embodiments, the number K of cluster centers in the Frequency Domain Aware Image Patch Clustering Module (FPCM) can be set according to the input feature map resolution, channel dimension, and the complexity of the artifacts to be detected. Preferably, K takes any value from 8 to 32; more preferably, K can take 16 or 24. By setting the number of cluster centers within the above range, the computational complexity can be considered while maintaining the ability to represent local artifacts, avoiding the aliasing of features in different frequency bands due to too few cluster centers, or the increase in redundant calculations due to too many cluster centers.

[0171] S2. To address the limitation that single-frequency domain analysis cannot fully cover the characteristics of generated artifacts due to their varying distribution across different frequency bands, a Spatial-Channel Frequency Interaction Module (SCFIM) is proposed. The SCFIM module performs dual-branch collaborative modeling of the input features along both spatial and channel paths. Combining two-dimensional discrete cosine transform and multi-scale pooling mechanisms, it achieves effective interaction between spatial high-frequency detail information and channel response information. This module clusters image patches in the frequency domain, explicitly modeling the distribution differences of multi-frequency band features, thereby highlighting potential anomaly patterns and improving the model's ability to detect complex forgery traces.

[0172] In step S2, the following is adopted: Figure 4 The Spatial-Channel Frequency Interaction Module (SCFIM) shown performs dual-branch collaborative enhancement of the input features using both spatial and channel paths, specifically including:

[0173] The spatial path performs a two-dimensional discrete cosine transform on the input feature map to obtain frequency domain features, and constructs a weighted matrix according to a preset frequency truncation ratio to retain the low-frequency region;

[0174] The high-frequency region is extracted by utilizing the difference between the original features and the low-frequency region. Then, the high-frequency region is mapped back to the spatial domain by inverse discrete cosine transform. After 1×1 convolution and sigmoid activation, it is multiplied element-wise with the original input features to obtain the spatial path output features. By separating low-frequency information and high-frequency information in the frequency domain, the local texture abrupt change region and edge discontinuity region are highlighted to enhance the spatial sensitivity to forgery traces.

[0175] The channel path performs a two-dimensional discrete cosine transform on the input feature map to extract high-frequency features of each channel. Global average pooling, global max pooling, and global median pooling are then applied to the high-frequency features of each channel to obtain multi-scale statistical responses. Global average pooling is used to characterize the overall distribution level of high-frequency responses within a channel, global max pooling is used to characterize the intensity of significant high-frequency activations within a channel, and global median pooling is used to suppress outlier interference under conditions of local anomalous noise and sparse distribution of fake boundaries, stably characterizing the central trend of high-frequency responses within a channel, thereby improving the ability to characterize local structural abrupt changes. After 1×1 convolution mapping, channel concatenation, convolution, and Sigmoid activation, the channel path output features are obtained.

[0176] Table 1 High-frequency feature pooling for each channel

[0177]

[0178] Preferably, step S2 includes the following steps:

[0179] S21. Construct a spatial path, perform frequency domain decomposition on the input features, and extract high-frequency spatial detail features.

[0180] Input feature map The input is fed into a spatial path, and the input features are mapped to the frequency domain using a two-dimensional discrete cosine transform to obtain the corresponding frequency domain representation. A frequency weighting matrix is ​​constructed to truncate and retain the low-frequency region.

[0181] (12)

[0182]

[0183] In the formula, The weight matrix is ​​preserved for the low-frequency region; This is the frequency cutoff ratio. and These represent the low-frequency retention ratios in the height and width directions, respectively. and These represent the height and width of the input feature map in the spatial dimension, respectively; and This is the position index in the frequency domain feature map; when the frequency coordinate is located within the preset low-frequency region, the corresponding position is set to 1, otherwise it is set to 0.

[0184] This represents the low-frequency features obtained after filtering using the low-frequency retention weight matrix; This represents the input features in the frequency domain. This represents element-wise multiplication. The low-frequency region retains the weight matrix; by multiplying the frequency domain features element by element with the weight matrix, the low-frequency region information is retained and non-low-frequency components are suppressed.

[0185] The corresponding frequency domain features are obtained based on the retained low-frequency components. The high-frequency region is then separated by subtracting the low-frequency region from the original frequency domain features. Finally, the high-frequency features are mapped back to the spatial domain using the inverse discrete cosine transform. Convolution and Sigmoid activation generate spatial attention responses, which are then multiplied element-wise with the original input features to obtain spatial path output features;

[0186] (13)

[0187] In the formula, Output features of the spatial path; The high-frequency features are obtained by removing the low-frequency region from the original frequency domain features; It is a two-dimensional inverse discrete cosine transform used to map high-frequency features back to the spatial domain; for Convolution mapping operation; Use the Sigmoid activation function; These are the original input features;

[0188] S22. Construct channel paths and perform multi-scale statistical modeling and enhancement of the high-frequency response of each channel.

[0189] High-frequency information of the input features is extracted using two-dimensional discrete cosine transform. Global average pooling, global max pooling, and global median pooling are then performed on the high-frequency regions of each channel. In the median pooling branch, adaptive reflection padding is first applied to the input high-frequency features according to the target output size, ensuring a uniform spatial division. Subsequently, the padded feature map is divided into non-overlapping local regions, and the median of each region is taken to obtain the median pooling result. This process provides stable statistical representation of the feature responses within each local region, suppresses the interference of extreme values ​​on the statistical results, and stably characterizes the central trend of the high-frequency responses within the local regions.

[0190] (14)

[0191] The pooling results are then processed... Convolutional mapping is used to enhance features, which are then concatenated along the channel dimension and then processed... Convolution and the Sigmoid activation function generate channel path output features;

[0192] (15)

[0193] In the formula, The output features of the channel path, These are intermediate enhancement features within the channel path;

[0194] S23. Integrate spatial path and channel path features to output spatial-channel joint enhancement results.

[0195] The spatial path output features and the channel path output features are fused by element-wise addition to achieve interactive coupling between spatial high-frequency detail information and channel response information. The fused features are then processed by convolution and normalization to obtain the final output of the spatial-channel frequency interaction module SCFIM.

[0196] (16)

[0197] In the formula, This refers to the fusion output characteristics of the Spatial Channel Frequency Interaction Module (SCFIM).

[0198] Through the above steps, SCFIM can simultaneously model high-frequency texture details in the spatial domain and multi-scale response features in the channel domain, effectively enhancing the model's sensitivity to minute texture artifacts, local distortions, and generation traces in deepfake images, thus providing a more robust feature representation for forgery image detection.

[0199] In some embodiments, the frequency cutoff ratio in the SCFIM module Used to determine the retention range of the low-frequency region. Preferably, and Each value is independently selected from 0.3 to 0.5. By controlling the low-frequency region within the above range, it is possible to preserve the basic contour and main structure information, and also to extract more discriminative high-frequency artifact information through the difference between the low-frequency and original frequency domain features.

[0200] S3. To address the issue of existing detection methods being insensitive to subtle artifacts such as unnatural local textures and discontinuous edges, a neighboring pixel relationship mechanism is introduced. A neighboring pixel relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model, thereby uncovering local artifact features in fine-grained regions. This mechanism, by modeling the statistical dependencies between adjacent pixels, can explicitly capture inconsistencies in local regions, improving the detection accuracy of the model in high-resolution and complex scenes.

[0201] In step S3, the neighborhood pixel relationship representation module (NPR) is used to model the local artifacts introduced during the upsampling process of the generative model. Specifically, this includes:

[0202] Input image scaled to The intermediate feature map obtained by the upsampling operation and the output image after convolutional layer processing are used to divide the intermediate feature map and the output image into two parts of size 1. A set of local image patches; taking the pixels in each local image patch as the basic unit, selecting any pixel as the reference pixel, constructing a relative difference relationship between the pixel and its neighboring pixels to characterize the dependency pattern between local pixels; using the relative difference relationship to characterize the local generation traces introduced by the upsampling operation and retained by the convolutional layer, thereby achieving a stable characterization of forged features in fine-grained regions.

[0203] Preferably, step S3 includes the following steps:

[0204] S31. Construct a local image patch representation for the upsampling process at the generator output.

[0205] Input features are , by scale The intermediate features are obtained after the upsampling operation. The output image is then obtained through a convolutional layer. ; intermediate feature map With output image Divide into sizes of Let the set of image patches be denoted as follows: and ,definition:

[0206] (17)

[0207] In the formula, For output image The set of pixels of a local image patch. These are the pixel values ​​within a local image patch arranged in a predetermined order.

[0208] S32. Construct the relative difference relationship between local pixels to characterize the neighborhood dependency features introduced by upsampling.

[0209] Using pixels within each image block as the basic unit, construct the relative difference relationship between pixels; using the pixel in the image block as the basic unit, construct the relative difference relationship between pixels; Using one reference pixel as a reference pixel, the difference relationship between the reference pixel and its neighboring pixels is calculated to obtain the corresponding neighborhood pixel relationship representation. ;

[0210] (18)

[0211] In the formula, For the first The set of relative difference relationships between neighboring pixels constructed using a reference pixel; For the first local image patch The pixel value of each reference pixel; For the first The pixel and the The relative difference between each reference pixel; Let be the index position of the reference pixel within this local image patch, and satisfy . ;

[0212] This relative relationship characterization can effectively describe the correlation patterns between local pixels, highlight statistical anomalies introduced by upsampling replication or interpolation mechanisms, and at the same time reduce the interference caused by changes in absolute pixel values.

[0213] S33. Represent the neighboring pixel relationships of each local image patch. According to the local image patches in the output image The original spatial arrangement in the image is combined to obtain a feature map of neighboring pixel relationships. Based on the constructed feature map, local generation traces in fine-grained regions are modeled to reveal the artifact features retained during the upsampling process of the generation model.

[0214] Through the above steps, the NPR module enables the model to maintain good forgery detection performance under different generation architectures and different generation sources, thereby effectively improving the generalization ability and robustness of deep forgery detection.

[0215] Example 2: The frequency-aware high-frequency enhancement network system for deepfake detection provided in this embodiment of the invention includes:

[0216] The input module is used to acquire the image to be detected and extract the initial feature map;

[0217] The Frequency Domain Aware Image Patch Clustering (FPCM) module is used to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map, and output representative features across frequency bands.

[0218] The Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of input features using both spatial and channel paths, and outputs jointly enhanced spatial-channel features.

[0219] The Neighbor Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model and output local artifact features.

[0220] The classification module is used to output the classification results of real images and fake images based on the cross-band representative features, spatial-channel joint enhancement features and local artifact features;

[0221] The input image is first processed by the Neighborhood Pixel Relationship (NPR) module to extract local pixel relative relationship features; these features are then processed by a shallow ResNet branch with an added SCFIM module for further feature extraction; simultaneously, the input image is processed by the FPCM module and then input into another ResNet branch for feature extraction; finally, the features output from the two branches are fused and sent to the classification module for discrimination. In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0222] This invention can be applied to the detection of uncompressed deepfake images. For such scenarios, the FPCM module is used to enhance the representation of local frequency band differences, the SCFIM module is used to enhance the interaction between spatial high-frequency details and channel responses, and the NPR module is used to capture local pixel-dependent features retained during the upsampling process of the generative model, thereby improving the detection capability of artifacts in facial edges, skin textures, hair regions, and organ detail regions.

[0223] This invention can be applied to deepfake image detection scenarios after compression, scaling, resampling, or blurring post-processing. In such scenarios, where some high-frequency information may be weakened, this invention uses an FPCM module to filter representative features across frequency bands, an SCFIM module to jointly enhance high-frequency details and channel responses, and an NPR module to maintain the ability to characterize local pixel relationship anomalies, thus maintaining good detection robustness even under post-processing conditions.

[0224] Experiment Example 1: Comparative Experiment. To verify the effectiveness of the frequency-aware high-frequency enhancement network method provided in this invention in deepfake detection tasks, this experiment example conducted a comparative experiment on the Deepfake Detection and Localization image (DDL-I) dataset. Evaluation metrics included accuracy (ACC) and average precision (AP). To ensure fairness in the comparison, each comparative method was tested under the same or equivalent data partitioning, training epochs, input resolution, and evaluation criteria. Table 2 shows the comparison results of the method of this invention and existing methods on the DDL-I dataset.

[0225] Table 2 Comparison results between the method of the present invention and existing methods

[0226]

[0227] Experimental results show that the detection accuracy and AP index of the method of the present invention are both excellent on this dataset, indicating that the FPCM module, SCFIM module and NPR module proposed in this invention can extract forgery features with strong discriminative ability from multiple perspectives such as frequency domain, spatial-channel interaction and local neighborhood relationship, thereby improving the performance of deep forgery detection.

[0228] Experiment Example 2: Module Ablation Experiment. To verify the contribution of each functional module in this invention to the overall detection performance, this experiment conducted a module ablation experiment under the same training conditions. Using a detection model containing only the ResNet backbone network as the baseline model, the detection results were tested with only the SCFIM module, only the FPCM module, only the NPR module, any combination of two modules, and simultaneously with the SCFIM, FPCM, and NPR modules. Table 3 shows the experimental results under different module combinations.

[0229] Table 3 Ablation experimental results under different module combinations

[0230]

[0231] As shown in Table 3, the detection performance improved after introducing any module on top of the baseline model, indicating that the FPCM, SCFIM, and NPR modules can enhance the forgery feature representation capabilities from different dimensions. The optimal result was achieved when all three modules were introduced simultaneously, demonstrating a good synergistic effect among them. Specifically, the FPCM module mainly improves the extraction capability of cross-band anomaly features, the SCFIM module mainly enhances the joint representation capability of spatial high-frequency details and channel responses, and the NPR module mainly strengthens the characterization capability of local neighborhood pixel anomaly relationships. The combination of these three modules is beneficial for improving the model's detection performance of deepfake images in complex scenes.

[0232] Experiment 3: Parameter Sensitivity Experiment. To further verify the rationality of the parameter settings of the Frequency Domain Aware Image Patch Clustering (FPCM) module in this invention, this experiment compares and analyzes the detection performance under different frequency band number settings, with only the FPCM module introduced and other training conditions remaining unchanged. Specifically, the number of frequency bands was set to 2, 4, 6, and 8, and tests were conducted under the same data partitioning, training rounds, input resolution, and evaluation criteria. Evaluation metrics included accuracy (ACC) and average precision (AP). Table 4 shows the experimental results under different frequency band number settings.

[0233] Table 4. Experimental results of parameter sensitivity under different frequency band settings

[0234]

[0235] Table 4 shows that, with only the FPCM module introduced, the model's detection performance gradually improves as the number of frequency bands increases from 2 to 6. When the number of frequency bands is set to 6, the accuracy and average precision reach their optimal values ​​of 75.44 and 86.81, respectively. When the number of frequency bands is further increased to 8, the detection performance slightly decreases. These results indicate that appropriately increasing the number of frequency bands is beneficial for enhancing the model's ability to represent forged features within different frequency ranges. However, too many frequency bands may lead to the dispersion or redundancy of frequency domain information, thus affecting the overall detection effect. In this embodiment, it is preferable to set the number of frequency bands in the FPCM module to 6.

[0236] Experiment 4: Visual Verification Experiment Based on Heatmaps. To further verify the ability of the method of this invention to represent forged regions, this experiment uses Gradient Weighted Class Activation Mapping (Grad-CAM) technology to perform visual analysis of the regions of interest of different models on the same test image, generating heatmaps that reflect the criteria used by the models for discrimination. To ensure the fairness of the comparison, each model generates a corresponding heatmap under the same input image and the same visualization conditions. Figure 5 The visualization results of different models on representative samples in the test set are shown. Figure 5 As can be seen, when determining the authenticity of an image, the high-response region of the method of this invention is mainly concentrated in forgery-related areas of the image, such as eye contours, mouth edges, and local texture anomalies, while the response in non-forgery background areas is relatively weak. In contrast, the response regions of some comparison methods are more dispersed, with some responding to background areas or areas not directly related to forgery, indicating that their focus on key forgery areas is relatively low. The above visualization results further demonstrate that the method of this invention can more effectively focus on local anomaly areas related to deepfakes, which is beneficial for enhancing the characterization ability of forgery areas and improving the interpretability of detection results.

[0237] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A frequency-aware high-frequency enhancement network method for deepfake detection, characterized in that, The method includes the following steps: S1. The frequency domain-aware image patch clustering module FPCM is used to divide the input feature map into local image patches. Cluster matching is performed based on the frequency domain information of each local image patch. The matched image patches within the cluster are aggregated and filtered according to the preset frequency band to obtain cross-frequency band representative features. High-frequency enhanced features are filtered according to the energy score of the cross-frequency band representative features. S2. The Spatial-Channel Frequency Interaction Module (SCFIM) is used to input the input features into the spatial path and the channel path respectively for dual-branch processing. The spatial path extracts spatial high-frequency detail information based on two-dimensional discrete cosine transform and multi-scale pooling, while the channel path extracts channel response information. The outputs of the spatial path and the channel path are interactively fused to realize the interaction and enhancement of spatial high-frequency detail information and channel response information. S3. The Nearest Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model, and to mine high-frequency local artifact features in fine-grained regions.

2. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 1, characterized in that, In step S1, the Frequency Domain Aware Image Patch Clustering (FPCM) module is used to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map, including: The input feature map is divided into multiple local image patches, and each local image patch is mapped into a low-dimensional feature vector using 1×1 convolution. Adaptive average pooling is applied to the input feature map to generate multiple cluster centers. The cluster affiliation of each local image patch is determined based on the cosine similarity between the feature vector corresponding to each local image patch and each cluster center. Aggregate the feature vectors corresponding to local image patches within the same cluster to obtain representative cluster features; Two-dimensional discrete cosine transform is performed on the representative features of each cluster, and inverse discrete cosine transform is performed after extracting different frequency components by combining six preset frequency band filters. Calculate the energy score of the feature corresponding to each frequency band, and select the local image patch with the highest energy score in each frequency band as the output feature.

3. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 2, characterized in that, The process of dividing the input feature map into local image patches, constructing cluster centers, and assigning clusters includes: Input feature map The image is divided into multiple local image patches to represent the texture structure and detail information of the image at a local scale; use Convolution maps each image patch, projecting the original high-dimensional features into a continuous low-dimensional feature space to obtain the vector representation of each image patch, so as to extract semantic information and local representation. ； In the formula, It is the feature set of all local image patches after mapping. For use in feature mapping Convolution operation, To divide the input feature map into multiple local image patches; Adaptive average pooling is applied to the input feature map to generate multiple cluster centers. The matching degree between each image patch vector and each cluster center is calculated based on cosine similarity, and each image patch is assigned to its corresponding cluster according to the maximum similarity principle. The calculation formula is as follows: ； In the formula, For the first The normalized cluster center vectors for Normalization operation, To perform adaptive average pooling on the feature set and generate Cluster centers, The number of cluster centers; ； In the formula, For the first The image patch and the first Cosine similarity between cluster centers For the first Feature vectors of image patches For vectors Norm, symbol This refers to the dot product operation of vectors. Cluster assignment of image patches is determined by maximum similarity, expressed as: ； In the formula, For the first The final cluster number to which each image patch belongs; To obtain similarity The cluster center index corresponding to the maximum value represents the image patch. Assigned to cluster .

4. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 3, characterized in that, Based on a preset frequency band, representative characteristics across frequency bands are obtained, including: Image patch vectors belonging to the same cluster are aggregated using the mean aggregation method to obtain the representative features of each cluster: ； In the formula, For the first Representative characteristics of a cluster To be assigned to the A set of image patch indexes in each cluster For set The number of elements in the middle; A two-dimensional discrete cosine transform is performed on the representative features of each cluster, and different frequency components are separated by a preset frequency band filter. The preset frequency band filter is constructed using a triangular banding method based on the anti-diagonal index of the two-dimensional discrete cosine spectrum. ； In the formula, For the first The set of frequency domain locations corresponding to each frequency band Frequency band index; The index of the spectrum in the horizontal direction; This is the index of the spectrum in the vertical direction; These are the frequency domain position coordinates; For the first The upper boundary of each frequency band; For the first The lower boundary of each frequency band; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total number of frequency bands pre-defined; No. The boundary of each frequency band is defined as: ； In the formula, For the first Location of each frequency band boundary; This is a floor function; The horizontal dimension of the frequency domain feature map; This represents the vertical dimension of the frequency domain feature map. The total range length of the spectrum anti-diagonal index; According to the The set of frequency domain locations corresponding to the frequency band is used to construct the first frequency band. Individual frequency band filters: ； In the formula, For the first The frequency band filter is located in the frequency domain. The value at; When this occurs, the frequency component at that frequency location is retained; When this occurs, the frequency component at that frequency position is suppressed; The enhanced frequency domain features are restored to the spatial domain by inverse discrete cosine transform, resulting in a feature representation that combines local structural information and frequency discrimination information. ； In the formula, For the first The frequency domain representation of representative features of a cluster obtained by performing a two-dimensional discrete cosine transform. For the first Each frequency band corresponds to a frequency band filter. For element-wise multiplication, It is a two-dimensional inverse discrete cosine transform. For the first The cluster in the th Spatial domain recovery features across a frequency band.

5. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 2, characterized in that, In step S1, representative image blocks are selected based on the frequency band energy score, and the cross-frequency band high-frequency enhancement features are output, including: Energy scores are calculated for each cluster feature. The feature amplitudes are accumulated in logarithmic form to compress the dynamic range of different image patches in the frequency band response. This makes the score difference between high-energy and low-energy image patches smoother and more reasonable, improving the stability and robustness of representative image patch selection. The calculation formula is as follows: ； In the formula, For the first The cluster in the th Energy fraction in each frequency band For the first The cluster in the th Spatial domain recovery characteristics across frequency bands; For feature map In the The first channel, the first line, number The eigenvalues ​​at the column, For the number of channels, The spatial size of the image patch. To take the absolute value, It is a very small positive constant; the energy score is used to measure the local texture intensity, detail richness, and high-frequency information salience of the image patch in the corresponding frequency band; Within each frequency band, the image patch with the highest score is selected as the representative output for that frequency band. ； In the formula, In the first The representative features ultimately selected within each frequency band The cluster index with the largest energy fraction.

6. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 1, characterized in that, In step S2, the Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of the input features using both spatial and channel paths, including: The spatial path performs a two-dimensional discrete cosine transform on the input feature map to obtain frequency domain features, and constructs a weighted matrix according to a preset frequency truncation ratio to retain the low-frequency region; The high-frequency region is extracted by utilizing the difference between the original features and the low-frequency region. Then, the high-frequency region is mapped back to the spatial domain by inverse discrete cosine transform. After 1×1 convolution and sigmoid activation, it is multiplied element-wise with the original input features to obtain the spatial path output features. By separating low-frequency information and high-frequency information in the frequency domain, the local texture abrupt region and edge discontinuity region are highlighted to enhance the spatial sensitivity to forgery traces. The channel path performs a two-dimensional discrete cosine transform on the input feature map to extract high-frequency features for each channel. Global average pooling, global max pooling, and global median pooling are then applied to the high-frequency features of each channel to obtain multi-scale statistical responses. Global average pooling is used to characterize the overall distribution level of high-frequency responses within a channel, global max pooling is used to characterize the intensity of significant high-frequency activations within a channel, and global median pooling is used to suppress outlier interference under conditions of local anomalous noise and sparse distribution of fake boundaries, stably characterizing the central trend of high-frequency responses within a channel to improve the ability to characterize local structural abrupt changes. After 1×1 convolution mapping, channel concatenation, convolution, and Sigmoid activation, the channel path output features are obtained.

7. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 6, characterized in that, In step S2, the Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of the input features using both spatial and channel paths. Specifically, this includes: (1) Construct a spatial path, perform frequency domain decomposition on the input features, and extract high-frequency spatial detail features; Input feature map The input is fed into a spatial path, and the input features are mapped to the frequency domain using a two-dimensional discrete cosine transform to obtain the corresponding frequency domain representation. A frequency weighting matrix is ​​constructed to truncate and retain the low-frequency region. ； ； In the formula, The weight matrix is ​​preserved for the low-frequency region; This is the frequency cutoff ratio. and These represent the low-frequency retention ratios in the height and width directions, respectively. and These represent the height and width of the input feature map in the spatial dimension, respectively; and This is the position index in the frequency domain feature map; when the frequency coordinate is located within the preset low-frequency region, the corresponding position is set to 1, otherwise it is set to 0. These are the low-frequency features obtained after filtering using the low-frequency retention weight matrix; This represents the input features in the frequency domain. This is an element-wise multiplication operation; The weight matrix is ​​preserved for the low-frequency region; By multiplying the frequency domain features element-wise with the weight matrix, low-frequency region information is preserved and non-low-frequency components are suppressed; Based on the retained low-frequency components, the corresponding frequency domain features are obtained. The high-frequency region is then separated by subtracting the low-frequency region from the original frequency domain features. Finally, the high-frequency features are mapped back to the spatial domain using the inverse discrete cosine transform. Convolution and Sigmoid activation generate spatial attention responses, which are then multiplied element-wise with the original input features to obtain spatial path output features; ； In the formula, Output features of the spatial path; The high-frequency features are obtained by removing the low-frequency region from the original frequency domain features; It is a two-dimensional inverse discrete cosine transform used to map high-frequency features back to the spatial domain; for Convolution mapping operation; Use the Sigmoid activation function; These are the original input features; (2) Construct channel paths and perform multi-scale statistical modeling and enhancement of the high-frequency response of each channel; High-frequency information of the input features is extracted by two-dimensional discrete cosine transform, and global average pooling, global max pooling and global median pooling are performed on the high-frequency regions of each channel respectively. In the median pooling branch, the input high-frequency features are first adaptively filled with reflections according to the target output size to uniformly divide the spatial size. Then, the feature map after edge filling is divided into non-overlapping local regions, and the median of the elements in each region is taken to obtain the median pooling result. This process performs stable statistics on the feature responses in each local region, suppresses the interference of extreme values ​​on the statistical results, and stably represents the central trend of high-frequency responses in local regions. ； The pooling results are then processed... Convolutional mapping is used to enhance features, which are then concatenated along the channel dimension and then processed... Convolution and the Sigmoid activation function generate channel path output features; ； In the formula, The output features of the channel path, These are intermediate enhancement features within the channel path; (3) Integrate spatial path and channel path features to output spatial-channel joint enhancement results; The spatial path output features and the channel path output features are fused by element-wise addition to achieve interactive coupling between spatial high-frequency detail information and channel response information. The fused features are then processed by convolution and normalization to obtain the final output of the spatial-channel frequency interaction module SCFIM. ； In the formula, This refers to the fusion output characteristics of the Spatial Channel Frequency Interaction Module (SCFIM).

8. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 1, characterized in that, In step S3, the nearest neighbor pixel (NPR) mechanism is used to model the relative relationships between local pixels during the upsampling process of the generative model. A neighboring pixel refers to another pixel located within the same local image patch as the reference pixel. The process includes: Input image scaled to The intermediate feature map obtained by the upsampling operation and the output image after convolutional layer processing are used to divide the intermediate feature map and the output image into two parts of size 1. A set of local image patches; Using pixels in each local image patch as the basic unit, any pixel is selected as a reference pixel to construct the relative difference relationship between the pixel and its neighboring pixels, so as to characterize the dependency pattern between local pixels. The relative difference relationship is used to characterize the local generation traces introduced by the upsampling operation and retained by the convolutional layer, thereby achieving a stable representation of forged features in fine-grained regions.

9. The frequency-aware high-frequency enhancement network method for deepfake detection according to claim 8, characterized in that, In step S3, the Nearest Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model. This specifically includes: (1) Construct a local image patch representation for the upsampling process at the generator output; Input features are , by scale The intermediate features are obtained after the upsampling operation. The output image is then obtained through a convolutional layer. ; intermediate feature map With output image Divide into sizes of Let the set of image patches be denoted as follows: and ,definition: ； In the formula, For output image The set of pixels of a local image patch. These are the pixel values ​​within a local image patch arranged in a predetermined order. (2) Construct the relative difference relationship between local pixels to characterize the neighborhood dependency features introduced by upsampling; Using pixels within each image block as the basic unit, construct the relative difference relationship between pixels; using the pixel in the image block as the basic unit, construct the relative difference relationship between pixels; Using one reference pixel as a reference pixel, the difference relationship between the reference pixel and its neighboring pixels is calculated to obtain the corresponding neighborhood pixel relationship representation. ; ； In the formula, For the first The set of relative difference relationships between neighboring pixels constructed using a reference pixel; For the first local image patch The pixel value of each reference pixel; For the first The pixel and the The relative difference between each reference pixel; Let the index position of the reference pixel be within this local image patch, and satisfy the following conditions: ; (3) Represent the neighboring pixel relationships of each local image patch According to the local image blocks in the output image The original spatial arrangement in the image is combined to obtain a feature map of neighboring pixel relationships. Based on the constructed feature map, local generation traces in fine-grained regions are modeled to reveal the artifact features retained during the upsampling process of the generation model.

10. A frequency-aware high-frequency enhancement network system for deepfake detection, characterized in that, This system is used to implement the frequency-aware high-frequency enhancement network method for deepfake detection as described in any one of claims 1 to 9, the system comprising: The input module is used to acquire the image to be detected and extract the initial feature map; The Frequency Domain Aware Image Patch Clustering (FPCM) module is used to perform local image patch division, cluster matching, intra-cluster aggregation, and frequency band filtering on the input feature map, and output representative features across frequency bands. The Spatial-Channel Frequency Interaction Module (SCFIM) is used to perform dual-branch collaborative enhancement of input features using both spatial and channel paths, and outputs jointly enhanced spatial-channel features. The Neighbor Pixel Relationship (NPR) module is used to model the relative relationships between local pixels during the upsampling process of the generative model and output local artifact features. The classification module is used to output the classification results of real images and fake images based on the cross-band representative features, spatial-channel joint enhancement features and local artifact features; The input image is first processed by the Neighbor Pixel Relationship (NPR) module to extract local pixel relative relationship features. These features are then processed by a shallow ResNet branch with an added SCFIM module for further feature extraction. Simultaneously, the input image is processed by the FPCM module and then input into another ResNet branch for feature extraction. Finally, the features output from the two branches are fused and sent to the classification module for discrimination.

Images