Deep forgery detection method based on multi-granularity collaborative attention mechanism

Abstract

The invention discloses a deep forgery detection method based on a multi-granularity collaborative attention mechanism, and the method comprises the following steps: extracting multi-level features through a backbone network, achieving the self-adaptive region division and weight fusion through a dynamic space grouping attention mechanism, and solving the problem that a conventional method is insufficient in attention to a distributed forgery region; a double-path channel decoupling module is designed to separate high-frequency artifacts and low-frequency semantic features, and feature coupling interference is eliminated; a cross-granularity feature interaction network is constructed, local details and global semantic features are collaboratively optimized, and the fusion efficiency of multi-scale forged clues is improved; and finally, a detection result is output through the classifier. According to the method, the defects of weak cross-domain generalization ability, low multi-granularity information utilization efficiency and the like caused by incomplete space attention coverage and channel feature interference in the prior art are overcome, the detection precision and robustness are remarkably improved, the stable performance is kept in a complex degradation scene, and the method can be widely applied to scenes such as digital content security auditing and identity authentication.

Description

Technical Field

[0001] The present invention relates to the field of deep fake detection, and in particular to a deep fake detection method based on a multi-granularity collaborative attention mechanism. Background Art

[0002] With the rapid development of generative artificial intelligence technologies such as generative adversarial networks (GANs) and diffusion models, deepfake technology can generate highly realistic fake images and videos, posing a serious threat to digital content security, forensics, financial identity authentication, and other fields. Deepfake detection technology has become a research hotspot in computer vision and multimedia security. Existing technologies mainly focus on feature extraction and classification model optimization, and have made considerable progress.

[0003] In the existing technology, the detection methods based on deep learning mainly adopt the following technical routes: the first is the frequency domain analysis method, which converts the image into the frequency domain space to capture the high-frequency artifact characteristics of the synthesized content. This type of method has high detection accuracy for obvious artifacts generated by early forgery technology, but with the evolution of forgery technology, the high-frequency noise of the generated content gradually approaches the real image distribution, resulting in a significant decrease in detection performance; the second is the attention mechanism enhancement method, which embeds channel or spatial attention modules in the convolutional neural network to enhance the model's attention to forgery-sensitive areas. However, this type of method generally uses global average pooling to compress the spatial dimension, resulting in excessive concentration of attention distribution on the central area of the face, and insufficient ability to capture distributed forgery clues such as hairline edges and skin texture; the third is the multimodal fusion method, which combines multimodal features such as facial movements, voiceprints, and physiological signals for joint detection. Although this method can improve the robustness in some scenarios, it faces problems such as high data acquisition cost and difficulty in cross-modal alignment, and is difficult to adapt to large-scale practical application needs.

[0004] Through the above analysis, the problems and defects of the existing technology are as follows:

[0005] (1) Insufficient spatial attention modeling capabilities: Traditional spatial attention mechanisms compress spatial information through global pooling operations, causing the model to over-focus on local salient areas while ignoring distributed artifact cues such as hairlines and skin microtexture. Experiments show that the attention coverage of existing methods is less than 61% complete, and performance degradation exceeds 18% in cross-dataset testing.

[0006] (2) Severe channel feature coupling interference: Conventional channel attention modules do not distinguish between high-frequency artifacts and low-frequency semantic features, making the model susceptible to interference from lighting changes, image compression noise, etc. For example, under JPEG compression conditions, the detection accuracy of existing methods drops by 23.7%.

[0007] (3) Inefficient multi-scale feature collaboration: Existing methods often use simple concatenation or element-by-element addition operations for features at different scales, lacking a fine-grained dynamic fusion mechanism. This makes it difficult for local detail features to effectively complement global semantic information. Visual analysis shows that the feature response intensity of key forgery areas is attenuated by 37%, which is particularly significant in low-resolution input scenarios. Summary of the Invention

[0008] In response to the problems existing in the prior art, the present invention provides a deep fake detection method based on a multi-granularity collaborative attention mechanism.

[0009] This paper provides a deepfake detection method based on a multi-granularity collaborative attention mechanism. Through the three technical breakthroughs of dynamic spatial grouping, frequency domain decoupling, and cross-granularity fusion, it can accurately capture and efficiently identify forgery traces. The specific process includes the following core steps:

[0010] Step 1: In the preprocessing stage, the input RGB image is normalized to a standard size, and brightness equalization is performed to eliminate interference from lighting differences to ensure the consistency of the input data. Subsequently, the pre-trained EfficientNet-B4 backbone network is used to extract feature maps of three layers, Conv3_x, Conv4_x, and Conv5_x, from shallow to deep layers. Shallow features retain high-frequency detail information, while deep features encode global semantics. Finally, bilinear interpolation upsampling is used to unify the resolution to form a multi-scale feature pyramid. This process fuses multi-level features, taking into account the expression of local details and overall structure, providing comprehensive information support for subsequent forgery detection.

[0011] Step 2: Generate an adaptive region partitioning template through a learnable 3×3 convolution, dynamically dividing the feature maps of each level into N×N sub-regions. The size and shape of the sub-regions are adaptively adjusted according to the input content. The attention weight is calculated independently for each sub-region: first, the regional feature vector is extracted through global average pooling, and then the regional importance is analyzed by a multi-layer perceptron. The Sigmoid function is applied to generate an attention coefficient of 0-1. Finally, the weight ratio of each sub-region is normalized through Softmax, and the weighted fusion of local attention maps is formed into a global spatial attention map to highlight forgery-sensitive areas.

[0012] Step three: The features enhanced with spatial attention are processed in two ways: the high-frequency path is filtered using the Laplacian operator to capture synthetic artifacts; the low-frequency path is filtered using a Gaussian low-pass filter to preserve low-frequency semantic information such as facial structure and skin color transitions. The high- and low-frequency features are then concatenated, and the fusion weights are learned through 1×1 convolution. The dynamic gating coefficients are generated using the Sigmoid function. The two types of features are weighted and fused according to the coefficients to suppress invalid information interference and enhance the forgery-related clues.

[0013] Step 4: 3×3 depthwise separable convolutions are used to model local details, reducing computational effort while capturing microtexture features. Simultaneously, a 3×3 dilated convolution with a dilation rate of 3 is used to expand the receptive field to a 23×23 pixel area, modeling the global spatial relationship between facial organs. After concatenating local detail features with global semantic features, their importance is analyzed through a fully connected layer, and a softmax is used to generate a weight distribution. This dynamically fuses the two types of features, achieving complementary optimization of cross-granularity information.

[0014] In step five, global average pooling is performed on the fused features to compress the spatial dimensions while preserving the channel dimensions. The features are then mapped to forgery probability values through a two-layer fully connected network. A sigmoid function is used to constrain the output to a range of 0-1, with a threshold of τ = 0.5. When the probability value ≥ τ, the image is considered forged; otherwise, it is authentic. This process achieves efficient and reliable classification decisions through end-to-end probability mapping.

[0015] Furthermore, in step 1, normalization uses a linear transformation to map pixel values to a uniform interval, avoiding feature deviations caused by uneven illumination. The backbone network, EfficientNet-B4, was selected for its ability to balance computational efficiency and feature representation. The Conv3_x layer corresponds to the early stages of the network, capturing detailed features such as edges and textures; the Conv4_x layer extracts medium-grained local patterns; and the Conv5_x layer focuses on global semantics. Feature alignment uses interpolation to adjust feature maps at each level to the same resolution, constructing a spatially aligned pyramid structure to ensure effective interaction of multi-scale information.

[0016] Furthermore, in step 2, the dynamic segmentation module uses convolutional kernels to learn the spatial distribution characteristics of input features, achieving flexible region segmentation. For example, in forged images, forged regions may exhibit discontinuous or abnormal textures, and dynamic segmentation can focus on such areas. During the attention calculation process, global average pooling compresses spatial information to generate region descriptors. The MLP learns region importance through nonlinear mapping, and ultimately enhances and suppresses noise in locally sensitive areas through weight fusion. This method significantly improves the adaptability and detection accuracy of spatial attention compared to traditional fixed grid partitioning.

[0017] Furthermore, in step three, high-frequency filtering enhances edge and texture mutations using a second-order differential operator, effectively capturing sharpening artifacts or interpolation artifacts commonly seen in forged images. Low-frequency filtering preserves the overall facial structure, preventing semantic information loss due to high-frequency noise. A dynamic gating mechanism uses convolutional layers to learn the contribution of features in different frequency bands. For example, in highly forged areas, high-frequency features may dominate, while in natural images, low-frequency features have a higher weight. This adaptive dual-path decoupling strategy achieves precise separation of forged and authentic features.

[0018] Furthermore, in step 4, depthwise separable convolution significantly reduces the number of parameters by decomposing the spatial and channel dimensions, making it suitable for mobile deployment. Atrous convolution expands the receptive field through interval sampling, capturing long-range dependencies. During the feature fusion stage, the fully connected layer learns the relative importance of local and global features. For example, when detecting subtle forgeries, local features are given a higher weight, while when judging overall facial harmony, global features dominate. This dynamic selection mechanism enhances the model's generalization across different forgery techniques.

[0019] Furthermore, in step five, global average pooling transforms spatial features into channel-dimensional statistics, mitigating the risk of overfitting in the fully connected layers. The fully connected network learns the discriminant boundaries for forged features through nonlinear transformations. The sigmoid function normalizes the outputs into probabilities, which intuitively reflect the confidence level of the image's forgery. The threshold τ is set based on empirically determined trade-offs between false positive rate and missed detection rate and can be adjusted based on practical needs. For example, in security scenarios, τ can be increased to reduce false positives, while in content moderation scenarios, τ can be decreased to improve recall. The final output can be visualized using heatmaps for enhanced interpretability.

[0020] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0021] This paper achieves a significant technological breakthrough in deepfake detection by constructing a multi-granularity collaborative attention mechanism. Addressing core issues commonly encountered in existing methods, such as incomplete spatial attention coverage, channel feature coupling interference, and inefficient multi-scale information fusion, the proposed technical solution utilizes a dynamic spatial grouping attention mechanism to adaptively partition image regions and independently calculate attention weights for each sub-region. This effectively overcomes the attention bias caused by traditional fixed grid divisions, significantly improving the model's ability to capture distributed forgery cues such as hairline edges and skin texture. Combined with a dual-path channel decoupling design, high-frequency artifact features are separated from low-frequency semantic features in the frequency domain and adaptively fused through a dynamic gating mechanism. This significantly reduces the impact of interference factors such as illumination changes and image compression on detection accuracy and enhances the model's adaptability to complex degraded scenarios. Furthermore, a cross-granularity feature interaction network is constructed, which addresses the inefficient fusion of multi-scale forgery cues through the coordinated optimization of local detail modeling and global semantic modeling. This allows for the full exploration and joint identification of forgery traces of varying granularity, such as microtexture anomalies and facial structural inconsistencies.

[0022] The technical solution of the present invention shows multiple advantages in detection accuracy, generalization ability and practical effectiveness. Compared with traditional methods, the completeness of its spatial attention coverage is improved to 89%, the sensitivity to high-frequency artifacts is improved by 37.5%, and the performance degradation in cross-dataset testing is reduced to 7.8%, which significantly improves the model's generalization ability for unknown forgery methods. When dealing with quality degradation scenarios such as JPEG compression and Gaussian blur, the detection accuracy retention rate exceeds 85%, showing excellent anti-interference robustness. At the same time, through lightweight designs such as deep separable convolution and dynamic gating, the model's computing efficiency is improved by 35.8% compared to mainstream solutions, achieving 30fps real-time processing capabilities on edge computing devices, and supporting full-scene deployment from cloud servers to mobile terminals.

[0023] This technical solution has significant social application value. It can provide social media platforms with highly effective false content filtering tools, increasing the accuracy of automated review to over 93% and reducing operating costs by 75% compared to manual review. In financial identity authentication scenarios, the false detection rate is controlled within 0.15%, effectively preventing the risk of biometric fraud. Furthermore, its core module can be expanded to other fields such as image tampering detection and medical image authenticity verification, demonstrating broad potential for technology transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is an overall architecture diagram of a deep fake detection method based on a multi-granularity collaborative attention mechanism provided by an embodiment of the present invention.

[0025] Figure 2 This is a dynamic spatial grouping attention (DSGA) flowchart of a deep fake detection method based on a multi-granularity collaborative attention mechanism provided by an embodiment of the present invention.

[0026] Figure 3 This is a structural diagram of a dual-path channel decoupling module (DCDM) of a deep fake detection method based on a multi-granularity collaborative attention mechanism provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] The specific implementation of the present invention is described in detail below using examples, allowing those skilled in the art to fully implement the technical solution. The implementation process uses a server equipped with an NVIDIA A100 GPU to complete model training. The deployment phase is adapted to the NVIDIA Jetson Xavier NX edge computing device. Input images are uniformly processed into a 256×256 pixel RGB format. The backbone network uses EfficientNet-B4, which loads ImageNet pre-trained weights and freezes the parameters of the first two stages to accelerate convergence. Features from three layers—Conv3_x (256×256×40), Conv4_x (128×128×112), and Conv5_x (64×64×192)—are extracted to construct a multi-scale pyramid. In the dynamic spatial grouping attention module, the 3×3 convolutional layer outputs 36 channels corresponding to 6×6 region partitions. Each sub-region undergoes global average pooling and is then fed into a two-layer fully connected network to generate the attention coefficient. The negative slope of the LeakyReLU activation function is set to 0.01, and the sub-region weights are fused using a Softmax temperature coefficient of 0.5 to generate a spatial attention map.

[0029] The high-frequency path of the dual-path channel decoupling module uses a fixed Laplacian kernel [[0, 1, 0], [1, -4, 1], [0, 1, 0]] to extract artifact features, and the low-frequency path is filtered through a 5×5 Gaussian filter (σ=1.5) to retain semantic information. The dual-path features are concatenated and then subjected to a 1×1 convolution to generate dynamic gating coefficients. The adaptive fusion is completed after Sigmoid function mapping.

[0030] In the cross-granularity feature interaction network, local detail modeling utilizes 3×3 depthwise separable convolutions, with the number of groups equal to the number of input channels, and point-by-point convolutions upscaled to 1.25 times the original channel size. Global semantic modeling establishes long-range dependencies through 3×3 dilated convolutions (dilation rate 3, padding 6), with weights initialized using a Xavier uniform distribution. The dynamic feature selection module concatenates local and global features and feeds them into a fully connected layer, where the dimensionality is compressed to 1 / 8 of the original channel size and the softmax fusion weights are output. During training, FocalLoss (α=0.25, Y=2.0) is used to balance the sample distribution. The AdamW optimizer uses an initial learning rate of 1e-4 and a weight decay coefficient of 0.05, along with a cosine annealing schedule. Data augmentation includes random flipping, rotation, cropping, as well as MixUp (α=0.2) and CutOut (10% occlusion) to enhance model robustness.

[0031] During the deployment phase, INT8 quantization calibration is performed through TensorRT. 500 validation images are selected to dynamically adjust the quantization range. A layer fusion strategy combines convolution, BN, and ReLU operations into a single kernel function, reducing video memory usage by 40%. During edge device deployment, threads are bound to the CPU's large cores and the GPU frequency is locked to 1.1GHz to achieve a balance between power consumption and performance. During the complete operation process, input image preprocessing takes 3ms, backbone network feature extraction takes 15ms, MGCA module dynamic attention calculation and dual-path fusion takes 14ms, cross-granular interaction network execution detailed modeling, global modeling, and dynamic fusion takes a total of 15ms, and classification decision and result output takes 2ms. The entire process latency is controlled within 30ms / frame, meeting real-time detection requirements.

Claims

1. A deep fake detection method based on multi-granularity collaborative attention mechanism, characterized by: The steps include: S1: Extract multi-level features of the input image through the pre-trained backbone network and construct a multi-scale feature pyramid; S2: Perform dynamic spatial grouping attention calculation on the feature map of each level to generate an attention weight map with adaptive region division; S3: A dual-path channel decoupling module is used to separate high-frequency artifact features and low-frequency semantic features, and then fused them through a dynamic gating mechanism; S4: Build a cross-granularity feature interaction network to collaboratively optimize local detail features and global semantic features; S5: Predict the forgery probability based on the fused features and output the detection results.

2. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 1 is characterized in that The dynamic spatial grouping attention calculation in S2 includes: S21: Generate N×N region partition template through a learnable 3×3 convolutional layer, where N∈[4,8]; S22: Calculate the attention weight for each sub-region independently, and use global average pooling and multi-layer perceptron to generate the regional importance coefficient; S23: Based on the Softmax-normalized sub-region weights, weighted fusion is performed to generate the global spatial attention map.

3. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 1 is characterized in that The dual-path channel decoupling module in S3 includes: S31: The high-frequency path uses the Laplacian operator to extract the high-frequency components of the image and learns artifact features through the convolution layer; S32: The low-frequency path uses a Gaussian low-pass filter to extract semantic features and preserve facial structure consistency; S33: Generate dynamic gating coefficients through 1×1 convolution and weighted fusion of high-frequency and low-frequency path outputs.

4. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 1 is characterized in that The cross-granularity feature interaction network in S4 includes: S41: Local granularity modeling unit, using 3×3 depthwise separable convolution to capture micro-texture details; S42: Global granularity modeling unit, using dilated convolution with a dilation rate of 3 to establish long-range spatial dependencies; S43: Dynamic feature selection mechanism, which generates fusion weights through the fully connected layer and the Softmax function to combine local and global features.

5. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 1 is characterized in that The classifier in S5 uses global average pooling to compress the spatial dimension and outputs the forgery probability value through a network including two fully connected layers.

6. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 2 is characterized in that The generation of the region division template in S21 adopts a channel number N 2 The convolutional layer of , where the preferred value of N is 6.

7. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 3 is characterized in that The kernel size of the Laplacian operator in S31 is 3×3, and the standard deviation σ of the Gaussian filter is set to 1.

5.

8. The deep fake detection method based on multi-granularity collaborative attention mechanism according to claim 4 is characterized in that The step size of the depthwise separable convolution in S41 is set to 1, and the filling mode is symmetric filling.

Images