Deep forgery detection method based on double-flow fusion and adaptive feature enhancement

Abstract

The invention discloses a deep counterfeiting detection method based on double-flow fusion and adaptive feature enhancement, and belongs to the technical field of computer vision and digital media security. According to the method, a double-flow feature extraction network is constructed, a high-low feature adaptive enhancement module (HLFAE) is adopted in a spatial flow to decompose multi-scale texture features, and micro texture expression of a forged area is enhanced in combination with expansion convolution and a channel attention mechanism; introducing a high-frequency sub-band capable of learning a discrete wavelet transform self-adaptive decomposition image into a frequency domain flow, and amplifying frequency domain artifact features through a convolutional network; a multi-modal enhanced feature attention module is designed to dynamically fuse space and frequency domain features, significant features are weighted through a multi-scale convolution kernel and a double attention mechanism, and feature interaction consistency is improved based on a cross-modal contrast enhancement (CMCE) module. According to the method, the highest AUC value of 99.63% is achieved on data sets such as FaceForce + + and Celeb-DFv2, the cross-domain generalization ability and the anti-disturbance robustness are remarkably improved, and the method is suitable for financial risk control and media content auditing.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and digital media security technology. Specifically, it is a deepfake detection method based on dual-stream fusion and adaptive feature enhancement. Background Technology

[0002] With facial recognition technology widely applied in key areas such as identity authentication, financial payments, and social entertainment, facial spoofing detection technology is increasingly important as a core defense for digital identity security. However, current detection technologies face severe challenges: on the one hand, the rapid development of artificial intelligence technologies such as Generative Adversarial Networks (GANs) and diffusion models has made spoofed images and videos extremely realistic, difficult for the human eye to distinguish; on the other hand, existing detection methods have significantly insufficient generalization ability when faced with unknown spoofing techniques, complex compression encoding, and common image perturbations (such as rotation and color jitter), making it difficult to cope with rapidly evolving spoofing techniques. Deepfake technology could be used to create false information, commit online fraud, and even threaten national security and social stability; therefore, the development of robust and universal facial spoofing detection technology is urgently needed.

[0003] Current face forgery detection methods face three major technical bottlenecks. First, most methods rely excessively on spatial domain appearance features (such as texture anomalies) while neglecting crucial information in the frequency domain, such as compression artifacts and resampling traces, resulting in incomplete feature representation. Second, common feature fusion strategies (such as direct concatenation or weighted summation) are often static or heuristic, unable to adaptively capture multi-scale, subtle forgery traces generated by different forgery methods, and also struggling to effectively model the inconsistencies between forged regions and the real background. Furthermore, the continuous downsampling operations during feature extraction in deep neural networks weaken the subtle differences (especially high-frequency details) between forged and non-forged regions, making it difficult for the model to capture subtle forgery clues. Although some studies have attempted to introduce multi-scale feature fusion or prior attention mechanisms to enhance feature representation, their performance degrades significantly when tested across datasets, making it difficult to maintain high accuracy and robustness in complex real-world scenarios.

[0004] Therefore, developing a new framework for face forgery detection that can adaptively fuse multimodal features (spatial and frequency domains), enhance discriminative feature representation, and significantly improve cross-domain generalization ability has become a key technical challenge that urgently needs to be overcome in this field. It is of great significance for maintaining the integrity system of digital society and protecting personal privacy and public interests.

[0005] A search revealed application publication number CN114677372B, which discloses a method and system for detecting deep forged images using noise perception fusion. The method includes: acquiring an image to be detected; extracting RGB stream features based on the acquired image and dual attention calculation to obtain multi-scale enhancement features; extracting noise domain features from the acquired image, and obtaining image noise stream features based on the obtained multi-scale enhancement features and the extracted noise domain features; fusing the obtained image noise stream features and the extracted RGB stream features to obtain image dual-stream fusion enhancement features; and completing image forgery detection based on the obtained image dual-stream fusion enhancement features and local consistency calculation.

[0006] This patent employs a progressive spatial attention module in the spatial domain to mine features beyond attention in an image. Compared to this approach, this patent uses an HLFAE module to separate high-level and low-level abstract features in the spatial domain. In the high-level abstract features, convolution kernels of different sizes are used to perform dilated convolution on the features to extract detailed information of the image. In the low-level abstract features, average pooling and max pooling are used to extract macroscopic information of the image. Finally, the two are fused. This approach can better mine detailed and texture information in the spatial domain and has lower computational complexity and better training stability.

[0007] This patent uses an SRM filter to extract noise information and fuses it with the information extracted from the original RGB stream. In contrast, this patent uses multi-level wavelet decomposition to mine frequency information in the image, obtaining different image information through a different technical approach.

[0008] The feature fusion module of this patent employs progressive spatial attention to perform traditional summation processing on the RGB stream and noise stream at different levels, ultimately obtaining global and local consistency features of the image to determine whether the image has been tampered with. This traditional summation process cannot effectively fuse the content of the two streams. In contrast, the MEFA module proposed in this patent uses CBAM to apply attention weighting to channels and space, effectively improving the expressive power of important features while suppressing irrelevant information. Furthermore, it extracts and aggregates multi-scale features through convolutional operations, helping the model improve its ability to process details at different scales in human faces. Summary of the Invention

[0009] This invention aims to solve the problems of the prior art. It proposes a deep forgery detection method based on dual-stream fusion and adaptive feature enhancement. The technical solution of this invention is as follows:

[0010] A deepfake detection method based on dual-stream fusion and adaptive feature enhancement includes the following steps:

[0011] Step 1: Convert the input original RGB image to the YCbCr color space to obtain the Y channel grayscale image; then, input the original RGB image and the Y channel grayscale image in parallel into the dual-branch structure of the model: the spatial flow focuses on extracting detailed textures and appearance features from the RGB information; the frequency domain flow applies a third-level discrete wavelet transform (DWT) to the Y channel image.

[0012] Step 2: In the spatial flow, a high-low feature adaptive enhancement module HLFAE is designed. By introducing extended convolution, the receptive field is expanded to capture multi-scale texture patterns, and a channel attention mechanism is combined to perform weighted fusion of high- and low-level features. In the frequency domain flow, the high-frequency subbands obtained by DWT decomposition are used to amplify the frequency domain artifact features through a convolutional network.

[0013] Step 3: Spatial and frequency domain features are first processed by the Feature Consistency Enhancement (CMCE) module. By calculating the cross-modal consistency map, the interaction and complementarity between the two feature streams are enhanced. Subsequently, the processed dual-stream features are fed into the Multimodal Enhanced Feature Attention (MEFA) module. MEFA uses multi-scale convolutional kernels and a dual attention mechanism (CBAM) to dynamically fuse spatial details and frequency domain cues and suppress irrelevant feature responses. Finally, the fused discriminative features are input into the feature processing layer of the Xception network for abstraction and classification, and the result of the image authenticity determination is output.

[0014] Furthermore, step 2, which involves capturing multi-scale textures in the spatial domain using the HLFAE module, specifically includes the following steps:

[0015] 1.11.1 From the input feature map Decomposition begins; low-abstract feature components are obtained through average pooling. Represents the original input image

[0016] 1.2 Enhanced high-level and low-level abstractions are subsequently implemented through... Operations are integrated;

[0017] 1.3 Multimodal Enhanced Feature Attention Module (MEFA) enhances convolutional neural networks by combining multi-scale feature extraction with a dual attention mechanism. The MEFA module can capture micro-texture patterns, meso-scale structures, and macro-scale contextual relationships through its parallel convolutional branches, while the CBAM component suppresses irrelevant activations and amplifies task-relevant features through sequential channel and spatial attention mechanisms.

[0018] Furthermore, step 1.11.1 decomposes the input feature map from the beginning. Low-abstraction feature components are obtained through average pooling, specifically including:

[0019]

[0020] By starting from the original input Subtracting the upsampled low-abstraction features yields the high-abstraction feature components that encapsulate high-frequency details of texture anomalies and edge discontinuities. ;

[0021]

[0022] in This indicates that interpolation is an upsampling operation. Ensure upsampling features and The spatial dimensions, namely height and width, are matched;

[0023] Highly abstract feature components are processed by a feature extractor to further amplify and identify forgery traces. ;

[0024]

[0025] The low-abstract feature components undergo a channel-based attention mechanism to enhance semantically salient patterns; the framework generates adaptive weights by integrating average pooling and max pooling features.

[0026]

[0027] here, Indicates a fully connected layer. It is the sigmoid activation function. This represents element-wise multiplication. This represents the feature map after the tie pooling process. This represents the feature map after max pooling. This represents a dimensionality-reducing fully connected layer, used to reduce computation and capture dependencies between channels in a low-dimensional space. This represents a fully connected layer of increased dimensionality, used for receiving... Output and restore its dimensions to the original number of channels.

[0028] Furthermore, the enhanced high-level and low-level abstraction capabilities described in step 1.2 are subsequently implemented through... The operations are integrated, specifically including:

[0029]

[0030] High-level and low-level abstract features are concatenated through channels, and the concatenated features are generated by convolutional layers. Processing to facilitate cross-feature interactions and reduce dimensionality;

[0031]

[0032] The concatenated features are processed by the output of convolution. Finally, the SEAttention module is applied to the fused features to dynamically recalibrate the channel weights. The residual connection integrates the improved features with the original input.

[0033]

[0034] in This indicates element addition. This represents the feature image after processing by the SEAttention module. This represents the final feature map obtained after spatial flow processing.

[0035] Furthermore, 1.3 specifically includes:

[0036] The input to the CMCE module consists of spatial features and frequency characteristics Composition, and cross-modal consistency graphs are calculated using element inner products;

[0037]

[0038] and This represents elements in two different feature maps that participate in the computation of the CMCE module, where and , Indicates the height of the feature map, Indicates the width of the feature map. This indicates the nth element in the feature map; subsequently, the correlation is mapped in the following way. Applied to and : and Feature maps of the spatial and frequency domains calculated by the CMCE module.

[0039]

[0040] Finally, the dual-stream feature map enhanced by the CMCE module was obtained;

[0041] 1) Multimodal Enhanced Feature Attention Module (MEFA) enhances convolutional neural networks by combining multi-scale feature extraction with a dual attention mechanism, with two input features... and First, it is processed through feature connections:

[0042]

[0043] Subsequently, multi-scale contextual features are extracted through parallel convolutions with kernel sizes of 1×1, 3×3, 5×5, and 7×7:

[0044]

[0045] Feature maps processed by dilated convolutions with different kernel sizes are then fused together using concatenated 1×1 convolutional layers:

[0046]

[0047] After being processed by cascaded and 1×1 convolutional layers, each input feature is processed by the Convolutional Block Attention (CBAM) module to generate a weighted feature map:

[0048]

[0049] The spatial and frequency domain features processed by the CBAM module are finally output by integrating these components through weighted summation and 3×3 convolution.

[0050]

[0051] The weights of the feature map weighted summation. The final feature map, after weighted summation and 3×3 convolution, enables the MEFA module to capture micro-texture patterns, mesoscale structures, and macro-scale contextual relationships through its parallel convolutional branches, while the CBAM component suppresses irrelevant activations and amplifies task-relevant features through sequential channels and spatial attention mechanisms.

[0052] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the deepfake detection method based on dual-stream fusion and adaptive feature enhancement as described in any one of the claims.

[0053] A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the deep forgery detection method based on dual-stream fusion and adaptive feature enhancement as described in any one of the claims.

[0054] The advantages and beneficial effects of this invention are as follows:

[0055] This invention proposes a deep forgery detection method based on dual-stream fusion and adaptive feature enhancement, aiming to address the problems of poor adaptability to cross-domain forgery patterns, insufficient capture of subtle forgery traces, and weak robustness under complex perturbations in existing face forgery detection technologies. This method significantly improves the accuracy and generalization ability of forgery detection by constructing a spatial-frequency domain dual-stream analysis architecture and combining hierarchical feature enhancement and a dynamic cross-modal fusion mechanism. The main advantages and beneficial effects of this invention are as follows:

[0056] 1. A Dual-Stream Collaborative Feature Extraction Framework: This invention captures texture detail anomalies through spatial flow while simultaneously utilizing frequency domain flow analysis to compress artifacts and high-frequency distortion, forming complementary feature representations. Specifically, learnable Discrete Wavelet Transform (DWT) adaptively decomposes high-frequency subbands, enabling targeted enhancement of frequency domain traces generated by different forgery methods. This design overcomes the limitations of relying solely on spatial or frequency domain features, improving sensitivity to subtle forgery traces from a multimodal perspective. This innovation corresponds to claim 1. Traditional frequency domain processing methods employ single-level wavelet decomposition, discarding low-frequency information and fusing high-frequency information to observe forgery traces in high-frequency regions. This innovation uses multi-level wavelet decomposition to process frequency information, further decomposing low-frequency information to allow the model to utilize the limited information within low frequencies, thus improving the model's learning ability.

[0057] 2. Hierarchical Adaptive Feature Enhancement Mechanism: A high-low feature adaptive enhancement module (HLFAE) is used to decompose and reconstruct spatial features at multiple scales. Combined with dilated convolution and channel attention mechanisms, this significantly enhances the model's ability to capture local anomalies in forged regions (such as edge discontinuities and texture distortion). This module maintains information integrity through residual connections, avoiding gradient vanishing and thus improving the model's robustness to detecting highly compressed, low-quality forged content. This innovation corresponds to step 2 of claim 1 and the content of claims 3 and 4. This innovation uses a self-designed HLAFE module to process spatial domain information, which, compared to traditional methods, can further mine the information characteristics of the spatial domain and fully utilize spatial domain information to prepare for subsequent feature fusion steps.

[0058] 3. Dynamic Cross-Modal Fusion and Generalization Optimization: Based on the Cross-Modal Contrast Enhancement (CMCE) and Multimodal Enhanced Feature Attention (MEFA) modules, the model can adaptively focus on highly discriminative feature regions and suppress irrelevant information interference. The introduction of multi-scale convolutional kernels and the dual attention mechanism (CBAM) enables dynamic interaction and weight allocation between spatial and frequency domain features. This effectively improves the model's cross-domain generalization ability against unknown forgery techniques (such as content generated by diffusion models), achieving leading performance (AUC up to 99.63%) on datasets such as FaceForensics++ and Celeb-DFv2. This innovation corresponds to step 2 of claim 1 and claim 5. This innovation uses the CMCE module and MEFA module to progressively and collaboratively process information in the spatial and frequency domains. Compared with the traditional channel cascading and feature weighting approach, the CMCE module of this innovation can enhance the contrast between corresponding regions of information in the spatial and frequency domains, allowing the model to learn the forgery traces in different information domains. The MEFA module can fully integrate information in the frequency and spatial domains, allowing the model to further focus on highly discriminative feature regions, thereby suppressing the interference of irrelevant information.

[0059] 4. Highly Efficient End-to-End Detection and Strong Practicality: This invention employs an end-to-end training approach, integrating dual-stream feature extraction, enhancement, and fusion processes into a unified framework, eliminating the need for complex multi-stage processing or additional prior knowledge import. This method is compatible with mainstream classification networks such as Xception, reducing computational overhead while maintaining high accuracy, making it suitable for real-time detection scenarios (such as financial identity verification and content security auditing). Furthermore, its strong robustness to common perturbations such as rotation and color jitter further enhances its industrial application value.

[0060] Dual-stream information fusion is a common information fusion framework. However, how to fully mine information from different information domains and further learn and fuse these two pieces of information from different information domains is a difficult point that requires design and consideration. This patent uses different methods to mine different information domains in both the spatial and frequency domains. In the information fusion part, it uses its own designed MEFA module and combines it with the CMCE module to fuse features, allowing the model to pay attention to hidden forgery traces. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the overall network model structure of a preferred embodiment of the present invention.

[0062] Figure 2 HLAFE modules represent information processing in the spatial domain;

[0063] Figure 3 This refers to the MEFA module, which performs information fusion in the spatial and frequency domains. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0065] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0066] This invention proposes a deep forgery detection method based on dual-stream fusion and adaptive feature enhancement. The overall network model structure of this method is shown in the attached figure. Figure 1 As shown, the core of this method lies in constructing a dual-stream architecture that includes spatial and frequency domain streams, and improving the model's ability to perceive subtle forgery traces and its cross-domain generalization performance through adaptive feature enhancement and fusion mechanisms. This method particularly focuses on addressing the shortcomings of existing technologies in generalizing to unknown forgery methods, complex compression coding, and common image perturbations. The specific implementation steps of this method are as follows:

[0067] Figure 1 The overall network model structure of this invention is shown in the diagram, which includes the following steps:

[0068] A deepfake detection method based on dual-stream fusion and adaptive feature enhancement includes the following steps:

[0069] Step 1: Convert the input original RGB image to the YCbCr color space and extract its luminance component (Y channel) to obtain the corresponding grayscale image. Then, input the original RGB image and the Y channel grayscale image in parallel into the dual-branch structure of the model. The spatial flow focuses on extracting detailed texture and appearance features from the RGB information; the frequency flow applies a three-level discrete wavelet transform (DWT) to the Y channel image, where the wavelet basis "db4" is a learnable parameter. By recursively decomposing the LL subband, high-frequency subbands (such as LH, HL, HH) are extracted at multiple scales, thereby amplifying frequency domain anomalies related to forgery operations. It is worth noting that to reduce interference from semantically consistent but forgery-insensitive low-frequency information, low-frequency (LL) subbands in each level of decomposition are actively excluded.

[0070] Step 2: In the spatial flow, a High-Low Feature Adaptive Enhancement (HLFAE) module is designed. This module first decomposes the input feature map into high and low abstract feature components. Low abstract features are obtained through average pooling to preserve the main structural information; high abstract features are obtained by subtracting the upsampled low abstract features from the original input to capture high-frequency details such as texture anomalies and edge discontinuities. After being processed by dilated convolution (expanding the receptive field and capturing multi-scale texture patterns) and channel attention mechanism (enhancing semantically salient patterns), the high and low abstract features are fused through concatenation and subsequent convolutions, and residual connections are used to preserve the original information flow, effectively enhancing the subtle differences between fake and non-fake regions.

[0071] In the frequency domain stream, the high-frequency subbands obtained from DWT decomposition are further amplified using a convolutional network to amplify frequency domain artifact features. Spatial and frequency domain features are first processed by a cross-modal contrast enhancement module (CMCE), which generates a consistency correlation graph by calculating the element-wise inner product between the two stream features, enhancing the interaction and complementarity between the two features. Subsequently, the processed two-stream features are fed into a multimodal enhanced feature attention module (MEFA). The MEFA module first concatenates the features, then uses parallel multi-scale convolutional kernels (1×1, 3×3, 5×5, 7×7) to extract contextual information, and uses a convolutional block attention module (CBAM) to weight the channel and spatial dimensions, dynamically fusing spatial details and frequency domain cues, and suppressing irrelevant feature responses.

[0072] Step 3: The fused features output from the MEFA module are input into the feature processing layer of the Xception network for further abstraction. A label-smoothed cross-entropy loss function is used for optimization, with the smoothing parameter ε set to 0.1, which helps prevent overfitting and improves generalization ability. Data augmentation strategies, such as random cropping and color dithering, are incorporated during training to further enhance the model's robustness to common perturbations in real-world scenes. Experiments on datasets such as FaceForensics++ (c23 compressed version) show that this method achieves leading detection performance (AUC up to 99.63%).

[0073] In summary, this invention effectively improves the accuracy and generalization ability of deepfake detection through dual-stream collaborative processing, hierarchical adaptive feature enhancement, and dynamic cross-modal fusion, showing significant advantages, especially when facing unknown forgery techniques and post-processed videos.

[0074] Furthermore, step 2, which involves capturing multi-scale textures in the spatial domain using the HLFAE module, specifically includes the following steps:

[0075] 1.1 This process begins with the decomposition of the input feature map. The low-abstraction feature components are obtained through average pooling, which suppresses high-frequency details and preserves the main structural information.

[0076]

[0077] By subtracting the upsampled low-abstract features from the original input, high-abstract feature components that encapsulate high-frequency details such as texture anomalies and edge discontinuities are obtained. This ensures spatial consistency for subsequent operations.

[0078]

[0079] in This indicates an interpolation (upsampling) operation. Ensure upsampling features and Spatial dimension (height, width) matching.

[0080] Highly abstract feature components are processed by a feature extractor to further amplify the identification of forgery traces.

[0081]

[0082] The low-abstract feature components undergo a channel-based attention mechanism to enhance semantically salient patterns. The framework generates adaptive weights by integrating average pooling and max-pooling features. This design aligns with the squeezing and excitation principles, enhancing feature representations through channel recalibration.

[0083]

[0084] here, Indicates a fully connected layer. It is the sigmoid activation function. This represents element-wise multiplication.

[0085] 1.2 Enhanced high-level and low-level abstractions are subsequently implemented through... The operations are integrated. This approach is chosen over simpler element-wise addition or multiplication because it preserves the complete, uncompressed flow of information from both paths by increasing the feature dimension rather than simply combining values ​​within the same dimension. Therefore, the cascaded feature set provides a more flexible and informative foundation for subsequent learning processes, enabling the model to learn the optimal fusion strategy directly from the data.

[0086]

[0087] Cascaded features are derived from convolutional layers Processing is used to facilitate cross-feature interactions and reduce dimensionality.

[0088]

[0089] Finally, the SEAttention module is applied to the fused features to dynamically recalibrate the channel weights, suppressing features with less information and emphasizing discriminative features. Residual connections integrate the improved features with the original input, which is crucial for maintaining gradient flow and reducing information loss during training.

[0090]

[0091] in This indicates element addition.

[0092] 1.3 Our feature fusion differs from previous methods; we avoid merging the two modes through direct connections or attention-weighted enhancements. Instead, we strive to ensure that the two branches retain their respective characteristics while capturing the interactions and interrelationships between the two modes.

[0093] 1) Specifically, the input of the CMCE module consists of spatial features. and frequency characteristics Composition. We calculate the cross-modal consistency graph using element-wise inner products.

[0094]

[0095] in and Subsequently, we mapped the correlation in the following way. Applied to and :

[0096]

[0097] Finally, we obtained the dual-stream feature map enhanced by the CMCE module.

[0098] 2) The Multimodal Enhanced Feature Attention (MEFA) module enhances convolutional neural networks by combining multi-scale feature extraction with a dual attention mechanism, particularly addressing issues of feature redundancy, information loss, and cross-scale adaptability. Two input features... and First, it is processed through feature connections:

[0099]

[0100] The next step is dimensionality reduction using a 1×1 convolutional layer:

[0101]

[0102] Subsequently, multi-scale contextual features were extracted using parallel convolutions with kernel sizes of 3×3, 5×5, and 7×7:

[0103]

[0104] These multi-scale features are then fused together using cascaded and 1×1 convolutional layers:

[0105]

[0106] Each input feature is processed by a Convolutional Block Attention (CBAM) module to generate a weighted feature map:

[0107]

[0108] The final output integrates these components using weighted summation and 3×3 convolution:

[0109]

[0110] This architecture enables the MEFA module to capture microscopic texture patterns, mesoscale structures, and macroscopic contextual relationships through its parallel convolutional branches, while the CBAM component suppresses irrelevant activations and amplifies task-relevant features through sequential channels and spatial attention mechanisms. This combined approach effectively addresses the trade-off between detail preservation and context integration. This resolution lays a solid foundation for visual recognition tasks that must handle real-world scale variations.

[0111] This invention introduces a novel deep forgery detection network—the Dual-Stream Adaptive Feature Enhancement Network (DSFA-Net)—by employing a dual-stream fusion and adaptive feature enhancement mechanism to improve the model's ability to perceive complex forgery traces and its cross-domain generalization performance. This network collaboratively processes the input image using spatial and frequency-domain streams, extracting texture details and frequency-domain artifact features respectively, and utilizes a High-Low Feature Adaptive Enhancement Module (HLFAE) to strengthen the capture of multi-scale forgery traces. Furthermore, dynamic interaction and optimized fusion of spatial and frequency-domain features are achieved through a Cross-Modal Contrast Enhancement Module (CMCE) and a Multi-Modal Enhanced Feature Attention Module (MEFA), significantly enhancing the model's sensitivity to subtle forgery traces. This invention effectively addresses the weak generalization capabilities of existing face forgery detection methods due to insufficient utilization of single-modal features, static feature fusion strategies, and poor cross-domain adaptability. This invention proposes a learnable Discrete Wavelet Transform (DWT) frequency-domain analysis mechanism that adaptively decomposes the high-frequency sub-bands of the image to capture compression artifacts and resampling traces, overcoming the limitations of traditional methods in fully utilizing frequency-domain information. Meanwhile, by employing dynamic attention weighting and multi-scale convolutional aggregation, irrelevant feature interference is further suppressed, enhancing the model's robustness under complex perturbations. Furthermore, this invention introduces feature consistency constraints during the feature fusion stage, resolving the alignment problem in spatial and frequency domain feature interactions and enhancing the consistent representation of forgery traces. Experimental results show that DSFA-Net achieves leading detection performance on mainstream deepfake detection datasets such as FaceForensics++, Celeb-DFv2, and WildDeepfake, exhibiting significant generalization advantages, especially in cross-domain tests. Compared to existing methods, this invention maintains stable performance when dealing with unknown forgery techniques, video compression perturbations, and complex scene changes, providing an innovative technical approach for deepfake detection that balances accuracy and generalization.

[0112] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

[0113] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A deepfake detection method based on dual-stream fusion and adaptive feature enhancement, characterized in that, Includes the following steps: Step 1: Convert the input original RGB image to the YCbCr color space to obtain a Y channel grayscale image; then, input the original RGB image and the Y channel grayscale image in parallel into the dual-branch structure of the model: the spatial flow focuses on extracting detailed textures and appearance features from the RGB information; The frequency domain flow applies a third-order discrete wavelet transform (DWT) to the Y-channel image; Step 2: In the spatial flow, a high-low feature adaptive enhancement module HLFAE is designed. By introducing extended convolution, the receptive field is expanded to capture multi-scale texture patterns, and a channel attention mechanism is combined to perform weighted fusion of high- and low-level features. In the frequency domain flow, the high-frequency subbands obtained by DWT decomposition are used to amplify the frequency domain artifact features through a convolutional network. Step 3: Spatial and frequency domain features are first processed by the Feature Consistency Enhancement (CMCE) module. By calculating the cross-modal consistency map, the interaction and complementarity between the two feature streams are enhanced. Subsequently, the processed dual-stream features are fed into the Multimodal Enhanced Feature Attention (MEFA) module. MEFA uses multi-scale convolutional kernels and a dual attention mechanism (CBAM) to dynamically fuse spatial details and frequency domain cues and suppress irrelevant feature responses. Finally, the fused discriminative features are input into the feature processing layer of the Xception network for abstraction and classification, and the result of the image authenticity determination is output.

2. The deepfake detection method based on dual-stream fusion and adaptive feature enhancement according to claim 1, characterized in that, Step 2, which captures multi-scale textures in the spatial domain using the HLFAE module, specifically includes the following steps: 1.1 From the input feature map Decomposition begins; low-abstract feature components are obtained through average pooling. This represents the original input image; 1.2 Enhanced high-level and low-level abstractions are subsequently implemented through... Operations are integrated; 1.3 Multimodal Enhanced Feature Attention Module (MEFA) enhances convolutional neural networks by combining multi-scale feature extraction with a dual attention mechanism. The MEFA module can capture micro-texture patterns, meso-scale structures, and macro-scale contextual relationships through its parallel convolutional branches, while the CBAM component suppresses irrelevant activations and amplifies task-relevant features through sequential channel and spatial attention mechanisms.

3. The deepfake detection method based on dual-stream fusion and adaptive feature enhancement according to claim 2, characterized in that, Step 1.11.1 involves analyzing the input feature map. Start processing; low Abstract feature components are obtained through average pooling, specifically including: By starting from the original input Subtracting the upsampled low-abstraction features yields the high-abstraction feature components that encapsulate high-frequency details of texture anomalies and edge discontinuities. ; in This indicates that interpolation is an upsampling operation. Ensure upsampling features and The spatial dimensions, namely height and width, are matched; Highly abstract feature components are processed by a feature extractor to further amplify and identify forgery traces. ; The low-abstract feature components undergo a channel-based attention mechanism to enhance semantically salient patterns; the framework generates adaptive weights by integrating average pooling and max pooling features. here, Indicates a fully connected layer. It is the sigmoid activation function. Indicates element-wise multiplication. This represents the feature map after the flat pooling process. This represents the feature map after max pooling. This represents a dimensionality-reducing fully connected layer, used to reduce computation and capture the dependencies between channels in a low-dimensional space. This represents a fully connected layer of increased dimensionality, used to receive... Output and restore its dimensions to the original number of channels.

4. The deepfake detection method based on dual-stream fusion and adaptive feature enhancement according to claim 3, characterized in that, The enhanced high-level and low-level abstraction capabilities described in step 1.2 are then implemented through... The operations are integrated, specifically including: High-level and low-level abstract features are concatenated through channels, and the concatenated features are generated by convolutional layers. Processing to facilitate cross-feature interactions and reduce dimensionality; The concatenated features are processed by the output of convolution. Finally, the SEAttention module is applied to the fused features to dynamically recalibrate the channel weights. The residual connection integrates the improved features with the original input. in This represents element addition. This represents the feature image after processing by the SEAttention module. This represents the final feature map obtained after spatial flow processing.

5. The deepfake detection method based on dual-stream fusion and adaptive feature enhancement according to claim 4, characterized in that, Specifically, 1.3 includes: 1) The input of the CMCE module consists of spatial features. and frequency characteristics Composition, and cross-modal consistency graphs are calculated using element inner products; and This represents elements in two different feature maps that participate in the computation of the CMCE module, where and , Indicates the height of the feature map, Indicates the width of the feature map. This indicates the nth element in the feature map; subsequently, the correlation is mapped in the following way. Applied to and : and Feature maps of the spatial and frequency domains after calculation by the CMCE module; Finally, the dual-stream feature map enhanced by the CMCE module was obtained; 2) Multimodal Enhanced Feature Attention Module (MEFA) enhances convolutional neural networks by combining multi-scale feature extraction with a dual attention mechanism, with two input features... and First, it is processed through feature connections: Subsequently, multi-scale contextual features are extracted through parallel convolutions with kernel sizes of 1×1, 3×3, 5×5, and 7×7: Feature maps processed by dilated convolutions with different kernel sizes are then fused together using concatenated 1×1 convolutional layers: After being processed by cascaded and 1×1 convolutional layers, each input feature is processed by the Convolutional Block Attention (CBAM) module to generate a weighted feature map: The spatial and frequency domain features processed by the CBAM module are finally output by integrating these components through weighted summation and 3×3 convolution. The weights of the feature map weighted summation. The final feature map, after weighted summation and 3×3 convolution, enables the MEFA module to capture micro-texture patterns, mesoscale structures, and macro-scale contextual relationships through its parallel convolutional branches, while the CBAM component suppresses irrelevant activations and amplifies task-relevant features through sequential channels and spatial attention mechanisms.

6. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the deep forgery detection method based on dual-stream fusion and adaptive feature enhancement as described in any one of claims 1 to 5.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the deep forgery detection method based on dual-stream fusion and adaptive feature enhancement as described in any one of claims 1 to 5.

Images