Deepfake detection method based on adaptive decomposition and high-frequency modulation

Abstract

The application provides a deep forgery detection method based on adaptive decomposition and high-frequency modulation, and has the characteristics that: an input image is acquired; in an auxiliary information processing branch: a high-frequency atlas is obtained by extracting a high-frequency part through discrete wavelet transform, and initial high-dimensional feature maps are obtained by performing feature extraction on the high-frequency atlas; the mixed initial high-dimensional feature maps are decomposed into two completely separated parts in semantics: a forgery-related feature map and a forgery-irrelevant feature map; in a main detection branch: an input image is subjected to feature extraction through a feature encoder to obtain an initial main feature map; a self-guided enhancement module is used to perform preliminary adaptive enhancement on the initial main feature map to obtain an enhanced main feature map; a high-frequency modulation cross-modal fusion module receives the enhanced main feature and the forgery-related feature map, and performs cross-modal inconsistency mining under high-frequency guidance once to obtain a fusion feature map used for classification finally; and the fusion feature map is used to complete forgery detection classification.

Description

Technical Field

[0001] This invention relates to the field of image forgery detection technology, and more specifically to a deep forgery detection method based on adaptive decomposition and high-frequency modulation. Background Technology

[0002] Currently, to improve the generalization ability of deepfake detection models against unknown attacks, the mainstream technical approach is to introduce auxiliary information to guide the backbone detection network to learn forgery features more accurately. This auxiliary information is usually a specific transformation of the input image, such as a frequency domain map, noise map, or depth map. The basic logic is to provide the model with an additional reference map containing forgery clues to amplify and highlight the forged region, guiding the model to learn more effectively. However, through in-depth research and analysis in this invention, two fundamentally overlooked technical flaws have been discovered in this seemingly effective strategy:

[0003] 1. Defect 1: The auxiliary information contains a large amount of irrelevant information, causing feature interference. Both frequency maps and depth maps, while containing forgery clues, inevitably also contain a large amount of normal structural and content information derived from the face itself (e.g., the facial contours and the distribution of facial features also generate high-frequency signals). Existing methods indiscriminately use these mixed auxiliary maps as guiding signals, which leads the detector to inevitably enhance both forgery-related features and forgery-related features while enhancing forgery features. This introduces a large amount of redundancy and interference, misleading the model's learning direction and limiting its upper limit of generalization performance.

[0004] 2. Deficiency Two: Lack of an explicit modeling mechanism for inconsistencies between real and fake features. Most existing methods rely on general classification losses (such as cross-entropy loss) to implicitly learn the differences between real and fake images. They lack a dedicated, explicit mechanism to capture and model the subtle inconsistencies between local regions of real and fake faces. Especially in high-quality, highly realistic fake images, these local differences are very weak, making it extremely difficult to learn them solely through backpropagation from the backbone network. This directly leads to insufficient detection capability for highly realistic fake samples. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a deep forgery detection method based on adaptive decomposition and high-frequency modulation, which solves the technical problems in the prior art where auxiliary information contains a large amount of forgery-irrelevant information, causing feature interference and lacking an explicit modeling mechanism for inconsistencies between genuine and fake features.

[0006] To achieve the above-mentioned technical objectives, in a first aspect, the present invention provides a deepfake detection method based on adaptive decomposition and high-frequency modulation, comprising the following steps:

[0007] Obtain the input image;

[0008] In the auxiliary information processing branch: the input image is extracted by discrete wavelet transform to obtain a high-frequency spectrum, and the feature encoder performs feature extraction on the high-frequency spectrum to obtain an initial high-dimensional feature map;

[0009] The adaptive forgery feature separation module decomposes the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map;

[0010] In the backbone detection branch: the input image is processed by a feature encoder to extract features and obtain an initial backbone feature map;

[0011] The self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map;

[0012] The high-frequency modulation cross-modal fusion module receives the enhanced backbone features and the forged related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain the final fused feature map for classification.

[0013] The fused feature map is used to complete the forgery detection and classification.

[0014] Compared with the prior art, the beneficial effects of the present invention include:

[0015] This invention proposes an innovative dual-branch deep forgery detection framework based on adaptive decomposition and high-frequency modulation, aiming to address the problem at its root. First, to address the issue of auxiliary features containing forgery-irrelevant information, this invention designs an adaptive forgery feature separation module. The core innovation of this module lies in its departure from treating mixed auxiliary information (such as high-frequency maps) as a whole. Instead, it intelligently and adaptively decomposes the input auxiliary feature map into two mutually exclusive parts: forgery-related features (containing only traces introduced by the forgery process) and forgery-irrelevant features (containing inherent structural and content information of the face). To achieve this clean decomposition, this invention introduces spatial exclusivity constraints and entropy maximization constraints. The former ensures that forgery-related and irrelevant features are not activated simultaneously at any spatial location; the latter encourages the model to learn more general forgery patterns across different forgery methods. In this way, this invention provides highly pure and interference-free guiding signals for subsequent detection.

[0016] To address the issue of inconsistency modeling deficiencies, this invention designs a high-frequency modulation cross-modal fusion module. The core function of this module is to explicitly guide and enhance the backbone network's ability to capture local inconsistencies between real and fake images by utilizing clean forgery-related features obtained from the adaptive forgery feature separation module. First, the high-frequency guidance part generates a spatial attention map using clean forgery-related high-frequency features. Then, the obtained features are modulated and mined, dynamically modulating features at different spatial locations in the backbone network using the spatial attention map, guiding the model to focus on regions where there are high-frequency differences between real and forged images. This invention also designs a gating unit to implicitly mine these inconsistencies without introducing additional, complex loss functions. Finally, through a two-step strategy of "decomposition and purification followed by guidance and enhancement," not only is an unprecedentedly clean guiding signal provided to the detector, but the model's ability to capture local inconsistencies is also explicitly enhanced, thus exhibiting generalization performance and robustness far exceeding existing technologies when facing unknown, high-quality forgery attacks.

[0017] According to some embodiments of the present invention, the adaptive forgery feature separation module decomposes the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map, including the steps of:

[0018] Two key, jointly optimized loss functions are introduced as constraints: spatial mutual exclusion constraint and entropy maximization constraint;

[0019] When the initial high-dimensional feature map F h The adaptive forged feature separation module then proceeds. Internally, this module uses two parallel, non-weight-sharing convolutional layers to separate the F... h Simultaneously mapped to two outputs: spoofed relevant feature map F hf Unrelated feature map F ho .

[0020] According to some embodiments of the present invention, spatial mutual exclusion constraints and entropy maximization constraints are as follows:

[0021] Spatial Mutual Exclusion Constraint L ex It is mandatory that at any spatial pixel location (h,w,c), F hf and F ho The activation values ​​of the features cannot all be positive simultaneously; this is achieved by minimizing the sum of squares of their pointwise products, as shown in the following formula:

[0022]

[0023] Entropy maximization constraint: Predetermine h learnable prototype-based fake feature cluster centers C k For F hfFor each feature vector in the dataset, calculate the probability p(k|F) of being assigned to each cluster center. hf The goal of the entropy maximization constraint is to make the probability distribution as uniform as possible, and the entropy maximization constraint loss function L... en This is achieved by minimizing negative entropy, as shown in the formula:

[0024] L en =-p(k|F hf )logp(k|F hf ).

[0025] According to some embodiments of the present invention, the self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map, including the following steps:

[0026] The initial backbone feature map F r Divided into two parts along the channel dimension C on average and

[0027] The high-frequency activation feature map M is extracted using a max pooling layer. h The high-frequency activation feature map is enhanced by a 1x1 convolution;

[0028] The low-frequency activation feature map M is then extracted using an average pooling layer. l It is enhanced by a 1x1 convolution;

[0029] The enhanced high-frequency activation feature map and low-frequency activation feature map are reassembled and then linked to the initial backbone feature map F using a learnable hyperparameter a. r Perform weighted fusion, as shown in the formula:

[0030]

[0031] w h and w l It is a convolutional layer, and ⊙ represents element-wise multiplication.

[0032] According to some embodiments of the present invention, a high-frequency modulated cross-modal fusion module receives the enhanced backbone features and the forged correlation feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain a final fused feature map for classification, including the following steps:

[0033] Enhance the backbone feature map F' r The query is transformed into a query(Query, F) through three parallel 1x1 convolutional layers. q ), key (Key, F) k) and Value (Value, F v );

[0034] Enhance the backbone feature map F' r Feature map F related to forgery from auxiliary branch hf After adding them together, the transformation is performed as shown in the formula:

[0035] F q =w q (F' r ),F k =w k (F' r +F hf ),F v =w v (F' r )

[0036] By forging related feature maps F hf Injected into the "key" F k This allows subsequent attention calculations to be acutely aware of the presence and location of forgeries;

[0037] Module Calculation Query F q and specially designed key F k The matrix multiplication between them yields a consistency matrix M. c ;

[0038] Using a lightweight convolutional network, the forged relevant feature map F is generated. hf Convert to a high-frequency gated weight graph M hf ;

[0039] High-frequency attention map M hf Element-wise multiplication by the consistency matrix M c ;

[0040] The gated attention weights are applied to the value F. v The results are then compared with the initial backbone feature map F. r Perform residual connections to obtain the final fused feature map F used for classification. m As shown below:

[0041]

[0042] Secondly, the technical solution of the present invention provides a deepfake detection system based on adaptive decomposition and high-frequency modulation, applied to the deepfake detection method based on adaptive decomposition and high-frequency modulation as described in any one of the first aspects, comprising:

[0043] The auxiliary information processing branch includes:

[0044] The high-frequency extraction module is used to extract the high-frequency part of the input image through discrete wavelet transform to obtain a high-frequency spectrum. The feature encoder performs feature extraction on the high-frequency spectrum to obtain an initial high-dimensional feature map.

[0045] An adaptive forgery feature separation module is used to decompose the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map;

[0046] The main detection branches include:

[0047] A feature encoder extracts features from the input image to obtain an initial backbone feature map;

[0048] The self-guided enhancement module is used to perform preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map.

[0049] The high-frequency modulation cross-modal fusion module receives the enhanced backbone features and the forged related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain the final fused feature map for classification.

[0050] The classifier uses the fused feature map to perform forgery detection classification.

[0051] According to some embodiments of the present invention, it further includes:

[0052] A gating unit is located between the high-frequency modulation cross-modal fusion module and the adaptive forgery feature separation module. The gating unit is used to explicitly mine and amplify the high-frequency inconsistencies in local areas of genuine and fake images.

[0053] Thirdly, the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the deep forgery detection method based on adaptive decomposition and high-frequency modulation as described in any one of the first aspects.

[0054] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0055] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein the abstract drawings are to be completely consistent with one of the drawings in the specification:

[0056] Figure 1 A flowchart of a deep forgery detection method based on adaptive decomposition and high-frequency modulation provided in an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0059] Reference Figure 1 , Figure 1 This is a flowchart illustrating a deepfake detection method based on adaptive decomposition and high-frequency modulation, provided in one embodiment of the present invention. The deepfake detection method based on adaptive decomposition and high-frequency modulation includes, but is not limited to, the following steps, and the step numbers should not be construed as limitations on the preceding or following steps:

[0060] Step S110: Obtain the input image;

[0061] Step S120, in the auxiliary information processing branch: the high-frequency part of the input image is extracted through discrete wavelet transform to obtain a high-frequency spectrum, and the feature encoder performs feature extraction on the high-frequency spectrum to obtain an initial high-dimensional feature map;

[0062] The adaptive forgery feature separation module decomposes the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map;

[0063] Step S130, in the backbone detection branch: the input image is processed by a feature encoder to extract features and obtain an initial backbone feature map;

[0064] The self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain the enhanced backbone feature map;

[0065] In step S140, the high-frequency modulation cross-modal fusion module receives the enhanced backbone features and the forged related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain the final fused feature map for classification.

[0066] Use fused feature maps to perform forgery detection and classification.

[0067] In one embodiment, the deep forgery detection method based on adaptive decomposition and high-frequency modulation includes the following steps: acquiring an input image; in the auxiliary information processing branch: the input image is processed by discrete wavelet transform to extract the high-frequency part to obtain a high-frequency spectrum, and a feature encoder is used to extract features from the high-frequency spectrum to obtain an initial high-dimensional feature map; in the adaptive forgery feature separation module, the mixed initial high-dimensional feature map is decomposed into two semantically completely separated parts: a forgery-related feature map and a forgery-irrelevant feature map; in the backbone detection branch: the input image is processed by a feature encoder to obtain an initial backbone feature map; in the self-guided enhancement module, the initial backbone feature map is initially adaptively enhanced to obtain an enhanced backbone feature map; a high-frequency modulation cross-modal fusion module receives the enhanced backbone feature map and the forgery-related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain a final fused feature map for classification; and the fused feature map is used to complete the forgery detection classification.

[0068] This invention extracts the high-frequency components of the input image using discrete wavelet transform to obtain a high-frequency spectrum. Combined with feature extraction of the entire input image, an initial backbone feature map is obtained. This approach fully utilizes information from different layers of the image, paying attention to both high-frequency details (which often contain key clues that can reveal forgery traces, such as splicing edges, which are more likely to show anomalies in high-frequency regions) and the overall features of the image. This allows subsequent detection to be based on a more comprehensive feature base, which helps improve the accuracy of detection.

[0069] The adaptive forgery feature separation module decomposes the mixed initial high-dimensional feature map into forgery-related feature maps and forgery-irrelevant feature maps, achieving complete semantic separation. This means that it can accurately focus on features closely related to forgery, filtering out features that are useless for forgery detection or may even interfere with it, making subsequent processing and judgment more targeted and reducing the possibility of false positives. The self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map. The adaptive approach can dynamically adjust the enhancement strategy according to the characteristics of the image itself, so that different input images can obtain the feature enhancement that best suits their own situation, further highlighting key features and improving the effectiveness of backbone features in detection.

[0070] The high-frequency modulation cross-modal fusion module performs high-frequency-guided cross-modal inconsistency mining, receiving and fusing enhanced backbone features and forgery-related feature maps. Since forged images often exhibit inconsistencies between high-frequency details and the overall features reflected in the backbone (e.g., texture and color in the forged area are inconsistent with the surrounding normal area), mining these cross-modal inconsistencies can keenly capture subtle clues of forgery, enhancing the ability to detect deepfakes. From comprehensive and accurate feature extraction to adaptive feature enhancement, and then to cross-modal fusion inconsistency mining, the entire process works collaboratively to more effectively identify deepfake images. Compared to some detection methods that only focus on a single feature or simply fuse features, it is expected to significantly improve accuracy and reduce false positives and false negatives.

[0071] Overall process overview:

[0072] This invention adopts a dual-branch architecture, with the auxiliary information processing branch processing the high-frequency information of the input image in parallel, and the adaptive forgery feature separation module decomposing the mixed features and extracting the pure forgery-related high-frequency clues.

[0073] Meanwhile, the main detection branch directly processes the original image, extracting basic content and structural features;

[0074] Finally, through a high-frequency modulation cross-modal fusion module, the pure forgery clues extracted from the auxiliary branches are used to intelligently and explicitly guide and modulate the features of the main branches to amplify and uncover the subtle inconsistencies between true and false features. Finally, the classifier outputs the detection results.

[0075] Auxiliary information processing branch:

[0076] To address the core issue of existing auxiliary information containing a large amount of forged and irrelevant information, thus causing feature interference, this invention designs an Adaptive Forged Feature Separation Module (AFFS). This module is specifically designed to extract pure, interference-free, and generalizable forged-related features from the mixed high-frequency features in the auxiliary information branch, providing high-quality guiding signals for the subsequent backbone network.

[0077] The input image I is first subjected to discrete wavelet transform (DWT) to extract its high-frequency components, forming a high-frequency spectrum I. h I h Subsequently, feature extraction is performed using a standard feature encoder (i.e., EfficientNet) to obtain an initial high-dimensional feature map F. h The purpose of this step is to map the high-frequency spectrum from the pixel space to the deep feature space to facilitate more complex subsequent processing.

[0078] Initial high-frequency feature map F hThis was then incorporated into the core innovation of this branch—the adaptive forgery feature separation module (the part within the dotted line in the red box in the figure). The goal of the adaptive forgery feature separation module is to separate the mixed features F h It intelligently decomposes into two semantically completely separate parts: the forgery-related part and the forgery-irrelevant part. Specifically, when the feature map F... h When entering the adaptive forged feature separation module, the module internally uses two parallel, weight-distributed convolutional layers to separate F... h Simultaneously mapped to two outputs: a forged relevant feature map f hf Unrelated feature map F ho To ensure the purity and generalization of these two output feature maps, this invention introduces two key, jointly optimized loss functions as constraints: spatial mutual exclusion constraint and entropy maximization constraint.

[0079] Spatial Mutual Exclusion Constraint L ex Based on the physical fact that any pixel in an image is either real or tampered with, there is no ambiguous intermediate state, this invention designs a loss function that mandates that at any spatial pixel location (h,w,c), F... hf and F ho The activation values ​​of the features cannot all be positive simultaneously. This is achieved by minimizing the sum of squares of their pointwise products, the core idea of ​​which can be intuitively understood using the following formula:

[0080]

[0081] This constraint ensures that the two feature maps are spatially mutually exclusive, fundamentally preventing the leakage of real texture information irrelevant to the forgery into the forgery-related feature map, thereby achieving pixel-level precise decoupling and purification.

[0082] While ensuring purity, this invention also requires F hf This invention aims to identify multiple different types of forgery techniques, rather than simply memorizing a specific artifact from the training data. To this end, the invention introduces an entropy maximization constraint. The invention predefines h learnable prototype forgery feature clustering centers C. k For f hf For each feature vector in the dataset, this invention calculates the probability p(k|F) that it is assigned to a particular cluster center. hf The goal of the entropy maximization constraint is to make the probability distribution as uniform as possible, rather than concentrating it on a few clusters. Its loss function L... en This is achieved by minimizing negative entropy, as shown in the formula:

[0083] L en =-p(k|F hf)logp(k|F hf )

[0084] Minimize L en Maximizing entropy forces the model to learn more generalized forgery patterns that are common to multiple forgery techniques, effectively avoiding overfitting to specific artifacts. Through this ingenious decomposition and dual constraints, the auxiliary information generation branch can extract a high-quality, high-purity, and highly generalized forgery-related feature map F from the original, mixed high-frequency information. hf This provides a perfect, interference-free guiding signal for subsequent cross-modal interactions with the main branches.

[0085] Main branch detection:

[0086] To address the lack of explicit modeling of inconsistencies between genuine and fake features in existing technologies, this invention designs a self-guided enhancement module and a high-frequency modulation cross-modal fusion module. These two modules work together to not only enhance the feature representation capabilities of the backbone network itself, but more importantly, they utilize the pure high-frequency cues extracted from auxiliary branches to proactively and explicitly uncover and amplify the subtle differences between genuine and fake images.

[0087] The input image I is first processed by a standard feature encoder to extract features, resulting in an initial backbone feature map F. r This feature map preserves the main content and structural information of the image, serving as the basis for subsequent inconsistency analysis. Initial backbone feature map F r It is then fed into a series of core modules of this invention, and its processing flow is as follows:

[0088] First, the self-guided enhancement module is responsible for the backbone feature F r Preliminary adaptive enhancements are performed to prepare for subsequent fusion. It also employs a dual-channel parallel processing strategy:

[0089] The module first inputs features F r Divided into two parts along the channel dimension C on average and

[0090] The high-frequency activation feature map M is extracted using a max pooling layer. h This feature is then enhanced by a 1x1 convolution.

[0091] The low-frequency activation feature map M is then extracted using an average pooling layer. l It is also enhanced by a 1x1 convolution.

[0092] The enhanced high- and low-frequency features are reassembled and then passed through a learnable hyperparameter a and the original input F. r Perform weighted fusion. As shown in the formula:

[0093]

[0094] The w here h and w l This is a convolutional layer, where ⊙ represents element-wise multiplication. This step ensures that while preserving the original information, the inherent high- and low-frequency components of the backbone features are effectively enhanced.

[0095] The high-frequency modulation cross-modal fusion module receives the enhanced backbone feature F' from the self-guided enhancement module. r And the pure forgery feature F from the auxiliary branch hf It then performs a high-frequency guided cross-modal inconsistency mining. Internally, it employs an innovative mechanism similar to Transformer self-attention. Specifically, the module first processes the enhanced backbone features F' r The query is transformed into a query(Query, F) through three parallel 1x1 convolutional layers. q ), key (Key, F) k ) and Value (Value, F v However, the "keys" here have been specially designed:

[0096] It is the main feature F' r With the pure high-frequency feature F from the auxiliary branch hf The addition is followed by transformation. As shown in the formula:

[0097] F q =w q (F' r ),F k =w k (F' r +F hf ),F v =w v (F' r )

[0098] By using pure, fake high-frequency information F hf Injected into the "key" F k This allows subsequent attention calculations to be acutely aware of the presence and location of the forgery.

[0099] The module first calculates query F q and specially designed key F k The matrix multiplication between them yields a consistency matrix M. c .

[0100] At the same time, the module uses a lightweight convolutional network to process the clean, high-frequency fake features F. hf Convert to a high-frequency gated weight graph M hf .

[0101] This weighted graph is equivalent to a fake region indicator, with values ​​between 0 and 1.

[0102] This invention will use high-frequency attention map M hf Element-wise multiplication by the consistency matrix M c .

[0103] This operation means that the present invention is only concerned with those areas within the high-frequency forgery region (by M). hf The inconsistency in characteristics exhibited by the indication (by M) c reflect).

[0104] This allows the model to very accurately focus on the core differences between real and fake images. Finally, the gated attention weights are applied to the value F. v The results are then compared with the original backbone features F. r Perform residual connections to obtain the final fusion feature F used for classification. m As shown in formula (15):

[0105]

[0106] Through this series of ingenious guiding and fusion steps, the main detection branch not only enhances itself but also, guided by pure high-frequency information, successfully and explicitly models the subtle inconsistencies between true and false features. Finally, F... m Complete the classification of counterfeit detection.

[0107] The effect after improvement:

[0108] The experiments in this invention mainly consist of two sets: one set verifies the generalization effect within and across domains, and the other set verifies the robustness under different forgery methods and compression levels. This invention uses AUC (Area Under the Curve) as the core evaluation metric. AUC comprehensively measures the classification performance of the model across all possible thresholds; the closer the value is to 1, the stronger and more reliable the model's detection capability.

[0109] Generalization performance verification experiments in intra-domain and cross-domain scenarios:

[0110] This experiment aims to evaluate the model's comprehensive performance in both intra-domain and cross-domain scenarios, which are core benchmarks for measuring the detector's practicality. Intra-domain testing involves training and testing on different subsets of forgery methods on the FaceForensics++ (FF++) dataset to assess the model's ability to fit known data distributions. Cross-domain testing follows the most stringent testing paradigm, training on the FF++ dataset and then testing on three completely unseen public datasets (CDF, ​​DFDCP, DFD) with different data sources and forgery techniques. This setup aims to simulate real-world scenarios where defenses face completely unknown sources and attack methods.

[0111]

[0112] Table 1 shows the generalization effect in intra-domain and cross-domain scenarios.

[0113] Table 1 illustrates the superior performance of this invention in two test scenarios. The method of this invention achieved an average AUC of 80.76% in cross-domain testing, ranking first among all compared methods and significantly surpassing many advanced methods, including UCF and UMFC. Compared to baseline methods such as Xception, the average cross-domain performance of this invention shows an absolute improvement of 4% to 5%. This proves that the strategy of introducing auxiliary information for guidance in this invention is correct and effective. Compared to methods such as UCF, which also introduce auxiliary information but contain irrelevant forged information, the method of this invention still has a significant performance advantage. This fully demonstrates that this method solves the fundamental defects of existing auxiliary information-based methods through decoupling and purification, successfully learning a more universal forgery feature representation, thus maintaining a very high detection level even when facing a large number of forgery attacks from unknown sources.

[0114] Robustness verification experiments across forgery methods and compression levels:

[0115] This experiment aims to evaluate the robustness of the model across more challenging forgery methods and compression levels. First, the model is trained using only a high-quality (HQ) version of one forgery method (FF-NT), and then tested on both high-quality (HQ) and low-quality (LQ, i.e., heavily compressed) versions of a completely different forgery method (FF-DF). The experimental results are shown in Table 2.

[0116]

[0117] Table 2 shows the generalization results across forgery methods and compression levels.

[0118] As can be seen, on the DF(HQ) test set, the AUC of this invention reaches 67.6%, significantly outperforming all comparative methods. This indicates that, through entropy maximization constraints, the adaptive separation module of this invention successfully learns a general forgery pattern across different forgery techniques (from NT to DF), rather than specific artifacts. On the DF(LQ) test set, the AUC of this invention reaches 86.5%. More notably, from HQ to LQ, the performance of other methods drops significantly by 21% to 28%, while the performance of this invention improves by 18.9%. This result demonstrates that the proposed scheme is extremely sensitive to inconsistencies. Image compression simultaneously destroys the texture of both real and forged regions, but the manner and extent of destruction differ, which amplifies the inconsistencies between them. The proposed scheme successfully captures this new inconsistency cue introduced by compression and transforms it into effective discriminative features, thereby achieving superior performance on low-quality images.

[0119] Then, the present invention uses compressed FF++ as training and tests it on unseen datasets CDF, WDF and DFDC to verify the model's learning ability in real detection scenarios (i.e., the collected training data may be compressed). The experimental results are shown in Table 3.

[0120]

[0121] Table 3 shows the learning ability performance on compressed data.

[0122] In summary, the proposed solution achieved excellent performance in various detection scenarios, fully demonstrating its generalization and robustness against unknown forgery methods, and proving its effectiveness. Besides setting new research benchmarks in key performance indicators, the proposed solution also offers a new research direction for addressing the core problem of the difficulty in generalizing forgery detectors.

[0123] This invention provides a deep forgery detection system based on adaptive decomposition and high-frequency modulation, comprising: an auxiliary information processing branch, including: a high-frequency extraction module, used to extract the high-frequency part of the input image through discrete wavelet transform to obtain a high-frequency spectrum; a feature encoder extracts features from the high-frequency spectrum to obtain an initial high-dimensional feature map; an adaptive forgery feature separation module, used to decompose the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map; a backbone detection branch, including: a feature encoder, used to extract features from the input image to obtain an initial backbone feature map; a self-guided enhancement module, used to perform preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map; a high-frequency modulation cross-modal fusion module, which receives the enhanced backbone feature map and the forgery-related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain a final fused feature map for classification; and a classifier, which uses the fused feature map to complete forgery detection classification.

[0124] Furthermore, it also includes a gating unit, which is located between the high-frequency modulation cross-modal fusion module and the adaptive forgery feature separation module. The gating unit is used to explicitly mine and amplify the high-frequency inconsistencies in local areas of genuine and fake images.

[0125] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0126] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, one embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described terminal embodiment, such that the processor performs the deep forgery detection method based on adaptive decomposition and high-frequency modulation described in the above-described embodiment.

[0128] Those skilled in the art will understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0129] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

[0130] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A deepfake detection method based on adaptive decomposition and high-frequency modulation, characterized in that, Including the following steps: Obtain the input image; In the auxiliary information processing branch: the input image is extracted by discrete wavelet transform to obtain a high-frequency spectrum, and the feature encoder performs feature extraction on the high-frequency spectrum to obtain an initial high-dimensional feature map; The adaptive forged feature separation module decomposes the mixed initial high-dimensional feature map into two semantically completely separate parts: a forged relevant feature map and a forged irrelevant feature map, including the following steps: Two key, jointly optimized loss functions are introduced as constraints: spatial mutual exclusion constraint and entropy maximization constraint; When the initial high-dimensional feature map The adaptive forgery feature separation module then proceeds. Internally, this module uses two parallel, weight-distributed convolutional layers to separate the forged features. Simultaneously mapped to two outputs: a forged related feature map. Unrelated feature maps to forgery ; In the backbone detection branch: the input image is processed by a feature encoder to extract features and obtain an initial backbone feature map; The self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map; The high-frequency modulation cross-modal fusion module receives the enhanced backbone features and the forged correlation feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain the final fused feature map for classification, including the following steps: Enhance the backbone feature map The query is transformed using three parallel 1x1 convolutional layers. ), key ) and Value ) ; Enhance the backbone feature map Feature maps related to forgery from auxiliary branches After adding them together, the transformation is performed as shown in the formula: By forging related feature maps Injected into the "key" This allows subsequent attention calculations to be acutely aware of the presence and location of forgeries; Module Calculation Query and specially designed keys Matrix multiplication between them yields a consistent matrix. ; Using a lightweight convolutional network, the forged relevant feature maps are generated. Convert to a high-frequency gated weight graph ; High-frequency attention maps Element-wise multiplication by the consistency matrix ; The gated attention weights are applied to the value. The results were then compared with the initial backbone feature map. Perform residual connections to obtain the final fused feature map used for classification. As shown below: ； The fused feature map is used to complete the forgery detection and classification.

2. The deep forgery detection method based on adaptive decomposition and high-frequency modulation according to claim 1, characterized in that, Spatial mutual exclusion constraint and entropy maximization constraint: Spatial Mutual Exclusion Constraint Forced at any spatial pixel location superior, and The activation values ​​of the features cannot all be positive simultaneously; this is achieved by minimizing the sum of squares of their pointwise products, as shown in the following formula: Entropy maximization constraint: Preset A learnable prototype forgery feature clustering center ,for For each feature vector in the dataset, calculate the probability of it being assigned to a specific cluster center. The goal of the entropy maximization constraint is to make the probability distribution as uniform as possible; the entropy maximization constraint loss function... This is achieved by minimizing negative entropy, as shown in the formula: 。 3. The deep forgery detection method based on adaptive decomposition and high-frequency modulation according to claim 1, characterized in that, The self-guided enhancement module performs preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map, including the following steps: The initial backbone feature map Along the channel dimension Divided into two equal parts and ; Its high-frequency activation feature map is extracted using a max pooling layer. The high-frequency activation feature map is enhanced by a 1x1 convolution; The low-frequency activation feature map is then extracted using an average pooling layer. It is enhanced by a 1x1 convolution; The enhanced high-frequency activation feature maps and low-frequency activation feature maps are reassembled and then passed through a learnable hyperparameter. With the initial backbone feature map Perform weighted fusion, as shown in the formula: and It is a convolutional layer. This represents element-wise multiplication.

4. A deepfake detection system based on adaptive decomposition and high-frequency modulation, applied to the deepfake detection method based on adaptive decomposition and high-frequency modulation as described in any one of claims 1-3, characterized in that, include: The auxiliary information processing branch includes: The high-frequency extraction module is used to extract the high-frequency part of the input image through discrete wavelet transform to obtain a high-frequency spectrum. The feature encoder performs feature extraction on the high-frequency spectrum to obtain an initial high-dimensional feature map. An adaptive forgery feature separation module is used to decompose the mixed initial high-dimensional feature map into two semantically completely separate parts: a forgery-related feature map and a forgery-irrelevant feature map; The main detection branches include: A feature encoder extracts features from the input image to obtain an initial backbone feature map; The self-guided enhancement module is used to perform preliminary adaptive enhancement on the initial backbone feature map to obtain an enhanced backbone feature map. The high-frequency modulation cross-modal fusion module receives the enhanced backbone features and the forged related feature map, and performs a high-frequency guided cross-modal inconsistency mining to obtain the final fused feature map for classification. The classifier uses the fused feature map to perform forgery detection classification.

5. The deepfake detection system based on adaptive decomposition and high-frequency modulation according to claim 4, characterized in that, Also includes: A gating unit is located between the high-frequency modulation cross-modal fusion module and the adaptive forgery feature separation module. The gating unit is used to explicitly mine and amplify the high-frequency inconsistencies in local areas of genuine and fake images.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the deep forgery detection method based on adaptive decomposition and high-frequency modulation as described in any one of claims 1 to 3.

Images