Method and device for solving deep forgery detection through annular frequency segmentation

Abstract

The invention discloses a method and device for solving depth forgery detection through annular frequency segmentation, and the method comprises the steps: obtaining a corresponding spectrogram If through the discrete Fourier transform of an original image, and inputting a depth forgery detection model; the deep forgery detection model comprises a FreqFormer module and a FreqFuser module, and the FreqFormer module further comprises a FreqAttn sub-module, and the FreqAttn sub-module comprises a FreqAttn sub-module and a FreqAttn sub-module; the FreqFormer module generates a frequency mark fF based on the spectrogram If, inputs the frequency mark fF into the FreqAttn sub-module, divides the frequency mark fF according to frequency bands, then performs in-band attention interaction and inter-band attention interaction to obtain an enhanced frequency band mark, and inputs the enhanced frequency band mark into the FreqFuser module; the FreqFuser module carries out attention interaction on the image block feature fI and an enhanced frequency band mark # imgabs0 # to obtain fusion frequency information; and carrying out attention interaction on the text feature # imgabs1 # and the category feature fc by utilizing fusion frequency information so as to calculate the category prediction probability and realize deep counterfeiting detection. According to the method, repeated frequency extraction and utilization are realized through the annular mask, and deep counterfeiting is more effectively checked.

Description

Technical Field

[0001] The present invention relates to deepfake detection technology, and in particular to a method and device for solving deepfake detection by using circular frequency segmentation. Background Art

[0002] In recent years, the development of deepfake technology has made highly realistic synthetic facial content (including images and videos) widely available, which poses significant security risks and threatens trust online and in society. To address these challenges, effective methods are urgently needed to detect facial forgeries.

[0003] For the detection of forged content, various methods have been developed in the spatial domain for effective detection. However, these methods often struggle to detect subtle alterations, such as slight modifications or compression artifacts. In contrast, introducing frequency domain analysis has been shown to effectively reveal unique patterns that are invisible in the spatial domain, thereby improving the accuracy of deepfake detection. Although frequency domain-based methods perform well, they suffer from a key flaw in the spectrum processing stage, specifically the direct processing of the spectrogram as block patches.

[0004] As attached Figure 1 (b) As shown in the first column, existing frequency-domain-based methods, whether using convolutional neural networks (CNN) or Vision Transformer (ViT) architectures, usually divide the spectrum into grid patches as input. However, this approach will lead to energy truncation when the frequency radius is inscribed in the input image size, as shown in the following figure. Figure 1 (b) Furthermore, unlike spatial signals, which have local correspondence, the spectrum is distributed radially, meaning that the characteristics of the signal are determined solely by its distance from the poles. Clearly, traditional patch-based methods do not align with the underlying characteristics of the frequency domain, and directly and crudely using patch-based methods for frequency extraction is an incomplete and insufficient operation. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem in the prior art that block patch segmentation of a spectrum graph results in incomplete and insufficient frequency utilization.

[0006] The technical solution adopted by the present invention to solve the technical problem is to provide a method for deep fake detection by using circular frequency segmentation, including the following steps:

[0007] Given an original image, a corresponding spectrum is obtained through discrete Fourier transform and input into a deep fake detection model for forgery detection; the deep fake detection model includes a FreqFormer module and a FreqFuser module, wherein the FreqFormer module also includes a FreqAttn submodule;

[0008] In the FreqFormer module, several concentric ring masks are designed based on the spectrogram. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several band markers. For each band marker, intra-band attention interaction and inter-band attention interaction are performed to obtain several enhanced band markers, which are input into the FreqFuser module.

[0009] A text encoder is used to generate a category text feature corresponding to the enhanced frequency band mark; an image encoder is used to generate an image block feature and a category feature based on the image block of the original image; and the category text feature, category feature and image block feature are input into the FreqFuser module;

[0010] The FreqFuser module performs attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

[0011] Preferably, the design of several concentric annular masks based on the spectrum graph and the generation of frequency markers after convolution and fully connected layers include the following steps:

[0012] Calculate Spectrum I f The length R of the inscribed circle radius determines the width of each ring, which is expressed as:

[0013] R = min(H,W) / 2;

[0014] R step =R / N r ;

[0015] Among them, N r Indicates the number of preset rings; H and W represent the spectrum graph I f The height and width of R step Indicates the width of the ring;

[0016] Generate N based on the width of each ring r concentric ring masks M i , expressed as:

[0017]

[0018] Among them, Mi represents the i-th ring mask;

[0019] Using the ring mask M i Get the frequency mark f F , expressed as:

[0020] P i =I f ⊙M i ;

[0021] f F =FC(Conv H×W (P i ));

[0022] Among them, P i Indicates the corresponding ring mask M i The ring patch; FC represents the fully connected layer, Conv H×W Indicates a convolution operation where the convolution kernel is as large as the original spectrum graph; ⊙ indicates a dot product operation.

[0023] Preferably, the frequency markers are divided according to frequency bands to obtain a number of frequency band markers, specifically: setting a low-medium frequency band threshold τ m and the mid-high frequency band threshold τ h , using these two frequency thresholds to F Divided into three frequency bands θ∈{low frequency, medium frequency, high frequency}.

[0024] Preferably, performing intra-band attention interaction and inter-band attention interaction on each frequency band marker to obtain a plurality of enhanced frequency band markers comprises the following steps:

[0025] Perform intra-band attention interaction for each band marker, expressed as:

[0026]

[0027] Among them, Intra-bandAttn represents the intra-band attention interaction, express Results of within-band attention interactions were conducted;

[0028] Calculate the band average feature, expressed as:

[0029]

[0030] Among them, Mean represents the average operation; represents the frequency band average characteristics;

[0031] Perform inter-band attention interaction for each band marker, expressed as:

[0032]

[0033] Among them, Inter-bandAttn represents the attention interaction between frequency bands, express Results of between-band attention interactions were conducted; Represents the average characteristics of all frequency bands The splicing result of

[0034] Mark each frequency band Perform enhancement, expressed as:

[0035]

[0036] in, for The low-mid frequency band threshold τ m and the mid-high frequency band threshold τ h The result after division.

[0037] Preferably, performing attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information comprises the following steps:

[0038] Using image block features f I and Enhanced Band Marking Obtain block features with different frequency perception Then with the category feature f c Interactively, generating frequency-specific features for each band Expressed as:

[0039]

[0040] Among them, MHA represents the multi-head self-attention mechanism;

[0041] Using image block features f I Fusion frequency marker f F , obtain full frequency perception block features

[0042] Block feature with frequency-specific characteristics Splice them together to obtain fusion frequency information

[0043] Preferably, the method of using the fusion frequency information to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model, includes the following steps:

[0044] Using fused frequency information The interaction between category text features and category features is realized as follows:

[0045]

[0046] Among them, η∈{c,T} represents the feature type; when η=T, f T Represents the category text feature; when η=c, f c Represents category characteristics;

[0047] The predicted probability is calculated using the interaction results, expressed as:

[0048]

[0049] Among them, sim represents the cosine similarity between two vectors, which measures the similarity between vectors; P(j) represents the corresponding classification category obtained by similarity calculation.

[0050] Preferably, the loss function of the deep fake detection model is:

[0051]

[0052] in, represents the contrast loss term, represents the orthogonal frequency division loss term, represents the integrated loss term; θ1,θ2∈{low frequency, medium frequency, high frequency}, represents the j-th enhanced frequency band feature of the θ1 frequency band, represents the kth enhanced band feature of the θ2 band, l1 and l2 represent and The length of θ i ,θ j ∈{low frequency, medium frequency, high frequency}, and They represent θ generated by the text encoder i Band text features and θ of the frequency band j Band text features of the band, represents θ i Class characteristics of frequency bands.

[0053] The present invention also provides a device for solving deep fake detection by using circular frequency segmentation, comprising:

[0054] A spectrum conversion module, given an original image, obtains a corresponding spectrum through discrete Fourier transform, and inputs it into a deep fake detection model for forgery detection; the deep fake detection model includes a FreqFormer module and a FreqFuser module, wherein the FreqFormer module also includes a FreqAttn submodule;

[0055] In the ring segmentation module, in the FreqFormer module, several concentric ring masks are designed based on the spectrum graph. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several band markers. For each band marker, intra-band attention interaction and inter-band attention interaction are performed to obtain several enhanced band markers, which are input into the FreqFuser module.

[0056] The feature encoding module generates a category text feature corresponding to the enhanced frequency band mark using a text encoder; generates an image block feature and a category feature based on the image block of the original image using an image encoder; and inputs the category text feature, category feature and image block feature into the FreqFuser module;

[0057] The frequency prediction module, the FreqFuser module, performs attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

[0058] The present invention has the following beneficial effects:

[0059] (1) The present invention proposes a frequency domain processing method based on circular segmentation, which overcomes the problem of frequency energy truncation caused by traditional block patch segmentation in spectrum graph processing, realizes full utilization and accurate grouping of frequency signals, thus being more in line with the radial characteristics of frequency domain signals and significantly improving the accuracy of forgery detection.

[0060] (2) The present invention designs a novel network architecture FreqNext, which includes two core modules: FreqFormer and FreqFuser. FreqFormer performs feature interaction within and between frequency bands through ring segmentation and FreqAttn mechanism respectively. FreqFuser further optimizes frequency domain features through vision-language fusion, successfully realizing efficient integration of multimodal information.

[0061] (3) Experimental results on multiple benchmark datasets demonstrate that our method achieves significant performance improvements in forgery detection tasks. For example, compared to the current best frequency-domain method, the AUC improves by 11.26% on the CelebDF-v2 dataset and by 12.45% on the DFDCP dataset, fully demonstrating the superiority of our method.

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1Schematic diagram of frequency truncation and different patch segmentation methods;

[0064] Figure 2 A diagram showing the steps of a method according to an embodiment of the present invention;

[0065] Figure 3 is a flow chart of a method according to an embodiment of the present invention;

[0066] Figure 4 Grad-CAM diagrams of different categories according to an embodiment of the present invention;

[0067] Figure 5 A feature visualization diagram of the embodiment of the present invention and other methods;

[0068] Figure 6 This is a schematic diagram showing the effect of different mask ratios on the original image according to an embodiment of the present invention;

[0069] Figure 7 2 is a structural diagram of a device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] See also Figure 1 As shown in the figure, directly using block patches for frequency extraction is an incomplete and insufficient operation. Inspired by the local similarity of spatial signals, the goal is to group frequency domain information based on frequency similarity. To this end, a new partitioning method is explored based on the radial distribution of the spectrum, and various types of frequency domain blocks are designed, including fan-shaped blocks with the same angle, annular blocks with the same radius, and a combination of the two, such as Figure 1 (b) The second to fourth columns show the best performance among these designs. This method can effectively group frequency signals close to each other, allowing for more accurate and meaningful analysis in the frequency domain.

[0071] To better utilize ring-patched input, a novel network architecture, FreqNext, is proposed as a deepfake detection model. FreqNext consists of two key components: FreqFormer and FreqFuser, which play distinct roles in optimizing patch processing and promoting cross-ring interaction, respectively. In the FreqFormer module, the spectrum is divided into concentric rings based on radius, with each ring representing a different frequency band, ranging from low to high frequency. Convolution operations are applied to these bands and embedded into feature tokens via a shared MLP. To enable effective interaction between these tokens, the FreqAttn mechanism is introduced, which first performs inter-band normalization and then intra-band interaction. Furthermore, an orthogonal frequency division loss is added to enhance the discriminability of each ring token, thereby achieving better feature separation. The FreqFuser module is then introduced to further enhance the comprehensiveness of features by combining it with a pre-trained vision-language model. This vision-language model has proven effective in solving image-related problems in multimodal scenarios. At this stage, frequency-domain features are processed with corresponding low-, medium-, and high-frequency text information. By fusing these frequency features with visual information, a robust feature set is constructed to determine the authenticity of the content.

[0072] For details, see Figure 2 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0073] S101: Given an original image, obtain the corresponding spectrum through discrete Fourier transform and input it into the deep fake detection model for fake detection;

[0074] In the FreqFormer module, several concentric ring masks are designed based on the spectrogram. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several band markers. Intra-band attention interaction and inter-band attention interaction are performed on each band marker to obtain several enhanced band markers, which are input into the FreqFuser module.

[0075] S103, using a text encoder to generate a category text feature corresponding to the enhanced frequency band mark; using an image encoder to generate an image block feature and a category feature based on the image block of the original image; inputting the text feature, category feature and image block feature into a FreqFuser module;

[0076] In S104, in the FreqFuser module, attention interaction is performed on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to pay attention to the text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

[0077] Specifically, the detailed process is as follows Figure 3 As shown, the deepfake detection model includes the FreqFormer module and the FreqFuser module, where the FreqFormer module also includes the FreqAttn submodule. The following details the processing flow of each step.

[0078] Specifically, in S101, for the input image I∈R H×W×C , first obtain its corresponding grayscale image I g ∈R H×W , then the initial frequency feature φ is obtained by discrete Fourier transform operation φ(I g ), due to the characteristics of discrete Fourier transform, it is centralized to obtain the spectrum image I of FreqFormer input f ∈R H×W .

[0079] Specifically, in S102, based on the analysis of frequency characteristics and related experiments, the ring mark is introduced into the frequency learning. f , calculate the radius of its inscribed circle R = min(H,W) / 2. Here, a hyperparameter N is defined r , representing I f The number of ring divisions is set to 28 in this embodiment of the present invention. The width of each ring R step =R / N r , according to R step , generate n concentric ring masks M i :

[0080]

[0081] Where i∈{0,1,…,n-1}. Use the ring mask M i Get Annular Patch (Annular Patch) i :

[0082] P i =I f ⊙M i .

[0083] For each P i , a convolution kernel of the same size as the original spectrogram is applied to Pi A convolution operation is performed on the patch token to obtain the patch token. Next, a fully connected layer (FC) is used to perform normalized non-identical distribution features to form frequency tokens. (where D f In FreqFormer, it represents the feature dimension. The specific calculation process is as follows:

[0084] f F =FC(Conv H×W (P i )).

[0085] Specifically, a FreqAttn submodule is designed in the FreqFormer module to promote the targeted learning of frequency-specific information. According to the previous research methods on frequency, in order to consider the different effects of different frequency bands, two frequency thresholds are set: low-medium frequency band threshold τ m and the mid-high frequency band threshold τ h These thresholds will f F Divided into three frequency bands Where θ∈{low frequency, medium frequency, high frequency}. For each frequency band feature Apply multi-head attention mechanism to enhance the information within the frequency band:

[0086]

[0087] Multiple frequency tokens are divided into low-band, mid-band, and high-band in ascending order. Each band contains multiple tokens. Intra-bandAttn is the interaction of attn within each band: tokens within each band learn from each other through a multi-head attention mechanism. Inter-band Attn follows Intra-band Attn and learns information across frequency bands: the token information within each band is averaged and the head-attention mechanism is used to learn between different frequency bands.

[0088] After refining the feature representation within each frequency band, the average feature within the frequency band is calculated. To represent the information of each frequency band. The average values of low, medium and high frequency bands are concatenated into a vector To carry out subsequent cross-band interactions. The cross-band attention mechanism captures the interaction between frequency bands, thereby generating

[0089]

[0090] Finally, the residual connection enables each token to learn the residual information of its frequency band. By subtracting its mean And add the cross-band attention results To update:

[0091]

[0092] Specifically, in S103, f I Image features are directly extracted using Image Encoder. The Image Encoder feature extractor in the embodiment of the present invention uses CLIP ViT / B-16. CLIP is an image-text multimodal encoder proposed in previous work. For the category description text, a natural language description similar to "This is a [real\fake] face image" is used to construct it. Each category has multiple descriptions, which are passed through the text encoder E T The obtained multiple features are averaged to represent the corresponding text features of the category as the category text features The frequency band descriptions of different frequency bands are as follows: low frequency is described as “large smooth areas and related skin textures in face images”, medium frequency is described as “medium details in face images”, and high frequency is described as “clear edges and overall contours in face images”. T Process these descriptions to generate frequency-aware band text features Among them, θ∈{low frequency, medium frequency, high frequency}, in order to improve the diversity and richness of text description, multiple related description texts are integrated for ensemble learning to achieve descriptive representation.

[0093] Specifically, in the S104, the frequency description text guides the frequency learning. In FreqFuser, the frequency-based text description is first applied in a cross-modal manner to enhance the distinction between different frequency bands, and then the features are fused based on the rich frequency information. First, cross-modal frequency division enhancement is introduced to highlight the characteristics of a specific frequency band through frequency-aware text. In the multi-band fusion process, the correlation between frequency information and image blocks is considered, rather than the correlation with category language information. In order to obtain frequency-aware category features, first let the image block feature f I With three frequency band characteristics These frequency-specific block features are then interacted with the category features to generate frequency-specific features for each frequency band.

[0094]

[0095] In refining frequency-specific features Then, use f again I Fusion of full frequency features To obtain full frequency aware block features These features are related to frequency-specific features Splice and generate fusion frequency information Maximize the utilization efficiency of frequency information. MHA stands for Multi-head Self-Attention. Both Intra-bandAttn and Inter-bandAttn in the embodiments of the present invention adopt the multi-head self-attention mechanism.

[0096] Specifically, in order to make the classification image and category text features capture rich frequency information, a multimodal frequency perception module is designed. The interaction between category and text features is realized:

[0097]

[0098] Where η∈{c,T} represents two feature types, f T Represents all categories of text features The predicted probability is calculated as follows:

[0099]

[0100] Specifically, during the training of the deep fake detection model, the FLIP method is followed and all its loss function terms are As part of the total loss. In addition, considering the input f of FreqFormer F Contains all frequency information, but there may be redundancy between frequency bands. Here, an orthogonal frequency-division loss is introduced to enhance the mutual orthogonality between frequency bands:

[0101]

[0102] Where θ1,θ2∈{low frequency, medium frequency, high frequency}, l1 and l2 represent and length; represents the j-th enhanced frequency band feature of the θ1 frequency band, represents the kth enhanced frequency band feature of the θ2 frequency band.

[0103] In order to make the frequency-specific features interact with the frequency-aware text features, contrastive learning is adopted:

[0104]

[0105] where θ i ,θ j ∈{low frequency, medium frequency, high frequency}.

[0106] Finally, during the training process, As an integrated part, combined and To construct the overall training loss function:

[0107]

[0108] In the inference stage, the score value of P(i) is used as the calculation method for the final output category probability.

[0109] The embodiments of the present invention were verified by experiments, and the experimental results are as follows: The present invention was trained and tested on deep fake detection datasets (FF++, CelebDF-v2, DFDC, DFD, WildDeepfake). FF++ (two versions with high quality (HQ) and low quality (LQ)) is the most widely used dataset, which contains four types of manipulation techniques, namely Deepfakes (DF), Face2Face (F2F), FaceSwap (FS) and NeuralTextures (NT). The CelebDF-v2 dataset includes 590 real videos and 5,639 high-quality fake videos, which are generated by an improved deep fake algorithm. WildDeepfake is a real-world dataset containing 3,805 real sequences and 3,509 fake sequences. DFD and DFDCP are large-scale datasets shot by professional actors in multiple scenes.

[0110] The experiment established two protocols for evaluating generalization ability: cross-dataset testing and cross-type testing. We used the equal error rate (EER) and area under the receiver operating characteristic curve (AUC) as performance metrics for the proposed method to compare with existing methods. The results are shown in Table 1 below.

[0111] Table 1. Comparison of the FreqNext method with other methods

[0112]

[0113] As can be seen from the table, FreqNext generally outperforms all listed methods on unseen test datasets, and the advantage is often very significant. For example, in testing on the WildDeepfake dataset, FreqNext achieved an AUC score of 83.30%, 2.86% higher than the second-best method, FLIP. In addition, compared with traditional Deepfake detection methods, the AUC score of the embodiment of the present invention is significantly improved by approximately 10%. On the challenging DFDCP dataset, the method of the embodiment of the present invention improves the AUC score by approximately 1% compared to the second-best method.

[0114] From the perspective of leveraging either RGB or frequency information, methods that fail to fully exploit frequency features do not demonstrate significant advantages over RGB-based methods. In contrast, FreqNext demonstrates significant performance gains by digging deeper and specifically applying frequency information. Furthermore, the obtained results demonstrate significant improvements over methods that rely solely on RGB information, further validating the critical role of frequency information in deepfake detection.

[0115] In order to clarify the performance of the model and the basis for judgment, Grad-CAM diagrams under different categories are drawn. Figure 4 As shown in Figure 2, the gradient of the real face (green) is evenly concentrated on the entire face, while the gradients of different attack types clearly highlight the common attack areas around the eyes and mouth. In addition, additional t-SNE feature visualization diagrams and the impact of different mask ratios on the original image are also added. Figure 5 and Figure 6 Through more visual analysis, it is shown that the model focuses on task-related features, such as eyes and mouth, which is beneficial for generalization judgment.

[0116] In summary, the proposed FreqNext method has good results in both performance and interpretability. Judging from the main evaluation indicator AUC, it has achieved the best detection performance in the current Deepfake Detection task.

[0117] See also Figure 7 FIG. 1 is a structural diagram of a device according to an embodiment of the present invention, comprising:

[0118] The spectrum conversion module 701, given an original image, obtains a corresponding spectrum through discrete Fourier transform, and inputs the spectrum into a deep fake detection model for forgery detection; the deep fake detection model includes a FreqFormer module and a FreqFuser module, wherein the FreqFormer module also includes a FreqAttn submodule;

[0119] In the ring segmentation module 702, in the FreqFormer module, several concentric ring masks are designed based on the spectrogram. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several frequency band markers. Intra-band attention interaction and inter-band attention interaction are performed on each frequency band marker to obtain several enhanced frequency band markers, which are input into the FreqFuser module.

[0120] The feature encoding module 703 generates a category text feature and a frequency band text feature corresponding to the enhanced frequency band mark using a text encoder; generates an image block feature and a category feature based on the image block of the original image using an image encoder; and inputs the text feature, category feature, and image block feature into the FreqFuser module;

[0121] In the frequency prediction module 704, FreqFuser module, attention interaction is performed on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to pay attention to the text features and category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

[0122] Extensive evaluation of the embodiments of the present invention on multiple benchmark datasets demonstrates that this method significantly improves performance in cross-domain tasks. Compared to the current frequency-domain SOTA method, FreqNext achieves an 11.26% AUC improvement on CelebDF-v2 and a 12.45% improvement on DFDCP. Furthermore, compared to the RGB-based SOTA method, the method of the embodiments of the present invention achieves a 0.99% improvement, validating the effectiveness of the embodiments of the present invention in improving deepfake detection performance.

[0123] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for solving deep fake detection by ring frequency segmentation, characterized in that: The following steps are involved: Given an original image, a corresponding spectrum is obtained through discrete Fourier transform and input into a deep fake detection model for forgery detection; the deep fake detection model includes a FreqFormer module and a FreqFuser module, wherein the FreqFormer module also includes a FreqAttn submodule; In the FreqFormer module, several concentric ring masks are designed based on the spectrogram. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several band markers. For each band marker, intra-band attention interaction and inter-band attention interaction are performed to obtain several enhanced band markers, which are input into the FreqFuser module. A text encoder is used to generate a category text feature corresponding to the enhanced frequency band mark; an image encoder is used to generate an image block feature and a category feature based on the image block of the original image; and the category text feature, category feature and image block feature are input into the FreqFuser module; The FreqFuser module performs attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

2. The method for solving deep fake detection by ring frequency segmentation according to claim 1 is characterized in that: The method designs several concentric ring masks based on the spectrum graph, generates frequency labels after convolution and fully connected layers, and includes the following steps: Calculate Spectrum I f The length R of the inscribed circle radius determines the width of each ring, which is expressed as: R = min(H,W) / 2; R step =R / N r ; Among them, N r Indicates the number of preset rings; H and W represent the spectrum graph I f The height and width of R step Indicates the width of the ring; Generate N based on the width of each ring r concentric ring masks M i , expressed as: Among them, M i represents the i-th ring mask; Using the ring mask M i Get the frequency mark f F , expressed as: P i =I f ⊙M i ; f F =FC(Conv H×W (P i )); Among them, P i Indicates the corresponding ring mask M i The ring patch; FC represents the fully connected layer, Conv H×W Indicates a convolution operation where the convolution kernel is as large as the original spectrum graph; ⊙ indicates a dot product operation.

3. The method for solving deep fake detection by ring frequency segmentation according to claim 1 is characterized in that: The frequency markers are divided into frequency bands to obtain a number of frequency band markers, specifically: setting the low-medium frequency band threshold τ m and the mid-high frequency band threshold τ h , using these two frequency thresholds to F Divided into three frequency bands θ∈{low frequency, medium frequency, high frequency}.

4. The method for solving deep fake detection by ring frequency segmentation according to claim 3 is characterized in that: The step of performing intra-band attention interaction and inter-band attention interaction on each frequency band marker to obtain a plurality of enhanced frequency band markers includes the following steps: Perform intra-band attention interaction for each band marker, expressed as: Among them, Intra-bandAttn represents the intra-band attention interaction, express Results of within-band attention interactions were conducted; Calculate the band average feature, expressed as: Among them, Mean represents the average operation; represents the frequency band average characteristics; Perform inter-band attention interaction for each band marker, expressed as: Among them, Inter-bandAttn represents the attention interaction between frequency bands, express Results of between-band attention interactions were conducted; Represents the average characteristics of all frequency bands The splicing result of Mark each frequency band Perform enhancement, expressed as: in, for The low-mid frequency band threshold τ m and the mid-high frequency band threshold τ h The result after division.

5. The method for solving deep fake detection by ring frequency segmentation according to claim 1 is characterized in that: The step of performing attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information includes the following steps: Using image block features f I and Enhanced Band Marking Obtain block features with different frequency perception Then with the category feature f c Interactively, generating frequency-specific features for each band Expressed as: Among them, MHA represents the multi-head self-attention mechanism; Using image block features f I Fusion frequency marker f F , obtain full frequency perception block features Block feature with frequency-specific characteristics Splice them together to obtain fusion frequency information 6. The method for solving deep fake detection by ring frequency segmentation according to claim 5 is characterized in that: The method of using the fusion frequency information to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model, includes the following steps: Using fused frequency information The interaction between category text features and category features is realized as follows: Among them, η∈{c,T} represents the feature type; when η=T, f T Represents the category text feature; when η=c, f c Represents category characteristics; The predicted probability is calculated using the interaction results, expressed as: Among them, sim represents the cosine similarity between two vectors, which measures the similarity between vectors; P(j) represents the corresponding classification category obtained by similarity calculation.

7. The method for solving deep fake detection by ring frequency segmentation according to claim 5 is characterized in that: The loss function of the deep fake detection model is: in, represents the contrast loss term, represents the orthogonal frequency division loss term, represents the integrated loss term; θ1,θ2∈{low frequency, medium frequency, high frequency}, represents the j-th enhanced frequency band feature of the θ1 frequency band, represents the kth enhanced band feature of the θ2 band, l1 and l2 represent and The length of θ i ,θ j ∈{low frequency, medium frequency, high frequency}, and They represent θ generated by the text encoder i Band text features and θ of the frequency band j Band text features of the band, represents θ i Class characteristics of frequency bands.

8. A device for solving deep fake detection by ring frequency segmentation, characterized in that: include: A spectrum conversion module, given an original image, obtains a corresponding spectrum through discrete Fourier transform, and inputs it into a deep fake detection model for forgery detection; the deep fake detection model includes a FreqFormer module and a FreqFuser module, wherein the FreqFormer module also includes a FreqAttn submodule; In the ring segmentation module, in the FreqFormer module, several concentric ring masks are designed based on the spectrum graph. After passing through the convolution and fully connected layers, frequency markers are generated and input into the FreqAttn submodule. The frequency markers are divided into frequency bands to obtain several band markers. For each band marker, intra-band attention interaction and inter-band attention interaction are performed to obtain several enhanced band markers, which are input into the FreqFuser module. The feature encoding module generates a category text feature corresponding to the enhanced frequency band mark using a text encoder; generates an image block feature and a category feature based on the image block of the original image using an image encoder; and inputs the category text feature, category feature and image block feature into the FreqFuser module; The frequency prediction module, the FreqFuser module, performs attention interaction on the image block features and the enhanced frequency band labels to obtain fused frequency information; the fused frequency information is used to perform attention interaction on the category text features and the category features, thereby calculating the category prediction probability as the output of the deep fake detection model.

Images