Efficient face deep counterfeiting detection method based on multi-frequency feature aggregation

Abstract

The invention relates to a high-efficiency face deep counterfeiting detection method based on multi-frequency feature aggregation, belongs to the technical field of computer vision and pattern recognition, focuses on detection of a face counterfeiting image, and provides a high-efficiency face deep counterfeiting detection method based on multi-scale spatial feature and multi-frequency domain feature fusion. The method aims at improving the robustness and generalization ability of a detection model under the conditions of coping with different counterfeiting means, noise and frequency distortion, and is suitable for various deep counterfeiting detection application scenes. The multi-context and frequency aggregation network provided by the invention aims to realize efficient and robust deep counterfeiting detection through innovative design of a multi-core aggregator and a multi-frequency aggregator module; the defects of an existing deep forgery detection technology in the aspects of space and frequency feature collaborative modeling, multi-scale feature extraction, high-frequency detail processing and cross-domain generalization ability are overcome.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and pattern recognition technology, and in particular to an efficient face deep fake detection method based on multi-frequency feature aggregation. Background Art

[0002] With the rapid development of generative modeling technology, deep fake facial technology has a potential social and ethical impact far greater than other forms of fake non-face content because of its more realistic imitation of personal identity.

[0003] Current deepfake detection technology has developed a variety of methods, which can be mainly divided into three categories: detection methods based on basic features (naive detectors), detection methods based on spatial domains (spatial detectors), and detection methods based on frequency domains (frequency detectors). However, these methods usually rely on the extraction of heuristic features rather than enhancing the detection capabilities of forged features from the backbone of the network model. Although these methods have improved the detection effect to a certain extent, there are still significant limitations, especially when dealing with evolving forgery techniques and handling different forms of image degradation (such as noise, compression). In addition, traditional detection models still need to be improved in terms of cross-domain generalization capabilities, and it is difficult to effectively detect new forged content that does not appear in the training data.

[0004] In recent years, some studies have attempted to enhance the generalization ability of detection models through strategies such as multi-task learning and heuristic forged data generation, aiming to improve the model's adaptability and ability to distinguish new forgery techniques. However, existing detection models (such as XceptionNet and EfficientNet) mainly tend to learn global features and fail to fully capture the more subtle local forgery features in deep forgery content, resulting in limitations in detecting high-quality forgery content.

[0005] However, existing deepfake detection methods have the following major shortcomings, which lead to deficiencies in detection accuracy, robustness, and cross-domain generalization capabilities:

[0006] 1. Limitations of Separation Processing of Spatial and Frequency Features

[0007] Existing deep fake detection technologies usually rely on single spatial features or frequency feature processing. Traditional spatial feature detection methods focus on extracting forgery traces from local areas, but lack the integration of multi-scale contextual information. Therefore, when processing subtle forgery features, they are easily interfered by background or noise. At the same time, although frequency feature detection can focus on frequency anomalies in forged content, it lacks the support of spatial information and often has weak ability to locate specific forged parts. The limitations of this separation processing will lead to unstable detection effects of the model under different forgery methods or complex environments.

[0008] 2. Insufficient detail capture due to single-scale convolution

[0009] Many existing deepfake detection models rely on single-scale convolutional networks, such as XceptionNet and EfficientNet. These models usually extract features at a fixed scale and have difficulty effectively dealing with multi-scale feature transformations in forged samples. For example, forgery techniques produce forged features of different scales in different areas of the face (such as eyes, lips, skin texture), and single-scale convolutions cannot fully capture these subtle differences. As a result, the model tends to ignore key details in forged samples, resulting in a decrease in detection accuracy.

[0010] 3. Insufficient processing of high-frequency information

[0011] Traditional deepfake detection methods have limited capabilities when processing high-frequency information. For example, classic convolutional networks such as ResNet50 have weak responses in high-frequency areas, making it difficult for the model to capture high-frequency details hidden in the forged content. Many forgery techniques have difficulty accurately reproducing high-frequency features when generating forged images, such as the authenticity of skin texture and edge details, so these high-frequency features are key to detecting forged samples. However, existing technologies fail to pay sufficient attention to these high-frequency anomalies, resulting in the model's lack of high recognition of forged samples.

[0012] 4. The model’s cross-domain generalization ability is insufficient

[0013] Existing detection methods work well when trained on specific datasets, but perform poorly on different datasets or new forged samples. This is because these methods rely too much on specific patterns in the training data and are unable to adapt to the diversity of samples brought about by the continuous evolution of forgery techniques. The lack of cross-domain generalization capability will cause the model to lose its effectiveness when facing new forgery techniques, reducing its robustness in practical applications.

[0014] In summary, the existing technologies have obvious deficiencies in the collaborative modeling of spatial and frequency features, multi-scale feature extraction, high-frequency information processing, and cross-domain generalization capabilities. Summary of the invention

[0015] In view of the above problems, the present invention provides an efficient face deep fake detection method based on multi-frequency feature aggregation. The present invention focuses on detecting face forgery images. Through the fusion of multi-scale spatial features and multi-frequency domain features, the present invention aims to improve the robustness and generalization ability of the detection model in dealing with different forgery methods, noise and frequency distortion, and is suitable for various deep fake detection application scenarios; the multi-context and frequency aggregation network (MkfaNet) proposed in the present invention aims to achieve efficient and robust deep fake detection through the innovative design of multi-kernel aggregator (MKA) and multi-frequency aggregator (MFA) modules, which solves the shortcomings of existing deep fake detection technology in collaborative modeling of spatial and frequency features, multi-scale feature extraction, high-frequency detail processing and cross-domain generalization capability.

[0016] The present invention provides an efficient face deep fake detection method based on multi-frequency feature aggregation, comprising:

[0017] Step S1, let n=1, when n=1, it represents the first data domain; obtain the forged image corresponding to the nth data domain;

[0018] A face deep fake detection model is established using a multi-core aggregator module and a multi-frequency aggregator module; the multi-frequency aggregator module includes: a high-frequency noise enhancement module and a dynamic weight regression module;

[0019] The multi-core aggregator module includes an embedding layer, a multi-scale convolution layer, and an adaptive selection layer;

[0020] Step S2, inputting the forged image corresponding to the nth data domain into the face deep forgery detection model; obtaining the adaptive features of the forged image corresponding to the nth data domain based on the multi-core aggregator module, and representing the adaptive features of the forged image corresponding to the nth data domain as multi-scale spatial information of the forged image corresponding to the nth data domain;

[0021] Preferably, the specific steps of obtaining the adaptive features of the forged image corresponding to the nth data domain in step S2 include:

[0022] Step S21, obtaining a forged image corresponding to the nth data domain, inputting the forged image corresponding to the nth data domain into the multi-core aggregator module, adjusting the resolution of the forged image based on the embedding layer Embedding Stem and mapping it to a preset dimensional space, and obtaining image features of the forged image corresponding to the nth data domain;

[0023] Step S22, inputting the image features of the forged image corresponding to the nth data domain into the multi-scale convolution layer, performing multi-scale convolution processing through multiple groups of convolution kernels, and obtaining the multi-scale features of the forged image corresponding to the nth data domain;

[0024] Step S23, inputting the multi-scale features of the forged image corresponding to the n-th data domain into the adaptive selection layer, weighting each scale feature through the attention mechanism, and obtaining the attention weights of multiple scales of the forged image corresponding to the n-th data domain;

[0025] Step S24, based on the attention weights of multiple scales of the forged image corresponding to the nth data domain, adaptive selection is performed to obtain adaptive features of the forged image corresponding to the nth data domain, and the adaptive features corresponding to the nth data domain are represented as multi-scale spatial information of the forged image corresponding to the nth data domain;

[0026] Preferably, the multi-scale spatial information of the corresponding forged image in step S24 is expressed as:

[0027]

[0028] Among them, Y is the output feature map, which is the feature generated after the convolution operation and is used for subsequent neural network processing; W k is the kth convolution kernel, which is a trainable parameter used to capture the local pattern of input features, such as edges, textures, etc.; F is the input forged image, b k is the bias term of the kth convolution kernel, and * represents the convolution operation.

[0029] Preferably, the expression of the attention weights of multiple scales of the corresponding forged image in step S24 is:

[0030] A=σ(W A GlobalPool(F)+b A )

[0031] Where σ(·) is the activation function, A is the attention weight of the important features of the forged image,

[0032] W A is the weight matrix of the forged image, which is used to weight the input features during the linear transformation process; GlobalPool(F) is the global pooling operation applied to the forged image F as input to reduce the dimension of the feature map and extract global information, b A is a bias term for the fake image, which is used to provide additional adjustment capabilities in the linear transformation.

[0033] Step S3, obtaining updated multi-frequency information of the forged image corresponding to the nth data domain based on the multi-frequency aggregator;

[0034] Preferably, the multi-frequency aggregator module in step S3 is used to enhance the sensitivity of the model to high-frequency information, thereby more accurately distinguishing between real and forged images;

[0035] Preferably, the specific steps of obtaining the updated multi-frequency information of the forged image corresponding to the nth data domain in step S3 include:

[0036] Step S31, inputting the forged image corresponding to the nth data field into the multi-frequency aggregator module;

[0037] Based on the high-frequency noise enhancement module, a fast Fourier transform FFT is performed to perform frequency decomposition to obtain low-frequency information and high-frequency information of the forged image corresponding to the nth data domain; the low-frequency information includes the overall structure of the image; the high-frequency information includes the details and texture of the forged image;

[0038] Step S32, performing convolution processing on the low-frequency information of the forged image to extract the structural information features of the forged image; performing refinement convolution operation and adaptive weighting processing on the high-frequency information to obtain the high-frequency forged detail features of the forged image;

[0039] Step S33: Based on the dynamic weight regression module, the structural information features of the forged image and the high-frequency forged detail features of the forged image are adaptively weighted and fused to obtain multi-frequency information of the forged image;

[0040] Step S34: input the multi-frequency information of the forged image into the high-frequency noise enhancement module, use a bandpass filter to extract the high-frequency features, and obtain the high-frequency features F of the forged image. HF , the expression is:

[0041] F HF =BandPass(F)

[0042] BandPass(·) is a bandpass filter that extracts the high-frequency components of the image.

[0043] Step S35: After convolution and adaptive normalization, the high-frequency features of the forged image are weightedly fused with the multi-frequency information of the forged image to obtain updated multi-frequency information of the forged image, expressed as:

[0044] F HFNEM =σ(W·F HF )·F Original

[0045] Among them, σ(·) is the Sigmoid activation function, W is the learnable weight, and F Original It is the multi-frequency information of the forged image.

[0046] Preferably, the expression of the multi-frequency information of the forged image is:

[0047] F MFA =α·F DC +β·F HC

[0048] Among them, F MFA is the aggregated feature map, which is the result of weighted fusion of DC component and HC component; F DC is the DC component of the input feature, which mainly represents low-frequency information, such as the overall structure of the image; F HC is the high-frequency component HC of the input feature, which mainly represents high-frequency information, such as edge and detail features; α is the first learnable parameter, which is used to adjust the weight of the DC component DC; β is the second learnable parameter, which is used to adjust the weight of the high-frequency component HC.

[0049] Step S4: Based on the feature fusion mechanism, the multi-scale spatial information of the forged image corresponding to the n-th data domain and the updated multi-frequency information of the forged image corresponding to the n-th data domain are fused to obtain the comprehensive features of the forged image in the n-th data region;

[0050] Step S5: Use the attention-based weight generator to predict the forgery probability weight W of each pixel in the forged image corresponding to the nth data region. DFWR , the expression is:

[0051] W DFWR =Softmax(f Attention (F)

[0052] Among them, f Attention (·) is the attention mechanism network, and F represents the input feature map.

[0053] Step S6, adjusting the classification loss function based on the forgery possibility weight of each pixel in the forged image corresponding to the nth data region, obtaining an updated forged image of the nth data region, and using the updated forged image corresponding to the nth data region as the forged image corresponding to the (n+1)th data region;

[0054] Preferably, the classification loss function is expressed as:

[0055]

[0056] Among them, L Weighted is the classification loss, expressed as weighted cross entropy loss, W DFWR is the dynamic weight, L CE is the standard cross entropy loss.

[0057] Step S7, determine whether n is greater than or equal to N, where N represents the total number of data domains. If so, obtain the final face deep fake detection model. If not, set n=n+1 and return to step S1.

[0058] Step S8: Perform face deep fake detection based on the final face deep fake detection model.

[0059] The feature fusion mechanism in the present invention is to fuse the outputs of the multi-core aggregator and the multi-frequency aggregator to obtain a comprehensive feature representation of the forged image. The fusion process adopts a layer-by-layer weighted average method to adaptively integrate spatial information and frequency information to ensure that consistent discriminant features can be extracted from different forged images.

[0060] In the present invention, multiple data regions are trained for cross-domain adaptation. During the training phase, the model is optimized using samples from different data domains to ensure high detection accuracy in cross-domain applications.

[0061] The present invention utilizes diverse data sets to enhance the generalization ability of the model to counterfeiting techniques.

[0062] Furthermore, the different data domains include shooting equipment, lighting conditions or ethnic characteristics. The diversity of data domains reflects the complexity of actual application scenarios and puts forward higher requirements for the generalization of the model.

[0063] The diversified data includes integrating multiple data domains, data enhancement or data style transfer, which can effectively make up for the limitations of a single data domain and improve the detection performance and robustness of the model in different scenarios.

[0064] The present invention also includes, during the training process, using a loss function to optimize the forgery detection performance, wherein the loss function is expressed as:

[0065]

[0066] Among them, L CE The cross entropy loss is used to optimize the classification performance of the model.

[0067] The High Frequency Noise Enhancement Module (HFNEM) in this invention significantly improves the detection accuracy of high-quality forged images, especially in scenarios with abnormal distribution of high-frequency forged details, such as inconsistent textures or abnormal distribution of subtle noise, and enhances the detection capability of high-quality forged content (such as Deepfake images generated using advanced generative models).

[0068] The present invention introduces a dynamic weight regression module, which improves the detection model's specific response ability to the forged area by calculating the dynamic weight of the forged area, especially in complex image scenes, with higher forged positioning accuracy; in the training stage, the classification ability of the forged area is optimized first to reduce the interference of the non-forged area on the detection results; in complex scenes (such as highly compressed images or partial forgeries), the positioning accuracy and robustness of the model are improved. The present invention has achieved a breakthrough in the detection of high-frequency forged data areas and the optimization of forged area weights, so that the performance of the detection model in different forgery scenes is further improved.

[0069] Compared with the prior art, the present invention has at least the following beneficial effects:

[0070] (1) The present invention uses a multi-core aggregator (MKA) module to perform feature extraction at multiple scales, so that within the model receptive domain, the change in the expansion rate of the convolution kernel can adapt to features of different scales, thereby achieving comprehensive capture of forged image details and structural features. Such a design exhibits higher accuracy and robustness in processing different types of forgeries, especially when dealing with complex backgrounds and tiny feature differences. The adaptive selection mechanism of the multi-core aggregator not only improves the fine-grained detection capability of the model, but also reduces the interference of irrelevant features, thereby optimizing the overall detection effect;

[0071] (2) The multi-frequency aggregator (MFA) module in the present invention decomposes the image into DC and high-frequency components. Through independent processing and adaptive fusion, it not only enhances the model's sensitivity to high-frequency forgery details, but also avoids detection bias caused by missing or abnormal high-frequency features. For example, subtle changes in key high-frequency information such as skin texture and edges in forged images can be accurately identified. The model is more robust in the detection of complex forged samples, significantly improving the ability to distinguish forged images.

[0072] (3) Through the synergistic effect of the MKA and MFA modules, the present invention can maintain excellent generalization capabilities on different data domains and forged samples. The multi-scale and multi-band feature aggregation design ensures the consistency and accuracy of the model in cross-domain detection tasks, enabling more flexible application and deployment in diverse scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0074] Figure 1 It is a schematic diagram of an efficient face deep fake detection method based on multi-frequency feature aggregation in an embodiment of the present invention;

[0075] Figure 2 This is a schematic diagram of an efficient face deep fake detection method based on multi-frequency feature aggregation in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0077] A specific embodiment of the present invention, as Figure 1-2 , discloses an efficient face deep fake detection method based on multi-frequency feature aggregation. In order to illustrate the effectiveness of the method proposed by the present invention, the above technical solution of the present invention is described in detail through a specific embodiment below. The specific implementation steps are as follows:

[0078] Step S1, let n=1, when n=1, it represents the first data domain; obtain the forged image corresponding to the nth data domain;

[0079] A face deep fake detection model is established using a multi-core aggregator module and a multi-frequency aggregator module; the multi-frequency aggregator module includes: a high-frequency noise enhancement module and a dynamic weight regression module;

[0080] The multi-core aggregator module includes an embedding layer, a multi-scale convolution layer, and an adaptive selection layer;

[0081] Step S2, inputting the forged image corresponding to the nth data domain into the face deep forgery detection model; obtaining the adaptive features of the forged image corresponding to the nth data domain based on the multi-core aggregator module, and representing the adaptive features of the forged image corresponding to the nth data domain as multi-scale spatial information of the forged image corresponding to the nth data domain;

[0082] Preferably, the specific steps of obtaining the adaptive features of the forged image corresponding to the nth data domain in step S2 include:

[0083] Step S21, obtaining a forged image corresponding to the nth data domain, inputting the forged image corresponding to the nth data domain into the multi-core aggregator module, adjusting the resolution of the forged image based on the embedding layer Embedding Stem and mapping it to a preset dimensional space, and obtaining image features of the forged image corresponding to the nth data domain;

[0084] Step S22, inputting the image features of the forged image corresponding to the nth data domain into the multi-scale convolution layer, performing multi-scale convolution processing through multiple groups of convolution kernels, and obtaining the multi-scale features of the forged image corresponding to the nth data domain;

[0085] Step S23, inputting the multi-scale features of the forged image corresponding to the n-th data domain into the adaptive selection layer, weighting each scale feature through the attention mechanism, and obtaining the attention weights of multiple scales of the forged image corresponding to the n-th data domain;

[0086] Step S24, based on the attention weights of multiple scales of the forged image corresponding to the nth data domain, adaptive selection is performed to obtain adaptive features of the forged image corresponding to the nth data domain, and the adaptive features corresponding to the nth data domain are represented as multi-scale spatial information of the forged image corresponding to the nth data domain;

[0087] Preferably, the multi-scale spatial information of the corresponding forged image in step S24 is expressed as:

[0088]

[0089] Among them, Y is the output feature map, which is the feature generated after the convolution operation and is used for subsequent neural network processing; W k is the kth convolution kernel, which is a trainable parameter used to capture the local pattern of input features, such as edges, textures, etc.; F is the input forged image, b k is the bias term of the kth convolution kernel, and * represents the convolution operation.

[0090] Preferably, the expression of the attention weights of multiple scales of the corresponding forged image in step S24 is:

[0091] A=σ(W A GlobalPool(F)+b A )

[0092] Where σ(·) is the activation function, A is the attention weight of the important features of the forged image,

[0093] W A is the weight matrix of the forged image, which is used to weight the input features during the linear transformation process; GlobalPool(F) is the global pooling operation applied to the forged image F as input to reduce the dimension of the feature map and extract global information, b A is a bias term for the fake image, which is used to provide additional adjustment capabilities in the linear transformation.

[0094] Step S3, obtaining updated multi-frequency information of the forged image corresponding to the nth data domain based on the multi-frequency aggregator;

[0095] Preferably, the multi-frequency aggregator module in step S3 is used to enhance the sensitivity of the model to high-frequency information, thereby more accurately distinguishing between real and forged images;

[0096] Preferably, the specific steps of obtaining the updated multi-frequency information of the forged image corresponding to the nth data domain in step S3 include:

[0097] Step S31, inputting the forged image corresponding to the nth data field into the multi-frequency aggregator module;

[0098] Based on the high-frequency noise enhancement module, a fast Fourier transform FFT is performed to perform frequency decomposition to obtain low-frequency information and high-frequency information of the forged image corresponding to the nth data domain; the low-frequency information includes the overall structure of the image; the high-frequency information includes the details and texture of the forged image;

[0099] Step S32, performing convolution processing on the low-frequency information of the forged image to extract the structural information features of the forged image; performing refinement convolution operation and adaptive weighting processing on the high-frequency information to obtain the high-frequency forged detail features of the forged image;

[0100] Step S33: Based on the dynamic weight regression module, the structural information features of the forged image and the high-frequency forged detail features of the forged image are adaptively weighted and fused to obtain multi-frequency information of the forged image;

[0101] Step S34: input the multi-frequency information of the forged image into the high-frequency noise enhancement module, use a bandpass filter to extract the high-frequency features, and obtain the high-frequency features F of the forged image. HF , the expression is:

[0102] F HF =BandPass(F)

[0103] BandPass(·) is a bandpass filter that extracts the high-frequency components of the image.

[0104] Step S35: After convolution and adaptive normalization, the high-frequency features of the forged image are weightedly fused with the multi-frequency information of the forged image to obtain updated multi-frequency information of the forged image, expressed as:

[0105] F HFNEM =σ(W·F HF )·F Original

[0106] Among them, σ(·) is the Sigmoid activation function, W is the learnable weight, and F Original It is the multi-frequency information of the forged image.

[0107] Preferably, the expression of the multi-frequency information of the forged image is:

[0108] F MFA =α·F DC +β·F HC

[0109] Among them, F MFA is the aggregated feature map, which is the result of weighted fusion of DC component and HC component; F DC is the DC component of the input feature, which mainly represents low-frequency information, such as the overall structure of the image; F HC is the high-frequency component HC of the input feature, which mainly represents high-frequency information, such as edge and detail features; α is the first learnable parameter, which is used to adjust the weight of the DC component DC; β is the second learnable parameter, which is used to adjust the weight of the high-frequency component HC.

[0110] Step S4: Based on the feature fusion mechanism, the multi-scale spatial information of the forged image corresponding to the n-th data domain and the updated multi-frequency information of the forged image corresponding to the n-th data domain are fused to obtain the comprehensive features of the forged image in the n-th data region;

[0111] Step S5: Use the attention-based weight generator to predict the forgery probability weight W of each pixel in the forged image corresponding to the nth data region. DFWR , the expression is:

[0112] W DFWR =Softmax(f Attention (F)

[0113] Among them, f Attention (·) is the attention mechanism network, and F represents the input feature map.

[0114] Step S6, adjusting the classification loss function based on the forgery possibility weight of each pixel in the forged image corresponding to the nth data region, obtaining an updated forged image of the nth data region, and using the updated forged image corresponding to the nth data region as the forged image corresponding to the (n+1)th data region;

[0115] Preferably, the classification loss function is expressed as:

[0116]

[0117] Among them, L Weighted is the classification loss, expressed as weighted cross entropy loss, W DFWR is the dynamic weight, L CE is the standard cross entropy loss.

[0118] Step S7, determine whether n is greater than or equal to N, where N represents the total number of data domains. If so, obtain the final face deep fake detection model. If not, set n=n+1 and return to step S1.

[0119] Step S8: Perform face deep fake detection based on the final face deep fake detection model.

[0120] The feature fusion mechanism in the present invention is to fuse the outputs of the multi-core aggregator and the multi-frequency aggregator to obtain a comprehensive feature representation of the forged image. The fusion process adopts a layer-by-layer weighted average method to adaptively integrate spatial information and frequency information to ensure that consistent discriminant features can be extracted from different forged images.

[0121] In the present invention, multiple data regions are trained for cross-domain adaptation. During the training phase, the model is optimized using samples from different data domains to ensure high detection accuracy in cross-domain applications.

[0122] The present invention utilizes diverse data sets to enhance the generalization ability of the model to counterfeiting techniques.

[0123] Furthermore, the different data domains include shooting equipment, lighting conditions or ethnic characteristics. The diversity of data domains reflects the complexity of actual application scenarios and puts forward higher requirements for the generalization of the model.

[0124] The diversified data includes integrating multiple data domains, data enhancement or data style transfer, which can effectively make up for the limitations of a single data domain and improve the detection performance and robustness of the model in different scenarios.

[0125] The present invention also includes, during the training process, using a loss function to optimize the forgery detection performance, wherein the loss function is expressed as:

[0126]

[0127] Among them, L CE The cross entropy loss is used to optimize the classification performance of the model.

[0128] The High Frequency Noise Enhancement Module (HFNEM) in this invention significantly improves the detection accuracy of high-quality forged images, especially in scenarios with abnormal distribution of high-frequency forged details, such as inconsistent textures or abnormal distribution of subtle noise, and enhances the detection capability of high-quality forged content (such as Deepfake images generated using advanced generative models).

[0129] The present invention introduces a dynamic weight regression module, which improves the detection model's specific response ability to the forged area by calculating the dynamic weight of the forged area, especially in complex image scenes, with higher forged positioning accuracy; in the training stage, the classification ability of the forged area is optimized first to reduce the interference of the non-forged area on the detection results; in complex scenes (such as highly compressed images or partial forgeries), the positioning accuracy and robustness of the model are improved. The present invention has achieved a breakthrough in the detection of high-frequency forged data areas and the optimization of forged area weights, so that the performance of the detection model in different forgery scenes is further improved.

[0130] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An efficient face deep fake detection method based on multi-frequency feature aggregation, characterized in that: include: Step S1, let n=1, when n=1, it represents the first data field; Get the forged image corresponding to the nth data domain; A face deepfake detection model is built using a multi-core aggregator module and a multi-frequency aggregator module; Step S2, inputting the forged image corresponding to the nth data domain into the face deep forgery detection model; Obtaining an adaptive feature of the forged image corresponding to the nth data domain based on the multi-core aggregator module, and representing the adaptive feature of the forged image corresponding to the nth data domain as multi-scale spatial information of the forged image corresponding to the nth data domain; Step S3, obtaining updated multi-frequency information of the forged image corresponding to the nth data domain based on the multi-frequency aggregator; Step S4: Based on the feature fusion mechanism, the multi-scale spatial information of the forged image corresponding to the n-th data domain and the updated multi-frequency information of the forged image corresponding to the n-th data domain are fused to obtain the comprehensive features of the forged image in the n-th data region; Step S5: using a weight generator based on an attention mechanism to predict the forgery possibility weight of each pixel in the forged image corresponding to the nth data region; Step S6, adjusting the classification loss function based on the forgery possibility weight of each pixel in the forged image corresponding to the nth data region, obtaining an updated forged image of the nth data region, and using the updated forged image corresponding to the nth data region as the forged image corresponding to the (n+1)th data region; Step S7, determine whether n is greater than or equal to N, where N represents the total number of data domains. If so, obtain the final face deep fake detection model. If not, set n=n+1 and return to step S1; Step S8: Perform face deep fake detection based on the final face deep fake detection model.

2. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 1 is characterized in that: The multi-core aggregator module includes an embedding layer, a multi-scale convolution layer, and an adaptive selection layer.

3. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 1 is characterized in that: The multi-frequency aggregator module includes a high-frequency noise enhancement module and a dynamic weight regression module.

4. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 2 is characterized in that: The specific steps of obtaining the adaptive features of the forged image corresponding to the nth data domain in step S2 include: Step S21, obtaining a forged image corresponding to the nth data domain, inputting the forged image corresponding to the nth data domain into the multi-core aggregator module, adjusting the resolution of the forged image based on the embedding layer and mapping it to a preset dimensional space, and obtaining image features of the forged image corresponding to the nth data domain; Step S22, inputting the image features of the forged image corresponding to the nth data domain into the multi-scale convolution layer, performing multi-scale convolution processing through multiple groups of convolution kernels, and obtaining the multi-scale features of the forged image corresponding to the nth data domain; Step S23, inputting the multi-scale features of the forged image corresponding to the n-th data domain into the adaptive selection layer, weighting each scale feature through the attention mechanism, and obtaining the attention weights of multiple scales of the forged image corresponding to the n-th data domain; Step S24: Based on the attention weights of multiple scales of the forged image corresponding to the nth data domain, adaptive selection is performed to obtain adaptive features of the forged image corresponding to the nth data domain, and the adaptive features corresponding to the nth data domain are represented as multi-scale spatial information of the forged image corresponding to the nth data domain.

5. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 3 is characterized in that: The specific steps of obtaining the updated multi-frequency information of the forged image corresponding to the nth data domain in step S3 include: Step S31, inputting the forged image corresponding to the nth data field into the multi-frequency aggregator module; Based on the fast Fourier transform FFT, frequency decomposition is performed to obtain the low-frequency information and high-frequency information of the forged image corresponding to the nth data domain; Step S32, performing convolution processing on the low-frequency information of the forged image to extract the structural information features of the forged image; performing refinement convolution operation and adaptive weighting processing on the high-frequency information to obtain the high-frequency forged detail features of the forged image; Step S33, adaptively weighting and fusing the structural information features of the forged image and the high-frequency forged detail features of the forged image to obtain multi-frequency information of the forged image; Step S34, inputting the multi-frequency information of the forged image into the high-frequency noise enhancement module, using a bandpass filter to extract high-frequency features, and obtaining high-frequency features of the forged image; Step S35 : After convolution and adaptive normalization, the high-frequency features of the forged image are weightedly fused with the multi-frequency information of the forged image to obtain updated multi-frequency information of the forged image.

6. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 5 is characterized in that: The low-frequency information includes the overall structure of the image; the high-frequency information includes the details and texture of the forged image.

7. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 4 is characterized in that: The multi-scale spatial information of the corresponding forged image in step S24 is expressed as: Among them, Y is the output feature map, which is the feature generated after the convolution operation and is used for subsequent neural network processing; W k is the kth convolution kernel, which is a trainable parameter used to capture the local pattern of input features, such as edges, textures, etc.; F is the input forged image, b k is the bias term of the kth convolution kernel, and * represents the convolution operation.

8. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 4 is characterized in that: The expression of the attention weights of multiple scales of the corresponding forged image in step S24 is: A=σ(W A ·GlobalPool(F)+b A ) Where σ(·) is the activation function, A is the attention weight of the important features of the forged image, W A is the weight matrix of the forged image, GlobalPool(·) is the global pooling operation function, b A is the bias term for the fake image.

9. The efficient face deep fake detection method based on multi-frequency feature aggregation according to claim 1 is characterized in that: The expression of the multi-frequency information of the forged image is: F MFA =α·F DC +β·FHC HC Among them, F MFA is the multi-frequency information of the forged image; F DC is the DC component of the input characteristic; F HC is the high-frequency component HC of the input feature; α is the learnable parameter one; β is the learnable parameter two.

Images