Efficient detection method for Deepfake forged image
Through CRE technology and ViT model combined with CNN to extract global and local features of the image and cross-fusion, the problem of difficult to detect subtle modifications in the existing technology is solved, and a more efficient detection effect is achieved.
Patent Information
- Application Number
- CN202411698873.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-04-15
AI Technical Summary
The prior art is difficult to effectively detect and distinguish subtlely modified Deepfake fake images, especially in the same image, the normal and tampered areas are difficult to distinguish by the naked eye, resulting in poor detection results.
Channel Random Erasing (CRE) technology is used to randomly erase some pixel channels of the image, combine the Vision Transformer (ViT) model and the convolutional neural network (CNN) to extract global and local features, and cross-fusion of these features is used to identify them using a classifier.
It significantly improves the robustness and detection accuracy of the model under different interference conditions, enhances the recognition ability of complex forged images, and improves the detection effect.
Smart Images
Figure CN119832262A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image detection, and in particular relates to a method for efficiently detecting Deepfake forged images. Background Art
[0002] In recent years, the rapid development of Deepfake technology has triggered numerous research discussions on digital Deepfake and information credibility, as it can be used to create fake videos, leading to the spread of false information and potential privacy violations. Therefore, in order to mitigate the negative impact of Deepfake, Deepfake detection technology has emerged and has made significant breakthroughs in detection. Although significant progress has been made in Deepfake detection of single-frame images or video frames in recent years, the continuous improvement of Deepfake technology has made it possible for people to modify specific parts of an image using simple forgery software. For example, people can modify parts such as eyes or noses, and even make multiple modifications based on the initial changes. The same manipulation steps in different orders can produce different spatial manipulation trajectories. Although the labels of the manipulated areas may be the same, the generator may cause changes in the spatial features of other parts during the manipulation process. Therefore, there may still be significant differences between forged images with the same manipulation labels but different operation orders.
[0003] Although the principles behind these Deepfake manipulations seem complicated, they are very similar to image description tasks. Previous state-of-the-art methods treat sequence Deepfake detection as a special image description task, using enhanced autoregressive models to detect manipulated regions. This approach has shown some effectiveness in detecting sequence manipulation. However, since the dataset we use contains images with relatively small resolution and subtle facial modifications, normal and tampered regions in the same image may not be distinguishable by the naked eye, although at the data level, there are significant modality differences between them, which greatly reduces the effectiveness of feature extraction when used directly.
[0004] Therefore, there is an urgent need to design an efficient detection method for Deepfake images to solve the problem of Deepfake images or videos that cannot be distinguished by the naked eye. Summary of the invention
[0005] In view of the defects of the above-mentioned prior art, the purpose of the present invention is to provide an efficient detection method for Deepfake forged images. First, the CRE technology is used to randomly erase a part of the pixels in the image, which provides finer-grained control and makes the generated image have higher diversity and complexity. It can not only effectively simulate the image damage scenes in the real world, but also significantly improve the robustness of the model under different interference conditions; then the ViT model and CNN network are used to generate global features and local features, and the global features and local features are cross-fused, so that the local features of CNN and the global features of ViT complement each other to achieve better detection effect; finally, the fused image is recognized by a classifier, and the recognition result is output.
[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0007] A method for efficiently detecting Deepfake forged images, characterized by comprising:
[0008] S1. Acquire image data and perform standardization on the image data;
[0009] S2. Using CRE technology, the standardized image data is processed to obtain a simulated forged image;
[0010] S3. Use the ViT model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation containing the global information of the image;
[0011] S4. Use a convolutional neural network to perform a convolution operation on the forged image, obtain local detail information of the forged image, and generate a local feature representation map;
[0012] S5. Fusing the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information;
[0013] S6. Input the feature map C into the classifier to perform authenticity detection and identification of forged parts.
[0014] Preferably, in step S2, the process of obtaining a simulated forged image includes:
[0015] S201. Randomly select an erased area in the standardized processed image;
[0016] S202. In the selected erasing area, assign a specific channel of each pixel to the average of the three channels.
[0017] Preferably, in step S201, the process of selecting the erasing area in the standardized processed image includes:
[0018] S2011. Randomly select a point P in the normalized image, and mark the coordinates of point P as (Xe, Ye);
[0019] S2012. Based on the coordinates of point P, a random erasure area is obtained in combination with W and H;
[0020] Among them, the area of the erase area Ie is set to (Xe, Ye, Xe+We, Ye+He), and its width satisfies the condition that Xe plus We is less than or equal to W, and its height satisfies the condition that Ye plus He is less than or equal to H.
[0021] Preferably, in step S5, the process of obtaining a feature map of global and local information fusion includes:
[0022] S501. Process the global feature map and the local feature map respectively to obtain a global feature representation map A and a local feature representation map B;
[0023] S502. Perform feature fusion on the global feature representation graph A and the local feature representation graph B.
[0024] Preferably, in step S501, the process of processing the global feature map and the local feature map respectively includes:
[0025] S5011. Use ResNet50 as the backbone network to embed the input patch containing position information into the sequence X p ∈R N×d The local information feature extraction of the image is performed to obtain the feature map list {X i ,i=1:k} and position information;
[0026] S5012.To X i Decompose and obtain feature tables c1, c2, c3, c4 at different levels;
[0027] S5013. Perform linear transformation and flattening on c1, c2, c3, and c4 respectively;
[0028] S5014. Add layer embedding information to c2, c3, and c4 to enhance feature representation;
[0029] S5015. The feature tables c1, c2, c3, and c4 of the processed local feature representation graph B and the global feature representation graph A are respectively concatenated in the channel dimension to form a global feature representation graph A and a local feature representation graph B with local spatial semantics.
[0030] Preferably, in step S501, c1, c2, c3, and c4 are respectively subjected to linear transformation and flattening processing. The specific process includes: c1 is directly subjected to linear transformation, c2 and c3 are flattened and latitude-adjusted after linear transformation, and c4 is first subjected to maximum pooling processing, and then subjected to linear transformation, flattening, and dimension adjustment.
[0031] Preferably, in step S502, the global feature representation graph A is defined as F vit , the local feature representation map B is defined as F sp The fusion process of the global feature representation map A and the local feature representation map B includes multiple times, and each fusion input is the output of the previous fusion, and the final fusion map is the output of the last fusion. The four fusion processes are the same, specifically:
[0032] S5021. Features As a query, spatial features As key-value pairs, the multi-scale deformation attention mechanism is used to transform the spatial features Fusion to input features The output features are obtained
[0033] S5022. As a query, As key-value pairs, they are then fused using a multi-scale deformation attention mechanism.
[0034] Preferably, in step S502-1, the spatial feature Fusion to input features The process formula is:
[0035]
[0036] Among them, norm is LayerNorm, attention layer Attention is sparse attention, γ i To learn the vector, is the output feature after fusion;
[0037] In step S502-2, As a query, The formula for merging key-value pairs is:
[0038]
[0039] in, It is the output of the previous module.
[0040] The second object of the present invention is to provide a system for efficiently detecting Deepfake forged images, characterized in that it includes:
[0041] An acquisition module, used for acquiring image data and performing standardization processing on the image data;
[0042] The forged image module uses CRE technology to process the standardized image data to obtain a simulated forged image;
[0043] Generation module 1 uses the ViT model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation containing the global information of the image;
[0044] Generation module 2 uses a convolutional neural network to perform a convolution operation on the forged image, obtains local detail information of the forged image, and generates a local feature representation map;
[0045] The fusion module fuses the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information;
[0046] The output module inputs the feature map C into the classifier to perform authenticity detection and identification of forged parts.
[0047] The third object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute an efficient method for detecting Deepfake forged images.
[0048] The beneficial effects of the present invention are as follows: the present invention discloses an efficient detection method for Deepfake forged images. Compared with the prior art, the improvements of the present invention are:
[0049] (1) The present invention uses Channel Random Erasing (CRE) technology to assign a specific channel of each pixel (such as a channel in RGB) to the mean of the three channels while retaining the information of other channels. This can reduce modality differences and enhance feature extraction, retain more feature information of the image, and significantly enhance the robustness and generalization ability of the model.
[0050] (2) The present invention uses the Vision Transformer (ViT) model to perform global feature extraction instead of the traditional Transformer, which does not rely on a fixed length of input data and provides greater flexibility in processing image data. In addition, ViT's global attention mechanism captures the features of the entire image, giving it a significant advantage in global feature extraction. These features and advantages enable ViT to better capture complex patterns and structures in images and provide superior performance in global feature extraction.
[0051] (3) The present invention combines the global feature information of the image extracted by the Vision Transformer (ViT) model with the local feature information captured by CNN, so that the local features of CNN and the global features of ViT complement each other. This method enhances the generalization ability, improves the performance of the model in this task, and achieves better detection results. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a flowchart of the efficient forged image detection method of the present invention;
[0053] Figure 2 The process of the efficient detection method of forged images of the present invention is as follows Figure 2 ;
[0054] Figure 3 Flowchart of the cross-attention mechanism of the efficient forged image detection method of the present invention. DETAILED DESCRIPTION
[0055] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0056] Embodiment 1:
[0057] See attached Figure 1-2 An efficient detection method for Deepfake forged images shown in FIG. 1 includes:
[0058] S1. Obtain image data and perform image standardization;
[0059] Specifically, the image data includes picture data and video data; the image standardization processing includes: adjusting the image size to adapt it to the input requirements of the model;
[0060] S2. Using CRE technology, the standardized image is processed to obtain a simulated forged image;
[0061] Specifically, the Channel Random Erasing (CRE) technology is used to assign a specific channel of each pixel of the image (such as a channel in RGB) to the mean of the three channels, while retaining the information of other channels. By introducing the diversity of training data, the robustness and generalization ability of the model are significantly improved.
[0062] CRE technology provides finer-grained control by selectively erasing pixels of specific channels, making the generated images more diverse and complex. This enhancement technology can not only effectively simulate image corruption scenarios in the real world, but also significantly improve the robustness of the model under different interference conditions. CRE technology is particularly suitable for deep learning tasks that need to deal with various image interference and forgery scenarios. In the Deepfake detection task, CRE can help the model better identify and deal with complex forgery techniques, and improve detection accuracy and reliability. Experimental results show that the model using CRE technology has shown significant robustness improvement on multiple public datasets. Compared with the traditional Random Erasing method, CRE not only has obvious advantages in enhancing diversity and complexity, but also can more effectively improve the generalization ability and practical application effect of the model. By introducing CRE technology, our model shows stronger adaptability and stability when facing complex image forgery and tampering scenarios. This data enhancement method provides new ideas and technical support for further improving the robustness of deep learning models.
[0063] Furthermore, the process of obtaining a simulated forged image using the CRE technology includes:
[0064] S201 randomly selects the erased rectangular area Ie in the standardized processed image; the specific process includes:
[0065] S201-1. Randomly select a point P in the standardized image and mark the coordinates of point P as (Xe, Ye);
[0066] S201-2. Based on the coordinates of point P, a randomly erased rectangular area Ie is obtained by combining W and H;
[0067] Specifically, when Xe plus We (the width of the randomly erased rectangular box) is less than or equal to W (the width of the original image), and Ye plus He (the height of the randomly erased rectangular box) is less than or equal to H (the height of the original image), the randomly erased rectangular area Ie is set to the rectangular area of (Xe, Ye, Xe+We, Ye+He), otherwise, point P is randomly generated again until a suitable Ie is selected.
[0068] S202. In the selected erasing area Ie, a specific channel (such as one channel in RGB) of each pixel is assigned the average of the three channels, while retaining the information of other channels.
[0069] This method is different from the traditional Random Erasing method. The Random Erasing method erases pixels of all channels at the same time, while CRE assigns a specific channel of each pixel (such as one channel in RGB) to the average of the three channels while retaining the information of other channels. This method provides finer-grained control and simulates more complex image damage scenarios.
[0070] S3. Use the Vision Transformer (ViT) model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation that contains the global information of the image;
[0071] Specifically, ViT's self-attention mechanism can effectively aggregate the global information in the forged image and provide a comprehensive feature description. The specific steps are:
[0072] S301. Segment the forged image to obtain multiple patches of fixed size;
[0073] Split the input forged image X into N non-overlapping patches;
[0074] S302. Perform linear transformation on the segmented patch;
[0075] Use the linear projection matrix E to transform each patch and map it to a high-dimensional space to obtain patch embedding. Let the dimension of patch embedding be D. The image after linear transformation is
[0076] S303. Add position encoding to the linearly transformed patch: In the Transformer model, the self-attention mechanism learns position encoding by calculating the dependencies between different input patches. However, standard self-attention does not directly consider position information. Therefore, a method is needed to encode the position of the patch so that the model can utilize this information.
[0077] S304. Perform feature extraction on the output image of step S303 through the Transformer block: multiple self-attention layers and fully connected layers alternately perform information processing and feature extraction.
[0078] S305. Pooling global features: The output of the last Transformer block is usually subjected to global average pooling or maximum pooling to obtain a global feature representation.
[0079] S4. Use a convolutional neural network (Resnet50) to perform a convolution operation on the forged image in step S2, extract local detail information of the forged image, and generate a local feature representation map;
[0080] Specifically, in the convolution operation of ResNet50, the multi-level features of the forged image are extracted through multi-layer convolution and residual structures, gradually capturing the features from the bottom edge information to the high-level semantic information. First, the image undergoes the initial 7x7 convolution and maximum pooling operation to extract low-level features and reduce the spatial dimension, and then enters multiple residual modules (such as Conv2_x, Conv3_x, Conv4_x, Conv5_x). Each module achieves the expansion of feature channels, extraction of spatial features and dimensionality reduction through a combination of 1x1, 3x3, and 1x1 convolutions. At the same time, the jump connection retains the input features, helping the model to better learn complex features. After layer-by-layer processing of multiple residual modules, the feature map is gradually condensed into a high-dimensional, global feature representation. Finally, the feature vector is quantized through the global average pooling layer to generate a local feature representation map.
[0081] S5. Fusing the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information;
[0082] Specifically, four Cross-Attentions are performed between the global feature representation and the generated local feature representation. Figure 3 As shown in the figure), the global feature representation map and the local feature representation map extracted by the ViT model and the CNN network are fused to obtain a feature map of global and local information fusion, so as to achieve an effective combination of global and local information. The fused features combine global and local information and can more comprehensively characterize the image content, thereby enhancing the detection ability of the model. The specific steps are:
[0083] S501. Data processing: Process the global feature map and the local feature map separately to facilitate the subsequent fusion operation; the processing steps of the global feature map and the local feature map are the same, and the specific processing steps include:
[0084] S501-1. Use ResNet50 as the backbone network to embed the input patch containing position information into the sequence X p ∈R N×d The local information feature extraction of the image is performed to obtain the feature map list {X i ,i=1:k} and position information;
[0085] S501-2. To X i Decompose and obtain feature tables c1, c2, c3, c4 at different levels;
[0086] X iDecompose and obtain feature tables c1, c2, c3, c4 at different levels. The shapes of these feature maps are shape (B, N, H, W), among which c4 is obtained by 16 times downsampling;
[0087] S501-3. Perform linear transformation and flattening on c1, c2, c3, c4 respectively;
[0088] Specifically, c1 directly applies linear transformation, c2 and c3 are flattened and dimensionally adjusted after linear transformation, and c4 is first subjected to maximum pooling and then linearly transformed, flattened, and dimensionally adjusted.
[0089] S501-4. Add layer embedding information to c2, c3, and c4 to enhance feature representation;
[0090] S501-5. The feature tables c1, c2, c3, and c4 of the processed local feature representation map and the global feature representation map are respectively concatenated in the channel dimension to form a global feature representation map A and a local feature representation map B with local spatial semantics;
[0091] S502. Perform feature fusion on the global feature representation map A and the local feature representation map B to obtain a feature map C that fuses global and local information;
[0092] The global feature representation map A and the local feature representation map B are cross-fused, and the global feature representation map A is defined as F vit , the local feature representation map B is defined as F sp ; For the specific process, please refer to Figure 2 The entire feature fusion process includes four cross-attention modules (i.e., MSDeformAttn), which are connected in series. The input of each module is the output result of the previous fusion. In the i-th (i=1, 2, 3, 4) fusion process, the module fusion process is as follows:
[0093] S502-1. Fusion to In the characteristics As a query, spatial features As key-value pairs, the multi-scale deformation attention (MSDeformAttn) mechanism is used to transform the spatial features Fusion to input features The output features are obtained
[0094] The fusion formula is:
[0095]
[0096] Among them, norm is LayerNorm, attention layer Attention is sparse attention, γ i To learn the vector, is the output feature after fusion, is the output of the previous module;
[0097] In the first module, when i=1, represents the local feature map generated by Resnet50, represents the global feature map generated by the Vit network. During the fusion process, we apply a learnable vector γ i ∈R D To balance the output and input features of the attention layer It is initialized to 0. This initialization strategy ensures that F vit The feature distribution of will not change significantly due to the fusion of spatial priors, thus making better use of the pre-trained weights of ViT.
[0098] S502-2. The result obtained in step S502-1 The features are integrated into A more comprehensive feature map C of fusion information is obtained; the specific process is:
[0099] In order to enable The features in are extracted and integrated into our In this paper, we designed an extraction module (Extractor) to The features are further extracted and fused. The extraction module is mainly composed of a multi-scale deformation attention mechanism (MSDeformAttn) and a convolutional feedforward neural network (ConvFFN). We first normalize the input features, then calculate the attention weights through the multi-scale deformation attention mechanism, and finally further process the features through the convolutional feedforward neural network. As a query, As a key-value pair,
[0100] The fusion formula is:
[0101]
[0102] in, is the output of the previous module;
[0103] This method essentially converts the Feature maps and local information The feature maps are cross-fused with each other to obtain a feature map C with richer features containing global and local information. As the input of the spatial prior features of the next cycle, a total of 4 cycles ( Figure 2 shown);
[0104] S6. Input the feature map C into the Transformer classifier to perform authenticity detection and identification of forged part sequences.
[0105] Specifically: the fused feature map C is sent to the traditional Transformer for final feature extraction to obtain the sequence forgery relationship, and the final feature of the FFN layer of the classifier is used as the output.
[0106] The present invention also provides a system for efficiently detecting Deepfake forged images, which is characterized by comprising:
[0107] An acquisition module, used for acquiring image data and performing standardization processing on the image data;
[0108] The forged image module uses CRE technology to process the standardized image data to obtain a simulated forged image;
[0109] Generation module 1 uses the ViT model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation containing the global information of the image;
[0110] Generation module 2 uses a convolutional neural network to perform a convolution operation on the forged image, obtains local detail information of the forged image, and generates a local feature representation map;
[0111] The fusion module fuses the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information;
[0112] The output module inputs the feature map C into the classifier to perform authenticity detection and identification of forged parts.
[0113] The present invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a method for efficiently detecting Deepfake forged images.
[0114] Embodiment 2:
[0115] In order to demonstrate the effectiveness and superiority of our method, Example 2 provides a comprehensive comparison between us and the current SOTA method, which is widely regarded as the most effective solution in the field of sequential Deepfake detection. Table 1 shows the comparison of the detection results of the present invention and the current SOTA method, including sequential Deepfake detection of facial components and attribute operations.
[0116] Table 1 Experimental results comparing this method with the current SOTA method
[0117]
[0118] In Table 1, we find that our proposed method achieves significant improvements in both Fixed-Acc and Adaptive-Acc. The full model (CVCNet) achieves the highest Fix-ACC of 73.32% and 69.47% for facial components and facial attributes. This is a significant improvement over SeqFakeFormer and SECA, which achieve 72.65% and 68.86% respectively. However, it also proves that adaptive detection (Ada-ACC) is more challenging than fixed-length detection (Fix-ACC), with generally lower scores among all methods. In addition, it can be seen from the table that the detection performance of facial component manipulation is better than that of facial attribute manipulation.
[0119] Ablation experiment
[0120] The model of the present invention mainly consists of two modules: data enhancement and CVC. We conducted an ablation study to evaluate the impact of each module on the overall performance. The study includes four model variants: the full model (including both modules), the model without CRE, the model without CVC, and the model without these two modules.
[0121] Table 2 Performance indicators of each module
[0122]
[0123] In Table 2, taking the facial component as an example, we check the Fix-ACC accuracy, and the accuracy of the full model is 73.32%. After removing CRE, the accuracy drops to 73.13%. After removing CVC, the accuracy drops to 73.07%, and the model without both modules performs the worst with an accuracy of 72.65%. These results confirm that these two modules significantly improve the model's ability to detect Deepfake manipulations, and the full model achieves the highest performance in all indicators, while removing either module leads to a significant drop in performance. This shows that both data augmentation and feature fusion are crucial for effective sequential Deepfake detection.
[0124] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. An efficient detection method for Deepfake forged images, characterized by: include: S1. Acquire image data and perform standardization on the image data; S2. Using CRE technology, the standardized image data is processed to obtain a simulated forged image; S3. Use the ViT model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation containing the global information of the image; S4. Use a convolutional neural network to perform a convolution operation on the forged image, obtain local detail information of the forged image, and generate a local feature representation map; S5. Fusing the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information; S6. Input the feature map C into the classifier to perform authenticity detection and identification of forged parts.
2. According to claim 1, a method for efficiently detecting Deepfake forged images is characterized by: In step S2, the process of obtaining a simulated forged image includes: S201. Randomly select an erased area in the standardized processed image; S202. In the selected erasing area, assign a specific channel of each pixel to the average of the three channels.
3. The method for efficiently detecting Deepfake forged images according to claim 2, characterized in that: In step S201, the process of selecting an erasure area in the normalized image includes: S2011. Randomly select a point P in the normalized image, and mark the coordinates of point P as (Xe, Ye); S2012. Based on the coordinates of point P, a random erasure area is obtained in combination with W and H; Among them, the area of the erase area Ie is set to (Xe, Ye, Xe+We, Ye+He), and its width satisfies the condition that Xe plus We is less than or equal to W, and its height satisfies the condition that Ye plus He is less than or equal to H.
4. The method for efficiently detecting Deepfake forged images according to claim 1, characterized in that: In step S5, the process of obtaining the feature map of global and local information fusion includes: S501. Process the global feature map and the local feature map respectively to obtain a global feature representation map A and a local feature representation map B; S502. Perform feature fusion on the global feature representation graph A and the local feature representation graph B.
5. The method for efficiently detecting Deepfake forged images according to claim 4, characterized in that: In step S501, the process of processing the global feature map and the local feature map respectively includes: S5011. Use ResNet50 as the backbone network to embed the input patch containing position information into the sequence X p ∈R N×d The local information feature extraction of the image is performed to obtain the feature map list {X i ,i=1:k} and position information; S5012.To X i Decompose and obtain feature tables c1, c2, c3, c4 at different levels; S5013. Perform linear transformation and flattening on c1, c2, c3, and c4 respectively; S5014. Add layer embedding information to c2, c3, and c4 to enhance feature representation; S5015. The feature tables c1, c2, c3, and c4 of the processed local feature representation graph B and the global feature representation graph A are respectively concatenated in the channel dimension to form a global feature representation graph A and a local feature representation graph B with local spatial semantics.
6. The method for efficiently detecting Deepfake forged images according to claim 5, characterized in that: In step S501, c1, c2, c3, and c4 are linearly transformed and flattened respectively. The specific process includes: c1 is directly linearly transformed, c2 and c3 are flattened and dimensionally adjusted after linear transformation, and c4 is firstly subjected to maximum pooling processing, and then linearly transformed, flattened, and dimensionally adjusted.
7. The method for efficiently detecting Deepfake forged images according to claim 5, characterized in that: In step S502, the global feature representation graph A is defined as F vit , the local feature representation map B is defined as F sp The fusion process of the global feature representation map A and the local feature representation map B includes multiple times, and each fusion input is the output of the previous fusion, and the final fusion map is the output of the last fusion. The four fusion processes are the same, specifically: S5021. Features As a query, spatial features As key-value pairs, the multi-scale deformation attention mechanism is used to transform the spatial features Fusion to input features The output features are obtained S5022. As a query, As key-value pairs, they are then fused using a multi-scale deformation attention mechanism.
8. The method for efficiently detecting Deepfake forged images according to claim 7, characterized in that: In step S502-1, the spatial feature Fusion to input features The process formula is: Among them, norm is LayerNorm, attention layer Attention is sparse attention, γ i To learn the vector, is the output feature after fusion; In step S502-2, As a query, The formula for merging key-value pairs is: in, It is the output of the previous module.
9. A system for efficiently detecting Deepfake forged images, characterized by: include: An acquisition module, used for acquiring image data and performing standardization processing on the image data; The forged image module uses CRE technology to process the standardized image data to obtain a simulated forged image; Generation module 1 uses the ViT model to capture the global context information of the forged image through the self-attention mechanism and generate a feature representation containing the global information of the image; Generation module 2 uses a convolutional neural network to perform a convolution operation on the forged image, obtains local detail information of the forged image, and generates a local feature representation map; The fusion module fuses the global feature representation map and the local feature representation map to obtain a feature map C that fuses global and local information; The output module inputs the feature map C into the classifier to perform authenticity detection and identification of forged parts.
10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.