Deepfake Detection Method with Multi-branch DLMDMLP and Adversarial Generation

Abstract

A multi-branch DLMDMLP and adversarial generation deepfake detection method, belonging to the field of deepfake monitoring and multimedia security technology, includes obtaining a forged image or video dataset and performing data preprocessing; improving the DeepFake-Adapter model to obtain an improved KaleidoDynAdv DeepFake Detection deepfake detection model; training the improved KaleidoDynAdv DeepFake Detection deepfake detection model from step 2; and using the trained deepfake detection model for forgery detection. This method utilizes open-source datasets for training to improve deepfake detection discrimination capabilities (AUC), accuracy (ACC), and generalization, while reducing the equal error rate (EER) and improving social security.

Description

Technical Field

[0001] The present invention belongs to the field of deep fake monitoring and multimedia security technology, and in particular relates to a deep fake detection method based on multi-branch DLMDMLP and adversarial generation, namely KaleidoDynAdv DeepFake Detection (KDA-DFD for short). Background Art

[0002] As short videos, online videos, and online images become commonplace, many criminals are using AI or photo editing to forge original videos or images, distorting the truth and causing significant harm to society. Due to the continuous advancement of forgery technology, the human eye has difficulty distinguishing forged photos or videos. Traditional detectors overfit to specific forgery patterns (such as local textures), resulting in significant performance degradation across datasets. For example, FaceX-ray detects discontinuities at the fusion boundary of facial regions, and F3-Net uses discrete cosine transforms (DCTs) to extract frequency feature differences. However, these methods lack generalization capabilities for unseen forgery techniques and are susceptible to post-processing interference such as compression and blurring.

[0003] Existing deepfake detection methods face significant challenges. Pre-trained Visual Transformers (ViTs) have been introduced to the detection field due to their powerful semantic modeling capabilities. DeepFake-Adapters fuse ViT's high-level semantics with low-level forgery features through a two-stage adapter, improving cross-dataset generalization, but neglecting the processing of local features. AVFFs capture audio-visual correspondences (such as speech-lip synchronization) through self-supervised contrastive learning and a cross-modal complementary masking strategy, but rely on large-scale annotated data and lack robustness to modality mismatches in complex scenarios. To address these issues, the present invention aims to achieve robust detection of unknown forgery techniques by designing a dynamic perceptual gated multi-scale forgery detection perception, spatial dynamic anchor chain priors, gradient-isolated dynamic adversarial feature generation adapters, and a dynamic gated multi-stream forgery detection network. This design improves the model's discriminative ability (AUC), accuracy (ACC), and generalization, addressing the shortcomings of existing technologies in generalized deepfake detection. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a deep fake detection method based on multi-branch DLMDMLP and adversarial generation, which is trained using open source datasets to improve the deep fake detection discrimination ability AUC, accuracy ACC and generalization, reduce the equal error rate EER, and improve social security.

[0005] The multi-branch DLMDMLP and adversarial deep fake detection method includes the following steps, which are performed in sequence:

[0006] Step 1: Obtain a forged image or video dataset and perform data preprocessing;

[0007] Step 2: Improve the DeepFake-Adapter model to obtain the improved KaleidoDynAdvDeepFake Detection deep fake detection model;

[0008] Step 3: Train the improved KaleidoDynAdv DeepFake Detection model from Step 2;

[0009] Step 4: Use the trained deep fake detection model to perform fake detection.

[0010] The improved KaleidoDynAdv DeepFake Detection model described in step 2 includes a dynamic perception gated multi-scale forgery detection perceptron DLMDMLP, a spatial dynamic anchor chain prior module, a gradient-isolated dynamic adversarial feature generation adapter, and a dynamic gated multi-stream forgery detection network.

[0011] The dynamic perception gated multi-scale forgery detection perceptron DLMDMLP adopts a local-global dynamic gate control feature fusion mechanism. Local features use depthwise separable convolution, and global features use fully connected layers. Local features and global features are fused through a dynamic gating mechanism; gating weights are generated using global feature information; the standard sequence structure is restored through a sequence reconstruction operation, and then the function enters a learnable multi-scale feature fusion structure for processing. The fused function enters a variance-driven dynamic activation function to adaptively adjust the nonlinear response, then enters a fully connected layer to restore the dimension, and finally enters a random dropout layer. During the forward propagation process, the random dropout randomly sets the output of the input neuron to 0 with a probability of 0.1 to prevent overfitting.

[0012] The fusion process of the local-global dynamic gate control feature fusion mechanism includes:

[0013] S1: Separate the data features into local feature carriers and global feature tags. The local feature carriers are Patches image blocks, and the global feature tags are cls_token classification tags, which are recorded as:

[0014] (1)

[0015] (2)

[0016] in represents the output feature, B is the batch size, N is the sequence length, and C is the feature dimension; It is the feature representation assigned to cls_token. is the feature representation assigned to Patches; 0:1 means selecting the element with index 0 on the sequence dimension, and 1:N means selecting all elements from index 1 to the end on the sequence dimension;

[0017] S2: Local features enter the local branch for depthwise separable convolution. 3*3 depthwise separable convolution is used to perform spatial convolution on the image block sequence. The formula is:

[0018] (3)

[0019] in represents the local branch output feature, stands for depthwise separable convolution, It is a tensor deformation operation that changes the dimensional structure of the tensor without changing the data content. It is the feature representation assigned to Patches;

[0020] The global features enter the global branch and are expanded in dimension through the fully connected layer. The specific formula is:

[0021] (4)

[0022] in represents the global branch output feature, is the weight matrix, is the bias vector, represents the output features;

[0023] S3: The corresponding features after processing by the local branch and the global branch are fused in the dynamic gate fusion module. The formula is:

[0024] (5)

[0025] (6)

[0026] in represents the gate weight tensor, is the weight matrix, is the bias vector, It means that after the feature vector of each patch is normalized, the mean and variance are calculated along the feature dimension. Represents the sequence features of the fully connected transformation; represents the local branch output feature, represents the final fusion feature;

[0027] S4: Reconstruct the processed features into a sequence. The reconstruction formula is:

[0028] (7)

[0029] Among them, Y is the final output feature, represents the final fusion feature, The cls_token representing the global branch.

[0030] The learnable multi-scale feature fusion method is as follows: first, a multi-scale downsampling operation is performed, and the formula is:

[0031] (8)

[0032] Where s is the scale factor, N is the length of the input sequence, and max(1,·) ensures that the pooling size is at least 1 to avoid the scale factor being too large, which will cause the sequence length to be 0. Represents the length of the sequence after downsampling;

[0033] After downsampling, adaptive pooling operation is performed on each scale. The input shape is (B, N, C), which is converted to (B, C, N) through Permute operation. The sequence length is downsampled from N to , using Permute to convert back (B, ,C),The formula for feature transformation,pooling operation is:

[0034] (9)

[0035] Among them, the Permute operation means rearranging the dimensions of the tensor according to the specified dimension order. AdaptiveAvgPool1D is a function in PyTorch, which belongs to the adaptive average pooling operation and is used to pool one-dimensional data. Y represents the features of the input pooling layer after multi-scale downsampling. is the feature after downsampling;

[0036] After pooling, the corresponding feature transformation and upsampling operations are performed. The specific formula is:

[0037] (10)

[0038] (11)

[0039] Among them, GELU is the activation function; Used for upsampling operation to restore the processed feature sequence to the original sequence length; and are the weights and biases of the linear layer, is the downsampled feature, It is the feature after the linear layer + GELU;

[0040] The weights and biases of the linear layer are calculated by matrix multiplication and activated by the activation function before entering the upsampling stage. The feature shape before upsampling is converted into (B, C, ),pass Change the sequence length from Restore to N and use Permute to convert back to (B, N, C) to be consistent with the input dimension;

[0041] After upsampling, the fusion module is used for weight normalization and feature fusion. The features at different scales are weighted and summed according to their corresponding weights to obtain the final features after multi-scale fusion. The formula is:

[0042] (12)

[0043] (13)

[0044] in is the learnable weight of each scale branch, initialized to 1; Softmax is a normalization calculation used to convert a numerical vector into a probability distribution vector representing the probability of each category; is the normalized weight of each scale branch in multi-scale fusion, is the final feature after multi-scale fusion, It is the feature tensor after processing with the corresponding scale.

[0045] The calculation formula of the variance-driven dynamic activation function is:

[0046] Variance calculation: (14)

[0047] Slope calculation: (15)

[0048] Dynamic Activation: (16)

[0049] Where Z is the input feature tensor, b is the batch index, i is the sequence position index, B is the batch size, N is the sequence length, and H is the feature dimension. is the feature average variance, Var is the variance calculation, τ is the variance threshold, β is the slope adjustment coefficient, λ is the dynamic slope factor, the activation function slope used, is the indicator function, which is 1 when ν>τ and 0 otherwise. The subscript R in the lower right corner is the real number operation, which converts the Boolean value into a floating point number. σ refers to the standard Sigmoid activation function, and 1.702 is the approximate parameter of the standard activation function. Represents the calculation of tensor element-by-element multiplication, and DyGELU(Z) represents the final variance-driven dynamic activation function.

[0050] After the image enters the spatial dynamic anchor chain prior module, it passes through the Stem layer, and after a 3×3 convolution with a step size of 2, it undergoes normalization and activation function operations, and enters the dynamic dilatedConv unit for dynamic dilated convolution operations. It dynamically fuses features according to the local gradient intensity, retains details in high-texture areas, and captures global anomalies in smooth areas; it passes through the lightweight texture retention unit LightTexturePreserver to protect texture information through the residual structure, enhances high-frequency textures on the left branch, and jumps to the right to retain the original details. Finally, after the maximum pooling operation, the processed feature map is output; the processed feature map enters three convolution layers with the same structure in sequence, EnhancedConv2 layer, EnhancedConv3 layer and EnhancedCon v4 layer, after entering the convolutional layer, it is first processed by the dynamic dilated convolution unit DynamicDilatedConv, and then enters the forgery pattern enhancement unit ForgeryPatternEnhancer, which explicitly enhances the forgery traces through the predefined kernel, splices the original features and pseudo-selected features, and fuses the information through a 1x1 volume. After a 3×3 convolution with a step size of 2, as well as normalization and activation function operations, the processed features are passed to the next layer. The feature map processed after the EnhancedConv4 layer enters the embedding mapping layer, and the outputs of the EnhancedConv2 layer, EnhancedConv3 layer and EnhancedConv4 layer are sequentially subjected to 1×1 convolution operations, dimension reshaping and dimension permutation.

[0051] The dynamic dilated convolution unit DynamicDilatedConv adopts a dual-branch structure of small receptive field and large receptive field, and uses the gradient predictor to generate a dynamic weight map α, and fuses the features of the small receptive field and the large receptive field according to the generated α. The formula is:

[0052] (17)

[0053] (18)

[0054] (19)

[0055] (20)

[0056] Among them, x is the input feature, is the gradient prediction convolution kernel, is a dynamic weight graph, BN is a batch normalization operation, * represents a convolution operation, represents the Sigmoid activation function, is the small receptive field convolution kernel, RELU is the activation function, Represents a small receptive field feature, is the convolution kernel with large receptive field, is the large receptive field feature, and y is the output feature;

[0057] The lightweight texture preservation unit LightTexturePreserver protects high-frequency artifacts during feature downsampling through a residual structure. It is processed through two branches: texture enhancement and skipping. In the texture enhancement branch, a 3×3 convolution with a stride of 2, a normalization operation, and an activation function operation are performed in sequence. In the skipping branch, identity mapping is performed, and the residuals of the results of the two branches are added. The formula is:

[0058] (twenty one)

[0059] Where x is the input feature, is the texture enhancement convolution kernel, + is the residual connection, y is the output feature, BN is the normalization operation, and ReLU is the activation function;

[0060] The Forgery Pattern Enhancer is processed through a predefined filter, which contains a blocking kernel and an edge detection kernel. Then, it is processed through depthwise separable convolution to extract forged features and concatenate them with the original input features. After that, 1×1 convolution, normalization, and activation function operations are performed in sequence to perform feature enhancement fusion. The formula is:

[0061] (twenty two)

[0062] (twenty three)

[0063] (twenty four)

[0064] (25)

[0065] in is the edge detection kernel, is the blockiness kernel, Forged pattern features, is a depth-wise separable convolution, RELU is the activation function, BN is a batch normalization operation, * represents a convolution operation, For channel splicing, is a 1×1 convolution kernel, x is the input feature, and y is the output feature.

[0066] The gradient-isolated dynamic adversarial feature generation adapter includes an optional adversarial feature generation module, which includes two linear layers and a GELU activation function. After receiving the downsampled feature tensor, the optional adversarial feature generation module inputs the feature generator, performs a first-level linear transformation, performs dimensionality compression, and after nonlinear activation of the GELU function, performs a second-level linear transformation, performs dimensionality recovery, and performs adaptive perturbation modulation on the processed features, including feature mean calculation, gating coefficient generation, and intensity modulation, and then injects the perturbation and outputs it. The calculation formula is:

[0067] (26)

[0068] (27)

[0069] If the module is activated and in training mode, then (28), if other circumstances, then (29)

[0070] Among them, A is the resistance strength coefficient, is the weight of the first layer of the generator, is the bias of the first layer of the generator, is the second layer weight of the generator, is the bias of the second layer of the generator, down is the down-projection feature; GELU is the activation function; B is the batch size, L is the sequence length, and d is the feature dimension; represents the disturbance generation, is the global average of the feature generator output, To perform nonlinear transformation on the mean value, ensure that the perturbation intensity is within a reasonable range; is the feature generator function, i is the batch dimension index, j is the sequence length dimension index, k is the feature dimension index; σ is the Sigmoid function.

[0071] The dynamic gated multi-stream forgery detection network input image enters the VIT branch and the lightweight CNN branch at the same time. The features output from the CNN branch are flattened and linearly projected, and then dynamically fused with the features of the VIT branch. The fused feature sequence enters the 12-layer improved Transformer block, each layer of which contains a self-attention mechanism and DLMDMLP. Each stage is followed by a degree-isolated dynamic adversarial feature generation adapter, in which the features are bidirectionally interacted with the spatial dynamic anchor chain prior module in three key layers. Finally, the multi-scale spatial features are spliced ​​and layer-normalized, the CLS token vector is extracted, and the true and false probabilities are output through the fully connected classification head. The lightweight CNN branch extracts features through a three-layer convolutional network, each layer has a kernel of 3, a stride of 2, and a padding of 1. After each convolution, a ReLU activation function is activated, and the spatial size is adjusted by adaptive average pooling. The calculation formula is:

[0072] (30)

[0073] (31)

[0074] Among them, LightweightCNN(x) is the lightweight convolutional neural network module operation, x is the input feature, is the CNN branch output, is the output height, is the input height, p is the filling size, d is the void rate, k is the convolution kernel size, and s is the step size;

[0075] The features output from the CNN branch undergo feature flattening and linear projection operations. Feature flattening includes spatial flattening and dimension rearrangement. The formula for the two-step operation is:

[0076] (32)

[0077] (33)

[0078] (34)

[0079] Among them, flat is the spatial flattening operation, and reshape is the operation of changing the shape of the tensor, rearranging the dimensions of the tensor while keeping the total number of elements unchanged; is the CNN branch output, B is the batch size, C is the number of channels, N is the number of spatial locations N=H×W, H is the feature map height, W is the feature map width; is the CNN feature tensor after spatial flattening and dimensional rearrangement, and permute is the dimensional order of the rearranged tensor; is the feature after projection, is the projection weight, is the projection offset;

[0080] The feature splicing, gate generation and weighted fusion are performed on the processed lightweight CNN branch features and VIT branch features, and then the feature splicing, gate generation and weighted fusion are performed, waiting for the next step. The formulas for the three operations are:

[0081] (35)

[0082] (36)

[0083] (37)

[0084] in, is the VIT branch feature, For the calculation of splicing along the channel dimension, is the feature after projection, is the feature after splicing, is the Sigmoid activation function, Gate bias, is the gate weight, is the fused feature.

[0085] Through the above design scheme, the present invention can bring the following beneficial effects: the multi-branch DLMDMLP and adversarial generation deep fake detection method improves the accuracy and robustness of fake detection, is more suitable for fake detection in different scenarios and with different content, can more accurately detect fake videos or images, and improve social security. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0087] Figure 1 Schematic diagram of the process flow of the deep fake detection method based on the multi-branch DLMDMLP and adversarial generation of the present invention.

[0088] Figure 2 This is the overall architecture diagram of the KDA-DFD model used in the multi-branch DLMDMLP and adversarial deep fake detection method of the present invention.

[0089] Figure 3 Schematic diagram of the structure of the dynamic perception gated multi-scale forgery detection sensor of the deep forgery detection method of the present invention with multi-branch DLMDMLP and adversarial generation.

[0090] Figure 4 Schematic diagram of the spatial dynamic anchor chain prior module structure of the deep fake detection method based on the multi-branch DLMDMLP and adversarial generation of the present invention.

[0091] Figure 5 Schematic diagram of the gradient-isolated dynamic adversarial feature generation adapter structure for the deep fake detection method of the present invention using multi-branch DLMDMLP and adversarial generation.

[0092] Figure 6 This is a graph showing experimental results of a specific implementation of the multi-branch DLMDMLP and adversarial deep fake detection method of the present invention. DETAILED DESCRIPTION

[0093] In order to better understand the purpose, structure and function of the present invention, the multi-branch DLMDMLP and the deep fake detection method (KDA-DFD) of the present invention are further described in detail below with reference to the accompanying drawings.

[0094] The present invention uses the improved model KDA-DFD for deep forgery detection to detect video or image forgeries, reduce the spread of false information, and maintain multimedia security and social stability.

[0095] like Figures 1-6 As shown, Figure 1 A flowchart of a multi-branch DLMDMLP and a deep fake detection method (KDA-DFD) for adversarial generation according to an embodiment of the present invention is provided. In this embodiment, the multi-branch DLMDMLP and the deep fake detection method (KDA-DFD) for adversarial generation according to the present invention are described in detail using the CIFAKE dataset as an example. Figure 1 The specific steps are as follows:

[0096] S1-1: Obtain a classified CIFAKE dataset, wherein the CIFAKE dataset is a CIFAKE dataset obtained from an open source dataset, which includes a portion of original images and a portion of forged images;

[0097] The specific content of the dataset is published in

[0098] https: / / github.com / jordan-bird / CIFAKE-Real-and-AI-Generated-Synthetic-Images

[0099] Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers offeatures from tiny images.

[0100] Bird, JJ and Lotfi, A., 2024. CIFAKE: Image Classification andExplainable Identification of AI-Generated Synthetic Images. IEEE Access.

[0101] S1-2: Process the acquired images and classify and place the original images and the forged images.

[0102] S2-1: Improve the original DeepFake-Adapter model to generate an improved model KDA-DFD. The overall architecture of KDA-DFD used in this invention is shown in the figure below: Figure 2 shown

[0103] S2–2: KDA-DFD adopts a three-way parallel processing architecture: the input image simultaneously enters the spatial dynamic anchor chain prior module, the lightweight CNN branch and the ViT trunk branch. The features output from the CNN branch are subjected to feature flattening and linear projection operations, and then dynamically fused with the features of the VIT branch.

[0104] The lightweight CNN branch extracts features through a 3-layer convolutional network. The kernel of each convolution layer is 3, the stride is 2, and the padding is 1. After each convolution layer, the ReLU activation function is activated, and finally the spatial size is adjusted through adaptive average pooling. The calculation formula is:

[0105] (30)

[0106] (31)

[0107] Among them, LightweightCNN(x) is the lightweight convolutional neural network module operation, x is the input image, is the CNN branch output, is the output height, is the input height, p is the padding size, d is the hole rate, k is the convolution kernel size, and s is the step size.

[0108] The feature flattening and linear projection operations are performed on the features output from the CNN branch. Feature flattening includes spatial flattening and dimension rearrangement. The formula for the two-step operation is:

[0109] (32)

[0110] (33)

[0111] (34)

[0112] Where flat is the spatial flattening operation, and reshape is the operation that reshapes the tensor, rearranging its dimensions while preserving the total number of elements. B is the batch size, C is the number of channels, and N is the number of spatial locations (N = H × W), where H is the feature map height and W is the feature map width. It is the CNN feature tensor after spatial flattening and dimension rearrangement, and permute is to rearrange the dimension order of the tensor. is the feature after projection, is the projection weight, is the projection offset.

[0113] The feature splicing, gate generation and weighted fusion are performed on the processed lightweight CNN branch features and VIT branch features, and then the feature splicing, gate generation and weighted fusion are performed, waiting for the next step. The formulas for the three operations are:

[0114] (35)

[0115] (36)

[0116] (37)

[0117] in, is the VIT branch feature, For the calculation of splicing along the channel dimension, is the feature after projection, is the feature after splicing, is the Sigmoid activation function, Gate bias, is the gate weight, is the fused feature.

[0118] S2–3: After the image enters the spatial dynamic anchor chain prior module, it first passes through the Stem layer, which refers to a specific layer in the image processing process. After the image enters, it first undergoes a 3×3 convolution with a step size of 2, and then performs normalization and activation function operations. After processing, it enters the dynamic dilated convolution unit (DynamicDilatedConv) for dynamic dilated convolution operations. According to the local gradient intensity, the features are dynamically fused, the high-texture area retains details, and the smooth area captures global anomalies. After that, it passes through the lightweight texture retention unit (LightTexturePreserver) to protect the texture information through the residual structure, the left branch enhances the high-frequency texture, and the right jump retains the original details. Finally, after the maximum pooling operation, the processed feature map is output, and then it enters the three convolution layers EnhancedConv2, EnhancedConv3, and EnhancedConv4 in sequence. They have the same structure. After entering each convolutional layer, they are first processed by the dynamic dilated convolution unit (DynamicDilatedConv). After processing, they enter the forgery pattern enhancement unit (ForgeryPatternEnhancer) to explicitly enhance the forgery traces through the predefined kernel, splice the original features and pseudo-selected features, and fuse the information through a 1x1 volume. After processing, they pass through a 3×3 convolution with a step size of 2, as well as normalization and activation function operations, and pass the processed features to the next layer. After the EnhancedConv4 layer, the processed feature map enters the final embedding mapping layer, which performs 1×1 convolution operations, dimension reshaping, and dimension permutation on the outputs of the EnhancedConv2 layer, EnhancedConv3 layer, and EnhancedConv4 layer in sequence to ensure the final dimension.

[0119] The dynamic dilated convolution unit (DynamicDilatedConv) adopts a dual-branch structure (small receptive field + large receptive field) and uses a gradient predictor to generate a dynamic weight map α. The features of the small receptive field and the large receptive field are then fused according to the generated α. The formula is:

[0120] (17)

[0121] (18)

[0122] (19)

[0123] (20)

[0124] Among them, x is the input feature, is the gradient prediction convolution kernel, is a dynamic weight graph, BN is a batch normalization operation, * represents a convolution operation, represents the Sigmoid activation function, is the small receptive field convolution kernel, RELU is the activation function, Represents a small receptive field feature, is the convolution kernel with large receptive field, is the large receptive field feature, and y is the output feature.

[0125] The LightTexturePreserver protects high-frequency artifacts during feature downsampling by designing a residual structure. After entering, it is processed through two branches: texture enhancement and skipping. In the texture enhancement branch, a 3×3 convolution with a stride of 2, normalization, and activation function operations are performed sequentially, while in the skipping branch, identity mapping is directly performed. Finally, the residuals of the results of the two branches are added. The formula is:

[0126] (twenty one)

[0127] Where x is the input feature, is the texture enhancement convolution kernel, + is the residual connection, and y is the output feature.

[0128] The Forgery Pattern Enhancer is processed through a predefined filter, which includes a blocking kernel and an edge detection kernel. It then undergoes depthwise separable convolution to extract forged features and concatenates them with the original input features. It then sequentially undergoes 1×1 convolution, normalization, and activation function operations to perform feature enhancement fusion. The formula is:

[0129] (twenty two)

[0130] (twenty three)

[0131] (twenty four)

[0132] (25)

[0133] in is the edge detection kernel, is the blockiness kernel, Forged pattern features, is a depth-wise separable convolution, RELU is the activation function, BN is a batch normalization operation, * represents a convolution operation, For channel splicing, is a 1×1 convolution kernel, x is the input feature, and y is the output feature.

[0134] S2–4: After the features are dynamically fused through the gated fusion mechanism between the VIT branch and the lightweight CNN branch, the CLS token (classification tag) and positional encoding are added before entering a 12-layer improved Transformer block. Each layer incorporates a self-attention mechanism and a dynamic-aware gated multi-scale forgery detection perceptron (DLMDMLP). Internally, this is processed using a gradient-segregated dynamic adversarial feature generation adapter. After three key layers (0-3 / 4-7 / 8-11), a cross-modal bidirectional interaction is performed: the feature injector injects spatial features into the ViT attention mechanism, and the feature extractor feeds ViT semantic information back to optimize the spatial features. Finally, the concatenated multi-scale spatial features undergo layer-normalization, and the CLS token vector is extracted. The judgment is then output through a fully connected classification head.

[0135] S2–5: Features after entering the dynamic-perceptual gated multi-scale forgery detection perceptron (DLMDMLP). The input features first enter the local and global feature processing modules. Compared to previous processing methods, we adopt a local-global dynamic gated feature fusion mechanism, using depthwise separable convolution for local features and fully connected layers for global features. The local and global features are then fused through a dynamic gating mechanism, and gating weights are generated using global feature information. A sequence reconstruction operation is then performed to restore the standard sequence structure, and the input is passed to the learnable multi-scale feature fusion structure for processing. The fused function enters a variance-driven dynamic activation function to adaptively adjust the nonlinear response. It then enters a fully connected layer for dimensionality restoration and then a random dropout layer. During the forward propagation, this layer randomly sets the output of the input neurons to 0 with a probability of 0.1 to prevent overfitting.

[0136] The local and global feature processing module first performs feature separation into patches (image blocks) and cls_tokens (classification tags). Patches are the carriers of local image features, while cls_tokens are global feature tags. The feature formula after separation is:

[0137] (1)

[0138] (2)

[0139] in represents the output feature, B is the batch size, N is the sequence length, and C is the feature dimension; It is the feature representation assigned to cls_token. is the feature representation assigned to Patches; 0:1 means selecting the element with index 0 on the sequence dimension, and 1:N means selecting all elements from index 1 to the end on the sequence dimension; after feature separation, the local features enter the local branch for depthwise separable convolution, using 3*3 depthwise separable convolution, and only performing spatial convolution on the image block sequence. The specific formula is:

[0140] (3)

[0141] in represents the local branch output feature, stands for depthwise separable convolution, It is a tensor deformation operation that changes the dimensional structure of the tensor without changing the data content. is the feature representation assigned to Patches. The global variable enters the global branch and is expanded in dimension through the fully connected layer. The specific formula is:

[0142] (4)

[0143] in represents the global branch output feature, is the weight matrix, is the bias vector, Represents the output feature. The corresponding features after processing by the local branch and the global branch are fused in the dynamic gate fusion module. The formula is:

[0144] (5)

[0145] (6)

[0146] in represents the gate weight tensor, is the weight matrix, is the bias vector, It means that after the feature vector of each patch is normalized, the mean and variance are calculated along the feature dimension. Represents the sequence features of the fully connected transformation; represents the local branch output feature, Represents the final fusion feature. Finally, the processed features are sequence reconstructed, and the reconstruction formula is:

[0147] (7)

[0148] Among them, Y is the final output feature, represents the final fusion feature, The cls_token representing the global branch.

[0149] Furthermore, the learnable multi-scale feature fusion structure, after entering the learnable multi-scale feature fusion structure, first performs a multi-scale downsampling operation, each scale corresponds to a different branch, and the multi-scale downsampling formula is:

[0150] (8)

[0151] Here, s is the scale factor, N is the input sequence length, and max(1,·) is calculated to ensure that the pooling size is at least 1 to avoid the scale factor being too large, which will cause the sequence length to be 0. Represents the length of the sequence after downsampling. After downsampling, adaptive pooling operation is performed on each scale. The input shape is (B, N, C), which is converted to (B, C, N) through Permute operation, and then the sequence length is downsampled from N to Then use Permute to convert back to (B, ,C), to facilitate subsequent feature transformation, the formula for the pooling operation is:

[0152] (9)

[0153] The Permute operation rearranges the dimensions of a tensor according to a specified dimensional order. It is a common method for adjusting data dimensions to adapt to different operational requirements in deep learning. AdaptiveAvgPool1D is a function in PyTorch that belongs to the adaptive average pooling operation and is used to pool one-dimensional data (usually sequence data). The Y in this formula refers to the feature of the input pooling layer after multi-scale downsampling. Obtained from the previous step. is the downsampled feature.

[0154] After pooling, the corresponding feature transformation and upsampling operations are performed. The specific formula is:

[0155] (10)

[0156] (11)

[0157] Among them, GELU is the activation function. It is mainly used for upsampling operations to restore the feature sequence after processing (such as pooling, feature transformation, etc.) to the original sequence length. and are the weights and biases of the linear layer, is the downsampled feature, is the feature after the linear layer + GELU. The weights and biases of the linear layer are calculated through matrix multiplication and activated by the activation function before entering the upsampling stage. The feature shape before upsampling is converted to (B, C, ), and then by Change the sequence length from Restore it to N, and then use Permute to convert it back to (B, N, C) to match the input dimension.

[0158] After upsampling, the model enters the fusion module for weight normalization and feature fusion. The features at different scales are weighted and summed according to their corresponding weights to obtain the final features after multi-scale fusion. This reflects the adaptive fusion of features at different scales and enhances the model's ability to capture multi-granularity semantics. The specific formula is:

[0159] (12)

[0160] (13)

[0161] in is the learnable weight of each scale branch, initialized to 1; Softmax is a normalization calculation used to convert a numerical vector into a probability distribution vector representing the probability of each category; is the normalized weight of each scale branch in multi-scale fusion, is the final feature after multi-scale fusion, It is the feature tensor after processing with the corresponding scale.

[0162] The calculation formula of the variance-driven dynamic activation function is:

[0163] Variance calculation: (14)

[0164] Slope calculation: (15)

[0165] Dynamic Activation: (16)

[0166] Where Z is the input feature tensor, b is the batch index, i is the sequence position index, B is the batch size, N is the sequence length, and H is the feature dimension. is the feature average variance, Var is the variance calculation, τ is the variance threshold, β is the slope adjustment coefficient, λ is the dynamic slope factor, the activation function slope used, is the indicator function, which is 1 when ν>τ and 0 otherwise. The subscript R in the lower right corner is the real number operation, which converts the Boolean value into a floating point number. σ refers to the standard Sigmoid activation function, and 1.702 is the approximate parameter of the standard activation function. Represents the calculation of tensor element-by-element multiplication, and DyGELU(Z) represents the final variance-driven dynamic activation function.

[0167] S2–6: Features entering the gradient-isolated dynamic adversarial feature generation adapter. After inputting the features, they undergo down-projection (a projection transformation similar to dimensionality reduction), followed by an optional adversarial feature generation block, followed by a Relu activation function, a "dropout" operation (randomly inactivating some neurons to prevent overfitting), and an up-projection operation. Finally, scaling and a +Residual connection are performed. During the forward pass, when adversarial training is enabled (enable_adv=True and the model is in training mode), the feature generator perturbs the output of the down-projection (down_proj). Specifically, the perturbation delta is added to the output of down_proj, followed by nonlinear activation and subsequent processing. The perturbation delta is scaled using an adjustable strength parameter (adv_strength) and a sigmoid gating mechanism (to stabilize the perturbation). To prevent the added module's computation from interfering with the gradient features of the original adapter, the generator's computation is wrapped in the torch.no_grad() context.

[0168] After receiving the downsampled feature tensor, the optional adversarial feature generation module inputs it into the feature generator for processing. First, a first-level linear transformation is performed to perform dimensionality compression. After the nonlinear activation of the GELU function, a second-level linear transformation is performed to perform dimensionality recovery. The processed features are subjected to adaptive perturbation modulation, including feature mean calculation, gating coefficient generation, and intensity modulation. Then, perturbations are injected and output. The calculation formula is:

[0169] (26)

[0170] (27)

[0171] If the module is activated and in training mode, then (28), if other circumstances, then (29)

[0172] Among them, A is the resistance strength coefficient, is the weight of the first layer of the generator, is the bias of the first layer of the generator, is the second layer weight of the generator, is the bias of the second layer of the generator, down is the down-projection feature; GELU is the activation function; B is the batch size, L is the sequence length, and d is the feature dimension; represents the disturbance generation, is the global average of the feature generator output, To perform nonlinear transformation on the mean value, ensure that the perturbation intensity is within a reasonable range; is the feature generator function, i is the batch dimension index, j is the sequence length dimension index, k is the feature dimension index; σ is the Sigmoid function.

[0173] S3-1: Place the real images and fake images of the CIFAKE dataset separately and divide them into training and testing use, of which 100,000 are used for training and 20,000 are used for testing.

[0174] S3-2: Configure the training environment. The experiment uses Python 3.9 on a Windows system, equipped with an 18-vCPU AMD EPYC 9754 128-Core Processor, an RTX 3090 (24GB) GPU, 60GB of video memory, and CUDA 11.7.

[0175] S3-3 trains the processed dataset in the environment. During training, set and output the loss function, model discrimination ability (AUC), accuracy (ACC), and error rate (EER) for each round. Adjust training parameters based on the results of each round, such as adjusting the learning rate and stopping training early, to prevent overfitting and ensure good performance on the test set.

[0176] Model testing and application phase (S4-1 to S4-2):

[0177] S4-1: After training, the model is tested using the test set. Images from the test set are fed into the trained model. The model determines whether an image is forged and outputs the model's AUC, ACC, and EER. Evaluation metrics such as AUC, ACC, and EER are calculated on the test set.

[0178] S4-2: The performance of the improved model KDA-DFD and the basic model DeepFake-Adapte is compared, as shown in Table 1.

[0179] Table 1

[0180] Methods / Evaluation Metrics AUC ACC EER DeepFake-Adapter 99.1269% 95.1450% 4.5900% KDA-DFD 99.6155% 97.0700% 2.9900%

[0181] In summary, this model has greatly improved the model's discriminative ability AUC, accuracy ACC, and equal error rate EER, and can perform deep fake detection tasks better and more accurately.

[0182] The foregoing description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by any person skilled in the art within the technical scope disclosed herein and within the spirit and principles of the present invention shall be covered by the scope of protection of the present invention. Furthermore, any matters not described in detail in this specification constitute prior art known to those skilled in the art.

[0183] It should be noted that, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

Claims

1. A multi-branch DLMDMLP and adversarial generation deepfake detection method, characterized by: The process includes the following steps, which are performed in sequence: Step 1: Obtain a forged image or video dataset and perform data preprocessing; Step 2: Improve the DeepFake-Adapter model to obtain the improved KaleidoDynAdv DeepFakeDetection deep fake detection model; Step 3: Train the improved KaleidoDynAdv DeepFake Detection model from Step 2; Step 4: Use the trained deepfake detection model to perform forgery detection; The improved KaleidoDynAdv DeepFake Detection model described in step 2 includes a dynamic-perceptual gated multi-scale forgery detection perceptron (DLMDMLP), a spatial dynamic anchor chain prior module, a gradient-isolated dynamic adversarial feature generation adapter, and a dynamic gated multi-stream forgery detection network. The dynamic perception gated multi-scale forgery detection perceptron DLMDMLP adopts a local-global dynamic gate control feature fusion mechanism. Local features use depthwise separable convolution, and global features use fully connected layers. Local features and global features are fused through a dynamic gating mechanism; gating weights are generated using global feature information; the standard sequence structure is restored through a sequence reconstruction operation, and then the function enters a learnable multi-scale feature fusion structure for processing. The fused function enters a variance-driven dynamic activation function, adaptively adjusts the nonlinear response, enters a fully connected layer for dimensionality recovery, and finally enters a random dropout layer. During the forward propagation process, the random dropout randomly sets the output of the input neuron to 0 with a probability of 0.1 to prevent overfitting. The calculation formula of the variance-driven dynamic activation function is: Variance calculation: Slope calculation: λ = 1 + β·II R (ν>τ) (15) Dynamic activation: DyGELU(Z) = Z⊙σ(1.702·λ·Z) (16) where Z is the input feature tensor, b is the batch index, i is the sequence position index, B is the batch size, N is the sequence length, H is the feature dimension, ν is the feature mean variance, Var is the variance calculation, τ is the variance threshold, β is the slope adjustment coefficient, λ is the dynamic slope factor, the slope of the activation function used, II R is the indicator function, which is 1 when ν>τ and 0 otherwise. The subscript R in the lower right corner is the real number operation, which converts the Boolean value into a floating point number. σ refers to the standard Sigmoid activation function, 1.702 is the approximate parameter of the standard activation function, ⊙ represents the calculation of tensor element-by-element multiplication, and DyGELU (Z) represents the final variance-driven dynamic activation function. After the image enters the spatial dynamic anchor chain prior module, it passes through the Stem layer, and after a 3×3 convolution with a step size of 2, it undergoes normalization and activation function operations, and enters the dynamic dilatedConv unit for dynamic dilated convolution operations. It dynamically fuses features according to the local gradient intensity, retains details in high-texture areas, and captures global anomalies in smooth areas; it passes through the lightweight texture retention unit LightTexturePreserver to protect texture information through the residual structure, enhances high-frequency textures on the left branch, and jumps to the right to retain original details. Finally, after the maximum pooling operation, the processed feature map is output; the processed feature map enters three convolution layers with the same structure in sequence, EnhancedConv2 layer, EnhancedConv3 layer and After entering the convolutional layer, the EnhancedConv4 layer is first processed by the dynamic dilated convolution unit DynamicDilatedConv and then enters the forgery pattern enhancement unit ForgeryPatternEnhancer, which explicitly enhances forgery traces through a predefined kernel, concatenates original features with pseudo-selected features, and fuses information through a 1x1 volume. After a 3x3 convolution with a stride of 2, normalization, and activation function operations, the processed features are passed to the next layer. The feature map processed after the EnhancedConv4 layer enters the embedding mapping layer, and the outputs of the EnhancedConv2, EnhancedConv3, and EnhancedConv4 layers are sequentially subjected to 1x1 convolution operations, dimension reshaping, and dimension permutation. The gradient-isolated dynamic adversarial feature generation adapter includes an optional adversarial feature generation module, which includes two linear layers and a GELU activation function. After receiving the downsampled feature tensor, the optional adversarial feature generation module inputs the feature generator, performs a first-level linear transformation, performs dimensionality compression, and after nonlinear activation of the GELU function, performs a second-level linear transformation, performs dimensionality recovery, and performs adaptive perturbation modulation on the processed features, including feature mean calculation, gating coefficient generation, and intensity modulation, and then injects the perturbation and outputs it. The calculation formula is: G(z)=W2·GELU(W1·z+b1)+b2 (26) If the module is active and in training mode, then δ = A·G(down)·σ(μ) (28) If otherwise, then δ = 0 (29) Among them, A is the adversarial strength coefficient, W1 is the weight of the first layer of the generator, b1 is the bias of the first layer of the generator, W2 is the weight of the second layer of the generator, b2 is the bias of the second layer of the generator, down is the down-projection feature; GELU is the activation function; B is the batch size, L is the sequence length, and d is the feature dimension; G(down) represents perturbation generation, μ is the global average output of the feature generator, and σ(μ) is the nonlinear transformation of the average to ensure that the perturbation strength is within a reasonable range; G(z) is the feature generator function, i is the batch dimension index, j is the sequence length dimension index, and k is the feature dimension index; σ is the Sigmoid function; The dynamic gated multi-stream forgery detection network input image enters the VIT branch and the lightweight CNN branch at the same time. The features output from the CNN branch are flattened and linearly projected, and then dynamically fused with the features of the VIT branch. The fused feature sequence enters the 12-layer improved Transformer block, each layer of which contains a self-attention mechanism and DLMDMLP. Each stage is followed by a degree-isolated dynamic adversarial feature generation adapter, in which the features are bidirectionally interacted with the spatial dynamic anchor chain prior module in three key layers. Finally, the multi-scale spatial features are spliced ​​and layer-normalized, the CLS token vector is extracted, and the true and false probabilities are output through the fully connected classification head. The lightweight CNN branch extracts features through a three-layer convolutional network, each layer has a kernel of 3, a stride of 2, and a padding of 1. After each convolution, a ReLU activation function is activated, and the spatial size is adjusted by adaptive average pooling. The calculation formula is: x cnn =LightweightCNN(x) (30) Among them, LightweightCNN(x) is the lightweight convolutional neural network module operation, x is the input feature, x cnn is the CNN branch output, H out is the output height, H in is the input height, p is the filling size, d is the void rate, k is the convolution kernel size, and s is the step size; The features output from the CNN branch undergo feature flattening and linear projection operations. Feature flattening includes spatial flattening and dimension rearrangement. The formula for the two-step operation is: flat(x cnn )=reshape(x cnn ,[B,C,N]) (32) x cnn_flat =permute(flat(x cnn ),[0,2,1]) (33) x′ cnn =W proj ·x cnn_flat +b proj (34) Where flat is the spatial flattening operation, reshape is the operation of changing the shape of the tensor, rearranging the dimensions of the tensor while keeping the total number of elements unchanged; cnn is the CNN branch output, B is the batch size, C is the number of channels, N is the number of spatial locations N = H × W, H is the feature map height, W is the feature map width; x cnn_flat is the CNN feature tensor after spatial flattening and dimensional rearrangement, and permute is the dimensional order of the rearranged tensor; x′ cnn is the feature after projection, W proj is the projection weight, b proj is the projection offset; The feature splicing, gate generation and weighted fusion are performed on the processed lightweight CNN branch features and VIT branch features, and the gate generation and weighted fusion are performed, waiting for the next step. The formulas for the three operations are: combined=[x vit ;x′ cnn ] (35) gate=σ(W g ·combined+b g ) (36) x fuse =gate⊙x vit +(1-gate)⊙x′ cnn (37) Among them, x vit is the VIT branch feature, [;] is the calculation of splicing along the channel dimension, x′ cnn is the feature after projection, combined is the feature after splicing, σ is the Sigmoid activation function, b g Gate bias, gate is the gate weight, x fuse is the fused feature.

2. The multi-branch DLMDMLP and adversarial generation deepfake detection method according to claim 1, characterized in that: The fusion process of the local-global dynamic gate control feature fusion mechanism includes: S1: Separate the data features into local feature carriers and global feature tags. The local feature carriers are Patches image blocks, and the global feature tags are cls_token classification tags, which are recorded as: X CLS =X[B,0:1,C] (1) X patches =X[B,1:N,C] (2) where X∈R B*N*C represents the output feature, B is the batch size, N is the sequence length, and C is the feature dimension; X CLS is the feature representation assigned to cls_token, X patches is the feature representation assigned to Patches; 0:1 means selecting the element with index 0 on the sequence dimension, and 1:N means selecting all elements from index 1 to the end on the sequence dimension; S2: Local features enter the local branch for depthwise separable convolution. 3*3 depthwise separable convolution is used to perform spatial convolution on the image block sequence. The formula is: X Local =DWConv(Reshape(X patches )) (3) where X Local Represents the local branch output feature, DWConv represents the depth separation convolution, Reshape is a tensor deformation operation, which changes the dimension structure of the tensor without changing the data content, X patches It is the feature representation assigned to Patches; The global feature enters the global branch and is expanded in dimension through the fully connected layer. The specific formula is: X global =W g ·X+b g (4) where X global Represents the global branch output feature, W g is the weight matrix, b g is the bias vector, X represents the output feature; S3: The corresponding features after processing by the local branch and the global branch are fused in the dynamic gate fusion module. The formula is: G=σ((W g ·x+b g )(LayerNorm(X global,1: )) (5) X fused =G⊙X Local +(1-G)⊙X global,1: (6) where G represents the gate weight tensor, W g is the weight matrix, b g is the bias vector, LayerNorm means that after the feature vector of each patch is normalized, the mean and variance are calculated along the feature dimension, X global,1: represents the sequence feature of the fully connected transformation; X Local Represents the local branch output feature, X fused represents the final fusion feature; S4: Reconstruct the processed features into a sequence. The reconstruction formula is: Y=[X global,:1 ;X fused ] (7) Among them, Y is the final output feature, X fused represents the final fusion feature, X global,:1 The cls_token representing the global branch.

3. The multi-branch DLMDMLP and adversarial generation deepfake detection method according to claim 1, characterized in that: The learnable multi-scale feature fusion structure is as follows: first, a multi-scale downsampling operation is performed, and the formula is: Where s is the scale factor, N represents the input sequence length, and max(1,·) ensures that the pooling size is at least 1 to avoid the scale factor being too large and causing the sequence length to be 0; P s Represents the length of the sequence after downsampling; After downsampling, adaptive pooling operation is performed on each scale. The input shape is (B, N, C), which is converted to (B, C, N) through Permute operation. The sequence length is downsampled from N to P through AdaptiveAvgPool1D. s , use Permute to return (B,P s ,C),The formula for feature transformation,pooling operation is: AND s =Permute(AdaptiveAvgPool1D(Permute(Y),P s )) (9) Among them, the Permute operation means rearranging the dimensions of the tensor according to the specified dimension order. AdaptiveAvgPool1D is a function in PyTorch, which belongs to the adaptive average pooling operation and is used to pool one-dimensional data. Y represents the feature of the input pooling layer after multi-scale downsampling. Y s is the feature after downsampling; After pooling, the corresponding feature transformation and upsampling operations are performed. The specific formula is: Among them, GELU is the activation function; LinearInterpolate is used for upsampling operation to restore the processed feature sequence to the original sequence length; W S and b S are the weights and biases of the linear layer, Y s is the downsampled feature, It is the feature after the linear layer + GELU; The weights and biases of the linear layer are calculated by matrix multiplication and activated by the activation function before entering the upsampling stage. The feature shape before upsampling is converted into (B, C, P) through the Permute operation. s ), and use LinearInterpolate to reduce the sequence length from P s Restore to N and use Permute to convert back to (B, N, C) to be consistent with the input dimension; After upsampling, the fusion module is used for weight normalization and feature fusion. The features at different scales are weighted and summed according to their corresponding weights to obtain the final features after multi-scale fusion. The formula is: a s =Softmax(W s ) (12) AND out =∑ s∈S a s ·AND n (13) Where W s is the learnable weight of each scale branch, initialized to 1; Softmax is a normalization calculation used to convert a numerical vector into a probability distribution vector representing the probability of each category; α s is the normalized weight of each scale branch in multi-scale fusion, Y out is the final feature after multi-scale fusion, Y n It is the feature tensor after processing with the corresponding scale.

4. The multi-branch DLMDMLP and adversarial generation deepfake detection method according to claim 1, characterized in that: The dynamic dilated convolution unit DynamicDilatedConv adopts a dual-branch structure of small receptive field and large receptive field, and uses the gradient predictor to generate a dynamic weight map α, and fuses the features of the small receptive field and the large receptive field according to the generated α. The formula is: α=σ(BN(W g *x)) (17) f s =RELU(BN(W S *x)) (18) f l =RELU(BN(W l *x)) (19) y=α⊙f s +(1-α)⊙f l (20) Among them, x is the input feature, W g is the gradient prediction convolution kernel, α is the dynamic weight map, BN is the batch normalization operation, * represents the convolution operation, σ represents the Sigmoid activation function, W S is the convolution kernel with small receptive field, RELU is the activation function, f s Represents the small receptive field feature, W l is the convolution kernel with large receptive field, f l is the large receptive field feature, and y is the output feature; The lightweight texture preservation unit LightTexturePreserver protects high-frequency artifacts during feature downsampling through a residual structure. It is processed through two branches: texture enhancement and skipping. In the texture enhancement branch, a 3×3 convolution with a stride of 2, a normalization operation, and an activation function operation are performed in sequence. In the skipping branch, identity mapping is performed, and the residuals of the results of the two branches are added. The formula is: y=ReLU(BN(W t *x))+x (21) Where x is the input feature, W t is the texture enhancement convolution kernel, + is the residual connection, y is the output feature, BN is the normalization operation, and ReLU is the activation function; The Forgery Pattern Enhancer is processed through a predefined filter, which contains a blocking kernel and an edge detection kernel. Then, it is processed through depthwise separable convolution to extract forged features and concatenate them with the original input features. After that, 1×1 convolution, normalization, and activation function operations are performed in sequence to perform feature enhancement fusion. The formula is: f p =DepthwiseConv(x;{K edge ,K block ,...}) (24) y=ReLU(BN(W f *[x||f p ])) (25) where K edge is the edge detection kernel, K block is the block effect kernel, f p is the forged pattern feature, DepthwiseConv is the depthwise separable convolution, RELU is the activation function, BN is the batch normalization operation, * represents the convolution operation, [||] is the channel splicing, W f is a 1×1 convolution kernel, x is the input feature, and y is the output feature.

Images