Annular residual network image stitching tampering detection method based on Swin Transform

Abstract

The invention is applicable to the technical field of digital image processing, and provides a Swin Transform-based annular residual network image stitching tampering detection method, which comprises the following steps of: constructing a detection network, and capturing global context and local details by adopting the Swin Transform by an encoder of the detection network; an SE module is introduced into jump connection between an encoder and a decoder, and self-adaptive weighting of a feature channel is achieved; and the decoder adopts an RRU-Net integrated with a multi-scale feature fusion module so as to accurately position a multi-scale tampering region. And then, offline training is carried out on the network by using the annotation data set, and finally, online detection is carried out by using the trained model. According to the method, the problems that an existing method is insensitive to weak edge artifacts, positioning of a multi-scale tampering area is inaccurate, and robustness of post-processing operation is poor are effectively solved, and the accuracy and positioning precision of image stitching tampering detection are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, and particularly relates to a method for detecting image stitching tampering based on a ring residual network using Swing Transformer. Background Technology

[0002] Image splicing tampering refers to a method of digital image manipulation in which image fragments from different sources are copied and pasted into a target image to forge or conceal the original content. Its detection aims to automatically determine whether an image has been tampered with and locate the tampered area.

[0003] Existing detection methods are mainly divided into traditional methods and deep learning methods. Traditional methods include those based on essential image attributes (such as color, texture, and noise), imaging device attributes (such as CFA interpolation and sensor noise), image compression attributes (such as DCT coefficients and block artifacts), and hashing techniques. These methods generally suffer from sensitivity to changes in image resolution, post-processing operations (such as filtering and compression), device stability, and hash collisions, exhibiting poor robustness and difficulty in precise localization.

[0004] Deep learning-based detection methods, especially encoder-decoder architectures (such as Ring Residual U-Net, or RRU-Net), have achieved performance improvements through end-to-end learning. However, existing models still have limitations: when the stitched region is visually highly consistent with the original image, traditional convolution struggles to capture subtle edge artifacts; when faced with multi-scale, shape-variable tampered regions, a single receptive field cannot capture both the overall picture and details; post-processing operations can easily erase tampering clues, resulting in insufficient model feature discrimination power, coarse boundary localization, and challenges to accuracy and localization precision. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detecting image stitching tampering based on a ring residual network using Swing Transformer, aiming to solve the problems mentioned in the background art.

[0006] The present invention is implemented as follows: a method for detecting image stitching tampering based on a ring residual network using Swin Transformer, comprising the following steps: Step 1: Construct an image stitching and tampering detection network based on Swing Transformer and multi-scale feature fusion; The image stitching and tampering detection network includes: The encoder based on the Swin Transformer uses the Swin Transformer network to extract multi-scale global and local features from the input image. Based on the skip connection of the SE module (Squeeze-and-Excitation Module), the SE module is introduced on the feature transfer path between the encoder and the decoder to perform channel attention weighting on the feature map output by the encoder. The RRU-Net decoder based on multi-scale feature fusion adopts the RRU-Net network and introduces a multi-scale feature fusion module after at least one upsampling layer of the decoder to fuse features of different scales and locate the tampered region. Step 2: Offline training of the image stitching tampering detection network; The constructed image splicing tampering detection network was trained offline using a dataset containing tampered images and their ground truth masks to obtain a trained image splicing tampering detection network model. Step 3: Online detection; The model directly outputs a mask map of the tampered region with the same size as the input image, thus completing the detection and localization of image stitching tampering.

[0007] In a further technical solution, in step 1, the encoder based on Swing Transformer includes an image block partitioning module and Stage 1 to Stage 4 modules connected in sequence. The image block partitioning module is used to divide the input image into multiple non-overlapping image blocks; The Stage1 module includes a linear embedding layer and at least one Swing Transformer Block for shallow feature extraction of image patches; Each of the Stage2 to Stage4 modules includes an image fusion layer and at least one Swin TransformerBlock, used to downsample the feature map, upscale the channels, and extract deep features, outputting feature maps of different scales.

[0008] A further technical solution is that the Swin Transformer Block includes a layer normalization layer, a window multi-head self-attention layer, a sliding window multi-head self-attention layer, and a multilayer perceptron, which performs feature transformation and enhancement through residual connections.

[0009] A further technical solution is that the SE module includes one global average pooling layer, two fully connected layers, one ReLU activation function, one Sigmoid activation function, and a recalibration operation. Channel weights are generated through squeezing, activation, and scaling steps and multiplied with the input feature map.

[0010] A further technical solution is that the RRU-Net decoder includes multiple multi-scale feature fusion modules, multiple annular residual units, and a convolutional layer; The multi-scale feature fusion module is used to extract multi-scale features through convolutional kernels of different sizes, and after weighted fusion by a self-attention mechanism, feature enhancement is performed through dilated convolution. The annular residual unit includes a residual propagation path and a residual feedback path, which are used to enhance the feature differences between the unaltered region and the tampered region through forward convolution and backward feedback.

[0011] A further technical solution is that the working process of the annular residual unit is as follows: Step 1: The input features pass through a residual propagation path containing two convolution operations to obtain the intermediate output; Step 2: The intermediate output is traced back to the unit input through the residual feedback path and summed with the original input features; Step 3: Pass the result obtained in step 2 through the residual propagation path again to obtain the final output.

[0012] In a further technical solution, in step 2, the network is trained using the public dataset CASIA V1.0, with a training set to test set ratio of 8:2; the loss function is binary cross-entropy loss; the optimization is performed using stochastic gradient descent, and the initial learning rate, momentum, weight decay, batch size, and training epochs are set.

[0013] The image stitching tampering detection method based on the Swin Transformer-based ring residual network provided in this invention has the following beneficial effects: (1) Stronger global modeling capability: Using Swing Transformer as the encoder, the model efficiently captures long-distance pixel dependencies and global context information through local window and shift window strategies, which improves the model's ability to perceive tampering traces, especially weak edge artifacts, and enhances feature discrimination and model robustness.

[0014] (2) Smarter feature reuse: The SE module is introduced in the skip connection, which enables the network to adaptively adjust the channel importance of encoder features according to the content of the input image, strengthen key clues, suppress background noise, and thus pass more discriminative features to the decoder, thereby improving the overall detection performance.

[0015] (3) More refined multi-scale localization: The multi-scale feature fusion module is integrated into the RRU-Net decoder. By combining feature extraction and fusion of different receptive fields, the model can effectively capture both large and small tampered regions at the same time. Combined with the iterative optimization mechanism of the annular residual unit, more accurate pixel-level tampered region boundary localization is achieved.

[0016] (4) Higher accuracy and robustness: Through the organic combination of the above technologies, the method of the present invention has achieved excellent detection accuracy on public datasets (such as 94.8% on CASIA V1.0) and exhibits good robustness to post-processing operations such as compression and filtering. Attached Figure Description

[0017] Figure 1 A flowchart of the image stitching tampering detection method based on the Swin Transformer-based ring residual network provided in this embodiment of the invention; Figure 2 A flowchart of a Swing Transformer-based encoder; Figure 3 A flowchart of the Swing Transformer Block module in the Swing Transformer encoder; Figure 4 A flowchart of the MLP Block in the Swing Transformer Block module; Figure 5 The flowchart for the SE module; Figure 6 A flowchart of the RRU-Net decoder based on multi-scale feature fusion; Figure 7 A flowchart for the multi-scale feature fusion module; Figure 8 This refers to the execution process of a ring-shaped residual unit. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0020] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting image stitching tampering based on a loop residual network using the Swing Transformer, comprising the following steps: Step 1: Construct an image stitching and tampering detection network based on Swing Transformer and multi-scale feature fusion; The network consists of a Swing Transformer-based encoder, a skip connection based on the SE module, and an RRU-Net decoder based on multi-scale feature fusion. The functions of each component are described below: Swing Transformer-based encoder: As attached Figure 2 As shown, the Swin Transformer is used as the encoder instead of a traditional convolutional network. It is a visual Transformer model based on a self-attention mechanism, addressing many challenges in transferring the Transformer from the language domain to the vision domain. Designed specifically for vision tasks, it processes multi-scale information in a hierarchical and windowed manner, improving computational efficiency and feature extraction capabilities. The network consists of an image patch partitioning module and four stages (Stage 1-Stage 4 modules).

[0021] The specific implementation process is as follows: First, the input image is divided into multiple non-overlapping small blocks by the image block partitioning module, and then input into the four Stage1-Stage4 modules for processing to construct feature maps of different sizes.

[0022] (1) The Stage 1 module consists of a linear embedding layer and two Swin TransformerBlocks. Its function is to extract shallow geometric details and obtain feature maps while maintaining the highest resolution. .

[0023] The linear embedding layer maps each small image patch into a low-dimensional vector space to obtain the initial feature embedding map. .

[0024] The Swin Transformer Block is primarily used for feature extraction and transformation; its structure is shown in the attached figure. Figure 3As shown, it consists of four LayerNorm (LN) layers, a Window Multi-head Self-Attention (W-MSA) layer, a Shifted-Window Multi-head Self-Attention (SW-MSA) layer, and two Multi-Layer Perceptrons (MLPs). Its specific operation is as follows: The input feature embedding map first passes through the LayerNorm layer to eliminate scale differences between different features, reduce variations in input distribution between network layers, and improve training convergence and stability; then it is input to the Window Multi-head Self-Attention (W-MSA) layer. The W-MSA layer divides the input feature map into non-overlapping small windows, and performs multi-head self-attention calculations independently within each window. This reduces computational cost and captures fine-grained local features. The specific steps are as follows: 1) Divide the feature map into Size (usually selected) The window.

[0025] 2) Encode the spatial dimension information of a single window feature to form an input vector containing the positional features. .

[0026] 3) For each input vector Perform a linear transformation to generate the corresponding query vector. key vector Sum value vector Its mathematical description is: (1); in, , and This is the weight matrix.

[0027] 4) Divide these transformed vectors into h "heads", each head having its own independent Query, Key, and Value matrices. Perform a scaled dot product operation on each head to obtain the attention output for that head, mathematically described as follows: (2); in, It is the dimension of the key vector; For attention output, For activation functions; 5) Concatenate the attention outputs of all heads together, then fuse them through a linear layer to obtain the final attention output vector, which can be mathematically described as follows: (3); (4); in, For multi-head attention output, This means concatenating the outputs of each multi-head attention along the feature dimension. For the first The output of each attention head, , The preset number of heads.

[0028] W-MSA Output and Feature Embedding Map Adding the residuals yields a new enhanced feature map. Then, it is input again into the layer normalization layer and then into the multilayer perceptron (MLP).

[0029] The role of a multilayer perceptron (MLP) is to perform a non-linear transformation on the features after the attention mechanism, enabling the model to fit more complex functional relationships. Its structure is shown in the attached figure. Figure 4 As shown, it consists of two fully connected layers, a GELU activation function, and two dropout layers. The specific working process is as follows: Enhanced feature maps The model enters the first fully connected layer for dimensionality increase to increase its capacity and provide a richer feature space for subsequent nonlinear transformations. Then, it enters the GELU activation function of the nonlinear activation unit for high-dimensional nonlinear transformation. Subsequently, it enters the random deactivation layer to randomly discard some features to prevent the model from overfitting.

[0030] The second fully connected layer performs dimensionality reduction, bringing the increased dimensionality back to the original feature dimensions. This ensures consistency between the input and output dimensions of the MLP, facilitating integration with residual connections. Finally, the system re-enters a random deactivation layer, generating a result that is then compared with... The residuals are summed to obtain the output of the first block. .

[0031] Output of the first block It is input into the normalization layer, and then into SW-MSA.

[0032] SW-MSA employs a shifted window design, allowing windows of adjacent blocks to overlap. This enables features from different windows to indirectly complement each other, overcoming the structural limitations of W-MSA. The specific operation involves shifting the feature map horizontally and vertically. Each pixel creates an overlapping area between adjacent windows, enabling cross-window information interaction; shifting will produce... To avoid computational redundancy, SW-MSA employs cyclic shifting and masking mechanisms within the edge window, ensuring that subsequent attention calculations are only effective within the original continuous region. Finally, similar to W-MSA, within the processed window, the features of each window are linearly projected to obtain the query (Q), key (K), and value (V). Multi-head attention scores are calculated, and the results from each head are concatenated and linearly projected to obtain the output features. .

[0033] and The enhanced output features are obtained by summing the residuals. It then enters the layer normalization layer again, then enters the multilayer perceptron (MLP), and finally... The residuals are summed to obtain the output of the second block, which is also the feature map output of Stage 1. .

[0034] (2) Stages 2-4 consist of one image fusion layer (Patch Merging) and two Swin Transformer Blocks, one image fusion layer (Patch Merging) and six Swin Transformer Blocks, and one image fusion layer (Patch Merging) and two Swin Transformer Blocks, respectively. Each of these three stages outputs a feature map. , and .

[0035] The role of the image fusion layer (Patch Merging) is to downsample the feature map while increasing the number of channels. By downsampling layer by layer (reducing spatial resolution) and channel upscaling (increasing feature dimension), the original pixel information of the input image is gradually transformed into a more abstract and semantic feature representation. At the same time, by combining the alternating use of W-MSA and SW-MSA of the Swing Transformer Block, spatial dependencies at different scales are captured in each stage.

[0036] Skip connections based on SE modules combine channel attention mechanisms with residual connections. The core idea is to dynamically adjust the weights of different channels, allowing the model to focus more on the features of important channels and suppress irrelevant or redundant information, thereby improving feature utilization efficiency.

[0037] As attached Figure 5As shown, the SE module contains one global average pooling layer, two fully connected layers, one ReLU activation function, one Sigmoid activation function, and a scale operation. Its operation mainly includes three key steps: squeezing, excitation, and scaling. The specific operation steps are as follows: (1) Squeezing: Perform global average pooling on each channel of the input feature map to obtain a dimension of A 3D vector, where each element corresponds to the integrated result of global information from one channel, is used as the input feature map. For example, its mathematical description is: (5); in, Indicates the first The result of global average pooling for each channel. Represents the feature map of the input In the The first channel, the first line, number The value of the element at the column position. It is the spatial size of the feature map.

[0038] (2) Excitation: The extruded material is obtained through the first fully connected layer. Dimensionality reduction of dimensional vectors to ,in It is a hyperparameter called the dimensionality reduction ratio, and nonlinearity is introduced using the ReLU activation function; the second fully connected layer re-upgrades the dimensionality of the reduced vector to its original dimension. The number of channels is restored to be consistent with the number of input channels. The activation function used is Sigmoid, which maps the output values ​​to the range of 0 to 1, thus obtaining the weight of each channel. Its mathematical description is as follows: (6); in, It is the input of the first fully connected layer. It is the output of the second fully connected layer. dimensional vector, and These are the weight matrices for the first and second fully connected layers, respectively. and These are the ReLU and Sigmoid activation functions, respectively.

[0039] (3) Scaling: The weights obtained in the activation step are multiplied by the pixels at each spatial location of the corresponding channel of the input feature map. The mathematical expression is as follows: (7); Using this method, the output feature map of Stage 1 in the encoder... Stage 2 upscaling output feature map Stage3 output feature map After channel weighting by the SE module, it becomes a feature map. It is directly passed to the decoder.

[0040] RRU-Net decoder based on multi-scale feature fusion: as attached Figure 6 As shown, the decoder uses an RRU-net network, which mainly consists of three multi-scale feature fusion modules, three annular residual units, and one... Composition of convolutional layers.

[0041] The multi-scale feature fusion module is attached. Figure 7 As shown, it consists of three convolutional layers with different kernel sizes, a self-attention mechanism, and dilated convolutions. The specific implementation process of the multi-scale feature fusion module is as follows: input feature map, use convolutional kernels of different sizes ( , , To extract multi-scale features, three feature maps of different scales are obtained. Then, a self-attention mechanism is used to adaptively select the most important features. The feature maps of different scales are then weighted and fused. The fused feature map is input into a dilated convolutional layer. Dilated convolution increases its receptive field by inserting holes between the elements of the convolutional kernel without increasing the number of parameters. While maintaining resolution, it can capture a wider range of contextual information and perform feature enhancement.

[0042] As attached Figure 8 The image shows a ring-shaped residual element, mainly composed of residual propagation and residual feedback. Its working process is as follows: Step 1: The input image is used as the input to the circular residual unit. After two convolution operations and residual concatenation, the output is obtained.

[0043] Step 2: The output obtained in Step 1 is fed back to the input of the circular residual unit through residual feedback and summed with the original input image.

[0044] Step 3: Using the result obtained in Step 2 as input, perform residual propagation again using the same method as in Step 1 to obtain the final output of the annular residual unit.

[0045] The two convolutional operations in residual propagation forward the feature map, with the two convolutional layers utilizing the ReLU normalized activation function to reduce parameter dependencies. Residual feedback mimics the human brain's memory consolidation mechanism, integrating input feature information to make the differences in image attributes between tampered and untampered regions more apparent.

[0046] The mathematical description of residual propagation is as follows: (8); in, For input, For output, Indicates the first Layer weights, Indicates linear projection. The residual mapping to be learned is mathematically described as follows: (9); in, These are the weights of the convolutional layer.

[0047] The mathematical expression for residual feedback is: (10); in, To enhance output, Represents a linear projection function. It is the Sigmoid activation function, which performs non-linear processing on the results of linear projection.

[0048] Convolution is used to adjust the number of channels, and the final output image size is... .

[0049] Step 2: Offline training of the image stitching tampering detection network; The manipulated image to be detected is input into the image manipulation detection network constructed in step 1, and the network is trained offline. The public dataset CASIA V1.0 is used for both training and testing, with the training and testing sets split in an 8:2 ratio. CASIA V1.0 contains 920 manipulated images, mostly with a resolution of 384×256. The forged regions contain various shapes, and post-processing methods including filtering, blurring, and deformation are applied to hide manipulation traces. The loss function uses binary cross-entropy to obtain the image loss in pixels. During training, the initial learning rate is 0.01, stochastic gradient descent is used, with a momentum of 0.9, weight decay of 0.005, and a batch size of 16. A total of 100 training rounds are performed. This method achieves an accuracy of 94.8% on the CASIA V1.0 dataset.

[0050] Step 3: Online detection; The image to be detected is input into the network model trained in step 2. The model directly outputs a tampered region localization mask of the same size as the input image, thus completing the detection and localization of image splicing tampering.

[0051] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting image stitching tampering based on a ring-shaped residual network using Swing Transformer, characterized in that, Includes the following steps: Step 1: Construct an image stitching and tampering detection network based on Swing Transformer and multi-scale feature fusion; The image stitching and tampering detection network includes: The encoder based on the Swin Transformer uses the Swin Transformer network to extract multi-scale global and local features from the input image. Based on the skip connection of the SE module, the SE module is introduced on the feature transfer path between the encoder and the decoder to perform channel attention weighting on the feature map output by the encoder. The RRU-Net decoder based on multi-scale feature fusion adopts the RRU-Net network and introduces a multi-scale feature fusion module after at least one upsampling layer of the decoder to fuse features of different scales and locate the tampered region. Step 2: Offline training of the image stitching tampering detection network; The constructed image splicing tampering detection network was trained offline using a dataset containing tampered images and their ground truth masks to obtain a trained image splicing tampering detection network model. Step 3: Online detection; The model directly outputs a mask map of the tampered region with the same size as the input image, thus completing the detection and localization of image stitching tampering.

2. The image stitching tampering detection method based on Swing Transformer and circular residual network according to claim 1, characterized in that, In step 1, the Swin Transformer-based encoder includes an image block partitioning module and Stage 1 to Stage 4 modules connected in sequence. The image block partitioning module is used to divide the input image into multiple non-overlapping image blocks; The Stage1 module includes a linear embedding layer and at least one Swing Transformer Block for shallow feature extraction of image patches; Each of the Stage2 to Stage4 modules includes an image fusion layer and at least one Swin TransformerBlock, used to downsample the feature map, upsample the channels, and extract deep features, outputting feature maps of different scales.

3. The image stitching tampering detection method based on the Swing Transformer-based ring residual network according to claim 2, characterized in that, The Swin Transformer Block includes a layer normalization layer, a window multi-head self-attention layer, a sliding window multi-head self-attention layer, and a multilayer perceptron, which performs feature transformation and enhancement through residual connections.

4. The image stitching tampering detection method based on Swing Transformer-based circular residual network according to claim 1, characterized in that, The SE module includes one global average pooling layer, two fully connected layers, one ReLU activation function, one Sigmoid activation function, and a recalibration operation. It generates channel weights through squeezing, activation, and scaling steps and multiplies them with the input feature map.

5. The image stitching tampering detection method based on a ring residual network according to claim 1, characterized in that, The RRU-Net decoder includes multiple multi-scale feature fusion modules, multiple annular residual units, and a convolutional layer. The multi-scale feature fusion module is used to extract multi-scale features through convolutional kernels of different sizes, and after weighted fusion by a self-attention mechanism, feature enhancement is performed through dilated convolution. The annular residual unit includes a residual propagation path and a residual feedback path, which are used to enhance the feature differences between the unaltered region and the tampered region through forward convolution and backward feedback.

6. The image stitching tampering detection method based on the Swin Transformer-based ring residual network according to claim 5, characterized in that, The working process of the annular residual unit is as follows: Step 1: The input features pass through a residual propagation path containing two convolution operations to obtain the intermediate output; Step 2: The intermediate output is traced back to the unit input through the residual feedback path and summed with the original input features; Step 3: Pass the result obtained in step 2 through the residual propagation path again to obtain the final output.

7. The image stitching tampering detection method based on Swing Transformer-based circular residual network according to claim 1, characterized in that, In step 2, the network is trained using the public dataset CASIA V1.0, with a training set to test set ratio of 8:2; the loss function is binary cross-entropy loss; the optimization method is stochastic gradient descent, and the initial learning rate, momentum, weight decay, batch size, and training epochs are set.

Images