An image tampering detection method integrating dynamic attention mechanism and residual noise analysis

Abstract

The present invention designs an image tampering detection method that integrates a dynamic attention mechanism and residual noise analysis. The network uses an SRM module for preprocessing at the input stage to extract noise features and splice them with the original RGB image to enhance the recognition of details. The encoder part integrates a DR-SE attention mechanism, which dynamically adjusts the compression ratio according to different inputs to optimize model performance and generalization ability. The network further integrates a feature enhancement residual module (DRH) and incorporates a gating signal G to selectively enhance or suppress features. The present invention improves the recognition ability of subtle and highly realistic tampering, and accurately highlights the imperceptible residual signal through the SRM module. The improved residual structure and DR-SE attention mechanism further optimize the management of information flow and feature representation, making the present invention have significant practical value in the field of image tampering detection.

Description

Technical Field

[0001] The present invention relates to the field of computer vision image processing, and in particular to an image tampering detection method that integrates a dynamic attention mechanism and residual noise analysis. Background Art

[0002] In this era of rapid digital development, images, as a crucial medium for information dissemination, are crucial for their authenticity and integrity. With the advancement of image manipulation technology, the realism of forged images is so astonishing that it is becoming increasingly difficult to distinguish authenticity with the naked eye. This phenomenon not only challenges traditional notions such as "seeing is believing" and "pictures are the truth," but also significantly undermines public trust in the authenticity of digital images. Therefore, developing efficient image tampering detection technology is crucial for ensuring the accuracy of news and the credibility of social media content. Advances in image tampering detection technology are expected to provide more effective support for safeguarding information authenticity and social stability. Research on image tampering detection technology can be divided into two main phases. The first phase involves image tampering detection based on traditional algorithms, which primarily rely on the detection of tampering traces and the consistency of the inherent properties of the imaging device. The second phase involves image tampering detection and localization based on deep learning. With the introduction of deep neural networks, detection methods have become more sophisticated and complex. Convolutional neural networks (CNNs), in particular, have been widely used for image tampering detection and localization.

[0003] Reference 1 (RAO Yuan, NI Jiangqun. A deep learning approach to detection of splicing and copy-move forgeries in images [C] | Proceedings of the IEEE International Workshop on Information Forensics and Security. Abu Dhabi, United Arab Emirates. IEEE, 2016: 1-6.) uses the SRM filter as the core of the convolutional layer to effectively extract residual information. After training the CNN with image blocks, it is combined with the SVM classifier to obtain the image tampering detection result. However, this method fails to provide accurate tampering localization. Reference 2 (XIAO Bin, WEI Yang, BI Xiuli, et al. Image splicing forgery detection combining coarse to refined convolutional neural network and adaptive clustering [J]. Information Sciences, 2020, 511: 172-191.) combines two networks for detection. First, the image is input into the VGG16 coarse network for preliminary recognition, and then the result is further optimized through the VGG19 fine network. However, the computational complexity is duplicated and the segmentation accuracy is not high. Reference 3 (BI Xiuli, WEI Yang, XIAO Bin, et al. RRU-net: The ringed residual U-net for image splicing forgery detection [C] ∥ Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, CA, USA. IEEE, 2019: 30-39.) designed a U-net to detect image splicing forgery types. The network constructs its encoding and decoding structure by incorporating multiple ring residual units, but the model's generalization is not high when faced with scattered attention points and image noise.

[0004] Patent 1 (Tian Xiuxia, Li Huaqiang, Zhou Aoying. A method for blindly detecting digital image tampering based on a dual-channel network [P]. Shanghai: CN110992238B, 2023-10-17.) builds separate R-FCN models for the color image channel and the steganalysis channel to perform blind detection of digital image tampering. Patent 2 (Yan Caiping, Wei Huajian, Li Hong. A method for detecting image tampering based on a convolutional neural network [P]. Zhejiang: CN116883805A, 2023-10-13.) uses a convolutional neural network to determine the location of tampered images. Patent 3 (Xia Chenxing, Cao Huizhen, Ge Bin et al. A highly robust method for detecting image tampering [P]. Anhui: CN116665027A, 2023-08-29.) provides a highly robust method for detecting image tampering. However, these methods have difficulty identifying subtle image tampering in complex scenes and cannot adaptively focus on key areas within the image. This patent uses a residual noise analysis module and designs a dynamic attention mechanism module to achieve the detection and positioning of tampered images. Summary of the Invention

[0005] The present invention demonstrates an image tampering detection method that integrates a dynamic attention mechanism and residual noise analysis. In the input stage, the SRM module is used to preprocess the image, extract noise features, and the noise map is spliced with the original RGB image and input into the codec network. This design enhances the ability to recognize image edges and detail features. The D-SE attention mechanism is integrated into the encoder structure to dynamically adjust the compression ratio to adapt to different inputs, thereby enhancing model performance and generalization capabilities. A feature enhancement residual module (DRH) is added and a gating signal G is embedded to selectively enhance or suppress features. The decoder is upsampled four times, using transposed convolution, convolution, and ReLU activation functions at each step. Skip connections are used to fuse the encoder and decoder feature maps to further improve feature representation capabilities.

[0006] 1. A method for detecting image tampering by integrating a dynamic attention mechanism and residual noise analysis, comprising the following steps:

[0007] S1. Data preprocessing: The original images of the dataset are evenly cropped from the top and bottom of the image and adjusted to a fixed width and height of 384×256;

[0008] S2, feature encoding: Apply the dynamic attention mechanism and residual noise analysis network to perform deep feature extraction on the data processed by S1. Its feature extraction backbone network structure includes the SRM module, the feature enhancement residual module and the dynamic attention mechanism module;

[0009] S3, feature decoding: Use the features extracted in S2 to perform four upsampling, use transposed convolution, convolution and ReLU activation functions, introduce skip connections, fuse the feature maps of the encoder and decoder, and finally obtain the segmentation result;

[0010] The S2 comprises the following steps:

[0011] The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows:

[0012] S21. Preprocess the image using the SRM module, which contains a specially designed convolution kernel for extracting noise-like features. The noise image is then combined with the original image using a concat operation to fuse the features and form the input data for the codec.

[0013] S22. The input data then passes through a feature-enhanced residual module, which consists of two 3×3 convolution kernels, batch normalization, a ReLU activation function, and a residual layer with a simple gating mechanism. The residual feedforward recalls the input feature information to solve the gradient degradation problem in deep networks. The feature enhancement integrates the input feature information to make the difference in image attributes between the untampered and tampered areas more obvious. The feature feedforward is defined as:

[0014] y1=F(x,{W i})+G(x)

[0015] Among them, x and y1 are the input and output of feature feedforward, W i is the weight of the i-th layer, F(x,{W i}) is the residual mapping to be learned, which can be visualized as F = W2σ(x*W1)+b, where σ represents the Relu function, b is the bias, G(x) = S*g(x) is obtained through the gating mechanism, where S represents the sigmoid function, and g(x) is the feature extracted by the 1×1 convolution layer.

[0016] Feature enhancement is defined as:

[0017] y2=x*S*G(y1)+x

[0018] Among them, x is the input, y1 is the output of feature feedforward, and y2 is the input of feature enhancement;

[0019] S23. A dynamic attention mechanism module is introduced into the feature enhancement module to adapt to different levels and diverse inputs. The feature map H×W×C is compressed to 1×1×C through hybrid pooling to achieve the conversion of spatial features to global features. The compression formula is:

[0020]

[0021] The excitation part involves the fully connected layer and activation function. The dimension reduction ratio Reduction in the excitation part is dynamically adjusted. After the dynamic compression DR module, this module includes two fully connected layers and Mish and Sigmoid functions, and uses the Clamp function to control the compression ratio R in the range of [4,16]. The formula of the excitation part is:

[0022] q=σ(W2(W1a))

[0023] The weighted feature formula is:

[0024] X′=q*a c

[0025] In this process, W2 and W1 are the weight matrices of the fully connected layer, σ is the activation function, and after the vector a passes through the first fully connected layer, the dimension changes from 1×1×C to 1×1×C / R. After passing through the second fully connected layer, the dimension changes from 1×1×C / R to 1×1×C. Finally, the generated weight vector q is used to assign weights to the feature map X.

[0026] 2. S3 includes the following steps:

[0027] S31, the decoding path performs an upsampling operation consisting of transposed convolution and batch normalization, fusing the upsampled feature map with the corresponding feature map of the encoder stage, thereby utilizing the deep features captured in the encoding stage, and introducing the feature enhancement residual module from the S21 stage to further enhance the feature expression capability of the decoding stage;

[0028] S32. Introduce skip connections to fuse the feature maps extracted by the encoder with the feature maps obtained by upsampling to enhance the feature representation ability of the model, further restore the resolution of the feature maps, and convert the processed feature maps into the final segmentation maps through 1×1 convolution.

[0029] Beneficial effects:

[0030] 1. This paper proposes an image tampering detection method that integrates a dynamic attention mechanism and residual noise analysis. It addresses the challenges faced by traditional image tampering detection technologies in dealing with complex tampering scenarios, especially in identifying subtle tampering and handling highly realistic tampering content.

[0031] 2. The present invention specifically utilizes the convolution part of the SRM module to more accurately highlight residual signals that are usually imperceptible to the naked eye but may indicate that the image has been tampered with;

[0032] 3. This invention improves the residual structure and introduces a gating mechanism. The network can dynamically adjust the propagation strength of each feature according to the content of the current data, allowing the network to manage the information flow more finely and focus on the key information of the final task;

[0033] 4. This paper proposes the DR-SE (Dynamic Reduction Squeeze-and-Excitation) attention mechanism. By dynamically adjusting the reduction ratio, the network can flexibly adjust the degree of information compression and feature activation according to different levels and diverse inputs, thereby achieving more accurate feature representation and performance optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flowchart of the image tampering detection method of the present invention;

[0035] Figure 2 It is the overall network structure diagram of the image tampering detection method of the present invention;

[0036] Figure 3 This is a structural diagram of the dynamic attention (DR-SE) module in the image tampering detection method of the present invention;

[0037] Figure 4 It is a structural diagram of the residual noise analysis module in the image tampering detection method of the present invention. Specific implementation plan

[0038] To provide a clearer understanding of the technical features, objectives, and beneficial effects of the present invention, an embodiment of the present invention is further described with reference to the accompanying drawings. The embodiment is intended only to further illustrate the present invention and is not to be construed as limiting the scope of protection of the present invention. Non-essential improvements and adjustments made by those skilled in the art based on the contents of the present invention also fall within the scope of protection of the present invention.

[0039] 1. A method for detecting image tampering by integrating a dynamic attention mechanism and residual noise analysis, comprising the following steps:

[0040] S1. Data preprocessing: The original images of the dataset are evenly cropped from the top and bottom of the image and adjusted to a fixed width and height of 384×256;

[0041] S2, feature encoding: Apply the dynamic attention mechanism and residual noise analysis network to perform deep feature extraction on the data processed by S1. Its feature extraction backbone network structure includes the SRM module, the feature enhancement residual module and the dynamic attention mechanism module;

[0042] S3, feature decoding: Use the features extracted in S2 to perform four upsampling, use transposed convolution, convolution and ReLU activation functions, introduce skip connections, fuse the feature maps of the encoder and decoder, and finally obtain the segmentation result;

[0043] The S2 comprises the following steps:

[0044] The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows:

[0045] S21. Preprocess the image using the SRM module, which contains a specially designed convolution kernel for extracting noise-like features. The noise image is then combined with the original image using a concat operation to fuse the features and form the input data for the codec.

[0046] S22. The input data then passes through a feature-enhanced residual module, which consists of two 3×3 convolution kernels, batch normalization, a ReLU activation function, and a residual layer with a simple gating mechanism. The residual feedforward recalls the input feature information to solve the gradient degradation problem in deep networks. The feature enhancement integrates the input feature information to make the difference in image attributes between the untampered and tampered areas more obvious. The feature feedforward is defined as:

[0047] y1=F(x,{W i})+G(x)

[0048] Among them, x and y1 are the input and output of feature feedforward, W i is the weight of the i-th layer, F(x,{W i}) is the residual mapping to be learned, which can be visualized as F = W2σ(x*W1)+b, where σ represents the Relu function, b is the bias, G(x) = S*g(x) is obtained through the gating mechanism, where S represents the sigmoid function, and g(x) is the feature extracted by the 1×1 convolution layer.

[0049] Feature enhancement is defined as:

[0050] y2=x*S*G(y1)+x

[0051] Among them, x is the input, y1 is the output of feature feedforward, and y2 is the input of feature enhancement;

[0052] S23. A dynamic attention mechanism module is introduced into the feature enhancement module to adapt to different levels and diverse inputs. The feature map H×W×C is compressed to 1×1×C through hybrid pooling to achieve the conversion of spatial features to global features. The compression formula is:

[0053]

[0054] The excitation part involves the fully connected layer and activation function. The dimension reduction ratio Reduction in the excitation part is dynamically adjusted. After the dynamic compression DR module, this module includes two fully connected layers and Mish and Sigmoid functions, and uses the Clamp function to control the compression ratio R in the range of [4,16]. The formula of the excitation part is:

[0055] q=σ(W2(W1a))

[0056] The weighted feature formula is:

[0057] X′=q*a c

[0058] In this process, W2 and W1 are the weight matrices of the fully connected layer, σ is the activation function, and after the vector a passes through the first fully connected layer, the dimension changes from 1×1×C to 1×1×C / R. After passing through the second fully connected layer, the dimension changes from 1×1×C / R to 1×1×C. Finally, the generated weight vector q is used to assign weights to the feature map X.

[0059] 2. S3 includes the following steps:

[0060] S31, the decoding path performs an upsampling operation consisting of transposed convolution and batch normalization, fusing the upsampled feature map with the corresponding feature map of the encoder stage, thereby utilizing the deep features captured in the encoding stage, and introducing the feature enhancement residual module from the S21 stage to further enhance the feature expression capability of the decoding stage;

[0061] S32. Introduce skip connections to fuse the feature maps extracted by the encoder with the feature maps obtained by upsampling to enhance the feature representation ability of the model, further restore the resolution of the feature maps, and convert the processed feature maps into the final segmentation maps through 1×1 convolution.

[0062] Simulation experiment

[0063] In order to quantitatively evaluate the segmentation ability of the network constructed by the present invention, the following evaluation indicators are used: precision, recall, and F1 score. The segmentation network of the present invention achieved 0.959, 0.935, and 0.944 in precision, recall, and F1 score, respectively, indicating good segmentation effect.

[0064] The above describes the method of the present invention. Those skilled in the art can implement the method of the present invention based on the description of this content. Based on the above content of the present invention, other embodiments obtained by those skilled in the art without making any creative work should fall within the scope of protection of the present invention.

Claims

1. An image tampering detection method integrating dynamic attention mechanism and residual noise analysis, characterized in that: The following steps are involved: S1. Data preprocessing: The original images of the dataset are evenly cropped from the top and bottom of the image and adjusted to a fixed width and height of 384×256; S2, feature encoding: Apply dynamic attention mechanism and residual noise analysis network to perform deep feature extraction on the data processed by S1, Its feature extraction backbone network structure includes SRM module, feature enhancement residual module and dynamic attention mechanism module; S3, feature decoding: Use the features extracted in S2 to perform four upsampling, use transposed convolution, convolution and ReLU activation functions, introduce skip connections, fuse the feature maps of the encoder and decoder, and finally obtain the segmentation result; The S2 comprises the following steps: The feature extraction backbone network is constructed by a convolutional neural network. The specific network structure is as follows: S21. Preprocess the image using the SRM module, which contains a specially designed convolution kernel for extracting noise-like features. The noise image is then combined with the original image using a concat operation to fuse the features and form the input data for the codec. S22. The input data then passes through a feature-enhanced residual module, which consists of two 3×3 convolution kernels, batch normalization, a ReLU activation function, and a residual layer with a simple gating mechanism. The residual feedforward recalls the input feature information to solve the gradient degradation problem in deep networks. The feature enhancement integrates the input feature information to make the difference in image attributes between the untampered and tampered areas more obvious. The feature feedforward is defined as: y1=F(x,{W i })+G(x) Among them, x and y1 are the input and output of feature feedforward, W i is the weight of the i-th layer, F(x,{W i }) is the residual mapping to be learned, which can be visualized as F = W2σ(x*W1)+b, where σ represents the Relu function, b is the bias, G(x) = S*g(x) is obtained through the gating mechanism, where S represents the sigmoid function, and g(x) is the feature extracted by the 1×1 convolution layer. Feature enhancement is defined as: y2=x*S*G(y1)+x Among them, x is the input, y1 is the output of feature feedforward, and y2 is the input of feature enhancement; S23. A dynamic attention mechanism module is introduced into the feature enhancement module to adapt to different levels and diverse inputs. The feature map H×W×C is compressed to 1×1×C through hybrid pooling to achieve the conversion of spatial features to global features. The compression formula is: The excitation part involves the fully connected layer and activation function. The dimension reduction ratio Reduction in the excitation part is dynamically adjusted. After the dynamic compression DR module, this module includes two fully connected layers and Mish and Sigmoid functions, and uses the Clamp function to control the compression ratio R in the range of [4,16]. The formula of the excitation part is: q=σ(W2(W1a)) The weighted feature formula is: X′=q*a c In this process, W2 and W1 are the weight matrices of the fully connected layer, σ is the activation function, and after the vector a passes through the first fully connected layer, the dimension changes from 1×1×C to 1×1×C / R. After passing through the second fully connected layer, the dimension changes from 1×1×C / R to 1×1×C. Finally, the generated weight vector q is used to assign weights to the feature map X.

2. The image tampering detection method integrating dynamic attention mechanism and residual noise analysis according to claim 1 is characterized in that: The S3 includes the following steps: S31, the decoding path performs an upsampling operation consisting of transposed convolution and batch normalization, fusing the upsampled feature map with the corresponding feature map of the encoder stage, thereby utilizing the deep features captured in the encoding stage, and introducing the feature enhancement residual module from the S21 stage to further enhance the feature expression capability of the decoding stage; S32. Introduce skip connections to fuse the feature maps extracted by the encoder with the feature maps obtained by upsampling to enhance the feature representation ability of the model, further restore the resolution of the feature maps, and convert the processed feature maps into the final segmentation maps through 1×1 convolution.

Images