Image tampering positioning method based on edge guidance and multi-scale feature fusion

Abstract

The invention discloses an image tampering positioning method and system based on edge guidance and multi-scale feature fusion. The method comprises the following steps: firstly, constructing a deep learning framework based on a Vision Transformer (ViT) backbone network, and realizing efficient modeling of local and global tampering features in an image in combination with a content awareness residual module; in order to improve the accuracy and robustness of tampering region detection, an edge guiding strategy is designed, and the strategy combines a Sobel operator, morphological operation and an edge segmentation loss function to reinforce tampering boundary feature expression, so that the sensitivity to an unnaturally fused region is improved. The invention further provides a multi-scale supervision mechanism, a coordinate attention module is combined, the model is guided to fuse semantic features under different scales, and the adaptive detection capability of the model on tampering regions with different scales is enhanced. In the implementation process, in the training stage, the detection performance of the model on image tampering is gradually improved by optimizing all modules of the deep learning network; and then, inputting an image to be detected by using the trained network, and automatically identifying and positioning a tampering region in the image. The method can effectively detect and position the tampering area of the image, has a wide application prospect, and can provide a more reliable and efficient image tampering detection solution especially in the fields of digital forensics, image content security and the like.

Description

Technical Field

[0001] This invention relates to the field of image tampering detection technology, and in particular to an image tampering detection method and system based on edge guidance and multi-scale feature fusion. Background Technology

[0002] With the rapid development of image tampering technology and the widespread use of various image editing software, ordinary users can edit and modify image content without professional knowledge. While this greatly facilitates image creation and visual dissemination, it also brings serious privacy leaks and information security risks. In practical applications, malicious image tampering has been widely used to create false information, mislead public opinion, and even to commit fraud, extortion, and other illegal and criminal activities, leading to a series of serious social security problems. Therefore, how to effectively detect and accurately locate image tampering has become an important research topic in the fields of computer vision and information security.

[0003] Currently, common image tampering methods mainly include splicing (transferring part of an image to another image), copy-paste (copying a part of the same image and pasting it to another location), and image removal (deleting a target area from the original image and repairing it with visually natural background content). These tampering operations are often accompanied by secondary editing techniques such as image rotation, brightness adjustment, boundary smoothing, and compression, making the tampered area more natural and difficult to identify, greatly increasing the difficulty of detection.

[0004] Although image tampering detection technology has made some progress in recent years, with deep learning-based detection models showing good performance on multiple datasets, many challenges remain in practical applications. First, feature extraction from tampered regions is difficult. Finely edited tampered regions are visually very similar to real regions, and existing detection models often struggle to accurately capture these subtle artifacts or inconsistencies, leading to frequent false positives and false negatives. Second, most existing methods fail to fully utilize image edge information. In fact, the edges of tampered regions often exhibit artifact features such as discontinuities, blurred boundaries, and residual noise, which are crucial for identifying tampering. Because existing detection methods rely heavily on deep convolutional features or salient region extraction, neglecting the guiding role of edge details, the model's detection performance deteriorates in complex scenes, especially when dealing with small or irregularly shaped tampered regions.

[0005] In summary, existing technologies still have significant shortcomings in terms of precise identification of tampered regions, edge feature extraction, and model robustness. There is an urgent need for a new image tampering detection method that can integrate contextual semantics, spatial structure, and edge guidance information to achieve accurate detection and localization of various complex tampering behaviors. Summary of the Invention

[0006] This invention discloses an image tampering detection method based on edge guidance and multi-scale feature fusion, which aims to improve the accuracy of image tampering detection and localization. By introducing multiple functional modules and fusing feature information at different scales, this invention effectively solves the problems of incomplete capture of detailed features, insufficient utilization of edge information, and poor model robustness in existing methods for image tampering detection.

[0007] In this invention, the image tampering detection process begins with the input image. The input image is first segmented into multiple small blocks through a patch embedding operation, and then features are extracted using a ViT backbone network and a context-aware residual module. The output of this step is the intermediate features of the image, which are further processed by the context-aware residual module to enhance the perception of details in the tampered region. Next, a multi-scale supervised module further optimizes the extracted features. This module processes the features at multiple resolutions through convolutional operations to integrate fine-grained local features and global contextual information, generating enhanced multi-scale feature maps. Finally, a feature fusion module uses an attention mechanism to fuse these feature maps, generating a comprehensive feature representation. A binary mask is then generated by the decoder to accurately locate the tampered region.

[0008] The context-aware residual module, an innovation of this invention, aims to enhance the extraction capability of local features and improve the robustness of the model in complex contexts by combining global contextual information. The module employs a depthwise separable convolutional structure, including depthwise convolution and pointwise convolution, to efficiently capture intra-channel and inter-channel features, reduce computational burden, and effectively integrate feature information. This module further enhances the model's perception of subtle tampering regions through residual connections, particularly for the extraction of detailed and texture features.

[0009] The multi-scale supervision module enhances the model's ability to detect tampered regions of different sizes by fusing feature information across scales. It generates feature maps of different resolutions through multiple upsampling and downsampling operations and utilizes the CoordAttention module to model spatial and channel information. The CoordAttention module first performs global pooling on the input feature map to generate global descriptive information in the height and width directions, then generates attention weights through convolution operations, and finally applies these attention weights to the original feature map, enhancing its focus on tampered regions.

[0010] To further enhance the model's ability to perceive boundary regions, this invention designs an edge-guided strategy, particularly in the extraction and enhancement of edge features. This strategy utilizes the Sobel operator and morphological operations to generate edge masks, combining local gradient features of the image with macroscopic boundary information to accurately capture the edges of tampered regions. Through comparative supervision with edge labels, the model can gradually learn to focus on the boundary features of tampered regions, thereby improving detection accuracy.

[0011] Finally, this invention designs a composite loss function that combines pixel-level cross-entropy loss and edge supervision loss, improving the overall model performance through joint optimization. During training, the model further improves its ability to detect complex tampered regions by simultaneously optimizing the overall image segmentation performance and the accurate modeling of edge regions, especially exhibiting higher robustness under noisy interference and blurred boundary conditions.

[0012] In summary, this invention proposes a novel image tampering detection method through the collaborative work of multiple modules. This method can effectively improve the detection and localization capabilities of complex tampered areas and achieves excellent performance on multiple public datasets. It also has strong anti-noise interference capabilities and robustness, and is widely applicable to the fields of image forensics and content security.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] (1) Combination of Context-Aware Residual Module and Multi-Scale Supervision Module: By effectively combining the context-aware residual module and the multi-scale supervision module, this invention significantly improves the model's ability to perceive artifacts in tampered regions and its adaptability at multiple scales. The context-aware residual module enhances the capture of subtle tampered regions and local textures, while the multi-scale supervision module can handle various tampering forms at multiple scales, thereby improving the model's detection accuracy in different tampering scenarios.

[0015] (2) Fusion of attention mechanism and edge guidance strategy: This invention improves the detection accuracy of the model by introducing a fusion attention mechanism and an edge guidance strategy, while enhancing the model's localization ability in complex boundary regions. The attention mechanism can adaptively strengthen the features of important regions, while the edge guidance strategy further helps the model accurately capture the boundary information of the tampered region, especially in cases of complex backgrounds or blurred tampered regions, demonstrating excellent robustness.

[0016] (3) Optimization of the composite loss function: The composite loss function optimization method proposed in this invention effectively enhances the information synergy between modules. By combining segmentation loss and edge loss, the model can better handle the boundaries and details of the tampered region, thereby improving the model's generalization ability and robustness in complex image tampering scenarios. Especially under high noise and blurred boundaries, it can still achieve accurate detection and localization.

[0017] (4) Excellent scalability and adaptability: The overall method proposed in this invention has excellent scalability and adaptability, and can be widely applied to various practical scenarios, such as media content authentication, judicial evidence collection, and network security. Whether for different types of tampered images or for different detection tasks, the method of this invention can provide effective solutions and has broad application prospects. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the network framework described in this invention.

[0019] Figure 2 This is a schematic diagram of the image tampering described in this invention.

[0020] Figure 3 This is a schematic diagram of the edge mask generation method described in this invention.

[0021] Figure 4 The F1 scores of this invention are compared with those of state-of-the-art methods on the CASIAV1, Coverage, and Columbia datasets.

[0022] Figure 5 This paper compares the F1 scores of the present invention with those of state-of-the-art methods on the CASIAV1 dataset in terms of robustness.

[0023] Figure 6 A comparison of the visualization effects of different methods. Detailed Implementation

[0024] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0026] This invention relates to a deep learning network structure for image tampering detection and localization, capable of pixel-level detection and precise localization of tampered areas. The network structure is as follows: Figure 1 As shown, it mainly includes a content-aware residual module, a multi-scale supervision module (which constructs a multi-scale feature map through a coordinate feature pyramid to simultaneously capture detailed information and global location information), and an attention feature fusion module (which enhances feature representation through an improved attention mechanism).

[0027] Within the framework proposed in this invention, the detection process first starts with the input tampered image. In the beginning, such as Figure 2 As shown. Among them, This indicates that the image has a high degree of detail. ,width It includes three channels: RGB. The goal of this invention is to generate a binary mask with the same dimensions as the input image. ,Right now In this context, white (1) represents the tampered area, and black (0) represents the normal area.

[0028] First, the input image is divided into segments by the Patch Embedding operation. There are several small patches, each with a size of [size missing]. In this operation, the image is divided into fixed-size blocks, and each block is mapped to... The feature vectors form a preliminary feature representation.

[0029] Next, these segmented image patches are processed by a feature extraction module consisting of a visual Transformer backbone network and a content-aware residual module to extract intermediate features. ,in: .

[0030] This feature extraction module fully leverages the advantages of the ViT network in modeling long-range dependencies, while enhancing the ability to extract local edge and texture features through a context-aware residual module.

[0031] Subsequently, the intermediate features were extracted. The data is fed into a multi-scale supervision module, which refines the features at multiple resolutions through a series of convolutional operations, generating enhanced feature maps at different scales. These multi-scale feature maps are then spatially concatenated to form a comprehensive representation that takes into account both local details and global semantics.

[0032] To further enhance feature representation capabilities, the concatenated integrated feature map is fed into the attention-based feature fusion module. This module employs an improved attention mechanism, dynamically adjusting feature importance weights in both channel and spatial dimensions to enhance the representation of salient features and suppress redundant information.

[0033] Finally, the fused feature maps are mapped to a prediction mask by the decoder. This enables precise segmentation and localization of tampered areas. The entire process can not only effectively detect large-scale tampering, but also meticulously capture tiny pixel-level tampered areas, significantly improving detection accuracy and robustness.

[0034] 1. Content-Aware Residual Module. To improve the accuracy of tampered region detection, this invention employs a Visual Transformer (ViT) as the backbone network. Its self-attention mechanism effectively captures the dependencies between long-distance pixels in the image, thereby assisting the model in distinguishing between natural backgrounds and tampered regions. However, the ViT network itself also has some limitations: on the one hand, ViT performs poorly in fine-grained boundary detection and the perception of small-scale tampered regions; on the other hand, ViT lacks a multi-scale feature fusion mechanism, making it difficult to simultaneously handle large-scale tampering and minor modifications. Furthermore, although the window-based attention mechanism alleviates the computational burden of high-resolution images, it also limits the interaction of cross-window information, resulting in incomplete context modeling.

[0035] To address the aforementioned issues, this invention introduces a context-aware residual module after the ViT backbone network, aiming to enhance local feature extraction and context fusion capabilities, thereby improving the model's robustness and generalization in complex tampering scenarios. Specifically, the context-aware residual module combines the advantages of convolutional neural networks (CNNs) in local pattern recognition, employing depthwise separable convolution to efficiently capture intra-channel and inter-channel features. This process reduces computational overhead by decomposing standard convolution operations and promotes effective feature integration. Its expression is: .

[0036] in, Indicates the input feature map, This represents the enhanced feature map after depthwise separable convolution. This process significantly enhances the model's ability to perceive edge and texture features in small-scale tampered regions.

[0037] To further enhance feature representation capabilities, the context-aware residual module introduces a global context fusion mechanism. First, it utilizes adaptive average pooling from... Extract global feature vectors ,Right now: .

[0038] in, This represents the global average feature of each channel. Subsequently, channel attention weights are generated using a two-layer fully connected neural network. The calculation process is as follows: .

[0039] in, and These are the weight matrices of the fully connected layer. This indicates a modified linear unit activation function. This represents the Sigmoid activation function. This step enables dynamic weighting of important channels, improving the discriminative power of the features. Finally, skip connections are used to connect the input features... With enhanced features The features are fused to obtain the final output features. Its expression is: .

[0040] in, As the output of the context-aware residual module, it contains rich feature information after local enhancement and global modeling.

[0041] Through the above design, the context-aware residual module not only improves the model's ability to perceive fine-grained boundaries and small target tampering, but also enhances the modeling effect of multi-scale context, effectively making up for the shortcomings of ViT in high-resolution feature processing, and providing a high-quality feature foundation for subsequent multi-scale supervision and feature fusion.

[0042] 2. Multi-scale Supervision Module. To further enhance the model's ability to detect tampered regions at different scales, this invention designs a multi-scale supervision module. This module aims to fully integrate spatial and semantic features at different levels, thereby balancing the fine-grained boundaries of tampered regions with overall semantic consistency.

[0043] Specifically, the multi-scale supervision module utilizes a series of convolution operations. The feature map generated by the context-aware residual module Perform multi-scale processing to generate feature maps of different resolutions. : ,

[0044] in It is the number of channels in the feature map. and These are the height and width of the feature map, respectively.

[0045] The core of this module lies in jointly modeling spatial and channel information through the CoordAttention module. The CoordAttention module first processes the feature map... Global pooling is performed in both the height and width directions to generate two unidirectional global descriptions. and :

[0046] in, and These represent adaptive average pooling operations along the height and width directions, respectively. , Next, the features from the two directions are concatenated and the channel dimensions are compressed using a 1×1 convolution: {Y}_{i}=Act(\, BN(\, \, {Conv}_{1}(\, \, [\, Perm(\, {f}_{ih})\, \cdot {f}_{iw}]))),\, \, \, i\in \left \{{1,2,,3,4} \right \}

[0047] in, It is a 1×1 convolution. It is a normalization operation. This represents the ReLU activation function, [\, \cdot \, ] This indicates a splicing operation. This indicates a dimensionality transformation operation. Subsequently, the fused features... Features decomposed into height and width directions and Attention weights are then generated through independent convolutions. and :

[0048] in, This represents the Sigmoid function. Finally, the attention weights are... and Acting on the input feature map And generate CoordAttention-enhanced feature maps through residual connections. :

[0049] Through the above process, the multi-scale supervision module explicitly models spatial location information while maintaining the integrity of the global context, enabling the network to focus more accurately on salient regions, thereby improving robustness in complex backgrounds.

[0050] 3. Attention Feature Fusion Module. To fully leverage the complementarity of challenges at different scales and capture both local and global features, an attention mechanism enhanced by multi-scale feature alignment and fusion is used to optimize feature representation. This invention designs an attention feature fusion module. This module first takes feature maps representing different receptive field information from local details to global semantics as input, and then uses bilinear interpolation upsampling to unify this information to the same spatial resolution. The fused feature maps ensure that information at different scales can be directly fused in subsequent operations. Through this unified scale alignment operation, the decoder module can avoid information loss caused by differences in feature size, while simplifying subsequent computations.

[0051] Specifically, this module first models the dependencies between channels by dynamically assigning weights to each channel. (Fusing feature maps) Channel weights are generated using two 1×1 convolutions and a ReLU activation function. : .

[0052] Then applied to the feature map: .

[0053] This operation significantly enhances the ability to model global features by amplifying key channel features and weakening redundant or noisy channels. Subsequently, the modeling of local spatial information is further strengthened. A fused feature map is calculated using average pooling and max pooling operations. Average value in the channel dimension and maximum value The two are concatenated and then fed into a 7×7 convolutional layer to generate spatial weights. : .

[0054] The generated spatial weights Ws are applied to the features after channel processing. : .

[0055] This operation, by explicitly modeling local detail information, is more sensitive to feature extraction from edge regions and complex textures, especially near the boundaries of the manipulated region, which can significantly improve prediction accuracy and robustness. Finally, random deactivation is used to reduce the risk of overfitting, and a 1×1 convolutional layer is used to generate the prediction mask image. : .

[0056] in AttentionFusion, by combining channel and spatial attention, not only addresses the issue of insufficient feature interaction in the decoder but also enhances its ability to perceive detailed regions. The combined effect of the attention mechanism makes the final prediction results more robust to background complexity, boundary blurring, and noise interference.

[0057] 4. Loss Function. To improve the detection accuracy of tampered region boundaries, this invention proposes an edge mask extraction module that combines the dilatational erosion edge algorithm and the Sobel operator. For example... Figure 3 As shown in the generated edge mask, the boundary of the tampered region is usually the part where the tampering features are most obvious, and it is also the key to distinguishing the real region from the tampered region. This method generates a more accurate edge mask by fusing local gradient features and macroscopic boundary information, providing important support for the localization and refinement of the tampered region.

[0058] First, the Sobel operator is used to process the input binary mask image. Processing is performed, and the horizontal gradient is calculated separately. and vertical gradient Using the gradient magnitude formula: .

[0059] Generate gradient edge map To remove noise and preserve significant edge features, for Apply thresholding: .

[0060] in, This is a threshold parameter used to filter out low-intensity edge responses, preserving more important boundary information. This method can effectively capture the detailed edge features of the tampered region. Next, edge features are further extracted through mathematical morphology operations. Specifically, for the mask image... Expand ( ) and corrosion ( Perform the operation and calculate the difference between the two: .

[0061] in, It is The cross-shaped structural element is only present in the first... row and number The pixel value in the column is 1, and the values ​​in the other positions are 0. The value of is approximately equal to the boundary width, used to control the intensity of dilation and erosion operations. Dilation expands the boundaries of bright areas, while erosion shrinks the dark areas of the inverted image; the difference between the two accurately captures the boundary distribution and width features. Finally, the Sobel edge detection results are... With morphological edge detection results Blend to generate the final edge mask : .

[0062] This method cleverly combines the advantages of gradient detection and morphological edge extraction, enabling the generated edge mask to preserve details of the tampered region while improving the coherence and robustness of the boundaries. Through this fusion strategy, the mask not only accurately reflects the edge characteristics of the tampered region but also effectively avoids the noise or detail loss problems that may arise from using a single method.

[0063] Furthermore, this invention introduces a combined loss function that integrates segmentation and edge detection to fully utilize edge information to guide the model optimization process. Specifically, this invention first uses a binary mask image... and the generated edge mask Based on this, the joint loss function Defined as: .

[0064] in, Represents the predicted mask image, symbol This indicates a pointwise multiplication operation, used to mask non-edge regions and emphasize the model's focus on edge regions. Loss term. and Both methods employ the binary cross-entropy loss function; the former primarily optimizes the overall segmentation of the tampered region, while the latter focuses on accurate modeling of the edge region. Hyperparameters By adjusting the relative weights of these two losses, this invention sets... This guides the model to invest more learning capacity in edge regions.

[0065] To demonstrate the effectiveness of this invention, experiments were conducted on publicly available image manipulation datasets CASIAV1, Coverage, and Columbia. The F1 score was used as the evaluation metric for the experiments.

[0066] Performance comparison of different algorithms with the present invention, as follows: Figures 4-6 As shown in the figure, the image tampering localization method based on edge guidance and multi-scale feature fusion proposed in this invention has excellent performance.

[0067] Furthermore, the method of the present invention is not limited to being executed in the chronological order described in the specification, but may also be executed in other chronological orders, in parallel, or independently. Therefore, the execution order of the method described in this specification does not constitute a limitation on the technical scope of the present invention.

[0068] Although the invention has been disclosed above through the description of specific embodiments, it should be understood that all the embodiments and examples described above are exemplary and not restrictive. Those skilled in the art can design various modifications, improvements, or equivalents to the invention within the spirit and scope of the appended claims. These modifications, improvements, or equivalents should also be considered to be included within the protection scope of the invention.

Claims

1. A method and system for detecting image tampering based on edge guidance and multi-scale feature fusion, characterized in that, Includes the following steps: (1) The input image is segmented and feature vectors are embedded to form the initial patch representation of the image; (2) Input the initial patch into a backbone network based on visual Transformer to extract global semantic features; (3) A context-aware residual module is connected after the backbone network. The context-aware residual module includes a depth-separable convolutional structure, a global channel attention module and a skip connection structure to enhance texture perception and context modeling capabilities. (4) A multi-scale feature supervision module is used to model the feature maps output by multiple intermediate layers. The multi-scale supervision module includes a multi-scale downsampling structure and a coordinate attention mechanism to extract spatial and channel information at different scales. (5) Input the multi-scale feature map into the attention feature fusion module, which includes a channel attention branch and a spatial attention branch, and fuse them to obtain a unified attention map for strengthening the response of the tampered region; (6) An edge guidance strategy is introduced during the training phase. The Sobel operator and dilation and erosion operation are used to generate the target edge mask and compare it with the predicted edge to calculate the edge guidance loss. (7) Construct a composite loss function, and combine binary cross-entropy loss, edge guidance loss and Dice loss for training optimization; (8) The fused feature map is restored to a tampered region mask of the same size as the input image by the decoding module, so as to realize the localization and detection of image tampering.

2. The method according to claim 1, characterized in that, The context-aware residual module includes: (1) Depth-separable convolutional units are used to extract local texture features of the input image with a low number of parameters; (2) Global channel attention mechanism, which achieves weighted importance of feature map channels by performing global average pooling, fully connected mapping and Sigmoid activation on each channel; (3) Residual connection path, used to add the module input and output, thereby preserving the original feature information and enhancing the gradient propagation ability of the network.

3. The method according to claim 1, characterized in that, The multi-scale supervision module includes: (1) Feature extraction paths at multiple scales are implemented by using convolution operations with a stride of 2 to construct feature maps at different resolutions; (2) Coordinate attention mechanism, which encodes global information in the horizontal and vertical directions respectively, and generates an attention guidance map by combining the inter-channel dependencies; (3) Feature fusion operation: After fusing the attention guidance maps at various scales, the model is applied to the main feature map to guide the model to focus on the tampered areas at different scales.

4. The method according to claim 1, characterized in that, The attention feature fusion module includes: (1) Channel attention branch: The importance weight of each channel is obtained through global average pooling and fully connected mapping network in the channel dimension; (2) Spatial attention branch, which compresses and weights the fused feature map in the spatial dimension to highlight the location features of potential tampering areas in the image; (3) Weighted fusion mechanism, which is used to jointly apply channel attention and spatial attention to the backbone feature map to obtain an enhanced tamper response map.

5. The method according to claim 1, characterized in that, The edge guidance strategy includes: (1) Apply the Sobel operator to the real mask image to extract edge information and obtain the target edge image; (2) Morphological dilation and erosion are performed on the target edge map to enhance the continuity and salience of the edges; (3) Apply the same Sobel operator to the mask map predicted by the model to extract the predicted edge map; (4) Compare the difference between the predicted edge and the target edge, and calculate the edge guidance loss to guide the model to learn the accurate tampering boundary.

Images