Image tampering detection method and system for multi-granularity mask supervision and storage medium
By using the ConvNeXt architecture and BiFPN for multi-scale feature fusion, combined with the supervision mechanisms of Focal Loss and Dice Loss, the problem of high computational complexity and insufficient adaptability in image tampering detection in existing technologies is solved, and efficient and accurate detection of various tampering methods is achieved.
Patent Information
- Application Number
- CN202511798608.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-02
- Publication Date
- 2026-02-06
AI Technical Summary
Existing image tampering detection technologies have high computational complexity when processing high-resolution images, making it difficult to meet real-time detection requirements. They also lack adaptability to various tampering methods, especially when tampering performance degrades for non-JPEG compressed or uncompressed images.
A multi-granularity mask-supervised image tampering detection method is proposed. The ConvNeXt architecture is used as the backbone network, combined with BiFPN to achieve multi-scale feature fusion, and a joint supervision mechanism of Focal Loss and Dice Loss is adopted to obtain multi-scale enhanced features through feature extraction, fusion and detection layers.
It significantly improves the model's inference speed and generalization performance, enabling image processing in milliseconds and enhancing the detection accuracy and robustness against various tampering techniques, especially strengthening the ability to identify small-scale tampered regions and difficult samples.
Smart Images
Figure CN121482584A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and artificial intelligence, and specifically to a multi-granularity mask-supervised image tampering detection method, system, and storage medium. Background Technology
[0002] With the rapid development of self-media and social networks, information dissemination channels have become increasingly diversified, and the cost of information acquisition has significantly decreased. However, the resulting issue of information authenticity has become increasingly prominent. Due to the widespread public perception that "a picture is worth a thousand words," rumors accompanied by forged images are more easily believed and spread rapidly, with some even having a serious negative impact on social order. In the commercial sector, especially for companies with numerous offline retail outlets, traditional manual inspection methods suffer from significant problems of low efficiency and high cost. While remote review based on image content can effectively reduce costs, the inherent susceptibility of images to tampering, coupled with the inefficiency and instability of manual visual inspection, severely restricts the widespread application of this method.
[0003] Current mainstream image tampering detection and localization technologies mainly include two representative schemes: IML-ViT and Cat-Net. IML-ViT, as the first localization model based on Visual Transformer (ViT), is specifically designed for the high-resolution input, edge supervision, and multi-scale supervision requirements of image tampering localization tasks. Its core innovation lies in adopting a ViT with a window mechanism, replacing global attention with window attention, significantly reducing computational complexity while maintaining high-resolution processing capabilities; it also introduces a Simple Feature Pyramid Network (SFPN) to construct multi-scale feature maps for multi-scale supervision, and innovatively employs a morphology-based edge loss algorithm, strengthening edge feature learning by handling the dilation and erosion operations of the real mask. Cat-Net proposes an end-to-end fully convolutional network architecture. Its key breakthrough lies in achieving the first-ever collaborative analysis of RGB and DCT domain information. By designing a dedicated DCT flow to learn the binary volume representation of quantized DCT coefficients, it effectively captures JPEG compression artifacts, giving the model special robustness to JPEG double compression and demonstrating performance superior to traditional histogram methods in detecting double JPEG compression. While the IML-ViT model reduces some computational burden through its window attention mechanism, it still faces significant computational complexity when processing ultra-high-resolution images. The in-window attention computation and convolution / deconvolution operations in multi-scale feature processing result in high overall computational costs, making it difficult to meet the demands of real-time detection scenarios. Furthermore, this model lacks adaptability to complex tampering scenarios. When images are simultaneously subjected to multiple tampering methods such as stitching, erasure, and generative manipulation, different tampering features interfere with each other, making it difficult for the model to effectively distinguish and integrate features. The Cat-Net approach, on the other hand, has significant application limitations. Its performance heavily relies on the assumption that "the source files of the stitched images are all camera-compressed." When processing tampering of uncompressed or non-JPEG compressed images, the model's performance drops sharply. In addition, its feature extraction dimension is singular, overly focusing on JPEG compression artifacts, lacking the ability to detect uncompressed tampering features such as illumination adjustments and geometric transformations, thus limiting its applicability in diverse tampering scenarios.
[0004] Therefore, there is an urgent need for a general image tampering detection platform to solve the above-mentioned problems in existing technologies, which can efficiently and accurately identify various types of image tampering behaviors, thereby improving detection accuracy and adaptability in specific application scenarios. Summary of the Invention
[0005] The purpose of this invention is to provide a multi-granularity mask-supervised image tampering detection method, system, and storage medium to solve the technical problems of low accuracy and efficiency in image tampering detection in the prior art.
[0006] To achieve the above objectives, embodiments of the present invention provide a multi-granularity mask-supervised image tampering detection method, comprising: Acquire the image to be detected; The image to be detected is input into the feature extraction layer of the image tampering detection model to extract feature maps of different scales; The feature maps of different scales are input into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features. The multi-scale enhanced features are input into the image tampering detection layer of the image tampering detection model to obtain image detection information.
[0007] Optionally, acquiring the image to be detected includes: Obtain the initial image to be detected; The initial image to be detected is subjected to data cleaning, edge extraction, and image filling to obtain the image to be detected.
[0008] Optionally, the feature extraction layer includes a first convolutional layer, a first normalization layer, at least one feature block, and at least one downsampling layer.
[0009] Optionally, the feature block includes a first depthwise convolutional layer, a second normalization layer, a second convolutional layer, a first activation layer, a third convolutional layer, a first channel-wise scaling layer, and a first random depth layer. The second normalization layer, the second convolutional layer, the first activation layer, the third convolutional layer, the first channel-wise scaling layer, and the first random depth layer are sequentially connected in series after the first depthwise convolutional layer. The input end of the first depthwise convolutional layer is residually connected to the output end of the first random depth layer.
[0010] Optionally, the image tampering detection model is obtained by training the feature extraction layer, feature fusion layer, and image tampering detection layer on real sample images and tampered sample images. The loss function for training the image tampering detection model is given by formula (1). (1) in, For the total loss function, For the segmentation loss function, For edge loss function, For visual loss function, For classification loss function, for The weighting coefficients.
[0011] Optionally, the feature maps at different scales are input into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features, including: The feature maps of different scales are processed to have a unified channel dimension. Expand the pyramid hierarchy through downsampling; Top-down feature fusion injects rich semantic information from high-level features into low-level features, while bottom-up feature fusion transfers details and location information from low-level features to high-level features to obtain enhanced feature maps.
[0012] Optionally, a top-down feature fusion approach can be adopted to inject rich semantic information from high-level features into low-level features, including: Upsample the (i+1)th feature map to obtain the (i+1)th upsampled feature; Depth-separable convolution is used to refine the (i+1)th upsampled feature and the ith feature map to obtain the ith preliminary enhanced feature map; Bottom-up feature fusion is employed to pass details and location information from lower-level features to higher-level features, including: The (i-1)th feature map is downsampled to obtain the (i-1)th downsampled feature. The (i-1)th downsampled feature is weighted and fused with the i-th preliminary enhanced feature map and the i-th feature map to obtain the i-th enhanced feature map.
[0013] Optionally, the multi-scale enhanced features are input into the image tampering detection layer of the image tampering detection model to obtain image detection information, including: An initial segmentation map is generated based on the fused multi-scale enhanced features; The initial segmentation image is upsampled to the size of the original input image using bilinear interpolation to obtain pixel-level segmentation output; Intermediate layer feature maps are selected from the feature maps at different scales, and a classification head is used to detect the intermediate layer feature maps to obtain image-level classification output; The final image detection result is obtained based on the pixel-level segmentation output and the image-level classification output.
[0014] On the other hand, the present invention also provides a multi-granularity mask-supervised image tampering detection system, the detection system comprising: The image acquisition module is used to acquire the image to be detected. The feature extraction module is used to extract features from the image to be detected in order to obtain feature maps at different scales; The feature fusion module is used to fuse feature maps of different scales to obtain multi-scale enhanced feature maps. The image tampering detection module is used to detect the multi-scale enhanced feature map to obtain detection results; A processor is connected to the image acquisition module, feature extraction module, feature fusion module, and image tampering detection module, and the processor is configured to perform any of the methods described above.
[0015] In another aspect, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement any of the methods described above.
[0016] The beneficial effects of this invention are: This invention employs the computationally efficient ConvNeXt architecture as the backbone network, significantly improving model inference speed. It achieves millisecond-level processing on standard hardware, significantly outperforming existing ViT-based solutions and providing feasibility for real-time image authentication applications. Secondly, this invention enhances the model's generalization performance. Even after training or fine-tuning on a small training set, it maintains stable detection capabilities against various unknown tampering methods, effectively overcoming the over-reliance of existing methods on specific training data. In terms of structural design, multi-scale feature fusion is achieved through a Feature Pyramid Network (FPN), combined with early high-resolution feature maps introduced during the decoding stage. This allows the model to simultaneously capture macroscopic semantic information and microscopic edge features of the tampered region, thereby significantly improving edge localization accuracy while maintaining the integrity of the tampered region. Furthermore, this invention innovatively employs a joint supervision mechanism of Focal Loss and Dice Loss, effectively solving the common problem of positive and negative sample imbalance in tampering detection. In particular, it enhances the sensitivity to small-scale tampered regions and difficult samples, comprehensively improving the model's robustness and accuracy under different tampering scenarios.
[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 A flowchart of a multi-granularity mask-supervised image tampering detection method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an image tampering detection model according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the feature extraction layer structure according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a feature block structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the process of feature fusion in a feature fusion layer of an image tampering detection model based on an embodiment of the present invention, where feature maps of different scales are input into the model. Figure 6 This is a schematic diagram of the structure of a BiFPN according to an embodiment of the present invention; Figure 7 A flowchart of an image tampering detection method according to an embodiment of the present invention; Figure 8 A schematic diagram of the overall architecture for model training according to an embodiment of the present invention; Figure 9 This is a schematic diagram illustrating the detection effect of an image tampering detection model according to an embodiment of the present invention; Figure 10 This is a schematic diagram illustrating the detection performance of the SOTA model according to one embodiment of the present invention. Detailed Implementation
[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.
[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.
[0021] like Figure 1 The diagram shows a flowchart of a multi-granularity mask-supervised image tampering detection method according to an embodiment of the present invention. Figure 1 The detection method may include the following steps: In step S10, the image to be detected is acquired; In step S11, the image to be detected is input into the feature extraction layer of the image tampering detection model to extract feature maps at different scales; In step S12, feature maps of different scales are input into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features; In step S13, the multi-scale enhanced features are input into the image tampering detection layer in the image tampering detection model to obtain image detection information.
[0022] In such Figure 1In the multi-granularity mask-supervised image tampering detection method shown, step S10 is used to acquire the image to be detected. In this embodiment, to ensure the accuracy of the image tampering detection results, the image to be detected is usually preprocessed. Further, the specific method for image preprocessing in step S10 can be of various forms known to those skilled in the art. In one example of the present invention, step S10 may include data cleaning, edge extraction, and image filling of the image to be detected. Specifically, data cleaning may involve filtering the image to remove images of excessively low quality. Edge extraction may involve extracting edges from the ground truth according to a set pixel value, which is used for edge loss calculation. Image filling may involve filling all images to 1024*1024. Further, in this example, the image data format is JPEG or TIFF image format, and the label data is PNG format.
[0023] After obtaining the preprocessed image to be detected, it is input into the image tampering detection model for detection. In this embodiment, the image tampering detection model can use ConvNeXt (pure convolutional neural network) as the backbone, which outperforms Swin-Transformer while significantly reducing complexity and computation. BiFPN is also added for multi-scale supervision, improving both the model's recognition accuracy and generalization performance. DiceLoss+FocalLoss allows the model to focus more on improving the difficult terms, while EdgeLoss enables the model to output more accurate edges and accelerates convergence. Specifically, in this example, the image tampering detection model can be as follows: Figure 2 The diagram shows a feature extraction layer, a feature fusion layer, and an image tampering detection layer.
[0024] Step S11 is used to input the image to be detected into the feature extraction layer of the image tampering detection model, extracting useful information from the input image from shallow to deep, i.e., feature maps at different scales. In this example, the feature extraction layer may include a first convolutional layer, a first normalization layer, at least one feature block, and at least one downsampling layer.
[0025] In one embodiment of the present invention, the feature extraction layer may be as follows: Figure 3 As shown, the feature pyramid consists of a first convolutional layer, a first normalization layer, a first feature block, a first downsampling layer, a second feature block, a second downsampling layer, a third feature block, a third downsampling layer, and a fourth feature block. The four feature blocks and three downsampling layers alternate to form a feature pyramid. At each fixed resolution, multiple identical blocks are used for deep feature extraction. The downsampling layers halve the size of the feature map while doubling the number of channels, reducing computation while increasing the richness and abstraction of the features.
[0026] Furthermore, in this example, the feature block (ConvNeXt Block) can be as follows: Figure 4 As shown, the network includes a first depthwise convolutional layer, a second normalization layer, a second convolutional layer, a first activation layer, a third convolutional layer, a first channel-wise scaling layer, and a first random depthwise layer. The second normalization layer, the second convolutional layer, the first activation layer, the third convolutional layer, the first channel-wise scaling layer, and the first random depthwise layer are sequentially connected in series after the first depthwise convolutional layer. The input of the first depthwise convolutional layer is residually connected to the output of the first random depthwise layer. The first depthwise convolutional layer uses a 7x7 depthwise separable convolution, employing a large kernel to capture contextual information within a window. It is then normalized, and then enlarged and activated using a 1x1 convolution. The residual connection between the input of the first depthwise convolutional layer and the output of the first random depthwise layer prevents overfitting and ensures smooth gradient flow, facilitating the training of deep networks.
[0027] In one specific embodiment, the backbone network (ConvNeXt-Base) variant parameters are depths=[3,3,27,3], dims=[128,256,512,1024]. The structural composition and corresponding description are shown in Table 1. Table 1 Backbone Network Structure Description
[0028] The backbone network outputs the following feature sets: backbone_features=[C2,C3,C4,C5], which is the complete feature pyramid generated by the backbone network, containing features at all levels from high-resolution details to low-resolution semantics. c2_skip_feature=C2([B,128,256,256]), which is the highest resolution feature map, containing the richest spatial detail information (such as edges and textures).
[0029] bifpn_input_features=[C3,C4,C5]([B,256,128,128],[B,512,64,64],[B,1024,32,32]), which are feature maps of different scales used for the input feature fusion layer.
[0030] Step S12 is used to input feature maps of different scales into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features. In this embodiment, the specific method for feature fusion of feature maps of different scales in step S12 can be various forms known to those skilled in the art. In one example of the present invention, step S12 can be to use a BiFPN multi-scale supervision module to perform bidirectional fusion of features from multiple different convolutional stages, enabling the model to learn features at more scales. Step S12 may include, for example... Figure 5 The steps shown are described in this. Figure 5 In this context, step S12 may include: In step S20, feature maps of different scales are processed to unify the channel dimension. In step S21, the pyramid levels are expanded by downsampling; In step S22, top-down feature fusion is used to inject rich semantic information of high-level features into low-level features, and bottom-up feature fusion is used to pass the details and location information of low-level features to high-level features to obtain an enhanced feature map.
[0031] In such Figure 5 In the method shown, step S20 is used to unify the channel dimension, providing a unified feature representation space for subsequent feature fusion and ensuring that features at different levels can be operated on element-wise. In this example, feature maps of different scales can be unified to 256 channels. Specifically, the input is: bifpn_input_features(C3,C4,C5fromConvNeXt-Base), fpn_channels=256. BiFPN first unifies the number of channels of C3, C4, and C5 to fpn_channels(256) through 1x1 convolutions (p_input_convs): C3([B,256,128,128])->Conv(256,256)->P3_in([B,256,128,128]); C4([B,512,64,64])->Conv(512,256)->P4_in([B,256,64,64]); C5([B,1024,32,32])->Conv(1024,256)->P5_in([B,256,32,32]); The pyramid levels are downsampled and expanded in step S21. In this example, the structure of BiFPN can be as follows: Figure 6 As shown, by using convolutional downsampling with a stride of 2, deeper pyramid levels P6 and P7 are constructed, providing richer scale coverage and enhancing adaptability to multi-scale targets.
[0032] Step S22 is used to process these feature maps of different scales through top-down and bottom-up approaches and weighted feature fusion. Specifically, in this example, top-down feature fusion is used to inject rich semantic information from high-level features into low-level features, including: Upsample the (i+1)th feature map to obtain the (i+1)th upsampled feature; Depth-separable convolution is used to refine the (i+1)th upsampled feature and the ith feature map to obtain the ith preliminary enhanced feature map; Bottom-up feature fusion is employed to pass details and location information from lower-level features to higher-level features, including: The (i-1)th feature map is downsampled to obtain the (i-1)th downsampled feature. The (i-1)th downsampled feature is weighted and fused with the ith preliminary enhanced feature map and the ith feature map to obtain the ith enhanced feature map.
[0033] In one specific embodiment of the invention, top-down feature fusion, for example, upsamples the higher-level, more semantically informative P5 features to match the size of P4. The upsampled P5 is then weighted and fused with the original P4 input to generate a preliminarily enhanced P4 feature. Bottom-up feature fusion, for example, downsamples the lower-level, more detailed P3 features to match the size of P4. The preliminarily enhanced P4 feature generated in the top-down feature fusion step is weighted and fused again with the downsampled P3 feature and the original P4 from another path. Finally, a convolutional block is used for fine-tuning to output the final enhanced P4_out. That is, the generation of P4_out simultaneously incorporates semantics from higher levels (P5), details from lower levels (P3), and the original information from the current layer (P4).
[0034] In this example, the output multi-scale augmented features are fpn_output_features = [P3_out, P4_out, P5_out, P6_out, P7_out], and each feature map has fpn_channels (256) channels. P3_out: [B, 256, 128, 128], P4_out: [B, 256, 64, 64], P5_out: [B, 256, 32, 32], P6_out: [B, 256, 16, 16], P7_out: [B, 256, 8, 8].
[0035] Step S13 is used to input multi-scale enhanced features into the image tampering detection layer in the image tampering detection model to obtain image detection information. In this embodiment, the specific method for image tampering detection in step S13 can be of various forms known to those skilled in the art. In one example of the present invention, step S13 may include, for example... Figure 7 The steps shown are described in this. Figure 7 In this context, step S13 may include: In step S30, an initial segmentation map is generated based on the fused multi-scale enhanced features; In step S31, the initial segmentation map is upsampled to the size of the original input image using bilinear interpolation to obtain pixel-level segmentation output; In step S32, intermediate layer feature maps are selected from feature maps of different scales, and a classification head is used to detect the intermediate layer feature maps to obtain image-level classification output; In step S33, the final image detection result is obtained based on the pixel-level segmentation output and the image-level classification output.
[0036] In such Figure 7 In the method shown, step S30 is used to generate an initial segmentation map. In this example, the features output by BiFPN, which contain rich multi-scale contextual information, can be integrated first. Then, the integrated features, which are rich in semantics but have low resolution, are fused with the highest-resolution detail features extracted by the backbone network to restore accurate spatial boundaries. Based on the best fused features, a preliminary segmentation map is generated.
[0037] In a specific embodiment of the present invention, the input is: bifpn_features_list (i.e., fpn_output_features), c2_feature (i.e., C2: [B, 128, 256, 256]). c2_feature is the highest resolution detail feature extracted from the backbone network. The decoder embedding dimension is set to 256, i.e., decoder_embed_dim = 256, and batch normalization is used, i.e., norm_type = "BN" (default). The detailed architecture description of the segmentation decoding head is shown in Table 2: Table 2. Architecture description of the segmentation decoder head
[0038] A 1x1 convolution kernel maps 256 channels of features to one channel. The 1x1 convolution acts as a pixel-wise classifier, outputting a raw score (logit) for each location indicating whether it is a tampered region. The output logits_low_res is a low-resolution (256x256) prediction map, i.e., the initial segmentation map.
[0039] Step S31 is used to upsample the initial segmentation map to the size of the original input image using bilinear interpolation to obtain pixel-level segmentation output. Specifically, in this example, bilinear interpolation can be used to upsample logits_low_res from 256x256 to 1024x1024. The output mask_pred_logits has a size of [B, 1, 1024, 1024]. This is the pixel-level segmentation output of the model, where each pixel value represents the confidence that the point is a tampered region. The structure description of the segmentation output upsampling is shown in Table 3: Table 3 Description of the split output upsampling structure
[0040] Step S32 is used to select intermediate layer feature maps from feature maps of different scales, and use the classification head to detect the intermediate layer feature maps to obtain image-level classification output, providing a global tampering judgment at the image level. This complements the segmentation task. Sometimes the tampered area in the image is very small, and the classification head can serve as a robust backup judgment. Specifically, in this example, the input can be selected from the C4 feature [B,512,64,64] of the backbone network, because it is in the intermediate layer and balances details and semantic information. That is, input: features_for_clf_head, the default selection is backbone_features[2] (i.e., C4:[B,512,64,64]). classification_head_input_dim=512. The structure description of the classification head is shown in Table 4: Table 4. Structural Description of the Classification Header
[0041] Global average pooling is used to average the entire feature map of each channel, resulting in a 512-dimensional vector. This compresses spatial information into a global descriptor, allowing the network to focus solely on "what is present" rather than the location of features. A fully connected layer maps this 512-dimensional vector to a scalar value through two linear layers (with ReLU and Dropout in between). The final output, `tamper_logits`, has dimensions [B, 1]. This is the model's image-level classification output, representing the confidence level that the entire image has been tampered with.
[0042] In step S33, the final image detection result is obtained based on the pixel-level segmentation output and the image-level classification output. Specifically, in this example, the output results include segmentation mask prediction and image-level classification prediction. The segmentation mask prediction (logits) is `mask_pred_logits`, with a size of [batch_size, 1, 1024, 1024], representing one predicted value for each pixel (e.g., the logit value of the tampered region). The image-level classification prediction (logits) is `tamper_logits`, with a size of [batch_size, 1]. This represents the logit value indicating whether the entire image has been tampered with.
[0043] The image tampering detection model described above is trained using real and tampered sample images, focusing on feature extraction, feature fusion, and image tampering detection layers. In this example, the overall training architecture of the model can be as follows: Figure 8 As shown. In this example, the data sources used for training can include (1) CASIAV2.0 provided by the Institute of Automation, Chinese Academy of Sciences: 7491 real images and 5123 tampered images, with two tampering methods: copy-move and splice. (2) CASIAV1.0 provided by the Institute of Automation, Chinese Academy of Sciences: 800 real images and 921 tampered images, mainly for splice operation. (3) Liqun display photos provided by China Tobacco: a total of 88305 unlabeled photos, some of which were manually tampered with and used to test the model performance. The data sources are preprocessed and stored in step S10. The image data is in jpeg or tif image format, the label data is in png format, and the dataset is stored in the file system.
[0044] In this example, the loss function used to train the model includes, but is not limited to, segmentation loss, edge loss, and visual loss. In a specific embodiment of the present invention, the loss function used to train the image tampering detection model can be as shown in formula (1): (1) in, For the total loss function, For the segmentation loss function, For edge loss function, For visual loss function, For classification loss function, for The weighting coefficients.
[0045] In this example, the model training strategy could be to use the CASIAV2.0 dataset, with images of size 1024*1024, a batch size of 2, and 8 batches to update parameters; use the AdamW optimizer with a learning rate of 1e-5; use LayerNorm; and use gradient clipping with a limit of 1.0.
[0046] There are many evaluation metrics in the field of image tampering localization, such as AUC, IoU, and F1. However, AUC often reflects an overly optimistic metric and is difficult to accurately measure the model's localization accuracy. IoU, on the other hand, cannot effectively handle the extreme imbalance between negative and positive samples, which is particularly common in the IMDL dataset because the tampered area in the image is usually very small. Therefore, in this example, the F1 score is used as the evaluation metric for image tampering detection. Specifically, in this example, the image tampering detection model in this embodiment of the invention is evaluated using the CASIAV1.0 dataset, and the average pixel-level F1 score is 0.781 (the detection effect is as follows). Figure 9 As shown), the SOTA model's F1 score is 0.721 (detection performance is as shown). Figure 10 (As shown). Therefore, it can be seen that the image tampering detection model in this embodiment of the invention exhibits better robustness, sharper edges, and more accurate range compared to other models across different datasets in actual testing.
[0047] On the other hand, the present invention also provides a multi-granularity mask-supervised image tampering detection system, the detection system comprising a target image acquisition module, a feature extraction module, a feature fusion module, an image tampering detection module, and a processor. The target image acquisition module is used to acquire the target image. The feature extraction module is used to extract features from the target image to obtain feature maps at different scales. The feature fusion module is used to fuse the feature maps at different scales to obtain multi-scale enhanced feature maps. The image tampering detection module is used to detect the multi-scale enhanced feature maps to obtain detection results. The processor is connected to the target image acquisition module, the feature extraction module, the feature fusion module, and the image tampering detection module, and the processor is configured to execute any of the methods described above.
[0048] In another aspect, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement any of the methods described above.
[0049] The beneficial effects of this invention are: This invention employs the computationally efficient ConvNeXt architecture as the backbone network, significantly improving model inference speed. It achieves millisecond-level processing on standard hardware, significantly outperforming existing ViT-based solutions and providing feasibility for real-time image authentication applications. Secondly, this invention enhances the model's generalization performance. Even after training or fine-tuning on a small training set, it maintains stable detection capabilities against various unknown tampering methods, effectively overcoming the over-reliance of existing methods on specific training data. In terms of structural design, multi-scale feature fusion is achieved through a Feature Pyramid Network (FPN), combined with early high-resolution feature maps introduced during the decoding stage. This allows the model to simultaneously capture macroscopic semantic information and microscopic edge features of the tampered region, thereby significantly improving edge localization accuracy while maintaining the integrity of the tampered region. Furthermore, this invention innovatively employs a joint supervision mechanism of Focal Loss and Dice Loss, effectively solving the common problem of positive and negative sample imbalance in tampering detection. In particular, it enhances the sensitivity to small-scale tampered regions and difficult samples, comprehensively improving the model's robustness and accuracy under different tampering scenarios.
[0050] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0051] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0053] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0054] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0055] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0056] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0057] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0058] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A multi-granularity mask-supervised image tampering detection method, characterized in that, The detection method includes: Acquire the image to be detected; The image to be detected is input into the feature extraction layer of the image tampering detection model to extract feature maps of different scales; The feature maps of different scales are input into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features. The multi-scale enhanced features are input into the image tampering detection layer of the image tampering detection model to obtain image detection information.
2. The detection method according to claim 1, characterized in that, Acquiring the image to be detected includes: Obtain the initial image to be detected; The initial image to be detected is subjected to data cleaning, edge extraction, and image filling to obtain the image to be detected.
3. The detection method according to claim 1, characterized in that, The feature extraction layer includes a first convolutional layer, a first normalization layer, at least one feature block, and at least one downsampling layer.
4. The detection method according to claim 3, characterized in that, The feature block includes a first depthwise convolutional layer, a second normalization layer, a second convolutional layer, a first activation layer, a third convolutional layer, a first channel-wise scaling layer, and a first random depth layer. The second normalization layer, the second convolutional layer, the first activation layer, the third convolutional layer, the first channel-wise scaling layer, and the first random depth layer are sequentially connected in series after the first depthwise convolutional layer. The input end of the first depthwise convolutional layer is residually connected to the output end of the first random depth layer.
5. The detection method according to claim 1, characterized in that, The image tampering detection model is obtained by training the feature extraction layer, feature fusion layer, and image tampering detection layer on real sample images and tampered sample images. The loss function for training the image tampering detection model is given by formula (1). ，（1） in, For the total loss function, For the segmentation loss function, For edge loss function, For visual loss function, For classification loss function, for The weighting coefficients.
6. The detection method according to claim 1, characterized in that, The feature maps of different scales are input into the feature fusion layer of the image tampering detection model for feature fusion processing to obtain multi-scale enhanced features, including: The feature maps of different scales are processed to have a unified channel dimension. Expand the pyramid hierarchy through downsampling; Top-down feature fusion injects rich semantic information from high-level features into low-level features, while bottom-up feature fusion transfers details and location information from low-level features to high-level features to obtain enhanced feature maps.
7. The detection method according to claim 6, characterized in that, A top-down feature fusion approach is used to inject rich semantic information from high-level features into low-level features, including: Upsample the (i+1)th feature map to obtain the (i+1)th upsampled feature; Depth-separable convolution is used to refine the (i+1)th upsampled feature and the ith feature map to obtain the ith preliminary enhanced feature map; Bottom-up feature fusion is employed to pass details and location information from lower-level features to higher-level features, including: The (i-1)th feature map is downsampled to obtain the (i-1)th downsampled feature. The (i-1)th downsampled feature is weighted and fused with the i-th preliminary enhanced feature map and the i-th feature map to obtain the i-th enhanced feature map.
8. The detection method according to claim 1, characterized in that, The multi-scale enhanced features are input into the image tampering detection layer of the image tampering detection model to obtain image detection information, including: An initial segmentation map is generated based on the fused multi-scale enhanced features; The initial segmentation image is upsampled to the size of the original input image using bilinear interpolation to obtain pixel-level segmentation output; Intermediate layer feature maps are selected from the feature maps at different scales, and a classification head is used to detect the intermediate layer feature maps to obtain image-level classification output; The final image detection result is obtained based on the pixel-level segmentation output and the image-level classification output.
9. A multi-granularity mask-supervised image tampering detection system, characterized in that, The detection system includes: The image acquisition module is used to acquire the image to be detected. The feature extraction module is used to extract features from the image to be detected in order to obtain feature maps at different scales; The feature fusion module is used to fuse feature maps of different scales to obtain multi-scale enhanced feature maps. The image tampering detection module is used to detect the multi-scale enhanced feature map to obtain detection results; A processor is connected to the image acquisition module, feature extraction module, feature fusion module, and image tampering detection module, and the processor is configured to perform the method as described in any one of claims 1 to 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 8.