Image tampering detection method fusing noise residual error and compression artifact, and program product

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image tampering detection method fusing noise residual errors and compression artifacts and a program product. According to the scheme, noise residual features of compression artifact features of an original image are extracted through an error level analysis technology and an airspace rich model, and then an image tampering detection model with quality adaptability is constructed by combining the two types of features. The network model firstly extracts tampering features from image compression features and steganography noise dimensions through a convolutional layer in an ELA branch and an SRM branch, and then performs multi-scale feature coding through a lightweight MobileNetV2 network; feature interaction enhancement is realized in combination with an SECA attention mechanism; and finally, generating a pixel-level binary mask representing the tampered area through a feature fusion strategy. According to the scheme, the compression characteristic and the noise characteristic of the image can be fully utilized, and the detection precision of the model on the low-quality image and the robustness in a multi-quality scene are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image tampering detection method fusing noise residual and compression artifacts, a corresponding computer program product, a storage medium and an image tampering recognition device. BACKGROUND

[0002] With the development of image editing technology, tampering images becomes easier and easier, and tampering means is increasingly complex and diverse; including simple copy-paste, splicing and advanced filter processing and AI face changing. The tampered image is often difficult to distinguish from the real image in vision. This phenomenon has serious consequences in the fields of news media, judicial evidence, e-commerce and the like. Therefore, accurately detecting and positioning the tampered area in the image is of vital importance to maintaining the credibility of digital images and social stability.

[0003] In recent years, deep learning technology has made significant progress in the field of image tampering detection and has become a research hotspot in this field. However, most of the existing deep learning-based image tampering detection methods assume that the input image is of high quality, ignoring the variability of image quality in actual applications. Studies have shown that image quality degradation can significantly reduce the performance of tampering detection models, especially in low-resolution and high-compression scenarios. On the one hand, noise and blur in low-quality images can interfere with the extraction of tampering traces, leading to model false positives or false negatives, and reducing the robustness of the model in feature extraction. For example, JPEG compression can destroy the edge continuity of the tampered area. On the other hand, models trained on a single quality dataset are difficult to adapt to other quality scenarios, showing significant performance degradation, i.e., the cross-domain generalization ability of the model is reduced. For example, a model trained on a high-quality image dataset may have a significant reduction in detection accuracy when processing low-quality images. SUMMARY

[0004] In order to solve the problem that the detection accuracy of the existing image tampering detection method on the compressed low-quality image is significantly reduced, the application provides an image tampering detection method fusing noise residual and compression artifacts, and a corresponding computer program product, a storage medium and an image tampering recognition device.

[0005] The application adopts the following technical solutions:

[0006] An image tampering detection method fusing noise residual and compression artifacts, comprising:

[0007] An image tamper detection model for identifying tampered regions in an input image and outputting a corresponding region binary mask is constructed. In the image tamper detection model, first, an input image is obtained by an error level analysis tool and a spatial rich model processing to obtain an ELA image and an SRM image. Next, the ELA image is sequentially subjected to a convolution layer with a fixed 2 weight kernel of 5x5 and a MobileNetV2 network for feature extraction to obtain a first feature map F1. The SRM image is sequentially subjected to a convolution layer with a learnable weight kernel of 3x3 and a MobileNetV2 network for feature extraction to obtain a second feature map F2; F1 and F2 are concatenated to obtain a comprehensive feature map F3; F3 is subjected to a convolution layer with a 1x1 kernel for feature compression and fusion, and then subjected to a hybrid attention mechanism SECA for feature refinement, and the refined feature is subjected to a 1x1 convolution layer and a bilinear interpolation-based upsampling module to obtain a predicted binary mask.

[0008] The dataset is divided into a training set and a test set according to 9:1, and a binary classification cross-entropy loss L BCE The image tamper detection model is trained and tested as a loss function, the model parameters with the best performance after testing are reserved, and are used for tamper region identification of an input image.

[0009] As a further improvement of the present application, the error level analysis tool is used for secondary compression of the image with a fixed quality factor, and the residual error between the original image and the compressed image is calculated to form an ELA image. The ELA image is a heat map that marks different regions whether they have been tampered with.

[0010] The spatial rich model contains 30 filters that can generate noise residuals based on the center pixel; the spatial rich model is used to generate an SRM image representing the noise residual distribution of the input image through the 30 filters.

[0011] As a further improvement of the present application, in the MobileNetV2 network, the input features are sequentially processed by a 1x1 nonlinear convolution layer, a 3x3 depth separable convolution layer, and a 1x1 linear convolution layer, and the feature map obtained is connected with the original input feature to obtain the final output feature. The activation function of the nonlinear convolution layer is the ReLU function, and the activation function of the linear convolution layer is Linear.

[0012] As a further improvement of the present application, the depth separable convolution layer of the MobileNetV2 network contains 7 expansion bottleneck modules, and the output channel numbers of the 7 expansion bottleneck modules are 16, 24, 32, 64, 96, 160 and 320, respectively.

[0013] As a further improvement of the present application, in the mixed attention mechanism SECA, the input features are first obtained by global average pooling to obtain the global spatial information of the feature map, and then 1D convolution operation is used to enhance the different scale information of the feature map; then the QKV mapping in the self-attention mechanism calculates the spatial attention weight and weights the aggregation features to obtain the refined fusion features.

[0014] As a further improvement of the present application, the feature refinement process of the mixed attention mechanism SECA includes:

[0015] ⅰ、For the given input feature Figure X , Through adaptive average pooling AdaptiveAvgPool2d, the spatial dimension is compressed to obtain the channel statistics z,

[0016] z=AdaptiveAvgPool2d(X),

[0017] Where B represents the batch size, C represents the number of channels, H represents the image height, and W represents the image width.

[0018] ⅱ、Through one-dimensional convolution Conv1D with kernel size k, cross-channel interaction is captured, and channel weight α is generated through Sigmoid,

[0019] α=σ(Conv1D(Y))

[0020] Y=transpose(squeeze(z)),

[0021] In the above formula, σ is the Sigmoid function; Y represents the result of reshaping each channel statistics z by dimension; squeeze(·) represents the dimension increasing operation, and transpose(·) represents the dimension exchange operation.

[0022] ⅲ、Using the channel weight to perform channel weighting on the input feature map, the output feature E of the ECA module is obtained:

[0023]

[0024] In the above formula, represents channel-by-channel multiplication.

[0025] ⅳ、Flatten E in the spatial dimension to sequence X ECA , And generate query vector Q, key vector K and value vector V through linear transformation:

[0026]

[0027] In the above formula, Wq denotes the query vector weight; W k denotes the key vector weight; W v denotes the value vector weight.

[0028] ⅴ、Through Softmax normalization, a spatial correlation weight A is obtained,

[0029] A = Softmax(QK T ).

[0030] ⅵ, the value vector is weighted and aggregated using the spatial correlation weight, and the spatial structure is recovered, to obtain the output feature X of the SECA module Attn ,

[0031] X Attn = Reshape(AV).

[0032] As a further improvement of the application, the expression of the binary cross-entropy loss L BCE is:

[0033]

[0034] In the above formula, y i denotes the true label of the i-th pixel; y i = 0 indicates no tampering, and y i = 1 indicates that there is tampering; p i denotes the probability that the i-th pixel predicted by the model belongs to the tampered region, p i ∈ [0, 1]; N denotes the total number of pixels of the image.

[0035] The application also includes a computer program product comprising a computer program which, when executed by a processor, creates a trained image tampering detection model in the image tampering detection method of fusing noise residuals and compression artifacts as described above. The image tampering detection model is used to identify tampered regions contained in an input image and generate a corresponding binary mask.

[0036] The application also includes a storage medium having stored therein a computer program which, when executed by a processor, creates a trained image tampering detection model in the image tampering detection method of fusing noise residuals and compression artifacts as described above. The image tampering detection model is used to identify tampered regions contained in an input image and generate a corresponding binary mask.

[0037] The application also comprises an image tampering recognition device, which comprises a memory, a processor, and a computer program stored in the memory and running in the processor, wherein the processor executes the computer program to create the trained image tampering detection model in the image tampering detection method fusing noise residuals and compression artifacts as described above. The image tampering detection model is used to identify the tampered region contained in the input image and generate the corresponding binary mask.

[0038] The technical scheme provided by the application has the following beneficial effects:

[0039] The application provides an image tampering detection method fusing noise residuals and compression artifacts. The scheme extracts the compression characteristics and noise characteristics of the image through a double-branch structure respectively, performs efficient feature extraction using a MobileNetV2 network, and enhances the feature fusion effect through an SECA attention module, thereby significantly improving the detection accuracy and robustness of the model in complex scenes. The scheme overcomes the limitations of traditional image tampering detection schemes in low-quality images and multi-quality scenes.

[0040] In particular, in the feature extraction stage, the scheme performs error level analysis (ELA) and spatial rich model (SRM) processing on the original image through the two branches respectively, obtains a noise residual map of the ELA heat map, and combines the feature information of the two branches to improve the adaptability and detection accuracy of the model for different quality images. ELA can effectively detect tampering traces in JPEG format images by comparing the differences between images at different compression qualities and highlighting the feature changes of the tampered region. SRM focuses on the noise distribution in the image and extracts the noise residuals of the image using a high-pass filter. These residuals will change significantly when the image is tampered with. By fusing these two features, the model can fully utilize the compression characteristics and noise characteristics of the image to improve the recognition ability of the tampered region, especially in low-quality image scenarios.

[0041] In the feature fusion stage, the application first realizes deep interaction of multi-modal features through channel splicing and attention enhancement mechanism. The SRM branch extracts image noise residual features through fixed convolution, focusing on local statistical anomalies that may exist in the tampered region, such as JPEG compression artifacts, edge discontinuity of copy-paste operations, etc. The ELA branch captures the global distribution shift caused by the difference in image compression quality through learnable convolution; such as block effect, chroma distortion caused by re-compression, etc. After encoding by MobileNetV2, channel dimension splicing is adopted, and finally 1x1 convolution is used for cross-modal feature reconstruction. This operation not only realizes feature dimension compression but also establishes cross-branch channel-level association, enabling nonlinear interaction between noise features and compression artifact features at spatially corresponding positions, thereby enhancing the joint representation ability of tampering traces. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and together with the description serve to explain the application, and, together with the description, to specify the application without limitation. In the drawings:

[0043] Figure 1 The step flow chart of the image tampering detection method for fusion noise residual and compression artifacts provided in Embodiment 1 of the present application.

[0044] Figure 2 The network architecture diagram of the image tampering detection model constructed in Embodiment 1 of the present application.

[0045] Figure 3 The comparison diagram of the original image and the ELA heat map of the un-tampered and tampered images in Embodiment 1 of the present application.

[0046] Figure 4 The working principle diagram of the 30 high-pass filters used in the spatial rich model in Embodiment 1 of the present application.

[0047] Figure 5 The comparison diagram of the noise residual image obtained by processing through the spatial rich model and the original image in Embodiment 1 of the present application.

[0048] Figure 6 The architecture comparison diagram of the MobileNetV2 and the traditional MobileNetV1 module used in Embodiment 1 of the present application.

[0049] Figure 7 The principle diagram of the hybrid attention mechanism SECA used in Embodiment 1 of the present application.

[0050] Figure 8 The visualization image of the tampering detection results of the present application and the control group scheme on different data sets in the test experiment.

[0051] Figure 9 The F1 score change curve of the present application and the control group scheme on the CASIAv1 data set in the test experiment with the image compression quality factor.

[0052] Figure 10 The F1 score change curve of the present application and the control group scheme on the Coverage data set in the test experiment with the image compression quality factor.

[0053] Figure 11 The F1 score change curve of the present application and the control group scheme on the Columbia data set in the test experiment with the image compression quality factor. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and not to limit the present application.

[0055] Example 1

[0056] Most of the existing deep learning-based image tamper detection methods assume that the input image is of high quality, ignoring the variability of image quality in actual applications, and thus the performance is severely degraded when the forged image is a JPEG compressed image. To solve this problem, the present embodiment proposes an image tamper detection model that fuses noise residual and compression artifact dual-modal feature enhancement. It fuses the compression artifact features of error level analysis (ELA) and the noise residual features of spatial rich model (SRM) to construct a dual-branch detection network with quality adaptability. The scheme first designs an ELA-SRM dual-flow preprocessing layer to extract tamper features from the image compression characteristics and steganographic noise dimensions to obtain corresponding ELA images and SRM images. Then, a lightweight MobileNetV2 network is introduced for multi-scale feature coding, and SECA attention mechanism is combined to realize feature interaction enhancement; finally, a pixel-level tamper probability map is generated through feature fusion strategy. This method can perform better detection accuracy than existing schemes on standard datasets and JPEG forged images with various compression quality factors, and has more outstanding performance.

[0057] Specifically, the image tamper detection method provided by the present embodiment fuses noise residual and compression artifact as shown in Figure 1 The method comprises the following steps:

[0058] Step one: build an image tamper detection model for identifying tampered regions in the input image and outputting the corresponding region binary mask.

[0059] As shown in Figure 2 The image tamper detection model includes an ELA flow processing layer (also known as ELA branch) composed of a 5x5 convolution layer with a fixed convolution kernel, an SRM flow processing layer (also known as SRM branch) composed of a convolution layer with a learning convolution kernel, a feature coding module composed of MobileNetV2, a feature compression and fusion module composed of a 1x1 convolution layer, a feature refining module composed of a hybrid attention mechanism SECA, and an output module composed of a 1x1 convolution layer and a bilinear interpolation-based up-sampling module.

[0060] The original image to be detected is subjected to an error level analysis (ELA) tool and a spatial rich model (SRM) processing to obtain corresponding ELA and SRM images; the two types of images are input into two branches of an image tampering detection model, the former can highlight potential tampering traces by calculating pixel differences of the image under different compression qualities, and the latter can capture noise distribution in the image, thereby providing more rich feature information for the model. In this embodiment, the sizes of the feature images extracted by the two branches are both 256x256x3.

[0061] Next, the feature images extracted by the two branches are subjected to convolution operations respectively to extract local features of the respective images. Among them, the ELA branch outputs an ELA image which is subjected to a convolution layer with a fixed weight of 5x5; the SRM branch outputs an SRM image which is subjected to a 3x3 convolution layer with a learnable weight. The feature images extracted by the two convolution layers are respectively input into a MobileNetV2 network for further feature extraction and processing. After MobileNetV2 coding, the two branches generate high-dimensional feature maps F1 and F2 of 1280 channels respectively.

[0062] After feature extraction, the high-dimensional feature maps F1 and F2 obtained by the two branches are connected by using a channel dimension concatenation strategy to form a comprehensive feature map F3 with an expanded channel number of 2560. The connected feature vector is subjected to a 1x1 convolution layer for cross-modal feature reorganization.

[0063] Subsequently, a hybrid attention mechanism SECA (Scale-wise Attenion) is applied to the cross-modal reorganized features output by the 1x1 convolution layer for feature refinement. The SECA module is an attention module combining an ECA (Efficient Channel Attention) module and a Softmax Attention, which can improve the performance of the model in the image feature extraction task through multi-scale information fusion and global feature weighting, enhance the model's ability to capture key features by learning the correlation and dependency between features, and improve its feature refinement capability in the complex scenario of image tampering detection.

[0064] Finally, the feature map processed by the SECA module is subjected to a further feature processing by a 1x1 convolution layer, and a bilinear interpolation is used to perform an upsampling operation on the obtained feature map to restore the spatial resolution of the feature map, thereby obtaining a binary mask representing the tampered region in the predicted feature image.

[0065] It needs to be particularly pointed out that in the embodiment, the image tampering detection model directly uses the ELA image and the SRM image generated by the ELA tool and the pre-trained spatial rich model as the input. The ELA tool and the spatial rich model are used as the front-end data processing module of the image tampering detection model, and the front-end data processing module is used to generate the corresponding ELA image and SRM image according to the original image to be detected. In other embodiments, the ELA tool and the spatial rich model can also be directly integrated into the image tampering detection model as a feature extraction module to realize multi-dimensional feature information extraction on the input original image.

[0066] Step two: divide the data set into a training set and a test set according to 9:1, and use the binary classification cross-entropy loss L BCE The image tampering detection model is trained and tested as a loss function. The model parameters with the best performance after testing are reserved and used for tampering area identification of the input image.

[0067] In the training phase of the network model, for the original tampered image containing label information (i.e. the real mask) collected, the error level analysis tool and the spatial rich model can be used to pre-generate the heat map (i.e. the ELA image) and the noise residual image (i.e. the SRM image) corresponding to each tampered image, then the size of the tampered image and its corresponding real mask is uniformly adjusted to 256*256, and finally a group of sample data is formed by using the ELA image, the SRM image and the real mask corresponding to each tampered image, and a data set containing a large amount of sample data is obtained.

[0068] The binary classification cross-entropy loss L BCE used in the training phase is as follows:

[0069]

[0070] In the above formula, y i represents the real label of the i-th pixel; y i = 0 represents no tampering, y i = 1 represents tampering; p i represents the probability that the i-th pixel predicted by the model belongs to the tampered area, p i ∈ [0, 1]; N represents the total number of pixels of the image.

[0071] In order to evaluate the detection performance of the trained image tampering detection model, the pixel-level F1 score is used as the evaluation index in the embodiment, and the F1 score is more suitable for the image tampering positioning task. Specifically, the output mask is converted into a binary mask based on the threshold, and compared with the real mask GT. Different F1 scores under a fixed threshold (0.5) and an optimal threshold are used to evaluate the detection performance of the network model.

[0072] Compared with various existing deep learning-based image tamper detection methods, the scheme provided in the embodiment combines the heat map generated by error level analysis and the noise residual generated by the spatially rich model to improve the adaptability and detection accuracy of the model for images of different qualities. Among them, ELA analysis can effectively detect tampering traces in JPEG format images. By comparing the differences between images under different compression qualities, the feature changes of the tampered region are highlighted. SRM focuses on the noise distribution in the image, and uses a high-pass filter to extract the noise residual of the image. These residuals will change significantly when the image is tampered with. By fusing and refining these two features, the new image tamper detection model constructed in the embodiment can fully utilize the compression characteristics and noise characteristics of the image to improve the recognition ability of the tampered region, especially in low-quality image scenarios, showing stronger robustness.

[0073] In order to make the principle and performance advantages of the new image tamper detection model constructed in the embodiment more clear, the following will be described in detail:

[0074] (I) Error Level Analysis (ELA) tool

[0075] The ELA tool is a data processing tool that can realize tamper detection based on image compression characteristics and output a heat map representing tampering traces. This tool does not need to rely on image metadata or external databases, and can quickly locate suspicious tampered regions through the compression characteristics of the image itself. The basic principle of the ELA tool is to use the data loss characteristics of images in lossy compression formats (such as JPEG) to detect tampered regions. When a JPEG format image is changed and saved again in the same format, each save will lose some data. By resaving the picture at a predetermined compression rate (for example, 95%), then checking the difference between the original picture and the resaved picture, that is, performing secondary compression on the image with a fixed quality factor, the residual error (i.e. error level) between the original image and the compressed image can be calculated to form an ELA heat map. After generating the ELA image of the original image, the unaltered region will be marked with almost the same error level, and the altered region will be marked with a different error level.

[0076] As Figure 3 shown, after the original image is processed by the error level analysis tool, the tampered region (such as copy-move, clone, splice) in the image usually undergoes an independent compression or saving process, and its residual compression error will be contrasted with the background region, and then highlighted in the form of a highlighted area in the ELA heat map. Therefore, the position of the tampered region contained in the image after ELA processing can be roughly seen from the obtained ELA image. For example, Figure 3 part (a) of the original image and its heat map after error level analysis,Figure 3 Part (b) is the heat map of the tampered image after ELA. For the convenience of comparison, the heat map is contrast-inverted.

[0077] (ii) Spatial Rich Model (SRM)

[0078] SRM is a steganalysis technique that extracts strong features from the noise residuals of an image by applying a number of high-pass filters. These features are closely related to the details of the image, and changes to the image will affect the corresponding residuals. To convert the image into a noise residual map, this embodiment applies 30 high-pass filters designed by Fridrich and Kodovsky to generate noise residual maps of the input image, which can generate noise residuals based on the center pixel. Among them, the 30 high-pass filters used in this embodiment are as shown in Figure 4 Figure 5 The comparison chart of the noise residual map obtained by processing the input using the high-pass filter shown in Figure 4 and its original image is shown.

[0079] (iii) MobileNetV2

[0080] MobileNetV2 is a lightweight convolutional neural network model, which is an upgraded version of MobileNetV1, aiming to improve the efficiency and accuracy of the model while maintaining low computational cost and small memory occupancy. Figure 6 Part (a) of Figure 6 Part (b) of shows the convolution block structure diagram of MobileNetv1 and improved MobileNetV2 respectively. Among them, MobileNetV1 replaces standard convolution with depth separable convolution, i.e. decomposes standard convolution into depth convolution and point convolution, which respectively play the roles of filtering and linear combination. But it has two defects, on the one hand, the structure is too simple, cannot reuse image features, and does not perform feature fusion operation. On the other hand, the depth separable convolution will cause information loss when processing low-dimensional data. Depth separable convolution has no ability to change the number of channels due to its own computational characteristics, and the output channel number depends on the input channel number of the upper layer. The improved MobileNetV2 overcomes the above defects.

[0081] ​MobileNetV2 can better capture complex features and context information in images while maintaining computational efficiency through Inverted Residual Blocks and Linear Bottlenecks modules. Considering that tampered regions of images often have complex texture and edge features, the embodiment utilizes the deeper network structure and stronger feature extraction capability of MobileNetV2 to effectively extract these features, thereby improving the accuracy of image tampering detection.

[0082] In combination Figure 6 It can be seen that in the MobileNetV2 network, the input features are processed by a 1x1 nonlinear convolution layer, a 3x3 depth separable convolution layer, and a 1x1 linear convolution layer in turn, and the resulting feature map is connected in residual with the original input features to serve as the final output feature. The activation function of the nonlinear convolution layer is the ReLU function, and the activation function of the linear convolution layer is Linear.

[0083] Among them, the inverted residual structure in MobileNetV2 is opposite to the traditional residual structure. In the traditional residual structure, the number of input and output channels is large, while the number of channels of the intermediate convolution layer is small, forming a bottleneck structure. In MobileNetV2, the number of input and output channels is small, and the number of channels of the intermediate depth separable convolution layer is large, forming an inverted bottleneck structure. MobileNetV2 takes an opposite approach, increasing the number of channels at the beginning, then extracting features through depth separable convolution, and finally using 1x1 convolution to reduce dimension. Effectively utilize the high-dimensional representation in the middle and maintain information flow through skip connection. Depth separable convolution is a key feature of the MobileNet series, which decomposes the standard convolution into two steps: depthwise convolution and pointwise convolution. Depthwise convolution applies a separate filter to each channel of the input, while pointwise convolution mixes these outputs, increasing the feature dimension. This can significantly reduce the number of parameters and computational requirements. In the inverted residual block, the convolution in the expansion part is usually nonlinear, but MobileNetV2 adds a linear bottleneck, i.e. 1x1 pointwise convolution, after the expansion part, with a linear activation function. This helps to preserve more information and reduce computational load.

[0084] In detail, the specific network structure of MobileNetV2 is shown in Table 1,

[0085] Table 1: Network structure of MobileNetV2 used in the embodiment

[0086]

[0087] (IV) Hybrid attention mechanism (SECA)

[0088] In the image tampering detection model designed in this embodiment, the feature images extracted by the two input branches are subjected to deep interaction of multi-modal features through channel splicing and attention enhancement mechanism in the feature fusion stage. Among them, the SRM branch extracts image noise residual features through a fixed weight convolution layer, focusing on the local statistical anomalies that may exist in the tampered region (such as JPEG compression artifacts, edge discontinuity of copy-paste operation), and the ELA branch captures the global distribution offset caused by image compression quality difference (such as blocking effect and chroma distortion caused by re-compression) through a convolution layer with learnable weights. The feature information extracted by the two branches is encoded by MobileNetV2 to generate two high-dimensional feature maps with 1280 channels respectively. The model uses channel dimension splicing (concat) strategy to expand the two 1280-channel feature maps to 2560 channels, and then performs cross-modal feature reorganization through 1x1 convolution. This operation not only realizes feature dimension compression (reduced to 1280 channels), but more importantly, establishes channel-level association across branches, enabling nonlinear interaction between noise features and compression artifact features at spatially corresponding positions, thereby enhancing the joint representation capability of tampering traces.

[0089] To further optimize the fusion effect of the feature information extracted by different branches, the image tampering detection model of this embodiment also introduces a SECA hybrid attention mechanism for feature refinement. As shown in Figure 7 the SECA hybrid attention mechanism first uses the efficient channel attention mechanism ECA to obtain the global spatial information of the feature map through global average pooling, and then uses the 1D convolution operation to enhance the different scale information of the feature map, so that the model can more effectively capture the features at different scales. Based on ECA, SECA further models the pixel-level long-range dependency relationship using the self-attention mechanism, calculates the spatial attention weight through Query-Key-Value mapping, and enables the model to capture the context difference between the tampered region and the original region.

[0090] The dual-attention collaborative mechanism in the SECA used in this embodiment effectively suppresses the interference of background texture and focuses on the potential tampered region in both channel and spatial dimensions. In the image tampering detection task, this fusion method can simultaneously utilize the dual evidence of local noise anomaly and global distribution offset, significantly improving the detection robustness of complex tampering operations (such as splicing, erasing, and content generation).

[0091] Specifically, the feature refinement process of the hybrid attention mechanism SECA used in this embodiment includes the following steps:

[0092] i. For the characteristics of a given input Figure X , The spatial dimension is compressed by AdaptiveAvgPool2d to obtain the channel statistics z,

[0093] z = AdaptiveAvgPool2d(X),

[0094] Where B represents the batch size, C represents the number of channels, H represents the image height, and W represents the image width.

[0095] ii. Capture cross-channel interaction through one-dimensional convolution Conv1D with kernel size k, and generate channel weights α through Sigmoid,

[0096] α = σ(Conv1D(Y))

[0097] Y = transpose(squeeze(z)),

[0098] In the above formula, σ is the Sigmoid function; Y represents the result of reshaping each channel statistics z by dimension; squeeze(·) represents the dimension increase operation, and transpose(·) represents the dimension exchange operation.

[0099] iii. Use channel weights to perform channel weighting on input feature maps to obtain the output feature E of the ECA module:

[0100]

[0101] In the above formula, represents channel-by-channel multiplication.

[0102] iv. Flatten E in the spatial dimension to a sequence X ECA , And generate query vector Q, key vector K and value vector V through linear transformation:

[0103]

[0104] In the above formula, W q represents the query vector weight; W k represents the key vector weight; W v represents the value vector weight.

[0105] v. Get the spatial correlation weight A through Softmax normalization,

[0106] A = Softmax(QK T ).

[0107] v. The value vector is aggregated and the spatial structure is recovered by using the spatial correlation weight to obtain the output feature X of the SECA module Attn ,

[0108] X Attn = Reshape(AV).

[0109] Embodiment 2

[0110] In order to realize efficient application of the image tampering detection method for fusing noise residual and compression artifacts provided in embodiment 1, the embodiment further provides a computer program product, a storage medium and an image tampering identification using the corresponding detection method.

[0111] Specifically, the computer program product provided in the embodiment includes a computer program, which, when executed by a processor, creates a trained image tampering detection model in the image tampering detection method for fusing noise residual and compression artifacts as in embodiment 1. The image tampering detection model is used to identify the tampered region contained in the input image and generate the corresponding binary mask.

[0112] The storage medium provided in the embodiment stores a computer program, which, when executed by a processor, creates a trained image tampering detection model in the image tampering detection method for fusing noise residual and compression artifacts as in embodiment 1. The image tampering detection model is used to identify the tampered region contained in the input image and generate the corresponding binary mask.

[0113] The image tampering identification device provided in the embodiment includes a memory, a processor, and a computer program stored in the memory and running in the processor. When the processor executes the computer program, a trained image tampering detection model in the image tampering detection method for fusing noise residual and compression artifacts as in embodiment 1 is created. The image tampering detection model is used to identify the tampered region contained in the input image and generate the corresponding binary mask.

[0114] The image tampering identification device provided in the embodiment is essentially a computer device, which can be an embedded module embedded in a computer device with data receiving, processing and storage functions, or an independent computer device with only image tampering detection function. For example, the computer device in the embodiment can be a smart terminal, a tablet computer, a notebook computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including a standalone server, or a server cluster composed of multiple servers).

[0115] The computer device indicated in the embodiment includes at least but is not limited to a memory and a processor which can be connected to each other in communication through a system bus. The memory (i.e. a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g. an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory can be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory can also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the memory can include both the internal storage unit and the external storage device of the computer device. In the embodiment, the memory is usually used to store an operating system and various application software installed on the computer device, etc. In addition, the memory can also be used to temporarily store various data which have been output or will be output.

[0116] The processor in some embodiments can be a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor is usually used to control the overall operation of the computer device. In the embodiment, the processor is used to run the program code or process the data stored in the memory.

[0117] Test Experiment

[0118] In order to verify the performance of the image tampering detection method for fusing noise residual and compression artifacts provided in the embodiment, the technical personnel formulate an experimental plan, compare and test the present application scheme and other comparative schemes on multiple standard data, and verify the contribution of different components in the network model of the present application in the final performance through the ablation experiment. The experimental content is as follows:

[0119] I. Experimental setup

[0120] (1.1) The present application scheme

[0121] The experiment is implemented using pytorch on a machine equipped with Nvidia GeForce GTX 1050 GPU and 8 GB memory. The image tampering detection model of the present application is trained on the CASIAv2 dataset, and the CASIA v2.0 dataset is composed of 7492 original images and 5124 modified images in different formats. The training set and the test set are divided according to the ratio of 9:1. Before training the image tampering detection model, the ELA tool and the spatial rich model are used to generate the ELA heat map and the noise residual image corresponding to all tampered images in the dataset, respectively. Then the tampered images and the corresponding real mask are resized to 256*256, and finally the ELA heat map and the noise residual image corresponding to the tampered image are directly input into the network model for training.

[0122] In the experiment, the model is trained for 100 times, and the adam optimizer is used for each training iteration, the learning rate is set to 0.001, and the binary classification cross entropy loss is used as the loss function.

[0123] (1.2) Control group scheme

[0124] In the experiment, the model is trained for 100 times, and the adam optimizer is used for each training iteration, the learning rate is set to 0.001, and the binary classification cross entropy loss is used as the loss function.

[0125] HP-FCN adopts a high-pass filtered image residual based fully convolutional network, which obtains image residuals through a high-pass pre-filtering module to enhance the repair traces, and then uses a ResNet block to extract discriminative features. This makes it have certain advantages in detecting repaired tampered images, and can effectively capture the differences between repaired and unrepaired areas in the residual domain. ManTra-Net is also a high-pass filtered image residual based fully convolutional network, which has a similar structure to HP-FCN, but may have differences in some details. MVSS-Net learns general features through multi-view feature learning and multi-scale supervision, while preventing false positives. The core idea is to use the noise distribution and boundary artifacts around the tampered area to learn semantic-independent features, thereby improving the sensitivity to tampering in new data. IML-ViT is based on Vision Transformer (ViT), which can naturally capture long-range dependencies and non-semantic differences in images using self-attention mechanisms, and introduces a simple feature pyramid network (SFPN) that can amplify artifacts at multiple scales, thereby more comprehensively capturing features of tampered areas. HiFi-Net is a hierarchical image forgery detection and localization model, composed of a multi-branch feature extractor, a localization module, and a classification module. The branches of the feature extractor classify the corresponding level of forgery attributes, and the localization module and the classification module segment the pixel-level forgery area and detect the image-level forgery, respectively.

[0126] II. Comparative experiments and data analysis

[0127] (2.1) Experimental content

[0128] This experiment tests the performance of the present invention (ours) and five control group schemes on four standard data sets NIST16, CASIAv1, Columbia, and Coverage without additional compression operations. During the experiment, the pixel-level F1 more suitable for image tampering localization tasks is used as the evaluation index to evaluate the detection performance of different schemes on the four public data sets. Specifically, the performance evaluation stage converts the output mask into a binary mask based on the threshold, and compares it with the true mask GT. In order to more comprehensively evaluate each scheme, this experiment compares the F1 scores of each scheme under the fixed threshold (0.5) and the optimal threshold, respectively.

[0129] (2.2) Experimental results and analysis

[0130] In the comparative experiment, the F1 scores of the present invention and the control group schemes are shown in Table 1, Figure 8 and the partial visual detection results of different schemes are shown.

[0131] Table 1: Fixed threshold F1 scores and optimal threshold F1 scores of each model on different data sets

[0132]

[0133]

[0134] From the above data, it can be seen that the F1 score (at the optimal threshold) of ManTra-Net on the NIST16 dataset reaches 0.724, which is outstanding. This is because the tampered images in the NIST16 dataset have specific characteristics, such as certain types of repair traces in the residual domain that are more easily captured by the network structure of ManTra-Net. However, on other datasets, its performance is affected by the type of tampering and image characteristics, and is not as good as on the NIST16 dataset. MVSS-Net performs relatively evenly on multiple datasets, such as the F1 score (at the optimal threshold) of 0.741 on the CASIAv1 dataset and 0.654 on the Columbia dataset. This indicates that MVSS-Net can adapt to different types of tampering detection and has a certain universality. However, due to the complexity of its feature learning and insufficient targeting of specific tampering characteristics, its performance on certain specific datasets is not as good as networks specifically designed for a certain type of tampering. The performance of IML-ViT varies on different datasets, and the model lacks generalization, especially when dealing with specific types of tampering, the performance will fluctuate significantly.

[0135] The proposed scheme achieves relatively better performance on the four standard datasets, especially in the F1 score with a fixed threshold (0.5), which reaches the best overall performance, such as 0.826 on the Coverage dataset and 0.794 on the Columbia dataset. And Figure 8 The visualization results also show that the proposed scheme can still accurately locate the tampered region when dealing with fake images with complex tampering boundaries.

[0136] The reasons for analyzing the above-mentioned superior performance are mainly as follows: the scheme of the application adopts a double-branch structure to process SRM (structure preserving transform) features and ELA (error level analysis) features respectively. The SRM branch is used to capture the structural information of the image, which can effectively preserve the key features such as edges and textures in the image, and is helpful for detecting tampering such as image splicing, which needs to pay attention to the change of structure. The ELA branch enhances the error information in the image, so that the traces of the tampered area are more obvious, especially for the case of repair tampering, which needs to capture subtle differences. Both branches are subjected to feature extraction by Mobilenetv2 network, which is a high-efficiency and lightweight network structure, and can extract features with strong discriminability while ensuring certain calculation efficiency. Finally, the features of the two branches are fused, and through 1x1 convolution, SECA module and up-sampling operation, the final tampered area prediction map is obtained.

[0137] The double-branch structure of the application can effectively cope with different types of tamper detection tasks. The SRM branch can accurately capture the splicing edge by using the structural information when dealing with splicing tampering, while the ELA branch plays an important role in repair tampering detection, and makes the difference between the repair area and the normal area more obvious by enhancing the error information. Thanks to the comprehensive use of different types of features by the double-branch structure and the powerful feature extraction capability of Mobilenetv2, the application can also locate the tampered area more clearly and accurately, whether it is a complex texture area or a light and shadow change area, it can better adapt to and detect the tampered part.

[0138] Thirdly, ablation experiment of the scheme of the application

[0139] In order to verify the effectiveness of each component in the double-branch network which fuses error level analysis (ELA) and spatial rich model (SRM), the technical personnel also conducted an ablation experiment. The experiment was carried out on the NIST16 dataset, which contains 564 tampered images, covers three main tampering operations of image splicing, copy moving and removal repair, and provides pixel-level tampered area mask to support accurate evaluation of model performance.

[0140] The ablation experiment takes the F1 score under the best threshold as the evaluation index, and the F1 score is the harmonic mean of accuracy and recall, which can comprehensively reflect the detection accuracy and recall ability of the model for the tampered area. The ablation experiment compares the performance of the model in the absence of the ELA branch, the SRM branch and the hybrid attention mechanism SECA with the performance difference of the complete scheme; finally, the ablation experiment results are shown in the following table:

[0141] Table 3: Ablation experiment results of the scheme of the application

[0142]

[0143] Analyzing the above table data can be known: when only using the ELA branch, the F1 score is only 0.230. This shows that the ELA branch has limitations in detecting JPEG image tampering alone. Although ELA can effectively detect tampering traces in JPEG format images by comparing the differences between images under different compression qualities to highlight the feature changes of the tampered area, it lacks sufficient mining of image detail features and is easily disturbed by image complex background and noise, resulting in more false positives and false negatives.

[0144] When only using the SRM branch for tampering detection, the F1 score reaches 0.549. This reflects the advantage of the SRM branch in extracting image noise residual features. SRM extracts strong features from image noise residuals by applying a high-pass filter, which are closely related to image details, and changes to the image will affect the corresponding residuals. However, the SRM branch mainly focuses on the noise distribution of the image, and the perception of the boundary and overall structure information of the tampered area is relatively weak, resulting in some tampered areas being unable to be accurately identified, especially in the case of complex background and multiple tampering operations.

[0145] When using the ELA and SRM dual-branch fusion for image tampering detection at the same time, the F1 score significantly improves to 0.795. This fully proves the effectiveness of the dual-branch structure. The ELA branch provides information about the compression characteristics of the image and can effectively detect tampering traces in JPEG format images; while the SRM branch analyzes from the perspective of image noise residuals and extracts noise features of the tampered area. The combination of the two achieves complementary advantages, enabling the model to utilize the compression characteristics and noise characteristics of the image to enhance the recognition ability of the tampered area.

[0146] After adding the SECA attention module to the dual-branch structure, the F1 score further improves to 0.811. This shows that the SECA module can enhance the model's ability to represent key features and further improve detection accuracy.

[0147] The above ablation experiment results show that the components in the dual-branch network proposed by the present application cooperate with each other and jointly improve the performance of image tampering detection. The dual-branch structure effectively integrates the compression characteristics and noise characteristics of the image, and the SECA attention module further enhances the feature expression ability, enabling the model to exhibit excellent detection performance when processing low-quality images.

[0148] Four, robustness comparison of different schemes

[0149] In practical applications, the accuracy of image tamper detection can be affected by various factors, such as noise and blur in low-quality images. These factors can interfere with the extraction of tamper traces, leading to false positives or false negatives in the model. In order to evaluate the effectiveness of the present solution in identifying image tampering under the above conditions, the technical personnel further developed a robustness analysis experiment. The robustness analysis experiment systematically evaluates the robustness of the present solution and the control group solutions (ManTra-Net, HiFi-Net, LML-ViT) in the face of different JPEG compression quality factors (QF) for sample images in three datasets (CASIAv1, Coverage, and Columbia). In the experiment, the threshold is fixed at 0.5, and the F1 score of each method is evaluated at multiple QF values (100, 90, 80, 70, 60, 50). The final experimental results of different solutions are shown in Table 1. Figure 9- Figure 11

[0150] It is worth noting that ManTra-Net and HiFi-Net use extensive data augmentation strategies including JPEG compression during training, while LML-ViT does not explicitly mention the use of such augmentation. This difference may affect the fairness of the comparison, so this experiment mainly focuses on the comparison between the present solution and ManTra-Net and HiFi-Net solutions.

[0151] Analysis Figure 9- Figure 11 From the data in Table 1, it can be seen that as the JPEG compression quality factor (QF) decreases, the F1 scores of all methods show a downward trend, indicating that JPEG compression has a significant impact on the performance of image tamper localization tasks. However, the method proposed in this paper exhibits higher robustness at all QF values, with F1 scores consistently superior to or comparable to existing methods. Taking the CASIAv1 dataset as an example, when QF decreases from 100 to 50, the F1 score of ManTra-Net decreases from 0.471 to 0.285, and the F1 score of HiFi-Net decreases from 0.523 to 0.241, while the F1 score of the present solution only decreases from 0.612 to 0.417, showing stronger anti-compression ability. In addition, experimental data show that the present solution performs particularly well under low QF (such as 50). This feature is particularly important in practical applications, as many social networking platforms (such as Facebook and WeChat) typically use low QF (about 70) for image compression. The present solution can effectively cope with the compression challenges in this real-world scenario.

[0152] ​The reason why the application achieves the above advantages is that the application combines the ELA and SRM feature extraction methods relative to other models trained on a single quality dataset. ELA can effectively detect tampering traces in JPEG format images, but may fail when facing a large number of post-processing or lossless format images. SRM focuses on the noise distribution in the image, and can provide strong support for the model even in the case of insufficient ELA features. The application enhances the adaptability to images of different quality factors by combining the two features of ELA and SRM. In this way, the application scheme can still maintain a high detection accuracy when facing low-quality images, showing strong robustness. However, the existing scheme trained on a single quality dataset is often difficult to adapt to other quality scenarios, resulting in a significant decline in performance.

[0153] The above only describes the preferred embodiments of the application and is not intended to limit the application. Any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A method for detecting image tampering by fusing noise residuals and compression artifacts, characterized in that, It includes: Construct an image tampering detection model to identify tampered regions in an input image and output a binary mask of the corresponding region; In the image tampering detection model, the input image is first acquired and processed by an error level analysis tool and a spatial rich model to obtain an ELA image and an SRM image. The ELA image is then processed sequentially through a 5×5 convolutional layer with fixed weights and a MobileNetV2 network for feature extraction to obtain a first feature map F1. The SRM image is then processed sequentially through a 3×3 convolutional layer with learnable weights and a MobileNetV2 network for feature extraction to obtain a second feature map F2. F1 and F2 are then concatenated to obtain a comprehensive feature map F3. F3 performs feature compression and fusion through a 1×1 convolutional layer, then refines the features through a hybrid attention mechanism SECA, and the resulting fused features are then processed by a 1×1 convolutional layer and an upsampling module based on bilinear interpolation to obtain the predicted binary mask. The dataset is divided into training and test sets, and a binary classification cross-entropy loss L is used. BCE The image tampering detection model is trained and tested using the loss function. The model parameters with the best performance after testing are retained and used to identify tampered regions in the input image.

2. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 1, characterized in that: The error level analysis tool is used to perform secondary compression on the image with a fixed quality factor and calculate the residual between the original image and the compressed image to form an ELA image; the ELA image is a heatmap that marks whether different regions have been tampered with by error level. The spatial rich model contains 30 filters that can generate noise residuals based on the center pixel; the spatial rich model is used to generate an SRM image that characterizes the noise residual distribution of the input image through the 30 filters.

3. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 1, characterized in that: In the MobileNetV2 network, the input features are processed sequentially through a 1×1 nonlinear convolutional layer, a 3×3 depthwise separable convolutional layer, and a 1×1 linear convolutional layer. The resulting feature map is then residually concatenated with the original input features to serve as the final output feature. The activation function of the nonlinear convolutional layer is the ReLU function, and the activation function of the linear convolutional layer is the Linear function.

4. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 3, characterized in that: The depthwise separable convolutional layer of the MobileNetV2 network contains 7 layers of extended bottleneck modules, with the number of output channels of the 7 extended bottleneck modules being 16, 24, 32, 64, 96, 160 and 320 respectively.

5. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 1, characterized in that: In the hybrid attention mechanism SECA, the input features first obtain global spatial information of the feature map through global average pooling, and then enhance the different scale information of the feature map using 1D convolution operation; then, spatial attention weights are calculated by QKV mapping in the self-attention mechanism and the features are weighted and aggregated to obtain the refined features.

6. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 5, characterized in that, The feature refinement process of the hybrid attention mechanism SECA includes: i. For a given input feature map X, The channel statistics z are obtained by compressing the spatial dimension using AdaptiveAvgPool2d. z = AdaptiveAvgPool2d(X), Where B represents the batch size, C represents the number of channels, H represents the image height, and W represents the image width; ii. Cross-channel interactions are captured using a one-dimensional convolution Conv1D with a kernel size of k, and channel weights α are generated using a Sigmoid function. α=σ(Conv1D(Y)) Y = transpose(squeeze(z)), In the above formula, σ is the Sigmoid function; Y represents the result of reshaping the statistics z of each channel according to the dimensions; queeze(·) represents the dimension addition operation, and transpose(·) represents the dimension exchange operation; iii. Apply channel weights to the input feature map to obtain the output feature E of the ECA module: In the above formula, This indicates multiplication by channel; iv. Flatten E into a sequence X in spatial dimensions. ECA , And generate the query vector Q, key vector K, and value vector V through linear transformation: In the above formula, W q W represents the query vector weight; k W represents the key vector weights; v Represents the weights of the value vector; v. The spatial correlation weight A is obtained through Softmax normalization. A=Softmax(QK T ), vi. By using spatial correlation weights to weight and aggregate the value vectors and restore the spatial structure, the refined feature X output by the SECA module is obtained. Attn , X Attn =Reshape(OFF).

7. The image tampering detection method that fuses noise residuals and compression artifacts as described in claim 1, characterized in that: The binary cross-entropy loss L BCE The expression is: In the above formula, y i This represents the true label of the i-th pixel; y i =0 indicates no alteration, y i =1 indicates that tampering has occurred; p i p represents the probability that the model predicts the i-th pixel belongs to the tampered region. i ∈[0,1]; N represents the total number of pixels in the image.

8. A computer program product comprising a computer program, characterized in that: When the computer program is executed by the processor, it creates a trained image tampering detection model in the image tampering detection method that fuses noise residuals and compression artifacts as described in any one of claims 1-7. The image tampering detection model is used to identify tampered regions contained in the input image and generate corresponding binary masks.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it creates a trained image tampering detection model in the image tampering detection method that fuses noise residuals and compression artifacts as described in any one of claims 1-7. The image tampering detection model is used to identify tampered regions contained in the input image and generate corresponding binary masks.

10. An image tampering detection device, comprising a memory, a processor, and a computer program stored in the memory and running in the processor, characterized in that: When the processor executes the computer program, it creates a trained image tampering detection model in the image tampering detection method that fuses noise residuals and compression artifacts as described in any one of claims 1-7. The image tampering detection model is used to identify tampered regions contained in the input image and generate corresponding binary masks.

Images