A highly robust image tampering detection method

Abstract

The application belongs to the field of information security, and provides a high-robustness image tampering detection method, which comprises the following steps: 1) obtaining multi-scale feature extraction by using a hole convolution, and using Concat to obtain a global feature representation, so that the loss of shallow layer features after multiple convolutions can be prevented, and a larger receptive field can be obtained, and the effect of small target tampering detection is improved; 2) extracting noise by using BayarConv, and the noise extractor is better than an SRM filter, and Bayarconv is developed to enhance the noise inconsistency between the manipulated area and the real area in a given image; 3) using a residual block, batch normalization and an activation function to capture spatial features, and the structure is a conventional encoder, and the main purpose is to cross-fuse with semantic features; 4) using a cross-attention mechanism to fuse spatial features and semantic features, and the fused features are input into an up-sampling layer and a convolution layer for detection, and a hybrid loss function is used to train the model. Experimental data on a large number of public data sets show that the cross-attention mechanism provided in the application can not only accurately predict the tampering position, but also more accurately identify the tampering type.

Description

[0001] This invention relates to the field of information security, and more specifically, to a highly robust method for detecting image tampering. Background technology:

[0002] The statements in this section only relate to the background technology in connection with this invention and do not necessarily constitute prior art.

[0003] Robust image tampering detection methods play an important role in the field of information security. They do not target a specific type of tampering, but rather locate and determine the type of tampering by addressing common tampering types such as copy-move, splicing, and inpainting.

[0004] In recent years, with advancements in image editing technology and software, low-cost image manipulation or tampering techniques have become widely available. Among these techniques, splicing, copying / moving, and deleting are the most common operations. Image splicing involves copying areas from a real image and pasting them into other images; copying / moving involves copying and pasting areas within the same image; and removing areas eliminated from a real image. These techniques can arbitrarily modify images or videos, generating forged images that are difficult for the human eye to distinguish from the real thing. While this technology has brought significant progress to the field of digital image processing, it has also been maliciously used in areas such as politics and pornography, causing considerable harm to society. Therefore, distinguishing between real and tampered images has become increasingly challenging. Most existing methods target only one type of tampering; however, these methods perform poorly when detecting multiple types of image manipulation.

[0005] Designing a robust model architecture to obtain more accurate localization and tamper type identification is a hot research topic in image tamper detection. Convolutional neural networks have proven to be very effective for two-dimensional image signal processing. However, unlike ordinary image tamper detection methods, robust detection methods use richer and more numerous datasets, making training relatively difficult. Therefore, this method uses the random forest algorithm to randomly select images from various types of datasets to reduce training complexity. Furthermore, how to utilize the complementary information of shallow and deep features is also a key issue in image tamper detection. Summary of the Invention

[0006] To alleviate the aforementioned problems, we propose a highly robust detection method in this invention. This method consists of four parts: a noise perception module, an encoder-decoder, a multi-scale feature extraction module, and a feature fusion module. We utilize BayarConv to generate noisy features, residual blocks, batch normalization, and activation functions to capture spatial information. Then, we perform convolution operations on the spatial feature groups to obtain the final spatial geometric features. Simultaneously, we utilize parallel dilated convolutional structures to obtain deep tampering information at different scales. Then, we employ a cross-attention mechanism to fuse spatial and semantic features to address the issue of complementary information between shallow and deep features. Finally, we train the entire network end-to-end, achieving better prediction performance.

[0007] The technical solution of this invention is to provide a robust image tampering detection method, which includes the following steps:

[0008] 1. Using dilated convolution to obtain multi-scale feature extraction, and using Concat to obtain global feature representation, can not only prevent the loss of shallow features after multiple convolutions, but also obtain a larger receptive field and improve the effect of small target tampering detection.

[0009] 1.1) First, set the convolution kernel to 3×3 and set the dilated convolution ratio d={1,3,5} for each layer. Different dilated convolution ratios can be used to explore image forgery information from different angles.

[0010] 1.2) Secondly, each layer uses ResNet101 to learn multi-scale semantic tampering features, and uses the output features of the last convolutional layer of ResNet for operation classification.

[0011] 2. Noise extraction is performed using BayarConv, a noise extraction method that outperforms the SRM filter. BayarConv was developed to enhance the noise inconsistency between the manipulated and real regions in a given image.

[0012] 2.1) First, given a convolutional filter with ω as the parameter and ω(0,0) as the center element, Bayarconv imposes two constraints: ω(0,0) = -1 and ∑ i,j！＝0 ω(i,j)=1 After each training iteration, the constraint is applied to ω.

[0013] 2.2) Next, the given image X is passed through a Bayarconv layer with a kernel size of 5×5×3 to obtain its full-size noise view as Bayarconv(x). After passing through the encoder, the output is an array of K feature maps from the last ResNet block of its backbone.

[0014] 3. Spatial features are captured by using residual blocks, batch normalization, and activation functions. This construction is a conventional encoder, and its main purpose is to cross-fuse with semantic features.

[0015] 3.1) First, we use convolutional layers to design the encoder so that the network can understand the appearance and spatial relationship (context) between tampered and untampered. An encoder consists of residual blocks, batch normalization and activation functions. After each residual block, a max pooling operation is performed to give translation invariance.

[0016] 3.2) Subsequently, the residual block can be conveniently optimized for residual mapping and can be well trained for more layers.

[0017] 3.3) Finally, the spatial and semantic features from the encoder are fed into the fusion module to obtain more accurate discriminative features.

[0018] 4. Spatial and semantic features are fused using a cross-attention mechanism. The fused features are then input into the upsampling layer and convolutional layer for detection, and the model is trained using a hybrid loss function.

[0019] 4.1) First, convert spatial features into query Q. s Convert the semantic features of an image into key K i Sum V i As shown in the following formula:

[0020]

[0021] Where S represents the spatial feature vector, B represents the semantic feature vector, and W represents the semantic feature vector. Q W K W V These represent the query weight matrix, key weight matrix, and value weight matrix, respectively.

[0022] 4.2) Next, an attention mechanism is used for calculation, as shown in the following formula:

[0023]

[0024] Where K i T K represents i The transpose of .

[0025] The attention obtained above is then input into a 1D convolutional layer and a max-pooling layer to obtain the final fused features. This improves the tampering boundary information and optimizes the performance of tampering region localization.

[0026] 4.3) The fused features are input into the Decoder, and each decoder follows the basic operations of upsampling, convolution and batch normalization, which improves the tampering boundary information and optimizes the performance of tampering region localization.

[0027] 4.4) The loss generated by the prediction positioning is calculated using SmoothL1 loss.

[0028]

[0029] Where f(x) i ) represents the predicted value, y i Represents the actual value.

[0030] 4.5) The loss generated by classification is achieved by using the cross-entropy loss function for semantic tampering classification.

[0031]

[0032] Where M represents the number of categories (usually 3), y ic This means that if the true class of sample i is equal to c, then p is 1; otherwise, p is 0. ic This represents the predicted probability that observed sample i belongs to category c.

[0033] The advantages of this invention are as follows: This invention reduces training complexity by randomly selecting various datasets for training, uses a BayarConv noise generator for noise extraction, and obtains rich spatial geometric features through an encoder-decoder. Simultaneously, it utilizes dilated convolution to obtain multi-scale feature extraction, and uses Concat to obtain a global feature representation. A cross-attention mechanism is used to fuse spatial and semantic features, and the fused features are then input into the decoder for detection. The final fused features are obtained by inputting into a 1D convolutional layer and a max-pooling layer. This improves the accuracy of tamper boundary information and optimizes the performance of tamper region localization. Attached Figure Description

[0034] Figure 1 Network Flowchart

[0035] Figure 2 Cross-attention network module Detailed Implementation

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Furthermore, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0037] The flowchart framework of this invention is as follows Figure 1 As shown, this invention is a highly robust image tampering detection method, and its specific operation is described below:

[0038] 1. Using dilated convolution to obtain multi-scale feature extraction, and using Concat to obtain global feature representation, can not only prevent the loss of shallow features after multiple convolutions, but also obtain a larger receptive field and improve the effect of small target tampering detection.

[0039] 1.1) First, set the convolution kernel to 3×3 and set the dilated convolution ratio d={1,3,5} for each layer. Different dilated convolution ratios can be used to explore image forgery information from different angles.

[0040] 1.2) Secondly, each layer uses ResNet101 to learn multi-scale semantic tampering features, and uses the output features of the last convolutional layer of ResNet for operation classification.

[0041] 2. Noise extraction is performed using BayarConv, a noise extraction method that outperforms the SRM filter. BayarConv was developed to enhance the noise inconsistency between the manipulated and real regions in a given image.

[0042] 2.1) First, given a convolutional filter with ω as the parameter and ω(0,0) as the center element, Bayarconv imposes two constraints: ω(0,0) = -1 and ∑ i,j！＝0 ω(i,j)=1 After each training iteration, the constraint is applied to ω.

[0043] 2.2) Next, the given image X is passed through a Bayarconv layer with a kernel size of 5×5×3 to obtain its full-size noise view as Bayarconv(x). After passing through the encoder, the output is an array of K feature maps from the last ResNet block of its backbone.

[0044] 3. Spatial features are captured by using residual blocks, batch normalization, and activation functions. This construction is a conventional encoder, and its main purpose is to cross-fuse with semantic features.

[0045] 3.1) First, we use convolutional layers to design the encoder so that the network can understand the appearance and spatial relationship (context) between tampered and untampered. An encoder consists of residual blocks, batch normalization and activation functions. After each residual block, a max pooling operation is performed to give translation invariance.

[0046] 3.2) Subsequently, the residual block can be conveniently optimized for residual mapping and can be well trained for more layers.

[0047] 3.3) Finally, the spatial and semantic features from the encoder are fed into the fusion module to obtain more accurate discriminative features.

[0048] 4. Spatial and semantic features are fused using a cross-attention mechanism. The fused features are then input into the upsampling layer and convolutional layer for detection, and the model is trained using a hybrid loss function.

[0049] 4.1) First, convert spatial features into query Q. s Convert the semantic features of an image into key K i Sum V i As shown in the following formula:

[0050]

[0051] Where S represents the spatial feature vector, B represents the semantic feature vector, and W represents the semantic feature vector. Q W K W V These represent the query weight matrix, key weight matrix, and value weight matrix, respectively.

[0052] 4.2) Next, an attention mechanism is used for calculation, as shown in the following formula:

[0053]

[0054] Where K i T K represents i The transpose of .

[0055] The attention obtained above is then input into a 1D convolutional layer and a max-pooling layer to obtain the final fused features. This improves the tampering boundary information and optimizes the performance of tampering region localization.

[0056] 4.3) The fused features are input into the Decoder. Each decoder includes upsampling, convolution and batch normalization, which improves the tampering boundary information and optimizes the performance of tampering region localization.

[0057] 4.4) The loss generated by the predicted positioning is calculated using Smooth L1 loss.

[0058]

[0059] Where f(x) i ) represents the predicted value, y i Represents the actual value.

[0060] 4.5) The loss generated by classification is achieved by using the cross-entropy loss function for semantic tampering classification.

[0061]

[0062] Where M represents the number of categories, y ic This means that if the true class of sample i is equal to c, then p is 1; otherwise, p is 0.ic This represents the predicted probability that observed sample i belongs to category c.

[0063] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A high-robustness image tampering detection method, characterized in that, The method includes the following steps: 1) Dilated convolution is used to obtain multi-scale feature extraction, and Concat can be used to obtain global feature representation; 2) Use BayarConv as a noise extractor to extract noise features; 3) Spatial features are captured by using residual blocks, batch normalization, and activation functions. The main purpose of this construction is to cross-fuse with semantic features. 4) Use a cross-attention mechanism to fuse spatial and semantic features. The fused features are then input into the upsampling layer and convolutional layer for detection, and the model is trained using a hybrid loss function. The specific method for step 4) is as follows: 5.1) First, convert spatial features into queries , convert semantic features of the image into keys and values , as shown in the following formula: ，(1) wherein S represents a spatial feature vector, B represents a semantic feature vector, respectively represent a query weight matrix, a key weight matrix, and a value weight matrix.​​ 5.2) Next, an attention mechanism is used for calculation, as shown in the following formula: ，(2) wherein denotes transpose of the above obtained attention is inputted into a 1-dimensional convolution layer and a max-pooling layer to obtain the final fused features; 5.3) Input the fused features into the Decoder, where the decoder includes upsampling, convolution, and batch normalization; 5.4) The loss generated by the prediction and positioning is calculated using Smooth L1 loss. ，(3) in Indicates the predicted value. Represents the actual value; 5.5) The loss generated by classification is achieved by using the cross-entropy loss function for semantic tampering classification; ，(4) where M denotes the number of classes, denotes 1 if the true class of sample i equals c, and 0 otherwise, denotes the predicted probability that observation i belongs to class c.

2. The method of claim 1, wherein the method is a high-robust image tampering detection method. The specific method for step 1) is as follows: 2.1) First, set the convolution kernel to... And set the dilated convolution rate d for each layer. Different dilated convolution rates can be used to explore image forgery information from different perspectives; 2.2) Secondly, each layer uses ResNet101 to learn multi-scale semantic tampering features, and uses the output features of the last convolutional layer of ResNet for operation classification.

3. The method of claim 1, wherein the method is a high-robust image tampering detection method. The specific method for step 2) is as follows: 3.1) First, given a convolutional filter centered at ω(0,0) with parameter ω, Bayarconv imposes two constraints: ω(0,0) = -1 and After each training iteration, constraints are applied to ω; 3.2) Next, the given image X is passed through a Bayarconv layer with a kernel size of 5×5×3 to obtain its full-size noise view as Bayarconv(x). After being encoded, the output is an array of K feature maps from the last ResNet block of its backbone.

4. The method of claim 1, wherein the method is a high-robust image tampering detection method. The specific method for step 3) is as follows: 4.1) First, convolutional layers are used to design the encoder, enabling the network to understand the appearance and spatial context relationship between tampered and untampered layers. An encoder consists of residual blocks, batch normalization, and activation functions. After each residual block, a max pooling operation is performed to give translation invariance. 4.2) Subsequently, the residual block can easily optimize the residual mapping and can effectively train more layers; 4.3) Finally, the spatial and semantic features from the encoder are fed into the fusion module to obtain more accurate discriminative features.

Images