Image Tampering Detection Method Based on CNN Multi-Scale Feature Extraction and Hierarchical Fusion

Abstract

This invention discloses an image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion, comprising the following steps: inputting the image to be tested into a noise preprocessing module for noise feature analysis, and outputting a feature map after processing by SRM and constrained convolutional layers; combining the obtained feature map with the original input image and inputting it into basic blocks to obtain a unified feature representation; inputting the unified feature representation into a multi-scale feature extraction module for feature extraction; the decoder of the multi-scale feature extraction module performs pixel refinement on the features to obtain mask 4; upsampling and hierarchical feature output of the feature map obtained by the attention module in the decoder of the multi-scale feature extraction module to obtain mask 1, mask 2, and mask 3; fusing the mask 1, mask 2, and mask 3 obtained by hierarchical feature fusion with the mask 4 obtained by the decoder in the multi-scale feature extraction module to obtain the final image tampering region prediction map.

Description

Technical Field

[0001] This invention relates to an image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion, belonging to the field of image passive evidence tampering detection technology. Background Technology

[0002] Image tampering detection technology is an forensic technique that analyzes the features of digital images to verify their authenticity and integrity. This technology plays a crucial role in information security, data communication, and many other areas, and is of great significance for ensuring data security and maintaining social stability. However, image tampering detection still faces significant challenges: low accuracy in locating tampered areas, insufficient sensitivity to small-area tampering, and poor model robustness.

[0003] In recent years, the research and development of deep learning has progressed rapidly, with new models constantly emerging in fields such as computer vision, semantic segmentation, and image processing, and recognition accuracy continuously improving. Compared to traditional methods with many limitations based on manual design, deep learning models are faster and more efficient at image processing and possess more powerful feature learning capabilities. Researchers have introduced deep learning into the field of image tampering detection, allowing the network to learn richer tampering features. For example, deep learning image forensics methods using Daubechies wavelet features are used to detect image splicing; VGG is used to extract image tampering features, and then the tampered region edge mask is used to further refine the tampered region mask. However, these network models still have some problems, such as most frameworks typically only being able to locate tampered regions of a limited size, high model complexity, high cost, and lack of interpretability. With the continuous emergence and upgrading of various image processing software, image tampering traces are becoming increasingly difficult to detect and can even be indistinguishable from genuine images. Image tampering detection continues to face challenges, requiring further improvements in detection performance to ensure the authenticity and originality of digital images. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide an image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion. By utilizing multi-scale feature extraction and hierarchical fusion networks to mine multi-scale fine-grained features, the method solves the problem of low accuracy in locating tampered regions in existing technologies.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion includes the following steps:

[0007] The image to be tested is input into the noise preprocessing module for noise feature analysis. After processing by SRM and constrained convolutional layers, the output is a feature map. The obtained feature map is combined with the original input image and then input into the basic block to obtain a unified feature representation.

[0008] The unified feature representation obtained by the noise preprocessing module is input into the multi-scale feature extraction module for feature extraction. The decoder of the multi-scale feature extraction module performs pixel refinement on the features to obtain the mask 4.

[0009] The feature maps obtained by the attention module in the decoder of the multi-scale feature extraction module are upsampled and layered to output mask 1, mask 2, and mask 3.

[0010] The masks 1, 2, and 3 obtained by fusion of hierarchical features are fused with the mask 4 obtained by the decoder in the multi-scale feature extraction module to obtain the final image tampering region prediction map.

[0011] Furthermore, the aforementioned input of the image to be tested into the noise preprocessing module for noise feature analysis includes the following steps:

[0012] The input image to be tested is fed into the SRM filter and the constrained convolutional layer respectively. The SRM filter analyzes the noise of the image to be tested and outputs the first feature map. The constrained convolutional layer analyzes the semantic content information of the image to be tested and outputs the second feature map.

[0013] The first feature map, the second feature map, and the original input image to be tested are fused together and then input into the basic block to obtain a unified feature representation.

[0014] Furthermore, the calculation method for the aforementioned constrained convolutional layer to analyze the semantic content information of the image under test is as follows:

[0015] W k (0,0)=-1

[0016] ∑ m,m≠0 W k (m,n)=1

[0017] In the formula, W k (m,n) represents the value at (m,n) in the k-th layer convolutional filter, with the center value of the convolutional filter set to (0,0). W k Updated as the entire network is trained.

[0018] Furthermore, the aforementioned step of inputting the unified feature representation obtained by the noise preprocessing module into the multi-scale feature extraction module for feature extraction includes:

[0019] The unified feature representation is input into the multi-scale feature extraction module. The attention block of the encoder in the multi-scale feature extraction module extracts features to capture the tampered region features. The extracted features are supplemented by the res2 block in the bridging block to make up for the feature loss in the previous feature extraction process and output the third feature map.

[0020] The Pearson coefficient and percentage pooling are used to identify similar regions in the image and output the fourth feature map.

[0021] The decoder of the multi-scale feature extraction module performs pixel refinement on the third and fourth feature maps to obtain mask 4.

[0022] Furthermore, the feature supplementation steps for the aforementioned res2 block include:

[0023] The input feature map is divided into four subsets of the same size: P1, P2, P3, and P4.

[0024] Except for the first feature set which is directly obtained as subset R1 without convolution processing, the remaining feature sets are input into a 3x3 convolution block for processing.

[0025] The corresponding subsets R1, R2, R3, and R4 are obtained. These subsets are then fused and input into a 1x1 convolutional block to obtain the required third feature map.

[0026] Furthermore, the aforementioned steps for determining similar regions in an image using the Pearson coefficient and percentage pooling include:

[0027] Calculate the Pearson coefficient ρ(i,j):

[0028]

[0029]

[0030] Where M is the dimension of the feature tensor, () T It is a transpose operation. yes The standardized version and They are The mean and standard deviation, This represents two random blocks on the image;

[0031] Perform percentage pool calculations:

[0032] T'X[i] = sort(TX[i])

[0033] Where TX[i] is the fractional vector obtained after calculating the Pearson coefficient, and T'X[i] is the sum of all fractional vectors obtained after calculating the Pearson coefficient;

[0034] The two regions with high scores are identified as similar regions, and the corresponding fourth feature map is output.

[0035] Furthermore, the aforementioned upsampling of the feature maps obtained by the attention module in the decoder of the multi-scale feature extraction module to output hierarchical features includes the following steps:

[0036] The third feature map obtained from res2 and the fourth feature map obtained after similar matching blocks are used as part of the input of the ordinary block of the decoder in the multi-scale feature extraction module. The attention block on the decoder scales and upsamples the input third and fourth feature maps to obtain layered mask maps: mask 1, mask 2, and mask 3.

[0037] The loss function used in the mask output process is:

[0038]

[0039] Among them G k This represents the true value label, P. k The label represents the predicted output.

[0040] Furthermore, in the aforementioned step of fusing the masks 1, 2, and 3 obtained from the hierarchical feature fusion with the mask 4 obtained from the decoder in the multi-scale feature extraction module, the total loss function used is:

[0041]

[0042] in L represents the output mask of the m-th layer. mask4 This represents the output of mask 4, α m β and β represent the weights corresponding to the loss functions in different processes.

[0043] This invention proposes an efficient multi-scale feature extraction and fusion network for copy-and-move forgery detection, consisting of a noise preprocessing module, a multi-scale feature extraction network, and a hierarchical feature fusion scheme. The noise preprocessing module aims to eliminate noise by highlighting inconsistencies between the tampered region and the real region. Simultaneously, to improve feature extraction capabilities from the tampered region, this invention proposes a multi-scale feature extraction network to mine rich location information and more refined features. Furthermore, this invention develops a hierarchical feature fusion scheme that compensates for the loss of useful contextual information through an upsampling strategy, achieving accurate localization of the tampered region.

[0044] The beneficial effects achieved by this invention are as follows:

[0045] Multi-scale feature extraction networks can uncover richer and more refined features. The hierarchical fusion scheme compensates for the loss of useful contextual information through an upsampling strategy, achieving accurate localization of the tampered region. Compared with existing technologies, the method proposed in this invention has a significant improvement in the accuracy of localizing the tampered region. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating the image tampering detection method of the present invention;

[0047] Figure 2 This is a schematic diagram of the SRM filter and constrained convolutional layer (Constrained Conv) of the present invention, as well as the feature mapping of the combination of the two;

[0048] Figure 3 This is a flowchart illustrating the multi-scale feature extraction module of the present invention;

[0049] Figure 4 This is a flowchart of the res2 block in the bridging block of the multi-scale feature extraction module of the present invention;

[0050] Figure 5 This is a flowchart illustrating the hierarchical feature fusion module of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0052] This embodiment discloses a copy-paste image tampering detection method based on CNN multi-scale feature extraction and hierarchical feature fusion, such as... Figure 1 As shown, it includes the following steps:

[0053] S10: The image to be tested is input into the noise preprocessing module for noise feature analysis. After processing by SRM and constrained convolutional layers, the output is a feature map. The obtained feature map is combined with the original input image and then input into the basic block to obtain a unified feature representation.

[0054] S20, the unified feature representation obtained by the noise preprocessing module is input into the multi-scale feature extraction module. The encoder attention block and bridging block in the multi-scale feature extraction module perform feature extraction. The decoder of the multi-scale feature extraction module performs pixel refinement on the features to obtain the mask 4.

[0055] S30, Upsample the feature maps obtained by the attention module in the decoder of the multi-scale feature extraction module to output layered features, and obtain mask 1, mask 2, and mask 3;

[0056] S40, the three masks obtained from the hierarchical features are fused with the mask obtained from the multi-scale feature extraction module to obtain the final image tampering region prediction map.

[0057] In step S10, the image to be tested is input into the noise preprocessing module for noise feature analysis, including the following steps:

[0058] Noise feature analysis includes SRM kernel analysis and constrained convolutional layer analysis. The input image to be tested undergoes both SRM kernel analysis and constrained convolutional layer analysis. SRM kernel analysis analyzes the noise in the image to be tested and outputs a first feature map; constrained convolutional layer analysis analyzes the semantic content information of the image to be tested and outputs a second feature map. The calculation method is as follows:

[0059] W k (0,0)=-1

[0060] ∑ m,m≠0 W k (m,n)=1

[0061] In the formula, W k (m,n) represents the value at (m,n) in the k-th layer convolutional filter, with the center value of the convolutional filter set to (0,0). W k Updated as the entire network is trained, the SRM filter kernel analysis, constrained convolutional layer analysis (Constrained Conv), and the feature map resulting from their combination are visualized as follows: Figure 2 As shown;

[0062] The obtained first feature map and second feature map are fused with the original input image to be tested, and then input into the basic block to obtain a unified feature representation.

[0063] In step S20, the unified feature representation is input into the multi-scale feature extraction module for feature extraction. The process of the multi-scale feature extraction module is as follows: Figure 3 As shown, it includes the following steps:

[0064] The unified feature representation is input into the multi-scale feature extraction module. The attention block of the encoder in the multi-scale feature extraction module extracts features to capture the tampered region features. The extracted features are supplemented by the res2 block in the bridging block to make up for the feature loss in the previous feature extraction process and output the third feature map.

[0065] The feature completion process for the res2 block is as follows: Figure 4As shown: The feature maps, which take the attention blocks from the encoder in the multi-scale feature extraction module as input, are divided into four subsets: P1, P2, P3, and P4. Each feature has the same scale size, but the number of channels is one-quarter of the number of channels in the input feature map. Except for the first feature set, which is directly obtained as subset R1 without convolution processing, the other feature sets are input into a 3x3 convolution block for processing to obtain the corresponding subsets R1, R2, R3, and R4. These subsets are then fused and input into a 1x1 convolution block to obtain the third feature map.

[0066] The encoder's end is connected to the decoder's end via a similarity matching block. Because copying and tampering involves copying and pasting a portion of image content to other locations within the same image, the copied region and the original region share similar characteristics. The similarity matching block uses the Pearson coefficient and percentage pooling to determine similar regions in the image. The Pearson coefficient ρ(i,j) is calculated as follows:

[0067]

[0068]

[0069] Where M is the dimension of the feature tensor, () T It is a transpose operation. yes The standardized version and They are The mean and standard deviation, This represents two random blocks in the image.

[0070] The percentage pool is calculated as follows:

[0071] T'X[i] = sort(TX[i])

[0072] Where TX[i] is the fractional vector obtained after calculating the Pearson coefficient, and T'X[i] is the sum of all fractional vectors obtained after calculating the Pearson coefficient;

[0073] Based on the calculation results of Pearson coefficient and percentage pooling, in an image, the higher the score, the higher the similarity, thus identifying two similar regions and outputting the corresponding fourth feature map;

[0074] The decoder of the multi-scale feature extraction module performs pixel refinement on the fourth feature map obtained through the similarity matching block and the third feature map obtained through the res2 block to obtain mask 4.

[0075] In step S30, the corresponding feature maps in the multi-scale feature extraction module are upsampled and output through the attention block of the decoder, including the following steps:

[0076] The third feature map obtained from res2 and the fourth feature map obtained after similarity matching are used as part of the input to the ordinary block of the decoder in the multi-scale feature extraction module. The attention block on the decoder upsamples the input third and fourth feature maps and outputs them in layers. Through scaling, upsampling, and other operations, layered mask images are obtained: mask 1, mask 2, and mask 3. A schematic diagram of the layered fusion module is shown below. Figure 5 As shown. The loss function used in the mask output process is:

[0077]

[0078] Among them G k This represents the true value label, P. k The label represents the predicted output. Adjusting the loss function will result in a higher pixel localization accuracy.

[0079] In step S40, the steps of fusing the three masks obtained by hierarchical feature fusion with the mask obtained by the decoder in the multi-scale feature extraction module to obtain the final image tampering region prediction map include:

[0080] The three masks obtained by the hierarchical feature fusion module are fused with the mask obtained by the decoder of the multi-scale feature extraction module through element-wise addition to obtain the final tampered region prediction map. Therefore, the total loss function of the entire method is as follows:

[0081]

[0082] in l represents the mask output of the m-th layer. mask4 α represents the output of mask 4. m β and β represent the weights corresponding to the loss functions in different processes.

[0083] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion, characterized in that, The method comprises the following steps: The image to be tested is input into a noise preprocessing module for noise feature analysis, and a feature map is output after processing by an SRM and a constraint convolution layer; the obtained feature map is combined with the original input image, and then input into a basic block to obtain a unified feature representation; The unified feature representation obtained by the noise preprocessing module is input into a multi-scale feature extraction module for feature extraction; a decoder of the multi-scale feature extraction module performs pixel refinement on the features to obtain a mask 4; The feature map obtained by the attention module in the decoder of the multi-scale feature extraction module is up-sampled and layer features are output to obtain a mask 1, a mask 2 and a mask 3; The mask 1, the mask 2 and the mask 3 obtained by fusing the layer features are fused with the mask 4 obtained by the decoder of the multi-scale feature extraction module to obtain a final image tampering region prediction map; The step of inputting the unified feature representation obtained by the noise preprocessing module into the multi-scale feature extraction module for feature extraction comprises: The unified feature representation is input into the multi-scale feature extraction module; an attention block of an encoder in the multi-scale feature extraction module extracts tampering region features; a res2 block in a bridging block supplements the features to make up for feature loss in the previous feature extraction process, and outputs a third feature map; Pearson coefficients and percentage pools are used to determine similar regions on the image, and a fourth feature map is output; The decoder of the multi-scale feature extraction module performs pixel refinement on the third feature map and the fourth feature map to obtain a mask 4.

2. The image tamper detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 1, characterized in that, The step of inputting the image to be tested into the noise preprocessing module for noise feature analysis comprises the following steps: The input image to be tested is input into an SRM filter and a constraint convolution layer; the SRM filter analyzes the noise of the image to be tested and outputs a first feature map; the constraint convolution layer analyzes the semantic content information of the image to be tested and outputs a second feature map; The first feature map, the second feature map and the original input image to be tested are fused, and then input into a basic block to obtain a unified feature representation.

3. The image tampering detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 2, characterized in that, The calculation method of the constraint convolution layer for analyzing the semantic content information of the image to be tested comprises: ， ， wherein represents the value at (0,0) in the kth layer convolutional filter, the center value of the convolutional filter is set to (0,0), W represents the value at (0,0) in the kth layer convolutional filter, the center value of the convolutional filter is set to (0,0), W k is updated as the entire network is trained.

4. The image tamper detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 1, characterized in that, The feature supplementing step of the res2 block comprises: The input feature map is divided into four subsets P1, P2, P3 and P4 of the same size; Except that the first feature set is directly obtained as a subset R1 without convolution processing, the remaining feature sets are input into a 3x3 convolution block for processing; The corresponding subsets R1, R2, R3 and R4 are obtained, and these subsets are fused and input into a 1x1 convolution block to obtain the required third feature map.

5. The image tamper detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 1, characterized in that, The step of using Pearson coefficients and percentage pools to determine similar regions on the image comprises: Pearson's coefficient was calculated Calculation: ， ， Where M is the dimension of the feature tensor, ( ) T It is a transpose operation. [i]Yes The standardized version of [i] They are The mean and standard deviation of [i] [i] represents two random blocks in the image; The percentage pool is calculated: ， where TX[i] is the score vector after calculating the Pearson coefficient, is the sum of all score vectors after calculating the Pearson coefficient; The two regions with high scores are determined as similar regions, and the corresponding fourth feature map is output.

6. The image tamper detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 1, characterized in that, The step of up-sampling and layer feature outputting the feature map obtained by the attention module in the decoder of the multi-scale feature extraction module comprises the following steps: The third feature map obtained by res2 and the fourth feature map obtained by the similar matching block are input as part of the decoder normal block in the multi-scale feature extraction module, and the attention block on the decoder scales and up-samples the input third feature map and fourth feature map to obtain layered mask maps: mask 1, mask 2, and mask 3. In the process of outputting the mask, the loss function used is: ， wherein represents the true value label, represents the predicted output label.

7. The image tamper detection method based on CNN multi-scale feature extraction and hierarchical fusion according to claim 1, characterized in that, In the step of fusing the mask 1, the mask 2, and the mask 3 obtained by fusing the layered features and the mask 4 obtained by the decoder in the multi-scale feature extraction module, the total loss function selected is: m∈{1, 2, 3}， where M is the dimension of the feature tensor, output mask representing the mth layer, output of mask 4, and representing the weights corresponding to the loss function in different processes.

Images