Double-view-angle-based copy movement tampering detection method

Abstract

The invention discloses a double-view-angle-based copy moving tampering detection method, which comprises the following steps: tampering feature extraction: extracting features of different scales from an image to be detected by using ShuffleNetV2, and fusing the features through an adaptive receptive field selection module ARFSM to obtain shallow features and deep features containing accurate tampering features; tampering region positioning: using double-view-angle self-correlation calculation and a shallow similarity attention mechanism in the shallow features to obtain similarity information and tampering features for positioning source and target regions; and tampering region classification: capturing accurate tampering features from the deep features through at least three adaptive receptive field selection modules connected in series for classifying source and target regions. According to the method, high-precision positioning and classification of the copy moving tampering region are realized.

Description

Technical Field

[0001] The invention relates to the technical field of image security, and in particular to a copy, move and tamper detection method based on dual viewing angles. Background Art

[0002] With the increasing popularity of digital image editing tools, digital images can be easily edited and tampered with without leaving any obvious traces of modification. Image editing software such as PhotoShop and Meitu XiuXiu have powerful functions such as portrait beauty, image beautification, image stitching, and adding watermarks, making image editing more intelligent. The use of image editing software is to facilitate people's lives or for entertainment, but it can also have a negative impact on many aspects of social life through malicious manipulation, such as spreading fake news, online rumors, insurance fraud, and extortion. Therefore, judging the integrity and authenticity of the content of the picture plays an important role and significance in maintaining judicial justice and maintaining the order of daily life.

[0003] There are many types of image tampering operations, which are mainly divided into: image synthesis, computer-generated false images, and image enhancement. In image synthesis, copy-move tampering is the most common and easiest way to change the content of an image. It refers to copying and pasting one or more image areas to other locations in the same image. Early image copy-move tampering detection was based on a machine learning framework, combining feature extraction and classifier training. In 2003, Fridrich et al. proposed a copy-move forgery detection technique based on dictionary sorting of DCT coefficients. The image is divided into overlapping blocks of fixed size. DCT coefficients are extracted for each block, and then the feature matrix is ​​dictionary sorted, and similar blocks are found using Euclidean distance. In 2012, Li et al. proposed extracting features by using polar harmonic transform (PHT). This method is specifically designed to handle copy-move tampering with rotation characteristics. However, traditional copy-move tampering detection is only designed for a certain attribute of the image. The performance of this type of method mainly relies on feature extraction methods with high expressive power and relies on strong expert knowledge. Its detection rate and robustness need to be improved. These feature extraction-based tampering detection algorithms often find it difficult to achieve high-quality and efficient information security protection in practical applications.

[0004] With the development of neural network and other technologies, a large number of digital image copy-move forgery forensics technologies based on neural networks have been born, but their performance in distinguishing source / target regions, the accuracy of tampered region positioning, and the integrity of detected tampered regions still need to be improved. The existing network extracts similarity information imperfectly and does not fully utilize the tampering features contained in shallow features, which are very important for the positioning and classification of tampered regions. Therefore, this study hopes to aggregate more complete similarity information through dual perspectives, better utilize shallow features and the selection of receptive fields to obtain more accurate tampering features, improve the accuracy of image copy-move forgery detection (CMFD), achieve accurate positioning of tampered regions, and accurately distinguish source / target regions. In recent years, Wu et al. proposed a dual-branch BusterNet, which is the first CMFD method that can distinguish source and target regions. It uses the Simi-Det branch to locate the source and target regions through similarity information, and the parallel Mani-Det branch distinguishes the target region at the same time. Subsequently, Chen et al. proposed two serially constructed sub-networks, namely CMSDNet and STRDNet, in which CMSDNet first locates the source / target region by similarity, and then STRDNet further distinguishes the source and target regions. Liu et al. proposed a two-stage framework with autocorrelation deep matching and scheme bonding, in which the first framework is used to find high similarity regions, while the second framework is used to reduce errors and improve incomplete regions. Weng et al. proposed a tampering region detection framework UCM-Net similar to the U-Net architecture, and constructed UCM-Net-S and UCM-Net-L to detect tampering regions of different sizes respectively. Recently, Weng et al. proposed a new diffusion model-based network DCM-Net, which locates and classifies tampering regions through a high similarity box filling strategy, an adaptive feature combination strategy, and a dual attention detection head. Summary of the invention

[0005] The purpose of the present invention is to provide a copy-move-and-tamper detection method based on dual viewing angles.

[0006] The technical solution adopted by the present invention is:

[0007] The copy-move tampering detection method based on dual perspectives includes the following steps:

[0008] Step 1, tampering feature extraction: Use ShuffleNetV2 to extract features of different scales from the image to be detected, and fuse them through ASFFM to obtain shallow features and deep features containing more accurate tampering features;

[0009] Step 2: Tampered area location: After dual-view autocorrelation calculation and shallow similarity attention mechanism in the shallow features, similarity information and tampered features are obtained to locate the source and target areas;

[0010] Step 3: Classification of tampered areas: The deep features are passed through three series-connected adaptive receptive field selection modules to capture more accurate tampered features for classifying the source and target areas.

[0011] Further, step 1 includes the following steps:

[0012] Step 1.1, feature acquisition: Input the image to be detected into DV-Net with ShuffleNetV2 as the backbone network. ShuffleNetV2 uses a module consisting of convolution, pooling and 3 ShuffleNetV2 blocks to extract image features, and obtain 4 layers of image features of different scales;

[0013] Step 1.2, feature enhancement: The original features obtained in step 1.1 are obtained in the stereo attention through the feature guidance module, and the stereo attention is multiplied by the features sampled by CAEAFE, so as to weight the upsampled features and highlight the tampering features;

[0014] Step 1.3, feature fusion: The weighted features obtained in step 1.2 are added to the original features to obtain the fused features. Specifically, the input image size is h×w×3, and the 4-layer feature representation obtained by ShuffleNetV2 is Where c represents the number of feature map channels, h represents the height of the input image, and w represents the width of the input image; a cross-scale feature fusion method ASFFM is used, that is, v1 is the original feature, the size is kept unchanged, and it is fused with v2 to obtain the shallow feature f1; v3 is the original feature, the size is kept unchanged, and it is fused with v4 to obtain the deep feature f2, so as to achieve more effective fusion of features of different scales and improve the expression ability of tampering features in the fused features. Furthermore, step 1.2 specifically includes the following steps:

[0015] Step 1.2.1, channel attention: Global average pooling GAP and global maximum pooling GMP are used to aggregate spatial context information of the original input features respectively. After splicing the outputs of the two branches in the channel dimension, point-by-point convolution PWConv is used to integrate the two different spatial context information, and the attention on the channel is obtained through Sigmoid.

[0016] Step 1.2.2, spatial attention: Use deep convolution DWConv to convolve the original features along the spatial dimension, and then use point-by-point convolution PWConv to linearly combine the spatial information across channels to achieve spatial information interaction. Finally, obtain spatial attention through Sigmoid;

[0017] Step 1.2.3, stereo attention: After obtaining the channel and spatial attention through steps 1.2.1 and 1.2.2, multiply the two element by element to obtain stereo attention.

[0018] Step 1.2.4, feature guidance: The stereo attention obtained in step 1.2.3 is multiplied element by element with the features upsampled by CARAFE, so as to weight the upsampled features and highlight the tampered features.

[0019] Further, step 2 includes the following steps:

[0020] Step 2.1, similarity information in direction: Use cosine similarity to obtain similarity information in direction, that is, the shallow feature map f1 extracted from the cross-scale feature fusion module in step 1.1 has a size of The cosine similarity is calculated for each feature point on the feature map f1 and all other feature points on f1. The specific formula is as follows:

[0021]

[0022] in, and are each 1×1×c block of the feature points in the feature map; then the size of the cosine similarity set in the direction is adjusted to After the channel dimension is sorted in descending order, the first c are selected as the similarity information S in the direction c ;

[0023] Step 2.2, numerical similarity information: Use Euclidean distance to obtain numerical similarity information, that is, on the shallow feature map f1 extracted after the cross-scale feature fusion module in step 1.1, calculate the Euclidean distance between each feature point on the feature map f1 and all other feature points on f1. The specific formula is as follows:

[0024]

[0025] in, and For each 1×1×c block of the feature points in the feature map, the numerical similarity obtained is Adjust the size to The smaller the Euclidean distance, the greater the similarity. Therefore, after arranging in ascending order of channel dimensions, select S E The first c values ​​are used as the similarity information in the direction. In order to unify the cosine similarity, the larger the cosine value, the greater the similarity, the conversion is performed using formula (3);

[0026]

[0027] Step 2.3, dual-view combination: Multiply the directional and numerical similarity information extracted in step 2.1 and step 2.2 to obtain the dual-view similarity information, where ⊙ represents the multiplication of the elements at the corresponding positions of the matrix. The specific formula is as follows:

[0028]

[0029] Step 2.4, shallow similarity attention mechanism: The specific steps are as follows: compress the dual-view similarity information obtained in step 2.3 through the Mean function to The size of the sigmoid function is then converted into a similarity attention mechanism S a , and S a Multiply it back to the shallow feature f1 to obtain the feature map b1 that contains both similarity information and tampering features.

[0030] Furthermore, the detection head predicts the coordinates of the source / target area based on feature b1, and GFiocal Loss and GIoU Loss are used as loss functions to calculate the closeness between the predicted box and the ground truth box. Through training, the error between the predicted box output by the detection head and the ground truth box is minimized, so that the network can generate a box containing the complete tampered area based on the features.

[0031] Furthermore, the specific steps of step 3 are as follows:

[0032] Step 3.0, take the deep feature f2 as the input feature g;

[0033] Step 3.1, pass the input feature g through Sigmoid to obtain attention λ:

[0034] λ=Sigmoid(g) (5),

[0035] Step 3.2, the input feature g is passed through the upper branch to maintain the fine-grained features, and is multiplied by λ to obtain the fine-grained feature allocation weight;

[0036] Step 3.3, the input feature g passes through the lower branch and the convolution layer with a convolution kernel of 3×3 to expand the receptive field, obtain the context feature, and multiply it with (1-λ) to obtain the context feature allocation weight;

[0037] Step 3.4, fuse the features of the two branches. The fusion formula is as follows:

[0038] ARFSM(g)=g×λ+(1-λ)×Conv 3×3 (g) (6),

[0039] Step 3.5, determine whether the fusion times of the current fusion feature reaches the set times; if yes, execute step 3.6; otherwise, take the current fusion feature ARFSM(g) as the input feature g and execute step 3.1;

[0040] Step 3.6, output the current fused features to obtain a feature map b2 with a suitable receptive field.

[0041] Furthermore, the number of fusions is set to 3. Furthermore, the expression formula of the feature map b2 in step 3.6 is as follows:

[0042] b2=ARFSM(ARFSM(ARFSM(f2))) (7).

[0043] Furthermore, the detection head predicts the category of the area according to b2, and uses GFOcal Loss as the loss function to calculate the closeness between the predicted label and the ground truth label; by reducing the loss through training, the error between the label predicted by the detection head and the ground truth label is made as small as possible, so that the network can accurately classify the tampered area according to the tampered features.

[0044] The present invention adopts the above technical scheme, integrates dual-view self-correlation calculation (DV-SCC), shallow similarity attention mechanism (SSAM), adaptive receptive field selection module (ARFSM) and cross-scale features fusion module (ASFFM) to realize the location and classification of the copy mobile tampering area. Since simply using the similarity information of a single perspective will make the similarity information acquisition incomplete, for this reason, the present invention proposes dual-view self-correlation calculation (DV-SCC) to aggregate similarity information from two perspectives in terms of direction and value to obtain more complete similarity information. SSAM can obtain shallow features that emphasize similarity information and contain tampering features by converting the similarity information obtained by DV-SCC into a similarity attention mechanism and multiplying it back to the shallow features, so that the network can locate the tampering area according to the similarity information and can further distinguish the source / target area from the similar area that appears accidentally according to the tampering features. The adaptive receptive field selection module (ARFSM) can select appropriate receptive fields for deep features containing more semantic information, thereby extracting more precise tampering features and enabling the network to classify the source / target regions more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments;

[0046] Figure 1 It is a schematic diagram of the structure of the dual-viewing-based copy-move-and-tamper detection network DV-Net adopted by the present invention;

[0047] Figure 2 This is the structural diagram of the cross-scale feature fusion ASFFM of the present invention. Implementation

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0050] Unlike previous CMFD networks, similarity information can only be captured from a single perspective, which cannot guarantee the capture of complete similarity. DV-Net fuses the directional and numerical similarity information through DV-SCC to further aggregate more complete similarity information. In addition, unlike previous CMFD networks that only use similarity information to locate similar regions while ignoring tampering features, we innovatively weight the shallow features of similarity information through a shallow similarity attention module (SSAM), which emphasizes similarity information while retaining tampering characteristics, significantly improving the localization accuracy of the source / target region. Multiple serial adaptive receptive field selection modules (ARFSMs) can adaptively select the appropriate receptive field according to the size of the tampered region, improving the classification accuracy of the source / target region.

[0051] like Figure 1 As shown in FIG. 2 , the present invention discloses a dual-viewing-based copy-move-tampering detection method, which includes the following steps:

[0052] Step 1, tampering feature extraction: extract features of different scales from the input image, and more effectively fuse deep features and shallow features through the cross-scale feature fusion module, so that the fused features contain more accurate tampering features, so that the subsequent training network model can better locate and classify the tampered area; Step 1 specifically includes the following steps:

[0053] Step 1.1, obtain features: Input the tampered image into DV-Net with ShuffleNetN2 as the backbone network. ShuffleNetV2 uses a module consisting of convolution, pooling and 3 ShuffleNetV2 blocks to extract image features, and obtain 4 layers of image features of different scales. The obtained 4 layers of features are expressed as Where c represents the number of feature map channels, h represents the height of the input image, and w represents the width of the input image;

[0054] Step 1.2, feature enhancement: The original features obtained in step 1.1 are passed through the feature guidance module to obtain stereoscopic attention, and the stereoscopic attention is multiplied by the features upsampled by CAEAFE, so as to weight the upsampled features and highlight the tampering features. Its structure is as follows Figure 2 As shown in the figure, feature enhancement is mainly divided into the following four parts:

[0055] (1) Channel attention: Global average pooling (GAP) and global maximum pooling (GMP) are used to aggregate spatial context information of the original input features. After the outputs of the two branches are concatenated in the channel dimension, point convolution (PWConv) is used to integrate the two different spatial context information, and the attention on the channel is obtained through Sigmoid.

[0056] (2) Spatial attention: Use deep convolution DWConv to convolve the original features along the spatial dimension, and then use point-by-point convolution PWConv to linearly combine the spatial information across channels to achieve spatial information interaction. Finally, the spatial attention is obtained through Sigmoid.

[0057] (3) Stereoscopic attention: Multiply the channel and spatial attention obtained from (1) and (2) to obtain stereoscopic attention.

[0058] (4) Feature guidance: Multiply the stereo attention obtained in step 1.2.3 with the features upsampled by CARAFE, weight the upsampled features, and highlight the tampered features.

[0059] Using stereo attention can highlight the important features of the original features, which is beneficial to reduce the aliasing effect caused by the subsequent fusion of features of different scales.

[0060] Step 1.3, feature fusion: Add the weighted feature map obtained in step 1.2 to the original feature to obtain the fused feature; its structure is as follows Figure 2 As shown. Taking f2 as an example, feature extraction is mainly divided into the following three steps:

[0061] (1) Since the feature map sizes obtained by ShuffleNetV2 in step 1.1 are different, as shown in Figure 2 As shown in , the sizes of v3 and v4 are different. Therefore, in order to better integrate v3 and v4, it is necessary to convert feature maps at different scales into feature maps of the same size. First, the size is The feature map v3 remains unchanged, and CAEAFE upsampling is performed on the feature map v4 to obtain a feature map of the same size as v3.

[0062] (2) Multiply the stereo attention obtained in step 1.2 by the upsampled feature map of v4.

[0063] (3) Add the feature map obtained in (2) to the original feature v3 to obtain the fused feature f2.

[0064] The fusion features extracted through the above steps can not only make the features more effectively integrated, but also improve the expression ability of the tampered features in the fusion features. It is beneficial to improve the network's subsequent positioning and classification capabilities of the tampered areas.

[0065] Step 2: Tampering with the regional location; specifically including the following steps:

[0066] Step 2.1, dual-view autocorrelation calculation: If only cosine similarity or Euclidean distance is used, similarity information can only be captured from a single angle, which cannot guarantee the capture of complete similarity information for some tampered images. A dual-view autocorrelation calculation combining Euclidean distance and cosine similarity is proposed to aggregate more comprehensive similarity information from two perspectives: direction and value. The dual-view autocorrelation calculation is mainly divided into the following three steps:

[0067] (1) Similarity information in direction: Cosine similarity is used to obtain similarity information in direction, that is, the shallow feature map f1 extracted after the cross-scale feature fusion module in step 1.1, whose size is The cosine similarity is calculated for each feature point on the feature map f1 and all other feature points on f1. The specific formula is as follows:

[0068]

[0069] in, and are each 1×1×c block of the feature points in the feature map f1. Then the size of the cosine similarity set in all directions is adjusted to After sorting in descending order, select the first c in the channel dimension as the similarity information S in the direction c ;

[0070] (2) Numerical similarity information: The numerical similarity information is obtained using the Euclidean distance, that is, on the shallow feature map f1 extracted after the cross-scale feature fusion module in step (1), the Euclidean distance is calculated for each feature point on the feature map f1 and all other feature points on f1. The specific formula is as follows:

[0071]

[0072] in, and are each 1×1×c block of the feature points in the feature map. The numerical similarity obtained Adjust the size to The smaller the Euclidean distance, the greater the similarity. Therefore, after arranging in ascending order of channel dimensions, select S E The first c values ​​are used as the similarity information in the direction. In order to unify the cosine similarity, the larger the cosine value, the greater the similarity, the conversion is performed using formula (3);

[0073]

[0074] Step 2.3, dual-view combination: Multiply the directional and numerical similarity information extracted in step 2.1 and step 2.2 to obtain the dual-view similarity information, where ⊙ represents the multiplication of the elements at the corresponding positions of the matrix. The specific formula is as follows:

[0075]

[0076] Step 2.4, shallow similarity attention mechanism: The specific steps are as follows: compress the dual-view similarity information obtained in step 2.3 through the Mean function to The size of the sigmoid function is then converted into a similarity attention mechanism S a , and S a Multiply it back to the shallow feature f1 to obtain the feature map b1 that contains both similarity information and tampering features.

[0077] Furthermore, the detection head predicts the coordinates of the source / target area based on feature b1, and GFiocal Loss and GIoU Loss are used as loss functions to calculate the closeness between the predicted box and the ground truth box. Through training, the error between the predicted box output by the detection head and the ground truth box is minimized, so that the network can generate a box containing the complete tampered area based on the features.

[0078] Step 3: Classification of tampered areas: The positioning results obtained in step 2 are combined with deep features to classify tampered images. The classification is mainly divided into the following steps:

[0079] Step 3.0, take the deep feature f2 as the input feature g;

[0080] Step 3.1, pass the input feature g through Sigmoid to obtain attention λ:

[0081] λ=Sigmoid(g) (5),

[0082] Step 3.2, the input feature g is passed through the upper branch to maintain the fine-grained features, and is multiplied by λ to obtain the fine-grained feature allocation weight;

[0083] Step 3.3, the input feature g passes through the lower branch and the convolution layer with a convolution kernel of 3×3 to expand the receptive field, obtain the context feature, and multiply it with (1-λ) to obtain the context feature allocation weight;

[0084] Step 3.4, fuse the features of the two branches. The fusion formula is as follows:

[0085] ARFSM(g)=g×λ+(1-λ)×Conv 3×3 (g) (6),

[0086] Step 3.5, determine whether the fusion times of the current fusion feature reaches the set times; if yes, execute step 3.6; otherwise, take the current fusion feature ARFSM(g) as the input feature g and execute step 3.1;

[0087] Step 3.6, output the current fused features to obtain a feature map b2 with a suitable receptive field.

[0088] Furthermore, the number of fusions is set to 3. Furthermore, the expression formula of the feature map b2 in step 3.6 is as follows:

[0089] b2=ARFSM(ARFSM(ARFSM(f2))) (7).

[0090] Furthermore, the detection head predicts the category of the area according to b2, and uses GFOcal Loss as the loss function to calculate the closeness between the predicted label and the ground truth label; by reducing the loss through training, the error between the label predicted by the detection head and the ground truth label is made as small as possible, so that the network can accurately classify the tampered area according to the tampered features.

[0091] Effect description: The compared networks are mainly divided into the following three categories: 1. CMFD networks based on image segmentation: CMSDNet, UCM-Net and LHCM-Net; 2. CMFD networks based on target detection: DCM-Net; 3. CMFD networks that can distinguish between source and target areas: BusterNet and STRD-Net. For different tampered areas, UCM-Net needs to be divided into two networks: UCM-Net-L (for large tampered areas) and UCM-Net-S (for small tampered areas).

[0092] In order to evaluate the performance of the present invention, DV-Net and the above three types of networks are experimentally compared; BusterNet and STRD-Net can segment the source and target regions. The evaluation criteria used are pixel-level F1 and frame-level F1.

[0093] Pixel-level F1: The predicted result is converted into a black-and-white binary image, where the pixels in the box area are white, indicating that the pixels are predicted to be tampered with, and the pixels outside the box area are black, indicating that the pixels are predicted to be true. Then, the image is calculated with the ground truth mask. Let Tp, Fp, and FN be the number of correctly detected pixels, the number of pixels incorrectly detected as forged pixels, and the number of omitted forged pixels, respectively.

[0094]

[0095] F1 at the box level: The irregular mask area is expanded into the corresponding minimum rectangular area, and then the IoU is calculated with the area of ​​the predicted box, where IoU is the intersection and union ratio of the predicted box area and the expanded mask area. When the IoU is greater than a predefined threshold (for example, 0.5), the predicted box is classified as a positive sample. In contrast, samples with IoU less than the predefined threshold are considered negative samples. In addition, if the IoU between the ground truth box and any predicted box is less than the threshold, the ground truth box is considered to be a false negative sample. Tp is the number of true positive samples, Fp is the number of false positive samples, and FN is the number of false negative samples.

[0096] The three databases used in the experiment are USC-ISI, CoMoFoD, and COVERAGE. In terms of all indicators on the three benchmark datasets, DV-Net performs best among all the compared networks. From Tables 2 and 3, it can be seen that although DV-Net provides the same pixel-level F1 as UCM-Net-L-DenseNet121 on the COVERAGE dataset, it obtains significantly higher pixel-level F1 (close to 0.3 higher) than UCM-Net-L-DenseNet121 on the CoMoFoD dataset. In summary, DV-Net has stronger generalization performance than other networks and can locate source / spot regions more accurately. From Tables 2 and 3, it can be seen that DV-Net has higher applicability and stronger universality.

[0097] Table 1 Performance comparison of five methods on the USC-ISI dataset

[0098] Network Precision Recall F1 BusterNet - - 0.46 CMSDNet - - 0.69 UCM-Net-S-VGG16 0.85 0.84 0.83 UCM-Net-L-DenseNet121 0.56 0.52 0.51 LHCM-Net 0.90 0.86 0.87 DCM-Net (pixel level) 0.85 0.83 0.84 DCM-Net (frame level) 0.87 0.85 0.85 DV-Net (Pixel Level) 0.88 0.91 0.89 DV-Net (frame level) 0.88 0.90 0.89

[0099] Table 2 Performance comparison of five methods on the COVERAGE dataset

[0100] Network Precision Recall F1 BusterNet 0.65 0.61 0.58 CMSDNet 0.69 0.66 0.62 UCM-Net-S-VGG16 0.65 0.67 0.63 UCM-Net-L-DenseNet121 0.69 0.84 0.73 LHCM-Net 0.73 0.75 0.70 DCM-Net (pixel level) 0.70 0.72 0.68 DCM-Net (frame level) 0.71 0.73 0.70 DV-Net (Pixel Level) 0.74 0.73 0.73 DV-Net (frame level) 0.74 0.75 0.74

[0101] Table 3 Performance comparison of five methods on the CoMoFoD dataset

[0102] Network Precision Recall F1 BusterNet 0.57 0.49 0.49 CMSDNet - - 0.51 UCM-Net-S-VGG16 0.55 0.57 0.58 UCM-Net-L-DenseNet121 0.31 0.37 0.33 LHCM-Net 0.63 0.59 0.57 DCM-Net (pixel level) 0.60 0.62 0.61 DCM-Net (frame level) 0.64 0.62 0.63 DV-Net (Pixel Level) 0.64 0.60 0.61 DV-Net (frame level) 0.64 0.62 0.63

[0103] As can be seen from Table 4, on the CoMoFoD dataset, the number of DV-Net correctly distinguished is increased by 35.8% compared with STRD-Net, the number of DV-Net correctly distinguished is 2.7 times the number of BusterNet correctly distinguished, and the number of DV-Net correctly distinguished is 2 more than DCM-Net; on the COVERAGE dataset, the number of DV-Net correctly distinguished is increased by 63% compared with STRD-Net, the number of DV-Net correctly distinguished is 6.7 times the number of BusterNet correctly distinguished, and the number of DV-Net correctly distinguished is 3 more than DCM-Net. In general, compared with BusterNet, DV-Net has better generalization ability.

[0104] Table 4 compares the discrimination performance of source and target by the number of correctly distinguished images.

[0105] Network CoMoFoD COVERAGE BusterNet 33 10 STRD-Net 67 41 DCM-Net 89 64 DV-Net 91 67

[0106] The technical advantages of the present invention compared to the prior art are: (1) ASFFM can more effectively fuse features of different scales and improve the expressiveness of tampering features in the fused feature map. (2) Unlike the existing CMFD network that only uses directional or numerical similarity information, the present invention proposes DV-Net that integrates directional and numerical similarity information to obtain more complete similarity information. (3) At the same time, after the dual-view autocorrelation calculation, the similarity information is converted into an attention mechanism through SSAM, which is used to weight the shallow feature map, emphasize similar information and retain tampering features, which is crucial for the positioning of the source / target area, so that the network can better locate the tampering area. (4) Unlike the previous ASPP or SK-attention, the present invention proposes an adaptive receptive field selection module that can select a suitable receptive field according to the size of the tampering area to capture more accurate tampering features, thereby improving the accuracy of source / target area classification.

[0107] The present invention proposes dual-view autocorrelation calculation, which can aggregate more complete similarity information from two perspectives, direction and value, which is conducive to the network's positioning of high-similarity areas. The similarity information obtained after the dual-view autocorrelation calculation is converted into a similarity attention mechanism and multiplied by shallow features, which can both emphasize the similarity information and retain the tampering features, and can further distinguish the source / target area from the accidental similar area, so that the network can more accurately locate the tampered area. The adaptive receptive field selection module can adaptively select the appropriate receptive field according to the size of the tampered area, thereby improving the classification accuracy of the source / target area.

[0108] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. A dual-view copy-move tampering detection method, characterized in that: The dual-view based copy-move tampering detection network adopted includes cross-scale feature fusion module ASFFM, dual-view autocorrelation calculation DV-SCC module, shallow similarity attention mechanism SSAM, and adaptive receptive field selection module ARFSM; The cross-scale feature fusion module ASFFM is used to fuse features of different scales to improve the expressiveness of tampering features in the fused feature map; the dual-view autocorrelation calculation DV-SCC module extracts more comprehensive similarity information from two perspectives: direction and value; the shallow similarity attention mechanism SSAM converts similarity information into an attention mechanism, which is used to weight shallow features to emphasize similarity information and retain tampering features; The adaptive receptive field selection module ARFSM is used to adaptively select a suitable receptive field according to the size of the tampered area to extract more accurate tampering features; The detection method includes the following steps: Step 1, tampering feature extraction: Use ShuffleNetV2 to extract features of different scales from the image to be detected, and fuse them through the adaptive receptive field selection module ARFSM to obtain shallow features and deep features containing accurate tampering features; Step 2: Tampered area location: Use dual-view autocorrelation calculation and shallow similarity attention mechanism in shallow features to obtain similarity information and tampered features to locate the source and target areas; Step 3, tampered area classification: The deep features are passed through at least three series-connected adaptive receptive field selection modules to capture accurate tampered features for classifying the source and target areas.

2. The dual-view copy-move tampering detection method according to claim 1 is characterized in that: Step 1 includes the following steps: Step 1.1, obtain features: input the image to be detected into the dual-view copy-move tampering detection network with ShuffleNetV2 as the backbone network. ShuffleNetV2 uses a module consisting of convolution, pooling and 3 ShuffleNetV2 blocks to extract image features to obtain 4 layers of original image features of different scales; Step 1.2, strengthen features: The original features of the image are passed through the feature guidance module to obtain stereo attention, and the stereo attention is multiplied by the features after upsampling by CARAFE, so as to weight the upsampled features to highlight the tampering features; Step 1.3, feature fusion: add the weighted features to the original features of the image to obtain the fused features.

3. The dual-view copy-move tampering detection method according to claim 1 is characterized in that: The input image size in step 1 is h×w×3, and the 4-layer feature representation obtained by ShuffleNetV2 is Among them, c represents the number of feature map channels, h represents the height of the input image, and w represents the width of the input image. In step 1, the adaptive receptive field selection module ARFSM is used for fusion, that is, v1 is the original feature, keeping the size unchanged, and fused with v2 to obtain the shallow feature f1; v3 is the original feature, keeping the size unchanged, and fused with v4 to obtain the deep feature f2, so as to achieve more effective fusion of features of different scales and improve the expression ability of tampering features in the fused features.

4. The dual-viewing-based copy-move-tampering detection method according to claim 3 is characterized in that: Step 1.2 includes the following steps: Step 1.2.1, channel attention: Global average pooling GAP and global maximum pooling GMP are used to aggregate spatial context information of the original input features respectively. After splicing the outputs of the two branches in the channel dimension, point-by-point convolution PWConv is used to integrate the two different spatial context information, and then Sigmoid is used to obtain channel attention; Step 1.2.2, spatial attention: Use deep convolution DWConv to convolve the original features along the spatial dimension, and then use point-by-point convolution PWConv to linearly combine the spatial information across channels to achieve spatial information interaction; finally, obtain spatial attention through Sigmoid; Step 1.2.3, stereo attention: multiply channel attention and spatial attention element by element to get stereo attention; Step 1.2.4, feature guidance: multiply the stereo attention by the features upsampled by CARAFE element by element, and weight the upsampled features to obtain weighted features to highlight the tampering features.

5. The dual-viewing-based copy-move tampering detection method according to claim 1 is characterized in that: Step 2 includes the following steps: Step 2.1, similarity information in direction: Use cosine similarity to obtain similarity information in direction, that is, the shallow feature map f1 extracted from the cross-scale feature fusion module in step 1.1 has a size of The cosine similarity is calculated for each feature point on the shallow feature map f1 and all other feature points on the shallow feature map f1. The specific formula is as follows: in, and are each 1×1×c block of the feature points in the feature map; then the size of the cosine similarity set in the direction is adjusted to After the channel dimension is sorted in descending order, the first c are selected as the similarity information S in the direction c ; Step 2.2, numerical similarity information: Use Euclidean distance to obtain numerical similarity information, that is, after the shallow feature map f1 extracted by the cross-scale feature fusion module in step 1.1, calculate the Euclidean distance between each feature point on the shallow feature map f1 and all other feature points on the shallow feature map f1. The specific formula is as follows: in, and For each 1×1×c block of the feature points in the feature map, the numerical similarity obtained is Adjust the size to After sorting in ascending order in the channel dimension, select S E The first c of are used as the similarity information in the direction, and converted using formula (3) to obtain Among them, 1 is a three-dimensional tensor of all 1s; Step 2.3, dual-view combination: multiply the extracted directional and numerical similarity information to obtain the dual-view similarity information. The specific formula is as follows: Among them, ⊙ represents the multiplication of elements at corresponding positions of the matrix; Step 2.4, shallow similarity attention mechanism: compress the dual-view similarity information through the Mean function to The size is then converted into the similarity attention mechanism S through the Sigmoid function. a , and S a Multiply it back to the shallow feature map f1 to obtain the feature map b1 that contains both similarity information and tampering features.

6. The dual-view copy-move tampering detection method according to claim 1 is characterized in that: The specific steps of step 3 are as follows: Step 3.0, take the deep feature f2 as the input feature g; Step 3.1, pass the input feature g through Sigmoid to obtain attention λ: λ=Sigmoid(g) (5), Step 3.2, the input feature g is passed through the upper branch to maintain the fine-grained features, and is multiplied by λ to obtain the fine-grained feature allocation weight; Step 3.3, the input feature g passes through the lower branch and the convolution layer with a convolution kernel of 3×3 to expand the receptive field, obtain the context feature, and multiply it with (1-λ) to obtain the context feature allocation weight; Step 3.4, fuse the features of the two branches. The fusion formula is as follows: ARFSM(g)=g×λ+(1-λ)×Conv 3×3 (g) (6), Step 3.5, determine whether the fusion times of the current fusion feature reaches the set times; if yes, execute step 3.6; otherwise, take the current fusion feature as the input feature g and execute step 3.1; Step 3.6, output the current fused features to obtain a feature map b2 with a suitable receptive field.

7. The dual-viewing-based copy-move-tampering detection method according to claim 6 is characterized in that: In step 3.5, set the number of fusions to 3.

8. The dual-viewing-based copy-move-tampering detection method according to claim 7 is characterized in that: The expression formula of feature map b2 in step 3.6 is as follows: b2=ARFSM(ARFSM(ARFSM(f2))) (7).

Images