Self-adaptive sampling mechanism and double-step self-correlation copy movement tampering detection method

Abstract

The invention discloses an adaptive sampling mechanism and two-step self-correlation copy moving tampering detection method, and provides an adaptive sampling mechanism ASM, and the number and position of sampling points are automatically adjusted according to the characteristics of an input image, so that a network is helped to better detect tampered areas with different sizes. According to the method, low-calculation-amount double-step autocorrelation is provided, the high-similarity region on the high-resolution feature map is extracted through the feature correlation on the low-resolution feature map, and the correlation is only calculated among the features of the high-similarity region, so that the calculation amount is greatly reduced. According to the invention, a double-branch adaptive fusion module is provided, and a light-weight multi-scale cavity convolution module is adopted to adaptively fuse local context features before LD-SCC and related features after LD-SCC, so that the detection performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of image security technology, and in particular to a copy-move tampering detection method using an adaptive sampling mechanism and a double-step autocorrelation. Background Art

[0002] With the rapid decline in the cost of mobile devices, anyone can use their mobile phone to take photos anytime and anywhere. At the same time, the rapid progress and popularization of image editing software have made it easy to tamper with the content of digital images, and the traces of tampering are almost undetectable [1, 2]. Considering that digital images are often used as key evidence in news reporting, political events, crime investigations, and insurance claims, the consequences of forged images can be very serious [3, 4]. Therefore, it is particularly important to accurately detect whether an image has been tampered with. There are many ways to tamper with images. Among them, copy-move forgery (CMF) is a very common and simple tampering method. This method copies and pastes one or more areas of an image (source area) to another location in the same image (target area). In order to make the tampering look more realistic, this method usually also adjusts the brightness and contrast of the copied area and performs geometric processing such as scaling and rotation.

[0003] In recent years, many copy-move forgery detection (CMFD) methods have been proposed, which can be mainly divided into two categories, namely, methods based on hand-crafted features [5, 7-14] and methods based on deep learning [6, 15-19]. The former can be further divided into block-based methods [7, 9-11] and key point-based methods [8, 12-14]. Block-based methods usually divide suspicious images into overlapping or non-overlapping blocks, and then use methods such as discrete cosine transform [9], principal component analysis

[10] or Zernik transform moment

[11] to extract the features of each image block, and then detect the Copy-Move tampering area by comparing the similar features between blocks. Block-based methods are relatively simple, but the amount of calculation is large and it is difficult to cope with complex geometric transformations. Key point-based methods first extract the features of key points by using methods such as SIFT

[12] , SURF

[13] or ORB

[14] , and then use the similarity between key point features to match and identify tampering areas. Although keypoint-based methods are more robust to geometric transformations, they have difficulty detecting smooth copied and moved tampered areas.

[0004] In recent years, deep learning, especially convolutional neural networks, has been widely used in various fields. Methods based on deep learning also provide effective solutions for CMFD. BusterNet proposed by Wu et al.

[15] is the first CMFD model based on convolutional neural networks that can be trained end-to-end. It consists of two parallel branches, one for detecting tampered target areas and the other for detecting highly similar copy-move tampered areas. However, the parallel structure of BusterNet requires the two branches to be correctly positioned at the same time in order to correctly classify the source and target areas. To solve this problem, Chen et al.

[16] proposed two serial sub-networks, namely CMSDNet and STRDNet, where CMSDNet is used to detect highly similar areas in the input image, and STRDNet is used to distinguish the source and target areas from the detected highly similar areas. Zhu et al.

[17] proposed AR-Net, which combines the adaptive attention mechanism with the residual refinement module and is an end-to-end CMFD network. Liu et al.

[18] proposed a two-stage CMFD framework that combines self-depth matching with key point matching. Weng et al. [6] proposed two independent U-shaped CMFD networks, namely UCM-Net-L and UCM-Net-S. UCM-Net-L is responsible for detecting large-area tampered areas, while UCM-Net-S is responsible for detecting small-area tampered areas. Each network contains three modules, namely, feature extraction module (FEM), tampered region localization module (TRLM), and multiple cross-layer connections between FEM and TRLM. Although UCM-Net-L and UCM-Net-S have achieved excellent detection performance in detecting large-area and small-area tampered areas, both networks need to be trained separately, which increases the cost of computing resources. In order to reduce the computational overhead while maintaining good detection performance, Shi et al.

[19] proposed a lightweight high-precision network LHCM-Net that integrates a low-cost self-correlation calculation (SCC), a gated feature fusion module (GFFM), and a faster residual-U-block (FRSU) equipped with a FasterNet block

[20] .

[0005] So far, the detection process of most CMFD networks based on deep learning is to first use a convolutional neural network to extract the features of the input image, then calculate the similarity between the features through the self-correlation calculation (SCC) module to obtain the correlation map, and finally use the convolution layer and upsampling layer to decode the correlation map to obtain the final detection result. However, most CMFD networks have defects in the above stages. First, in the feature extraction stage, the commonly used standard convolution layer cannot adaptively adjust the number and position of sampling points according to the area and shape of the tampered area, which increases the difficulty of the network to detect tampered areas of different sizes. In addition, the SCC in the CMFD network aims to calculate the similarity between each feature point in the feature map, which will inevitably generate expensive computational burdens when used for high-resolution feature maps. Finally, the existing CMFD network only uses the correlation features after the self-correlation module to obtain the tampering detection results, ignoring the local context features extracted by the backbone network, which leads to inaccurate prediction of the edges and interiors of the tampered area by the model.

[0006] In the feature extraction stage of the prior art, the existing CMFD network uses a standard convolution layer to extract features from the input image, and the sampling position of the standard convolution layer is fixed, so it is difficult to effectively extract the features of tampered areas of different areas and shapes at the same time. In addition, since the autocorrelation module in the CMFD network needs to calculate the similarity between the two feature points on the feature map, which brings huge computational overhead, most CMFD networks currently only use the autocorrelation module on the low-resolution feature map. However, this inevitably leads to these networks ignoring the similarity between high-resolution features, which damages the performance of the model. Finally, the existing CMFD network only uses the convolution layer and the upsampling layer after the autocorrelation module to process the correlation feature map obtained from the autocorrelation module to obtain the copy-move tampering detection result. Although this method takes into account the correlation information between features, it ignores the local context information of the feature itself, which leads to the model's detection results for the edges and insides of the tampered area being inaccurate. Summary of the invention

[0007] The purpose of the present invention is to provide an adaptive sampling mechanism and a two-step autocorrelation copy-move tampering detection method, which can achieve excellent detection performance on multiple data sets and is lightweight and has low computational overhead.

[0008] The technical solution adopted by the present invention is:

[0009] The adaptive sampling mechanism and the double-step autocorrelation copy-move tampering detection method include the model construction part, the model training and reasoning part, and the specific steps are as follows:

[0010] Step 1: Construct a copy-move tampering detection network model with an adaptive sampling mechanism and two-step autocorrelation. The network model performs the following steps:

[0011] Step 1-1, feature extraction: Input the image to be detected into AL-Net with ASM-Net as the backbone network. ASM-Net consists of 4 stages. The first two stages are composed of a "CBG 3-2" and two "CBG 3-1" set in sequence, which are used to extract low-level features on the input image, thereby helping AL-Net to better detect small-area tampering areas; the last two stages are composed of a "CBG 3-2" set in sequence and three ASM blocks that can automatically adjust the number and position of sampling points, which are used to adaptively extract high-level features on the input image, thereby helping AL-Net to better detect large-area tampering areas; the feature maps output by the four stages of ASM-Net are F1, F2, F3 and F4 respectively; among them, CBG 3-2 represents a convolution block composed of a standard convolution layer of 3×3 size and a step size of 2, a Batch Normalization layer and a GELU activation function; CBG 3-1 represents a standard convolution layer of 3×3 size and a step size of 1, a Batch Normalization layer and a GELU activation function. A convolutional block consisting of a Normalization layer and a GELU activation function; ASMblock consists of an adaptive sampling mechanism (ASM) and two point-wise convolutions (PWConv) and cross-layer connections;

[0012] Furthermore, the adaptive sampling mechanism ASM in step 1-1 performs the following steps:

[0013] Step 1-1-1, first input feature map E in The intermediate feature map E is obtained through "Convα-1" mid , the shape is W ε ×H ε ×(β 2 +1); “Convα-1” indicates a standard convolutional layer with a kernel size of α and a stride of 1; and Represents the input feature map E in The width and height of 2 +1 is the number of output channels of "Convα-1";

[0014] Step 1-1-2, intermediate feature map E mid Each sampling point E on mid (i,j) is split into the first channel and the remaining β 2 There are two parts of the channel, where i and j are the sampling points E midThe horizontal and vertical coordinates of (i,j), E mid The value of the first channel in (i,j) represents the number of sample points at position (i,j) predicted by "Convα-1", and The remaining β 2 The predicted values ​​of the channels will be reshaped into a 2D matrix of shape β×β Each value in (m,n∈[1,β]), represents the weight of each feature point in the β×β range centered at coordinate (i,j) predicted by “Convα-1”;

[0015] Step 1-1-3: Since the number of sampling points required for each location is different, ASM calculates the number of sampling points based on the predicted number of sampling points. filter Center front The largest weight value is assigned to the rest of the weight values, and the new weight matrix is ​​obtained Its formula is shown in formula (4):

[0016]

[0017] in, express Middle Large weight values, and Represents rounding down and rounding up respectively. hour, The purpose is to allow "Convα-1" to automatically learn the number of sampling points required for each position through back propagation

[0018] Step 1-1-4, ASM will Perform L1-Normalization and get And with the input feature map E in By performing weighted summation, we get the output feature map E out , its formula is shown in formula (5):

[0019]

[0020] Furthermore, α and β are set to 5 in the third stage of ASM-Net and are set to 3 in the fourth stage of ASM-Net.

[0021] Step 1-2, feature fusion: Input the obtained feature maps F1, F2, F3 and F4 into the feature fusion module (FFM) for feature fusion to obtain the output feature map F f ;

[0022] Furthermore, step 1-2 specifically includes the following steps:

[0023] Step 1-2-1, the feature map F4 is input into two "Conv 3-1" for further feature extraction; Conv 3-1 represents a standard convolution with a size of 3×3 and a step size of 1;

[0024] Step 1-2-2, input the result obtained in step 1-2-1 into the global average pooling layer (global average pooling, GAP) to obtain the weight;

[0025] Step 1-2-3, perform element-wise product on the weights obtained in step 1-2-2 and the F1, F2, F3, and F4 after channel concatenation to obtain a weighted concatenated feature map, retaining the key information in the four feature maps;

[0026] Step 1-2-4: Input the weighted concatenated feature map into "CBG 1-1" and two "FasterNet blocks" for feature fusion to obtain the output feature map F f .

[0027] Specifically, in order for AL-Net to automatically filter the key information in the four feature maps according to the characteristics of the input image, FFM first inputs F4 into two "Conv 3-1" for further feature extraction, and then inputs the result into the global average pooling layer (GAP) to obtain the weight. FFM multiplies the obtained weights by element-wise product with F1, F2, F3 and F4 after channel concatenation to retain the key information in the four feature maps. The weighted concatenated feature map will be input into "CBG 1-1" and two "FasterNetblocks" for feature fusion to obtain the output feature map f of FFM. f , the structure of FFM is as follows Figure 5 shown.

[0028] Step 1-3, correlation feature extraction: extract the feature map F obtained in step 1-2 fA low-cost dual-step self-correlation calculation (LD-SCC) is input to extract the correlation information in the feature map.

[0029] Furthermore, steps 1-3 specifically include the following steps:

[0030] Step 1-3-1, LD-SCC takes the input feature map F f Downsampling to obtain low-resolution feature maps Then according to The correlation between the two features in the filter is f Highly similar regions in

[0031] Step 1-3-2, LD-SCC only in F f The self-correlation calculation (SCC) is performed between the features of the highly similar regions to obtain the output correlation feature map F d This method not only reduces the impact of redundant low-similarity information, improves the detection performance of AL-Net, but also significantly reduces the computational overhead. The flowchart of LD-SCC is shown in Figure 4 shown.

[0032] Specifically, in the first step, LD-SCC first performs a The input feature map F f Perform k-fold average pooling (AP) downsampling to obtain a low-resolution feature map Shape in The present invention sets k to 8. Then, LD-SCC Further reshape into the shape of The matrix And calculate The cosine similarity (CS) between two feature points is calculated as shown in formula (6):

[0033]

[0034] Among them, T represents the transpose operation, Represents The cosine similarity between the u-th feature point and the v-th feature point, which also represents F f The similarity between the uth k×k pixel block and the vth k×k pixel block. Finally, LD-SCC divides F according to the k×k partitioning method. fTransform into shape Tensor for For each k×k pixel block in , LD-SCC is based on The similarity information in is extracted to obtain the t k×k pixel blocks with the highest similarity, and the shape is Tensor Here, t is set to 20 in the present invention.

[0035] In the second step, LD-SCC will Each feature point in is only similar to the feature points of the t pixel blocks with the highest similarity to the pixel block where it is located. Each pixel block in To understand the connection between the t pixel blocks in , LD-SCC first transforms Transpose, then and Perform matrix multiplication, the formula is shown in formula (7):

[0036]

[0037] Finally, LD-SCC will The shape is restored to Then sort along the channel dimension and retain the first g largest values ​​along the channel dimension, resulting in a shape of The output feature map of Here, g is set to 32 in the present invention.

[0038] Step 1-4, tampering area positioning: Step 1-2 to obtain the fusion feature map F f Correlation feature map F obtained from step 1-3 d Input to the bi-branch adaptive fusion module (BAFM) for fusion. The first branch of BAFM converts F d Input MDCM for multi-scale feature extraction, and the second branch extracts F f The input point convolution is used to interact information between channels. Then, the output results of the two branches are concatenated in the channel dimension and input to MDCM together. MDCM will comprehensively consider F d The correlation characteristics in F f The multi-level features in , and multi-scale feature extraction is performed on both, so as to more accurately d and F fGenerate weights and retain the key information of both. BAFM splits the weights in the channel dimension to obtain two tensors with the same shape as the feature map, and weights the two feature maps. Finally, BAFM concatenates the two weighted feature maps in the channel dimension and inputs them into the lightweight FRSU for feature fusion to obtain the final detection result of AL-Net.

[0039] Specifically, the structure DWDCa-bc of the multi-scale dilated convolution module (MDCM) represents a depth-wise dilated convolution layer with a size of a×a, a stride of b, and a dilation rate of c. The multi-scale dilated convolution module DWDC first uses three series-connected depth-wise dilated convolution layers (DWDC) to extract multi-scale features on the input feature map. Compared with using three parallel DWDCs, the series structure can obtain a larger receptive field with the same amount of parameters, thereby better capturing multi-scale features and generating weights more accurately. The three output feature maps are respectively subjected to point convolution for information interaction between channels, and are subjected to batch normalization (BN) layers and nonlinear GELU activation functions to ensure feature diversity. Finally, the three feature maps will be spliced ​​in the channel dimension and fused in the input point convolution layer to obtain the final output result.

[0040] Step 2: Model training and reasoning:

[0041] Step 2-1, data preprocessing: read the training samples from the training set of the USC-ISI dataset and perform preprocessing to obtain the training data required for training; the preprocessing includes resizing, data enhancement and normalization.

[0042] Data preprocessing is a key step in CMFD tasks, which can improve the stability of the model during training and the performance during testing. The data preprocessing of the present invention includes the following steps:

[0043] (1) Reading data: The present invention reads training samples from the training set of the USC-ISI dataset. Each sample includes an original image and a corresponding label, and the label contains the category information to which each pixel in the original image belongs.

[0044] (2) Resizing: To ensure that all training samples have the same size, the present invention scales the original image and the corresponding label at the same time. The present invention uniformly adjusts the length and width of all images to 320×320. This step ensures that the model can process input images of the same size, simplifying the training process.

[0045] (3) Data enhancement: In this stage, the present invention randomly rotates and horizontally flips the input image and the corresponding label at the same time according to a certain probability. Random rotation and random horizontal flipping can enhance the model's ability to detect different angles and symmetries of the image.

[0046] (4) Normalization: The present invention uses statistical data of ImageNet to normalize images. This dataset contains a large number of images from various scenes. Using its mean and variance for normalization can help the model learn more efficiently, thereby accelerating the convergence speed.

[0047] Step 2-2, model training: randomly divide the training data into multiple batches, use the cross entropy loss function to guide the AL-Net training process; select the stochastic gradient descent method (SGD) to adjust the model parameters by estimating the gradient of the loss function to the parameters to optimize the performance; after a set number of rounds of traversal training, complete the model training and obtain the final model;

[0048] Furthermore, the training process of step 2-2 is set to 60 rounds, each round representing a complete traversal of the entire dataset.

[0049] Specifically, first, the present invention randomly divides the training data into multiple batches, each batch containing 16 samples. This batch method can effectively utilize the computing power of the GPU, ensuring that the model can process multiple samples in parallel, thereby improving the training efficiency. Next, the present invention uses a cross-entropy loss function to guide the training process of AL-Net. This loss function evaluates the deviation between the model prediction and the actual label. By minimizing this loss, AL-Net can adapt to the training data more accurately, thereby improving its performance in the CMFD task. In order to optimize the parameters of the model, the present invention selects the stochastic gradient descent method (SGD). This method adjusts the model parameters by estimating the gradient of the loss function to the parameters, thereby optimizing the performance. In addition, the entire training process is set to 60 rounds, each round representing a complete traversal of the entire data set.

[0050] Step 2-3, model reasoning: perform preprocessing on the image to be detected to make it completely consistent with the parameters used in model training, and then input it into the final model to analyze the image through forward propagation to estimate the probability of each pixel belonging to the tampered area and display it in the form of a probability map, and mark all pixels in the probability map whose predicted probability exceeds the set value as tampered pixels, thus completing the accurate identification of the tampered area of ​​the image.

[0051] Further, the value is set to 0.5.

[0052] Specifically, the model obtained from step 2-2 is equipped with the weight parameters learned during training, which provide support for the subsequent copy mobile image tampering detection task. First, this step performs necessary preprocessing on the image to be detected, including resizing and normalization, to ensure that it is exactly the same as the parameters used during training. Then, the preprocessed image is input into the trained model. The model analyzes the image through forward propagation and estimates the probability that each pixel belongs to the tampered area. The results are presented in the form of a probability map, in which the value of each pixel represents the possibility of the pixel being tampered. Finally, this step marks all pixels in the probability map output by AL-Net with a predicted probability exceeding 0.5 as tampered pixels, thus completing the accurate identification of the tampered area of ​​the image.

[0053] The present invention also discloses a copy-move tampering detection network with adaptive sampling mechanism and double-step autocorrelation, including an ASM-Net module, a double-branch adaptive fusion module BAFM, and a double-step autocorrelation module; ASM-Net includes 4 stages, the first two stages are composed of a "CBG 3-2" and two "CBG 3-1" set in sequence, which are used to extract low-level features on the input image, thereby helping AL-Net to better detect small-area tampering areas; the last two stages are composed of a "CBG3-2" and three ASM blocks that can automatically adjust the number and position of sampling points set in sequence, which are used to adaptively extract high-level features on the input image, thereby helping AL-Net to better detect large-area tampering areas; the feature maps output by the four stages of ASM-Net are F1, F2, F3 and F4 respectively; wherein CBG 3-2 represents a convolution block composed of a standard convolution layer of 3×3 size and step size of 2, a BatchNormalization layer and a GELU activation function; CBG 3-1 represents a standard convolution layer of 3×3 size and step size of 1, a BatchNormalization layer and a GELU activation function. The convolution block consists of a Normalization layer and a GELU activation function; the ASM block consists of an adaptive sampling mechanism (ASM) and two point-wise convolutions (PWConv) and cross-layer connections; the feature maps F1, F2, F3 and F4 are input to the feature fusion module (FFM) for feature fusion to obtain the output feature map F f ;

[0054] LD-SCC takes as input the feature map F f Downsampling to obtain low-resolution feature maps Then according to The correlation between the two features in the filter is fIn the second step, LD-SCC is only in the high similarity region of F f The self-correlation calculation (SCC) is performed between the features of the highly similar regions to obtain the output correlation feature map F d ;

[0055] BAFM includes at least two MDCMs. The first branch of BAFM will be F d The first MDCM is input for multi-scale feature extraction, and the second branch performs point convolution on the input for information exchange between channels. The output results of the two branches will be spliced ​​in the channel dimension and input into the second MDCM together. The multi-scale dilated convolution module DWDC uses three serially connected deep dilated convolution layers DWDC to extract multi-scale features on the input feature map to obtain three feature maps. The three feature maps are respectively subjected to point convolution for information exchange between channels, and are subjected to batch normalization (BN) layer and nonlinear GELU activation function to ensure feature diversity. Finally, the three feature maps will be spliced ​​in the channel dimension and fused with the input point convolution layer to obtain the final output result.

[0056] The present invention adopts the above technical solution. Unlike deformable convolution, which needs to obtain sampling points located at floating point coordinates through bilinear interpolation, the sampling point coordinates of ASM are all integers, thus reducing the computational overhead. An adaptive sampling mechanism (ASM) is proposed, which can automatically adjust the number and position of sampling points according to the characteristics of the input image, thereby helping the network to better detect tampering areas of different sizes. The present invention proposes a low-cost dual-step self-correlation calculation (LD-SCC), which first extracts high-similar areas on high-resolution feature maps through feature correlation on low-resolution feature maps, and then only calculates correlation between features in high-similar areas, thereby greatly reducing the amount of calculation. The present invention proposes a bi-branch adaptive fusion module (BAFM), which uses a lightweight multi-scale hole convolution module to adaptively fuse local context features before LD-SCC and related features after LD-SCC, thereby improving detection performance. The present invention combines these three structures to design a lightweight, low-computational-overhead, high-precision CMFD network AL-Net. Experimental results on four public datasets show that AL-Net is superior to the above-mentioned related CMFD networks in terms of detection accuracy, number of parameters, and computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments;

[0058] Figure 1 ASM-Net backbone network structure diagram of the present invention;

[0059] Figure 2 This is a schematic diagram of the ASM block structure of the present invention;

[0060] Figure 3 This is a schematic diagram of the structure of the input feature fusion module FFM of the present invention;

[0061] Figure 4 Schematic diagram of the process of low computational cost two-step autocorrelation LD-SCC;

[0062] Figure 5 It is a schematic diagram of the structure of the dual-branch adaptive fusion module BAFM;

[0063] Figure 6 It is a schematic diagram of the structure of the multi-scale dilated convolution module MDCM;

[0064] Figure 7 It is a schematic diagram of the structure of the copy-move tampering detection network of the present invention with an adaptive sampling mechanism and a double-step autocorrelation;

[0065] Figure 8 The following is a schematic diagram of the F1 comparison of 6 methods on 4 public datasets;

[0066] Fig. 9 Schematic diagram for comparison of parameters and computational complexity of the six methods. Implementation

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0068] like Figures 1 to 9 As shown in FIG. 1 , the present invention discloses a copy-move tampering detection method with an adaptive sampling mechanism and a double-step autocorrelation, including a model construction part, a model training and reasoning part, and the specific steps are as follows:

[0069] Step 1, model construction: Step 1-1, feature extraction: set the resolution to The 3-channel image to be detected is input into AL-Net with ASM-Net as the backbone network;

[0070] like Figure 1As shown, ASM-Net consists of 4 stages, of which the first two stages are composed of one "CBG 3-2" and two "CBG 3-1" set in sequence, which are used to extract low-level features on the input image, thereby helping AL-Net to better detect small-area tampering areas. The last two stages are composed of one "CBG 3-2" and three ASM blocks that can automatically adjust the number and position of sampling points. Their function is to adaptively extract high-level features on the input image, thereby helping AL-Net to better detect large-area tampering areas. The feature maps output by the four stages of ASM-Net are F1, F2, F3 and F4 respectively. As shown Figure 1 As shown in the figure, the ASM-Net backbone network structure diagram, where CBG a-b represents a convolution block consisting of a standard convolution layer of size a×a and stride b, a Batch Normalization layer, and a GELU activation function. For example, CBG 3-2 represents a convolution block consisting of a standard convolution layer of size 3×3 and stride 2, a Batch Normalization layer, and a GELU activation function. The ASM block consists of an adaptive sampling mechanism (ASM) and two point-wise convolutions (PWConv) and cross-layer connections. The structure of the ASM block is shown in the figure. Figure 2 As shown, Conve-f represents a standard convolution with a size of e×e and a step size of f. For example, Conv 1-1 represents a standard convolution with a size of 1×1 and a step size of 1.

[0071] ASM first takes the input feature map E in The intermediate feature map E is obtained through "Convα-1" mid , the shape is "Convα-1" means a standard convolutional layer with a kernel size of α and a stride of 1; and Represents the input feature map E in The width and height of 2 +1 is the number of output channels of "Convα-1"; α and β are set to 5 in the third stage of ASM-Net and 3 in the fourth stage of ASM-Net. Intermediate feature map E mid Each sampling point E on mid (i,j), is split into two parts, the first channel and the remaining β 2 channels, i and j are sampling points E mid The horizontal and vertical coordinates of (i,j); the value of one of the channels represents the number of sample points at position (i,j) predicted by "Convα-1", and The remaining β 2 The predicted values ​​of the channels will be reshaped into a 2D matrix of shape β×β Each value in (m,n∈[1,β]) represents the weight of each feature point in the β×β range centered at coordinate (i,j) predicted by “Convα-1”. Since the number of sampling points required for each position is different, ASM is based on the predicted number of sampling points. filter Center front The largest weight value is assigned to the rest of the weight values, and the new weight matrix is ​​obtained Its formula is shown in formula (4):

[0072]

[0073] in, express Middle Large weight values, and Represents rounding down and rounding up respectively. hour, The purpose is to allow "Convα-1" to automatically learn the number of sampling points required for each position through back propagation

[0074] Finally, ASM will Perform L1-Normalization and get And with the input feature map E in By performing weighted summation, we get the output feature map E out , its formula is shown in formula (5):

[0075]

[0076] Step 1-2, feature fusion: Input the feature maps F1, F2, F3 and F4 obtained in step 1-1 into the feature fusion module (FFM) for feature fusion. In order to allow AL-Net to automatically filter the key information in the four feature maps according to the characteristics of the input image, FFM first inputs F4 into two "Conv 3-1" for further feature extraction, and then inputs the results into the global average pooling layer (global average pooling, GAP) to obtain the weights. FFM performs element-wise product multiplication (element-wise product) of the obtained weights with F1, F2, F3 and F4 after channel concatenation to retain the key information in the four feature maps. The weighted concatenated feature map will be input into "CBG 1-1" and two "FasterNet blocks" for feature fusion to obtain the output feature map F of FFM. f , the structure of FFM is as follows Figure 5 shown.

[0077] Step 1-3, correlation feature extraction: extract the feature map F obtained in step 1-2 f A low-cost dual-step self-correlation calculation (LD-SCC) is used to extract the correlation information in the feature map. In the first step, LD-SCC takes the feature map F as input. f Downsampling to obtain low-resolution feature maps Then according to The correlation between the two features in the filter is f In the second step, LD-SCC is only in the high similarity region of F f The self-correlation calculation (SCC) is performed between the features of the highly similar regions to obtain the output correlation feature map F d This method not only reduces the impact of redundant low-similarity information, improves the detection performance of AL-Net, but also significantly reduces the computational overhead. The flowchart of LD-SCC is shown in Figure 4 shown.

[0078] Specifically, in the first step, LD-SCC first performs a The input feature map F f Perform k-fold average pooling (AP) downsampling to obtain a low-resolution feature map Shape in The present invention sets k to 8. Then, LD-SCC Further reshape into the shape of The matrix And calculate The cosine similarity (CS) between two feature points is calculated as shown in formula (6):

[0079]

[0080] Among them, T represents the transpose operation, Represents The cosine similarity between the u-th feature point and the v-th feature point, which also represents F f The similarity between the uth k×k pixel block and the vth k×k pixel block. Finally, LD-SCC divides F according to the k×k partitioning method. f Transform into shape Tensor for For each k×k pixel block in , LD-SCC is based on The similarity information in is extracted to obtain the t k×k pixel blocks with the highest similarity, and the shape is Tensor Here, t is set to 20 in the present invention.

[0081] In the second step, LD-SCC will Each feature point in is only similar to the feature points of the t pixel blocks with the highest similarity to the pixel block where it is located. Each pixel block in To understand the connection between the t pixel blocks in , LD-SCC first transforms Transpose, then and Perform matrix multiplication, the formula is shown in formula (7):

[0082]

[0083] Finally, LD-SCC will The shape is restored to Then sort along the channel dimension and retain the first g largest values ​​along the channel dimension, resulting in a shape of The output feature map of Here, g is set to 32 in the present invention.

[0084] Step 1-4, tampering area positioning: Step 1-2 to obtain the fusion feature map F fCorrelation feature map F obtained from step 1-3 d Input the bi-branch adaptive fusion module (BAFM) for fusion. The structure of BAFM is as follows: Figure 5 shown.

[0085] The structure of the multi-scale dilated convolution module (MDCM) is as follows: Figure 6 As shown in the figure, DWDCa-bc represents a depth-wise dilated convolution layer with a size of a×a, a stride of b, and a dilation rate of c. It first uses three series-connected depth-wise dilated convolution layers (DWDC) to extract multi-scale features on the input feature map. Compared with using three parallel DWDCs, the series structure can obtain a larger receptive field with the same amount of parameters, thereby better capturing multi-scale features and generating weights more accurately. The three output feature maps are respectively subjected to point convolution for information interaction between channels, and are subjected to batch normalization (BN) layers and nonlinear GELU activation functions to ensure feature diversity. Finally, the three feature maps will be spliced ​​in the channel dimension and fused in the input point convolution layer to obtain the final output result.

[0086] The first branch of BAFM will be F d Input MDCM for multi-scale feature extraction, and the second branch extracts F f The input point convolution is used to interact information between channels. Then, the output results of the two branches are concatenated in the channel dimension and input to MDCM together. MDCM will comprehensively consider F d The correlation characteristics in F f The multi-level features in , and multi-scale feature extraction is performed on both, so as to more accurately d and F f Generate weights and retain the key information of both. BAFM splits the weights in the channel dimension to obtain two tensors with the same shape as the feature map, and weights the two feature maps. Finally, BAFM concatenates the two weighted feature maps in the channel dimension and inputs them into the lightweight FRSU for feature fusion to obtain the final detection result of AL-Net. The overall structure of AL-Net is shown in the figure below. Figure 7 As shown:

[0087] Step 2: Model training and reasoning:

[0088] Step 2-1, data preprocessing: Data preprocessing is a key step in the CMFD task, which can improve the stability of the model during training and the performance during testing. The data preprocessing of the present invention includes the following steps:

[0089] (1) Reading data: The present invention reads training samples from the training set of the USC-ISI dataset. Each sample includes an original image and a corresponding label, and the label contains the category information to which each pixel in the original image belongs.

[0090] (2) Resizing: To ensure that all training samples have the same size, the present invention scales the original image and the corresponding label at the same time. The present invention uniformly adjusts the length and width of all images to 320×320. This step ensures that the model can process input images of the same size, simplifying the training process.

[0091] (3) Data enhancement: In this stage, the present invention randomly rotates and horizontally flips the input image and the corresponding label at the same time according to a certain probability. Random rotation and random horizontal flipping can enhance the model's ability to detect different angles and symmetries of the image.

[0092] (4) Normalization: The present invention uses statistical data of ImageNet to normalize images. This dataset contains a large number of images from various scenes. Using its mean and variance for normalization can help the model learn more efficiently, thereby accelerating the convergence speed.

[0093] Step 2-2, model training: First, the present invention randomly divides the training data into multiple batches, each batch containing 16 samples. This batch method can effectively utilize the computing power of the GPU, ensuring that the model can process multiple samples in parallel, thereby improving training efficiency. Next, the present invention uses a cross-entropy loss function to guide the training process of AL-Net. This loss function evaluates the deviation between the model prediction and the actual label. By minimizing this loss, AL-Net can adapt to the training data more accurately, thereby improving performance in the CMFD task. In order to optimize the parameters of the model, the present invention selects the stochastic gradient descent method (SGD). This method adjusts the model parameters by estimating the gradient of the loss function to the parameters to optimize performance. In addition, the entire training process is set to 60 rounds, each round representing a complete traversal of the entire data set.

[0094] Step 2-3, model reasoning: The model obtained from step 2-2 is equipped with the weight parameters learned during training, which provide support for the subsequent copy mobile image tampering detection task. First, this step performs necessary preprocessing on the image to be detected, including resizing and normalization, to ensure that it is exactly the same as the parameters used during training. Then, the preprocessed image is input into the trained model. The model analyzes the image through forward propagation and estimates the probability that each pixel belongs to the tampered area. The results are presented in the form of a probability map, in which the value of each pixel represents the possibility of the pixel being tampered. Finally, this step marks all pixels in the probability map output by AL-Net with a predicted probability exceeding 0.5 as tampered pixels, thus completing the accurate identification of the tampered area of ​​the image.

[0095] The experimental results of the present invention show that the detection performance of AL-Net is compared with that of BusterNet, CMSDNet, UCM-Net and LHCM-Net, and the evaluation index used is F1-score.

[0096] F1-score combines two important performance indicators, precision and recall. Precision is the ratio of the number of pixels that the model correctly predicts as tampered pixels to the number of pixels that the model always predicts as tampered pixels. The calculation formula of Precision is shown in formula (1).

[0097]

[0098] Among them, TP represents true positives, that is, the number of pixels where the model correctly predicts tampered pixels as tampered pixels, and FP represents false positives, that is, the number of pixels where the model incorrectly predicts real pixels as tampered pixels.

[0099] Recall refers to the ratio of the number of pixels that the model correctly predicts as tampered pixels to the total number of tampered pixels. The calculation formula of Recall is shown in formula (2).

[0100]

[0101] Where FN represents false negatives, i.e., the number of pixels where the model incorrectly predicts tampered pixels as real pixels.

[0102] F1-score is the harmonic mean of Precision and Recall, and is used to evaluate the model's balanced performance on these two indicators. The F1 score ranges from 0 to 1, where the closer the value is to 1, the better the model has achieved a balance between precision and recall. The calculation formula for F1-score is shown in formula (3):

[0103]

[0104] This experiment used four public datasets: USC-ISI, COVERAGE, CoMoFoD, and CASIA. The USC-ISI dataset contains 100,000 copy-move type tampered images, including 80,000 training set images, 10,000 validation set images, and 10,000 test set images. This experiment selected the training set to train AL-Net, and selected the test set, as well as the COVERAGE, CASIA, and CoMoFoD datasets to test the detection performance of AL-Net. The CASIA dataset contains 1,313 pairs of copy-move type tampered samples from the CASIA v2.0 dataset. The CoMoFoD dataset contains 200 pairs of test samples and 4,800 pairs of test samples that have undergone post-processing attacks of different types and intensities. The COVERAEG dataset contains 100 copy-move tampered images and their corresponding labels. The test results are as follows. Figure 8 As shown, the present invention compares the proposed AL-Net with several other advanced CMFD models. It can be seen that the detection performance of AL-Net on all data sets exceeds all current networks, except that the detection performance on the COVERAGE data set is on par with UCM-Net-L-DenseNet121. Specifically, UCM-Net designs two sub-networks for detecting tampered areas of different sizes, namely UCM-Net-L-DenseNet121 and UCM-Net-S-VGG16, among which UCM-Net-L-DenseNet121 is a sub-network specially designed for detecting large-size tampered areas. It uses a deeper DenseNet121 backbone network to extract features of large-area tampered areas. Therefore, it achieves excellent detection performance on the COVERAGE data set, which mainly contains images of large-area tampered areas. However, it performs poorly on the CoMoFoD data set, which mainly contains images of small-area tampered areas. Similarly, UCM-Net-S-VGG16 is a subnetwork designed specifically for detecting small-sized tampered areas. It uses a shallower VGG16 backbone network to extract features of small-sized tampered areas on the image. Therefore, it can achieve a higher F1 on the CoMoFoD dataset, but a poorer F1 on the COVERAGE dataset. Unlike UCM-Net, the AL-Net proposed in the present invention benefits from the adaptive sampling attention (ASM) that can automatically adjust the receptive field, and can achieve excellent detection performance on datasets containing tampered areas of different sizes with only a single network architecture. In addition, Fig. 9As shown in the figure, thanks to the low-cost dual-step self-correlation calculation (LD-SCC) and the low-parameter bi-branch adaptive fusion module (BAFM), AL-Net has the lowest number of parameters and floating point operations (FLOPs). These experimental results strongly demonstrate the high efficiency of AL-Net.

[0105] The present invention adopts the above technical solution. Since the current CMFD network uses a standard convolution layer for feature extraction, and the sampling position of the standard convolution layer is fixed, it is difficult to effectively extract the features of tampered areas with different areas and shapes. In order to solve this problem, the present invention proposes an adaptive sampling mechanism (ASM), which automatically learns the sampling point positions and the number of sampling points required for tampered areas of different areas and shapes through a convolution layer, thereby more accurately capturing the features of the tampered area. In addition, based on ASM, the present invention proposes ASMblock, which first uses ASM to aggregate features from the spatial dimension, and then uses two point convolutions to fuse the aggregated features in the channel dimension. Compared with variable convolution, ASM block has a smaller computational overhead. Based on ASM block, the present invention constructs a new backbone network, ASM-Net. ASM-Net extracts local context features of different ranges through layers of different depths, thereby providing AL-Net with information different from scales for detecting tampered areas and enhancing the detection performance of AL-Net.

[0106] Since directly applying the autocorrelation module to the high-resolution feature map will bring huge computational overhead, most CMFD networks currently only use the autocorrelation module on the low-resolution feature map. However, this inevitably leads to these networks ignoring the similarity between high-resolution features, reducing the performance of the model. In order to solve this problem, the present invention proposes a low-cost dual-step self-correlation calculation (LD-SCC), which first screens the high-similar regions on the high-resolution feature map through the feature correlation on the low-resolution feature map, excludes redundant information, and then only calculates the correlation of features between high-similar regions, which not only greatly reduces the computational overhead of AL-Net, but also maintains excellent detection performance.

[0107] The existing CMFD network only uses the correlation features after the autocorrelation module to obtain the tampering detection results, ignoring the local context features extracted by the backbone network, which leads to the inaccurate prediction of the edge and interior of the tampered area by the model. In order to solve this problem, the present invention proposes a bi-branch adaptive fusion module (BAFM), which uses a lightweight multi-scale dilated convolution module (MDCM) to adaptively fuse the local features before LD-SCC and the correlation features after LD-SCC, thereby improving the detection performance of AL-Net.

[0108] The present invention combines these three structures to design a lightweight, low computational overhead, and high-precision CMFD network AL-Net. Experimental results on four public datasets show that AL-Net is superior to the above-mentioned related CMFD networks in terms of detection accuracy, number of parameters, and amount of computation.

[0109] Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians of the art without making creative work are within the scope of protection of the present application.

Claims

1. Adaptive sampling mechanism and two-step autocorrelation copy-move tampering detection method, characterized by: It includes the model construction part, model training and reasoning part. The specific steps are as follows: Step 1: Construct a copy-move tampering detection network model with an adaptive sampling mechanism and two-step autocorrelation. The network model performs the following steps: Step 1-1, feature extraction: input the image to be detected into AL-Net with ASM-Net as the backbone network to extract low-level features and high-level features for detecting different area sizes; ASM-Net consists of 4 stages. The first two stages are composed of a "CBG 3-2" and two "CBG 3-1" blocks, which are used to extract low-level features on the input image, so as to help AL-Net better detect small-area tampering areas. The last two stages are composed of a "CBG 3-2" and three ASM blocks that can automatically adjust the number and position of sampling points, which are used to adaptively extract high-level features on the input image, so as to help AL-Net better detect large-area tampering areas. The feature maps output by the four stages of ASM-Net are F1, F2, F3 and F4 respectively. Among them, CBG 3-2 represents a convolution block composed of a standard convolution layer of 3×3 size and stride 2, a Batch Normalization layer and a GELU activation function; CBG 3-1 represents a convolution block composed of a standard convolution layer of 3×3 size and stride 1, a BatchNormalization layer and a GELU activation function; ASM block consists of an adaptive sampling mechanism ASM, two point convolutions PWConv and cross-layer connections. Step 1-2, feature fusion: Input the obtained feature maps F1, F2, F3 and F4 into the feature fusion module FFM for feature fusion to obtain the output feature map F f ; Step 1-3, correlation feature extraction: extract the feature map F obtained in step 1-2 f Input low-computation two-step autocorrelation LD-SCC to extract correlation information in feature maps; Step 1-4, tampering area positioning: Step 1-2 to obtain the fusion feature map F f Correlation feature map F obtained from step 1-3 d Input to the dual-branch adaptive fusion module BAFM for fusion; the first branch of BAFM converts F d Input MDCM for multi-scale feature extraction, and the second branch extracts F f The input point convolution is used to exchange information between channels. The output results of the two branches will be spliced ​​in the channel dimension and input into MDCM together. MDCM will comprehensively consider F d The correlation characteristics in F f The multi-level features in , and multi-scale feature extraction is performed on both, so as to obtain F d and F f Generate weights and retain the key information of both. BAFM splits the weights in the channel dimension to obtain two tensors with the same shape as the feature map, and weights the two feature maps. BAFM concatenates the two weighted feature maps in the channel dimension and inputs them into the lightweight FRSU for feature fusion to obtain the tampering area positioning detection result of AL-Net. Step 2: Model training and reasoning: Step 2-1, data preprocessing: read the training samples from the training set of the USC-ISI dataset and perform preprocessing to obtain the training data required for training; preprocessing includes resizing, data enhancement and normalization; Step 2-2, model training: randomly divide the training data into multiple batches, use the cross entropy loss function to guide the AL-Net training process; select the stochastic gradient descent method SGD to adjust the model parameters by estimating the gradient of the loss function to the parameters to optimize the performance; after a set number of rounds of traversal training, complete the model training and obtain the final model; Step 2-3, model reasoning: perform preprocessing on the image to be detected to make it completely consistent with the parameters used in model training, and then input it into the final model to analyze the image through forward propagation to estimate the probability of each pixel belonging to the tampered area and display it in the form of a probability map. In the probability map, all pixels with predicted probabilities exceeding the set value are marked as tampered pixels, completing the accurate identification of the tampered area of ​​the image.

2. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 1 is characterized in that: In step 1-1, the adaptive sampling mechanism ASM performs the following steps: Step 1-1-1, first pass the input feature map Ein through "Convα-1" to obtain the intermediate feature map E mid , the shape is "Convα-1" indicates a standard convolutional layer with a kernel size of α and a stride of 1; and Represents the input feature map E in The width and height of 2 +1 is the number of output channels of "Conv α-1"; Step 1-1-2, intermediate feature map E mid Each sampling point E on mid (i, j) is split into the first channel and the remaining β 2 There are two parts of the channel, where i and j are the horizontal and vertical coordinates of the sampling point Emid (i, j), respectively. E mid The value of the first channel in (i, j) represents the number of sample points at position (i, j) predicted by "Conv α-1", and The remaining β 2 The predicted values ​​of the channels will be reshaped into a 2D matrix of shape β×β Each value in (m, n∈[1,β]), represents the weight of each feature point in the β×β range centered at coordinate (i, j) predicted by "Conv α-1"; Step 1-1-3: Since the number of sampling points required for each location is different, ASM calculates the number of sampling points based on the predicted number of sampling points. filter Center front The largest weight value is assigned to the rest of the weight values, and the new weight matrix is ​​obtained Its formula is shown in formula (4): in, express Middle Large weight values, and Represents rounding down and rounding up respectively; Step 1-1-4, ASM will Perform L1-Normalization and get And with the input feature map E in By performing weighted summation, we get the output feature map E out , its formula is shown in formula (5):

3. The copy-move tampering detection method based on the adaptive sampling mechanism and the two-step autocorrelation according to claim 2 is characterized in that: α and β are set to 5 in the third stage of ASM-Net and are set to 3 in the fourth stage of ASM-Net.

4. The copy-move tampering detection method based on the adaptive sampling mechanism and the two-step autocorrelation according to claim 2 is characterized in that: Step 1-2 specifically includes the following steps: Step 1-2-1, the feature map F4 is input into two "Conv 3-1" for further feature extraction; Conv 3-1 represents a standard convolution with a size of 3×3 and a step size of 1; Step 1-2-2, input the result obtained in step 1-2-1 into the global average pooling layer GAP to obtain the weight; Step 1-2-3, multiply the weight obtained in step 1-2-2 by the channel spliced ​​F1, F2, F3 and F4 element by element to obtain the weighted spliced ​​feature map, retaining the key information in the four feature maps; Step 1-2-4: Input the weighted concatenated feature map into "CBG 1-1" and two "FasterNet blocks" for feature fusion to obtain the output feature map F f .

5. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 2 is characterized in that: Steps 1-3 specifically include the following steps: Step 1-3-1, LD-SCC takes the input feature map F f Downsampling to obtain low-resolution feature maps Then according to The correlation between the two features in the filter is f Highly similar regions in Step 1-3-2, LD-SCC only in F f The autocorrelation calculation SCC is performed between the features of the highly similar regions to obtain the output correlation feature map F d .

6. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 5 is characterized in that: Step 1-3-1 specifically includes the following steps: Step 1-3-1-1, LD-SCC first performs The input feature map F f Perform k-fold average pooling downsampling AP to obtain a low-resolution feature map Shape in Step 1-3-1-2, LD-SCC will Further reshape into the shape of The matrix And calculate The cosine similarity CS between two feature points is calculated as shown in formula (6): Among them, T represents the transpose operation, Represents The cosine similarity between the u-th feature point and the v-th feature point, which also represents F f The similarity between the u-th k×k pixel block and the v-th k×k pixel block; Step 1-3-1-3, LD-SCC divides F into f Transform into shape Tensor for For each k×k pixel block in , LD-SCC is based on The similarity information in is extracted to obtain the t k×k pixel blocks with the highest similarity, and the shape is Tensor 7. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 5 is characterized in that: Step 1-3-2 specifically includes the following steps: Step 1-3-2-1, LD-SCC will Each feature point in is only similar to the feature points of the t pixel blocks with the highest similarity to the pixel block where it is located. Step 1-3-2-2, to establish Each pixel block in To understand the connection between the t pixel blocks in , LD-SCC first transforms Transpose, then and Perform matrix multiplication, the formula is shown in formula (7): Step 1-3-2-3, LD-SCC will The shape is restored to Then sort along the channel dimension and retain the first g largest values ​​along the channel dimension, resulting in a shape of The output feature map of 8. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 1 is characterized in that: In steps 1-4, the multi-scale dilated convolution module DWDC first uses three serially connected deep dilated convolution layers DWDC to extract multi-scale features on the input feature map to obtain three feature maps; the three feature maps are respectively subjected to point convolution for information exchange between channels, and are then subjected to batch normalization (BN) layer and nonlinear GELU activation function; finally, the three feature maps are concatenated in the channel dimension and fused with the input point convolution layer to obtain the final output result.

9. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 1 is characterized in that: The data preprocessing of step 2-1 includes the following steps: (1) Reading data: Read training samples from the training set of the USC-ISI dataset. Each sample contains the original image and the corresponding label. The label contains the category information of each pixel in the original image. (2) Resizing: Scale the original image and the corresponding label at the same time, and adjust the length and width of all images to 320×320; (3) Data enhancement: The input image and the corresponding label are randomly rotated and horizontally flipped at a certain probability to enhance the model's ability to detect different angles and symmetries of the image; (4) Normalization: Use ImageNet statistics to normalize images. Using the mean and variance of the ImageNet dataset for normalization can help the model learn more efficiently, thereby accelerating convergence.

10. The copy-move tampering detection method based on the adaptive sampling mechanism and double-step autocorrelation according to claim 1 is characterized in that: The training process of step 2-2 is set to 60 rounds, and the setting value of step 2-3 is 0.5.

Images