Contour-guided mutual aggregation and duplication mobile remote sensing image tampering detection method

Abstract

The present invention belongs to the technical field of image tampering detection, and particularly relates to a mutually aggregated copy-and-move remote sensing image tampering detection method based on contour guidance, comprising step 1, inputting a copy-and-move tampered image; step 2, similar region detection; step 3, tampered edge detection; step 4, information fusion; step 5, contour detection; step 6, class information fusion; step 7, contour-guided region modeling; step 8, outputting a tampered detection image. The present invention solves the problem of information redundancy in the feature extraction process and solves the problem of global modeling feature confusion.

Description

Technical Field

[0001] The invention belongs to the technical field of image tampering detection, and in particular relates to a contour-guided mutual aggregation copy mobile remote sensing image tampering detection method. Background Art

[0002] The copy-move operation is one of the common image tampering methods. This operation copies an area in the image (source domain) and pastes it to another area of ​​the same image (tampered domain) to change the meaning of the entire image. Since the tampered area and the source domain in the copy-move tampering detection image are from the same image, the brightness and contrast of the two are highly similar, which poses a challenge to copy-move forgery detection. In addition, forgers usually perform post-processing operations on the tampered image, such as blurring and scaling, which also becomes a major problem in copy-move detection.

[0003] At present, the cutting-edge tampering detection method uses the Two-stageNet network architecture, which essentially only calculates the correlation between pixels to obtain similar area features to achieve tampering area positioning, and improves the accuracy of the tampering area through subsequent refinement methods. However, this method has the following problems:

[0004] First, the autocorrelation calculation method of the image cannot accurately obtain the tampering information and edge information. The tampered area is obtained by copying the original domain to other locations in the image, so there are differences in the surrounding pixels of the source domain and the tampered domain. In order to hide the traces of tampering, the tamperer usually performs post-processing operations such as blurring on the tampered image. After blurring, the difference in the surrounding pixels of the source domain and the tampered domain will lead to differences in the edge area information of similar target objects. At this time, only detecting the pixel area through pixel-by-pixel similarity calculation is very likely to cause errors in the detection of the target edge area, and the integrity of the detection of similar objects cannot be guaranteed. Secondly, the source domain has not undergone any operation, it is just a part of the original image, and it is more likely to cause errors in the detection and edge extraction of the source domain.

[0005] Second, the global modeling of features can easily cause feature confusion and fail to obtain accurate detection results. Specifically, existing methods fail to consider internal modeling of regions from the perspective of objects, including modeling within the tampered domain, modeling within the tampered edge domain, and modeling within the background domain to refine the detection results. The existing method Two-stageNet refines the coarse similarity region obtained by similarity calculation by adding a unique convolution structure. It only further extracts features from the coarse features, and does not model and optimize different objects from the essence of the related regions. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a contour-guided mutual aggregation copy mobile remote sensing image tampering detection method. In the feature extraction stage, feature difference and jump multiplication are used to solve the problem of information redundancy in the feature extraction process, while ensuring that important information in the features will not be lost; a method for modeling within the object domain is proposed, in which the contour object domain (contour class and object class) and the contour background domain (contour class and background class) are modeled in the contour-guided area respectively, to solve the problem of confusion in global modeling features.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] The contour-guided mutual aggregation copy mobile remote sensing image tampering detection method comprises the following steps:

[0009] Step 1, input the copy move tampered image;

[0010] Step 2: Similar area detection:

[0011] The input image is processed by the similar region detection module to obtain a rough similar region Fc;

[0012] Step 3: Tampering edge detection:

[0013] The input image is processed by the edge detection module to obtain the tampered edge Fe;

[0014] Step 4: Information fusion:

[0015] After obtaining the rough similar region Fc and the tampered edge Fe, the tampered region feature Fst is obtained by information intersection fusion, and then the rough similar region Fc and the tampered region feature Fst are used to make a difference to obtain the tampered source domain Fs, and the tampered edge Fe and the tampered region feature Fst are fused and multiplied to obtain the tampered region Ft;

[0016] Step 5: Contour detection:

[0017] After obtaining the tampered area Ft and the tampered source domain Fs, the wavelet transform edge detection algorithm is used to obtain the respective tampered contours Fte and source domain contours Fse. After obtaining the contour information, the tampered area Ft is subtracted from the tampered contour Fte to obtain the tampered object Ftb. Similarly, the tampered source domain Fs is subtracted from the source domain contour Fse to obtain the source domain object Fsb.

[0018] Step 6: Class information fusion:

[0019] After obtaining the tampered object Ftb, tampered contour Fte, source domain object Fsb and source domain contour Fse, the four are added and fused, and each element is assigned contour class K, object class D, and background class B to obtain an image L;

[0020] Step 7: Modeling the contour-guided area:

[0021] Use contour class K to guide object class D, and contour class K to guide background class B to perform separate modeling within the area;

[0022] Step 8: Output tampering detection image:

[0023] The results of modeling in step 7 are fused, and the tampered source domain Fs and tampered area Ft are used for auxiliary distinction to obtain the tampering detection image.

[0024] Furthermore, in step 2, the specific steps of the similar area detection module are as follows:

[0025] Step 201: First, input the image into the encoder to extract its features. The last layer of convolution uses a dilated convolution to increase the receptive field. Then, three convolution operations with different convolution rates are performed on it to obtain features F1, F2, and F3.

[0026] Step 202: Feature F1 passes through the element weight attention module to obtain feature A1, and A1 is multiplied by F1 to obtain feature F'1; at the same time, A1 passes through the filter to perform a 1-A1 operation to obtain A'1, and then A'1 is multiplied by F2 to obtain F 2d , the same F 2d After inputting the element weight attention module, we get A2, A2 and F 2d After multiplication, we get F'2. At the same time, A2 is filtered, 1-A2 is performed, and then multiplied with F3 to get F'3.

[0027] Among them, the weighted attention is a parallel structure, including two parts: the element weight distribution branch composed of the softmax function and the position attention branch;

[0028] Step 203, concatenate the obtained F'1, F'2 and F'3, and then perform L2 normalization operation, position attention calculation, autocorrelation calculation, Relu activation function, L2 normalization, and finally upsampling to obtain a rough similar area Fc of the same size as the input image, which only contains three parts: source domain object, tampered domain object and background object.

[0029] Furthermore, in step 3, the specific steps of edge detection module processing are as follows:

[0030] The input image passes through different feature processing stages of the ResNet network to obtain features M1, M2, M3, and M4. These four features are then passed through a filter and a residual module respectively to obtain features Fe1, Fe2, Fe3, and Fe4. Fe1 is subtracted from Fe2 and then passed through a residual module to obtain F21. F21 is then subtracted from Fe3 to obtain F32. Finally, the obtained F21, F32, and Fe4 are adaptively fused to obtain the tampered edge Fe.

[0031] Furthermore, step 7 is as follows: the contour class K and the object class D are respectively subjected to two linear layer operations to obtain D1, D2 and K1, K2, and then the D1 and K1 features are subjected to attention operation to obtain AKD, and then convolution operations are performed respectively, and the features after each convolution are respectively multiplied with the features originally subjected to the linear layer D2, K2, and the multiplication results are concatenated to obtain KD. In the concatenated result KD, the object class has both the guidance of the contour class to the object class and the important information in the object class; similarly, the result KB after the contour class and background class modeling is obtained.

[0032] Furthermore, step 8 is as follows: KD and KB are adaptively fused to obtain a binary classification tampering detection result T with a complete structure, and the binary classification result is auxiliaryly distinguished using the tampering source domain Fs and the tampering area Ft, and finally a three-classification tampering detection image is obtained.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] (1) In the process of image feature extraction, three convolution rates of different sizes are used to perform feature subtraction (subtraction of large convolution rate features with low convolution rate features of the previous level) and jump multiplication (subtraction of large convolution rate features with low convolution rate features of the previous level, and then multiplication with large convolution rate features) to solve the problem of information redundancy in the feature extraction process, and jump multiplication ensures that important information in the features will not be lost. At the same time, in order to fully consider the information of each element in the feature extraction process, a position attention mechanism is designed. Considering the different weights of elements at the same position and different channels, a weight value should not be used in general, and an element weight allocation mechanism is designed. The two are combined into the current element weight attention module. Therefore, feature subtraction and jump multiplication solve the problem of information redundancy in the feature extraction process, while ensuring that important information in the features will not be lost. Element weight attention solves the problem of single weight allocation at the element position and at different channels at the same position.

[0035] (2) A method for modeling in the object domain is proposed. After obtaining the tampered object, tampered contour, source domain object, and source domain contour, the four are fused. Then, the contour-guided region modeling is performed in the contour object domain (contour class and object class) and the contour background domain (contour class and background class) to solve the problem of confusion in global modeling features. By aggregating the features in the domain, the object information can contain both the guidance of the contour to the object information and the important information in the object. Similarly, the background information can contain both the guidance of the contour to the background information and the important information in the background. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0037] Figure 1 It is a structural diagram of a similar region detection module of the present invention;

[0038] Figure 2 It is a structural diagram of the edge detection module of the present invention;

[0039] Figure 3 is a schematic diagram of the filter structure in the edge detection module of the present invention;

[0040] Figure 4 Schematic diagram of the structure of the residual module in the edge detection module of the present invention;

[0041] Figure 5 is a network structure diagram of the present invention;

[0042] Figure 6 is a boundary perception map according to an embodiment of the present invention;

[0043] The represents feature addition, Indicates that the characteristics are poor. Represents feature multiplication. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0045] Combination Figure 1-Figure 5 This embodiment provides a contour-guided mutual aggregation copy mobile remote sensing image tampering detection method, comprising the following steps:

[0046] Step 1. Input the copy-move tampered image: The input is a copy-move tampered image with a size of H*W.

[0047] Step 2: Similar area detection:

[0048] The input image is processed by the similar region detection module to obtain a rough similar region Fc.

[0049] like Figure 1 As shown, the specific steps of the similar area detection module are as follows:

[0050] Step 201: First, input the image into the encoder to extract its features. The last layer of convolution uses a dilated convolution to increase the receptive field. Then, three convolution operations with different convolution rates are performed on it to obtain features F1, F2, and F3.

[0051] Step 202: In order to fully extract feature information to reflect the information inconsistency of different elements on the channel, a weighted attention module is designed. The weighted attention is a parallel structure, including an element weight allocation branch composed of a softmax function and a position attention branch.

[0052] Feature F1 passes through the element weighted attention module to obtain feature A1, and A1 is multiplied by F1 to obtain feature F'1; at the same time, A1 passes through the filter to perform a 1-A1 operation to obtain A'1, and then A'1 is multiplied by F2 to obtain F 2d , the same F 2d After inputting the element weight attention module, we get A2, A2 and F 2d After multiplication, we get F'2, and A2 is filtered and multiplied by F3 to get F'3. In this way, while eliminating redundant information and retaining important information, the characteristics of different channels of each position element are taken into account.

[0053] Step 203, concatenate the obtained F'1, F'2 and F'3, and then perform L2 normalization operation, position attention calculation, autocorrelation calculation, Relu activation function, L2 normalization, and finally upsampling to obtain a rough similar area Fc of the same size as the input image, which only contains three parts: source domain object, tampered domain object and background object.

[0054] Step 3: Tampering edge detection:

[0055] For the input image, it should be input not only into the similar region detection module, but also into the edge detection module.

[0056] The input image is processed by the edge detection module to obtain the tampered edge Fe.

[0057] like Figure 2 As shown, the specific steps of edge detection module processing are as follows:

[0058] The input image is input into a ResNet network. The ResNet network can be implemented using the structure of the existing technology. In this embodiment, the ResNet network is set to four feature extraction stages ( Figure 2 Represented as module 1, module 2, module 3, and module 4 respectively), after different feature processing stages of the ResNet network, features M1, M2, M3, and M4 are obtained, and then these four features are respectively passed through a filter and a residual module to obtain features Fe1, Fe2, Fe3, and Fe4. Fe1 is subtracted from Fe2 and then passed through a residual module to obtain F21, and then the obtained F21 is subtracted from Fe3 to obtain F32. Finally, the obtained F21, F32, and Fe4 are adaptively fused to obtain the tampered edge Fe.

[0059] The principles of the filter and the residual module are the principles of the prior art. In this embodiment, the filter is as follows: Figure 3 As shown in , it includes filter convolution, batch normalization, L2 normalization and activation layer (activation function). The input of the filter is processed by filter convolution, batch normalization, L2 normalization and activation layer (activation function) in sequence, and then multiplied with the input of the filter again. The result is used as the output of the filter. Figure 4 As shown, it includes a convolution layer, batch normalization, and an activation layer (activation function). In this embodiment, the input of the residual module first undergoes a 1×1 convolution operation to extract features, and then undergoes a 3×3 convolution, batch normalization, activation function, and 3×3 convolution operation in sequence. The obtained features are again added to the features extracted by the 1×1 convolution operation, and the result of the addition is used as the output of the residual module.

[0060] Step 4: Information fusion:

[0061] like Figure 5 As shown, after obtaining the rough similar area Fc and the tampered edge Fe, the tampered area feature Fst is obtained by information intersection and fusion, and then the rough similar area Fc and the tampered area feature Fst are subtracted to obtain the tampered source domain Fs, and the tampered edge Fe and the tampered area feature Fst are fused and multiplied to obtain the tampered area Ft.

[0062] Step 5: Contour detection:

[0063] like Figure 5 As shown, after obtaining the tampered area Ft and the tampered source domain Fs, the wavelet transform edge detection algorithm is used to obtain their respective tampered contours Fte and source domain contours Fse. After obtaining the contour information, the tampered area Ft is subtracted from the tampered contour Fte to obtain the tampered object Ftb. Similarly, the tampered source domain Fs is subtracted from the source domain contour Fse to obtain the source domain object Fsb.

[0064] Through the above method, the tampered object Ftb, the tampered outline Fte, the source domain object Fsb and the source domain outline Fse are obtained.

[0065] Step 6: Class information fusion:

[0066] like Figure 5 As shown in FIG, after obtaining the tampered object Ftb, the tampered contour Fte, the source domain object Fsb and the source domain contour Fse, the four are added and fused, and each element is assigned a contour class K, an object class D, and a background class B to obtain an image L; among the different objects in this image, the object boundaries are not connected to each other, as shown in FIG. Figure 6 A boundary perception map (example map) is shown for easy understanding. Figure 6 In the figure, circles represent object 1 pixels, squares represent boundary pixels, and triangles represent object 2 pixels.

[0067] Step 7: Modeling the contour-guided area:

[0068] Use the contour class K to guide the object class D, and the contour class K to guide the background class B to perform separate modeling within the area.

[0069] like Figure 5 As shown in the figure, the details are as follows: the contour class K and the object class D are respectively subjected to two linear layer operations to obtain D1, D2 and K1, K2, and then the D1 and K1 features are subjected to attention operation to obtain AKD, and then convolution operations are performed respectively. The features after convolution are respectively multiplied with the features originally subjected to the linear layer D2, K2, and the multiplication results are concatenated to obtain KD. In the concatenated result KD, the object class has both the guidance of the contour class to the object class and the important information in the object class.

[0070] Similarly, the result KB after contour and background modeling is obtained.

[0071] Step 8: Output tampering detection image:

[0072] The results of modeling in step 7 are fused, and the tampered source domain Fs and tampered area Ft are used for auxiliary distinction to obtain the tampering detection image.

[0073] Specifically, KD and KB are adaptively fused to obtain a complete binary classification tampering detection result T. In order to better distinguish the source area and the tampered area, the tampered source domain Fs and the tampered area Ft are used to assist in distinguishing the binary classification results, and finally a three-classification tampering detection image is obtained, as shown in Figure 5 The output in .

[0074] As a preferred embodiment, a contour-guided mutual aggregation replication mobile remote sensing image tampering detection network (system) is also provided, and the network structure is as follows: Figure 5As shown, it mainly includes edge detection module, similar area detection module, information intersection and fusion, wavelet transform edge detection, contour-guided area modeling and other parts. The data processing process of each part can be found in the tampering detection method recorded above, which will not be repeated here.

[0075] Different from the previous copy-move tampering detection, the present invention proposes an element weighted attention and edge detection method to obtain accurate tampering information and edge information, and uses attention to model inside the object through the in-domain modeling method to solve the problem of global modeling feature confusion.

[0076] The loss function of the present invention is described as follows:

[0077] In order to optimize the proposed network, the total loss function is defined as:

[0078] ;

[0079] Among them, L edg represents the edge loss calculated using the sample similarity metric function, L det represents the detection loss calculated using cross entropy. α and β are learnable parameters.

[0080] ;

[0081] Among them, y i ∈{0,1} indicates whether the i-th pixel in the image is an edge pixel of the tampered area. It represents the probability that the pixel i is predicted to be a tampered edge, and W and H represent the number of pixels contained in the width and height of the image respectively.

[0082] ;

[0083] Among them, p i ∈{0,1} indicates whether the i-th pixel belongs to the area to be detected. It indicates the probability that the pixel is predicted to belong to the target area, w1 and w2 are hyper parameters.

[0084] The network was implemented using the PyTorch deep learning framework. The RSCMSD dataset was used for experiments. The dataset was divided into 8:1:1 for training, validation, and testing. During training, the batch size was set to 16, and the Adam optimizer was used for optimization. The initial learning rate was set to 1e-2. All experiments were performed on RTX3090 devices.

[0085] In summary, the present invention (1) uses different convolution rates to extract features in the feature extraction stage. After obtaining the features, the high-order and low-order are subtracted to eliminate the redundant information in the high-order. The subtraction result is then multiplied with the high-order information to prevent the loss of important information features. At the same time, an element weight attention module is designed, which takes into account both the position information of the elements and the weight influence of the elements on different channels. (2) An intra-domain modeling method is proposed, which changes the traditional global modeling method of copying mobile image tampering detection into an internal modeling method in three areas: the background class, the contour class, and the object class. A modeling module in the contour-guided area is designed to solve the problem of global modeling feature confusion, thereby accurately detecting the source domain, tampering domain, and background domain by aggregating intra-domain features and separating inter-domain features.

[0086] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A contour-guided mutual aggregation copy mobile remote sensing image tampering detection method, characterized in that: The following steps are involved: Step 1, input the copy move tampered image; Step 2: Similar area detection: The input image is processed by the similar region detection module to obtain a rough similar region Fc; In step 2, the specific steps of the similar area detection module are as follows: Step 201: First, input the image into the encoder to extract its features. The last layer of convolution uses a dilated convolution to increase the receptive field. Then, three convolution operations with different convolution rates are performed on it to obtain features F1, F2, and F3. Step 202: Feature F1 passes through the element weight attention module to obtain feature A1, and A1 is multiplied by F1 to obtain feature F'1; at the same time, A1 passes through the filter to perform a 1-A1 operation to obtain A'1, and then A'1 is multiplied by F2 to obtain F 2d , the same F 2d After inputting the element weight attention module, we get A2, A2 and F 2d After multiplication, we get F'2. At the same time, A2 is filtered, 1-A2 is performed, and then multiplied with F3 to get F'3. Among them, the weighted attention is a parallel structure, including two parts: the element weight distribution branch composed of the softmax function and the position attention branch; Step 203, concatenate the obtained F'1, F'2 and F'3, and then perform L2 normalization operation, position attention calculation, autocorrelation calculation, Relu activation function, L2 normalization, and finally up-sampling to obtain a rough similar region Fc of the same size as the input image, which only contains three parts: the source domain object, the tampered domain object and the background object; Step 3: Tampering edge detection: The input image is processed by the edge detection module to obtain the tampered edge Fe; In step 3, the specific steps of edge detection module processing are as follows: The input image is processed by the ResNet network at different feature processing stages to obtain M1, M2, M3, and M4. These four features are then passed through a filter and a residual module to obtain features Fe1, Fe2, Fe3, and Fe4. Fe1 is subtracted from Fe2 and then passed through a residual module to obtain F21. F21 is then subtracted from Fe3 to obtain F32. Finally, the obtained F21, F32, and Fe4 are adaptively fused to obtain the tampered edge Fe. Step 4: Information fusion: After obtaining the rough similar region Fc and the tampered edge Fe, the tampered region feature Fst is obtained by information intersection fusion, and then the rough similar region Fc and the tampered region feature Fst are used to make a difference to obtain the tampered source domain Fs, and the tampered edge Fe and the tampered region feature Fst are fused and multiplied to obtain the tampered region Ft; Step 5: Contour detection: After obtaining the tampered area Ft and the tampered source domain Fs, the wavelet transform edge detection algorithm is used to obtain the respective tampered contours Fte and source domain contours Fse. After obtaining the contour information, the tampered area Ft is subtracted from the tampered contour Fte to obtain the tampered object Ftb. Similarly, the tampered source domain Fs is subtracted from the source domain contour Fse to obtain the source domain object Fsb. Step 6: Class information fusion: After obtaining the tampered object Ftb, tampered contour Fte, source domain object Fsb and source domain contour Fse, the four are added and fused, and each element is assigned contour class K, object class D, and background class B to obtain an image L; Step 7: Modeling the contour-guided area: Use contour class K to guide object class D, and contour class K to guide background class B to perform separate modeling within the area; Step 7 is as follows: the contour class K and the object class D are respectively subjected to two linear layer operations to obtain D1, D2 and K1, K2, and then the D1 and K1 features are subjected to attention operations to obtain AKD, and then convolution operations are performed respectively, and the features after convolution are respectively multiplied with the features that originally passed through the linear layer D2, K2, and the multiplication results are spliced ​​to obtain KD. In the spliced ​​result KD, the object class contains both the guidance of the contour class to the object class and the important information of the object class; similarly, the result KB after the contour class and background class modeling is obtained; Step 8: Output tampering detection image: The results of modeling in step 7 are fused, and the tampered source domain Fs and tampered area Ft are used for auxiliary distinction to obtain the tampering detection image.

2. The contour-guided mutual aggregation copy mobile remote sensing image tampering detection method according to claim 1 is characterized in that: Step 8 is as follows: KD and KB are adaptively fused to obtain a binary classification tampering detection result T with a complete structure, and the binary classification result is auxiliaryly distinguished using the tampering source domain Fs and the tampering area Ft, and finally a three-classification tampering detection image is obtained.

Images