A method for image tampering detection based on deep learning

Abstract

The present invention discloses an image tampering detection method based on deep learning, comprising the following steps: acquiring an image data set and preprocessing the data set to obtain a preprocessed training set; constructing a coarse-grained channel compensation module; constructing a fine-grained noise feature separation attention mechanism module; building an end-to-end encoding and decoding image tampering detection neural network model, adding the coarse-grained channel compensation module and the fine-grained noise feature separation attention mechanism module, inputting a tampering feature map into the neural network model to obtain a tampering region segmentation probability map; using the preprocessed training set, optimizing a binary cross entropy loss function using a back propagation algorithm, iteratively training the neural network model to obtain a trained image tampering detection neural network model, and using the trained image tampering detection neural network model to complete image tampering detection. The method solves the problem of feature loss in the convolution and pooling processes of the existing image tampering detection network model.

Description

Technical Field

[0001] The present invention relates to the field of image forensics, and in particular to an image tampering detection method based on deep learning. Background Art

[0002] Digital images are important information carriers, and their authenticity and integrity should be protected. Currently, common tampered images can be divided into three categories: cropping and splicing, copying and moving, and content modification. To better detect tampered images, researchers have proposed image tampering detection technology. Image tampering detection technology based on deep learning detects tampered images by designing deep neural networks. Depending on the different tampering feature extraction strategies, it is specifically divided into two categories: the first category is a deep network that combines traditional image processing technology, using image processing technology to extract feature information during pre-training or convolution; the second category is a structural detection network, which enhances the performance of the tampering detection network through the design of the network structure, such as using skip connections or attention mechanisms to extract tampering feature information. By adding branch structures with different input streams to the structural detection network, the network's detection generalization and robustness can be enhanced.

[0003] Current image tampering detection methods based on deep learning ignore the possibility of feature information loss in the convolution and pooling processes, resulting in a decrease in detection accuracy. In addition, when there are multiple tampered areas in the same image, the existing algorithms do not distinguish between tampered features and non-tampered features during feature extraction during the convolution and pooling processes, resulting in the loss of feature boundary information of the tampered image and treating the two tampered areas as the same target, resulting in poor detection accuracy. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present invention provides an image tampering detection method based on deep learning, which solves the problem of feature loss during the convolution and pooling processes in the existing image tampering detection network model.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: to provide an image tampering detection method based on deep learning, comprising the following steps:

[0006] S1. Obtain an image dataset and preprocess it to obtain a preprocessed training set;

[0007] The image dataset includes a plurality of tampered images containing tampered area masks;

[0008] S2, constructing a coarse-grained channel compensation module;

[0009] S3. Construct a fine-grained noise feature separation attention mechanism module;

[0010] S4. Based on the coarse-grained channel compensation module and the fine-grained noise feature separation attention mechanism module, an end-to-end encoding and decoding image tampering detection neural network model is constructed;

[0011] S5. Use the preprocessed training set and the back-propagation algorithm to optimize the binary cross entropy loss function, iteratively train the end-to-end encoding and decoding image tampering detection neural network model, obtain the trained image tampering detection neural network model, and use the trained image tampering detection neural network model to complete image tampering detection.

[0012] Furthermore: Step S1 includes the following sub-steps:

[0013] S11, obtain the CASIA2 tampered image dataset as the initial tampered image training set;

[0014] S12. Obtain the COCO image dataset, crop the objects in the COCO image dataset, and then splice them onto the CASIA2 tampered image dataset as a tampered image training set;

[0015] S13, preprocessing the images in the tampered image training set, and obtaining an image set of size C×H×W after scale transformation as the preprocessed training set;

[0016] Among them, C represents the number of channels, H represents the image length, and W represents the image width.

[0017] Furthermore, the method for processing the input image by the coarse granularity channel compensation module in step S2 includes the following steps:

[0018] S21. Perform local and global feature extraction on feature information of the input image using dilated convolutions of different convolution scales to obtain first feature maps of different scales, and perform channel splicing on the first feature maps of different scales to obtain a second feature map;

[0019] S22: Input the second feature map into a convolutional layer, and output the tampered feature map after channel compensation.

[0020] Furthermore, step S21 includes the following sub-steps:

[0021] S2101, using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 2 to process the input image to obtain a first local feature of the input image;

[0022] S2022. Process the input image using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 5 to obtain a second local feature of the input image.

[0023] S2023. Pass the input image through a convolution layer with a convolution kernel of 1×1, and perform absolute value processing on the output convolution eigenvalue to obtain a first eigenvalue;

[0024] S2024, passing the input image through a convolution layer with a convolution kernel of 1×1, and performing mean pooling processing on the output convolution eigenvalues ​​to obtain a second eigenvalue;

[0025] S2025. Perform channel concatenation on the first eigenvalue and the second eigenvalue to obtain a third eigenvalue;

[0026] S2026. Pass the third eigenvalue through a convolution layer with a convolution kernel of 1×1 to obtain a fourth eigenvalue.

[0027] S2027. Process the fourth eigenvalue using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 1 to obtain a global feature of the input image.

[0028] S2028. Use the first local feature of the input image, the second local feature of the input image, and the global feature of the input image as first feature maps of different scales, and perform channel splicing on the first feature maps of different scales to obtain a second feature map.

[0029] Furthermore, the method for processing the input image by the fine-grained noise feature separation attention mechanism module in step S3 includes the following steps:

[0030] S31, passing the input image through a convolution layer with a convolution kernel of 1×1 and performing normalization processing to obtain a third feature map;

[0031] S32, dividing the third feature map into two channels using a threshold, respectively representing two feature information greater than the threshold and less than the threshold, performing threshold truncation on the two divided channels, and performing channel splicing on the feature values ​​on both sides after the threshold truncation to obtain a fourth feature map;

[0032] S33, pass the input image through a convolution layer with a convolution kernel of 1×1, use the output convolution value as the input of the softmax function, and add 1 to the result output by the softmax function to obtain a fifth feature map;

[0033] S34, multiplying the attention weight of the fourth feature map by the fifth feature map to obtain a sixth feature map;

[0034] S35, using average pooling to process the input image to obtain a seventh feature map;

[0035] S36. Use the sixth feature map and the seventh feature map as inputs of the softmax function, add 1 to the result output by the softmax function, and output the actively enhanced feature map.

[0036] Furthermore, the end-to-end encoding and decoding image tampering detection neural network model in step S4 includes one input module, four encoding modules, and four decoding modules, namely encoding module E1, encoding module E2, encoding module E3, encoding module E4, decoding module D1, decoding module D2, decoding module D3, and decoding module D4;

[0037] The internal structure of each encoding module is the same, and the internal structure of each decoding module is the same; the connection relationship between the encoding modules and the decoding modules is as follows: the output end of the encoding module E1 is respectively connected to the input end of the encoding module E2 and the input end of the decoding module D4, the output end of the encoding module E2 is respectively connected to the input end of the encoding module E3 and the input end of the decoding module D3, the output end of the encoding module E3 is respectively connected to the input end of the encoding module E4 and the input end of the decoding module D2, and the output end of the encoding module E4 is connected to the output end of the decoding module D1;

[0038] The encoding module E2 is the next layer structure of the encoding module E1, the encoding module E3 is the next layer structure of the encoding module E2, the encoding module E4 is the next layer structure of the encoding module E3, the decoding module D2 is the next layer structure of the decoding module D1, the decoding module D3 is the next layer structure of the decoding module D2, the decoding module D4 is the next layer structure of the decoding module D3, and the decoding module D1 is the next layer structure of the encoding module E4;

[0039] The corresponding relationship between the encoding module and the decoding module is: the encoding module E1 corresponds to the decoding module D4, the encoding module E2 corresponds to the decoding module D3, the encoding module E3 corresponds to the decoding module D2, and the encoding module E4 corresponds to the decoding module D1.

[0040] Furthermore: the method for processing the input image by the input module includes the following steps:

[0041] S401, processing the input image through a double-layer zigzag dilated convolution structure to obtain an eighth feature map;

[0042] S402, performing a jump connection between the input image and the eighth feature map to obtain a ninth feature map;

[0043] S403 : Use the eighth feature map as the first input image of the encoding module E1 , use the ninth feature map as the second input image of the encoding module E1 , and output them to the encoding module E1 .

[0044] Furthermore: the method for processing the input image by the encoding module includes the following steps:

[0045] S411, performing maximum pooling processing on the first input image to obtain a tenth feature map;

[0046] S412: Input the tenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the tenth feature map to obtain an eleventh feature map;

[0047] S413, passing the eleventh feature map through a three-layer zigzag-shaped dilated convolution structure to obtain a twelfth feature map;

[0048] S414: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twelfth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a thirteenth feature map.

[0049] S415, using a convolution layer with a convolution kernel of 1×1 to obtain the fourteenth feature map;

[0050] S416, performing a jump connection on the thirteenth feature map and the fourteenth feature map to obtain a fifteenth feature map;

[0051] S417: Use the fifteenth feature map as the first input image of the next layer structure and the third input image of the corresponding decoding structure, and use the eleventh feature map as the second input image of the next layer structure.

[0052] Furthermore, the method for processing the input image by the decoding module includes the following steps:

[0053] S421, passing the first input image through a deconvolution layer to obtain a sixteenth feature map;

[0054] S422, performing channel concatenation on the sixteenth feature map and the third input image, and passing the resulting channel concatenation image through a double-layer zigzag dilated convolution structure to obtain a seventeenth feature map;

[0055] S423, pass the seventeenth feature map through a convolution layer with a convolution kernel of 1×1 to obtain an eighteenth feature map;

[0056] S424: Input the eighteenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the eighteenth feature map to obtain a nineteenth feature map;

[0057] S425, passing the nineteenth feature map through a three-layer zigzag dilated convolution structure to obtain a twentieth feature map;

[0058] S426: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twentieth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a twenty-first feature map.

[0059] S427, performing a jump connection on the twenty-first feature map and the eighteenth feature map to obtain a twenty-second feature map;

[0060] S428: Determine whether the current decoding module is decoding module D4:

[0061] If so, the twenty-second feature map is passed through a convolution layer with a convolution kernel of 1×1 to obtain a twenty-third feature map, and the twenty-third feature map is used as the output of the end-to-end encoding and decoding image tampering detection neural network model;

[0062] If not, proceed to step S429;

[0063] S429: Use the twenty-second feature map as the first input image of the next layer structure, and use the twenty-fourth feature map as the second input image of the next layer structure.

[0064] Furthermore: Step S5 includes the following sub-steps:

[0065] S51, randomly initialize the end-to-end encoding and decoding image tampering detection neural network model, and input the preprocessed training set into the network model in set batches for forward propagation to obtain the tampering area segmentation probability map after forward propagation;

[0066] S52, using the binary cross entropy loss function to calculate the loss value of the tampered region segmentation probability map after forward propagation and the tampered region mask corresponding to the input pre-trained training set image, and performing error back propagation to train the end-to-end codec image tampering detection network model, and selecting the model weight that minimizes the model loss within the training round as the trained image tampering detection neural network model;

[0067] S53: Use the trained image tampering detection neural network model to detect the input image to be detected, and complete the image tampering detection.

[0068] The beneficial effects of the present invention are:

[0069] 1. This paper proposes an image tampering detection network that combines coarse and fine granularity, uses a codec network model to detect tampered images, and realizes digital image forensics;

[0070] 2. The coarse-grained channel compensation module used in the present invention can avoid the loss of tampering boundary features during network convolution, compensate for the global convolution feature information, better adapt to the feature distribution of tampered images, and improve the detection accuracy and robustness of the network model;

[0071] 3. The present invention involves the detection of images with multiple tampered regions. The model uses a fine-grained noise feature separation attention mechanism. Through the feature information threshold segmentation strategy, it can better extract feature information from images with multiple tampered regions and improve the detection effect of images with multiple tampered regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Schematic diagram of the process of image tampering detection method based on deep learning in the present invention.

[0073] Figure 2 Schematic diagram of the coarse-grained channel compensation module of the present invention.

[0074] Figure 3 Schematic diagram of the fine-grained noise feature separation attention mechanism module of the present invention.

[0075] Figure 4 Schematic diagram of the neural network model structure for image tampering detection of the present invention.

[0076] Figure 5 Schematic diagram of the encoding module in the image tampering detection neural network model structure of the present invention.

[0077] Figure 6 Schematic diagram of the decoding module in the image tampering detection neural network model structure of the present invention.

[0078] Figure 7 This is a test image for single tampering region image detection according to the present invention.

[0079] Figure 8 This is a test image for detecting multiple tampering regions in the present invention. DETAILED DESCRIPTION

[0080] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0081] like Figure 1 As shown, in one embodiment of the present invention, a method for detecting image tampering based on deep learning is provided, comprising the following steps:

[0082] S1. Obtain an image dataset and preprocess it to obtain a preprocessed training set;

[0083] The image dataset includes a plurality of tampered images containing tampered area masks;

[0084] S2, constructing a coarse-grained channel compensation module;

[0085] S3. Construct a fine-grained noise feature separation attention mechanism module;

[0086] S4. Based on the coarse-grained channel compensation module and the fine-grained noise feature separation attention mechanism module, an end-to-end encoding and decoding image tampering detection neural network model is constructed;

[0087] S5. Use the preprocessed training set and the back-propagation algorithm to optimize the binary cross entropy loss function, iteratively train the end-to-end encoding and decoding image tampering detection neural network model, obtain the trained image tampering detection neural network model, and use the trained image tampering detection neural network model to complete image tampering detection.

[0088] The step S1 includes the following sub-steps:

[0089] S11, obtain the CASIA2 tampered image dataset as the initial tampered image training set;

[0090] S12. Obtain the COCO image dataset, crop the objects in the COCO image dataset, and then splice them onto the CASIA2 tampered image dataset as a tampered image training set;

[0091] S13, preprocessing the images in the tampered image training set, and obtaining an image set of size C×H×W after scale transformation as the preprocessed training set;

[0092] Among them, C represents the number of channels, H represents the image length, and W represents the image width.

[0093] like Figure 2 As shown, in one embodiment of the present invention, the method for processing the input image by the coarse granularity channel compensation module in step S2 includes the following steps:

[0094] S21. Perform local and global feature extraction on feature information of the input image using dilated convolutions of different convolution scales to obtain first feature maps of different scales, and perform channel splicing on the first feature maps of different scales to obtain a second feature map;

[0095] The step S21 includes the following sub-steps:

[0096] S2101, using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 2 to process the input image to obtain a first local feature of the input image;

[0097] S2022. Process the input image using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 5 to obtain a second local feature of the input image.

[0098] S2023. Pass the input image through a convolution layer with a convolution kernel of 1×1, and perform absolute value processing on the output convolution eigenvalue to obtain a first eigenvalue;

[0099] S2024, passing the input image through a convolution layer with a convolution kernel of 1×1, and performing mean pooling processing on the output convolution eigenvalues ​​to obtain a second eigenvalue;

[0100] S2025. Perform channel concatenation on the first eigenvalue and the second eigenvalue to obtain a third eigenvalue;

[0101] S2026. Pass the third eigenvalue through a convolution layer with a convolution kernel of 1×1 to obtain a fourth eigenvalue.

[0102] S2027. Process the fourth eigenvalue using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 1 to obtain a global feature of the input image.

[0103] S2028. Taking the first local feature of the input image, the second local feature of the input image, and the global feature of the input image as first feature maps of different scales, and performing channel concatenation on the first feature maps of different scales to obtain a second feature map;

[0104] S22: Input the second feature map into a convolutional layer, and output the tampered feature map after channel compensation.

[0105] In this embodiment, a ReLU activation layer, a convolution layer with a convolution kernel of 1×1, and a group normalization layer are used to obtain the output channel-compensated tampering feature map.

[0106] like Figure 3 As shown, in one embodiment of the present invention, the method for processing the input image by the fine-grained noise feature separation attention mechanism module in step S3 includes the following steps:

[0107] S31, passing the input image through a convolution layer with a convolution kernel of 1×1 and performing normalization processing to obtain a third feature map;

[0108] S32, dividing the third feature map into two channels using a threshold, respectively representing two feature information greater than the threshold and less than the threshold, performing threshold truncation on the two divided channels, and performing channel splicing on the feature values ​​on both sides after the threshold truncation to obtain a fourth feature map;

[0109] S33, pass the input image through a convolution layer with a convolution kernel of 1×1, use the output convolution value as the input of the softmax function, and add 1 to the result output by the softmax function to obtain a fifth feature map;

[0110] S34, multiplying the attention weight of the fourth feature map by the fifth feature map to obtain a sixth feature map;

[0111] S35, using average pooling to process the input image to obtain a seventh feature map;

[0112] S36. Use the sixth feature map and the seventh feature map as inputs of the softmax function, add 1 to the result output by the softmax function, and output the actively enhanced feature map.

[0113] like Figure 4 As shown, the end-to-end encoding and decoding image tampering detection neural network model in step S4 includes 1 input module, 4 encoding modules and 4 decoding modules, namely encoding module E1, encoding module E2, encoding module E3, encoding module E4, decoding module D1, decoding module D2, decoding module D3, and decoding module D4;

[0114] The internal structure of each encoding module is the same, and the internal structure of each decoding module is the same; the connection relationship between the encoding modules and the decoding modules is as follows: the output end of the encoding module E1 is respectively connected to the input end of the encoding module E2 and the input end of the decoding module D4, the output end of the encoding module E2 is respectively connected to the input end of the encoding module E3 and the input end of the decoding module D3, the output end of the encoding module E3 is respectively connected to the input end of the encoding module E4 and the input end of the decoding module D2, and the output end of the encoding module E4 is connected to the output end of the decoding module D1;

[0115] The encoding module E2 is the next layer structure of the encoding module E1, the encoding module E3 is the next layer structure of the encoding module E2, the encoding module E4 is the next layer structure of the encoding module E3, the decoding module D2 is the next layer structure of the decoding module D1, the decoding module D3 is the next layer structure of the decoding module D2, the decoding module D4 is the next layer structure of the decoding module D3, and the decoding module D1 is the next layer structure of the encoding module E4;

[0116] The corresponding relationship between the encoding module and the decoding module is: the encoding module E1 corresponds to the decoding module D4, the encoding module E2 corresponds to the decoding module D3, the encoding module E3 corresponds to the decoding module D2, and the encoding module E4 corresponds to the decoding module D1;

[0117] The method for processing the input image by the input module comprises the following steps:

[0118] S401, processing the input image through a double-layer zigzag dilated convolution structure to obtain an eighth feature map;

[0119] S402, performing a jump connection between the input image and the eighth feature map to obtain a ninth feature map;

[0120] S403 : Use the eighth feature map as the first input image of the encoding module E1 , use the ninth feature map as the second input image of the encoding module E1 , and output them to the encoding module E1 .

[0121] In one embodiment of the present invention, the method for processing the input image by the encoding module includes the following steps:

[0122] S411, performing maximum pooling processing on the first input image to obtain a tenth feature map;

[0123] S412: Input the tenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the tenth feature map to obtain an eleventh feature map;

[0124] S413, passing the eleventh feature map through a three-layer zigzag-shaped dilated convolution structure to obtain a twelfth feature map;

[0125] S414: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twelfth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a thirteenth feature map.

[0126] S415, using a convolution layer with a convolution kernel of 1×1 to obtain the fourteenth feature map;

[0127] S416, performing a jump connection on the thirteenth feature map and the fourteenth feature map to obtain a fifteenth feature map;

[0128] S417: Use the fifteenth feature map as the first input image of the next layer structure and the third input image of the corresponding decoding structure, and use the eleventh feature map as the second input image of the next layer structure.

[0129] In one embodiment of the present invention, the method for processing the input image by the decoding module includes the following steps:

[0130] S421, passing the first input image through a deconvolution layer to obtain a sixteenth feature map;

[0131] S422, performing channel concatenation on the sixteenth feature map and the third input image, and passing the resulting channel concatenation image through a double-layer zigzag dilated convolution structure to obtain a seventeenth feature map;

[0132] S423, pass the seventeenth feature map through a convolution layer with a convolution kernel of 1×1 to obtain an eighteenth feature map;

[0133] S424: Input the eighteenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the eighteenth feature map to obtain a nineteenth feature map;

[0134] S425, passing the nineteenth feature map through a three-layer zigzag dilated convolution structure to obtain a twentieth feature map;

[0135] S426: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twentieth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a twenty-first feature map.

[0136] S427, performing a jump connection on the twenty-first feature map and the eighteenth feature map to obtain a twenty-second feature map;

[0137] S428: Determine whether the current decoding module is decoding module D4:

[0138] If so, the twenty-second feature map is passed through a convolution layer with a convolution kernel of 1×1 to obtain a twenty-third feature map, and the twenty-third feature map is used as the output of the end-to-end encoding and decoding image tampering detection neural network model;

[0139] If not, proceed to step S429;

[0140] S429: Use the twenty-second feature map as the first input image of the next layer structure, and use the twenty-fourth feature map as the second input image of the next layer structure.

[0141] The step S5 comprises the following sub-steps:

[0142] S51, randomly initialize the end-to-end encoding and decoding image tampering detection neural network model, and input the preprocessed training set into the network model in set batches for forward propagation to obtain the tampering area segmentation probability map after forward propagation;

[0143] In this embodiment, the binary cross function of the tampered image segmentation probability map is calculated according to the following formula:

[0144]

[0145] where y ij Indicates the category to which the pixel at position (i, j) belongs in the mask image, p ij is the predicted probability of the pixel at position (i, j), M and N represent the size of the convolution map, i and j represent the pixels in the convolution map, and during the training process, the tampered image training set obtained by S1 is used for training. The optimizer uses Adam, the learning rate is set to 0.001, and only the rotation enhancement dataset is used during training, and the anti-interference strategy enhancement training set is not used;

[0146] S52, using the binary cross entropy loss function to calculate the loss value of the tampered region segmentation probability map after forward propagation and the tampered region mask corresponding to the input pre-trained training set image, and performing error back propagation to train the end-to-end codec image tampering detection network model, and selecting the model weight that minimizes the model loss within the training round as the trained image tampering detection neural network model;

[0147] S53: Use the trained image tampering detection neural network model to detect the input image to be detected, and complete the image tampering detection.

[0148] In this example, the pixel values ​​in the segmentation probability map are transformed into the interval [0, 255] through linear transformation, and the grayscale image is saved as the detection result image of the tampered area. The larger the pixel value, the higher the probability that the pixel is in the tampered area.

[0149] In one embodiment of the present invention, the following experiments were performed to verify the performance of the present invention:

[0150] The experimental platform is an AMD Ryzen 7 2000X 3.70GHz CPU, 32GB RAM, NVIDIA GeForce RTX2070SUPER (8GB) independent graphics card, and the PyTorch framework on a 64-bit Windows 10 operating system;

[0151] The test set is the CASIA1 tampered image dataset, which contains 920 tampered images, and the Columnbia tampered image dataset, which contains 180 tampered images;

[0152] Evaluation indicators include F1 value, accuracy, and intersection-over-union ratio.

[0153] Comparison of tamper detection effects:

[0154] In order to demonstrate the advantages of the present invention in tamper detection, Table 1 summarizes the performance of various different methods on the CASIA1 test set and the years in which they were proposed. Table 2 summarizes the performance of various different methods on the Columbia test set and the years in which they were proposed. It can be seen from Tables 1 and 2 that the present invention is superior to other methods.

[0155] Table 1 Comparison of tamper detection performance of the proposed method and other algorithms on the CASIA1 test set

[0156]

[0157] Table 2 Comparison of tamper detection performance between the proposed method and other algorithms on the Columbia test set

[0158]

[0159] In order to prove the detection effect of the image tampering detection network model proposed in this invention on a single tampered area, the invention is compared with various different methods on the tampered image of a single tampered area. The detection results are shown in the figure below. Figure 5 To prove the detection effect of the image tampering detection network model proposed by this method in multiple tampering areas, the present invention is compared with various different methods on tampered images in multiple tampering areas. The detection results are shown in Figure 6 shown; from Figure 5 and Figure 6 It can be seen from the figure that the image tampering detection model of the present invention has excellent detection effects on tampered images with single tampering area and multiple tampering areas.

[0160] In the description of the present invention, it should be understood that the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of such features.

Claims

1. A method for detecting image tampering based on deep learning, characterized in that: The following steps are involved: S1. Obtain an image dataset and preprocess it to obtain a preprocessed training set; The image dataset includes a plurality of tampered images containing tampered area masks; S2, constructing a coarse-grained channel compensation module; The method for processing the input image by the coarse granularity channel compensation module in step S2 includes the following steps: S21. Perform local and global feature extraction on feature information of the input image using dilated convolutions of different convolution scales to obtain first feature maps of different scales, and perform channel splicing on the first feature maps of different scales to obtain a second feature map; S22, input the second feature map into a convolutional layer, and output the tampered feature map after channel compensation; S3. Construct a fine-grained noise feature separation attention mechanism module; The method for processing the input image by the fine-grained noise feature separation attention mechanism module in step S3 includes the following steps: S31, passing the input image through a convolution layer with a convolution kernel of 1×1 and performing normalization processing to obtain a third feature map; S32, dividing the third feature map into two channels using a threshold, respectively representing two feature information greater than the threshold and less than the threshold, performing threshold truncation on the two divided channels, and performing channel splicing on the feature values ​​on both sides after the threshold truncation to obtain a fourth feature map; S33, pass the input image through a convolution layer with a convolution kernel of 1×1, use the output convolution value as the input of the softmax function, and add 1 to the result output by the softmax function to obtain a fifth feature map; S34, multiplying the attention weight of the fourth feature map by the fifth feature map to obtain a sixth feature map; S35, using average pooling to process the input image to obtain a seventh feature map; S36, taking the sixth feature map and the seventh feature map as inputs of the softmax function, adding 1 to the result output by the softmax function, and outputting the actively enhanced feature map; S4. Based on the coarse-grained channel compensation module and the fine-grained noise feature separation attention mechanism module, an end-to-end encoding and decoding image tampering detection neural network model is constructed; S5. Use the preprocessed training set and the back-propagation algorithm to optimize the binary cross entropy loss function, iteratively train the end-to-end encoding and decoding image tampering detection neural network model, obtain the trained image tampering detection neural network model, and use the trained image tampering detection neural network model to complete image tampering detection.

2. The image tampering detection method based on deep learning according to claim 1 is characterized in that The step S1 includes the following sub-steps: S11, obtain the CASIA2 tampered image dataset as the initial tampered image training set; S12. Obtain the COCO image dataset, crop the objects in the COCO image dataset, and then splice them onto the CASIA2 tampered image dataset as a tampered image training set; S13, preprocessing the images in the tampered image training set, and obtaining an image set of size C×H×W after scale transformation as the preprocessed training set; Among them, C represents the number of channels, H represents the image length, and W represents the image width.

3. The image tampering detection method based on deep learning according to claim 1 is characterized in that The step S21 includes the following sub-steps: S2101, using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 2 to process the input image to obtain a first local feature of the input image; S2022. Process the input image using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 5 to obtain a second local feature of the input image. S2023. Pass the input image through a convolution layer with a convolution kernel of 1×1, and perform absolute value processing on the output convolution eigenvalue to obtain a first eigenvalue; S2024, passing the input image through a convolution layer with a convolution kernel of 1×1, and performing mean pooling processing on the output convolution eigenvalues ​​to obtain a second eigenvalue; S2025. Perform channel concatenation on the first eigenvalue and the second eigenvalue to obtain a third eigenvalue; S2026. Pass the third eigenvalue through a convolution layer with a convolution kernel of 1×1 to obtain a fourth eigenvalue. S2027. Process the fourth eigenvalue using a convolution layer with a convolution kernel of 3×3 and a dilated convolution scale of 1 to obtain a global feature of the input image. S2028. Use the first local feature of the input image, the second local feature of the input image, and the global feature of the input image as first feature maps of different scales, and perform channel splicing on the first feature maps of different scales to obtain a second feature map.

4. The image tampering detection method based on deep learning according to claim 1 is characterized in that The end-to-end encoding and decoding image tampering detection neural network model in step S4 includes one input module, four encoding modules and four decoding modules, namely encoding module E1, encoding module E2, encoding module E3, encoding module E4, decoding module D1, decoding module D2, decoding module D3, and decoding module D4; The internal structure of each encoding module is the same, and the internal structure of each decoding module is the same; the connection relationship between the encoding modules and the decoding modules is as follows: the output end of the encoding module E1 is respectively connected to the input end of the encoding module E2 and the input end of the decoding module D4, the output end of the encoding module E2 is respectively connected to the input end of the encoding module E3 and the input end of the decoding module D3, the output end of the encoding module E3 is respectively connected to the input end of the encoding module E4 and the input end of the decoding module D2, and the output end of the encoding module E4 is connected to the output end of the decoding module D1; The encoding module E2 is the next layer structure of the encoding module E1, the encoding module E3 is the next layer structure of the encoding module E2, the encoding module E4 is the next layer structure of the encoding module E3, the decoding module D2 is the next layer structure of the decoding module D1, the decoding module D3 is the next layer structure of the decoding module D2, the decoding module D4 is the next layer structure of the decoding module D3, and the decoding module D1 is the next layer structure of the encoding module E4; The corresponding relationship between the encoding module and the decoding module is: the encoding module E1 corresponds to the decoding module D4, the encoding module E2 corresponds to the decoding module D3, the encoding module E3 corresponds to the decoding module D2, and the encoding module E4 corresponds to the decoding module D1.

5. The image tampering detection method based on deep learning according to claim 4 is characterized in that: The method for processing the input image by the input module comprises the following steps: S401, processing the input image through a double-layer zigzag dilated convolution structure to obtain an eighth feature map; S402, performing a jump connection between the input image and the eighth feature map to obtain a ninth feature map; S403 : Use the eighth feature map as the first input image of the encoding module E1 , use the ninth feature map as the second input image of the encoding module E1 , and output them to the encoding module E1 .

6. The image tampering detection method based on deep learning according to claim 4 is characterized in that: The method for processing the input image by the encoding module includes the following steps: S411, performing maximum pooling processing on the first input image to obtain a tenth feature map; S412: Input the tenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the tenth feature map to obtain an eleventh feature map; S413, passing the eleventh feature map through a three-layer zigzag-shaped dilated convolution structure to obtain a twelfth feature map; S414: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twelfth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a thirteenth feature map. S415, using a convolution layer with a convolution kernel of 1×1 to obtain the fourteenth feature map; S416, performing a jump connection on the thirteenth feature map and the fourteenth feature map to obtain a fifteenth feature map; S417: Use the fifteenth feature map as the first input image of the next layer structure and the third input image of the corresponding decoding structure, and use the eleventh feature map as the second input image of the next layer structure.

7. The image tampering detection method based on deep learning according to claim 6, characterized in that: The method for processing the input image by the decoding module includes the following steps: S421, passing the first input image through a deconvolution layer to obtain a sixteenth feature map; S422, performing channel concatenation on the sixteenth feature map and the third input image, and passing the resulting channel concatenation image through a double-layer zigzag dilated convolution structure to obtain a seventeenth feature map; S423, pass the seventeenth feature map through a convolution layer with a convolution kernel of 1×1 to obtain an eighteenth feature map; S424: Input the eighteenth feature map into the fine-grained noise feature separation attention mechanism module, and multiply the output map of the fine-grained noise feature separation attention mechanism module by the eighteenth feature map to obtain a nineteenth feature map; S425, passing the nineteenth feature map through a three-layer zigzag dilated convolution structure to obtain a twentieth feature map; S426: Input the second input image to the coarse-grained channel compensation module, perform channel concatenation on the output image of the coarse-grained channel compensation module and the twentieth feature map, and pass the channel-concatenated image through a double-layer zigzag dilated convolution structure to obtain a twenty-first feature map. S427, performing a jump connection on the twenty-first feature map and the eighteenth feature map to obtain a twenty-second feature map; S428: Determine whether the current decoding module is decoding module D4: If so, the twenty-second feature map is passed through a convolution layer with a convolution kernel of 1×1 to obtain a twenty-third feature map, and the twenty-third feature map is used as the output of the end-to-end encoding and decoding image tampering detection neural network model; If not, proceed to step S429; S429: Use the twenty-second feature map as the first input image of the next layer structure, and use the twenty-fourth feature map as the second input image of the next layer structure.

8. The image tampering detection method based on deep learning according to claim 7, characterized in that: The step S5 comprises the following sub-steps: S51, randomly initialize the end-to-end encoding and decoding image tampering detection neural network model, and input the preprocessed training set into the network model in set batches for forward propagation to obtain the tampering area segmentation probability map after forward propagation; S52, using the binary cross entropy loss function to calculate the loss value of the tampered region segmentation probability map after forward propagation and the tampered region mask corresponding to the input pre-trained training set image, and performing error back propagation to train the end-to-end codec image tampering detection network model, and selecting the model weight that minimizes the model loss within the training round as the trained image tampering detection neural network model; S53: Use the trained image tampering detection neural network model to detect the input image to be detected, and complete the image tampering detection.

Images