A tampering detection method based on convolutional vision Mamba

Abstract

The present invention discloses a tampering detection method based on convolutional vision Mamba. The method first uses a seam cropping algorithm to process a data set, and reduces the image width according to different seam cropping rates to obtain a data set. Secondly, the processed original image is randomly cropped to the same size as the image processed with the corresponding seam cropping rate. Then, the data set is input into a network model based on convolutional vision Mamba, the classification loss is calculated using cross entropy, and the network model parameters are updated using back propagation. Finally, the image to be detected is input into the trained network model, and a detection result of whether the image has been processed by seam cropping is output. The present invention makes it easier to capture artifacts in the image and can effectively improve the detection accuracy when processing low seam cropping rate image detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital forensics and computer vision, and specifically is a tampering detection method based on convolutional vision Mamba. Background Art

[0002] With the increasing prevalence of smart devices, sharing digital images on social media has become increasingly common. Sharing digital images online sometimes requires resizing them to a specific size. This involves scaling, a technique known as image resizing. Traditional digital image resizing techniques primarily involve linear interpolation and cropping, both of which can significantly damage the original image's important content.

[0003] To achieve image scaling while minimizing the loss of important image content, Shai Avidan proposed a content-aware image scaling technique, known as seam carving, in 2007. The key idea behind this method is to predefine an energy calculation function, perhaps based on pixel gradients or color changes, and then calculate the energy value of each pixel in the digital image to be scaled. Pixels with smaller energy values ​​contribute less to the overall image and are therefore less important. Deleting these pixels results in less loss to the image. A dynamic programming algorithm is then used to calculate an eight-connected path of pixels from top to bottom (left to right) with the minimum sum of energy values ​​(considering all eight directions of the pixels). This path is known as a "seam." Deleting this path reduces the height (width) of the entire image by one pixel.

[0004] Because this technology can scale images while leaving minimal artifacts, making them difficult to detect with the naked eye, some criminals have exploited this technology to maliciously tamper with important digital images, distort facts, and spread false news online, severely impacting social stability. Consequently, the importance of detecting seam carving algorithms is growing, and it has become a key branch of digital image forensics. Summary of the Invention

[0005] The technical solution adopted by the present invention to solve the above problems is: a tampering detection method based on convolutional vision Mamba, comprising the following steps:

[0006] Step 1. Create a dataset: Use the Seam Carving algorithm to process the dataset, reduce the image width according to different Seam Carving rates, and obtain the final dataset.

[0007] Step 2. Unify the input image size: randomly crop the original image before processing to the same size as the image after processing with the corresponding Seam Carving rate.

[0008] Step 3. Train the network model. Input the prepared dataset into the network model based on convolutional vision Mamba, use cross entropy to calculate the classification loss, and use back propagation to update the network model parameters.

[0009] Step 4. Model testing: input the image to be tested into the trained network model and output the detection result of whether the image has been processed by Seam Carving.

[0010] In step 3, the convolutional visual Mamba-based network model includes a sequentially connected preprocessing module, an artifact feature extraction module, a feature scale unification module, and a visual Mamba module.

[0011] The preprocessing module consists of a high-pass filter with a 5×5 convolution kernel. Since most areas of the image after seam carving remain consistent with the original image content, a preprocessing layer is used to preprocess the image to help suppress the impact of image content on detection results. The convolution kernel used in the high-pass filter is as follows:

[0012]

[0013] The artifact feature extraction module consists of three different types of sequentially connected convolutional blocks. The first type of convolutional block includes a single convolution operation to expand the number of channels; the second type includes multiple densely connected convolutional blocks, with the outputs residually connected to the initial input; and the third type includes two convolutions with different kernel sizes and a stride of 2, which halves the size of the feature vector. The artifact feature extraction module is used to enhance the artifact features left after seam carving.

[0014] The feature scale unification module is implemented by adaptive maximum pooling. Since the cropped pixels of the image are different after each Seam Carving rate processing, the image is input into the feature scale unification module to output a feature vector of unified dimension.

[0015] The visual Mamba module uses the number of channels of the unified-dimensional feature vector as the number of patch blocks, flattens and linearly projects the patch blocks into different feature unit tokens, adds class tokens for subsequent classification, and after position encoding, inputs them into a module stacked with multiple Mamba encoders for further global artifact feature extraction, and finally inputs them into a multi-layer perceptron for detection and classification.

[0016] The Mamba encoder includes two one-dimensional convolutions in both forward and reverse directions and two SSMs in both forward and reverse directions, and uses the SiLU activation function for residual connection.

[0017] The present invention has the following beneficial effects:

[0018] The proposed method uses a high-pass filter to suppress image content and a convolutional neural network for local feature learning. It then enhances the local artifact features left after seam carving, inputting them into a visual Mamba module to convert them into a series of image representations for subsequent classification. By using the convolutional visual Mamba to learn local and global artifact features of the image, and modeling the extracted local artifact features using the bidirectional state space in the visual Mamba encoder, the model is more likely to capture artifacts in the image, effectively improving detection accuracy when processing images with low seam carving rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a Seam Carving tampering detection method based on convolutional vision Mamba provided by the present invention;

[0020] Figure 2 This is a schematic diagram of the overall network structure provided by the present invention;

[0021] Figure 3 This is a working diagram of the pre-processing module provided by the present invention;

[0022] Figure 4 This is a working diagram of the artifact feature extraction module provided by the present invention;

[0023] Figure 5 This is a working diagram of the characteristic scale unification module provided by the present invention;

[0024] Figure 6 This is a working diagram of the visual Mamba module provided by the present invention. DETAILED DESCRIPTION

[0025] In order to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be clear that the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] like Figure 1 As shown in FIG, a tampering detection method based on convolutional vision Mamba includes the following steps:

[0027] Step 1. Create a dataset: Use the Matlab-implemented Seam Carving algorithm to process the BossBase dataset. Reduce the image widths using different seam carving ratios to generate the final dataset. Split the dataset into training, validation, and test sets in an 8:1:1 ratio for subsequent model training and testing.

[0028] Step 2. Unify the input image size: During the model training phase, since the original image and the image processed by Seam Carving have different sizes, the subsequent model input feature dimensions cannot be unified. Therefore, during the data loading phase, the original image is randomly cropped to the same size as the corresponding image processed by Seam Carving.

[0029] Step 3. Train the network model. Input the prepared dataset into the network model based on convolutional vision Mamba, use cross entropy to calculate the classification loss, and use back propagation to update the network model parameters.

[0030] The functional expression of the cross entropy loss is as follows:

[0031]

[0032] Among them, n represents the batch size of training, C represents the total number of categories classified by the model, and y i,j represents the label value of sample i in the jth class, p i,j It represents the predicted probability that sample i belongs to the jth class.

[0033] Step 4. Model testing: Input the image to be tested into the trained model and output the test result of whether the image has been processed by SeamCarving.

[0034] like Figure 2 As shown, the network model includes a sequentially connected preprocessing module, an artifact feature extraction module, a feature scale unification module, and a visual Mamba module.

[0035] like Figure 3 As shown in the figure, the preprocessing module processes the image through a high-pass filter with a 5×5 convolution kernel, with padding set to 2 to ensure consistent input and output sizes. Since most areas of the image after seam carving remain consistent with the original image content, a preprocessing layer is used to preprocess the image to help suppress the impact of image content on detection results. The convolution kernel used in the high-pass filter is as follows:

[0036]

[0037] like Figure 4 As shown in the figure, the artifact feature extraction module contains three different blocks. The initial input feature dimension size is H×W. Block1 only contains one convolution operation, which is used to convert the number of channels to the specified size, and the feature dimension size remains unchanged; block2 contains two forms of dense connection and residual connection. Dense connection means that each input is the feature vector after the initial input and the output of each previous convolution operation are spliced ​​together. Specifically, the input feature m first undergoes a convolution operation, and then the convolution output conv1(m) is connected to the previous input m in dimension to obtain concat(m,conv1(m)), which is then used as the input of the next convolution. In the final 1×1 convolution, the number of output channels is set to the number of channels of the initial input feature, so that the feature dimension size remains unchanged; block3 uses two convolutions with different kernel sizes to process them separately, and then adds the outputs of the two. Since the step size is set to 2, the feature vector size is halved. By superimposing 4 block3s, the feature dimension size is changed to

[0038] like Figure 5 As shown, the feature resizing module uses adaptive max pooling to convert feature vectors to a specified size using the extracted features. Specifically, it adaptively calculates the kernel size, padding, and stride based on the input feature size and the target output feature size. Max pooling represents the region by taking the maximum value of the elements within a kernel-sized region. For example, if the elements within a 2×2 kernel-sized region are (x1, x2, x3, x4), then max(x1, x2, x3, x4) is used instead of (x1, x2, x3, x4), achieving feature resizing.

[0039] like Figure 6 As shown in the figure, the visual Mamba encoding module flattens the input features according to the preset patch block size to obtain multiple feature unit tokens, and randomly initializes a class token. This token can be inserted at any position between multiple tokens to aggregate global feature information for subsequent classification. Then, a position code is set for each token so that the model can know the relative position relationship of each feature patch block. The code is then input into multiple stacked Mamba encoders for global feature extraction. Finally, the output corresponding to the class token position is input into the multi-layer perceptron for final classification.

[0040] The Mamba encoder includes two one-dimensional convolutions in both forward and reverse directions and two SSMs in both forward and reverse directions.

[0041] Specifically, the input token is first normalized, and then the dimension of the input token is transformed through linear projection to obtain x and z.

[0042] x undergoes forward and reverse bidirectional processing respectively. The simulated bidirectional effect is mainly achieved by inverting x. Taking the forward direction as an example, a one-dimensional convolution operation is first performed on x, and then the state space model parameter matrices A, B, and C are calculated based on the output after the one-dimensional convolution. Then, the state space modeling operation is performed, and then the dot product is performed with z after the SiLU activation function. The reverse direction is achieved by inverting x and then performing the same forward processing as above. The forward and reverse outputs are added, and after linear projection, a residual connection is performed with the Mamba encoder input.

[0043] Mamba can process the input token in both forward and backward directions, simulating a bidirectional neural network layer, which can provide a richer understanding of the image context and help capture artifacts left after seam carving.

[0044] The convolutional vision-based Mamba seam carving tampering detection method can detect seam carving tampering images with high accuracy by training the model designed by the present invention. In particular, when processing images with low seam carving rates, the present invention can still maintain a high detection accuracy.

[0045] During the model training phase, the Adam optimizer was used, the initial learning rate was set to 0.0001, the weight decay value was set to 0.0002, and the number of training rounds was set to 100.

[0046] The seam carving tampering detection method based on convolutional vision Mamba provided by the present invention is compared with the existing detection methods. The experimental results are shown in Table 1:

[0047] Table 1

[0048] Xception ResNet YeNet SRNet ILFNet SCDNet The present invention 5％ 53.30％ 71.85％ 78.85％ 53.10％ 71.40％ 89.75％ 98.65％ 6％ 60.30％ 73.75％ 79.70％ 60.25％ 80.10％ 91.45％ 99.25％ 7％ 75.95％ 75.35％ 82.10％ 80.35％ 83.45％ 93.80％ 99.20％ 8％ 79.35％ 77.10％ 82.30％ 86.05％ 83.25％ 93.00％ 98.85％ 9％ 81.25％ 80.00％ 84.60％ 86.40％ 84.45％ 93.30％ 99.25％ 10％ 82.55％ 81.25％ 86.65％ 88.50％ 87.55％ 93.05％ 99.60％ 20％ 91.40％ 92.45％ 94.05％ 94.45％ 94.85％ 97.25％ 99.70％ 30％ 97.00％ 97.95％ 97.70％ 98.10％ 98.40％ 98.35％ 99.75％ 40％ 98.90％ 99.25％ 99.05％ 99.45％ 99.30％ 98.50％ 99.85％ 50％ 99.90％ 99.90％ 99.95％ 99.65％ 99.85％ 98.50％ 99.75％

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A tampering detection method based on convolutional vision Mamba, characterized in that: The following steps are involved: Step 1. Use the seam carving algorithm to process the dataset and reduce the image width according to different seam carving rates to obtain the dataset; Step 2. Randomly crop the original image before processing to the same size as the image after processing with the corresponding Seam Carving rate; Step 3. Input the dataset into the convolutional vision Mamba-based network model, use cross entropy to calculate the classification loss, and use backpropagation to update the network model parameters; The network model based on convolutional visual Mamba includes a preprocessing module, an artifact feature extraction module, a feature scale unification module, and a visual Mamba module constructed by sequential connections; The preprocessing module consists of a high-pass filter with a convolution kernel of 5×5. It preprocesses the image to suppress the influence of image content on the detection effect. The convolution kernel used by the high-pass filter is as follows: The artifact feature extraction module consists of three different convolution blocks connected sequentially. The first convolution block contains a convolution operation to expand the number of channels; the second contains multiple densely connected convolution blocks, and the output is residually connected to the initial input; the third contains two convolutions with different kernel sizes, and the stride is set to 2, so that the size of the feature vector is halved; The feature scale unification module is implemented by adaptive maximum pooling, and the image is input into the feature scale unification module to output a feature vector of unified dimension; The visual Mamba module uses the number of channels of the unified-dimensional feature vector as the number of patches, flattens and linearly projects the patches into different feature unit tokens, adds class tokens for subsequent classification, and after position encoding, inputs them into a module stacked with multiple Mamba encoders for global artifact feature extraction, and finally inputs them into a multi-layer perceptron for detection and classification; The Mamba encoder includes two one-dimensional convolutions in both forward and reverse directions and two SSMs in both forward and reverse directions, and uses the SiLU activation function for residual connection; Step 4. Input the image to be tested into the trained network model and output the detection result of whether the image has been processed by Seam Carving.

Images