Image processing method and terminal based on image tampering positioning model

Abstract

The present invention discloses an image processing method and terminal based on an image tampering localization model. The image tampering localization model includes: a preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules, and an output convolution layer. The image processing method includes: inputting a tampered image into the image tampering localization model, extracting manual features of the tampered image through a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer; inputting the manual features into a dual convolution feature extraction layer to obtain deep features; inputting the deep features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling, respectively; inputting the features obtained by upsampling into an output convolution layer, and outputting a final predicted mask map through the output convolution layer. The mask map is used to predict the tampered area of the tampered image. The present invention achieves better performance without the need for pre-training with a large-scale tampering dataset.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, a terminal, and a computer-readable storage medium based on an image tampering positioning model. Background Art

[0002] In recent years, the popularity of efficient image manipulation and editing tools such as Photoshop, After Effects, and GIMP, coupled with deep learning technology, has led to rapid advancements in the sophistication and realism of image manipulation techniques. When these fabricated images are used to create fake news and disseminate it online, they often pose significant risks to individuals and society. Therefore, tampering detection and location of manipulated images is essential and has broad application value.

[0003] With the development of deep learning technology, image tampering detection and localization methods in recent years are based on convolutional neural networks, such as RRUnet, Mantra-Net, H-LSTM, RGB-N, SPAN, PSCC-Net, HPFCN, DFCN, and MVSS-Net. Among them, Mantra-Net, RGB-N, and HPFCN all use three 5×5 high-pass filters from the Steganalysis Rich Model (SRM) to initialize the convolution kernel of the convolution layer. This convolution layer is then used to extract residual features of the image. Finally, the extracted features are input into the convolutional neural network to calculate the prediction results. The difference is that the SRM convolution layer of Mantra-Net and RGB-N is not learnable, while that of HPFCN is learnable.

[0004] Although a growing number of deep learning-based methods have achieved significant success in localizing image tampering, they still face challenges, such as robustness to post-processing of tampered images and generalizability to cross-dataset testing. To address this issue, most current methods are data-driven (such as DFCN, MVSS-Net, Mantra-Net, H-LSTM, RGB-N, SPAN, and PSCC-Net), often using large-scale image tampering datasets for pre-training and then fine-tuning on specific datasets. However, collecting and producing large-scale image tampering datasets with tens of thousands of items often requires significant manpower and resources.

[0005] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of the present invention is to provide an image processing method, terminal and computer-readable storage medium based on an image tampering localization model, aiming to solve the problem in the prior art that collecting and producing large-scale image tampering data sets of tens of thousands of levels requires a lot of manpower and material resources.

[0007] To achieve the above object, the present invention provides an image processing method based on an image tampering localization model, wherein the image tampering localization model includes: a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer; the image processing method based on the image tampering localization model includes:

[0008] Inputting the tampered image into the image tampering localization model, and extracting manual features of the tampered image through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer;

[0009] Inputting the manual features into a dual convolutional feature extraction layer to obtain deep features;

[0010] Input the depth features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling respectively;

[0011] The features obtained by upsampling are input to the output convolution layer, and the final predicted mask map is output through the output convolution layer. The mask map is used to predict the tampered area of the tampered image.

[0012] The image processing method based on the image tampering positioning model, wherein the DCT-Layer preprocessing convolution layer is composed of a DCT-Conv convolution layer and a truncation calculation, wherein the convolution kernel of the DCT-Conv is initialized by a DCT kernel of size 4*4*6 with a step size of 1.

[0013] The image processing method based on the image tampering positioning model, wherein the SRM-Layer preprocessing convolution layer is composed of an SRM-Conv convolution layer and a truncation calculation, wherein the convolution kernel of the SRM-Conv is initialized by 30 5*5 high-pass filters with a step size of 1.

[0014] The image processing method based on the image tampering localization model, wherein the dual convolutional feature extraction layer includes two convolutional layers, two batch normalization layers and two ReLU activation functions, wherein the convolution kernel size of the two convolutional layers is 3*3 and the step size is 1;

[0015] The dual convolution feature extraction layer is composed of a convolution layer, a batch normalization layer, a ReLU activation function, a convolution layer, a batch normalization layer and a ReLU activation function in series.

[0016] In the image processing method based on the image tampering localization model, the up-sampling convolution module and the down-sampling convolution module at the same level are jump-connected;

[0017] The first downsampling convolution module of the four downsampling convolution modules consists of block1 and block2, the second downsampling convolution module consists of block3, the third downsampling convolution module consists of block4, and the fourth downsampling convolution module consists of block5 and block6;

[0018] Each of the four up-sampling convolution modules consists of a bilinear interpolation upsampling layer and a double convolution feature extraction layer.

[0019] The image processing method based on the image tampering positioning model, wherein the convolution output layer consists of a convolution layer and a sigmoid activation function, wherein the convolution kernel size of the convolution layer is 1*1, the step size is 1 and the output channel is 1.

[0020] The image processing method based on the image tampering localization model, wherein the image processing method based on the image tampering localization model further includes:

[0021] Two tampered images are input into the image tampering localization model, a tampered area is selected according to the mask corresponding to one of the tampered images, and the tampered area is randomly pasted onto the other tampered image to output a new tampered image.

[0022] The image processing method based on the image tampering localization model, wherein the image processing method based on the image tampering localization model further includes:

[0023] A tampered image is input into the image tampering localization model, a tampered area is selected according to the mask corresponding to the tampered image, and the tampered area is randomly pasted to other areas of the tampered image to output a new tampered image.

[0024] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an image processing program based on an image tampering positioning model stored in the memory and runnable on the processor, and when the image processing program based on the image tampering positioning model is executed by the processor, the steps of the image processing method based on the image tampering positioning model as described above are implemented.

[0025] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image processing program based on an image tampering positioning model, and when the image processing program based on the image tampering positioning model is executed by a processor, the steps of the image processing method based on the image tampering positioning model as described above are implemented.

[0026] In the present invention, the image tampering positioning model includes: a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer; the image processing method includes: inputting the tampered image into the image tampering positioning model, extracting manual features of the tampered image through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer; inputting the manual features into the dual convolution feature extraction layer to obtain depth features; inputting the depth features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling respectively; inputting the features obtained by upsampling into the output convolution layer, and outputting the final predicted mask map through the output convolution layer, and the mask map is used to predict the tampering area of the tampered image. The present invention uses the mask corresponding to the tampered image to perform end-to-end expansion of the tampered image data volume. Without the need for pre-training with a large-scale image tampering dataset, the image tampering localization model is trained end-to-end through self-supervised data enhancement, thereby achieving tampering localization performance similar to or even better than that of the pre-training method. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the structure of the image tampering localization network (image tampering localization model) in the present invention;

[0028] Figure 2 It is a flow chart of a preferred embodiment of the image processing method based on the image tampering localization model of the present invention;

[0029] Figure 3 This is a structural diagram of the DCT-Layer preprocessing convolution layer in the image tampering localization model of the present invention;

[0030] Figure 4 This is a structural diagram of the SRM-Layer preprocessing convolutional layer in the image tampering localization model of the present invention;

[0031] Figure 5 It is a structural diagram of channel attention in the image tampering localization model of the present invention;

[0032] Figure 6 2 is a schematic diagram of an example of mask-based splicing self-supervised data enhancement in a preferred embodiment of an image processing method based on an image tampering localization model;

[0033] Figure 7 3. It is a schematic diagram of an example of mask-based copy-move self-supervised data enhancement in a preferred embodiment of an image processing method based on an image tampering localization model;

[0034] Figure 8 This is a schematic diagram of an example of prediction results in a preferred embodiment of an image processing method based on an image tampering localization model;

[0035] Figure 9 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The image processing method based on the image tampering positioning model described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the image tampering positioning model includes: a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a double convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer.

[0038] like Figure 2 As shown, the image processing method based on the image tampering positioning model includes:

[0039] Step S10: input the tampered image into the image tampering localization model, and extract the manual features of the tampered image through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer;

[0040] Step S20: inputting the manual features into a dual convolutional feature extraction layer to obtain deep features;

[0041] Step S30: input the depth features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling respectively;

[0042] Step S40: Input the features obtained by upsampling into the output convolution layer, and output the final predicted mask map through the output convolution layer, where the mask map is used to predict the tampered area of the tampered image.

[0043] Specifically, the image tampering localization model (EfficientUnet) belongs to a U-shaped fully convolutional neural network, which includes a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer, wherein the upsampling convolution modules and downsampling convolution modules at the same level are jump-connected. When performing inference learning on a deep learning model, the manual features extracted by the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer pass through four downsampling convolution modules, four upsampling convolution modules and an output convolution layer in sequence.

[0044] Specifically, the DCT-Layer preprocessing convolution layer is composed of a DCT-Conv convolution layer and a truncation calculation, wherein the convolution kernel of the DCT-Conv is initialized by a DCT kernel of size 4*4*6 with a step size of 1.

[0045] Specifically, the SRM-Layer preprocessing convolution layer consists of an SRM-Conv convolution layer and a truncation calculation, wherein the convolution kernel of the SRM-Conv is initialized by 30 5*5 high-pass filters with a step size of 1; in order to allow the 30 high-pass filters to give different sensitivities, an SRM-Conv convolution layer based on the attention mechanism, namely the CA-SRM-Layer preprocessing convolution layer, is constructed.

[0046] Furthermore, the CA-SRM-Layer preprocessing convolution layer adds a channel attention module to the SRM-Conv convolution layer based on the SRM-Layer preprocessing convolution layer.

[0047] Specifically, the dual convolution feature extraction layer includes two convolution layers, two batch normalization layers (BatchNormalization) and two ReLU activation functions, wherein the convolution kernel size of the two convolution layers is 3*3 and the stride is 1.

[0048] Furthermore, the dual convolutional feature extraction layer is composed of a convolutional layer, a batch normalization layer, a ReLU activation function, a convolutional layer, a batch normalization layer, and a ReLU activation function connected in series.

[0049] Specifically, if Figure 1As shown, the four down-sampling convolution modules are composed of the first six modules of EfficientNet, block1, block2, block3, block4, block5 and block6, where the first down-sampling convolution module down1 is composed of block1 and block2, the second down-sampling convolution module down2 is composed of block3, the third down-sampling convolution module down3 is composed of block4, and the fourth down-sampling convolution module down4 is composed of block5 and block6.

[0050] Specifically, the four up-sampling convolution modules are composed of a bilinear interpolation upsampling layer and a double convolution feature extraction layer.

[0051] Specifically, the convolution output layer consists of a convolution layer and a sigmoid activation function, wherein the convolution kernel size of the convolution layer is 1*1, the stride is 1 and the output channel is 1.

[0052] This paper draws on the deep learning networks EfficientNet and Unet to construct a new deep learning model, EfficientUnet. The network structure mainly consists of four down-sampling modules and four up-sampling modules, where the down-sampling modules and up-sampling modules at the same level are jump-connected. At the same time, to better capture weak image tampering signals, two pre-processing convolutional layers, DCT-Layer or SRM-Layer, used in the field of steganalysis, are optionally inserted before EfficientUnet. Finally, based on the existing dataset, a self-supervised data enhancement method is designed to train EfficientUnet. This self-supervised data enhancement achieves end-to-end expansion of the tampered dataset by generating splicing and copy-move tampered images in real time. Through a large number of experiments, it is proved that the present invention can achieve higher performance on most public datasets without the need for pre-training with a large number of tampered images.

[0053] The self-supervised data enhancement method includes mask-based splicing self-supervised data enhancement and mask-based copy-move self-supervised data enhancement, thereby achieving end-to-end real-time expansion of tampered images when training the model.

[0054] The mask-based splicing self-supervised data enhancement takes two tampered images as input, selects the tampered area according to the mask corresponding to one of the tampered images, and then randomly pastes it onto the other tampered image, thereby outputting a new tampered image.

[0055] The mask-based copy-move self-supervised data augmentation takes a tampered image as input, selects a tampered area according to the mask corresponding to the tampered image, and then randomly pastes it to other areas of the tampered image, thereby outputting a new tampered image.

[0056] The self-supervised data augmentation method proposed in this paper uses masks corresponding to tampered images to perform end-to-end augmentation of the tampered image data based on existing public data. Other tampering operations include image splicing and image copy-and-paste. Without the need for pre-training on a large-scale image tampering dataset, the proposed tampering localization model, EfficientUnet, is trained end-to-end using self-supervised data augmentation, achieving tampering localization performance similar to or even better than pre-trained methods.

[0057] The image tampering localization method of an embodiment of the present invention includes a model design stage, a data preparation and an experimental demonstration stage. The deep learning model used is an improved version of the U-shaped fully convolutional neural network EfficientUnet, and the data enhancement training method used is splicing self-supervised data enhancement and copy movement data enhancement.

[0058] The image tampering localization model of the present invention (i.e., the network model EfficientUnet) mainly includes a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules, and an output convolution layer, wherein the dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules, and the output convolution layer belong to the basic part of EfficientUnet, and its network structure is as follows: Figure 1 In general, the model proposed in this paper takes the tampered image as input, extracts manual features through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer, and then inputs the manual features into the deep features obtained by the dual convolution feature extraction layer. The deep features are then input into four down-sampling convolution modules and four up-sampling convolution modules, and finally the final predicted mask image is output through the output convolution layer.

[0059] First, if Figure 1The figure below introduces the basic components of EfficientNet: a dual convolutional feature extraction layer, four downsampling convolutional modules, four upsampling convolutional modules, and an output convolutional layer. The tampered input image first passes through the dual convolutional feature extraction layer (replacing the first convolutional layer in the original EfficientNet. Because the stride of the original convolutional layer is 2, the extracted feature size is halved, which is not conducive to extracting weak tampering signals). It then passes through four downsampling modules, downi (i = 1, 2, 3, 4). Unlike the four downsampling modules in Unet, EfficientUnet uses EfficientNet's blocks 1 and 2 as down1, block 3 as down2, block 3 as down3, and blocks 5 and 6 as down4. The number of channels in each downsampling module remains the same as in EfficientNet. This structural design is mainly due to the fact that EfficientNet performs a stride-2 convolution calculation in blocks 2, 3, 4, and 6 to downsample the features. Next, each upsampling module upi (i=1, 2, 3, 4) takes as input the input and output features of the downsampling module at the same level and outputs features with the same dimensions as the input features of the downsampling module at the same level. For example, the output feature dimensions of up1 are the same as the input feature dimensions of down4. After passing through four upsampling modules, the resulting features are input to the final output convolutional layer to obtain the final predicted image tampering mask.

[0060] DCT-Layer was first applied to single-channel JPEG image steganalysis to extract the frequency domain features of the image. This paper applies it to RGB channel images. The network structure of DCT-Layer is as follows: Figure 3 As shown in the figure, a 4×4×16 DCT kernel is first initialized to a convolutional layer (DCT-Conv) with an input channel of 1, an output channel of 16, and a convolution kernel size of 4×4. The parameters of the initialized convolution kernel are fixed and do not participate in training. After the DCT-Conv calculation, the obtained features are truncated according to the form of formula (1). Therefore, the DCT-Layer consists of DCT-Conv and truncation calculation. Its input is an RGB image of size W×H×3. The images of the three RGB channels are input into the DCT-Layer separately, and then the obtained features are concatinated into output features of size W×H×48.

[0061] Similarly, SRM-Layer is applied to steganalysis of color images. The detailed structure of SRM-Layer is as follows Figure 4As shown, it first uses 30 high-pass filters of SRM to initialize the convolution layer (SRM-Conv) with an input channel of 1, an output channel of 30, and a convolution kernel size of 5×5. The parameters of the convolution kernel after initialization are fixed and do not participate in training. After the SRM-Conv calculation, the obtained features are truncated in the form of formula (2). Therefore, SRM-Layer is composed of SRM-Conv and truncation calculation. Its input is an RGB image of size W×H×3. The images of the three RGB channels are input into the SRM-Layer respectively, and then the obtained features are concat into output features of W×H×90. CA-SRM-Layer (ChannelAttention SRM-Layer) is composed of SRM-Layer and a channel attention mechanism module (Channel AttentionModule, CAM). The structural diagram of CAM is shown in Figure 5 As shown in the figure, its purpose is to perform channel attention calculation on the output features of the RGB three channel images obtained through the SRM-Layer, so as to give different sensitivities to the 30 residual features obtained for each channel.

[0062] ; (1)

[0063] ; (2)

[0064] The embodiment of the present invention utilizes the DCT-layer or SRM-Layer in the field of forensics to construct an image tampering model EfficientUnet, which achieves a higher recognition rate than the general deep learning model; and applies self-supervised data enhancement training technology to achieve better performance without the need for pre-training on large-scale tampering data sets.

[0065] In the training phase, the self-supervised data enhancement proposed in the present invention includes splicing self-supervised data enhancement and copy-move self-supervised data enhancement, wherein the steps of splicing self-supervised data enhancement are as shown in Algorithm 1, and the mask-based splicing self-supervised data enhancement example is as follows: Figure 6 As shown; the steps for implementing copy-move self-supervised data enhancement are as shown in Algorithm 2, and the mask-based copy-paste self-supervised data enhancement example is as follows: Figure 7 As shown; the prediction results of the tampered image are as follows Figure 8 The training method of the present invention can achieve end-to-end expansion of new tampered data, thereby achieving the effect of not requiring additional large-scale tampered data set pre-training.

[0066] Algorithm 1 (splicing self-supervised data enhancement algorithm) is as follows:

[0067] Input: Two tampered images , and its corresponding mask , ;

[0068] Output: A new stitched tampered image and its corresponding mask ;

[0069] 1: Use the smallest possible rectangle to Middle crop tampered area and its corresponding mask , similarly, randomly from Cut out and Rectangular areas of the same size and its corresponding mask ;

[0070] 2: Cut out Paste the following formula into Generate a new tampered image on and its corresponding mask : , ;

[0071] 3: Pick a random number from 0 to 1: p = random(0, 1);

[0072] 4: If , then: cv2.seamlessClone( , , , center, cv2.NORMAL_CLONE) , where cv2.seamlessClone is used to perform Poisson fusion operation on the tampered area;

[0073] 5: End of assumption;

[0074] 6: Return the new stitched tampered image and its corresponding mask and .

[0075] Algorithm 2 (copy-move self-supervised data enhancement algorithm) is as follows:

[0076] Input: A tampered image and its corresponding mask ;

[0077] Output: A new copy-move tampered image and its corresponding mask ;

[0078] 1: Use the smallest possible rectangle to Middle crop tampered area and its corresponding mask , similarly, randomly from Cut out and Rectangular areas of the same size and its corresponding mask ;

[0079] 2: Pick a random number from 0 to 1: p = random(0, 1);

[0080] 3: If , then: Paste the following formula into Generate new tampered images and its corresponding mask :

[0081] , ;

[0082] 4: Otherwise: Cut out Paste the following formula into Generate new tampered images and its corresponding mask :

[0083] ;

[0084] 5: End of assumption;

[0085] 6: Return the new copy-move tampered image and its corresponding mask and .

[0086] For example, the experiment of this invention was trained and tested on the CASIA (v2 is the training set, v1 is the test set), COVERAGE, Columbia, and NIST16 datasets. In the comparison of experimental results, F1-score and pixel-level AUC were used as test indicators. For both indicators, higher indicators represent better performance.

[0087] In this example, EfficientUnet was trained and tested on NIST2016, Columbia, COVERAGE, and CASIA datasets, and compared with several deep learning-based algorithms (H-LSTM, RGB-N, Mantra-Net, SPAN, and PSCC-Net). The results are shown in Table 1. DCT-EfnUnet-b7 represents an EfficientUnet-b7 network consisting of a DCT layer and an EfficientNet-b7 backbone network; SRM-EfnUnet-b6 represents an EfficientUnet-b6 network consisting of an SRM layer and an EfficientNet-b6 backbone network; EfnUnet-b4 represents an EfficientUnet-b7 network consisting of an EfficientNet-b7 backbone network; and CA-SRM-EfnUnet-b2 represents an EfficientUnet-b2 network consisting of a CA-SRM layer and an EfficientNet-b2 backbone network.

[0088]

[0089] Table 1: Comparison with other advanced methods (* indicates the addition of self-supervised data augmentation)

[0090] In this example, EfficientUnet is trained on CASIA-v2 and then directly tested on the NIST2016, COVERAGE, Columbia, and CASIA-v1 datasets. CA-SRM-EfnUnet-b2 represents an EfficientUnet-b2 consisting of a CA-SRM-Layer and an EfficientNet-b2 backbone network; CA-SRM-Res34Unet replaces the EfficientUNet backbone network with Resnet34; the results in rows 2 and 5 represent the addition of self-supervised data augmentation to the training model. Furthermore, the MVSS-Net method used for comparison uses Resnet50 as its backbone network, and this method also introduces custom data augmentation for comparison. The results are shown in Table 2.

[0091]

[0092] Table 2: Comparison of model generalization tests across datasets (* indicates the addition of self-supervised data augmentation)

[0093] Finally, the model EfficientUnet in this example was implemented using the deep learning framework PyTorch and trained on an NVIDIA Tesla V100 GPU. During training, a batch size of 4 was used; the optimization function used Adam with a learning rate and weight decay of 0.0001 and 0.00001, respectively; and the loss function used was BCEDiceLoss (BCEDiceLoss = BCELoss + ×DiceLoss), where The backbone network EfficientNet in EfficientUnet is initialized with ImageNet pre-trained weights. For data augmentation, conventional data augmentation methods such as random cropping, rotation, flipping, median blurring, and JPEG compression are used. Self-supervised data augmentation is also implemented when training on the CASIA-v2 dataset.

[0094] Table 1 shows that, with the exception of the AUC performance on the NIST2016 dataset, the proposed model, EfficientUnet (including various combinations), combined with self-supervised data augmentation, outperforms state-of-the-art methods on all four datasets. On the NIST2016 dataset, the proposed model achieves a 2% higher F1 than the previous best method, PSCC-Net; on the COVERAGE dataset, the AUC and F1 are 2.8% and 10% higher, respectively, than the previous best method, PSCC-Net; on the Columbia dataset, the AUC and F1 are 6.3% and 16.9% higher than the previous best method, SPAN; and on the CASIA dataset, the AUC and F1 are 6.2% and 8.9% higher, respectively, than the previous best method, PSCC-Net.

[0095] Table 2 shows that after incorporating self-supervised data augmentation and training on the CASIA-v2 dataset, the EfficientUnet (CA-SRM-EfnUnet-b2) achieves the best AUC and F1 performance on the COVERAGE, CASIA, and PS-alibaba datasets, except for its F1 performance on the NIST2016 and Columbia datasets, where it is 0.8% and 10.1% lower than that of MVSS-Net, respectively. On the COVERAGE and CASIA datasets, its F1 performance outperforms that of MVSS-Net by 8.9% and 19.1%, respectively. On the PS-alibaba dataset, its AUC and F1 performance outperform those of CA-SMR-Resnet34Unet by 15.8% and 19.9%, respectively. In summary, the proposed EfficientNet, after training on the CASIA-v2 dataset, demonstrates excellent generalization performance.

[0096] In summary, the embodiments of the present invention take advantage of the organic integration of domain knowledge to achieve a higher recognition rate than general deep learning models, and apply self-supervised data enhancement training technology to achieve better performance without the need for pre-training on large-scale tampered datasets.

[0097] Furthermore, if Figure 9 As shown, based on the above-mentioned image processing method based on the image tampering positioning model, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 9 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0098] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal. Furthermore, the memory 20 may include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an image processing program 40 based on an image tampering positioning model. The image processing program 40 based on the image tampering positioning model can be executed by the processor 10, thereby implementing the image processing method based on the image tampering positioning model in the present application.

[0099] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program code or process data stored in the memory 20, such as executing the image processing method based on the image tampering positioning model.

[0100] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. Components 10-30 of the terminal communicate with each other via a system bus.

[0101] In one embodiment, when the processor 10 executes the image processing program 40 based on the image tampering localization model in the memory 20 , the steps of the image processing method based on the image tampering localization model are implemented.

[0102] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image processing program based on an image tampering positioning model, and when the image processing program based on the image tampering positioning model is executed by a processor, the steps of the image processing method based on the image tampering positioning model as described above are implemented.

[0103] In summary, the present invention provides an image processing method, a terminal and a computer-readable storage medium based on an image tampering positioning model, wherein the image tampering positioning model includes: a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer; the image processing method includes: inputting a tampered image into the image tampering positioning model, extracting manual features of the tampered image through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer; inputting the manual features into the dual convolution feature extraction layer to obtain depth features; inputting the depth features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling respectively; inputting the features obtained by upsampling into the output convolution layer, and outputting the final predicted mask map through the output convolution layer, wherein the mask map is used to predict the tampered area of the tampered image. The present invention uses the mask corresponding to the tampered image to perform end-to-end expansion of the tampered image data volume. Without the need for pre-training with a large-scale image tampering dataset, the image tampering localization model is trained end-to-end through self-supervised data enhancement, thereby achieving tampering localization performance similar to or even better than that of the pre-training method.

[0104] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0105] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When executed, the program can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0106] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An image processing method based on an image tampering localization model, characterized in that: The image tampering localization model includes: a DCT-Layer preprocessing convolution layer or an SRM-Layer preprocessing convolution layer, a dual convolution feature extraction layer, four downsampling convolution modules, four upsampling convolution modules and an output convolution layer; the image processing method based on the image tampering localization model includes: Inputting the tampered image into the image tampering localization model, and extracting manual features of the tampered image through the DCT-Layer preprocessing convolution layer or the SRM-Layer preprocessing convolution layer; The SRM-Layer preprocessing convolution layer consists of the SRM-Conv convolution layer and the truncation calculation, wherein the convolution kernel of the SRM-Conv is initialized by 30 5*5 high-pass filters with a step size of 1, and the 30 high-pass filters give different sensitivities; Inputting the manual features into a dual convolutional feature extraction layer to obtain deep features; The dual convolution feature extraction layer includes two convolution layers, two batch normalization layers and two ReLU activation functions, wherein the convolution kernel size of the two convolution layers is 3*3 and the stride is 1; The dual convolution feature extraction layer is composed of a convolution layer, a batch normalization layer, a ReLU activation function, a convolution layer, a batch normalization layer and a ReLU activation function in series; Input the depth features into four downsampling convolution modules and four upsampling convolution modules for downsampling and upsampling respectively; The up-sampling convolution module and the down-sampling convolution module at the same level are skip-connected; The first downsampling convolution module of the four downsampling convolution modules consists of block1 and block2, the second downsampling convolution module consists of block3, the third downsampling convolution module consists of block4, and the fourth downsampling convolution module consists of block5 and block6; Each of the four up-sampling convolution modules consists of a bilinear interpolation upsampling layer and a double convolution feature extraction layer; The features obtained by upsampling are input to the output convolution layer, and the final predicted mask map is output through the output convolution layer. The mask map is used to predict the tampered area of the tampered image.

2. The image processing method based on the image tampering localization model according to claim 1, characterized in that: The DCT-Layer preprocessing convolution layer is composed of a DCT-Conv convolution layer and a truncation calculation, wherein the convolution kernel of the DCT-Conv is initialized by a DCT kernel of size 4*4*6 with a step size of 1.

3. The image processing method based on the image tampering localization model according to claim 1, characterized in that: The convolution output layer consists of a convolution layer and a sigmoid activation function, wherein the convolution kernel size of the convolution layer is 1*1, the stride is 1 and the output channel is 1.

4. The image processing method based on the image tampering localization model according to claim 1, characterized in that: The image processing method based on the image tampering positioning model also includes: Two tampered images are input into the image tampering localization model, a tampered area is selected according to the mask corresponding to one of the tampered images, and the tampered area is randomly pasted onto the other tampered image to output a new tampered image.

5. The image processing method based on the image tampering localization model according to claim 1, characterized in that: The image processing method based on the image tampering positioning model also includes: A tampered image is input into the image tampering localization model, a tampered area is selected according to the mask corresponding to the tampered image, and the tampered area is randomly pasted to other areas of the tampered image to output a new tampered image.

6. A terminal, characterized in that: The terminal includes: a memory, a processor, and an image processing program based on an image tampering positioning model stored in the memory and runnable on the processor. When the image processing program based on the image tampering positioning model is executed by the processor, the steps of the image processing method based on the image tampering positioning model as described in any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an image processing program based on an image tampering positioning model, and when the image processing program based on an image tampering positioning model is executed by a processor, the steps of the image processing method based on an image tampering positioning model according to any one of claims 1 to 5 are implemented.

Images