A method for detecting inpainted images based on multi-level gating units

Abstract

A method for detecting repaired images based on a multi-level gating unit belongs to the field of multimedia information security and image recognition technology. The method first synthesizes a new repair dataset based on two existing datasets. Secondly, the dual-stream encoder branch extracts the spatial features and local noise residuals of different levels of the repaired image, splices them on the channel, and inputs them into the receptive field module. After passing through the multi-branch convolution layer and the hole convolution layer, the multi-scale features of the repaired image are extracted. Finally, according to the spatial features, local noise residuals and multi-scale features, the multi-level gating unit is used to generate gating features of different levels, and the decoder branch is used to perform feature fusion, and output the classification of the repaired image, the repair area positioning and the image segmentation results. The present invention solves the problems of poor detection accuracy and lack of generalization ability of existing methods for unknown repair means, suppresses interference features while highlighting the repair features.

Description

Technical Field

[0001] The present invention belongs to the technical field of multimedia information security and image recognition, and in particular relates to a repaired image detection method based on a multi-level gating unit. Background Art

[0002] With the rapid development of digital imaging technology and the popularity of image editing software, even ordinary people can easily modify complex image content using image restoration techniques. When image restoration technology is maliciously used in serious fields such as news media and judicial evidence collection, or when inflammatory restoration images are disseminated online, it can pose a significant threat to social fairness and stability. However, existing restoration detection methods still face numerous challenges. For example, they only target specific restoration algorithms for evidence collection, have poor detection accuracy for unknown restoration methods, and lack generalization capabilities. Furthermore, some algorithms use convolutional neural networks for feature extraction, which results in the network primarily learning the content features of the restored image and ignoring subtle traces left behind during the restoration process. Such algorithms also suffer from poor robustness to post-processing operations associated with restoration manipulation, such as JPEG compression and channel transmission noise. Therefore, a universal image restoration detection method is urgently needed that not only generalizes well to unknown restoration methods but also enables pixel-level detection of restored areas. Summary of the Invention

[0003] The purpose of the present invention is to provide a repaired image detection method that can determine whether an image has been repaired or tampered with, and can also provide a segmentation mask of the repaired area, thereby effectively improving the accuracy of locating the repaired area.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for detecting inpainted images based on a multi-level gating unit comprises the following steps:

[0006] Step S1: synthesize a new restoration dataset based on two existing datasets, divide it into a training set and a test set, and perform data preprocessing on the restoration images in the training set and the binary masks corresponding to the restoration images.

[0007] Step S2: Input any image in the training set into the dual-stream encoder branch, extract the spatial features and local noise residuals of different levels of the repaired image, and splice the top-level spatial features and local noise residuals on the channel.

[0008] Step S3: The feature image obtained after splicing is input into the receptive field module, and the multi-scale features of the repaired image are extracted through the multi-branch convolution layer and the hole convolution layer in the receptive field module.

[0009] Step S4: Input the spatial features, local noise residuals and multi-scale features into the multi-level gating unit to generate gating features at different levels.

[0010] Step S5: The decoder branch performs feature fusion on the input gated features of different levels and outputs the classification, repair area positioning and image segmentation results of the repaired image.

[0011] Step S6: The dual-stream encoder branch, receptive field module, multi-level gating unit and decoder constitute the inpainted image detection model. The decoder output of this round of training is input into the loss function of the pre-built inpainted image detection model, and the training loss value of the inpainted image detection model is calculated until the loss values calculated for all images in the training set are lower than the preset loss threshold, or until the number of training rounds reaches the preset rounds, to obtain a trained inpainted image detection model.

[0012] Furthermore, the dual-stream encoder branch includes a ResNet50 network for extracting spatial features and three high-pass filter structures for extracting noise residuals.

[0013] Furthermore, the three high-pass filtering structures include: a pre-filtering module (PF), a constrained convolution structure and an improved random high-pass filter (IRHP).

[0014] The step S1 comprises:

[0015] S11, repair and tamper the images in the two existing datasets to generate training sets and test sets for image repair detection, and modify the shapes to basic rectangles, circles, lines, etc.

[0016] S12, perform data augmentation on half of the repaired images in the training set: the image set before repair is denoted as D ori , the restored image set is D ip , the true mask corresponding to the restored image set is D gt , the following formula is used to enhance the data of the images in the training set:

[0017]

[0018] Among them, the intensity factor β∈(0,1], It is element-wise addition, and the strength factor β is used to control the strength of data augmentation.

[0019] S13, resize the resolution of all images in the training set and their corresponding ground-truth masks to 256×256.

[0020] S14: Randomly select 50% of the images in the training set to perform horizontal flipping, and the remaining images to perform vertical flipping.

[0021] S15, converts all pixel values of the three-channel images in the training set to the range of [0,1] through linear transformation.

[0022] The step S2 comprises:

[0023] S21, input image It is an RGB image, where H represents the image height, W represents the image width, and C represents the number of channels of the image. It is processed by the pre-filtering module PF, the constrained convolution structure, and the improved random high-pass filter IRHP to capture the local noise residual, and the residual features are obtained by splicing on the channel.

[0024] Furthermore, in step S21, for the input three-channel RGB image, the pre-filtering module PF performs convolution on each channel using a 3×3 high-pass filter kernel with a step size of 1. The obtained noise residual is spliced on the channel and used as the input of the next layer. The pre-filtering module PF includes three basic filters, and the kernel of the basic filter is:

[0025]

[0026] Furthermore, the first convolutional layer of the constrained convolutional structure in step S21 has the following constraints:

[0027]

[0028] in Indicates the weight of the kth convolution kernel at position (m,n) in the first convolution layer. is the center position of the convolution kernel.

[0029] Furthermore, the first convolution layer of the constrained convolution structure first initializes the weight of the center position of the convolution kernel to -1, and then normalizes the weights on its neighborhood so that their sum is 1.

[0030] Furthermore, in step S21, the IRHP filter randomly initializes a high-pass filter template of size 5×5, and the kernel of the basic filter is:

[0031]

[0032] Among them, w m ,m=1,2,…,N(N=24) and C are the convolution kernel weights, w mis a random variable that obeys a simple uniform distribution, and C is an unknown constant. In order to initialize the convolution kernel as a high-pass filter, it is necessary to make w m ,m=1,2,…, the sum of the mathematical expectations of N is -C. Then the weight w m The variance should satisfy the following formula:

[0033]

[0034] According to the above formula, w can be calculated j The uniform distribution interval is as follows:

[0035]

[0036] When the IRHP is initialized according to the 5×5 convolution kernel template, N=5×5-1=24, w m The uniform distribution interval is U(-0.7071,0) or U(0, 0.7071).

[0037] S22, the dual-stream encoder takes the input image The residual features E are input into the first four layers of ResNet50 pre-trained on the ImageNet dataset, and the spatial domain features F of each layer are extracted respectively. i ,i=0,1,2,3 and residual features E j ,j=0,1,2,3;

[0038] Among them, the features of the first four layers of the encoder output are:

[0039] S23, concatenates the top-level spatial feature F3 and the residual feature E3 on the channel and inputs them into the receptive field module.

[0040] Furthermore, the multi-branch convolutional layer of the receptive field module in step S3 contains four branches, the first three branches use a bottleneck layer (Bottleneck) as the top structure, and the last branch is a skip connection.

[0041] Among them, Bottleneck includes a 1×1 convolutional layer and two 3×3 convolutional layers connected sequentially to reduce the number of feature channels.

[0042] Furthermore, the dilated convolution layer of the receptive field module includes dilated convolutions with dilated rates of 1, 3, and 5, respectively, corresponding to convolution layers with different kernel sizes in the upper layer. The outputs of the three dilated convolution layers are concatenated on the channel, and the number of channels is reduced by 1×1 convolution, and the residual features are learned through jump connections.

[0043] The step S4 comprises:

[0044] S41, the calculation formula of the gate feature is as follows:

[0045] h i =Conv(Gap(Cat(F i ,E i )))

[0046]

[0047] Among them, Cat represents splicing on the channel, Gap represents global average pooling, Conv is the convolution layer (including convolution, pooling, and ReLU activation function), represents the element-wise dot product, G i,i＝0,...3 are the output features of the three different levels of the gated unit.

[0048] Furthermore, for the spatial features F from the dual-stream encoder branch i ,i=0,1,2,3 and residual features E j ,j=0,1,2,3, respectively, the spatial features and residual features of the same size are spliced on the channel to obtain the spliced features.

[0049] Furthermore, the spliced features are subjected to global average pooling and a convolution layer with a convolution kernel size of 1×1. The obtained feature h is element-wise dot product with the spliced features, and finally passed through a convolution layer with a kernel size of 3×3 to obtain a gated feature with the same size as the input feature.

[0050] Furthermore, the step S5 includes:

[0051] S51, for the gate feature G i,i>0 , the decoder uses bilinear interpolation to upsample it once, so that the length and width of the feature are expanded to twice the original.

[0052] S52, using convolutional layers to reduce the number of feature channels, and G i-1 Do element-wise addition.

[0053] Furthermore, the calculation formula of the decoder is as follows:

[0054]

[0055] Among them, Up represents bilinear interpolation upsampling, Conv is a convolutional layer with a kernel size of 3×3, The decoder further combines the gated features {G0, G1, G2, G3} by upsampling, convolution and element-by-element addition to obtain the decoding features

[0056] S54, the decoded feature S3 is converted into a single-channel prediction mask output by the repair image detection model through a classification head consisting of upsampling and Sigmoid activation function

[0057] When the step S6 trains the repair image detection model, the loss function L includes the focus loss L Focal and intersection loss L Iou , as shown below:

[0058]

[0059] Where y is the true mask of the input image, is the prediction mask output by the model, and λ1 and λ2 are hyperparameters.

[0060] The technical effects of the present invention are:

[0061] The present invention's inpainted image detection method, based on a multi-level gating unit, effectively addresses the problems of poor detection accuracy and lack of generalization for unknown inpainting methods in existing methods. Furthermore, the encoder-decoder architecture commonly used in image segmentation tasks often suffers from a lack of control over information exchange between the encoder and decoder. By constructing a multi-level gating unit to control the feature proportions at different levels of the encoder output, interfering features are suppressed while highlighting inpainted features. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 Flow chart of the method of the present invention;

[0063] Figure 2 This is a diagram of the network model structure in an embodiment of the present invention;

[0064] Figure 3 This is a structural diagram of the receptive field module in an embodiment of the present invention;

[0065] Figure 4 2 is a structural diagram of a gate control unit in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to better understand the technical solution of the present invention, the present invention is described in detail below with reference to the accompanying drawings and embodiments. It should be clear that the described embodiments and all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] Example 1:

[0068] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting a repaired image based on a multi-level gating unit, the method comprising the following steps:

[0069] Step S101: synthesize a new restoration dataset based on the Dresden and Places datasets, divide it into a training set and a test set, and perform data preprocessing on the restoration images in the training set and the binary masks corresponding to the restoration images.

[0070] Step S102: Input any image in the training set into the dual-stream encoder branch, which extracts spatial features and local noise residuals at different levels of the repaired image, and splices the top-level spatial features and noise features on the channel.

[0071] Step S103: Input the spliced feature image into the receptive field module, extract the multi-scale features of the repaired image through the multi-branch convolution layer and the hole convolution layer in the receptive field module, and input the multi-scale features into the multi-level gating unit.

[0072] Step S104: Generate gating features of different levels through a multi-level gating unit according to the input spatial features, local noise residuals and multi-scale features.

[0073] Step S105: The decoder branch performs feature fusion on the input gated features of different levels, and outputs the classification, restoration area positioning and image segmentation results of the restoration image.

[0074] Step S106: The dual-stream encoder branch, receptive field module, multi-level gating unit and decoder constitute a repaired image detection model, and the decoder prediction value of this round of training is input into the loss function of the pre-built repaired image detection model.

[0075] The training loss value of the repaired image detection model is calculated until the loss values calculated for all images in the training set are lower than the preset loss threshold, or until the number of training rounds reaches the preset number of rounds, and a trained repaired image detection model is obtained.

[0076] Example 2:

[0077] As an implementation method, in the embodiment of the present invention, the data preprocessing method is to generate a training set based on the Dresden and Places datasets, and the original image set is denoted as D ori , the restored image set is D ip , the corresponding real mask is D gt , and the following formula is used to enhance the data of the images in the training set:

[0078]

[0079] Among them, the intensity factor β∈(0,1], The intensity factor β is used to control the strength of data enhancement. The smaller the value of β, the fewer the restoration traces and the closer the restored image is to the original image.

[0080] After adjusting the resolution of all images in the training set and their corresponding ground-truth masks to 256×256, 50% of the images in the training set are randomly selected to perform horizontal flipping, and the remaining images are performed vertical flipping. Finally, all pixel values of the three-channel images in the training set are converted to the range of [0, 1] through linear transformation.

[0081] like Figure 2 The structural diagram of the constructed repaired image detection model shown in the figure includes: a dual-stream encoder branch, a receptive field module, a multi-level gating unit and a decoder branch.

[0082] The dual-stream encoder branches are used to extract the spatial features and noise residuals of the RGB image in parallel. The receptive field module is used to extract features with a large receptive field and rich contextual information. The multi-level gating unit is used to enable the model to measure the contribution of each encoder output block and highlight more discriminative restoration features. In the process of upsampling the output features of the gating unit, the decoder is used to fuse information at different levels.

[0083] As an implementable embodiment, in an embodiment of the present invention, the dual-stream encoder branch includes a ResNet50 network for extracting spatial features and three high-pass filtering structures for extracting noise residuals.

[0084] As an implementable embodiment, in an embodiment of the present invention, the backbone network of the encoder branch adopts a ResNet50 network; the ResNet50 network refers to the first four layers of the ResNet50 network pre-trained on the ImageNet dataset, and the number of convolution kernels used in the first four layers are 64, 256, 512 and 1024 respectively, and the output features of each layer are extracted.

[0085] As an implementable method, in an embodiment of the present invention, three high-pass filtering structures include: a pre-filtering module PF, a constrained convolution structure, and an improved random high-pass filter IRHP filter.

[0086] Specifically, the pre-filtering module PF uses a learnable pre-filtering module to extract the residual noise of the image, as shown in the figure below. For the input three-channel RGB image, the pre-filtering module PF performs convolution on each channel using a 3×3 high-pass filter kernel with a stride of 1. The resulting noise residuals are concatenated across the channels and used as the input to the next layer:

[0087]

[0088] Specifically, the constrained convolution structure can suppress image content features and adaptively learn low-order restoration features. The first convolution layer filter of the constrained convolution structure adds additional constraints to capture the changes in the relationship between adjacent pixels in the image caused by restoration. The constraints of the first convolution layer are as follows:

[0089]

[0090] in, Indicates the weight of the kth convolution kernel at position (m,n) in the first convolution layer. The first convolutional layer initializes the weight of the kernel center to -1 and then normalizes the weights of its neighborhood so that their sum is 1. The constrained convolutional architecture updates the weights via backpropagation after each training iteration, thus achieving adaptive learning of the image preprocessing convolution kernel.

[0091] Specifically, the IRHP filter is an improved random high-pass filter initialization method. When initializing the first layer weights of a deep neural network, the IRHP filter generates a set of random high-pass filters and maintains the stability of the data flow at the input and output of the convolution kernel.

[0092] Specifically, for the input features of the neural network, the IRHP filter randomly initializes a high-pass filter template of size 5×5, as shown in the following figure:

[0093]

[0094] Among them, w i,i＝1,2,…，N (N=24) and C are the convolution kernel weights, w i is a random variable that obeys a simple uniform distribution, C is an unknown constant, x i,i＝1,2,…，N (N=25) is the characteristic matrix of the input network. In order to initialize the convolution kernel as a high-pass filter, it is necessary to make w i,i＝1,2,…，N The sum of the mathematical expectations is -C. Then the weight w i The variance should satisfy the following formula:

[0095]

[0096] According to the above formula, w can be calculated i The uniform distribution interval is as follows:

[0097]

[0098] When initialized according to the 5×5 convolution kernel template, N=5×5-1=24, w iThe uniform distribution interval is U(-0.7071,0) or U(0, 0.7071). The following figure shows a randomly initialized convolution kernel of the IRHP filter:

[0099]

[0100] Specifically, a three-channel RGB image After passing through the pre-filtering module PF, the constrained convolution structure and the IRHP filter, the feature maps obtained are spliced on the channel, where H represents the image height, W represents the image width, and C represents the number of channels of the image. The spliced feature map y c ∈(H,W,15) is input to the first four layers of the ResNet50 network to mine richer noise residual features.

[0101] Specifically, the feature dimensions of each layer output of the dual-stream encoder provided by the embodiment of the present invention are:

[0102] Example 3:

[0103] On the basis of the above-mentioned embodiment 2, the embodiment of the present invention adopts a receptive field module to increase the receptive field of the features extracted by the dual-stream encoder.

[0104] Specifically, the specific structure of the receptive field module is as follows Figure 3 As shown in the figure, it mainly includes multi-branch convolutional layers and hole convolutional layers.

[0105] Specifically, the multi-branch convolutional layer contains four branches. The first three branches use Bottleneck as the top-level structure, and the last branch uses a skip connection. Bottleneck consists of sequentially connecting a 1×1 convolutional layer to reduce the number of feature channels and two 3×3 convolutional layers.

[0106] Specifically, the dilated convolution layer uses dilated convolutions with dilation rates of 1, 3, and 5, which correspond to the convolution layers of different kernel sizes in the upper layer respectively. After the outputs of the three dilated convolution layers are spliced on the channel, the number of channels is reduced through 1×1 convolution, and finally the residual features are learned through jump connections to protect the integrity of the input information.

[0107] Example 4:

[0108] On the basis of the above-mentioned embodiment 3, a multi-level gating unit is used in the embodiment of the present invention to enhance important restoration representations and suppress the activation of useless information.

[0109] Specifically, the specific structure of the multi-stage gate control unit is as follows: Figure 4 As shown, the spatial features F from the encoder branch i,i＝0,…3 and residual features E i,i＝0,...3The corresponding parts of the feature maps with the same size are taken and spliced on the channel; the spliced features first pass through global average pooling and a convolution layer with a convolution kernel size of 1×1, then do element-by-element dot product with the original features, and pass through a convolution layer with a kernel size of 3×3 to obtain gated features consistent with the input feature size.

[0110] Specifically, the calculation formula of the gate control unit is as follows:

[0111] h i =Conv(Gap(Cat(F i ,E i )))

[0112]

[0113] Among them, Cat represents splicing on the channel, Gap represents global average pooling, Conv is the convolution layer (including convolution, pooling, and ReLU activation function), represents the element-wise dot product, G i,i＝0,...3 is the output feature of the gate unit.

[0114] Example 5:

[0115] On the basis of the above embodiment 4, the embodiment of the present invention uses the decoder branch to process the output feature G of the gate control unit. i,i＝0,…3 The information at different levels is fused during upsampling.

[0116] Specifically, for the high-dimensional feature G i,i>0 The decoder first uses bilinear interpolation to upsample it, so that the length and width of the feature are expanded to twice the original, and then uses the convolution layer to reduce the number of channels of the feature, and then combines it with G i-1 Doing element-by-element addition, the decoder's calculation process can be expressed as follows:

[0117]

[0118] Among them, Up represents the upsampling operation, Conv is a convolutional layer with a kernel size of 3×3, The decoder combines the gated features {G0, G1, G2, G3} by upsampling, convolution, and element-by-element addition to obtain the decoding features.

[0119] Specifically, S3 is finally converted into a single-channel prediction mask output by the model through a classification head consisting of upsampling and Sigmoid activation function.

[0120] Example 6:

[0121] Based on the above embodiment 5, this embodiment of the present invention provides a hybrid loss function that combines focal loss and intersection-over-union loss to complete the iterative optimization of the model. The calculation formula is as follows:

[0122]

[0123] Among them, λ1 and λ2 are hyperparameters, L Focal is the focal loss, L Iou is the intersection loss.

[0124] Specifically, the focus loss L Focal A dynamic scaling factor α is introduced to reduce the weight of easily identifiable samples during training. This loss function effectively alleviates the problem of a large gap between the number of positive and negative samples in the training set by dynamically adjusting the ratio between the loss values of positive and negative samples. Its calculation formula is as follows:

[0125]

[0126] Where y is the true mask of the input image, is the prediction mask output by the model, and α,γ∈(0,1) are hyperparameters.

[0127] Specifically, the intersection-over-union loss L Iou It is used to evaluate the overlap between the predicted target and the real target in the image. The calculation formula is as follows:

[0128]

[0129] Among them, y ij To repair the true label of pixel (i, j) in the image, Represents the model's predicted value for pixel (i, j).

[0130] The inpainted image detection method based on a multi-level gating unit provided in an embodiment of the present invention constructs a multi-level gating unit to control the feature proportions of different levels output by the encoder, suppressing interference features while highlighting the inpainted features. The top-level features output by the encoder are spliced on the channel and then passed to the receptive field module to extract multi-scale information and enhance the discriminability of the inpainted features.

[0131] The output of the encoder branch is cascaded with the gating module and input into the decoder branch, which can provide effective context information for the decoder branch while minimizing interference information. During the training process, the data enhancement strategy is used to increase the proportion of difficult samples in the training set, increase the detection difficulty, and prompt the model to learn more accurate repair features.

[0132] In order to evaluate the advantages of the above method proposed by the present invention compared with the existing mainstream method, the present invention was conducted in Dresden 0 and Places[2] Model comparison experiments are conducted on the dataset.

[0133] The main models compared in this experiment include ManTra-Net [3] MVSS-Net [4] HP-FCN [5] and IID-Net [6] .

[0134] In order to fairly compare the performance of each model on the same dataset, the training set and validation set were prepared by Yu et al. [7] The proposed image restoration method is generated. The test set is generated by 10 restoration methods, six of which are based on deep learning: GC [7] , CA [8] SH [9] 、EC

[10] , LB

[11] and RN

[12] , the remaining 4 are traditional modification methods: TE

[13] 、NS

[14] 、PM

[15] and SG

[16] .

[0135] Tables 1 and 2 show the comparison results of AUC values and F1 scores of the proposed method and four existing methods on the test set generated by the traditional image restoration method.

[0136] Table 1 AUC values on the test set generated by traditional image restoration methods

[0137]

[0138] As can be seen from the data in the table, the proposed method surpasses the comparison algorithms in both AUC and F1 scores on almost all four test sets, achieving superior performance to mainstream models. On the test set synthesized using traditional restoration methods, the proposed method achieves an AUC exceeding 95, and an average F1 score of 85.84, exceeding the runner-up by 1.27 points.

[0139] Tables 3 and 4 show the AUC values and F1 scores of the present invention and four existing methods on the test set generated by the deep learning-based image restoration method.

[0140] Table 3 AUC values on the test set generated by deep learning-based image restoration methods

[0141]

[0142] Table 4 F1 scores on the test set generated by deep learning-based image restoration methods

[0143]

[0144] In order to verify the effectiveness of each component of the deep repair detection model based on multi-level gating units, the present invention conducted multiple ablation experiments on 10 test sets, and demonstrated the impact of each module on network performance using the average AUC value and F1 score as evaluation indicators. First, in the selection of the encoder backbone network, VGG-16, ResNet and ResNet50 were selected as the backbone network to test the output results of the model. Secondly, the effects of adding receptive field modules and multi-level gating units on the network prediction results were tested respectively. Finally, the impact of the data enhancement strategy with the core idea of increasing the difficulty of training sample detection on the detection results was verified.

[0145] Table 5 shows the results of four groups of ablation experiments. In terms of the selection of the backbone network, it can be seen that the best model performance is obtained by using ResNet50 as the backbone network. From the fourth column of Table 5, it can be seen that the addition of the receptive field module effectively improves the average AUC value and F1 score. This is because the receptive field module uses multi-branch convolution and pooling to flexibly adjust the receptive fields of different branches, which helps to extract the edge features of the repaired image. From the fifth column of Table 5, it can be seen that the multi-level gating unit constructed by the present invention greatly improves the average AUC value and F1 score of the network on all test sets, respectively, by about ten percentage points, which fully verifies the effectiveness of the gating unit for image repair detection tasks. Finally, based on the structure of the present invention, the data enhancement strategy implemented on the training set further improves the performance of the network, proving that increasing the detection difficulty of the training samples can prompt the model to learn more refined distinguishing features. From the average AUC value and F1 score, it can be seen that the components of the present invention effectively improve the model's ability to recognize the repaired area.

[0146] Table 5 Ablation experiment results

[0147]

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0149] References:

[0150] [1]Gloe T, R.The'Dresden Image Database'for benchmarking digitalimageforensics[C] / / Proceedings of the 2010ACM symposium on appliedcomputing.2010:1584-1590.

[0151] [2]Zhou B,Lapedriza A,Khosla A,et al.Places:A10million image databasefor scenerecognition[J].IEEE transactions on pattern analysis and machineintelligence,2017,40(6):1452-1464.

[0152] [3]Wu Y,AbdAlmageed W,Natarajan P.Mantra-net:Manipulation tracingnetwork for detectionand localization of image forgeries with anomalousfeatures[C] / / Proceedings of the IEEE / CVFConference on Computer Vision andPattern Recognition.2019:9543-9552.

[0153] [4]Chen X,Dong C,Ji J,et al.Image manipulation detection by multi-view multi-scalesupervision[C] / / Proceedings of the IEEE / CVF InternationalConference on Computer Vision.2021:14185-14193.

[0154] [5]Li H,Huang J.Localization of deep inpainting using high-pass fullyconvolutionalnetwork[C] / / Proceedings of the IEEE / CVF International Conferenceon Computer Vision.2019:8301-8310.

[0155] [6]Wu H,Zhou J.IID-Net:Image Inpainting Detection Network via NeuralArchitecture Search andAttention[J].IEEE Transactions on Circuits and Systemsfor Video Technology,2021.

[0156] [7]Yu J,Lin Z,Yang J,et al.Free-form image inpainting with gatedconvolution[C] / / Proceedings ofthe IEEE / CVF international conference oncomputer vision.2019:4471-4480.

[0157] [8]Yu J,Lin Z,Yang J,et al.Generative image inpainting withcontextual attention[C] / / Proceedings of the IEEE conference on computervision and pattern recognition.2018:5505-5514.

[0158] [9]Yan Z,Li X,Li M,et al.Shift-net:Image inpainting via deepfeaturerearrangement[C] / / Proceedings of the European conference on computervision(ECCV).2018:1-17.

[0159]

[10] Nazeri K,Ng E,Joseph T,et al.Edgeconnect:Generative imageinpainting with adversarial edgelearning[J].arXiv preprint arXiv:1901.00212,2019.

[0160]

[11] Wu H,Zhou J,Li Y.Deep generative model for image inpainting withlocal binary patternlearning and spatial attention[J].IEEE Transactions onMultimedia,2021,24:4016-4027.

[0161]

[12] Yu T,Guo Z,Jin X,et al.Region normalization for image inpainting[C] / / Proceedings of theAAAI conference on artificial intelligence.2020,34(07):12733-12740.

[0162]

[13] Telea A.An image inpainting technique based on the fast marchingmethod[J].Journal ofgraphics tools,2004,9(1):23-34.

[0163]

[14] Bertalmio M,Bertozzi AL,Sapiro G.Navier-stokes,fluid dynamics,andimage and videoinpainting[C] / / Proceedings of the 2001 IEEE Computer SocietyConference on Computer Visionand Pattern Recognition.CVPR 2001.IEEE,2001,1:I-I.

[0164]

[15] Herling J,Broll W.High-quality real-time video inpainting withPixMix[J].IEEE Transactionson Visualization and Computer Graphics,2014,20(6):866-879.

[0165]

[16] Huang J B,Kang S B,Ahuja N,et al.Image completion using planarstructure guidance[J].ACM Transactions on graphics(TOG),2014,33(4):1-10.

Claims

1. A method for detecting inpainted images based on a multi-level gating unit, characterized in that: The following steps are involved: Step S1: synthesize a new restoration dataset based on the two existing datasets, divide it into a training set and a test set, and perform data preprocessing on the restoration images in the training set and the binary masks corresponding to the restoration images; Step S2: Input any image in the training set into the dual-stream encoder branch, extract the spatial features and local noise residuals of different levels of the repaired image, and splice the top-level spatial features and local noise residuals on the channel; The dual-stream encoder branch includes a ResNet50 network for extracting spatial features and three high-pass filter structures for extracting noise residuals; The three high-pass filtering structures include: a pre-filtering module PF, a constrained convolution structure and an improved random high-pass filter IRHP; Step S3: Input the spliced feature image into the receptive field module, and extract the multi-scale features of the repaired image through the multi-branch convolution layer and the hole convolution layer in the receptive field module; The multi-branch convolutional layer of the receptive field module contains four branches, the first three branches use the bottleneck layer as the top structure, and the last branch is a skip connection; The dilated convolution layer includes dilated convolutions with dilation rates of 1, 3, and 5, which correspond to the convolution layers of the upper layer with different kernel sizes. The outputs of the three dilated convolution layers are concatenated on the channel, and the number of channels is reduced by 1×1 convolution, and the residual features are learned by skip connection. Step S4: Input the spatial features, local noise residuals and multi-scale features into the multi-level gating unit to generate gating features at different levels. The specific process is as follows: For the spatial feature F i ,i=0,1,2,3 and residual features E j ,j=0,1,2,3, respectively, splice the spatial features and residual features of the same size on the channel to obtain the spliced features; The spliced features are subjected to global average pooling and convolution layers, and the obtained feature h is element-wise dot product with the spliced features. Finally, a convolution layer is passed to obtain a gated feature with the same size as the input feature. Step S5: The decoder branch performs feature fusion on the input gated features at different levels and outputs the classification, restoration area positioning and image segmentation results of the restored image; Step S6: The dual-stream encoder branch, receptive field module, multi-level gating unit and decoder constitute the inpainted image detection model. The decoder output of this round of training is input into the loss function of the pre-built inpainted image detection model to calculate the training loss value of the inpainted image detection model. When the loss values calculated for all images in the training set are lower than the preset loss threshold, or the number of training rounds reaches the preset number, the trained inpainted image detection model is obtained.

2. The method for detecting inpainted images based on a multi-level gating unit according to claim 1, characterized in that: The specific process of step S1 is as follows: S11, repair and tamper the images in two existing datasets to generate training and test sets for image repair detection, and modify the shapes into basic rectangles, circles and lines; S12, perform data augmentation on half of the repaired images in the training set: the image set before repair is denoted as D ori , the restored image set is D ip , the true mask corresponding to the restored image set is D gt , the following formula is used to enhance the data of the images in the training set: Among them, the intensity factor β∈(0,1], is element-wise addition; S13, resize the resolution of all images in the training set and their corresponding ground-truth masks to 256×256; S14, randomly select 50% of the images in the training set to perform horizontal flipping operations, and the remaining images to perform vertical flipping operations; S15, converts all pixel values of the images in the training set to the range of [0,1] through linear transformation.

3. The method for detecting inpainted images based on a multi-level gating unit according to claim 2, characterized in that: The specific process of step S2 is as follows: S21, input image For RGB images, H represents the image height, W represents the image width, and C represents the number of channels of the image; the local noise residual is captured by the pre-filtering module PF, the constrained convolution structure and the improved random high-pass filter IRHP, and the residual features are obtained by splicing on the channel S22, input image The residual features E are input into the first four layers of the pre-trained ResNet50, and the spatial features F of each layer are extracted respectively. i ,i=0,1,2,3 and residual features E j ,j=0,1,2,3; Among them, the features of the first four layers of the encoder output are: S23, concatenate the top spatial feature F3 and the residual feature E3 on the channel.

4. The method for detecting inpainted images based on a multi-level gating unit according to claim 3, characterized in that: The specific process of step S5 is as follows: S51, for the gated feature G i,i>0 , the decoder uses bilinear interpolation to upsample it once, so that the length and width of the feature are expanded to twice the original; S52, using convolutional layers to reduce the number of feature channels, and G i-1 Do element-wise addition; The calculation formula of the decoder is as follows: Among them, Up represents bilinear interpolation upsampling, Conv is a convolutional layer with a kernel size of 3×3, The decoder combines the gated features {G0, G1, G2, G3} by upsampling, convolution, and element-by-element addition to obtain the decoding features. S54, the decoded feature S3 is converted into a single-channel prediction mask output by the repair image detection model through a classification head consisting of upsampling and Sigmoid activation function 5. The method for detecting inpainted images based on a multi-level gating unit according to claim 4, characterized in that: The loss function in step S6 includes the focus loss L Focal and intersection loss L Iou , as shown below: Where y is the true mask of the input image, is the prediction mask output by the model, and λ1 and λ2 are hyperparameters.

Images