A method and device for detecting tampered image data copied and pasted, a computer and a computer readable storage medium

Abstract

The application provides a copy-paste tampered image data detection method, device, computer and computer readable storage medium, and the method comprises the following steps: inputting a to-be-detected image into a double-branch feature extraction network to extract similarity features and edge artifact features respectively; inputting the multi-scale and multi-category features extracted by the double-branch into a feature fusion decoder module to perform feature fusion, so as to increase context complementary information and remove interference redundant information; and performing inverse decoding on the fused features through an attention module, and performing post-processing operations such as bilinear interpolation upsampling to output a final result. The application constructs a new deep learning framework, and completes a copy-paste tampered image source / target detection and positioning task by learning an existing public data set.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, computer, and computer-readable storage medium for detecting copied and pasted tampered image data. Background Technology

[0002] Image copy-paste tampering refers to copying certain areas (source areas) from an image and pasting these areas into other areas (target areas) of the same image to achieve the purpose of adding or removing content.

[0003] Current methods for detecting copy-paste tampering are mostly based on three steps: feature extraction, feature correlation matching, and post-processing.

[0004] Features refer to vectors calculated by algorithms that describe a local area or a series of pixels in an image. Copy-paste image manipulation can occur anywhere in an image, and its distribution is not predictable. Therefore, feature extraction of the entire image is often necessary for detection. The quality of the extracted features directly affects the final detection result. Feature correlation matching is a process of calculating the similarity between the aforementioned features and matching the source and target regions. The quality of the matching method affects matching accuracy and speed. Post-processing generally refers to the process of removing mismatches or feature decoding. The multiple steps and factors involved make copy-paste image manipulation detection a challenging technical problem.

[0005] The current mainstream solutions for detecting and locating copied and pasted images can be divided into two traditional methods: block feature-based methods and key point-based methods, as well as deep learning methods.

[0006] Traditional block-feature-based methods divide the image into overlapping blocks, then quantify the features of each sub-block, and finally perform feature matching between the sub-blocks. This method is effective for image manipulation tasks involving copy-paste; however, extracting accurate image block features requires certain prior conditions and has high algorithm complexity. This makes block-feature-based methods difficult to apply in practice.

[0007] Keypoint-based methods first extract keypoints from the image and perform feature descriptions on them. Then, they select similar regions by matching similar feature descriptions. Finally, post-processing is applied to reduce false matches. Keypoint-based methods are more efficient and robust than block-feature-based methods. However, when the image texture is not obvious or the detection area is smooth, the number of keypoints decreases sharply, causing the algorithm to fail to detect and locate tampered areas. Furthermore, the algorithm requires manual parameter adjustment; different parameter threshold settings under different conditions can lead to varying algorithm performance, requiring individual optimization for each module.

[0008] The first deep learning-based model first uses VGG16 to extract high-dimensional image features, then matches similar image locations by calculating feature cosine similarity, and finally uses an Inception-style decoder to decode the high-dimensional matching results into binary prediction results. This method showed some effectiveness on the test set, but there is still room for improvement in network performance. ZHU et al. added channel attention and position attention mechanisms to the feature extraction network, enabling the network to better focus on tampering semantic features. Recently, MVSS proposed a dual-branch network to simultaneously extract Bayar noise features and image edge features to locate tampering locations. However, since it does not only focus on detecting copy-paste tampering, its noise feature branch is not suitable for copy-paste image tampering localization tasks, which may disrupt the network's judgment.

[0009] On the other hand, while locating the tampering site is crucial in practical detection, distinguishing which regions were copied and which were pasted is equally important. Therefore, BusterNet proposed a two-branch end-to-end network and defined a source / target discrimination localization task (i.e., using three-class labels to segment the original region, source region, and target region). One branch detects the target region by extracting non-descriptive semantic features, while the other branch detects similar regions in the image by extracting similarity features. Finally, a three-class classifier distinguishes and locates the original, source, and target regions. DOA-GAN, using a generative adversarial model, also accomplishes the source / target discrimination localization task. Its generator is trained to generate three-class labels, while a discriminator is trained to determine whether the three-class labels are true or false.

[0010] Of the two existing methods, deep learning-based image copy-paste tampering detection and localization methods suffer from high computational cost in feature vector matching, resulting in an excessively large feature space for tampering detection. A common approach to avoid this high computational cost is to reduce the dimensionality of the feature space through downsampling. However, when the tampered region is small, downsampling significantly reduces the robustness of the feature representation, leading to low tampering detection accuracy. Furthermore, when the color and texture of the tampered region are similar to those of the background region, the extracted feature classification confusion matrix is ​​weak, resulting in a high false detection rate for tampering.

[0011] On the other hand, as research deepens, new requirements have been placed on image copy-paste tampering detection tasks, namely, the differentiation and localization of source / target regions in copy-paste tampering. Current research commonly distinguishes and localizes source / target regions based on the differences in their respective non-descriptive semantic features (the so-called non-descriptive semantic features refer to features extracted in the feature extraction step without targeted algorithmic design, using a deep learning network to forcibly fit the training set data distribution). However, in order to learn non-descriptive semantic features, networks often tend to overfit the dataset, leading to a significant decrease in network detection performance and weak interpretability.

[0012] Therefore, current research on deep learning-based image copy-paste tampering detection methods faces two challenges:

[0013] 1. Convolutional neural networks use convolution and downsampling operations to obtain long-range contextual information, but the resolution of features is continuously reduced in this process, losing a lot of image detail information, which is not conducive to subsequent pixel-level tampering detection tasks.

[0014] 2. Existing methods for classifying tampered source / target regions distinguish between source and target regions using non-descriptive semantic features. However, these methods tend to overfit the dataset, limiting the overall performance of the network and resulting in weak interpretability. Summary of the Invention

[0015] The technical problem to be solved by the present invention is to provide a method, device, computer and computer-readable storage medium for detecting copied and pasted tampered image data. A new deep learning framework is constructed to complete the task of detecting and locating the source / target of copied and pasted tampered images by learning from existing public datasets.

[0016] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0017] A first aspect includes a method for detecting copy-paste tampering of image data, the method comprising:

[0018] The image to be detected is input into a dual-branch feature extraction network to extract similarity features and edge artifact features respectively;

[0019] The multi-scale and multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion to increase contextual complementary information and remove interference and redundant information.

[0020] The fused features are de-decoded through an attention module, and then post-processed through bilinear interpolation upsampling and other operations to output the final result.

[0021] Furthermore, similarity features are extracted, including:

[0022] A The RGB color image is processed through four residual blocks of the RES-Net101 backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , ;

[0023] Assume that for any two points i on the feature map, the corresponding pixel value of j at that point is According to the formula:

[0024] ;

[0025] ;

[0026] The correlation between two feature pixels can be obtained, where, These represent the mean and standard deviation of the pixel value at point i for each channel, respectively.

[0027] The size was obtained through calculation. The similarity matrix is ​​obtained, and then input into the Inception-style inverse decoder and upsampled to complete the decoding. Finally, the result is used by a binary classifier to predict the binary coarse matching label, denoted as . .

[0028] Further edge artifact features include:

[0029] A The RGB color image is processed through four residual blocks of the RES-Net101 backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , ;

[0030] Assume that for any two points i on the feature map, the corresponding pixel value of j at that point is According to the formula:

[0031] ;

[0032] ;

[0033] The correlation between two feature pixels can be obtained, where, These represent the mean and standard deviation of the pixel value at point i for each channel, respectively.

[0034] The size was obtained through calculation. The similarity matrix is ​​obtained, and then input into the Inception-style inverse decoder and upsampled to complete the decoding. Finally, the result is used by a binary classifier to predict the binary coarse matching label, denoted as . .

[0035] Furthermore, the multi-scale, multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, including:

[0036] The four hierarchical features extracted from the similarity detection backbone network The artifact detection branch extracts features at four levels. Corresponding superposition as module input The calculation formula is as follows: in, For the i-th level feature of the artifact detection branch, For the i-th level feature of the similarity detection branch, This indicates feature channel overlay, and the resulting i-th layer feature is... Its size is ;

[0037] Features respectively Through a Convolution is used for resampling to fully integrate and connect the edge and similar information contained in the features.

[0038] Furthermore, the multi-scale, multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, which also includes:

[0039] The first Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Overlay;

[0040] Features are decoded using learnable decoding layers, each consisting of a... The decoding layer consists of a convolution and a deconvolution, and the formula is: In the above formula For the decoded first Layer features, For decoding function, This is for maxpooling operations.

[0041] The first Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Overlay;

[0042] Features are decoded using learnable decoding layers, each consisting of a... The decoding layer consists of a convolution and a deconvolution, and the formula is: In the above formula For the decoded first Layer features, For decoding function, This is for maxpooling operations.

[0043] Furthermore, the prediction map is a The size of the segmentation prediction map, denoted as ;

[0044] Will With label image Calculate the Dice loss, denoted as The calculation formula is as follows:

[0045] ;

[0046] in, Representative prediction chart The predicted probability value of the i-th pixel. This represents the label value of the i-th pixel in the mask label.

[0047] In a second aspect, a data processing apparatus is characterized by comprising:

[0048] The acquisition module is used to input the image to be detected into the dual-branch feature extraction network to extract similarity features and edge artifact features respectively;

[0049] The processing module is used to input the multi-scale and multi-class features extracted from the dual branches into the feature fusion decoder module for feature fusion, so as to increase the contextual complementary information and remove the interference and redundant information. The fused features are then passed through the attention module for inverse decoding, and after post-processing operations such as bilinear interpolation upsampling, the final result is output.

[0050] Thirdly, a computer for detecting copy-paste tampering of image data includes:

[0051] One or more processors;

[0052] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0053] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0054] The above-described solution of the present invention has at least the following beneficial effects:

[0055] The above-mentioned solution of the present invention requires no prior knowledge. Based on the generation characteristics of the source and target during the tampering process, a dual-branch network is constructed to simultaneously detect the similarity features and edge artifact features of the region. A top-down feature fusion method is designed to make up for the details lost during downsampling and avoid increasing the computational cost of feature matching. This invention greatly improves the accuracy and efficiency of the classification and source / target localization of copied and pasted tampered images. Attached Figure Description

[0056] Figure 1 A flowchart illustrating the image data tampering detection method provided in the embodiments of the present invention.

[0057] Figure 2 This is a schematic diagram of the overall network structure.

[0058] Figure 3 A schematic diagram of the branch structure for similarity feature extraction.

[0059] Figure 4 A schematic diagram of the branch structure for edge artifact feature extraction.

[0060] Figure 5 This is a schematic diagram of the Sobel layer structure.

[0061] Figure 6 This is a schematic diagram of the ERB module structure.

[0062] Figure 7 This is a schematic diagram of the fusion module structure.

[0063] Figure 8 This is a schematic diagram of the test results for the Uscisi dataset.

[0064] Figure 9 This is a schematic diagram of an image data tampering detection device provided in an embodiment of the present invention. Detailed Implementation

[0065] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0066] like Figure 1 As shown, an embodiment of the present invention proposes a method for detecting copied and pasted tampered image data, the method comprising the following steps:

[0067] Step 11: Feature extraction. Input the image to be detected into the dual-branch feature extraction network of this method to extract similarity features and edge artifact features respectively.

[0068] Step 12: Feature fusion. The multi-scale and multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, which adds contextual complementary information and removes interference and redundant information.

[0069] Step 13: Attention Decoding and Upsampling Post-processing. The fused features are de-decoded by the attention module and then post-processed by bilinear interpolation upsampling and other operations to output the final result.

[0070] In the data processing method described in this embodiment of the invention, no prior knowledge is required. Based on the generation characteristics of the source and target during the tampering process, a dual-branch network is constructed to simultaneously detect the similarity features and edge artifact features of the region. A top-down feature fusion method is designed to compensate for the details lost during downsampling and avoid increasing the computational cost of feature matching. This invention greatly improves the accuracy and efficiency of the classification and source / target localization of copied and pasted tampered images.

[0071] It should be noted that the network is divided into feature extraction and feature fusion, attention decoding, and post-processing. Feature extraction includes two branches with auxiliary tasks, where the auxiliary tasks guide the branches to extract corresponding features. The similarity detection branch is used to detect similar tampered regions in the image, while the artifact detection branch is used to detect subtle boundary artifacts of the tampered regions. The feature fusion module fuses the features output from the two branches in a top-down manner and outputs a three-class prediction map through dual-stream attention to complete pixel-level tamper prediction and source / target classification tasks.

[0072] Similarity detection branch network structure as follows Figure 3 As shown. In the similarity branch auxiliary task, we first take an image The RGB color image is processed through four residual blocks of the RES-Net101 backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , It is worth mentioning that only This will be used for similarity calculation in auxiliary tasks. This is because auxiliary tasks only require the guiding branch to extract semantic features of similar objects in the image; the last layer of features is sufficient to contain the necessary semantic information, and adding shallow feature information provides limited improvement. Furthermore, shallow features contain more feature details; excessively complex information can lead to excessively large computational loads, reducing network efficiency; on the other hand, complex information can interfere with the network's data fitting, increasing training difficulty. Therefore, in similarity auxiliary tasks, we only use... Perform cosine similarity calculation.

[0073] Assume that for any two points i on the feature map, the corresponding pixel value of j at that point is The correlation between two feature pixels can be obtained according to formulas (1) and (2). Wherein,

[0074] (1)

[0075] (2)

[0076] in , Each channel represents the current state of the channel. The mean and standard deviation of pixel values.

[0077] After the above calculations, a size of [value] can be obtained. The similarity matrix is ​​then input into an Inception-style inverse decoder and upsampled to complete the decoding. The final result is used by a simple binary classifier to predict binary coarse-matching labels, denoted as . The auxiliary task guides the branch to extract semantic features of similar objects in the image through such a similarity calculation process. In particular, the auxiliary task is only applied to the training process and does not participate in the testing and final prediction.

[0078] To avoid the network learning non-describable semantic features, we start from the characteristics of tampered region boundaries and define an auxiliary task for artifact detection, the structure of which is as follows: Figure 4 As shown.

[0079] Similar to similarity branches, take a sheet The RGB color image is processed through four residual blocks of the backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , These four scale features will be calculated through the Sobel layer and the ERB module respectively, and will output sizes of [size missing]. The feature maps, the structures of the Sobel layer and the ERB module are as follows: Figure 5 and Figure 6 As shown.

[0080] Unlike the similarity branch auxiliary task, the artifact detection branch auxiliary task uses... Features at four scales. This is because the auxiliary task of the artifact detection branch aims to guide the branch to extract feature information of image tampering boundary artifacts, while using only the deepest features would not suffice. It cannot contain enough detailed feature information. Therefore, by using feature information at multiple levels and making them complementary, the network can better complete the branch-assist task.

[0081] Specific methods are as follows Figure 4 As shown, a feature map containing only boundary artifact information is obtained through multi-level feature transfer and selection via a hierarchical ERB module structure. Finally, after sigmoid linear activation, a feature map of size [size missing] is output. The binary edge prediction map, denoted as .

[0082] This paper proposes a top-down multi-scale feature fusion method to integrate edge information from large-scale feature maps into small-scale feature maps. Four different levels of features are involved. Designed as input to the fusion module, its purpose is to enhance the contextual information between features. Specifically, by Figure 5 As shown, we extract four levels of features from the similarity detection backbone network. The artifact detection branch extracts features at four levels. Corresponding superposition as module input The formula is:

[0083] (3)

[0084] For the i-th level feature of the artifact detection branch, For the i-th level feature of the similarity detection branch, This indicates feature channel overlay, and the resulting i-th layer feature is... Its size is .

[0085] Subsequently, the features were respectively Through a Convolutional processing performs resampling. This is done to fully integrate and correlate edge and similarity information within the features. Resampling also reduces redundant channels, preventing sparse channel noise from negatively impacting subsequent operations.

[0086] Next, in order to complement and connect the contextual information at different levels, we designed to... Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Superposition. Finally, unlike previous bilinear interpolation decoding methods, we use learnable decoding layers to decode the features (each decoding layer consists of a...). (It consists of a convolution and a deconvolution). The decoding layer formula is:

[0087] (4)

[0088] In the above formula For the decoded first Layer features, For decoding function, This is for maxpooling operations.

[0089] After the above operations, we obtained four fused features. We believe that such a mechanism can transmit finer-grained feature information from the upper layers as a supplement and comparison to the lower-layer semantic features. This enhances the connection between contextual information while also removing noise from the lower-level semantic features.

[0090] It's worth noting that, theoretically, the design of the decoded feature resolution should not be too small, as this would lead to excessive loss of feature information and negatively impact model performance. On the other hand, excessively large resolution will also result in excessive computation and overly complex information, thus affecting model performance. Considering both aspects, the chosen approach is to set the decoded feature resolution to [missing information]. Unified as The goal is to create a model that balances performance and speed. Finally, after obtaining... Then, we feed it into a dual-attention output module to output a 3-channel RGB final prediction image.

[0091] The final predicted image output by the framework is a The size of the segmentation prediction map, denoted as Then we compared it with the label image. The Dice loss is calculated as follows: The formula is as follows:

[0092] (5)

[0093] Representative prediction chart The predicted probability value of the i-th pixel. This represents the label value of the i-th pixel in the mask label.

[0094] This invention, through experiments, reveals artifacts at the boundaries of the tampered target region in its edge artifact feature extraction module; that is, the edges of the target region exhibit discontinuities. Since attackers often rotate or scale the source image before pasting, the boundary points of the source image... After affine transformation, they become the boundary points of the target image. ( (Not integer values). Therefore, the boundary points of the target image have actually undergone quadratic interpolation. Using edge artifact features can, on the one hand, extract image edge features to give the model more fine-grained semantic information, which helps the network make more precise predictions in subsequent feature matching and other stages; on the other hand, by learning the artifact patterns existing in the S and T edge regions, the source and target categories of suspected tampered areas can be distinguished. The network does not need to overfit the dataset in order to learn non-descriptive semantic features of the image, thus affecting the overall performance of the network.

[0095] In the feature fusion network, a top-down feature fusion approach is constructed, which integrates high-resolution refined features into low-resolution features, making up for the details lost during downsampling, while avoiding increasing the computational cost of feature matching.

[0096] Compared with existing technologies, the greatest advantage of this invention lies in the fact that it requires no prior knowledge. Based on the generation characteristics of the source and target during the tampering process, a dual-branch network is constructed to simultaneously detect the similarity features and edge artifact features of the region. Furthermore, a top-down feature fusion method is designed to compensate for the detailed information lost during downsampling while avoiding increasing the computational cost of feature matching. This invention significantly improves the accuracy and efficiency of classifying and locating the source / target in copied-paste tampered images.

[0097] Ablation experiments on the Uscisi dataset have validated the effectiveness of the proposed method. Comparative experiments on multiple public datasets with state-of-the-art methods demonstrate the superior performance of the proposed method. The experiments consistently used F1-score and AUC values ​​as metrics.

[0098] (1) Ablation test

[0099] In the ablation experiments, we split the network, added modules step by step, and trained and evaluated them uniformly on the USCISI dataset. We first verified the effectiveness of the branch-assisted tasks. Specifically, we removed the artifact detection branch and used only RESNet-101 for direct end-to-end feature extraction, denoted as Our-w / o edg. Then, we evaluated the effectiveness of the fusion module. Specifically, we removed the feature fusion decoder from the complete model and directly superimposed the multi-level features extracted from the two branches, then output the classification prediction result through a dual attention module, denoted as Our-w / o fusion. In this experiment, BusterNet and DOA-GAN were used as benchmark networks, and the performance evaluation results are shown in Table 1.

[0100] Table 1. Performance indicators of each network in the ablation experiment

[0101]

[0102] Experimental results show that our method outperforms existing methods in both source and target regions. Furthermore, Figure 8 The results of our network's detection are shown. The first column is the input image, the second column is the three-class label corresponding to the image, and the third to seventh columns are the detection results of different methods on the input image. It can be seen that our method outperforms other current methods in terms of the details of the tampered region's edges. This is attributed to our dual-branch auxiliary task design, which allows the network to reduce noise-induced false detections and improve the performance of the network's detection results in terms of detail.

[0103] (2) Comparison experiments of each model on other datasets

[0104] Table 2 shows the pixel-level evaluation metrics of each model on the CASIA test set.

[0105]

[0106] Table 3 shows the pixel-level evaluation metrics for each model on the DEFACTO-cpmv test set.

[0107]

[0108] like Figure 9 As shown, an embodiment of the present invention also provides a copy-paste tampering image data detection device 20, comprising:

[0109] The acquisition module 21 is used to input the image to be detected into the dual-branch feature extraction network to extract similarity features and edge artifact features respectively;

[0110] The processing module 22 is used to input the multi-scale and multi-class features extracted from the dual branches into the feature fusion decoder module for feature fusion, so as to increase the contextual complementary information and remove the interference and redundant information; the fused features are then de-decoded by the attention module, and the final result is output after post-processing operations such as bilinear interpolation upsampling.

[0111] Optionally, similarity features can be extracted, including:

[0112] A The RGB color image is processed through four residual blocks of the RES-Net101 backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , ;

[0113] Assume that for any two points i on the feature map, the corresponding pixel value of j at that point is According to the formula:

[0114] ;

[0115] ;

[0116] The correlation between two feature pixels can be obtained. in 、 respectively This represents the mean and standard deviation of the pixel value at point i for each channel;

[0117] The size was obtained through calculation. The similarity matrix is ​​obtained, and then input into the Inception-style inverse decoder and upsampled to complete the decoding. Finally, the result is used by a binary classifier to predict the binary coarse matching label, denoted as . .

[0118] Optional edge artifact features include:

[0119] A The RGB color image is processed through four residual blocks of the backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , These four scale features will be calculated through the Sobel layer and the ERB module respectively, and will output sizes of [size missing]. Feature map;

[0120] Through a hierarchical ERB module structure, multi-level feature transfer and selection are performed to obtain a feature map containing only boundary artifact information. Finally, after sigmoid linear activation, the output size is... The binary edge prediction map, denoted as .

[0121] Optionally, the multi-scale, multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, including:

[0122] Through a hierarchical ERB module structure, multi-level feature transfer and selection are performed to obtain a feature map containing only boundary artifact information. Finally, after sigmoid linear activation, the output size is... The binary edge prediction map, denoted as ,

[0123] in, For the i-th level feature of the artifact detection branch, For the i-th level feature of the similarity detection branch, This indicates feature channel overlay, and the resulting i-th layer feature is... Its size is

[0124] Features respectively Through a Convolution is used for resampling to fully integrate and connect the edge and similar information contained in the features.

[0125] Optionally, the multi-scale, multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, and the module further includes:

[0126] The first Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Overlay;

[0127] Features are decoded using learnable decoding layers, each consisting of a... The decoding layer consists of a convolution and a deconvolution, and the formula is:

[0128] ,

[0129] In the above formula The i-th layer features after decoding For decoding function, This is for maxpooling operations.

[0130] Optionally, the multi-scale, multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion, and the module further includes:

[0131] Four fused features are obtained by decoding the features using a learnable decoding layer. ;

[0132] The four merged features The image is fed into a dual-attention output module to output a 3-channel RGB final prediction image.

[0133] Optionally, the prediction map is a The size of the segmentation prediction map, denoted as ;

[0134] Will With label image Calculate the Dice loss, denoted as The calculation formula is as follows:

[0135] ;

[0136] in, Representative prediction chart The predicted probability value of the i-th pixel. This represents the label value of the i-th pixel in the mask label.

[0137] It should be noted that this device is a device corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0138] Embodiments of the present invention also provide a computer for detecting copied and pasted tampered image data, comprising: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0139] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0145] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0146] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve by using their basic programming skills after reading the description of the present invention.

[0147] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0148] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0149] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting image data tampering through copying and pasting, characterized in that, The method includes: The image to be detected is input into a dual-branch feature extraction network to extract similarity features and edge artifact features respectively; The multi-scale and multi-class features extracted from the dual branches are input into the feature fusion decoder module for feature fusion to increase contextual complementary information and remove interference and redundant information. This includes: the four-level features extracted from the similarity detection branch backbone network. The artifact detection branch extracts features at four levels. Corresponding superposition as module input The calculation formula is as follows: in, For the i-th level feature of the artifact detection branch, For the i-th level feature of the similarity detection branch, This indicates feature channel overlay, and the resulting i-th layer feature is... Its size is ; respectively the features Through a Convolution is used for resampling to fully integrate and correlate edge and similarity information contained in the features; the 1st... Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Stacking; using learnable decoding layers to decode features, each decoding layer consisting of one The decoding layer consists of a convolution and a deconvolution, and the formula is: In the above formula For the decoded first Layer features, For decoding function, This is a maxpooling operation; features are decoded using a learnable decoding layer to obtain four fused features. ; The four merged features The image is fed into a dual-attention output module to output a 3-channel RGB final prediction image; The fused features are de-decoded through an attention module, and then post-processed through bilinear interpolation upsampling and other operations to output the final result.

2. The method for detecting copied and pasted tampered image data according to claim 1, characterized in that, Extract similarity features, including: A The RGB color image is processed through four residual blocks of the RES-Net101 backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , ; Assume that for any two points i on the feature map, the corresponding pixel value of j at that point is According to the formula: ； ； The correlation between two feature pixels can be obtained, where, These represent the mean and standard deviation of the pixel value at point i for each channel, respectively. The size was obtained through calculation. The similarity matrix is ​​obtained, and then input into the Inception-style inverse decoder and upsampled to complete the decoding. Finally, the result is used by a binary classifier to predict the binary coarse matching label, denoted as . .

3. The method for detecting copied and pasted tampered image data according to claim 2, characterized in that, Edge artifact features include: A The RGB color image is processed through four residual blocks of the backbone network to obtain four different scales of high-dimensional features, denoted as . The resolution sizes corresponding to the high-dimensional features at each scale are as follows: , , , These four scale features will be calculated through the Sobel layer and the ERB module respectively, and will output sizes of [size missing]. Feature map; Through a hierarchical ERB module structure, multi-level feature transfer and selection are performed to obtain a feature map containing only boundary artifact information. Finally, after sigmoid linear activation, the output size is... The binary edge prediction map, denoted as .

4. The method for detecting copied and pasted tampered image data according to claim 3, characterized in that, The prediction map is a The size of the segmentation prediction map, denoted as ; Will With label image Calculate the Dice loss, denoted as The calculation formula is as follows: ; in, Representative prediction chart The predicted probability value of the i-th pixel. This represents the label value of the i-th pixel in the mask label.

5. A device for detecting copied and pasted tampered image data, characterized in that, include: The acquisition module is used to input the image to be detected into the dual-branch feature extraction network to extract similarity features and edge artifact features respectively; The processing module is used to input the multi-scale and multi-class features extracted from the dual branches into the feature fusion decoder module for feature fusion, in order to increase contextual complementary information and remove interference and redundant information. This includes: extracting the four-level features from the similarity detection branch backbone network. The artifact detection branch extracts features at four levels. Corresponding superposition as module input The calculation formula is as follows: ,in, For the i-th level feature of the artifact detection branch, For the i-th level feature of the similarity detection branch, This indicates feature channel overlay, and the resulting i-th layer feature is... Its size is ; respectively the features Through a Convolution is used for resampling to fully integrate and correlate edge and similarity information contained in the features; the 1st... Features after layer resampling After adjusting the resolution using maxpooling, the result is passed to layer i+1. Stacking; using learnable decoding layers to decode features, each decoding layer consisting of one The decoding layer consists of a convolution and a deconvolution, and the formula is: In the above formula The i-th layer features after decoding For decoding function, This is a maxpooling operation; features are decoded using a learnable decoding layer to obtain four fused features. ; The four merged features The data is fed into a dual-attention output module to output a 3-channel RGB final prediction image; the fused features are then decoded by the attention module, and post-processing operations such as bilinear interpolation upsampling are performed to output the final result.

6. A computer for detecting copy-paste tampering of image data, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1-4.

Images