Multi-stage progressive image tampering detection and positioning method and device

Abstract

The invention discloses a multi-stage progressive image tampering detection and positioning method and device. The method is characterized by comprising the following steps: receiving an original image to be detected, and carrying out image segmentation on the original image to obtain a first segmented image and a second segmented image; and inputting the original image, the first segmented image and the second segmented image into a pre-trained multi-stage image tampering detection model one by one for feature processing, and determining a target tampered image corresponding to the original image. According to the method, a high-precision image tampering detection scheme is realized, a tampered area can be positioned more accurately, the marginal definition is remarkably improved, and the accuracy of image tampering detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multi-stage progressive image tampering detection and positioning method and device. Background Art

[0002] Image tampering is the falsification of the content in an image, so that the content displayed by the falsified image is different from that of the original image. Image tampering detection technology is used to identify whether an image has been tampered with and locate the specific falsified location in the falsified image. Image tampering technology can be divided into two categories: semantically related and semantically unrelated, based on the degree of its significant impact on the semantic information of the image. Among them, semantically related tampering technology includes three forms: copy and paste (i.e. copying part or all of the area in the image to other locations in the same image), move and delete (remove or remove one or more areas from the real image), and splice combination (copying part or all of the image to other images). These operations may significantly change the semantic content of the image. In contrast, semantically unrelated image tampering includes geometric transformations (such as scaling, rotation, translation, etc.), color space transformations (such as adjusting color, brightness and contrast, etc.), filtering operations (such as blurring, sharpening, etc.) and image compression, which usually have little impact on the semantics of the image, or even no impact at all. However, when these semantically unrelated tampering techniques are used in combination with semantically related tampering techniques, the traces left by the two may be intertwined, bringing additional challenges to the detection work, thereby affecting the final detection effect. In the prior art, deep learning-based solutions have proposed a variety of network models that can universally detect and locate different types of tampering. However, these network models currently have deficiencies in locating and / or detecting tampered areas, such as: fuzzy edge positioning of tampered areas, missed or false detection of tampered areas, and low accuracy in detecting tampered areas. Summary of the invention

[0003] The present invention provides a multi-stage progressive image tampering detection and positioning method and device to solve the technical problem in the prior art that deep learning has low precision and accuracy in positioning tampered areas and / or detecting tampered areas.

[0004] According to one aspect of the present invention, a multi-stage progressive image tampering detection and positioning method is provided, comprising:

[0005] Receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image;

[0006] The original image, the first segmented image and the second segmented image are input into a pre-trained multi-stage image tampering detection model one by one for feature processing to determine a target tampered image corresponding to the original image.

[0007] According to another aspect of the present invention, there is provided a multi-stage progressive image tampering detection and positioning device, comprising:

[0008] An image processing module is used to receive an original image to be detected, and perform image segmentation on the original image to obtain a first segmented image and a second segmented image;

[0009] The detection module is used to input the original image, the first segmented image and the second segmented image into a pre-trained multi-stage image tampering detection model one by one for feature processing, so as to determine a target tampered image corresponding to the original image.

[0010] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0011] at least one processor; and

[0012] A memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-stage progressive image tampering detection and positioning method described in any embodiment of the present invention.

[0014] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the multi-stage progressive image tampering detection and positioning method described in any embodiment of the present invention when executed.

[0015] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of a multi-stage progressive image tampering detection and location method is provided for an embodiment of the present invention;

[0019] Figure 2 A flowchart of a multi-stage progressive image tampering detection and positioning method provided by an embodiment of the present invention;

[0020] Figure 3 A flowchart of a multi-stage progressive image tampering detection and positioning method provided by an embodiment of the present invention;

[0021] Figure 4 A flowchart of a multi-stage progressive image tampering detection and positioning method provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of a multi-stage progressive image tampering detection and positioning device provided by an embodiment of the present invention;

[0023] Figure 6 A schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Figure 1 A flowchart of a multi-stage progressive image tampering detection and positioning method is provided for an embodiment of the present invention. This embodiment is applicable to the case of identifying and positioning tampering areas of forged images. The method can be performed by a multi-stage progressive image tampering detection and positioning device. The multi-stage progressive image tampering detection and positioning device can be implemented in the form of hardware and / or software. The multi-stage progressive image tampering detection and positioning device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0027] S110, receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image.

[0028] The original image may be an image object for image tampering detection. It should be noted that the original image may be a forged image with tampering or a normal image without tampering, and the present invention does not limit this.

[0029] The first segmented image may be two non-overlapping patches obtained by segmenting the original image. It should be noted that the first segmented image includes two non-overlapping patches, each of which is a part of the original image. Exemplarily, the original image is segmented vertically and evenly to obtain two patches of exactly the same size but non-overlapping, namely, the upper patch and the lower patch; each patch is twice the size of an original image.

[0030] The second segmented image may be four non-overlapping patches obtained by segmenting the original image. It should be noted that the second segmented image contains four complementary overlapping patches, each of which is also a part of the original image. Exemplarily, the original image is segmented vertically and horizontally evenly to obtain four patches of exactly the same size but non-overlapping, namely, the upper left patch, the upper right patch, the lower left patch, and the lower right patch; each patch is four times the size of the original image.

[0031] Specifically, in the present invention, an original image to be detected is received, the original image is first segmented, and a multi-patch hierarchy strategy is selected to segment the original image into non-overlapping patches to obtain a first segmented image and a second segmented image.

[0032] S120: Input the original image, the first segmented image, and the second segmented image into a pre-trained multi-stage image tampering detection model one by one for feature processing to determine a target tampered image corresponding to the original image.

[0033] The multi-stage image tampering detection model can be a neural network model that is pre-built and trained. Exemplarily, the multi-stage image tampering detection model can be built based on the model structure of MPRNet (Multi-Stage Progressive Image Restoration), by pre-building an initial model to be trained, obtaining multiple different forged images as image data sets for model training, and training the initial model through the image data sets to obtain a trained multi-stage image tampering detection model.

[0034] Optionally, in the present invention, a multi-stage image tampering detection model includes a first-stage network, a second-stage network and a third-stage network; the first-stage network is used to perform image detection on the second segmented image, the second-stage network is used to perform image detection on the first segmented image, the third-stage network performs detection on the original image, and the first-stage network is connected to the second-stage network, and the second-stage network is connected to the third-stage network. Through the connection of the networks at each stage, the accuracy of detecting and locating the tampered image area can be effectively improved.

[0035] Among them, the target tampered image can be an image composed of the image tampered area existing in the original image. Optionally, the target tampered image is set as a binary image, the image tampered area of ​​the original image can be set to white in the binary image, and the original image without image tampered area can be set to black in the binary image. Exemplary, for example, the original image shows football players on a football field, and the image tampered area exists in a basketball player among the football players. After the multi-stage image tampering detection model performs image detection on the original image, the output target tampered image is a binary image. In the binary image, the white area is the basketball player, and except for the basketball player, the other parts are black.

[0036] Specifically, the original image, the first segmented image, and the second segmented image are input into a pre-trained multi-stage image tampering detection model one by one for feature processing, and a target tampered image is output by the multi-stage image tampering detection model.

[0037] Optionally, in another optional embodiment of the present invention, the step of inputting the original image, the first segmented image, and the second segmented image into a pre-trained multi-stage image tampering detection model for feature processing one by one to determine a target tampered image corresponding to the original image includes:

[0038] Inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network respectively;

[0039] The first-stage network, the second-stage network and the third-stage network are used to perform feature processing on the original image, the first segmented image and the second segmented image to obtain a second tampered image corresponding to the first segmented image, a first tampered image corresponding to the second segmented image and a target tampered image corresponding to the original image.

[0040] The first-stage network can be a network structure for performing image detection on the second segmented image. The first-stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a first encoder-decoder subnetwork, and a supervised attention subnetwork.

[0041] The second-stage network may be a network structure for performing image detection on the first segmented image; the second-stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a second encoder-decoder subnetwork, and a supervisory attention subnetwork;

[0042] The third-stage network may be a network structure for performing image detection on the original image; the third-stage network includes two convolutional subnetworks, a spatial channel attention subnetwork, and an original resolution subnetwork; the original resolution subnetwork includes a first original resolution network block, a second original resolution network block, and a third original resolution network block;

[0043] Optionally, the convolutional subnetwork can be used to convolve the image to expand the image dimension;

[0044] Optionally, the spatial channel attention subnetwork can focus on features of specific spatial locations and different dimensions in the image, extract features of each dimension, and output a weighted feature map weighted by channel attention.

[0045] Optionally, the first encoder-decoder subnetwork can be constructed by a U-Net architecture, the first encoder-decoder subnetwork includes an encoder-decoder structure and channel attention introduced in the encoder-decoder structure, in the first encoder-decoder subnetwork, the encoder downsamples through bilinear interpolation and channel expansion convolution, the decoder upsamples through bilinear interpolation and channel reduction convolution, and the channel attention is used to connect the features across stages of the upsampling and downsampling stages for cross-stage fusion; the first encoder-decoder subnetwork can output feature maps of feature fusion.

[0046] Optionally, the second encoder-decoder subnetwork can be constructed by a U-Net architecture, and the first encoder-decoder subnetwork includes an encoder-decoder structure and channel attention introduced in the encoder-decoder structure. In the first encoder-decoder subnetwork, the encoder downsamples through bilinear interpolation and channel expansion convolution, and the decoder upsamples through bilinear interpolation and channel reduction convolution, and the channel attention is used to connect the features across stages of the upsampling and downsampling stages for cross-stage fusion; the second encoder-decoder subnetwork can output feature maps of feature fusion.

[0047] Optionally, the original resolution subnetwork can be ORsNet (Original Resolution Subnetwork); the original resolution subnetwork consists of a first original resolution network block, a second original resolution network block and a third original resolution network block; the first original resolution network block, the second original resolution network block and the third original resolution network block have the same network structure, which are all ORB (Oriented FAST and Rotated BRIEF) networks, and the ORB network can quickly detect and match the splicing points in the image; the original resolution subnetwork can analyze the relationship between objects in the image, infer the properties of the objects, and identify areas in the image that have no connection with the scene.

[0048] The supervised attention subnetwork can be used to focus on the tampered areas in the feature image and perform feature processing on the tampered areas; illustratively, the supervised attention subnetwork can be SAM (Supervised Attention Module).

[0049] Specifically, the second segmented image is input into the first-stage network of the multi-stage image tampering detection model. After the first-stage network detects the second segmented image, the first segmented image is input into the second-node network of the multi-stage image tampering detection model. After the second-stage network detects the first segmented image, the original image is input into the third-stage network of the multi-stage image tampering detection model for detection to obtain a target tampered image corresponding to the original image.

[0050] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0051] Figure 2 The flowchart of a multi-stage progressive image tampering detection and positioning method provided by the embodiment of the present invention is a specific process of the first stage network of the multi-stage image tampering detection model performing image detection on the second segmentation network. Figure 2 As shown, the method includes:

[0052] S210, receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image.

[0053] S220, respectively inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network.

[0054] S230. Perform convolution processing on the second segmented image through the convolution subnetwork to obtain a first convolution feature image.

[0055] Among them, the first convolution feature image can be a feature atlas obtained by convolution processing of the convolution subnetwork. The first convolution feature image includes 4 convolution feature maps. In the first stage network, the convolution subnetwork performs convolution processing on each patch of the second segmented image to obtain a convolution feature map corresponding to each patch. Exemplarily, the size of the convolution feature map corresponding to each patch is (batch_size, n_feat, H, W), where n_feat is the number of output channels of the convolution kernel of the convolution subnetwork, H and W are the spatial dimensions after convolution, H is the height, and W is the width. If an original image has 3 channels, a height of 512, and a width of 512, the patch image has 3 channels, a height of 256, and a width of 256, the convolution kernel is 3X3, and the convolution kernel of the convolution subnetwork is 96, then the output convolution feature map is (96, 256, 256).

[0056] Specifically, in the first stage network, the convolution subnetwork processes the four patches of the second segmented image in turn to obtain the convolution feature image of each patch, and obtains the first convolution feature image composed of four patches.

[0057] S240. Perform feature processing on the first convolution feature image through the spatial channel attention subnetwork to determine a first spatial channel attention map, and perform weighted multiplication of the first spatial channel attention map and the first convolution feature image to determine a first weighted feature image.

[0058] Among them, the first spatial channel attention map can be a weighted feature map corresponding to the first convolution feature image. It should be noted that the spatial channel attention subnetwork extracts the features of each channel through the attention mechanism. In the spatial channel attention subnetwork, the input image is subjected to convolution, parameterized rectified linear unit, convolution and global average pooling to obtain the probability map corresponding to the input image, and the probability map is subjected to convolution, rectified linear unit, convolution and sigmoid function again to obtain the first spatial channel attention map.

[0059] Optionally, the four convolution feature images of the first convolution feature image are feature processed by the spatial channel attention subnetwork to obtain a weight map corresponding to the convolution feature image, and the four weight maps are combined into a first spatial channel attention map, and the channel attention image and the first convolution feature image are weighted multiplied to determine the first weighted feature image. The first weighted feature image can be

[0060] S250. Downsample and upsample the first weighted feature image through the first encoder-decoder subnetwork to determine a first encoder-decoder output.

[0061] The first encoder-decoder output may be a feature image output after the encoder and decoder of the first encoder-decoder subnetwork encode, connect and decode the weighted feature image. The first encoder-decoder output includes three output images, namely, the first encoder output, the first decoder output and the first adjusted feature image.

[0062] Optionally, the encoder of the first encoder-decoder subnetwork is used to downsample the first weighted feature image. The specific downsampling process is as follows: in the encoder of the first encoder-decoder subnetwork, there is an input image and three outputs of different channel dimensions, and downsampling is performed through the bilinear interpolation and channel expansion convolution of the encoder, and three encoded images are output in sequence. Exemplarily, the convolution feature map corresponding to each patch is (96, 256, 256), where the channel is 96, and the length and width are 256 respectively; after passing through the spatial channel attention subnetwork, the feature map corresponding to the output of each patch is also (96, 256, 256), and after the two channel attention processing of the encoder, the first encoded image is (96, 256, 256); through a bilinear interpolation process, a channel expansion convolution process and two channel attention processes, the second encoded image is (144, 128, 128); again through a bilinear interpolation module, a channel expansion convolution process and two channel attention processes, the third encoded image is (192, 64, 64).

[0063] Optionally, after encoding each patch respectively, three coded images corresponding to each patch are obtained, the coded images of two adjacent patches are selected for corresponding splicing, and the first coded image, the second coded image and the third coded image of the two adjacent patches are respectively spliced ​​accordingly to obtain a spliced ​​image of two first coded images, a spliced ​​image of two second coded images, and a spliced ​​image of two third coded images, and the three spliced ​​images are used as the encoding results of the encoder of the first encoder-decoder subnetwork, and the three spliced ​​images are used as the output of the first encoder. Exemplarily, the three coded images corresponding to each tile are the first coded image (96, 256, 256), the second coded image (144, 128, 128) and the third coded image (192, 64, 64), and the adjacent tiles can select the upper left tile and the upper right tile and the lower left tile and the lower right tile, and the first coded images of the upper left tile and the upper right tile are spliced ​​to obtain a spliced ​​image of (96, 256, 512); the second coded images of the upper left tile and the upper right tile are spliced ​​to obtain a spliced ​​image of (144, 128, 256), and the third coded images of the upper left tile and the upper right tile are spliced ​​to obtain a spliced ​​image of (192, 64, 128). The coding image splicing process of the lower left patch and the lower right patch is the same as the coding image splicing process of the upper left patch and the upper right patch, and the spliced ​​image of the three coded images of the upper left patch and the upper right patch and the lower left patch and the lower right patch is output as the first encoder.

[0064] Optionally, the decoder of the first encoder-decoder subnetwork is used to upsample the first weighted feature image. Specifically, the upsampling process is performed by bilinear interpolation and channel reduction convolution, and cross-stage connection feature fusion is performed. The decoder of the first encoder-decoder subnetwork has inputs of three different channel dimensions and outputs three decoded images of different channel dimensions. The decoding process of the decoder of the first encoder-decoder sub-network is as follows: the input of the decoder of the first encoder-decoder sub-network in three different channel dimensions is the three spliced ​​images of two adjacent patches output by the decoder, the spliced ​​image of the third coded image of the two adjacent patches is processed by two channel attentions to obtain the first decoded image; the first decoded image is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image, and the first fused image is obtained by cross-stage connection feature addition and fusion with the feature image obtained by one channel attention processing of the spliced ​​image of the second coded image of the two adjacent patches, and the first fused image is subjected to two channel attention processing to obtain the second decoded image; the second coded image is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image, and the second fused image is obtained by one channel attention processing of the spliced ​​image of the first coded image of the two adjacent patches, and the second fused image is obtained by two channel attention processing. Then the decoder of the first encoder-decoder sub-network outputs three decoded images, namely the first decoded image, the second decoded image and the third decoded image. The three decoded images of every two adjacent tiles are used as the first decoder output; and the third decoded images of every two adjacent tiles are combined into a first adjusted feature image.

[0065] Exemplarily, the three decoder inputs of the decoder of the first encoder-decoder subnetwork are three encoded images of the upper left patch and the upper right patch and the lower left patch and the lower right patch. Taking the three spliced ​​images of the upper left patch and the upper right patch as an example, the specific decoding process is as follows: the three spliced ​​images are spliced ​​image (96, 256, 512), spliced ​​image (144, 128, 256) and spliced ​​image (192, 64, 128). In the decoding process, the spliced ​​image (192, 64, 128) is first subjected to two channel attention processes to obtain a first decoded image (192, 64, 128). The first decoded image is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image (144, 128, 256). The feature image obtained by the spliced ​​image (144, 128, 256) after one channel attention process is cross-stage connected feature addition fusion to obtain the first fused image (144, 12 8,256); the first fused image (144,128,256) is processed by two channel attentions to obtain a second decoded image (144,128,256); the second encoded image (144,128,256) is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image (96,256,512), which is fused with the feature image obtained by the spliced ​​image (96,256,512) after one channel attention process to obtain a second fused image (96,256,512); the second fused image (96,256,512) is processed by two channel attentions to obtain a third decoded image (96,256,512); the first decoded image (192,64,128), the second encoded image (144,128,256) and the third decoded image (96,256,512) are output as the first decoder.

[0066] S260: Perform feature processing on the first adjusted feature image and the second segmented image through the supervised attention subnetwork to determine the second weighted feature image and the first tampered image.

[0067] Among them, the second weighted feature image can be a weighted feature image obtained by the supervised attention subnetwork on the first adjusted feature image and the second segmented image through the attention mechanism; the first tampered image can be a detection result image obtained by the first stage network performing image detection on the second segmented image.

[0068] Optionally, the first-stage supervised attention subnetwork has two outputs, namely, a second weighted feature image and a first tampered image. The second weighted feature image can be used for feature processing of the second-stage network, and the first tampered image can be the image obtained by the first-stage network performing image tampering detection on the second segmented image.

[0069] Optionally, in another optional embodiment of the present invention, the supervisory attention subnetwork includes a first convolutional layer, a second convolutional layer, and a third convolutional layer; the performing feature processing on the first adjusted feature image and the second segmented image through the supervisory attention subnetwork to determine the second weighted feature image and the first tampered image includes:

[0070] Optionally, in the supervised attention subnetwork, the first adjusted feature image is connected to the first convolutional layer and the second convolutional layer respectively, the output of the second convolutional layer is added to the second segmented image and connected to the third convolutional layer, the third convolutional layer is connected to the Sigmoid function, the output of the Sigmoid function is multiplied by the output of the first convolutional layer, and then added to the first adjusted feature image to obtain the output of the supervised attention subnetwork.

[0071] Optionally, the first convolution layer is used to perform convolution processing on the first adjusted feature image to obtain channel convolution features, and the number of channels of the convolution kernel of the first convolution layer is the same as that of the first adjusted feature image; the second convolution layer is used to perform convolution processing on the first adjusted feature image to obtain a first channel image. Since the number of channels of the convolution kernel of the second convolution layer is the same as the number of channels of the second segmented image, the number of channels of the first channel image is the same as the number of channels of the second segmented image; the third convolution layer is used to perform convolution processing on the second weighted feature image to obtain a channel attention weight map. The channel attention weight map has weight coefficients ranging from 0 to 1. Since the number of channels of the convolution kernel of the third convolution layer is the same as the number of channels of the first adjusted feature image, the number of channels of the channel attention weight map is the same as that of the first adjusted feature image. Optionally, the process of the first stage of the supervised attention sub-network performing feature processing on the first adjusted feature image and the second segmented image is as follows: first, two different convolution processes are performed on the first adjusted feature image respectively, the first convolution layer is used to perform convolution processing on the first adjusted feature image to obtain channel convolution features, the second convolution layer is used to perform convolution processing on the first adjusted feature image to obtain a first channel image, the four slices corresponding to the first channel image and the second segmented image are added correspondingly to obtain a second weighted feature image, the third convolution layer is used to perform feature processing on the second weighted feature image to determine a channel attention weight map, the channel attention weight map and the channel convolution features are feature multiplied to obtain a second channel image, the second channel image and the first adjusted feature image are image added to determine a first tampered image. The first stage of the supervised attention sub-network outputs the second weighted feature image and the first tampered image respectively.

[0072] Optionally, another process in which the first-stage supervised attention subnetwork performs feature processing on the first adjusted feature image and the second segmented image is as follows: the first adjusted feature image is composed of the third decoded images of two adjacent patches, the four patches corresponding to the second segmented image are combined in the same way to obtain two patch combination images, the third decoded image of an adjacent patch is processed in turn with the patch combination image corresponding to the third decoded image, the third decoded image of an adjacent patch is convolved through the first convolution layer to obtain the channel convolution feature corresponding to the patch, the third decoded image of an adjacent patch is convolved through the second convolution layer to obtain the first channel image corresponding to the patch, the first channel image corresponding to the patch and the patch combination image are convolved. The combined images are added accordingly to obtain the weighted feature image corresponding to the slice, the weighted feature image corresponding to the slice is processed by the third convolutional layer, the channel attention weight map corresponding to the slice is determined, the channel attention weight map corresponding to the slice and the channel convolution feature are feature multiplied to obtain the second channel image corresponding to the slice, the second channel image corresponding to the slice and the third decoded image of an adjacent patch are added to determine the tampered image corresponding to the slice; the supervised attention sub-network then performs feature processing on the third decoded image of another adjacent patch, the weighted feature images corresponding to the two slices are image-connected to obtain the second weighted feature image; the tampered images corresponding to the two slices are image-connected to obtain the first tampered image.

[0073] S270. Determine the first fusion feature according to the first encoder output and the first decoder output.

[0074] Optionally, after obtaining the first encoder output and the first decoder output, a connection network is set between the first stage network and the second stage network, and the first encoder output and the first encoder output are convolved by a convolution network respectively, and the convolution results of the first encoder output and the first encoder output are added to obtain a first fusion feature. The first fusion feature can effectively improve the recognition and detection performance of the model through cross-stage feature fusion.

[0075] Optionally, since the first encoder outputs an encoded image including three channel dimensions and the first decoder outputs a decoded image including three channel dimensions, the encoded image and the decoded image of each channel dimension are convolved, and then feature splicing is performed to obtain a fused image of three channel dimensions, and the fused image of the three channel dimensions is used as a fused image of every two adjacent slices, and the two fused images of every two adjacent slices are combined into a first fused feature.

[0076] S280, connecting the first fusion feature, the first adjusted feature image, and the second weighted feature image to the second stage network, performing feature processing on the original image and the first segmented image through the second stage network and the third stage network, and determining the second tampered image and the target tampered image.

[0077] Specifically, the first fusion feature, the first adjusted feature image and the second weighted feature image are connected to the second-stage network, and the original image and the first segmented image are feature processed through the second-stage network and the third-stage network to obtain the second tampered image output by the second-stage network and the target tampered image output by the third-stage network.

[0078] Optionally, the network structure of the first stage network is as follows: a convolutional subnetwork is connected to the spatial channel attention subnetwork, which is connected to the first encoder-decoder subnetwork through the spatial channel attention subnetwork, and the second encoder-decoder subnetwork is connected to the supervisory attention subnetwork.

[0079] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0080] Figure 3 The flowchart of a multi-stage progressive image tampering detection and positioning method provided by the embodiment of the present invention is a specific process of the second stage network of the multi-stage image tampering detection model performing image detection on the first segmentation network. Figure 3 As shown, the method includes:

[0081] S310, receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image.

[0082] S320, respectively inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network.

[0083] S330. Perform feature processing on the second segmented image through the first stage network to obtain a first fused feature, a first adjusted feature image, a second weighted feature image and a first tampered image.

[0084] S340, performing convolution processing on the first segmented image through the convolution subnetwork to obtain a second convolution feature image.

[0085] Optionally, the convolution subnetwork of the second-stage network is exactly the same as the convolution subnetwork of the first-stage network, and the second convolution feature image can be a feature atlas obtained by convolution processing of the convolution subnetwork. The second convolution feature image includes 2 convolution feature maps. Exemplarily, in the second-stage network, the convolution subnetwork performs convolution processing on 2 patches of the first segmented image to obtain convolution feature maps corresponding to the 2 patches. Exemplarily, the original image is (3, 512, 512), and the images corresponding to the two patches are (3, 256, 512). The two patches are convoluted by the convolution subnetwork to obtain two convolution feature images (96, 256, 512).

[0086] Specifically, in the second stage network, the convolution subnetwork processes the two patches of the first segmented image in turn to obtain the convolution feature image of each patch, and obtains a second convolution feature image composed of two patches.

[0087] S350. Perform feature processing on the second convolution feature image through the spatial channel attention subnetwork to determine a second spatial channel attention map, and perform weighted multiplication of the second spatial channel attention map and the second convolution feature image to determine a third weighted feature image.

[0088] Among them, the second spatial channel attention map can be a weight feature map corresponding to the second convolution feature image.

[0089] Specifically, the two convolution feature images of the second convolution feature image are feature processed by the spatial channel attention sub-network to obtain a weight map corresponding to the convolution feature image, the two weight maps are combined into a second spatial channel attention map, the channel attention image and the second convolution feature image are weighted multiplied to determine a third weighted feature image. The third weighted feature image may be a feature image output by the spatial channel attention sub-network.

[0090] S360: Connect the third weighted feature image and the second weighted feature image to determine a fourth weighted feature image.

[0091] The fourth weighted feature image may be obtained by feature-connecting the third weighted feature image and the second weighted feature image.

[0092] Specifically, since the third weighted feature image and the second weighted feature image are feature images with the same channel dimension, the third weighted feature image and the second weighted feature image are fused according to the channel dimension to obtain a fused feature image, and the channel dimension of the fused feature image is twice that of the third weighted feature image. Through convolution processing, the feature dimension of the fused feature image is reduced by half to obtain a spliced ​​fourth weighted feature image.

[0093] S370. Downsample the third weighted feature image through the second encoder-decoder subnetwork, determine the second encoder output, and fuse the second encoder output with the first fusion feature to obtain a second fusion feature.

[0094] The second encoder output may be a feature image of the weighted feature image encoded by the encoder and decoder of the first encoder-decoder subnetwork. The second encoder output includes three encoded images for each tile image.

[0095] Optionally, the encoder of the second encoder-decoder subnetwork is used to downsample the second weighted feature image. The specific downsampling process for a slice image of the first segmented image is as follows: the encoder of the second encoder-decoder subnetwork has 1 input image and 3 outputs of different channel dimensions. Downsampling is performed through bilinear interpolation and channel expansion convolution of the encoder, and three encoded images are output in sequence. Exemplarily, the convolution feature map corresponding to each patch is (96, 256, 512), where the number of channels is 96, and the length and width are 256 and 512 respectively; after splicing the spatial channel attention subnetwork and the second weighted feature image, the feature map corresponding to the output of each patch is also (96, 256, 512), and after the two channel attention processing of the encoder, the first encoded image is (96, 256, 512); through a bilinear interpolation process, a channel expansion convolution process and two channel attention processes, the second encoded image is (144, 128, 256); and through a bilinear interpolation module, a channel expansion convolution process and two channel attention processes, the third encoded image is (192, 64, 128).

[0096] Optionally, since the encoder of the second encoder-decoder sub-network can obtain outputs of three different channel dimensions, namely the first encoded image, the second encoded image and the third encoded image, the first fusion feature also contains a fused image of the three channel dimensions of the slice, and each encoded image has a fused image of the channel dimension, the encoded image and the fused image of each channel dimension are added and spliced ​​to obtain a three-dimensional fused image, and the three-dimensional fused image is used as the fused image of each slice; after obtaining the three-dimensional fused image of each slice, the corresponding fused images of the same channel dimensions of the two slices are added and spliced ​​to obtain the three-dimensional fused image corresponding to the first segmented image, and the three-dimensional fused image corresponding to the first segmented image is used as the second fusion feature.

[0097] Specifically, the third weighted feature image is downsampled through the second encoder-decoder subnetwork, the second encoder output is determined, and the second encoder output and the first fusion feature are fused to obtain the second fusion feature.

[0098] S380. Upsample the second fused features through the second encoder-decoder sub-network to obtain a second decoder output and a second adjusted feature image.

[0099] Optionally, the decoder of the second encoder-decoder subnetwork is used to upsample the second fused features. Specifically, the upsampling process is performed through bilinear interpolation and channel reduction convolution, and cross-stage connection feature fusion is performed. The decoder of the second encoder-decoder subnetwork has inputs of three different channel dimensions and outputs decoded images of three different channel dimensions. The decoding process of the decoder of the second encoder-decoder sub-network is as follows: the input of the decoder of the first encoder-decoder sub-network with three different channel dimensions is the second fused feature output by the decoder, the fused image with the most channel dimensions in the second fused feature is processed by two channel attentions to obtain the first decoded image; the first decoded image is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image, the fused image with the second number of channels in the second fused feature is subjected to one channel attention to obtain a fused feature processed image, the feature image and the fused feature processed image are subjected to cross-stage connection feature addition fusion to obtain the first fused image, the first fused image is subjected to two channel attentions to obtain the second decoded image; the second encoded image is subjected to bilinear interpolation and channel reduction convolution to obtain a feature image, the last fused image in the second fused feature is subjected to one channel attention to obtain a fused feature processed image, the feature image and the fused feature processed image are subjected to cross-stage connection feature addition fusion to obtain the second fused image, the second fused image is subjected to two channel attentions to obtain the third decoded image. The third decrypted image is used as the second adjusted feature image, and the first decoded image, the second decoded image and the third decoded image are used as the output of the second decoder.

[0100] S390: Fusing the second adjusted feature image and the first adjusted feature image to determine a third fused feature.

[0101] The third fusion feature is a feature image obtained by fusion of the second adjusted feature image and the first adjusted feature image.

[0102] Specifically, since the channel dimensions of the second adjusted feature image and the first adjusted feature image are the same, the second adjusted feature image and the first adjusted feature image are feature spliced ​​and fused to determine the third fused feature.

[0103] S3100. Perform feature processing on the third fused feature and the first segmented image through the supervised attention subnetwork to determine the fifth weighted feature image and the second tampered image.

[0104] Among them, the fifth weighted feature image can be a weighted feature map obtained by the supervised attention subnetwork on the third fusion feature and the first segmented image through the attention mechanism; the second tampered image can be a detection result image obtained by the second stage network performing image detection on the first segmented image.

[0105] Optionally, the second-stage supervised attention subnetwork has two outputs, namely the fifth weighted feature image and the second tampered image. The fifth weighted feature image can be used for feature processing of the third-stage network, and the second tampered image can be the image obtained by the second-stage network performing image tampering detection on the first segmented image.

[0106] Optionally, the supervised attention subnetwork of the second-stage network also has a first convolutional layer, a second convolutional layer, and a third convolutional layer. The network structure of the supervised attention subnetwork of the second-stage network is exactly the same as the network structure of the supervised attention subnetwork of the first-stage network, and the manner of performing feature processing on the third fusion feature and the first segmented image is also the same, which is not described in detail in the present invention.

[0107] Specifically, the third fused feature and the first segmented image are feature processed through the supervised attention subnetwork to determine the fifth weighted feature image and the second tampered image.

[0108] S3110. Determine the fourth fusion feature according to the second fusion feature and the second decoder output.

[0109] The second fusion feature is obtained by performing feature processing on the second fusion feature and the second decoder output.

[0110] Optionally, after obtaining the second fused feature and the second decoder output, a connection network is set between the second stage network and the third stage network, and the second fused feature and the second decoder output are convolved by a convolution network respectively, and the convolution results of the second fused feature and the second decoder output are added to obtain a fourth fused feature. The fourth fused feature can effectively improve the recognition and detection performance of the model through cross-stage feature fusion.

[0111] Optionally, since the second fusion feature output includes a fused image of three channel dimensions, the second decoder output includes a decoded image of three channel dimensions, the fused image of each channel dimension is convolved with the decoded image, and then feature splicing is performed to obtain a fused image of three channel dimensions, and the fused image of three channel dimensions is used as the fused image of every two adjacent slices, and the two fused images of every two adjacent slices are combined into a fourth fusion feature.

[0112] Specifically, the fourth fusion feature is determined based on the second fusion feature and the second decoder output.

[0113] S3120. Connect the fourth fusion feature and the fifth weighted feature image to the third-stage network, perform feature processing on the original image through the third-stage network, and determine the target tampered image.

[0114] Specifically, the fourth fusion feature and the fifth weighted feature image are connected to the third stage network, and the original image is feature processed through the third stage network to determine the target tampered image.

[0115] Optionally, the network structure of the second stage network is as follows: a convolutional subnetwork is connected to the spatial channel attention subnetwork, which is connected to the second encoder-decoder subnetwork through the spatial channel attention subnetwork, and the second encoder-decoder subnetwork is connected to the supervisory attention subnetwork.

[0116] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0117] Figure 4 The flowchart of a multi-stage progressive image tampering detection and positioning method provided by the embodiment of the present invention is a specific process of the third stage network of the multi-stage image tampering detection model performing image detection on the original network. Figure 4 As shown, the method includes:

[0118] S410: Receive an original image to be detected, and perform image segmentation on the original image to obtain a first segmented image and a second segmented image.

[0119] S420, respectively inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network.

[0120] S430, performing feature processing on the second segmented image through the first stage network to obtain a first fusion feature, a first adjusted feature image, a second weighted feature image and a first tampered image.

[0121] S440. Perform feature processing on the first segmented image according to the first fused feature, the first adjusted feature image, and the second weighted feature image through the second stage network to obtain a fourth fused feature and a fifth weighted feature image.

[0122] S450, performing convolution processing on the original image through the convolution subnetwork to obtain a third convolution feature image.

[0123] Among them, the third convolution feature image is a feature image obtained by performing feature convolution processing on the original image by the convolution sub-network.

[0124] Optionally, the convolution subnetwork of the third stage network is exactly the same as the convolution subnetwork of the first stage network and the second stage network, and the second convolution feature image can be a feature map obtained by convolution subnetwork on the original image. Exemplarily, the original image is (3, 512, 512), and the convolution subnetwork performs convolution on the original image to obtain the third convolution feature image (3, 512, 512).

[0125] Specifically, in the third stage network, the convolution subnetwork performs convolution processing on the original image to obtain the third convolution feature image.

[0126] S460. Perform feature processing on the third convolution feature image through the spatial channel attention sub-network to determine a third spatial channel attention map, and perform weighted multiplication of the third spatial channel attention map and the third convolution feature image to determine a sixth weighted feature image.

[0127] Among them, the third spatial channel attention map can be a weight feature map corresponding to the third convolution feature image.

[0128] Specifically, the third convolution feature image is feature processed by the spatial channel attention sub-network to obtain a weight map corresponding to the third convolution feature image, i.e., the third spatial channel attention map, and the channel attention image and the third convolution feature image are weighted multiplied to determine the sixth weighted feature image. The sixth weighted feature image is a feature image output by the spatial channel attention sub-network.

[0129] S470. Determine the fourth convolution feature image by performing weighted fusion on the sixth weighted feature image and the fifth weighted feature image.

[0130] The fourth convolution feature image is a feature image obtained by weighted fusion of the sixth weighted feature image and the fifth weighted feature image.

[0131] Specifically, since the sixth weighted feature image and the fifth weighted feature image are feature images with the same channel dimension, the sixth weighted feature image and the fifth weighted feature image are spliced ​​according to the channel dimension to obtain a fused feature image, and the channel dimension of the fused feature image is twice that of the third weighted feature image. Through convolution processing, the feature dimension of the fused feature image is reduced by one quarter to obtain a spliced ​​fourth convolution feature image.

[0132] S480. Perform feature processing on the fourth convolution feature image and the fourth fusion feature through the original resolution sub-network to determine a fifth fusion feature.

[0133] Optionally, the fifth fused feature may be a fused feature image obtained by processing the fourth convolution feature image through the original resolution sub-network and gradually fusing the fourth fused features. In another optional embodiment of the present invention, the fourth convolution feature image and the fourth fused feature are subjected to feature processing through the original resolution sub-network to determine the fifth fused feature, including:

[0134] Inputting the fourth convolution feature image and the fourth fusion feature into the first original resolution network block for feature processing to determine the first original feature;

[0135] Inputting the first original feature and the fourth fused feature into the second original resolution network block for feature processing to determine the second original feature;

[0136] The second original feature and the fourth fused feature are input into the third original resolution network block for feature processing to determine the fifth fused feature.

[0137] Among them, the first original feature can be a feature processing result image of the fourth convolution feature image and the fourth fusion feature by the first original resolution network block.

[0138] Optionally, the first original resolution network block performs feature processing on the fourth convolution feature image: the fourth convolution feature image is directly input into the first original resolution network block for feature processing to obtain a feature-processed image; since the channel dimensions of the feature-processed image and the fused image with the lowest channel dimension in the fourth fusion feature are different, and the channel dimension of the feature-processed image is greater than the channel dimension of the fused image with the lowest channel dimension in the fourth fusion feature, the fused image with the lowest channel dimension in the fourth fusion feature is convolved to obtain a convolved fused image; the channel dimension of the convolved fused image is the same as the channel dimension of the feature-processed image; the two are feature-concatenated and fused to obtain the first original feature.

[0139] Among them, the second original feature can be a feature processing result image of the first original feature and the fourth fusion feature by the second original resolution network block.

[0140] Optionally, the second original resolution network block performs feature processing on the first original feature and the fourth fused feature: the first original feature is directly input into the second original resolution network block for feature processing to obtain a feature-processed image; since the channel dimensions of the feature-processed image and the fused image with the lowest channel dimension in the fourth fused feature are different, and the channel dimension of the feature-processed image is smaller than the channel dimension of the fused image with the second channel dimension in the fourth fused feature, convolution is performed on the fused image to obtain a convolved fused image; the channel dimension of the convolved fused image is the same as that of the feature-processed image; feature splicing is performed on the feature-processed image and the convolved fused image to obtain a spliced ​​feature image, thereby obtaining the second original feature.

[0141] Among them, the third original feature can be a feature processing result image of the second original feature and the fourth fusion feature by the third original resolution network block.

[0142] Optionally, the third original resolution network block performs feature processing on the first original feature and the fourth fused feature: the second original feature is directly input into the third original resolution network block for feature processing to obtain a feature-processed image; since the channel dimensions of the feature-processed image and the fused image with the lowest channel dimension in the fourth fused feature are different, and the channel dimension of the feature-processed image is smaller than the channel dimension of the fused image with the largest channel dimension in the fourth fused feature, convolution is performed on the fused image to obtain a convolved fused image; the channel dimension of the convolved fused image is the same as that of the feature-processed image; feature splicing is performed on the feature-processed image and the convolved fused image to obtain a spliced ​​feature image, thereby obtaining the fifth fused feature.

[0143] Specifically, the fourth convolution feature image and the fourth fused feature are input into the first original resolution network block for feature processing to determine the first original feature; the first original feature and the fourth fused feature are input into the second original resolution network block for feature processing to determine the second original feature; the second original feature and the fourth fused feature are input into the third original resolution network block for feature processing to determine the fifth fused feature.

[0144] S490. Perform convolution processing on the fifth fusion feature through the convolution sub-network to determine a fifth convolution feature image.

[0145] Among them, the fifth convolution feature image can be a feature image obtained by performing convolution processing on the fifth fusion feature by the convolution sub-network.

[0146] Specifically, the fifth fusion feature is convolved through a convolutional sub-network to determine the fifth convolution feature image.

[0147] S4100. Perform feature stitching on the fifth convolution feature image and the original image to obtain the target tampered image.

[0148] Specifically, the target tampered image is obtained by performing feature splicing on the fifth convolution feature image and the original image.

[0149] Optionally, in the present invention, by selecting the first adjustment feature image and the fifth fusion feature for tampering detection, a detection result value is obtained, and the detection result value reflects the accuracy of detecting the tampered area of ​​the original image. Exemplarily, the detection result value can be 0.99, which means that the recognition is accurate.

[0150] Optionally, the network structure of the third stage network is as follows: a convolutional sub-network is connected to the spatial channel attention sub-network, which is connected to the original resolution sub-network through the spatial channel attention sub-network, and the original resolution sub-network is connected to a convolutional sub-network.

[0151] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0152] Figure 5 The following is a schematic diagram of the structure of a multi-stage progressive image tampering detection and positioning device provided by an embodiment of the present invention. Figure 5 As shown, the device includes: an image processing module 510 and a detection module 520; wherein,

[0153] An image processing module 510 is used to receive an original image to be detected, and perform image segmentation on the original image to obtain a first segmented image and a second segmented image;

[0154] The detection module 520 is used to input the original image, the first segmented image and the second segmented image into a pre-trained multi-stage image tampering detection model one by one for feature processing to determine a target tampered image corresponding to the original image.

[0155] The technical solution of the embodiment of the present invention, by segmenting the original image to be detected into stages and performing step-by-step detection, deeply learns the extensive context information in the image through its wide receptive field, and establishes a connection between each two adjacent stages, so that the intermediate multi-scale context features extracted early can effectively enhance the subsequent intermediate features, further improving the accuracy of detection and positioning. The technical problem of low precision and accuracy of deep learning in locating tampered areas and / or detecting tampered areas in the prior art is solved. The present invention realizes a high-precision image tampering detection solution, which can more accurately locate tampered areas, significantly improve the edge clarity, and improve the accuracy of image tampering detection.

[0156] Optionally, the multi-stage image tampering detection model includes a first-stage network, a second-stage network and a third-stage network;

[0157] Optionally, the detection module 520 is specifically used for:

[0158] Inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network respectively;

[0159] The first-stage network, the second-stage network and the third-stage network are used to perform feature processing on the original image, the first segmented image and the second segmented image to obtain a second tampered image corresponding to the first segmented image, a first tampered image corresponding to the second segmented image and a target tampered image corresponding to the original image.

[0160] Optionally, the first stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a first encoder-decoder subnetwork and a supervised attention subnetwork;

[0161] Optionally, the detection module 520 is further configured to:

[0162] Performing convolution processing on the second segmented image through the convolution subnetwork to obtain a first convolution feature image;

[0163] Performing feature processing on the first convolution feature image through the spatial channel attention subnetwork to determine a first spatial channel attention map, and performing weighted multiplication of the first spatial channel attention map and the first convolution feature image to determine a first weighted feature image;

[0164] Downsampling and upsampling the first weighted feature image through the first encoder-decoder subnetwork to determine a first encoder-decoder output; wherein the first encoder-decoder output includes a first encoder output, a first decoder output, and a first adjusted feature image;

[0165] Performing feature processing on the first adjusted feature image and the second segmented image through the supervised attention subnetwork to determine the second weighted feature image and the first tampered image;

[0166] Determine the first fusion feature according to the first encoder output and the first decoder output;

[0167] The first fused feature, the first adjusted feature image and the second weighted feature image are connected to the second stage network, and feature processing is performed on the original image and the first segmented image through the second stage network and the third stage network to determine the second tampered image and the target tampered image.

[0168] Optionally, the second stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a second encoder-decoder subnetwork and a supervised attention subnetwork;

[0169] Optionally, the detection module 520 is further configured to:

[0170] Performing convolution processing on the first segmented image through the convolution subnetwork to obtain a second convolution feature image;

[0171] Performing feature processing on the second convolution feature image through the spatial channel attention subnetwork to determine a second spatial channel attention map, and performing weighted multiplication of the second spatial channel attention map and the second convolution feature image to determine a third weighted feature image;

[0172] Connecting the third weighted feature image and the second weighted feature image to determine a fourth weighted feature image;

[0173] Downsampling the fourth weighted feature image through the second encoder-decoder subnetwork, determining a second encoder output, and performing feature fusion on the second encoder output and the first fused feature to obtain a second fused feature;

[0174] Upsampling the second fused features through the second encoder-decoder subnetwork to obtain a second decoder output and a second adjusted feature image;

[0175] Performing feature fusion on the second adjusted feature image and the first adjusted feature image to determine a third fusion feature;

[0176] Performing feature processing on the third fused feature and the first segmented image through the supervised attention subnetwork to determine the fifth weighted feature image and the second tampered image;

[0177] Determine the fourth fused feature according to the second fused feature and the second decoder output;

[0178] The fourth fusion feature and the fifth weighted feature image are connected to the third-stage network, and feature processing is performed on the original image through the third-stage network to determine the target tampered image.

[0179] Optionally, the third stage network includes two convolutional subnetworks, a spatial channel attention subnetwork and an original resolution subnetwork; the original resolution subnetwork includes a first original resolution network block, a second original resolution network block and a third original resolution network block;

[0180] Optionally, the detection module 520 is further configured to:

[0181] Performing convolution processing on the original image through the convolution sub-network to obtain a third convolution feature image;

[0182] Performing feature processing on the third convolution feature image through the spatial channel attention subnetwork to determine a third spatial channel attention map, and performing weighted multiplication of the third spatial channel attention map and the third convolution feature image to determine a sixth weighted feature image;

[0183] Determine the fourth convolution feature image by weighted fusion of the sixth weighted feature image and the fifth weighted feature image;

[0184] Performing feature processing on the fourth convolution feature image and the fourth fusion feature through the original resolution sub-network to determine a fifth fusion feature;

[0185] Performing convolution processing on the fifth fusion feature through the convolution sub-network to determine a fifth convolution feature image;

[0186] The target tampered image is obtained by performing feature splicing on the fifth convolution feature image and the original image.

[0187] Optionally, the supervised attention subnetwork includes: a first convolutional layer, a second convolutional layer, and a third convolutional layer;

[0188] Optionally, the detection module 520 is further configured to:

[0189] Performing convolution processing on the first adjusted feature image through the first convolution layer to determine channel convolution features;

[0190] Performing image processing on the first adjusted feature image through the second convolution layer to determine a first channel image;

[0191] Adding the first channel image and the second segmented image to obtain the second weighted feature image;

[0192] The second weighted feature image is subjected to feature processing through the third convolutional layer to determine a channel attention weight map.

[0193] Determine a second channel image by performing feature multiplication of the channel attention weight map and the channel convolution feature;

[0194] The second channel image and the first adjustment feature image are added to determine the first tampered image.

[0195] Optionally, the detection module 520 is further configured to:

[0196] Inputting the fourth convolution feature image and the fourth fusion feature into the first original resolution network block for feature processing to determine the first original feature;

[0197] Inputting the first original feature and the fourth fused feature into the second original resolution network block for feature processing to determine the second original feature;

[0198] The second original feature and the fourth fused feature are input into the third original resolution network block for feature processing to determine the fifth fused feature.

[0199] The multi-stage progressive image tampering detection and positioning device provided in the embodiment of the present invention can execute the multi-stage progressive image tampering detection and positioning method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0200] Figure 6 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0201] like Figure 6As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0202] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer grid such as the Internet and / or various telecommunication grids.

[0203] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a multi-stage progressive image tampering detection and positioning method.

[0204] In some embodiments, the multi-stage progressive image tampering detection and positioning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the multi-stage progressive image tampering detection and positioning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the multi-stage progressive image tampering detection and positioning method in any other appropriate manner (for example, by means of firmware).

[0205] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0206] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0207] In the context of the present invention, a computer readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium may be a machine readable signal medium. More specific examples of machine readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0209] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), a blockchain grid, and the Internet.

[0210] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0211] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0212] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the multi-stage progressive image tampering detection and positioning method provided in any embodiment of the present invention are implemented. The method includes:

[0213] Receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image;

[0214] The original image, the first segmented image and the second segmented image are input into a pre-trained multi-stage image tampering detection model one by one for feature processing to determine a target tampered image corresponding to the original image.

[0215] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0216] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0217] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0218] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0219] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0220] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0221] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-stage progressive image tampering detection and location method, characterized in that ,include: Receiving an original image to be detected, and performing image segmentation on the original image to obtain a first segmented image and a second segmented image; The original image, the first segmented image and the second segmented image are input into a pre-trained multi-stage image tampering detection model one by one for feature processing to determine a target tampered image corresponding to the original image.

2. The method according to claim 1, characterized in that ,The multi-stage image tampering detection model includes a first stage network, a second stage network and a third stage network; The step of inputting the original image, the first segmented image, and the second segmented image into a pre-trained multi-stage image tampering detection model for feature processing one by one to determine a target tampered image corresponding to the original image comprises: Inputting the second segmented image into the first stage network, inputting the first segmented image into the second stage network, and inputting the original image into the third stage network respectively; The first-stage network, the second-stage network and the third-stage network are used to perform feature processing on the original image, the first segmented image and the second segmented image to obtain a second tampered image corresponding to the first segmented image, a first tampered image corresponding to the second segmented image and a target tampered image corresponding to the original image.

3. The method according to claim 2, characterized in that The first stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a first encoder-decoder subnetwork and a supervisory attention subnetwork; the feature processing of the original image, the first segmented image and the second segmented image by the first stage network, the second stage network and the third stage network includes: Performing convolution processing on the second segmented image through the convolution subnetwork to obtain a first convolution feature image; Performing feature processing on the first convolution feature image through the spatial channel attention subnetwork to determine a first spatial channel attention map, and performing weighted multiplication of the first spatial channel attention map and the first convolution feature image to determine a first weighted feature image; Downsampling and upsampling the first weighted feature image through the first encoder-decoder subnetwork to determine a first encoder-decoder output; wherein the first encoder-decoder output includes a first encoder output, a first decoder output, and a first adjusted feature image; Performing feature processing on the first adjusted feature image and the second segmented image through the supervised attention subnetwork to determine a second weighted feature image and the first tampered image; Determine a first fusion feature according to the first encoder output and the first decoder output; The first fused feature, the first adjusted feature image and the second weighted feature image are connected to the second stage network, and feature processing is performed on the original image and the first segmented image through the second stage network and the third stage network to determine the second tampered image and the target tampered image.

4. The method according to claim 3, characterized in that ,The second stage network includes a convolutional subnetwork, a spatial channel attention subnetwork, a second encoder-decoder subnetwork and a supervised attention subnetwork; The method of connecting the first fusion feature, the first adjustment feature image, and the second weighted feature image to the second stage network, performing feature processing on the original image and the first segmented image through the second stage network and the third stage network, and determining the second tampered image and the target tampered image comprises: Performing convolution processing on the first segmented image through the convolution subnetwork to obtain a second convolution feature image; Performing feature processing on the second convolution feature image through the spatial channel attention subnetwork to determine a second spatial channel attention map, and performing weighted multiplication of the second spatial channel attention map and the second convolution feature image to determine a third weighted feature image; Connecting the third weighted feature image and the second weighted feature image to determine a fourth weighted feature image; Downsampling the fourth weighted feature image through the second encoder-decoder subnetwork, determining a second encoder output, and performing feature fusion on the second encoder output and the first fused feature to obtain a second fused feature; Upsampling the second fused features through the second encoder-decoder subnetwork to obtain a second decoder output and a second adjusted feature image; Performing feature fusion on the second adjusted feature image and the first adjusted feature image to determine a third fusion feature; Performing feature processing on the third fused feature and the first segmented image through the supervised attention subnetwork to determine a fifth weighted feature image and the second tampered image; Determine a fourth fused feature according to the second fused feature and the second decoder output; The fourth fusion feature and the fifth weighted feature image are connected to the third-stage network, and feature processing is performed on the original image through the third-stage network to determine the target tampered image.

5. The method according to claim 4, characterized in that ,The third stage network includes two convolutional sub-networks, a spatial channel attention sub-network and an original resolution sub-network; the original resolution sub-network includes a first original resolution network block, a second original resolution network block and a third original resolution network block; The connecting the fourth fusion feature and the fifth weighted feature image to the third stage network, performing feature processing on the original image through the third stage network, and determining the target tampered image includes: Performing convolution processing on the original image through the convolution sub-network to obtain a third convolution feature image; Performing feature processing on the third convolution feature image through the spatial channel attention subnetwork to determine a third spatial channel attention map, and performing weighted multiplication of the third spatial channel attention map and the third convolution feature image to determine a sixth weighted feature image; Determine a fourth convolution feature image by weighted fusion of the sixth weighted feature image and the fifth weighted feature image; Performing feature processing on the fourth convolution feature image and the fourth fusion feature through the original resolution sub-network to determine a fifth fusion feature; Performing convolution processing on the fifth fusion feature through the convolution sub-network to determine a fifth convolution feature image; The target tampered image is obtained by performing feature splicing on the fifth convolution feature image and the original image.

6. The method according to claim 5, characterized in that , the supervisory attention subnetwork includes: a first convolutional layer, a second convolutional layer and a third convolutional layer; the feature processing of the first adjusted feature image and the second segmented image by the supervisory attention subnetwork to determine the second weighted feature image and the first tampered image includes: Performing convolution processing on the first adjusted feature image through the first convolution layer to determine channel convolution features; Performing image processing on the first adjusted feature image through the second convolution layer to determine a first channel image; Adding the first channel image and the second segmented image to obtain the second weighted feature image; Performing feature processing on the second weighted feature image through the third convolutional layer to determine a channel attention weight map; Determine a second channel image by performing feature multiplication of the channel attention weight map and the channel convolution feature; The second channel image and the first adjustment feature image are added to determine the first tampered image.

7. The method according to claim 6, characterized in that , the fourth convolution feature image and the fourth fusion feature are subjected to feature processing by the original resolution sub-network to determine the fifth fusion feature, including: Inputting the fourth convolution feature image and the fourth fusion feature into the first original resolution network block for feature processing to determine the first original feature; Inputting the first original feature and the fourth fused feature into the second original resolution network block for feature processing to determine the second original feature; The second original feature and the fourth fused feature are input into the third original resolution network block for feature processing to determine the fifth fused feature.

8. A multi-stage progressive image tampering detection and positioning device, characterized in that ,include: An image processing module is used to receive an original image to be detected, and perform image segmentation on the original image to obtain a first segmented image and a second segmented image; The detection module is used to input the original image, the first segmented image and the second segmented image into a pre-trained multi-stage image tampering detection model one by one for feature processing, so as to determine a target tampered image corresponding to the original image.

9. An electronic device, characterized in that , the electronic device comprises: at least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-stage progressive image tampering detection and positioning method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the multi-stage progressive image tampering detection and positioning method described in any one of claims 1-7 when executed.

Images