Image tampering detection method and device, equipment, storage medium and computer program product

Abstract

The invention discloses an image tampering detection method and device, equipment, a storage medium and a computer program product, and the method comprises the steps: calling a first network to process a first image, and obtaining an edge feature map; the edge feature map is used for describing the boundary of an abnormal region in the first image; calling a second network to process the first image to obtain a regional feature map; the region feature map is used for describing the content of an abnormal region in the first image; performing fusion processing on the edge feature map and the region feature map to obtain a fused feature map; calling a third network to process the fused feature map to obtain a first output; the first output is used for describing a tampered area in the first image.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image tampering detection method, device, equipment, storage medium and computer program product. Background Art

[0002] In the related art, a single model is called to perform tampering detection on an image to identify the tampered area in the image. However, the image to be tested often involves more than one tampering operation, and the uncertainty of the image type and the area of ​​the tampered area is high, resulting in low accuracy of tampering detection. Summary of the invention

[0003] To solve the related technical problems, the embodiments of the present application provide an image tampering detection method, apparatus, device, storage medium and computer program product.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present invention provides an image tampering detection method, which includes:

[0006] Calling the first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of the abnormal area in the first image;

[0007] Calling a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal region in the first image;

[0008] Fusing the edge feature map with the regional feature map to obtain a fused feature map;

[0009] Calling a third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

[0010] In the above solution, the first network includes a backbone network and a first long short-term memory (LSTM) network;

[0011] The backbone network is used to detect spatial domain features of the boundaries of abnormal regions in the image;

[0012] The first LSTM network is used to detect frequency domain difference characteristics between abnormal areas and normal areas in the image;

[0013] Correspondingly, calling the first network to process the first image to obtain an edge feature map includes:

[0014] Calling the backbone network and the first LSTM network in parallel to process the first image, and obtaining a processing result output by the backbone network and a processing result output by the first LSTM network;

[0015] The processing result of the backbone network output is integrated with the processing result of the first LSTM network output to obtain the edge feature map.

[0016] In the above solution, the backbone network includes a plurality of edge detection blocks and a plurality of first residual blocks;

[0017] The edge detection block is used to detect the spatial domain features of the boundaries of abnormal areas in the image;

[0018] The multiple first residual blocks are used to perform residual connection on the multiple edge detection blocks;

[0019] Correspondingly, when calling the backbone network to process the first image, based on the first image, the multiple edge detection blocks and the multiple first residual blocks in the backbone network are called in a set order.

[0020] In the above scheme, in the process of calling the multiple edge detection blocks and the multiple first residual blocks in the backbone network according to a set order, the method includes:

[0021] Calling the nth edge detection block to process the first input to obtain a second output; wherein n is a positive integer;

[0022] Calling the nth first residual block to process the second output to obtain a third output;

[0023] The first input and the third output are summed, or the second output and the third output are summed to obtain a processing result; the processing result is used to: input the n+1th edge detection block, or as the processing result output by the backbone network.

[0024] In the above scheme, the second network includes a spatial pyramid pooling module, a plurality of second residual blocks and a plurality of first attention blocks; the size of each of the plurality of second residual blocks decreases in sequence based on a set ratio;

[0025] The second residual block is used to extract content features of abnormal areas in the image;

[0026] The spatial pyramid pooling module is used to convert the input feature map into a feature map of a set size;

[0027] The first attention block is used to suppress information irrelevant to abnormal areas in the image;

[0028] Correspondingly, calling the second network to process the first image to obtain a regional feature map includes:

[0029] calling each second residual block and each first attention block in the second network in sequence to process the first image to obtain a fourth output;

[0030] The spatial pyramid pooling module is called to process the fourth output to obtain the regional feature map.

[0031] In the above scheme, the third network includes a second attention block and an abnormal perception block;

[0032] The second attention block is used to enhance the weight of the channel related to the tampering operation in the input feature map;

[0033] The anomaly sensing block is used to detect tampered areas in the image;

[0034] Correspondingly, calling the third network to process the fused feature map to obtain a first output includes:

[0035] Calling the second attention block to process the fused feature map to obtain an enhanced fused feature map;

[0036] The anomaly perception block is called to process a third input to obtain the first output; the third input represents a feature map obtained based on the enhanced fusion feature map.

[0037] In the above scheme, the anomaly perception block includes a second LSTM network and a plurality of Z-score pools, each of the plurality of Z-score pools corresponding to a pooling window of a set size;

[0038] Correspondingly, calling the abnormality perception block to process the third input to obtain the first output includes:

[0039] Calling the pooling window corresponding to each Z-score pool, performing average pooling processing on the third input, and obtaining a pooling result corresponding to each Z-score pool;

[0040] Based on the pooling result corresponding to each Z-score pool, Z-score calculation is performed to obtain the Z-score vector corresponding to each Z-score pool;

[0041] The second LSTM network is called to process the splicing feature to obtain the first output; the splicing feature is obtained by splicing the Z score vectors corresponding to the multiple Z score pools.

[0042] In the above scheme, the method further includes:

[0043] The fourth network, the fifth network and the sixth network are trained iteratively for multiple times based on multiple training samples until the set convergence condition is reached; the training samples represent the tampered images; the fourth network represents the first network before the training is completed, the fifth network represents the second network before the training is completed, and the sixth network represents the third network before the training is completed; wherein,

[0044] The performing one training iteration among multiple training iterations on the fourth network, the fifth network, and the sixth network based on the multiple training samples comprises:

[0045] Based on any of the above methods, calling the fourth network, the fifth network and the sixth network to process each of the multiple training samples to obtain a fifth output corresponding to each training sample; the fifth output is used to describe the tampered area in the corresponding training sample;

[0046] Based on the first loss value, updating the fourth network, the fifth network and the sixth network;

[0047] Wherein, the first loss value represents a combination of the second loss value and the third loss value;

[0048] The second loss value is obtained by calculating the fifth output corresponding to each training sample based on the binary cross entropy loss function;

[0049] The third loss value is obtained based on the calculation of the sixth output corresponding to each training sample by the binary cross entropy loss function; the sixth output is obtained based on the processing of the corresponding training sample by the second residual block and the first attention block in the fifth network.

[0050] The present application also provides an image tampering detection device, including:

[0051] A first calling unit is used to call a first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of an abnormal area in the first image;

[0052] A second calling unit is used to call a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal area in the first image;

[0053] A fusion unit, used for fusing the edge feature map with the regional feature map to obtain a fused feature map;

[0054] The third calling unit is used to call the third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

[0055] The embodiment of the present application also provides an electronic device, including a first processor and a first communication interface; wherein,

[0056] The first processor is used to call the first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of the abnormal area in the first image;

[0057] Calling a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal region in the first image;

[0058] fusing the edge feature map with the region feature map to obtain a fused feature map; and,

[0059] Calling a third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

[0060] An embodiment of the present application further provides an electronic device, comprising: a first processor and a first memory for storing a computer program that can be run on the processor,

[0061] Wherein, the first processor is used to execute the steps of any of the above methods when running the computer program.

[0062] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0063] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of any of the above methods when executed by a processor.

[0064] In the embodiment of the present application, the first network is called to process the first image to obtain an edge feature map, wherein the edge feature map is used to describe the boundary of the abnormal area in the first image; then, the second network is called to process the first image to obtain a regional feature map, wherein the regional feature map is used to describe the content of the abnormal area in the first image; thereafter, the edge feature map and the regional feature map are fused to obtain a fused feature map; the third network is called to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image. In the above scheme, the first network and the second network are first called to detect the abnormal area in the first image respectively, and then the edge feature map output by the first network and the regional feature map output by the second network are fused, which is equivalent to multi-branch detection based on the first network and the second network. Compared with the related technology, multiple network models are combined and called, which improves the accuracy of abnormal area detection. On this basis, the accuracy of tampering detection can be improved; and in the above scheme, the third network is also called to process the fused feature map to obtain the first output, which is equivalent to detecting the tampered area based on the abnormal area in the first image, narrowing the detection range, thereby further improving the accuracy of tampering detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of an implementation flow of an image tampering detection method provided in an embodiment of the present application;

[0066] Figure 2 A schematic diagram of the structure of an edge detection block provided in an embodiment of the present application;

[0067] Figure 3 A schematic diagram of the structure of a backbone network provided in an embodiment of the present application;

[0068] Figure 4 A schematic diagram of the structure of an abnormality sensing block provided in an embodiment of the present application;

[0069] Figure 5 A schematic diagram of the structure of an image tampering detection network provided for an application embodiment of the present application;

[0070] Figure 6 A flowchart of an image tampering detection method provided in an application embodiment of the present application;

[0071] Figure 7 A schematic diagram of the structure of an image tampering detection device provided in an embodiment of the present application;

[0072] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] With the development of image processing technology, digital images are facing the risk of being tampered with, and the traces of tampering are increasingly difficult to detect, which seriously affects the authenticity and originality of the image. In related technologies, a single model is used to detect tampering of images to identify the tampered areas in the image. However, the image to be tested often involves more than one tampering operation, and the uncertainty of the image type and the area of ​​the tampered area is high, resulting in low accuracy of tampering detection.

[0074] Based on this, in the embodiment of the present application, the first network is called to process the first image to obtain an edge feature map, wherein the edge feature map is used to describe the boundary of the abnormal area in the first image; then, the second network is called to process the first image to obtain a regional feature map, wherein the regional feature map is used to describe the content of the abnormal area in the first image; thereafter, the edge feature map and the regional feature map are fused to obtain a fused feature map; the third network is called to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image. In the above scheme, the first network and the second network are first called to detect the abnormal area in the first image respectively, and then the edge feature map output by the first network and the regional feature map output by the second network are fused, which is equivalent to multi-branch detection based on the first network and the second network. Compared with the related technology, multiple network models are combined and called, which improves the accuracy of abnormal area detection. On this basis, the accuracy of tampering detection can be improved; and in the above scheme, the third network is also called to process the fused feature map to obtain the first output, which is equivalent to detecting the tampered area based on the abnormal area in the first image, narrowing the detection range, thereby further improving the accuracy of tampering detection.

[0075] The present application is further described in detail below with reference to the accompanying drawings and embodiments.

[0076] The present application provides an image tampering detection method. Figure 1 , the method comprising:

[0077] Step 101: Call the first network to process the first image to obtain an edge feature map.

[0078] The edge feature map is used to describe the boundary of the abnormal area in the first image.

[0079] In practical applications, the first image can be understood as an electronic image to be tampered with, and tampering detection is to detect the tampered area in the image. For example, the first image can be a qualification certificate image, a chat record image, etc.

[0080] In practical applications, the first network can be used to identify the boundaries of abnormal areas in the image, that is, to perform edge detection. The abnormal area in the image can be understood as: an area that is unnaturally different from the surrounding content in the image to which it belongs. This difference does not belong to the original content in the image and usually indicates that there is an abnormality in the image. For example, the abnormality may include: the image has been tampered with.

[0081] In practical applications, the content of a region in an image may include the pixel content of the region, for example, pixel value and pixel size.

[0082] Step 102: Call the second network to process the first image to obtain a regional feature map.

[0083] The region feature map is used to describe the content of the abnormal region in the first image.

[0084] In practical applications, there may be multiple tampered areas of different sizes in the tampered image, and the second network can be used to capture the contents of these tampered areas, that is, to perform area detection.

[0085] It should be noted that step 101 and step 102 may be executed sequentially or simultaneously. When executed sequentially, there is no restriction on the order of execution.

[0086] Step 103: Fusing the edge feature map and the regional feature map to obtain a fused feature map.

[0087] In practical applications, the edge feature map and the regional feature map can be connected into a fused feature map based on fusion processing. The fused feature map contains the feature information of the edge feature map and the regional feature map. Compared with a single edge feature map or a single regional feature map, the fused feature map can more accurately describe the abnormal area in the image, thereby improving the accuracy of abnormal area detection.

[0088] For example, when there is a small amount of missing in the boundary of the abnormal area described by the edge feature map, the fused feature map can clarify the entire boundary of the abnormal area based on the description of the content of the corresponding abnormal area from the regional feature map, thereby improving the accuracy of abnormal area detection.

[0089] In practical applications, the fusion processing of the edge feature map and the region feature map may at least include: processing the edge feature map and the region feature map based on a connect function, or performing a dot product calculation on the edge feature map and the region feature map.

[0090] In practical applications, the first network and the second network can form an abnormal area detection network, wherein the first network can be regarded as a branch in the abnormal area detection network, and the second network can be regarded as another branch in the abnormal area detection network. Here, the edge feature map output by the first network and the region feature map output by the second network are fused, which can be regarded as a multi-branch detection based on the first network and the second network in the abnormal area detection network.

[0091] In practical applications, the network can be expressed as a model or a network model. Multi-branch detection based on the first network and the second network can also be understood as a combined call to the first network and the second network. Therefore, the embodiment of the present application can also be regarded as a combined call to multiple network models. Compared with related technologies, the detection results of multiple network models are integrated, thereby improving the accuracy of abnormal area detection.

[0092] In practical applications, the fused feature map can be used to detect tampered areas in the processed image. Therefore, on the basis of improving the accuracy of abnormal area detection, the embodiment of the present application also improves the accuracy of tampering detection.

[0093] Step 104: Call the third network to process the fused feature map to obtain a first output.

[0094] The first output is used to describe the tampered area in the first image.

[0095] In practical applications, the third network can further detect the tampered area in the abnormal area based on the abnormal area described in the fused feature map, thereby narrowing the detection range and further improving the accuracy of tampering detection.

[0096] In practical applications, the first output may be a mask output, and the mask output may be an image, each pixel in the image corresponds to a pixel in the first image. By superimposing the mask output with the first image, the tampered area in the first image can be determined.

[0097] Exemplarily, the first output may be a binary image, in which each pixel corresponds to a pixel at the same position in the first image, and the value of each pixel includes: 0 or 1, wherein the value 0 indicates that the pixel corresponding to the pixel in the first image has not been tampered with, and the value 1 indicates that the pixel corresponding to the pixel in the first image has been tampered with.

[0098] In the embodiment of the present application, the first network and the second network are first called separately to detect the abnormal area in the first image, and then the edge feature map output by the first network and the regional feature map output by the second network are fused, which is equivalent to multi-branch detection based on the first network and the second network. Compared with the related technology, multiple network models are called in combination to improve the accuracy of abnormal area detection. On this basis, the accuracy of tampering detection can be improved; and, the embodiment of the present application also calls the third network to process the fused feature map to obtain the first output, which is equivalent to detecting the tampered area based on the abnormal area in the first image, narrowing the detection range, thereby further improving the accuracy of tampering detection.

[0099] The first network, the second network and the third network are further described below.

[0100] In one embodiment, the first network includes a backbone network and a first LSTM network;

[0101] The backbone network is used to detect the spatial domain features of the boundaries of abnormal regions in the image;

[0102] The first LSTM network is used to detect the frequency domain difference characteristics between abnormal areas and normal areas in the image;

[0103] Correspondingly, the first network is called to process the first image to obtain an edge feature map, including:

[0104] The backbone network and the first LSTM network are called in parallel to process the first image, and a processing result output by the backbone network and a processing result output by the first LSTM network are obtained;

[0105] The processing result of the backbone network output is integrated with the processing result of the first LSTM network output to obtain an edge feature map.

[0106] In practical applications, the backbone network may include multiple edge detection blocks, each edge detection block is used to detect the spatial domain features of the boundary of an abnormal area in an image, and the connection method between these edge detection blocks can be a residual connection.

[0107] In practical applications, in the process of calling the first LSTM network to process the first image, the first image can be divided into blocks first, and then the first LSTM network can be called to process the divided first image. The first LSTM network can learn the correlation between the divided images, thereby assisting in identifying the frequency domain features of the boundaries of the abnormal regions, and the first LSTM network can also learn the spatial difference information of the boundaries of the abnormal regions based on the log-likelihood distance. In this way, after calling the first LSTM network to process the first image, the frequency domain difference features between the abnormal regions and the normal regions in the first image can be detected, thereby improving the recognition accuracy of the inter-class difference features, and then accurately identifying the frequency domain features of the boundaries of the abnormal regions.

[0108] In practical applications, the processing results of the backbone network output and the processing results of the first LSTM network output can be fused based on the connect function or dot multiplication calculation to obtain an edge feature map. The edge feature map combines the spatial domain features and frequency domain features of the boundary of the abnormal area in the first image, and can accurately describe the boundary of the abnormal area in the first image.

[0109] In one embodiment, the backbone network includes a plurality of edge detection blocks and a plurality of first residual blocks;

[0110] The edge detection block is used to detect the spatial domain features of the boundaries of abnormal regions in the image;

[0111] The plurality of first residual blocks are used to perform residual connection on the plurality of edge detection blocks;

[0112] Correspondingly, when the backbone network is called to process the first image, based on the first image, multiple edge detection blocks and multiple first residual blocks in the backbone network are called in a set order.

[0113] In practical applications, the edge detection block may include an edge enhancement layer, which may be used to enhance the relevance of the boundary, so that the edge detection block can more accurately identify the boundary of the abnormal area. Figure 2 A schematic diagram of the structure of an edge detection block is given.

[0114] In practical applications, the edge detection block and the first residual block can be understood as processing modules in the backbone network. The edge detection block and the first residual block can be arranged alternately. In the process of calling the backbone network, each processing module in the backbone network can be called in sequence according to the arrangement order, that is, each edge detection block and each first residual block are called in sequence.

[0115] Exemplarily, the backbone network may include two edge detection blocks, namely edge detection block 1 and edge detection block 2, and two first residual blocks, namely first residual block 1 and first residual block 2. In the process of calling the backbone network, the calling order of these processing modules can be: edge detection block 1, first residual block 1, edge detection block 2, first residual block 2.

[0116] In practical applications, the backbone network may further include a third residual block, which may have the same structure as the first residual block and may also be understood as a processing module in the backbone network. In the process of calling the backbone network to process the first image, the third residual block may be first called to process the first image to obtain a processing result, and then based on the processing result output by the third residual block, multiple edge detection blocks and multiple first residual blocks in the backbone network may be called in a set order.

[0117] In one embodiment, in the process of calling multiple edge detection blocks and multiple first residual blocks in the backbone network in a set order, the image tampering detection method provided by the embodiment of the present application includes:

[0118] Call the nth edge detection block to process the first input and obtain the second output; n is a positive integer;

[0119] Call the nth first residual block to process the second output to obtain a third output;

[0120] The first input and the third output are summed, or the second output and the third output are summed to obtain a processing result; the processing result is used to: input the n+1th edge detection block, or as the processing result output by the backbone network.

[0121] In practical applications, the first input can be obtained based on processing of the first image by a processing module before the nth edge detection block in the backbone network, and the first input can also be the first image.

[0122] In practical applications, the nth edge detection block can be any edge detection block in the backbone network. Here, for ease of understanding, an edge detection block is described as the nth edge detection block. The nth first residual block can be a subsequent processing module of the nth edge detection block.

[0123] In practical applications, the residual connection in the backbone network can include two modes:

[0124] Mode 1: The sum of the input of the nth edge detection block and the output of the nth first residual block is input to the n+1th edge detection block, or is used as the output of the backbone network.

[0125] Mode 2: The sum of the output of the nth edge detection block and the output of the nth first residual block is input into the n+1th edge detection block, or is output as the backbone network.

[0126] In practical applications, the input of the nth edge detection block is equivalent to the first input, the output of the nth edge detection block is equivalent to the second output, and the output of the nth first residual block is equivalent to the third output.

[0127] It should be noted that the backbone network may include both residual connections in the above two modes, and the nth edge detection block in mode 1 and mode 2 may not be the same edge detection block. Figure 3 A schematic diagram of a backbone network is given, which includes two modes of residual connections.

[0128] In practical applications, if the connection between edge detection blocks in the backbone network is not a residual connection, but based on the cascade of the first residual block, that is, the output of the previous processing module in the backbone network is directly used as the input of the next processing module, and the output of the last processing module is used as the output of the backbone network, then the output of the last processing module in the backbone network can be understood as a deep feature. Due to the cumulative effect caused by the cascade of edge detection blocks, the features will be over-extracted. Therefore, the deep features contain too much redundant information, which reduces the accuracy of the extracted boundary features and affects the subsequent tampering detection. However, if the output of the processing module before the last processing module is directly used, that is, the shallow features are used as the output of the backbone network, the subtle boundary information contained in the shallow features will disappear in the subsequent processing, which will still affect the subsequent tampering detection.

[0129] In the embodiment of the present application, the first residual block is used to perform residual connection on the edge detection block, so that the deep features and the shallow features can be combined, avoiding the accumulation effect, and ensuring the retention of subtle boundary information in the processing results output by the backbone network, thereby improving the accuracy of boundary feature detection. On this basis, the accuracy of tampering detection is improved. In practical applications, this boundary feature detection method that combines deep features and shallow features can also be expressed as multi-scale edge feature detection.

[0130] In one embodiment, the second network includes a spatial pyramid pooling module, a plurality of second residual blocks and a plurality of first attention blocks; the size of each of the plurality of second residual blocks decreases in sequence based on a set ratio;

[0131] The second residual block is used to extract the content features of the abnormal area in the image;

[0132] The spatial pyramid pooling module is used to convert the input feature map into a feature map of a set size;

[0133] The first attention block is used to suppress information irrelevant to abnormal regions in the image;

[0134] Correspondingly, the second network is called to process the first image to obtain a regional feature map, including:

[0135] Calling each second residual block and each first attention block in the second network in sequence to process the first image to obtain a fourth output;

[0136] The spatial pyramid pooling module is called to process the fourth output to obtain a regional feature map.

[0137] In practical applications, the second residual blocks and the first attention blocks can be arranged alternately. In the process of calling the second network, each second residual block and each first attention block can be called in sequence according to the arrangement order.

[0138] Exemplarily, the second network may include 4 second residual blocks, namely, second residual block 1, second residual block 2, second residual block 3 and second residual block 4, and 3 first attention blocks, namely, first attention block 1, first attention block 2 and first attention block 3; wherein, the sizes of second residual block 1, second residual block 2, second residual block 3 and second residual block 4 decrease in sequence based on a ratio of 2. In the process of calling the second network, the calling order of these second residual blocks and first attention blocks can be: second residual block 1, first attention block 1, second residual block 2, first attention block 2, second residual block 3, first attention block 3, second residual block 4.

[0139] In practical applications, the second residual block and the first attention block can be understood as processing modules in the second network. When the second network is called to process the first image, the output of the mth processing module in the second network can be directly used as the input of the m+1th processing module, where m is a positive integer; and the output of the last processing module can be used as the output of the second network. The input of the first processing module in the second network can be the first image.

[0140] Here, the first attention block is used to suppress information that is not related to the abnormal area in the image. In practical applications, the second network can adapt to different tampering operations based on the guidance of the first attention block to avoid misdetecting the untampered area as the tampered area. The tampering operation may include: copying, filling, and splicing.

[0141] Exemplarily, when the tampering operation corresponding to the tampered area in the image is copying, although the copied source area is the same as the content of the tampered area, the content of the source area belongs to the original content of the image. Therefore, the source area is not the tampered area, and the relevant information of the source area, such as the pixel position, can be regarded as information irrelevant to the abnormal area in the image; the second network can suppress the relevant information of the source area based on the first attention block, thereby avoiding misdetecting the source area as the tampered area.

[0142] In practical applications, the first attention block can be a squeeze-and-excitation (SE) attention block.

[0143] In practical applications, the spatial pyramid pooling module can be the last processing module in the second network. The spatial pyramid pooling module can be used to convert feature maps of any size into feature maps of a set size, increase the receptive field, and separate obvious context features in the feature map, thereby enhancing the feature expression ability of the feature map.

[0144] In practical applications, the fourth output can be a feature map. The regional feature map obtained by calling the spatial pyramid pooling module to process the fourth output enhances the feature expression capability compared to the fourth output, thereby providing a good data basis for subsequent tampering detection and helping to improve the accuracy of tampering detection.

[0145] In one embodiment, the third network includes a second attention block and an anomaly perception block;

[0146] The second attention block is used to enhance the weights of the channels related to the tampering operation in the input feature map;

[0147] The anomaly perception block is used to detect tampered areas in the image;

[0148] Correspondingly, the third network is called to process the fused feature map to obtain a first output, including:

[0149] Call the second attention block to process the fused feature map to obtain an enhanced fused feature map;

[0150] The anomaly perception block is called to process the third input to obtain the first output; the third input represents the feature map obtained based on the enhanced fusion feature map.

[0151] In practical applications, the second attention block may be a convolutional block attention module (CBAM). Based on the second attention block, the third network can more accurately detect more tampering operation traces, thereby increasing the accuracy of tampering detection.

[0152] In one embodiment, the anomaly perception block includes a second LSTM network and a plurality of Z-score pools, each of the plurality of Z-score pools corresponding to a pooling window of a set size;

[0153] Correspondingly, the abnormality perception block is called to process the third input to obtain the first output, including:

[0154] Call the pooling window corresponding to each Z-score pool, perform average pooling on the third input, and obtain the pooling result corresponding to each Z-score pool;

[0155] Based on the pooling result corresponding to each Z-score pool, Z-score calculation is performed to obtain the Z-score vector corresponding to each Z-score pool;

[0156] The second LSTM network is called to process the concatenated features to obtain a first output; the concatenated features are obtained by concatenating the Z score vectors corresponding to the multiple Z score pools.

[0157] In practical applications, multiple Z-score pools can be used to learn the differences between local features in the input feature map and the features of the tampered area in the corresponding image to be tested, so as to more accurately detect local anomalies in the image and improve the accuracy of tampering detection.

[0158] In practical applications, by calling the pooling window corresponding to each Z-score pool to perform average pooling on the third input, the abnormal local features in the third output can be extracted. Then, through Z-score calculation, the extracted abnormal local features can be classified to determine whether the image pixels corresponding to the abnormal local features belong to the tampered image pixels, that is, to determine the tampered area.

[0159] In practical applications, the size of the pooling window corresponding to each Z-score pool in the multiple Z-score pools may be different. The pooling windows corresponding to the multiple Z-score pools may include: a global pooling window and multiple local pooling windows, the size of the global pooling window is equal to the size of the feature map of the third input representation, and the size of the local pooling window is smaller than the size of the feature map of the third input representation.

[0160] Exemplarily, the abnormal perception block may include 4 Z-score pools, namely, Z-score pool 1, Z-score pool 2, Z-score pool 3 and Z-score pool 4, and the pooling windows corresponding to these 4 Z-score pools are respectively: pooling window 1, pooling window 2, pooling window 3 and pooling window 4; wherein, pooling window 1 may be a global pooling window, and pooling window 2, pooling window 3 and pooling window 4 may be local pooling windows. When the size of the feature map represented by the third input is 65×65, the size of pooling window 1 is 65×65, the size of pooling window 2 may be 31×31, the size of pooling window 3 may be 15×15, and the size of pooling window 4 may be 7×7.

[0161] In practical applications, in the process of calling the global pooling window to perform average pooling on the third input, the global average feature of the third input can be calculated, and the global average feature can be used as the corresponding pooling result.

[0162] In practical applications, the global average feature can be expressed as:

[0163]

[0164] Among them, δ F represents the global average feature, and F represents the third input. In practical applications, the feature map represented by the third input can be regarded as an H×W tensor, and F[i,j] can represent a local feature in the third input, specifically, the feature located in the i-th row and the j-th column in the third input.

[0165] For ease of explanation, we use Z F Characterizes the Z-score vector corresponding to the Z-score pool corresponding to the global pooling window. F It can include multiple vector elements, each of which can be understood as the Z score corresponding to a feature. F A Z score in can be expressed as:

[0166] Z F [i,j]=D F [i,j] / η F

[0167] Among them, Z F [i,j] represents the Z score corresponding to F[i,j]. F [i,j] can be used to describe the difference between F[i,j] and the global average feature, η F Characterizes the standard deviation of multiple features involved in the global average feature.

[0168] In practical applications, D F [i,j] can be expressed as: D F[i,j]=F[i,j]-δ F , η F It can be expressed as:

[0169]

[0170] In practical applications, when calling the local pooling window to perform average pooling on the third input, each of the multiple features in the third input can be used as a feature center, and the local pooling window can be called to calculate: the local average feature corresponding to each feature center.

[0171] For ease of explanation, we use Characterization: The local average feature corresponding to the feature center F[i,j] when the local pooling window of size n×n is called to process the third output. And, using Characterizes the Z-score vector corresponding to the Z-score pool corresponding to the local pooling window of size n×n, can include multiple vector elements, each of which can be understood as the Z score corresponding to a feature. A Z score in can be expressed as:

[0172]

[0173] Among them, Z F n×n [i,j] represents the Z score corresponding to F[i,j], It can be used to describe the difference between F[i,j] and the corresponding local average feature, η F n×n [i,j] represents the standard deviation of multiple features involved in the local average feature corresponding to F[i,j]. In practical applications, It can be expressed as:

[0174] In practical applications, the Z score vectors corresponding to multiple Z score pools can be concatenated to obtain a concatenated vector. The concatenated vector can be expressed as:

[0175]

[0176] in, represents the concatenated vector, Represents the Z-score vector corresponding to the Z-score pool corresponding to the local pooling window of size n1×n1, Characterization with size n k ×n k The Z score vector corresponding to the local pooling window, Z F Characterizes the Z-score vector corresponding to the Z-score pool corresponding to the global pooling window.

[0177] In practical applications, after obtaining the splicing vector, the splicing vector can be connected to the feature dimension to obtain the splicing feature. The splicing feature can be understood as: the obtained Z score vector is strung together with the size of the corresponding pooling window as the first dimension to obtain the feature combination.

[0178] In practical applications, the Z-score vector can be represented as a 2-dimensional tensor, based on which the concatenated feature can be represented as a 3-dimensional tensor.

[0179] In practical applications, calling the second LSTM network to process the splicing features can be understood as: in order to simulate the characteristic that the human eye will walk closer to observe an object when it cannot see it clearly, each Z score vector in the splicing features is evaluated based on the time series information to determine the tampered area. Specifically, the second LSTM network can analyze each Z score vector in the splicing features and serialize them from the time dimension to generate high-dimensional features with a time axis, and then obtain the first output. The high-dimensional features processed by the second LSTM network can be represented as a 4-dimensional tensor.

[0180] In practical applications, the anomaly perception block may further include a convolution block. After calling the second LSTM network to process the concatenated features, the convolution block may be called to process the high-dimensional features generated by the second LSTM network to convert the high-dimensional features into an output of a set format, for example, into a masked output, thereby obtaining a first output.

[0181] In practical applications, the second LSTM network can be a convolutional two-dimensional long short-term memory network (Conv2DLSTM, Convolutional Two-Dimensional Long Short-Term Memory Network) network.

[0182] For ease of understanding, Figure 4 The structure of the abnormal perception block is shown in FIG.

[0183] In practical applications, the image tampering detection method provided in the embodiment of the present application can be regarded as: processing the first image based on the trained image tampering detection network to obtain the first output. The trained image tampering detection network can be composed of the first network, the second network and the third network.

[0184] In one embodiment, the image tampering detection method provided by the embodiment of the present application further includes:

[0185] The fourth network, the fifth network and the sixth network are trained multiple times based on multiple training samples until the set convergence condition is reached; the training samples represent the tampered images; the fourth network represents the first network before the training is completed, the fifth network represents the second network before the training is completed, and the sixth network represents the third network before the training is completed; wherein,

[0186] Performing one training iteration among multiple training iterations on the fourth network, the fifth network, and the sixth network based on the multiple training samples includes:

[0187] Based on the image tampering detection method in any of the foregoing embodiments, calling the fourth network, the fifth network, and the sixth network to process each of the multiple training samples to obtain a fifth output corresponding to each training sample; the fifth output is used to describe the tampered area in the corresponding training sample;

[0188] Based on the first loss value, updating the fourth network, the fifth network and the sixth network;

[0189] Wherein, the first loss value represents a combination of the second loss value and the third loss value;

[0190] The second loss value is calculated based on the binary cross entropy loss function for the fifth output corresponding to each training sample;

[0191] The third loss value is obtained based on the calculation of the sixth output corresponding to each training sample by the binary cross entropy loss function; the sixth output is obtained based on the processing of the corresponding training sample by the second residual block and the first attention block in the fifth network.

[0192] In practical applications, the combination of the fourth network, the fifth network and the sixth network can be regarded as an image tampering detection network before training is completed, that is, an image tampering detection network in the training stage.

[0193] In practical applications, during a training iteration of the image tampering detection network in the training phase, the overall loss value can be obtained based on the output of the image tampering detection network in the training phase during the processing of the training samples and the sample labels corresponding to the training samples, and then the overall loss value is fed back to the image tampering detection network in the training phase, and based on the overall loss value, the network parameters in the image tampering detection network in the training phase are updated. Exemplarily, the network parameters can be the weights and biases of each node in the network. In this way, multiple training iterations are repeated until the set convergence condition is reached. Setting the convergence condition may include: the overall loss value is less than a set threshold.

[0194] In practical applications, the overall loss value can be understood as the degree of inconsistency between the predicted value obtained by the image tampering detection network based on the training sample in the training phase and the target value expected to be obtained by the network based on the training sample. The sample label corresponding to the training sample can be understood as the target value expected to be obtained by the image tampering detection network based on the training sample in the training phase. For example, the sample label corresponding to the training sample can be a mask output.

[0195] Here, based on the first loss value, the fourth network, the fifth network and the sixth network are updated. In practical applications, the first loss value can be understood as the overall loss value.

[0196] In practical applications, the fifth output corresponding to each training sample and the sample label corresponding to the training sample can be processed based on the cross entropy loss function to obtain a second loss value. The processing method corresponding to the fifth output can be understood by referring to the processing method corresponding to the first output. The second loss value can also be expressed as a positioning loss, which can be used to supervise the detection of tampered areas by the image tampering detection network in the training stage.

[0197] In practical applications, the loss function corresponding to the second loss value can be expressed as:

[0198] L1=y gt log(y p )+(1-y gt )log(1-log(y p ))

[0199] Among them, y gt Characterize the sample label corresponding to the training sample, y p The fifth output corresponding to the characterization training sample.

[0200] In practical applications, the sixth output corresponding to each training sample and the sample label corresponding to the training sample can be processed based on the cross entropy loss function to obtain a third loss value. The processing method corresponding to the sixth output can be understood by referring to the processing method corresponding to the fourth output. The third loss value can also be expressed as an auxiliary loss, which can be used to optimize the learning process of the image tampering detection network in the training phase.

[0201] In practical applications, the loss function corresponding to the third loss value can be expressed as:

[0202] L2=y gt log(y b )+(1-y gt )log(1-log(y b ))

[0203] Among them, y gt Characterize the sample label corresponding to the training sample, y bThe sixth output corresponding to the characterization training sample.

[0204] It should be noted that both L1 and L2 can be understood as cross entropy loss functions.

[0205] In practical applications, the second loss value and the third loss value may be combined to obtain the first loss value.

[0206] In practical applications, the first loss value may represent the sum of the second loss value and the first product, and the first product may represent the product of the set weight and the third loss value. The set weight may be understood as a parameter for balancing the influence of the second loss value and the third loss value on the training iteration.

[0207] In practical applications, the loss function corresponding to the first loss value can be expressed as:

[0208] L total =L1+aL2

[0209] Among them, α represents the set weight, L1 represents the loss function corresponding to the second loss value, and L2 represents the loss function corresponding to the third loss value.

[0210] In an embodiment of the present application, the image tampering detection network in the training stage is updated based on the first loss value during the training iteration process. The first loss value combines the second loss value and the third loss value, thereby supervising the detection of the tampered area by the image tampering detection network in the training stage. At the same time, the learning process is optimized, and the training effect of the image tampering detection network is improved. On this basis, the detection accuracy of the image tampering detection network after the training is improved, that is, the accuracy of image tampering detection is improved.

[0211] The present application is further described in detail below in conjunction with application examples.

[0212] The present application embodiment provides an image tampering detection network, see Figure 5 ,The image tampering detection network includes: a multi-scale edge detection branch, an abnormal region detection branch and a tampering detection branch.

[0213] Among them, the multi-scale edge detection branch is equivalent to the first network in the embodiment of the present application, the abnormal area detection branch is equivalent to the second network in the embodiment of the present application, and the tampering detection branch is equivalent to the third network in the embodiment of the present application.

[0214] The present application embodiment also provides an image tampering detection method, which can be performed based on the image tampering detection network in the present application embodiment. Figure 6 ,When performing image tampering detection, the main steps include:

[0215] Step 1: Call the multi-scale edge detection branch to process the image to be tested and obtain the edge feature map.

[0216] Here, the image to be tested is equivalent to the first image in the embodiment of the present application.

[0217] Step 2: Call the abnormal region detection branch to process the image to be tested and obtain the regional feature map.

[0218] It should be noted that step 1 and step 2 can be executed sequentially or simultaneously. When executed sequentially, there is no restriction on the execution order.

[0219] Step 3: Fuse the edge feature map and the regional feature map to obtain a fused feature map.

[0220] In practical applications, fusion processing can also be expressed as connection processing or connection fusion processing.

[0221] Step 4: Call the tampering detection branch to process the fused feature map to obtain the detection result.

[0222] Here, the detection result is equivalent to the first output in the embodiment of the present application.

[0223] In practical applications, the tampering detection branch can analyze the fused feature map, detect the tampered pixels in the image to be tested, and output the detection results, thereby realizing the detection of the tampered area.

[0224] In practical applications, before calling the image tampering detection network to process the image to be tested, the image tampering detection network before training can be trained iteratively based on the positioning loss and the auxiliary loss to obtain the image tampering detection network after training.

[0225] In the application embodiment of the present application, the multi-scale edge detection branch and the abnormal area detection branch are first called separately to detect the abnormal area in the image to be tested, and then the edge feature map output by the multi-scale edge detection branch and the area feature map output by the abnormal area detection branch are fused, which is equivalent to performing multi-branch detection based on the multi-scale edge detection branch and the abnormal area detection branch. Compared with the related technology, multiple network models are called in combination to improve the accuracy of abnormal area detection. On this basis, the accuracy of tampering detection can be improved; and, in the above scheme, the tampering detection branch is also called to process the fused feature map to obtain the detection result, which is equivalent to detecting the tampered area based on the abnormal area in the image to be tested, narrowing the detection range, thereby further improving the accuracy of tampering detection.

[0226] Based on the above embodiments, the present application also provides an image tampering detection device, see Figure 7 , the image tampering detection device comprises:

[0227] A first calling unit 71 is used to call a first network to process a first image to obtain an edge feature map; the edge feature map is used to describe the boundary of an abnormal area in the first image;

[0228] A second calling unit 72 is used to call a second network to process the first image to obtain a region feature map; the region feature map is used to describe the content of the abnormal region in the first image;

[0229] A fusion unit 73, configured to fuse the edge feature map with the region feature map to obtain a fused feature map;

[0230] The third calling unit 74 is used to call the third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

[0231] In one embodiment, the first network includes a backbone network and a first LSTM network;

[0232] The backbone network is used to detect spatial domain features of the boundaries of abnormal regions in the image;

[0233] The first LSTM network is used to detect frequency domain difference characteristics between abnormal areas and normal areas in the image;

[0234] Correspondingly, the first calling unit 71 calls the first network to process the first image to obtain an edge feature map, including:

[0235] Calling the backbone network and the first LSTM network in parallel to process the first image, and obtaining a processing result output by the backbone network and a processing result output by the first LSTM network;

[0236] The processing result of the backbone network output is integrated with the processing result of the first LSTM network output to obtain the edge feature map.

[0237] In one embodiment, the backbone network includes a plurality of edge detection blocks and a plurality of first residual blocks;

[0238] The edge detection block is used to detect the spatial domain features of the boundary of the abnormal area in the image;

[0239] The multiple first residual blocks are used to perform residual connection on the multiple edge detection blocks;

[0240] Correspondingly, when calling the backbone network to process the first image, the first calling unit 71 calls the multiple edge detection blocks and the multiple first residual blocks in the backbone network in a set order based on the first image.

[0241] In one embodiment, the first calling unit 71, in the process of calling the plurality of edge detection blocks and the plurality of first residual blocks in the backbone network according to a set order, is further configured to:

[0242] Calling the nth edge detection block to process the first input to obtain a second output; wherein n is a positive integer;

[0243] Calling the nth first residual block to process the second output to obtain a third output;

[0244] The first input and the third output are summed, or the second output and the third output are summed to obtain a processing result; the processing result is used to: input the n+1th edge detection block, or as the processing result output by the backbone network.

[0245] In one embodiment, the second network includes a spatial pyramid pooling module, a plurality of second residual blocks and a plurality of first attention blocks; the size of each of the plurality of second residual blocks decreases in sequence based on a set ratio;

[0246] The second residual block is used to extract content features of abnormal areas in the image;

[0247] The spatial pyramid pooling module is used to convert the input feature map into a feature map of a set size;

[0248] The first attention block is used to suppress information irrelevant to abnormal areas in the image;

[0249] Correspondingly, the second calling unit 72 calls the second network to process the first image to obtain a regional feature map, including:

[0250] calling each second residual block and each first attention block in the second network in sequence to process the first image to obtain a fourth output;

[0251] The spatial pyramid pooling module is called to process the fourth output to obtain the regional feature map.

[0252] In one embodiment, the third network includes a second attention block and an anomaly perception block;

[0253] The second attention block is used to enhance the weight of the channel related to the tampering operation in the input feature map;

[0254] The anomaly sensing block is used to detect tampered areas in the image;

[0255] Correspondingly, the third calling unit 74 calls the third network to process the fused feature map to obtain a first output, including:

[0256] Calling the second attention block to process the fused feature map to obtain an enhanced fused feature map;

[0257] The anomaly perception block is called to process a third input to obtain the first output; the third input represents a feature map obtained based on the enhanced fusion feature map.

[0258] In one embodiment, the anomaly perception block includes a second LSTM network and a plurality of Z-score pools, each of the plurality of Z-score pools corresponding to a pooling window of a set size;

[0259] Correspondingly, the third calling unit 74 calls the abnormality perception block to process the third input to obtain the first output, including:

[0260] Calling the pooling window corresponding to each Z-score pool, performing average pooling processing on the third input, and obtaining a pooling result corresponding to each Z-score pool;

[0261] Based on the pooling result corresponding to each Z-score pool, Z-score calculation is performed to obtain the Z-score vector corresponding to each Z-score pool;

[0262] The second LSTM network is called to process the splicing feature to obtain the first output; the splicing feature is obtained by splicing the Z score vectors corresponding to the multiple Z score pools.

[0263] In one embodiment, the image tampering detection device further includes a training unit, and the training unit is used to:

[0264] The fourth network, the fifth network and the sixth network are trained iteratively for multiple times based on multiple training samples until the set convergence condition is reached; the training samples represent the tampered images; the fourth network represents the first network before the training is completed, the fifth network represents the second network before the training is completed, and the sixth network represents the third network before the training is completed; wherein,

[0265] The performing one training iteration among multiple training iterations on the fourth network, the fifth network, and the sixth network based on the multiple training samples comprises:

[0266] Based on the image tampering detection method in any of the foregoing embodiments, calling the fourth network, the fifth network, and the sixth network to process each of the multiple training samples to obtain a fifth output corresponding to each training sample; the fifth output is used to describe the tampered area in the corresponding training sample;

[0267] Based on the first loss value, updating the fourth network, the fifth network and the sixth network;

[0268] Wherein, the first loss value represents a combination of the second loss value and the third loss value;

[0269] The second loss value is obtained by calculating the fifth output corresponding to each training sample based on the binary cross entropy loss function;

[0270] The third loss value is obtained based on the calculation of the sixth output corresponding to each training sample by the binary cross entropy loss function; the sixth output is obtained based on the processing of the corresponding training sample by the second residual block and the first attention block in the fifth network.

[0271] In actual application, the first calling unit 71, the second calling unit 72, the fusion unit 73, the third calling unit 74 and the training unit can all be implemented by a processor in the image tampering detection device.

[0272] It should be noted that: the image tampering detection device provided in the above embodiment only uses the division of the above program modules as an example when performing image tampering detection. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the image tampering detection device provided in the above embodiment and the image tampering detection method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0273] Based on the hardware implementation of the above program module, and in order to implement the method of the embodiment of the present application, the present application also provides an electronic device, referring to Figure 8 , the electronic device comprises:

[0274] The first communication interface 1 is capable of exchanging information with other devices;

[0275] The first processor 2 is connected to the first communication interface 1 to implement information exchange with other devices, and is used to execute the method provided by one or more technical solutions in the above embodiments when running a computer program. The computer program is stored in the first memory 3.

[0276] Specifically, the first processor 2 is used to call the first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of the abnormal area in the first image;

[0277] Calling a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal region in the first image;

[0278] fusing the edge feature map with the region feature map to obtain a fused feature map; and,

[0279] Calling a third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

[0280] In one embodiment, the first network includes a backbone network and a first LSTM network;

[0281] The backbone network is used to detect spatial domain features of the boundaries of abnormal regions in the image;

[0282] The first LSTM network is used to detect frequency domain difference characteristics between abnormal areas and normal areas in the image;

[0283] Correspondingly, the first processor 2 calls the first network to process the first image to obtain an edge feature map, including:

[0284] Calling the backbone network and the first LSTM network in parallel to process the first image, and obtaining a processing result output by the backbone network and a processing result output by the first LSTM network;

[0285] The processing result of the backbone network output is integrated with the processing result of the first LSTM network output to obtain the edge feature map.

[0286] In one embodiment, the backbone network includes a plurality of edge detection blocks and a plurality of first residual blocks;

[0287] The edge detection block is used to detect the spatial domain features of the boundaries of abnormal areas in the image;

[0288] The multiple first residual blocks are used to perform residual connection on the multiple edge detection blocks;

[0289] Correspondingly, when calling the backbone network to process the first image, the first processor 2 calls the multiple edge detection blocks and the multiple first residual blocks in the backbone network in a set order based on the first image.

[0290] In one embodiment, the first processor 2, in the process of calling the plurality of edge detection blocks and the plurality of first residual blocks in the backbone network according to a set order, is further configured to:

[0291] Calling the nth edge detection block to process the first input to obtain a second output; wherein n is a positive integer;

[0292] Calling the nth first residual block to process the second output to obtain a third output;

[0293] The first input and the third output are summed, or the second output and the third output are summed to obtain a processing result; the processing result is used to: input the n+1th edge detection block, or as the processing result output by the backbone network.

[0294] In one embodiment, the second network includes a spatial pyramid pooling module, a plurality of second residual blocks and a plurality of first attention blocks; the size of each of the plurality of second residual blocks decreases in sequence based on a set ratio;

[0295] The second residual block is used to extract content features of abnormal areas in the image;

[0296] The spatial pyramid pooling module is used to convert the input feature map into a feature map of a set size;

[0297] The first attention block is used to suppress information irrelevant to abnormal areas in the image;

[0298] Correspondingly, the first processor 2 calls the second network to process the first image to obtain a regional feature map, including:

[0299] calling each second residual block and each first attention block in the second network in sequence to process the first image to obtain a fourth output;

[0300] The spatial pyramid pooling module is called to process the fourth output to obtain the regional feature map.

[0301] In one embodiment, the third network includes a second attention block and an anomaly perception block;

[0302] The second attention block is used to enhance the weight of the channel related to the tampering operation in the input feature map;

[0303] The anomaly sensing block is used to detect tampered areas in the image;

[0304] Correspondingly, the first processor 2 calls the third network to process the fused feature map to obtain a first output, including:

[0305] Calling the second attention block to process the fused feature map to obtain an enhanced fused feature map;

[0306] The anomaly perception block is called to process a third input to obtain the first output; the third input represents a feature map obtained based on the enhanced fusion feature map.

[0307] In one embodiment, the anomaly perception block includes a second LSTM network and a plurality of Z-score pools, each of the plurality of Z-score pools corresponding to a pooling window of a set size;

[0308] Correspondingly, the first processor 2 calls the abnormality perception block to process the third input to obtain the first output, including:

[0309] Calling the pooling window corresponding to each Z-score pool, performing average pooling processing on the third input, and obtaining a pooling result corresponding to each Z-score pool;

[0310] Based on the pooling result corresponding to each Z-score pool, Z-score calculation is performed to obtain the Z-score vector corresponding to each Z-score pool;

[0311] The second LSTM network is called to process the splicing feature to obtain the first output; the splicing feature is obtained by splicing the Z score vectors corresponding to the multiple Z score pools.

[0312] In one embodiment, the first processor 2 is further configured to:

[0313] The fourth network, the fifth network and the sixth network are trained iteratively for multiple times based on multiple training samples until the set convergence condition is reached; the training samples represent the tampered images; the fourth network represents the first network before the training is completed, the fifth network represents the second network before the training is completed, and the sixth network represents the third network before the training is completed; wherein,

[0314] The performing one training iteration among multiple training iterations on the fourth network, the fifth network, and the sixth network based on the multiple training samples comprises:

[0315] Based on the method for image tampering detection in any of the foregoing embodiments, calling the fourth network, the fifth network, and the sixth network to process each of the multiple training samples to obtain a fifth output corresponding to each training sample; the fifth output is used to describe the tampered area in the corresponding training sample;

[0316] Based on the first loss value, updating the fourth network, the fifth network and the sixth network;

[0317] Wherein, the first loss value represents a combination of the second loss value and the third loss value;

[0318] The second loss value is obtained by calculating the fifth output corresponding to each training sample based on the binary cross entropy loss function;

[0319] The third loss value is obtained based on the calculation of the sixth output corresponding to each training sample by the binary cross entropy loss function; the sixth output is obtained based on the processing of the corresponding training sample by the second residual block and the first attention block in the fifth network.

[0320] It should be noted that the specific processing process of the first communication interface 1 can be understood by referring to the above method.

[0321] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus system 4.

[0322] The first memory 3 in the embodiment of the present application is used to store various types of data to support operations in the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0323] The method disclosed in the above embodiment of the present application can be applied to the first processor 2, or implemented by the first processor 2. The first processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 2. The above-mentioned first processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 2 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the first memory 3. The first processor 2 reads the information in the first memory 3 and completes the steps of the above method in combination with its hardware.

[0324] In an exemplary embodiment, the electronic device may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general purpose processors, controllers, MCUs, Microprocessors, or other electronic components to perform the aforementioned methods.

[0325] It can be understood that the first memory 3 of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0326] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, an electronic device storing a computer program, and the computer program can be executed by the first processor 2 of the electronic device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0327] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, and the computer program can be executed by the first processor 2 of the electronic device to complete the steps of any of the aforementioned methods.

[0328] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0329] The term "and / or" herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the three situations that A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "one or more" herein represents any combination of at least two of any one or more of a plurality. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0330] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

Claims

1. A method for detecting image tampering, characterized in that: The method comprises: Calling the first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of the abnormal area in the first image; Calling a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal region in the first image; Fusing the edge feature map with the regional feature map to obtain a fused feature map; Calling a third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

2. The method according to claim 1, characterized in that The first network includes a backbone network and a first long short-term memory (LSTM) network; The backbone network is used to detect spatial domain features of the boundaries of abnormal regions in the image; The first LSTM network is used to detect frequency domain difference characteristics between abnormal areas and normal areas in the image; Correspondingly, calling the first network to process the first image to obtain an edge feature map includes: Calling the backbone network and the first LSTM network in parallel to process the first image, and obtaining a processing result output by the backbone network and a processing result output by the first LSTM network; The processing result of the backbone network output is integrated with the processing result of the first LSTM network output to obtain the edge feature map.

3. The method according to claim 2, characterized in that The backbone network includes a plurality of edge detection blocks and a plurality of first residual blocks; The edge detection block is used to detect the spatial domain features of the boundaries of abnormal areas in the image; The multiple first residual blocks are used to perform residual connection on the multiple edge detection blocks; Correspondingly, when calling the backbone network to process the first image, based on the first image, the multiple edge detection blocks and the multiple first residual blocks in the backbone network are called in a set order.

4. The method according to claim 3, characterized in that In the process of calling the plurality of edge detection blocks and the plurality of first residual blocks in the backbone network according to a set order, the method comprises: Calling the nth edge detection block to process the first input to obtain a second output; wherein n is a positive integer; Calling the nth first residual block to process the second output to obtain a third output; The first input and the third output are summed, or the second output and the third output are summed to obtain a processing result; the processing result is used to: input the n+1th edge detection block, or as the processing result output by the backbone network.

5. The method according to claim 1, characterized in that The second network includes a spatial pyramid pooling module, a plurality of second residual blocks and a plurality of first attention blocks; the size of each of the plurality of second residual blocks decreases in sequence based on a set ratio; The second residual block is used to extract content features of abnormal areas in the image; The spatial pyramid pooling module is used to convert the input feature map into a feature map of a set size; The first attention block is used to suppress information irrelevant to abnormal areas in the image; Correspondingly, calling the second network to process the first image to obtain a regional feature map includes: calling each second residual block and each first attention block in the second network in sequence to process the first image to obtain a fourth output; The spatial pyramid pooling module is called to process the fourth output to obtain the regional feature map.

6. The method according to claim 1, characterized in that The third network includes a second attention block and an abnormal perception block; The second attention block is used to enhance the weight of the channel related to the tampering operation in the input feature map; The anomaly sensing block is used to detect tampered areas in the image; Correspondingly, calling the third network to process the fused feature map to obtain a first output includes: Calling the second attention block to process the fused feature map to obtain an enhanced fused feature map; The anomaly perception block is called to process a third input to obtain the first output; the third input represents a feature map obtained based on the enhanced fusion feature map.

7. The method according to claim 6, characterized in that The anomaly perception block includes a second LSTM network and a plurality of Z-score pools, each of the plurality of Z-score pools corresponding to a pooling window of a set size; Correspondingly, calling the abnormality perception block to process the third input to obtain the first output includes: Calling the pooling window corresponding to each Z-score pool, performing average pooling processing on the third input, and obtaining a pooling result corresponding to each Z-score pool; Based on the pooling result corresponding to each Z-score pool, Z-score calculation is performed to obtain the Z-score vector corresponding to each Z-score pool; The second LSTM network is called to process the splicing feature to obtain the first output; the splicing feature is obtained by splicing the Z score vectors corresponding to the multiple Z score pools.

8. The method according to any one of claims 1 to 7, further comprising: Performing multiple training iterations on the fourth network, the fifth network, and the sixth network based on multiple training samples until a set convergence condition is reached; The training sample represents the tampered image; the fourth network represents the first network before training is completed, the fifth network represents the second network before training is completed, and the sixth network represents the third network before training is completed; wherein, The performing one training iteration among multiple training iterations on the fourth network, the fifth network, and the sixth network based on the multiple training samples comprises: Based on the method described in any one of claims 1 to 7, calling the fourth network, the fifth network, and the sixth network, processing each of the multiple training samples, and obtaining a fifth output corresponding to each training sample; the fifth output is used to describe the tampered area in the corresponding training sample; Based on the first loss value, updating the fourth network, the fifth network and the sixth network; Wherein, the first loss value represents a combination of the second loss value and the third loss value; The second loss value is obtained by calculating the fifth output corresponding to each training sample based on the binary cross entropy loss function; The third loss value is obtained based on the calculation of the sixth output corresponding to each training sample by the binary cross entropy loss function; the sixth output is obtained based on the processing of the corresponding training sample by the second residual block and the first attention block in the fifth network.

9. An image tampering detection device, characterized in that: include: A first calling unit, used for calling a first network to process the first image to obtain an edge feature map; The edge feature map is used to describe the boundary of the abnormal area in the first image; A second calling unit is used to call a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal area in the first image; A fusion unit, used for fusing the edge feature map with the regional feature map to obtain a fused feature map; A third calling unit is used to call a third network to process the fused feature map to obtain a first output; The first output is used to describe a tampered area in the first image.

10. An electronic device, characterized in that: It includes a first processor and a first communication interface; wherein, The first processor is used to call the first network to process the first image to obtain an edge feature map; the edge feature map is used to describe the boundary of the abnormal area in the first image; Calling a second network to process the first image to obtain a regional feature map; the regional feature map is used to describe the content of the abnormal region in the first image; fusing the edge feature map with the region feature map to obtain a fused feature map; and, Calling a third network to process the fused feature map to obtain a first output; the first output is used to describe the tampered area in the first image.

11. An electronic device, characterized in that: include: a first processor and a first memory for storing a computer program executable on the processor, Wherein, when the first processor is used to run the computer program, the steps of the method described in any one of claims 1 to 8 are executed.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images