An image tampering detection method, device and electronic equipment

Abstract

The application provides an image tampering detection method and device and electronic equipment. The method comprises: performing segmentation processing based on target image data to be detected to obtain different target sub-image data to be detected, performing target feature extraction on each different target sub-image data to be detected to obtain a plurality of target image feature data of the entire target image data to be detected, inputting the plurality of target image feature data into a pre-trained image tampering detection and recognition model, and determining a detection result output by the pre-trained image tampering detection and recognition model as a target detection result. According to the embodiment of the application, the traditional image compression method is abandoned, the target image data to be detected is segmented to obtain different local image data, and the process does not lose the pixel points in the original image data. In this way, the accuracy of the detection result output by the image tampering detection and recognition model is higher, and a more accurate image tampering detection result is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image tampering detection method and device and electronic equipment. BACKGROUND

[0002] With the continuous development of technology, especially the development of picture PS (Photoshop) technology and AI (Artificial Intelligence) drawing technology, the Internet is filled with a large number of images modified by technology. For some business auditing practical application scenarios that need to use original images, if the audited image is an image tampered by PS or AI technology, it will inevitably increase the work burden of the auditing personnel or the auditing equipment.

[0003] In view of this, the related technology proposes to detect image tampering by machine learning algorithm or deep learning algorithm to identify whether the input image is a tampered image. However, the existing image tampering detection technology based on machine learning algorithm or deep learning algorithm usually compresses the input image and then detects the tampering of the compressed image. Since the image compression process will lose some details of the original input image, the existing image tampering detection method often has the problem of low detection accuracy. SUMMARY

[0004] Therefore, the present application provides an image tampering detection method, device and electronic equipment to solve the problem of low detection accuracy of the existing image tampering detection method. Among them:

[0005] In a first aspect, the present application provides an image tampering detection method, wherein the method comprises:

[0006] segmenting the target image data to be detected to obtain different target sub-image data to be detected, wherein each target sub-image data to be detected is local image data in the target image to be detected, and the coverage ranges of the local image data do not overlap with each other;

[0007] target feature extraction is performed on each target sub-image data to be detected to obtain target image feature data of the target image data to be detected;

[0008] The target image feature data is input into a pre-trained image tampering detection recognition model, and the detection result output by the pre-trained image tampering detection recognition model is determined as a target detection result.

[0009] In a second possible embodiment of the first aspect, the segmentation of the target image data to be detected to obtain different target sub-image data to be detected comprises:

[0010] Based on the original file size of the target image data to be detected, the target image data to be detected is segmented to obtain a plurality of target sub-image data of equal size.

[0011] In a third possible implementation of the first aspect, the target feature extraction is performed on each of the target sub-image data to be detected to obtain target image feature data of the target image data to be detected, including:

[0012] The target sub-image data to be detected is preprocessed to obtain a plurality of first target data streams of the target sub-image data to be detected;

[0013] According to a preset splicing rule, the first target data streams are spliced to obtain second target data streams of the target sub-image data to be detected;

[0014] The second target data streams of the target sub-image data to be detected are input into a preset image feature extraction model, and image feature data extracted by the preset image feature extraction model is determined as the target image feature data of the target image data to be detected.

[0015] In a fourth possible implementation of the third possible implementation of the first aspect, the second target data streams of the target sub-image data to be detected are input into a preset image feature extraction model, and image feature data extracted by the preset image feature extraction model is determined as the target image feature data of the target image data to be detected, including:

[0016] The second target data streams of the target sub-image data to be detected are extracted by the preset image feature extraction model to obtain feature vectors of the target sub-image data to be detected;

[0017] The feature vectors of the target sub-image data to be detected are preprocessed to obtain the target image feature data of the target image data to be detected, and the feature preprocessing at least includes feature fusion processing.

[0018] In a fifth possible implementation of the third or fourth possible implementation of the first aspect, the preset image feature extraction model includes a convolutional neural network model or a deep learning model based on a Transformer structure.

[0019] In a second aspect, the present application provides an image tampering detection device, wherein the device includes:

[0020] The segmentation processing module is configured to perform segmentation processing based on the target image data to be detected to obtain different target sub-image data to be detected, wherein each of the target sub-image data to be detected is local image data with a coverage range that does not overlap with those of other target sub-image data to be detected.

[0021] The feature extraction module is configured to perform target feature extraction on each of the target sub-image data to be detected to obtain target image feature data of the target image data to be detected.

[0022] The tampering recognition module is configured to input the target image feature data into a pre-trained image tampering detection and recognition model to determine a detection result output by the pre-trained image tampering detection and recognition model as a target detection result.

[0023] With reference to the second aspect, in a second possible implementation, the segmentation processing module is specifically configured to:

[0024] The target image data to be detected is segmented based on an original file size of the target image data to be detected to obtain a plurality of target sub-image data to be detected with equal sizes.

[0025] With reference to the second aspect, in a third possible implementation, the feature extraction module is specifically configured to:

[0026] The target sub-image data to be detected is preprocessed to obtain a plurality of first target data streams of the target sub-image data to be detected.

[0027] The first target data streams are spliced according to a preset splicing rule to obtain a second target data stream of the target sub-image data to be detected.

[0028] The second target data streams of the target sub-image data to be detected are input into a preset image feature extraction model, and image feature data extracted by the preset image feature extraction model is determined as target image feature data of the target image data to be detected.

[0029] With reference to the third possible implementation of the second aspect, in a fourth possible implementation, the inputting of the second target data streams of the target sub-image data to be detected into the preset image feature extraction model and the determination of image feature data extracted by the preset image feature extraction model as target image feature data of the target image data to be detected includes:

[0030] The second target data stream of the target sub-image data to be detected is subjected to feature extraction by the preset image feature extraction model to obtain a feature vector of the target sub-image data to be detected.

[0031] The feature vectors of the target subgraph data to be detected are preprocessed to obtain target image feature data of the target image data to be detected, wherein the feature preprocessing at least includes feature fusion processing.

[0032] In combination with the third or fourth possible implementation of the second aspect, in a fifth possible implementation, the preset image feature extraction model includes a convolutional neural network model or a deep learning model based on a Transformer structure.

[0033] In a third aspect, the present application provides an electronic device, wherein the electronic device includes:

[0034] a processor; and

[0035] a memory storing a program,

[0036] The program includes instructions that, when executed by the processor, cause the processor to perform the image tampering detection method of the first aspect.

[0037] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the image tampering detection method of the first aspect.

[0038] Advantages of the present application:

[0039] The embodiments of the present application provide an image tampering detection method, device and electronic device, wherein the method includes: performing segmentation processing based on target image data to be detected to obtain different target subgraph data to be detected, performing target feature extraction on each different target subgraph data to be detected to obtain a plurality of target image feature data of the entire target image data to be detected, inputting the plurality of target image feature data into a pre-trained image tampering detection recognition model, and determining a detection result output by the pre-trained image tampering detection recognition model as a target detection result. By selecting the embodiments of the present application, the traditional compression method for inputting original image data is abandoned, and the input target image data to be detected is segmented into different local image data. This process does not lose the pixel points in the original image data. Therefore, the plurality of target image feature data extracted based on each different local image data is more comprehensive, can more comprehensively express each detail of the target image data to be detected, can make the detection result output by the image tampering detection recognition model more accurate, and thus a more accurate image tampering detection result is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0040] More details, features and advantages of the present application will be disclosed in the following description of the exemplary embodiments in conjunction with the accompanying drawings, in which:

[0041] Figure 1 A possible flowchart of the image tampering detection method provided by the embodiments of the present application;

[0042] Figure 2a A possible schematic diagram of the target image to be detected provided by the embodiments of the present application;

[0043] Figure 2b A possible segmentation processing schematic diagram of the target image data to be detected provided by the embodiments of the present application;

[0044] Figure 2c Another possible segmentation processing schematic diagram of the target image data to be detected provided by the embodiments of the present application;

[0045] Figure 3a A possible flowchart of the target sub-image data feature extraction provided by the embodiments of the present application;

[0046] Figure 3b Another possible flowchart of the target sub-image data feature extraction provided by the embodiments of the present application;

[0047] Figure 4 Another possible flowchart of the target sub-image data feature extraction provided by the embodiments of the present application;

[0048] Figure 5 Another possible flowchart of the image tampering detection method provided by the embodiments of the present application;

[0049] Figure 6 A possible logical structure schematic diagram of the image tampering detection device provided by the embodiments of the present application;

[0050] Figure 7 A possible logical structure schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0051] Embodiments of the present application will be described in more detail by referring to the accompanying drawings. Although certain embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather these embodiments are provided for a more thorough and complete understanding of the present application. It is understood that the drawings and embodiments of the present application are for exemplary purposes only and should not be used to limit the scope of protection of the present application.

[0052] It should be understood that each of the steps recited in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0053] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms shall be construed accordingly. It should be noted that the use of "first", "second", or "third" to describe various embodiments is merely illustrative and is not to be construed as limiting of the order of the described steps or the order of the described elements. The terms "first", "second", and "third" are used to distinguish different elements, and do not imply a particular order or sequence of performing the steps or using the elements.

[0054] It should be noted that the modification of "one" or "more" mentioned in the present application is illustrative and not restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0055] As described in the foregoing background, in order to save computing resources and improve computing efficiency, the existing image tampering detection method usually compresses the to-be-detected image data and then detects image tampering based on the compressed image data. However, many details in the original to-be-detected image data are omitted in the image compression process, which further causes a large error in the detection result obtained by the final image tampering detection, and the detection accuracy is poor.

[0056] In view of this, in a first aspect, the present application provides an image tampering detection method, which can be applied to any electronic device with image tampering detection function, including but not limited to personal mobile terminals, computers or servers, etc.

[0057] As shown in the foregoing background, the image tampering detection method provided by the embodiments of the present application includes the following steps: Figure 1

[0058] S11, performing segmentation processing based on target to-be-detected image data to obtain different target to-be-detected sub-image data;

[0059] Among them, each target to-be-detected sub-image data is local image data with non-overlapping coverage in the target to-be-detected image;

[0060] S12, performing target feature extraction on each target to-be-detected sub-image data to obtain target image feature data of the target to-be-detected image data;

[0061] ​S13, input the target image feature data into the pre-trained image tampering detection and recognition model, and determine the detection result output by the pre-trained image tampering detection and recognition model as the target detection result.

[0062] In the embodiments of the present application, the target image data to be detected is segmented to obtain different target image sub-data, target feature extraction is performed on each different target image sub-data to obtain a plurality of target image feature data of the entire target image data to be detected, and the plurality of target image feature data is input into the pre-trained image tampering detection and recognition model, and the detection result output by the pre-trained image tampering detection and recognition model is determined as the target detection result.

[0063] In the embodiments of the present application, the traditional compression method for input original image data is abandoned, and the input target image data to be detected is segmented into different local image data, and the process does not lose the pixel points in the original image data. In this way, the plurality of target image feature data extracted based on each different local image data is more complete and can more comprehensively express the details of the target image data to be detected, so that the accuracy of the detection result output by the image tampering detection and recognition model is higher, and a more accurate image tampering detection result is obtained.

[0064] The steps S11 to S13 will be described in detail below:

[0065] In the embodiments of the present application, the target image data to be detected is image data to be determined whether there is a tampering trace in the image, which can be image data of any format. The presence of a tampering trace means that the original pixel points in the original image data collected by the image collection device are artificially or mechanically changed, so that the original pixel points in the original image data collected by the image collection device are lost, copied, deleted, modified, etc.

[0066] In one possible embodiment, the target image data to be detected can be image data to be detected for verifying the authenticity of the image to be audited in an actual application scenario, and the target image data to be detected is an electronic image provided by a user. In the embodiments of the present application, the target image data to be detected can be loaded by loading the electronic image transmitted by the user, or obtained by reading the electronic image under a specified storage path. In step S11, the loaded or read target image data to be detected can be directly segmented.

[0067] In another possible embodiment, the target image data to be detected can be training sample data with known image authenticity. This target image data is used to train the image tampering detection model. Based on the difference between the output of the image tampering detection model and the true / false results of the training sample data, the model's parameters are tuned to adjust its output accuracy. In this embodiment, the image tampering detection model includes an image segmentation processing unit, which performs step S11, segmenting the data based on the input training sample data.

[0068] In one embodiment, during step S11, segmentation processing is performed based on the target image data to be detected. This can be understood as dividing the target image data to be detected into N sub-images according to a preset size rule, with each sub-image corresponding to one target image data to be detected. The size of the N sub-images can vary depending on the preset size rule.

[0069] In another implementation, the preset size rule can be determined based on the importance of the content contained in the target image data to be detected. When performing step S11, the target detection image data can be divided into target detection sub-image data of different sizes based on the different importance of the content contained in the target image data to be detected, wherein the size of the target detection sub-image data is positively correlated with the importance of the content in the target detection sub-image data.

[0070] For example, such as Figure 2a As shown, if image tampering detection is required on a medical insurance reimbursement form, the target image data to be detected is a photograph of a medical insurance reimbursement form. In this case, the image may contain the following information: the hospital, patient information, a summary of the patient's medical treatment, details of the medical expenses, and the patient's reimbursement amount. Depending on the specific application, the importance of different areas of the image may vary. For example, the reimbursement amount for the patient's medical treatment may be more important than the hospital. Therefore, the image region containing the patient's reimbursement amount should be segmented larger than the region containing the hospital (i.e., as shown in the image). Figure 2b (As shown).

[0071] Based on this preset size rule, the situation can be determined according to the importance of the content contained in the target image data to be detected. When executing step S11, an image segmentation algorithm can be designed in advance according to the importance of the content contained in the target image data to be detected in the actual application scenario. The image segmentation algorithm will divide the target image data to be detected according to the pixel coordinate range of different contents to obtain different target sub-image data.

[0072] In the embodiments of the present application, the number of segmentation processing times is not limited to one time. For example, the target sub-image data obtained by segmentation in the embodiments of the present application can be subjected to secondary segmentation processing, and the second segmentation processing can be to divide the target sub-image data into several sub-images of the same size.

[0073] In a possible embodiment, when step S11 is performed, the following steps can be implemented:

[0074] Based on the original file size of the target image data to be detected, the target image data to be detected is subjected to segmentation processing to obtain a plurality of target sub-image data of the same size.

[0075] In the embodiments of the present application, the original file of the target image data to be detected can be the size of the memory space occupied by the target image data to be detected. For example, the original file size of the target image data to be detected is 1024 KB (Kilobyte). When step S11 is performed, based on the original file size of the target image data to be detected, the target image data to be detected can be input into a feature extraction network model constructed by a neural network model, and segmented by the size of the feature extraction network model.

[0076] Specifically, by means of neural network technology, a neural network model for feature extraction (hereinafter referred to as a feature extraction network model) is constructed. The size of the neural network model is different, and the corresponding input size is different. Accordingly, when step S11 is performed, the size of the target sub-image data to be detected obtained by segmentation processing of the target image data to be detected is also different.

[0077] In the embodiments of the present application, the size of the target sub-image data to be detected is different from the input size of the feature extraction network model. For example, if the size of the input layer of the feature extraction network model is 224*224 network layer, when step S11 is performed, essentially a 224*224 sliding window is used to slide in the target image data to be detected at a specified step size to obtain a plurality of target sub-image data of the same size.

[0078] For example, as shown in FIG. 2, the target image data to be detected is segmented into a plurality of target sub-image data of the same size by means of the feature extraction network model. Figure 2cAs shown, based on the original file size of the target to-be-detected image data required for image tampering detection, a feature extraction network model is designed, and then the network input size of the feature extraction network model is used to slide in the original target to-be-detected image data at a specified step size to obtain four target to-be-detected sub-image data of the same size. In the embodiment of the present application, the segmentation processing function can be a part of the implementable function of the image tampering detection model, and is implemented by a sub-model (segmentation processing sub-model) of the large model of the image tampering detection model.

[0079] According to the embodiment of the present application, the target to-be-detected sub-image data of the same size can be obtained from the original target to-be-detected image data directly according to the input size of the feature extraction model without compressing the original target to-be-detected image data. In this way, the details in the image are not lost. Compared with the traditional compression of the original image or the extracted features, which leads to the loss of the details of the tampered part, the embodiment of the present application can perform tampering detection on the target to-be-detected image data without losing the details of the original target to-be-detected image data, which can effectively guarantee the accuracy of the image tampering detection result.

[0080] Based on the target to-be-detected sub-image data obtained in step S11, in one possible embodiment, the target to-be-detected sub-image data can be processed as follows: Figure 3a As shown, the above step S12 can also be implemented by the following steps:

[0081] S121, image preprocessing is performed on the target to-be-detected sub-image data to obtain a plurality of first target data streams of the target to-be-detected sub-image data;

[0082] S122, according to a preset splicing rule, the first target data streams are spliced to obtain a second target data stream of the target to-be-detected sub-image data;

[0083] S123, the second target data stream of each target to-be-detected sub-image data is input into a preset image feature extraction model, and the image feature data extracted by the preset image feature extraction model is determined as the target image feature data of the target to-be-detected image data.

[0084] Because the traditional image tampering detection method is based on compressed images, and interpolation sampling is performed in the image compression process, and because the edge information of the tampered image is very little, the tampered information that can be preserved after interpolation compression will be even less, thereby affecting the accuracy of the image tampering detection. Therefore, in the embodiment of the present application, the image preprocessing on the target to-be-detected sub-image data in step S121 can include edge detection on the target to-be-detected sub-image data. Specifically, the target to-be-detected sub-image data can be processed as follows: Figure 3bAs shown, the edge operator is used to perform edge detection on the target subgraph data to be detected, and output a first target data stream. The edge operator can include a Canny operator, a Laplace operator, a Sobel operator, etc.

[0085] In the embodiments of the present application, edge detection refers to first-order or second-order derivation of the gray scale value or color of the edge part of the target subgraph data to be detected, and then determining the degree of change of the gray scale value or color of the edge part of the pixel point, so as to help extract the contour information of the edge. If the tampered part is located in the edge part, this method can enhance the information of the edge of the tampered part, and thus improve the accuracy of extracting the detailed features of the tampered part.

[0086] In step S121, the image preprocessing on the target subgraph data to be detected can also include data transformation on the target subgraph data to be detected. In the embodiments of the present application, data transformation refers to signal transformation on each digital signal in the target subgraph data to be detected, including fast Fourier transform (FFT), Gaussian filtering, and inverse fast Fourier transform (IFFT). In one possible embodiment, the data transformation can be performed as follows: Figure 3b As shown, the image preprocessing on the target subgraph data to be detected can include FFT transformation on the target subgraph data to be detected, high-pass filtering on the signal obtained by the FFT transformation, IFFT transformation, and output of a first target data stream. In this way, the noise in the image signal in the target subgraph data to be detected can be removed, which helps to improve the accuracy of tamper detection.

[0087] In step S121, the image preprocessing on the target subgraph data to be detected can also include error level analysis (ELA) on the target subgraph data to be detected. When an image is edited or saved again, the compression algorithm will re-encode the image, resulting in different error levels in the edited area and other areas. In the embodiments of the present application, the ELA detection on the target subgraph data to be detected can utilize the feature that the image is re-encoded when it is edited or saved again, and detect the editing traces in the image by repeatedly compressing and decompressing the image, and then calculating the error level between pixels.

[0088] As shown, Figure 3bAs shown, the ELA detection is performed on the target subgraph data to be detected, and another first target data stream can be output. In this way, the image tampering detection result can be given for the target subgraph data to be detected, and compared with the image tampering detection performed on the entire target subgraph data to be detected, the local subgraph of the target subgraph data to be detected can be more accurately determined whether tampering occurs, which helps to improve the accuracy of image tampering detection.

[0089] In the embodiment of the present application, as shown in the figure Figure 3b As shown, for the same target subgraph data to be detected, edge operator detection is performed to output a first target data stream, FFT, high-pass filtering and IFFT transformation are performed to output a first target data stream, and ELA detection is performed to output a first target data stream. The first target data stream obtained is different due to different image preprocessing methods, and the more image preprocessing methods, the more first target data streams obtained.

[0090] In step S122, a plurality of first target data streams are spliced to obtain a second target data stream of the target subgraph data to be detected. In the embodiment of the present application, the data stream splicing can be based on the input size of the target subgraph data to be detected to splice the input size of the image tampering detection model.

[0091] For example, assuming that the target subgraph data A1 is a subgraph of the target image data A, and the input size of A1 is C*H*W. Wherein, C represents the pixel channel value, such as A1 is RGB three channels, then C=3. H represents the height of the subgraph A1, and W represents the width of the subgraph A1.

[0092] Wherein, when the subgraph A1 is calculated by the edge operator, it is first converted into a gray image (the channel number is changed from 3 to 1), and then the size of the output after the edge detection by the edge operator is 1*H*W. The size of the output after the FFT, high-pass filtering and IFFT is 3*H*W, and the size of the output after the ELA is 3*H*W. The splicing of the three first target data streams is to splice the original channel number, that is, the size of the second target data stream obtained by splicing is 7*H*W. As can be seen, the embodiment of the present application does not lose image data, and more detailed data is extracted from the target image data to be detected, which helps to improve the accuracy of image tampering detection.

[0093] In the embodiment of the present application, when step S123 is performed, the second target data stream obtained by splicing each target sub-image data of the target image data to be detected is input into the preset image feature extraction model, wherein the second target data stream is the spliced data stream corresponding to each target sub-image data to be detected. For example, if the target image data A to be detected is divided into four target sub-image data (sub-image data A1, sub-image data A2, sub-image data A3 and sub-image data A4) for processing, and image preprocessing is performed on each sub-image data, a plurality of first target data streams are obtained.

[0094] For example, image preprocessing is performed on the sub-image data A1 to obtain the first target data stream a11, the first target data stream a12 and the first target data stream a13, and the first target data streams of the sub-image data A1 are spliced to obtain the second target data stream A_1 of the sub-image data A1. Image preprocessing is performed on the sub-image data A2 to obtain the first target data stream a21, the first target data stream a22 and the first target data stream a23, and the first target data streams of the sub-image data A2 are spliced to obtain the second target data stream A_2 of the sub-image data A2. Similarly, the second target data streams of each target image data to be detected can be obtained: the second target data stream A_1, the second target data stream A_2, the second target data stream A_3 and the second target data stream A_4.

[0095] Then, when step S123 is performed, the second target data stream obtained by splicing each target sub-image data of the target image data to be detected is input into the preset image feature extraction model, that is, the second target data stream A_1, the second target data stream A_2, the second target data stream A_3 and the second target data stream A_4 are input into the preset image feature extraction model to extract image feature data.

[0096] By selecting the embodiment of the present application, the image preprocessing method can be used to extract features of different tampering types based on original image cropping and segmentation processing, without changing the size of the target image data to be detected to ensure that image details are not lost and tampering features are not lost, which helps to obtain more tampering detail data in the target sub-image data to be detected, and further helps to improve the accuracy of the output result of the subsequent image tampering detection model.

[0097] In one possible embodiment, the above step S123 can further include:

[0098] S1231, for the second target data stream of the target sub-image data to be detected, performing feature extraction by the preset image feature extraction model to obtain a feature vector of the target sub-image data to be detected;

[0099] S1232, feature preprocessing is performed based on the feature vectors of each target subgraph data to be detected to obtain target image feature data of the target subgraph data to be detected.

[0100] In the embodiments of the present application, the feature preprocessing specifically includes feature extraction, normalization processing, and feature splicing. The feature fusion processing includes normalization processing and feature splicing.

[0101] In a possible embodiment, the preset image feature extraction model includes a CNN convolutional neural network model or a deep learning model based on a Transformer structure. The deep learning model based on the Transformer structure can include a swin-transformer feature extraction model. The structure of the preset feature extraction model can be as shown in Figure 4 When step S1231 is performed, the target subgraph data to be detected is input to the feature extraction part for one-time feature extraction, where the input of the feature extraction part is assumed to be C*H*W, and the final output is a feature map of N*H*W.

[0102] Then the N*H*W feature map is respectively subjected to the average pooling part and the maximum pooling part, and two feature vectors of feature=B*N are respectively output. Based on this, when step S1232 is performed, the correlation between the two feature vectors of B*N is calculated, that is, output=feature*feature T At this time, an output vector of size B*N*N can be obtained. Then, after normalization processing (compressing the feature values in the feature vector to 0-1) and feature splicing processing, feature fusion can be achieved, and an output feature vector of B*(2N 2 +2N) is obtained.

[0103] In this way, by selecting the embodiments of the present application, different sizes of feature information are obtained by average pooling and maximum pooling, respectively, and then the similarity between different features is calculated by correlation calculation, and further feature splicing is performed, so that a feature vector containing a large number of image tampering details can be obtained, which helps to improve the output result accuracy of the subsequent image tampering detection and recognition model.

[0104] Based on the target image feature data obtained in the step S12, a large amount of tampering detail information is actually obtained from each target to-be-detected sub-image data in the target to-be-detected image data, and the tampering details contained are more comprehensive. Then, in a possible embodiment, when the step S13 is performed, the target image feature data is directly input into the pre-trained image tampering detection and recognition model, and the detection result output by the image tampering detection and recognition model is determined as the target detection result.

[0105] In the embodiments of the present application, the pre-trained image tampering detection and recognition model is an image tampering detection and recognition model trained based on the feature vectors extracted by the feature extraction manner designed in the step S12 of the present application. Alternatively, in another possible embodiment, the pre-trained image tampering detection and recognition model is a neural network model constructed based on an image tampering detection algorithm, and the difference between the pre-trained image tampering detection and recognition model and a traditional image tampering detection and recognition model is only that the input feature vectors are different. In a possible embodiment, the pre-trained image tampering detection and recognition model is a sub-model of the target image tampering detection and recognition model, and the function of the sub-model is mainly used for image tampering detection on the feature vectors input into the sub-model, and the feature vectors are obtained by other sub-models of the target image tampering detection and recognition model based on the target to-be-detected image data performing the step S12.

[0106] For example, in a possible embodiment, the target image tampering detection and recognition model can be as shown in the following figure. Figure 5 As shown in the figure, the target to-be-detected image data is obtained by the input module of the target image tampering detection and recognition model, and then the step S11 is performed by the segmentation processing module of the target image tampering detection and recognition model to segment the target to-be-detected image data to obtain different target to-be-detected sub-image data. Then, each target to-be-detected sub-image data is input into the preset image feature extraction model, and the steps S12 are performed by the preset image feature extraction model to perform image preprocessing and image feature extraction on the target to-be-detected image data, to obtain each feature vector X1-Xn of the target to-be-detected sub-image data, and then each feature vector is fused by the Concat function to obtain the target image feature data X of the target to-be-detected image data. N Finally, the target image feature data is input into the classification module, and the step S13 is performed to obtain the classification result of whether there is tampering, such as 0 for no tampering and 1 for tampering.

[0107] Specifically, after the segmentation processing is completed, each subgraph data will generate a 7-channel input data through the image preprocessing module, and then the 7-channel input data is subsequently sent to the trained convolutional neural network model or swin-Transformer model, and for n subgraphs, n sublayer high-level sample expression (i.e., feature vectors of B*(2N 2 +2N) are obtained. Subsequently, the final classification is completed by sending the n subgraph high-level sample expression to the machine learning classifier.

[0108] In a possible embodiment, the classification result of each target subgraph data to be detected can be set as a threshold in the order of position by voting to determine whether the target image data to be detected is an output tampered image. Specifically, a full connection layer is added to the target image tampering detection and recognition model, the OUT output is calculated as a probability of a tampered image by the full connection layer, and the maximum prediction probability is taken as the prediction probability of the whole image.

[0109] For example, the prediction probability of target subgraph data A1 is 0.1, the prediction probability of target subgraph data A2 is 0.4, the prediction probability of target subgraph data A3 is 0.6, and the prediction probability of target subgraph data A4 is 0.8. Therefore, 0.8 is taken as the tampering probability of the target image data to be detected.

[0110] By segmenting the target image data to be detected into several target subgraph data to be detected, and then obtaining a feature vector containing a large number of image details by means of each target subgraph data to be detected, and performing image tampering detection by means of the feature vector, the present application can more comprehensively identify whether there is tampering trace in the target image data to be detected, and further solve the problem of low tampering recognition accuracy caused by loss of image details in the prior art.

[0111] In a second aspect, the present application provides an image tampering detection device, as shown in Figure 6 The device 600 comprises:

[0112] The segmentation processing module 601 is configured to perform segmentation processing based on the target image data to be detected to obtain different target subgraph data to be detected, wherein each target subgraph data to be detected is local image data in the target image data to be detected, and the coverage ranges of the target subgraph data to be detected do not overlap with each other.

[0113] The feature extraction module 602 is configured to perform target feature extraction on each target sub-image data to obtain target image feature data of the target image data.

[0114] The tampering recognition module 603 is configured to input the target image feature data into a pre-trained image tampering detection and recognition model to determine a detection result output by the pre-trained image tampering detection and recognition model as a target detection result.

[0115] With reference to the second aspect, in a second possible implementation, the segmentation processing module 601 is specifically configured to:

[0116] The target image data is segmented based on an original file size of the target image data to obtain a plurality of target sub-image data of equal size.

[0117] With reference to the second aspect, in a third possible implementation, the feature extraction module 602 is specifically configured to:

[0118] The target sub-image data is preprocessed to obtain a plurality of first target data streams of the target sub-image data.

[0119] The first target data streams are spliced according to a preset splicing rule to obtain a second target data stream of the target sub-image data.

[0120] The second target data stream of each target sub-image data is input into a preset image feature extraction model, and image feature data extracted by the preset image feature extraction model is determined as target image feature data of the target image data.

[0121] With reference to the third possible implementation of the second aspect, in a fourth possible implementation, the second target data stream of each target sub-image data is input into a preset image feature extraction model, and image feature data extracted by the preset image feature extraction model is determined as target image feature data of the target image data, including:

[0122] The second target data stream of the target sub-image data is subjected to feature extraction by the preset image feature extraction model to obtain a feature vector of the target sub-image data.

[0123] The target image feature data of the target image data is obtained based on the feature vector of each target sub-image data, and the feature preprocessing at least includes feature fusion processing.

[0124] In a fifth possible implementation of the third or fourth possible implementation of the second aspect, the preset image feature extraction model 602 comprises a convolutional neural network model, or a deep learning model based on a Transformer structure.

[0125] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0126] The exemplary embodiments of the present application also provide an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication. The memory stores a computer program capable of being executed by the at least one processor, and the computer program, when executed by the at least one processor, is configured to cause the electronic device to perform the method according to the embodiments of the present application.

[0127] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to the embodiments of the present application.

[0128] The exemplary embodiments of the present application also provide a computer program product comprising a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to the embodiments of the present application.

[0129] Reference Figure 7 A block diagram of the structure of an electronic device 700 that can be a server or a client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent a variety of digital electronic computer devices such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computer devices. The electronic device can also represent a variety of mobile devices such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0130] As Figure 7As shown, the electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0131] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 can be any type of device that can input information to the electronic device 700, and can receive inputted digital or character information, as well as generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 707 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 704 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0132] The computing unit 701 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above. For example, in some embodiments, the aforementioned image tampering detection method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the aforementioned image tampering detection method by any other appropriate means, such as by means of firmware.

[0133] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0138] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. An image tampering detection method, characterized by, The method comprises: dividing target image data to be detected into first target sub-image data of different sizes based on the importance of the content contained in the target image data to be detected; performing secondary segmentation processing on the first target sub-image data to divide the first target sub-image data into second target sub-image data of the same size, wherein each second target sub-image data is local image data in the target image to be detected, and the coverage ranges of the second target sub-image data do not overlap with each other; performing target feature extraction on each second target sub-image data to obtain target image feature data of the target image data to be detected, comprising: performing edge detection on the second target sub-image data to output a first target data stream; performing fast Fourier transform, Gaussian filtering, and inverse Fourier transform on the second target sub-image data to output a first target data stream; performing edit trace detection on the second target sub-image data to output another first target data stream; splicing each first target data stream according to a preset splicing rule to obtain a second target data stream of the second target sub-image data; inputting the second target data stream of each second target sub-image data into a preset image feature extraction model, and determining image feature data extracted by the preset image feature extraction model as target image feature data of the target image data to be detected; inputting the target image feature data into a pre-trained image tampering detection and recognition model, and determining a detection result output by the pre-trained image tampering detection and recognition model as a target detection result.

2. The method of claim 1, wherein, The method comprises: performing feature extraction on the second target data stream of the second target sub-image data by the preset image feature extraction model to obtain a feature vector of the second target sub-image data; performing feature preprocessing based on the feature vectors of each second target sub-image data to obtain target image feature data of the target image data to be detected, wherein the feature preprocessing at least comprises feature fusion processing.

3. The method of claim 1, wherein, The preset image feature extraction model comprises a convolutional neural network model or a deep learning model based on a Transformer structure.

4. An image tampering detection apparatus characterized by comprising: The device comprises: a segmentation processing module configured to divide target image data to be detected into first target sub-image data of different sizes based on the importance of the content contained in the target image data to be detected; performing secondary segmentation processing on the first target sub-image data to divide the first target sub-image data into second target sub-image data of the same size, wherein each second target sub-image data is local image data in the target image to be detected, and the coverage ranges of the second target sub-image data do not overlap with each other; The feature extraction module is configured to perform target feature extraction on each of the second target sub-image data to obtain target image feature data of the target image data, and includes: performing edge detection on the second target sub-image data to output a first target data stream; performing fast Fourier transform, Gaussian filtering, and inverse Fourier transform on the second target sub-image data to output a first target data stream; performing edit trace detection on the second target sub-image data to output another first target data stream; splicing each of the first target data streams according to a preset splicing rule to obtain a second target data stream of the second target sub-image data; inputting the second target data stream of each of the second target sub-image data into a preset image feature extraction model, and determining image feature data extracted by the preset image feature extraction model as the target image feature data of the target image data; The tamper detection module is configured to input the target image feature data into a pre-trained image tamper detection and recognition model, and determine a detection result output by the pre-trained image tamper detection and recognition model as a target detection result.

5. The apparatus of claim 4, wherein, The feature extraction module is specifically configured to: The feature extraction module is specifically configured to: performing feature extraction on the second target data stream of the second target sub-image data by the preset image feature extraction model to obtain a feature vector of the target sub-image data; performing feature preprocessing based on the feature vectors of each of the second target sub-image data to obtain the target image feature data of the target image data, wherein the feature preprocessing at least includes feature fusion processing; The preset image feature extraction model includes a convolutional neural network model or a deep learning model based on a Transformer structure.

6. An electronic device, comprising: The electronic device includes: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-3.

7. A non-transitory computer readable storage medium having computer instructions stored therein, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3.

Images