A real scene tampered image detection method, system, device and storage medium

Abstract

The application discloses a real scene tampered image detection method, system, device and storage medium, belongs to the technical field of image detection, solves the technical problem of low efficiency of artificial detection of tampered images, and the detection method comprises the steps of: cutting an initial training data set into pictures of a set size to form a training data set, marking the pictures after tampering in the training data set as X, and marking the mask graph of the tampering area as Y; inputting the training data set into Unet, Unet++ and LinkNet initial models respectively for training to obtain Unet models, Unet++ models and LinkNet models respectively; cutting a picture to be predicted into pictures of a set size and remembering the slice position; inputting the cut pictures into the Unet models, the Unet++ models and the LinkNet models respectively for inference, fusing three inference results to obtain a prediction result; slicing the prediction result, reassembling the prediction result back to the original picture size according to the remembered slice position, and obtaining a final detected tampering area prediction result.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and more specifically, to a method, system, device, and storage medium for detecting tampered images of real scenes. Background Technology

[0002] Currently, relying heavily on manual inspection and judgment of tampered images is an inefficient method.

[0003] Patent CN105844643B provides an image tampering detection method, which includes: Step 1, generating a brightness map and a local brightness change map of the image, and performing the following operations on the brightness map and the local brightness change map respectively: dividing them into M×N regions, and then performing block processing on the brightness map and the local brightness change map to obtain multiple overlapping blocks; Step 2, calculating the invariant moments for each overlapping block of the brightness map and the local brightness change map; Step 3, comparing the overlapping blocks through the invariant moment feature vectors of the brightness map and the local brightness change map.

[0004] The above detection method requires calculating the invariant moments for each overlapping block of the brightness map and the local brightness change map, which involves a large amount of computation. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art. The purpose of the present invention is to provide a method for detecting tampered images in real-world scenarios.

[0006] The second objective of this invention is to provide a real-world image tampering detection system.

[0007] The third objective of this invention is to provide a computer device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] To achieve the first objective mentioned above, the present invention provides a method for detecting tampered images in real-world scenes, comprising the following steps:

[0010] Step S1. Obtain publicly available tampered image datasets and create a self-made tampered dataset through manual or automated means. Combine the publicly available tampered image datasets and the self-made tampered datasets as the initial training dataset.

[0011] Step S2. Cut the initial training dataset from step S1 into images of a set size, remove the unaltered images from the cut dataset to form a training dataset, label the altered images in the training dataset as X, and label the mask image of the altered region as Y;

[0012] Step S3. Establish three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 is used for the backbone of all models, and the loss function is binary cross-entropy loss.

[0013] Step S4. Input the training dataset from step S2 into the initial Unet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet model is obtained.

[0014] Step S5. Input the training dataset from step S2 into the initial Unet++ semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet++ model is obtained.

[0015] Step S6. Input the training dataset from step S2 into the initial LinkNet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained LinkNet model is obtained.

[0016] Step S7. Cut the image to be predicted into images of a set size and remember the slice positions;

[0017] Step S8. Input the image cut in step S7 into the Unet model in step S4, the Unet++ model in step S5, and the LinkNet model in step S6 respectively for inference to obtain three inference results. Fuse the three inference results to obtain the prediction result.

[0018] Step S9. Slice the prediction results from step S8 and reassemble them back to the original image size according to the slice positions in step S7 to obtain the final predicted results of the detected tampered areas.

[0019] As a further improvement, the publicly disclosed tampered image dataset includes at least one of the following: Nist16 tampered dataset, COVERAGE tampered dataset, MICC_F600 tampered dataset, Columbia tampered dataset, and Alibaba Tianchi tampering algorithm competition dataset.

[0020] Furthermore, in step S1, manual creation includes at least one of splicing, copying and pasting, removing objects, text replacement, stamp replacement, and forging signatures; automated creation includes at least one of random erasure, random splicing, and random copying and pasting.

[0021] Furthermore, the set dimensions are 512 for both width and height.

[0022] Furthermore, in step S2, during the cutting process, the cutting positions of the tampered image and the Mask image are aligned, and the intersection edges of each image block retain 10% overlap. After the cutting is completed, based on the tampered area in the Mask image, Mask images with a tampered area of ​​less than or equal to 5% and their corresponding tampered images are randomly deleted. After the random deletion, the ratio of data with a tampered area of ​​less than or equal to 5% and data with a tampered area of ​​greater than 5% is maintained at 1:6.

[0023] Furthermore, in step S3, the initial parameters of the three semantic segmentation initial models are obtained by using ImageNet pre-trained model parameters.

[0024] Further, in step S4, step S5, or step S6, during the training process, X is subjected to at least one of the following processing: random flipping, translation, scaling, histogram equalization, brightness, gamma, sharpening, blurring, contrast, hue, saturation, JPEG compression, and normalization preprocessing; and Y is subjected to Label Smoothing preprocessing with a parameter of 0.1.

[0025] The training strategy is as follows: the K-fold strategy is used for splitting the training and validation datasets, where K=5, and the models trained by multiple folds are fused using the SWA method to obtain a single training model; the AdamW optimizer is used, and the learning rate is adjusted using a cosine annealing learning rate adjustment strategy. The initial learning rate, total number of training steps, warm-up steps, batch size, Early Stop, and Dropout parameters are optimized and adjusted based on the loss and accuracy feedback during single-fold model training.

[0026] To achieve the second objective mentioned above, the present invention provides a real-scene tampering image detection system, comprising:

[0027] The acquisition module is used to acquire publicly available tampered image datasets and create a self-made tampered dataset through manual or automated means. The publicly available tampered image datasets and the self-made tampered datasets are then combined as the initial training dataset.

[0028] The cutting module is used to cut the initial training dataset into images of a set size, remove unaltered images from the cut dataset to form a training dataset, and label the tampered images in the training dataset as X and the mask image that marks the tampered area as Y.

[0029] The training module is used to build three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 is used for the backbone of all models, and the loss function is binary cross-entropy loss.

[0030] And for inputting the training dataset into the initial Unet, Unet++, and LinkNet models respectively to train and obtain the Unet model, Unet++ model, and LinkNet model respectively;

[0031] The detection module is used to cut the image to be predicted into images of a set size and remember the slice positions. The cut images are then input into the Unet model, Unet++ model, and LinkNet model respectively for inference to obtain three inference results. The three inference results are fused to obtain the prediction result. Finally, the prediction result is sliced ​​and reassembled back to the original image size according to the remembered slice positions to obtain the final prediction result of the detected tampered area.

[0032] To achieve the third objective mentioned above, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method for detecting tampered images in a real scene.

[0033] To achieve the fourth objective mentioned above, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-mentioned method for detecting tampered images in a real-world scenario.

[0034] Beneficial effects

[0035] Compared with the prior art, the advantages of this invention are as follows:

[0036] This invention, based on the idea of ​​semantic segmentation model, trains multiple semantic segmentation models by combining image cutting and data augmentation, and uses model fusion method during inference detection to obtain more accurate tampering image detection results. This invention can detect tampering of image data in real-world scenarios with relatively accurate results, and has the advantages of high efficiency and low computational cost. Attached Figure Description

[0037] Figure 1 This is a flowchart of the detection method of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to specific embodiments shown in the accompanying drawings.

[0039] See Figure 1 A method for detecting image tampering in real-world scenarios includes the following steps:

[0040] Step S1. Obtain publicly available tampered image datasets. These datasets include at least one of the following: Nist16 tampered dataset, Coverage tampered dataset, MICC_F600 tampered dataset, Columbia tampered dataset, and Alibaba Tianchi tampering algorithm competition dataset. Publicly available tampered image datasets can also be sets of images collected internally by the enterprise using cameras, mobile phones, or computers, including images of notifications, announcements, certificates, documents, physical objects, and scene photos. For example, the Nist16 tampering dataset contains 1,000 tampered images and corresponding tampered region mask images for training; the COVERAGE tampering dataset contains 100 tampered images and corresponding tampered region mask images for training; the MICC_F600 tampering dataset contains 160 tampered images and corresponding tampered region mask images for training; the Columbia tampering dataset contains 183 high-resolution tampered images and corresponding tampered region mask images for training; the Alibaba Tianchi tampering algorithm competition dataset contains 11,600 high-resolution tampered images and corresponding tampered region mask images for training; and an enterprise collected 10,000 images, of which 2,000 were manually tampered with and had their tampered region mask images created for training, and 8,000 were tampered with and had their tampered region mask images created using automated scripts, totaling 10,000 self-made images and tampered region mask images for training.

[0041] Publicly available tampered image datasets are transformed into self-made tampered datasets through manual or automated means. Manual transformation includes at least one of splicing, copying and pasting, removing objects, replacing text, replacing seals, and forging signatures; automated transformation includes at least one of random erasure, random splicing, and random copying and pasting.

[0042] We combined publicly available tampered image datasets and self-made tampered datasets as the initial training dataset.

[0043] Step S2. Cut the initial training dataset from Step S1 into images of a set size, such as 512 pixels wide and 512 pixels high. Remove the unaltered images from the cut dataset to form the training dataset. Label the altered images in the training dataset as X, and label the altered regions as Y.

[0044] Specifically, during the cutting process, the cutting positions of the tampered image and the Mask image are aligned, and the intersection edges of each image block retain 10% overlap. After the cutting is completed, based on the tampered area in the Mask image, Mask images with tampered areas less than or equal to 5% and their corresponding tampered images are randomly deleted. After the random deletion, the ratio of data with tampered areas less than or equal to 5% and data with tampered areas greater than 5% is maintained at 1:6, forming the training dataset.

[0045] Step S3. Establish three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 is used for the backbone of all three models. The initial parameters of the three initial semantic segmentation models are the parameters of the ImageNet pre-trained model, and the loss function is binary cross-entropy loss.

[0046] Step S4. Input the training dataset from step S2 into the initial Unet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet model is obtained.

[0047] Step S5. Input the training dataset from step S2 into the initial Unet++ semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet++ model is obtained.

[0048] Step S6. Input the training dataset from step S2 into the initial LinkNet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained LinkNet model is obtained.

[0049] In steps S4, S5, and S6, during the training process, X is subjected to at least one of the following processing: random flipping, translation, scaling, histogram equalization, brightness adjustment, gamma adjustment, sharpening, blurring, contrast adjustment, hue adjustment, saturation adjustment, and JPEG compression, as well as normalization preprocessing. Y is subjected to Label Smoothing preprocessing with a parameter of 0.1.

[0050] The training strategy is as follows: the K-fold strategy is used for splitting the training and validation datasets, where K=5, and the models trained by multiple folds are fused using the SWA method to obtain a single training model; the AdamW optimizer is used, and the learning rate is adjusted using a cosine annealing learning rate adjustment strategy. The initial learning rate, total number of training steps, warm-up steps, batch size, Early Stop, and Dropout parameters are optimized and adjusted based on the loss and accuracy feedback during single-fold model training.

[0051] Step S7. Cut the image to be predicted into images of a set size (width and height are both 512), and remember the slice positions.

[0052] Step S8. Input the image segmented in step S7 into the Unet model in step S4, the Unet++ model in step S5, and the LinkNet model in step S6 respectively to obtain three inference results. Merge the three inference results to obtain the prediction result. Specifically, the average value of the three inference results is obtained by adding them together and dividing by the number of models.

[0053] Step S9. Slice the prediction results from step S8 to obtain slice results with a width and height of 512, and reassemble them back to the original image size according to the slice positions in step S7 to obtain the final predicted results of the detected tampered areas.

[0054] A real-world scene tampering image detection system, comprising:

[0055] The acquisition module is used to acquire publicly available tampered image datasets and create a self-made tampered dataset through manual or automated means. The publicly available tampered image datasets and the self-made tampered datasets are then combined as the initial training dataset.

[0056] The cutting module is used to cut the initial training dataset into images of a set size, remove the untampered images in the cut dataset to form the training dataset, and label the tampered images in the training dataset as X, and the mask image that marks the tampered area as Y;

[0057] The training module is used to build three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 is used for the backbone of all models, and the loss function is binary cross-entropy loss.

[0058] And for inputting the training dataset into the initial Unet, Unet++, and LinkNet models respectively to train them and obtain the Unet model, Unet++ model, and LinkNet model respectively;

[0059] The detection module is used to cut the image to be predicted into images of a set size and remember the slice positions. The cut images are then input into the Unet model, Unet++ model, and LinkNet model respectively for inference to obtain three inference results. The three inference results are fused to obtain the prediction result. Finally, the prediction result is sliced ​​and reassembled back to the original image size according to the remembered slice positions to obtain the final prediction result of the detected tampered area.

[0060] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method for detecting tampered images in a real-world scenario.

[0061] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for detecting tampered images in a real-world scenario.

[0062] This invention, based on the idea of ​​semantic segmentation model, trains multiple semantic segmentation models by combining image cutting and data augmentation, and uses model fusion method during inference detection to obtain more accurate tampering image detection results. It can detect tampering of image data in real-world scenarios with relatively high accuracy.

[0063] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A method for detecting tampered images in real-world scenes, characterized in that, Includes the following steps: Step S1. Obtain publicly available tampered image datasets and create a self-made tampered dataset through manual or automated means. Combine the publicly available tampered image datasets and the self-made tampered datasets as the initial training dataset. Step S2. Cut the initial training dataset from step S1 into images of a set size, remove the unaltered images from the cut dataset to form a training dataset, label the altered images in the training dataset as X, and label the mask image of the altered region as Y; Step S3. Establish three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 is used for the backbone of all models, and the loss function is binary cross-entropy loss. Step S4. Input the training dataset from step S2 into the initial Unet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet model is obtained. Step S5. Input the training dataset from step S2 into the initial Unet++ semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained Unet++ model is obtained. Step S6. Input the training dataset from step S2 into the initial LinkNet semantic segmentation model from step S3 for training. Configure the training strategy. When inputting the training dataset into the model, preprocess the datasets X and Y respectively. After convergence, the trained LinkNet model is obtained. Step S7. Cut the image to be predicted into images of a set size and remember the slice positions; Step S8. Input the image cut in step S7 into the Unet model in step S4, the Unet++ model in step S5, and the LinkNet model in step S6 respectively for inference to obtain three inference results. Fuse the three inference results to obtain the prediction result. Step S9. Slice the prediction results from step S8 and reassemble them back to the original image size according to the slice positions in step S7 to obtain the final predicted results of the detected tampered areas.

2. The method for detecting tampered images in a real-world scene according to claim 1, characterized in that, The publicly disclosed image manipulation datasets include at least one of the following: Nist16 manipulation dataset, COVERAGE manipulation dataset, MICC_F600 manipulation dataset, Columbia manipulation dataset, and Alibaba Tianchi manipulation algorithm competition dataset.

3. The method for detecting tampered images in a real-world scene according to claim 1, characterized in that, In step S1, manual creation includes at least one of splicing, copying and pasting, removing objects, text replacement, stamp replacement, and forging signatures; automated creation includes at least one of random erasure, random splicing, and random copying and pasting.

4. The method for detecting tampered images in a real-world scene according to claim 1, characterized in that, The specified dimensions are 512 for both width and height.

5. The method for detecting tampered images in a real-world scene according to claim 1, characterized in that, In step S2, the cutting positions of the tampered image and the Mask image are aligned during the cutting process, and the intersection edges of each image block retain 10% overlap. After the cutting is completed, based on the tampered area in the Mask image, Mask images with a tampered area of ​​less than or equal to 5% and their corresponding tampered images are randomly deleted. After the random deletion, the ratio of data with a tampered area of ​​less than or equal to 5% and data with a tampered area of ​​greater than 5% is maintained at 1:

6.

6. The method for detecting tampered images in a real-world scene according to claim 1, characterized in that, In step S3, the initial parameters of the three semantic segmentation initial models are the parameters of the ImageNet pre-trained model.

7. The method for detecting tampered images in a real scene according to claim 1, characterized in that, In step S4, step S5, or step S6, during the training process, X is subjected to at least one of the following processing: random flipping, translation, scaling, histogram equalization, brightness, gamma, sharpening, blurring, contrast, hue, saturation, and JPEG compression, as well as normalization preprocessing; and Y is subjected to LabelSmoothing preprocessing with a parameter of 0.

1. The training strategy is as follows: the K-fold strategy is used for splitting the training and validation datasets, where K=5, and the models trained by multiple folds are fused using the SWA method to obtain a single training model; the AdamW optimizer is used, and the learning rate is adjusted using a cosine annealing learning rate adjustment strategy. The initial learning rate, total number of training steps, warm-up steps, batch size, Early Stop, and Dropout parameters are optimized and adjusted based on the loss and accuracy feedback during single-fold model training.

8. A real-scene tampering image detection system, characterized in that, include: The acquisition module is used to acquire publicly available tampered image datasets and create a self-made tampered dataset through manual or automated means. The publicly available tampered image datasets and the self-made tampered datasets are then combined as the initial training dataset. The cutting module is used to cut the initial training dataset into images of a set size, remove unaltered images from the cut dataset to form a training dataset, and label the tampered images in the training dataset as X and the mask image that marks the tampered area as Y. The training module is used to build three initial semantic segmentation models: Unet, Unet++, and LinkNet. Resnest101 was used for all backbones, and the loss function for all was binary cross-entropy loss. And for inputting the training dataset into the initial Unet, Unet++, and LinkNet models respectively to train and obtain the Unet model, Unet++ model, and LinkNet model respectively; The detection module is used to cut the image to be predicted into images of a set size and remember the slice positions. The cut images are then input into the Unet model, Unet++ model, and LinkNet model respectively for inference to obtain three inference results. The three inference results are fused to obtain the prediction result. Finally, the prediction result is sliced ​​and reassembled back to the original image size according to the remembered slice positions to obtain the final prediction result of the detected tampered area.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a real-scene tampering image detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a real-scene tampering image detection method according to any one of claims 1 to 7.

Images