Image tampering region automatic labeling method

Abstract

The application discloses an image tampering region automatic labeling method, comprising the following steps: obtaining a tampered image and its corresponding original image to form an image pair; constructing and training a binary classifier by using unlabelled images through self-supervised learning, and using the binary classifier to classify the image pair composed of the tampered image and the original image; constructing and training a tampering region labeling model based on semantic denoising image difference; constructing and training a tampering region labeling model based on cross-layer matching; inputting the unlabelled image pair into the two models respectively to obtain tampering region labeling results; using a self-filtering algorithm to screen out suboptimal prediction results, and retaining the remaining results as the final labeling results. The image tampering region automatic labeling system of the application can automatically and accurately label a large amount of unlabelled manual tampering data on a network, greatly reduces the tampering region labeling cost of the tampered image, and thus relieves the problem of lack of manual labeling data in the field of image tampering detection.

Description

Technical Field

[0001] This invention relates to the technical fields of artificial intelligence and computer vision, and in particular to a method for automatically labeling image tampering areas. Background Technology

[0002] Digital images are an important medium for information dissemination and are widely present in people's daily lives. However, the rapid development of digital image processing technology has made digital images increasingly vulnerable to tampering by criminals, providing opportunities for fraud, spreading rumors, and other illegal activities, thus posing a threat to social security. Therefore, society urgently needs high-performance digital tampering detection systems to determine whether digital images have been tampered with and to locate the tampered areas.

[0003] Traditional tampering detection methods often suffer from poor robustness and generalization, and their detection capabilities in complex scenes are weak. Deep neural network-based methods offer better detection capabilities, but they are highly dependent on labeled training data. Manually altering and labeling images is time-consuming, labor-intensive, and extremely costly, while synthetic data can easily lead to model overfitting. These factors hinder the application of deep neural network-based digital tampering detection methods. Society urgently needs a system capable of efficiently and automatically labeling tampered regions in massive amounts of unlabeled tampered data to obtain a large amount of high-quality data for training tampering detection models.

[0004] Currently, the amount of manually tampered data with labels is small, and there is no model that can be trained using a small amount of manually tampered data to accurately and automatically label tampered areas in a large amount of unlabeled tampered data.

[0005] As shown in Table 2 of the existing technical document (SAFL-Net: Semantic-Agnostic Feature Learning Network with Auxiliary Plugins for Image Manipulation Detection), the existing manually labeled image tampering localization datasets are relatively small. The introduction section of that document reveals that manually performing image tampering and labeling the tampered regions is time-consuming and laborious, severely impacting the performance of deep learning-based image tampering region localization models. Synthetic data also struggles to adequately alleviate the lack of labeled data, and currently, there is no model capable of accurately automatically labeling tampered regions on a large volume of unlabeled tampered data using a small amount of labeled manually tampered data. The method of this invention can efficiently perform automatic labeling of tampered regions on massive amounts of unlabeled tampered data, obtaining a large amount of high-quality data for training the tampering detection model. This alleviates the overfitting problem of the tampering detection model and achieves significantly higher performance than existing methods. Summary of the Invention

[0006] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide a method, apparatus and medium for automatic annotation of image tampering areas.

[0007] The present invention is achieved by at least one of the following technical solutions.

[0008] An automatic annotation method for image tampering regions includes the following steps:

[0009] Obtain image pairs consisting of the tampered image and its corresponding original image;

[0010] A binary classifier was constructed and trained using unlabeled images through self-supervised learning, and then used to classify image pairs composed of tampered images and the original images.

[0011] Construct and train a tamper region labeling model based on semantic denoised image differences;

[0012] Construct and train a tampered region labeling model based on cross-layer matching;

[0013] Unlabeled image pairs are input into a tampered region labeling model based on semantic denoising image difference and a tampered region labeling model based on cross-layer matching, respectively, to obtain tampered region labeling results;

[0014] The suboptimal prediction results are filtered out using a self-filtering algorithm, and the remaining results are used as the final annotation results.

[0015] Furthermore, the step of training a binary classifier using unlabeled images through self-supervised learning and using it to classify image pairs composed of tampered images and the original images includes the following steps:

[0016] 1) Construct a binary classification model;

[0017] 2) During training, a random unlabeled image is selected as the original image;

[0018] 3) Randomly copy the original Figure 1 The block area is randomly scaled and then pasted to cover other random areas of the original image, and the image pairs before and after processing are labeled as Shared Probe category;

[0019] 4) Randomly copy the original Figure 1 The block area, after being randomly scaled, is pasted onto random areas of other images, and the image pair consisting of this original image and other images after the above random copying and pasting overlay process is labeled as the Shared Donor category;

[0020] 5) Train a binary classification model using the image pairs obtained in steps 3) and 4) and their corresponding category labels;

[0021] 1) Using the trained binary classification model, the image pairs consisting of the tampered image and its corresponding original image are classified into the Shared Probe category or the Shared Donor category.

[0022] Furthermore, the construction and training of the tamper region labeling model based on semantically denoised image differences includes:

[0023] The tampered region labeling model based on semantic denoising image difference has an encoder-decoder structure.

[0024] After scaling the original image and the tampered image to the same size, the difference is calculated, and the absolute value of the difference at the corresponding position is used as the difference image.

[0025] The original image, the tampered image, and the difference image are concatenated along the channel dimension and then input into the tampered region annotation model based on semantic denoising image difference to obtain the prediction result of the tampered region.

[0026] We train a tampered region labeling model based on semantic denoising image difference using images with labeled tampered regions.

[0027] Furthermore, the tampered region labeling model structure based on cross-layer matching includes a backbone network, a semantic enhancement and cross-layer matching module, and a decoder; the semantic enhancement and cross-layer matching module adopts a feature pyramid network (FPN).

[0028] Furthermore, the training of the tampered region labeling model based on cross-layer matching includes:

[0029] After scaling the original image and the tampered image to the same size, they are respectively input into the backbone network to extract multi-scale feature maps;

[0030] Multi-scale feature maps are fused using a feature pyramid approach to enhance the semantic information of high-resolution feature maps.

[0031] Matching feature maps are obtained through semantic enhancement and cross-layer matching modules;

[0032] The matching feature map is input into the decoder to obtain the final prediction result;

[0033] We train a cross-layer matching-based tampering region labeling model using images with tampered region labels.

[0034] Further, the step of inputting unlabeled image pairs into a tampered region labeling model based on semantically denoised image difference and a tampered region labeling model based on cross-layer matching to obtain tampered region labeling results includes:

[0035] Image pairs classified as Shared Probe are input into the tampered region labeling model based on semantic denoising image difference to obtain the prediction results of the tampered region;

[0036] Image pairs classified as Shared Donor are input into a tampered region labeling model based on cross-layer matching to obtain prediction results of tampered regions;

[0037] The prediction results of the tampered region are used as the automatic labeling results of the tampered region in the tampered image.

[0038] Furthermore, the step of using a self-filtering algorithm to remove suboptimal prediction results and retaining the results as the final annotation results includes:

[0039] The annotation results obtained by the tampered region annotation model based on semantic denoising image difference and the tampered region annotation model based on cross-layer matching are normalized;

[0040] Calculate the Quality Evaluation Score (QES):

[0041]

[0042] Where H and W are the height and width of the normalized prediction result mask, respectively, and p i,j For the normalized prediction result mask of the pixels at positions i and j, T h With T l These are predefined high and low thresholds, respectively;

[0043] Predictions with a Quality Evaluation Score (QES) below the threshold are filtered out, and the remaining predictions are used as the final tampered area labels.

[0044] Furthermore, the backbone network adopts ConvNeXt-Base pre-trained on the ADE20k dataset.

[0045] Furthermore, the encoder-decoder structure utilizes a VAN(B5)-UperNet pre-trained on the ADE20k dataset, based on the encoder-decoder structure, with 9 input channels.

[0046] Furthermore, the binary classifier is ConvNeXt-L pre-trained on the ImageNet dataset as the classification model, with an output of 2 categories.

[0047] Another technical solution adopted in this invention is:

[0048] An automatic labeling device for image tampering regions, comprising:

[0049] At least one processor;

[0050] At least one memory for storing at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the method described above.

[0052] Another technical solution adopted in this invention is:

[0053] A computer-readable storage medium storing a processor-executable program, which, when executed by a processor, performs the method described above.

[0054] Compared with existing technologies, the beneficial effects of the present invention are as follows:

[0055] (1) The automatic image tampering region labeling system of the present invention can accurately and efficiently label tampering regions on a large amount of unlabeled tampering data on the Internet by training with only a small amount of labeled manually tampered data, saving a lot of manpower costs and providing strong training data support for various image tampering detection systems; the method of the present invention has strong generalization and can be widely applied.

[0056] (2) This invention utilizes a self-supervised approach to train classifiers for Shared Probe and Shared Donor using unlabeled data, which is accurate and efficient, and reduces the reliance on manually labeled data.

[0057] (3) The tampered region labeling model based on semantic denoising image difference proposed in this invention effectively utilizes the image difference information between the tampered image and the original image in the SharedProbe image pair to reduce the difficulty of the task, and utilizes semantic information to efficiently improve the accuracy of the prediction results.

[0058] (4) The constraint image detection model based on cross-layer matching proposed in this invention greatly alleviates the problem of semantic misalignment of features at the same level caused by scaling operation in the tampering process through cross-layer semantic matching, thus improving the accuracy of tampering annotation prediction results.

[0059] (5) The self-filtering method based on mask prediction confidence proposed in this invention can automatically and accurately filter out substandard tampering labeling results without providing labels, thereby improving the overall quality and reliability of tampering labels. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of an automatic annotation method for image tampering regions according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of a self-supervised training binary classification model in an embodiment of the present invention;

[0063] Figure 3 This is a structural diagram of the cross-layer semantic matching model in an embodiment of the present invention;

[0064] Figure 4 This is a structural diagram of the decoder of the cross-layer semantic matching model in this embodiment of the invention;

[0065] Figure 5 This is a diagram showing the annotation results of the automatic annotation system for image tampering areas in an embodiment of the present invention. Detailed Implementation

[0066] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0067] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0068] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0069] Furthermore, in the description of this invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0070] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0071] like Figure 1 As shown, this embodiment provides a method for automatically labeling image tampering areas, including the following steps:

[0072] S1. Obtain the image pair consisting of the tampered image and its corresponding original image.

[0073] In this embodiment, a total of 2,000 pairs of labeled tampered images and their corresponding real images were collected, and a total of 120,000 pairs of unlabeled tampered images and their corresponding real images were collected.

[0074] S2. Construct and train a binary classifier using unlabeled images through self-supervised learning. Use the trained binary classifier to classify unlabeled tampered image pairs into the Shared Probe category or the Shared Donor category.

[0075] Specifically, step S2 includes steps S21-S24:

[0076] S21. Construct a binary classification model.

[0077] As an example, ConvNeXt-L, pre-trained on the ImageNet dataset, was chosen as the classification model, and its output number of categories was changed to 2.

[0078] S22. Dynamically and randomly generate Shared Probe category image pairs during binary classification model training.

[0079] Specifically, during training, an image is randomly selected as the original image, and the original image is randomly copied. Figure 1 A block region, after being randomly scaled, is pasted onto other random areas of the original image. Random JPEG compression and random Gaussian blur are applied to both the pre- and post-processed images, and the resulting image pairs are categorized as Shared Probes.

[0080] S23. Dynamically and randomly generate Shared Donor category image pairs during binary classification model training.

[0081] Specifically, during the training process, an image is randomly selected as the original image, and the original image is randomly copied. Figure 1 A block of data, after being randomly scaled, is pasted onto random areas of other images. The images before and after processing are then subjected to random JPEG compression and random Gaussian blurring, respectively. The resulting image pair, consisting of the original image and other images processed with the aforementioned random copying and pasting, is labeled as a Shared Donor.

[0082] S24. Train the model described in step S21 using the methods described in steps S22 and S23. Use cross-entropy loss to constrain the model and use the AdamW optimizer for optimization. The learning rate changes linearly and monotonically from the initial 1e-4 to 1e-6, and the number of training iterations is 100,000.

[0083] S25. Train a binary classification model using the collected image pairs and their corresponding category labels.

[0084] Specifically, using the binary classification model trained through steps S21-S24, the image pairs consisting of the tampered images and their corresponding real images collected in step S1 are classified into the Shared Probe category or the Shared Donor category.

[0085] S3. Construct and train a tampered region labeling model based on semantic denoising image differences.

[0086] Specifically, step S3 includes steps S31-S34:

[0087] S31. Construct a tampered region labeling model based on semantic denoising image difference.

[0088] Specifically, the prediction model is VAN(B5)-UPerNet, which is pre-trained on the ADE20k dataset based on an encoder-decoder structure, and the number of input channels is changed from 3 to 9.

[0089] S32. Scale the original image and the tampered image to the same size, with both length and width of 768, then calculate the difference. The absolute value of the difference at corresponding positions is used as the difference image.

[0090] S33. Concatenate the original image, the tampered image, and the difference image along the channel dimension, and input them into the model to obtain the prediction results of the tampered region.

[0091] S34. By comparing the prediction results from step S33 with the annotations, the error is backpropagated to train the tampered region annotation model based on cross-layer matching. The model is constrained by cross-entropy loss and optimized using the AdamW optimizer. The learning rate is linearly and monotonically varied from the initial 1e-4 to 1e-6, and the number of training iterations is 100,000.

[0092] S4. Construct and train a tampered region labeling model based on cross-layer matching.

[0093] S41. Construct a tampered region labeling model based on cross-layer matching. The model structure includes a backbone network, a semantic enhancement and cross-layer matching module, and a decoder, as detailed below:

[0094] S41.1 As an example, ConvNeXt-Base, pre-trained on the ADE20k dataset, is used as the backbone network of the model;

[0095] S41.2. Feature Pyramid Network (FPN) is used as the parameter structure for semantic enhancement and cross-layer matching modules;

[0096] S41.3, Use a multi-scale convolutional model as the decoder, such as Figure 4 As shown.

[0097] S42. Scale the image pair consisting of the tampered image and its corresponding real image to the same size. In this example, the image pair is scaled to an image size of 512 in length and 512 in width.

[0098] S43. Extract multi-scale features of input image pairs through the backbone network; for the input tampered image and its corresponding real image, four feature maps (number of channels, length, width) are obtained respectively: (128,128,128), (256,64,64), (512,32,32), and (1024,16,16). The four feature maps of the tampered image are denoted as fm1, fm2, fm3, and fm4, and the four feature maps of the original image are denoted as fo1, fo2, fo3, and fo4.

[0099] S44. Semantic enhancement is performed on multi-scale feature maps using a feature pyramid network with semantic enhancement and cross-layer matching modules. First, the number of channels in the fm4 and fo4 feature maps is changed to 512 using a 1*1 convolutional layer, while the spatial size remains unchanged. The resulting new feature maps are denoted as Fm4 and Fo4, respectively. Then, Fm4 and Fo4 are sequentially matched with the feature pyramid network. n =C3(C1(fm) n )+UPS(Fm n+1 )), Fo n =C3(C1(fo n )+UPS(Fo n+1 )) , n=1, 2, 3, calculate Fm3, Fm2, Fm1 and Fo3, Fo2, Fo1. Where C3 refers to a 3*3 convolutional layer with 512 input and 512 output channels, and C1 refers to a layer with fm input channels. n or fo nA 1*1 convolutional layer with 512 channels and 512 output channels, and a bilinear interpolation layer with an upsampling rate of 2.

[0100] S45. Calculate the cross-layer matching feature Fcorr, Fcorr = Concat([Corr(Fmi, Foj) for i = 1, 2, 3, 4 and for j = 1, 2, 3, 4]), where Concat refers to the feature dimension concatenation operation and Corr refers to the feature similarity matching operation function.

[0101] S46. Use a multi-scale decoder to decode the matching features. The structure of the multi-scale decoder is as follows: Figure 4 As shown, during the decoding process, the matching features are first reduced to a feature dimension of 512 using a 1x1 convolutional layer, and then multi-scale feature extraction is performed using four 3x3 convolutions with dilation rates of 1, 6, 12, and 18, respectively. The four extracted multi-scale feature maps are then concatenated along their feature dimensions and reduced to a feature dimension of 512 using a 1x1 convolutional layer. Finally, the predicted tampered region is obtained using the reduced features through a 1x1 convolutional layer.

[0102] S47. The model is constrained by cross-entropy loss and optimized using the AdamW optimizer. The learning rate changes linearly and monotonically from the initial 1e-4 to 1e-6, and the number of training iterations is 100,000.

[0103] S5. Input the unlabeled image pairs into the two models respectively to obtain the prediction results.

[0104] For the image pairs to be labeled collected in step S1, those that are classified and predicted as Shared Probe in step S25 are input into the model described in step S3 to obtain the labeling prediction result of the tampered region; for the image pairs to be labeled collected in step S1, those that are classified and predicted as Shared Donor in step S25 are input into the model described in step S4 to obtain the labeling prediction result of the tampered region.

[0105] S6. Use a self-filtering algorithm to remove suboptimal prediction results and retain the results as the final annotation results.

[0106] Specifically, step S6 includes the following steps S61-S63:

[0107] S61. Normalize the prediction result mask obtained in step S5 so that each pixel value in the mask image represents the probability that the location is a tampered area, and the probability value range is [0,1].

[0108] S62, According to the formula Calculate the Quality Evaluation Score (QES), where H and W are the height and width of the normalized prediction result mask, respectively, and pi,j For the normalized prediction result mask at position i,j, T h With T l These are predefined high and low thresholds, both set to 1 / 16.

[0109] S63. Screen out prediction results with a quality assessment score (QES) lower than the set threshold of 0.5, and use the remaining prediction results as the final tampered area label.

[0110] like Figure 5 As shown in Figure 5, this is the annotation result of the automatic annotation system for the tampered area of ​​the image.

[0111] This embodiment also provides an automatic image tampering region labeling device, including:

[0112] At least one processor;

[0113] At least one memory for storing at least one program;

[0114] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0115] This embodiment of the automatic image tampering region labeling device can execute the automatic image tampering region labeling method provided in the method embodiment of the present invention, and can execute any combination of the implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0116] This application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0117] This embodiment also provides a storage medium storing instructions or programs that can execute the automatic annotation method for image tampering areas provided in the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the method has the corresponding functions and beneficial effects.

[0118] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0119] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0120] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0122] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0123] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0124] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0125] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0126] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for automatically labeling image tampering regions, characterized in that, Includes the following steps: Obtain image pairs consisting of the tampered image and its corresponding original image; A binary classifier was constructed and trained using unlabeled images through self-supervised learning, and then used to classify image pairs composed of tampered images and the original images. A tamper region labeling model based on semantically denoised image difference is constructed and trained. The structure of the tamper region labeling model based on semantically denoised image difference is an encoder-decoder structure. The original image and the tampered image are scaled to the same size and then subtracted. The absolute value of the difference at the corresponding position is used as the difference image. The original image, the tampered image, and the difference image are concatenated along the channel dimension and input into the tamper region labeling model based on semantically denoised image difference to obtain the prediction result of the tampered region. The tamper region labeling model is trained using images with tamper region labels. A tampered region labeling model based on cross-layer matching is constructed and trained; the structure of the tampered region labeling model based on cross-layer matching includes a backbone network, a semantic enhancement and cross-layer matching module, and a decoder; The semantic enhancement and cross-layer matching module employs a Feature Pyramid Network (FPN). Training a tampered region labeling model based on cross-layer matching includes: The original image and the tampered image are scaled to the same size and then input into the backbone network to extract multi-scale feature maps. The multi-scale feature maps are fused using a feature pyramid to enhance the semantic information of the high-resolution feature maps. Matching feature maps are obtained through semantic enhancement and cross-layer matching modules. The matching feature maps are input into the decoder to obtain the final prediction result. A tampered region labeling model based on cross-layer matching is trained using images with tampered region labels. Unlabeled image pairs are input into a tampered region labeling model based on semantic denoising image difference and a tampered region labeling model based on cross-layer matching, respectively, to obtain tampered region labeling results; The suboptimal prediction results are filtered out using a self-filtering algorithm, and the remaining results are used as the final annotation results.

2. The method for automatically labeling image tampering regions according to claim 1, characterized in that, The method of training a binary classifier using unlabeled images through self-supervised learning and then using it to classify image pairs composed of tampered images and the original images includes the following steps: 1) Construct a binary classification model; 2) During training, a random unlabeled image is selected as the original image; 3) Randomly copy a region of the original image, scale it randomly, and then paste it to cover other random regions of the original image. Mark the images before and after processing as Shared Probe. 4) Randomly copy a region of the original image, scale it randomly, and paste it onto a random region of another image. Then, label the image pair consisting of this original image and the other images that have undergone the random copying and pasting process as the Shared Donor category. 5) Train a binary classification model using the image pairs obtained in steps 3) and 4) and their corresponding category labels; 6) Using the trained binary classification model, the image pairs consisting of the tampered image and its corresponding original image are classified into the SharedProbe category or the SharedDonor category.

3. The method for automatically labeling image tampering regions according to claim 2, characterized in that, The step of inputting unlabeled image pairs into a tampered region labeling model based on semantic denoising image difference and a tampered region labeling model based on cross-layer matching to obtain tampered region labeling results includes: Image pairs classified as Shared Probe are input into the tampered region labeling model based on semantic denoising image difference to obtain the prediction results of the tampered region; Image pairs classified as Shared Donor are input into a tampered region labeling model based on cross-layer matching to obtain prediction results of tampered regions; The prediction results of the tampered region are used as the automatic labeling results of the tampered region in the tampered image.

4. The method for automatically labeling image tampering regions according to claim 1, characterized in that, The process of using a self-filtering algorithm to remove suboptimal prediction results and retaining the remaining results as the final annotation results includes: The annotation results obtained by the tampered region annotation model based on semantic denoising image difference and the tampered region annotation model based on cross-layer matching are normalized; Calculate the Quality Evaluation Score (QES): Where H and W are the height and width of the normalized prediction result mask, respectively, and p i,j For the normalized prediction result mask of the pixels at positions i and j, T h With T l These are predefined high and low thresholds, respectively; Predictions with a Quality Evaluation Score (QES) below the threshold are filtered out, and the remaining predictions are used as the final tampered area labels.

5. The method for automatically labeling image tampering regions according to claim 1, characterized in that, The backbone network uses ConvNeXt-Base, which is pre-trained on the ADE20k dataset.

6. The method for automatically labeling image tampering regions according to claim 1, characterized in that, The encoder-decoder structure utilizes a VAN(B5)-UperNet pre-trained on the ADE20k dataset, with 9 input channels.

7. The method for automatically labeling image tampering regions according to any one of claims 1 to 6, characterized in that, The binary classifier is ConvNeXt-L, which is pre-trained on the ImageNet dataset, and outputs 2 classes.

Images