Image forgery detection method based on self-supervised contrast learning

Abstract

The invention relates to the technical field of image forgery detection, in particular to an image forgery detection method based on self-supervised contrast learning. According to the technical scheme, the method comprises the steps that an image to be detected is input into a trained image forgery detection model, and an image forgery detection result of the image to be detected is obtained; the image forgery detection model comprises a feature extractor which is pre-trained through self-supervised contrast learning and is used for extracting an image feature I of an image to be detected; the classifier is used for classifying the to-be-detected image into a real image or a forged image based on the image features I extracted by the feature extractor; the self-supervised contrast learning pre-training takes a real image set as a positive sample, and takes the real image set and a forged image set as negative samples. According to the method, feature distribution of a real image and a forged image is mined through self-supervised comparative learning, and the problem of generalization of a detection model is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image forgery detection, and in particular to an image forgery detection method based on self-supervised contrast learning. Background Art

[0002] With the rapid development of deep learning technology, deepfake technology has gradually become a prominent issue in the social and security fields. Deepfake technology uses deep learning algorithms to generate highly realistic fake facial images and videos. These technologies are not only widely used in entertainment, advertising, e-commerce and other fields, but also pose potential security risks to a certain extent, especially in social media, news dissemination and other fields. Deepfake facial images and videos are difficult to distinguish between true and false, which brings great risks and challenges to society.

[0003] At the same time, with the popularization of AIGC (artificial intelligence generated content) technology, the quality and authenticity of generated images, videos and other content have been further improved, making it more difficult to identify forged content. AIGC technology includes text-based image and video generation (such as StableDiffusion, DALL-E, Midjourney, Sora, etc.), text generated based on natural language, and speech based on sound synthesis. These technologies make the generation of forged content no longer limited to face synthesis, but expand to a wider range of multimedia forms, thereby increasing the complexity of forged content detection.

[0004] Another widespread forgery problem is image tampering. Traditional image processing software (such as Photoshop) makes image tampering operations very simple. Users can easily modify or replace certain parts of the image, changing the authenticity of the image. Although these image modification operations are relatively intuitive, in order to avoid being detected, image tampering often requires precise pixel-level operations, which increases the complexity of the forged content and also puts higher requirements on existing forgery detection technology.

[0005] In response to these forgery problems, the current forgery detection technology is mainly focused on the following directions: First, for the detection of deep fake faces, the authenticity of the image is mainly determined by analyzing the detailed features in the image (such as facial edge traces, skin texture, etc.). Secondly, for the detection of AIGC-generated images, the methods mostly rely on analyzing whether the logic of the image conforms to the laws of the real physical world, the laws of light reflection, and the unnatural characteristics of the image. Finally, for PS tampered images, researchers usually detect tampering traces through inconsistencies in the image (such as mismatched lighting, unnatural image edges, etc.).

[0006] However, these forgery detection methods are usually limited to a single task, and each type of forgery detection task relies on a separately trained classifier for identification. Different types of forgery images have different forgery methods, features, and detection difficulties, which leads to the existing detection technology generally having problems such as insufficient accuracy, slow speed, poor adaptability, and poor generalization when facing multiple forgery scenarios.

[0007] For the three types of forgery detection tasks, namely deep fake face detection, AIGC generated image detection and PS tampered image detection, there are currently two main methods: image forgery detection based on traditional methods and image forgery detection based on deep learning.

[0008] Image detection based on traditional methods. For deep fake face detection, it generally relies on manually extracted features and trains classifiers to identify fake images by analyzing these features. For example, the HOG (Histogram of Oriented Gradients) feature extraction method is used to encode local texture and structural information in the image, and then the image is classified by support vector machine (SVM). These methods are difficult to cover all potential information in fake images, and their generalization ability is poor for new fake methods or different fake images. For AIGC generated image detection, some basic statistical features in the image are analyzed (such as texture and noise, color and lighting, etc.) to distinguish between generated images and real images. AIGC generated images may be different from real images in terms of texture, noise distribution, etc. The generated images and real images are distinguished by calculating the high-order statistics of the image (such as high-frequency components, edge detection, noise spectrum, etc.). For PS tampered image detection, it relies on extracting low-level features or statistical features of the image (texture, edge and lighting, noise, color and hue, etc.), analyzing the consistency of local texture, edge consistency and lighting consistency to detect tampering traces in the image.

[0009] Image detection based on deep learning. For these three types of tasks, the forged content is identified by automatically learning the detailed features, texture information and unnatural forgery traces in the image. The current mainstream method outputs a binary classification result (such as "real" or "forged") and a confidence score through a trained binary classification detection model. For the deep forgery face detection task, when there are multiple faces in the image, the entire detection process is divided into two stages. It is necessary to rely on the face detection model for face detection first and then input the detected faces into the classifier for detection in sequence. The whole process is relatively cumbersome. For AIGC generated image detection, the detection accuracy of images where some areas in the image are generated by AIGC is very low, and the specific generated area cannot be located. In addition, since the model directly trains the classifier on the forgery detection dataset of the corresponding task, it lacks prior knowledge and is prone to overfitting. It achieves high accuracy on the same distribution data, but the accuracy drops sharply on the new method, and the generalization ability across datasets is low. Summary of the invention

[0010] The technical problem to be solved by the present invention is: in view of the above-mentioned problems, a method for detecting image forgery based on self-supervised contrastive learning is provided.

[0011] The technical solution adopted by the present invention is: an image forgery detection method based on self-supervised contrast learning, comprising: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for classifying the image to be detected as a real image or a forged image based on the image feature I extracted by the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

[0012] A method for detecting image forgery based on self-supervised contrastive learning, comprising: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for predicting whether the image to be detected is a real image or one of various preset types of forged images based on the image feature I extracted by the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

[0013] A method for detecting image forgery based on self-supervised contrastive learning, comprising: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for predicting whether the image to be detected is a real image or one of various preset types of forged images based on the image feature I extracted by the feature extractor; A forged region positioning module is used to generate a binary mask image of the forged region of the image to be detected based on the image to be detected, the classification result of the classifier and the image feature I of the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

[0014] The forged area positioning module includes: Image encoder, used to extract image features of the image to be detected II; The image decoder is used to generate a binary mask image of the forged area of ​​the image to be detected based on the image feature II of the image encoder, the classification result of the classifier and the image feature I of the feature extractor.

[0015] The self-supervised contrastive learning pre-training includes: The real image set is used as positive samples, and the real image set and the forged image set are used as negative samples; Extracting image features of an image in a positive sample by a feature extractor, and extracting image features of another image in the positive sample by another feature extractor; Extracting image features of real images in negative samples through a feature extractor, and extracting image features of forged images in negative samples through another feature extractor; Based on the image features extracted by the two feature extractors, the contrast loss is calculated to reduce the distance between positive samples and increase the distance between negative samples.

[0016] The self-supervised contrastive learning pre-training is performed using a self-distillation method.

[0017] The training of the image forgery detection model includes: Freeze the parameters of the feature extractor pre-trained by self-supervised contrastive learning in the image forgery detection model; Inputting the training sample set into the image forgery detection model for training to obtain a trained image forgery detection model; The samples in the training sample set include real images or forged images, image classification labels and forged region binary mask images corresponding to the forged images; the image classification labels include real image labels or forged image labels of various preset types.

[0018] The training of the image forgery detection model includes: For the feature extractor, LoRA is used to fine-tune a small number of incremental parameters.

[0019] The training of the image forgery detection model includes: Multi-label classification loss: , , in, is the true value of label i, which can be 0 or 1; is the probability value of the model predicting label i; C is the number of categories.

[0020] The training of the image forgery detection model includes: Localization region binary mask loss: , , in, is the true label of the i-th pixel, which takes a value of 0 or 1; is the predicted probability of the i-th pixel, indicating the probability that the pixel belongs to the forged area; N is the total number of pixels in the image; , in, Represents the number of intersection pixels between the real mask and the predicted mask; represents the number of non-zero pixels in the true mask; Represents the number of non-zero pixels in the predicted mask.

[0021] A storage medium stores a computer program executable by a processor, wherein the computer program implements the steps of the image forgery detection method based on self-supervised contrast learning when executed.

[0022] An image forgery detection device comprises a memory and a processor. The memory stores a computer program executable by the processor. When the computer program is executed, the steps of the image forgery detection method based on self-supervised contrast learning are implemented.

[0023] The beneficial effects of the present invention are: The present invention mines the feature distribution of real images and forged images through self-supervised contrastive learning to solve the generalization problem of the detection model. In contrastive learning, by comparing the differences between samples in the real image set, the representative features of the real image are learned; by the differences between samples in the real image set and samples in the forged image set, the feature differences between the two types of images are learned.

[0024] The present invention adds a classifier after the feature extractor pre-trained by the feature extractor. The classifier realizes accurate judgment of the authenticity of the image based on the representative features of the real image extracted by the feature extractor and the feature differences between the real and fake images.

[0025] The present invention adds a forged area positioning module after the feature extractor and the classifier. The forged area positioning module generates a forged area binary mask image based on the image features extracted by the feature extractor and the classification results of the classifier.

[0026] The forged area positioning module of the present invention is different from the traditional two-classification discrimination method (which only outputs whether the image is "real" or "forged"). The goal of the forged area positioning module is not only to determine whether the image is forged, but also to accurately locate the specific area in the image that is forged. This is of great value for subsequent image processing and analysis, especially in the fields of law enforcement, digital copyright protection, and media verification.

[0027] The present invention introduces self-supervised contrastive learning into the image forgery detection task, learns the deep feature representation in the image, mines the real image feature distribution and semantic feature information, and pre-understands the dynamic feature distribution of natural images. The model is not easy to overfit and has higher accuracy, stronger robustness and generalization. The pre-trained model lays a solid foundation for subsequent supervised training.

[0028] In the field of image forgery detection, although the detection tasks of various types of forged images (such as deep forged face detection, AIGC generated image detection, and PS tampered images) differ in details, they all belong to the category of forgery detection and involve unnatural changes in certain areas of the image. Therefore, the present invention adopts a multi-task joint unified training method to unify several types of tasks into one optimization goal. By sharing network layers or network modules, the model shares the underlying features when processing different tasks, so that the model can learn more general feature representations. The model is allowed to learn and optimize these types of tasks simultaneously under the same training framework, thereby improving the overall accuracy, robustness and generalization ability of the model. By sharing the underlying feature extraction network, it is avoided to train different network structures separately for each task, thereby saving training time and computing resources.

[0029] The present invention uses multi-task joint unified training to cross-integrate the features of different tasks, find complementary information between tasks, help the model capture multi-dimensional forgery features more comprehensively, and solve the accuracy problem of the detection model. Compared with the existing task-independent forgery detection technology, building a unified model to handle these types of tasks at the same time can significantly improve detection accuracy, model generalization ability and interpretable analysis ability, and improve the forgery detection effect across scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the framework of the image forgery detection model in the embodiment.

[0031] Figure 2 Schematic diagram of self-supervised contrastive learning pre-training in an embodiment. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0033] In the description of the present invention, the meaning of "a plurality" is two or more than two. If there is a description of "a first" or "a second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art.

[0034] Embodiment 1: This embodiment is a method for detecting image forgery based on self-supervised contrastive learning, which specifically includes the following steps: S100, obtaining an image to be detected; S200: Input the image to be detected into a trained image forgery detection model, and the image forgery detection model outputs an image forgery detection result corresponding to the image to be detected, which is a real image or a forged image.

[0035] In this embodiment, the image forgery detection model includes a feature extractor and a classifier, wherein the feature extractor uses a vision transformer as a backbone network and is pre-trained by self-supervised contrastive learning, and can be used to extract image features I of the image to be detected; the classifier is used to classify the image to be detected as a real image or a forged image based on the image features I extracted by the feature extractor.

[0036] In this example, the feature extractor is pre-trained using self-supervised contrastive learning, which includes: A. The real image set is used as positive samples, and the real image set and the forged image set are used as negative samples.

[0037] In this embodiment, different samples of a real image after data enhancement (such as cropping, rotation, etc.) are used as positive samples. Since the internal structure and features of the real image are relatively stable, they can provide consistent features for the model.

[0038] A sample of a real image after data augmentation (such as cropping, rotation, etc.) and a sample of a forged image after data augmentation are used as negative samples. This forces the model to learn to distinguish the features of real images from forged images, thereby enhancing its sensitivity to image features.

[0039] B. Extract image features of images in positive and negative samples through two feature extractors. Use one feature extractor to extract image features of images in positive samples, and use another feature extractor to extract image features of another image in positive samples; use one feature extractor to extract image features of real images in negative samples, and use another feature extractor to extract image features of forged images in negative samples.

[0040] C. Based on the image features extracted by the two feature extractors, the contrast loss is calculated to reduce the distance between positive samples and increase the distance between negative samples.

[0041] In this embodiment, the self-supervised contrastive learning pre-training adopts the self-distillation method, that is, the representation of a student model is compared with the representation of a momentum-quantized teacher model to learn a better feature representation. The images after different data enhancements are input into the student model and the teacher model respectively, and the corresponding features are obtained. The loss between the features is calculated to reduce the distance between the positive sample pairs and increase the distance between the negative sample pairs. The teacher model is not updated based on back propagation, but after the gradient of the student model is back-propagated, the weight is updated by exponential moving average.

[0042] The training of the image forgery detection model in this embodiment includes: Ⅰ. Freeze the parameters of the feature extractor pre-trained by self-supervised contrastive learning in the image forgery detection model, do not fine-tune the full parameters, and use LoRA to fine-tune a small number of incremental parameters; Ⅱ. Input the training sample set into the image forgery detection model for training to obtain a trained image forgery detection model.

[0043] In this example, the samples in the training sample set include images and image classification labels of the images, wherein the images are real images or forged images, and the image classification labels are real image labels or forged image labels corresponding to the images in the samples.

[0044] Embodiment 2: This embodiment is basically the same as Embodiment 1, except that in this embodiment, the classifier is used to predict the probability that the image to be detected is a real image or a forged image of each preset type based on the image feature I extracted by the feature extractor.

[0045] In this example, the preset types of fake images include deep fake face images, AIGC generated images, and PS tampered images.

[0046] In this example, the image to be detected is input into the trained image forgery detection model, and the image forgery detection model outputs the image forgery detection result corresponding to the image to be detected, which is one of the four categories: real image, deep fake face image, AIGC generated image, and PS tampered image.

[0047] The training of the image forgery detection model in this embodiment includes: Ⅰ. Freeze the parameters of the feature extractor pre-trained by self-supervised contrastive learning in the image forgery detection model, do not fine-tune the full parameters, and use LoRA to fine-tune a small number of incremental parameters; Ⅱ. Input the training sample set into the image forgery detection model for training to obtain a trained image forgery detection model.

[0048] In this example, the samples in the training sample set include images and image classification labels of the images, where the images are real images or forged images, and the image classification labels are real image labels corresponding to the images in the samples, deep forged face image labels, AIGC generated image labels, or PS tampered image labels.

[0049] Embodiment 3: This embodiment is a method for detecting image forgery based on self-supervised contrastive learning, which specifically includes the following steps: S100, obtaining an image to be detected; S200. Input the image to be detected into a trained image forgery detection model. The image forgery detection model outputs an image forgery detection result corresponding to the image to be detected. The image forgery detection result includes one of four categories: real image, deep forged face image, AIGC generated image, and PS tampered image. When the image to be detected is a forged image, the image forgery detection result also includes a binary mask image of the forged area.

[0050] The image forgery detection model in this embodiment includes a feature extractor, a classifier, and a forged area positioning module.

[0051] In this example, the feature extractor uses the vision transformer as the backbone network, and after pre-training with self-supervised contrastive learning, it can be used to extract image features of the image to be detected I.

[0052] In this embodiment, the classifier is used to classify the image to be detected into one of the four categories: real image, deep fake face image, AIGC generated image, and PS tampered image based on the image feature I extracted by the feature extractor.

[0053] In this example, the forged area positioning module includes an image encoder and an image decoder, wherein the image encoder is used to extract image features II of the image to be detected; the image decoder is used to generate a binary mask image of the forged area of ​​the image to be detected based on the image features II of the image encoder, the classification results of the classifier and the image features I of the feature extractor.

[0054] In this embodiment, the image forgery detection model first extracts features of the input image through a pre-trained backbone network, and then transmits the extracted image feature vectors to the classifier and the forged area positioning module respectively. The classifier judges the authenticity of the image. If it is a real image, the judgment result is directly output. If it is a forged image, the image decoder outputs the corresponding forged area binary mask to indicate the specific location of the forged area.

[0055] In this example, the feature extractor is pre-trained using self-supervised contrastive learning, which includes: A. The real image set is used as positive samples, and the real image set and the forged image set are used as negative samples.

[0056] In this embodiment, different samples of a real image after data enhancement (such as cropping, rotation, etc.) are used as positive samples. Since the internal structure and features of the real image are relatively stable, they can provide consistent features for the model.

[0057] A sample of a real image after data augmentation (such as cropping, rotation, etc.) and a sample of a forged image after data augmentation are used as negative samples. This forces the model to learn to distinguish the features of real images from forged images, thereby enhancing its sensitivity to image features.

[0058] B. Extract image features of images in positive and negative samples through two feature extractors. Use one feature extractor to extract image features of images in positive samples, and use another feature extractor to extract image features of another image in positive samples; use one feature extractor to extract image features of real images in negative samples, and use another feature extractor to extract image features of forged images in negative samples.

[0059] C. Based on the image features extracted by the two feature extractors, the contrast loss is calculated to reduce the distance between positive samples and increase the distance between negative samples.

[0060] In this embodiment, the self-supervised contrastive learning pre-training adopts the self-distillation method, that is, the representation of a student model is compared with the representation of a momentum-quantized teacher model to learn a better feature representation. The images after different data enhancements are input into the student model and the teacher model respectively, and the corresponding features are obtained. The loss between the features is calculated to reduce the distance between the positive sample pairs and increase the distance between the negative sample pairs. The teacher model is not updated based on back propagation, but after the gradient of the student model is back-propagated, the weight is updated by exponential moving average.

[0061] The training of the image forgery detection model in this embodiment includes: Ⅰ. Freeze the parameters of the feature extractor pre-trained by self-supervised contrastive learning in the image forgery detection model, do not fine-tune the full parameters, and use LoRA to fine-tune a small number of incremental parameters; Ⅱ. Input the training sample set into the image forgery detection model for training to obtain a trained image forgery detection model.

[0062] In this example, the samples in the training sample set include images (real images or forged images), image classification labels of the images, and binary mask images of the forged areas corresponding to the forged images. The image classification labels are deep forged face image labels, AIGC generated image labels, or PS tampered image labels corresponding to the images in the samples.

[0063] The loss function of the image forgery detection model in this embodiment mainly consists of two parts: multi-label classification loss and localization region binary mask loss.

[0064] Multi-label classification loss: , , in, is the true value of label i, which can be 0 or 1. is the probability value of the model predicting label i (that is, the probability of predicting the label to be 1). C is the number of categories.

[0065] Localization region binary mask loss: , , in, is the true label of the i-th pixel, which takes the value of 0 or 1. is the predicted probability of the ith pixel, indicating the probability that the pixel belongs to the forged region. N is the total number of pixels in the image.

[0066] , in, Represents the number of intersection pixels between the true mask and the predicted mask. Represents the number of non-zero pixels in the true mask (i.e., the true target area size). Represents the number of non-zero pixels in the predicted mask (i.e., the size of the predicted object region).

[0067] Total loss: , in, is the multi-label classification loss, and are both binary mask losses of the localization region. y is the predicted category probability, is the true label. is the binary mask of the predicted forged area, is the real binary mask image.

[0068] Embodiment 4: This embodiment is a storage medium on which a computer program executable by a processor is stored. When the computer program is executed, the steps of the image forgery detection method based on self-supervised contrast learning in this embodiment are implemented.

[0069] Embodiment 5: This embodiment is an image forgery detection device, which has a memory and a processor. The memory stores a computer program that can be executed by the processor. When the computer program is executed, the steps of the image forgery detection method based on self-supervised contrast learning in Embodiment 1 are implemented.

[0070] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0071] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0072] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the above methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0073] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0074] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0075] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0076] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0077] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0078] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for detecting image forgery based on self-supervised contrastive learning, characterized in that: include: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for classifying the image to be detected as a real image or a forged image based on the image feature I extracted by the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

2. A method for detecting image forgery based on self-supervised contrastive learning, characterized in that: include: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for predicting whether the image to be detected is a real image or one of various preset types of forged images based on the image feature I extracted by the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

3. A method for detecting image forgery based on self-supervised contrastive learning, characterized in that: include: Inputting the image to be detected into the trained image forgery detection model to obtain the image forgery detection result of the image to be detected; The image forgery detection model comprises: Feature extractor, pre-trained by self-supervised contrastive learning, is used to extract image features of the image to be detected I; A classifier, used for predicting whether the image to be detected is a real image or one of various preset types of forged images based on the image feature I extracted by the feature extractor; A forged region positioning module is used to generate a binary mask image of the forged region of the image to be detected based on the image to be detected, the classification result of the classifier and the image feature I of the feature extractor; The self-supervised contrastive learning pre-training uses a real image set as positive samples and a real image set and a forged image set as negative samples.

4. The image forgery detection method based on self-supervised contrastive learning according to claim 3, characterized in that: The forged area positioning module includes: Image encoder, used to extract image features of the image to be detected II; The image decoder is used to generate a binary mask image of the forged area of ​​the image to be detected based on the image feature II of the image encoder, the classification result of the classifier and the image feature I of the feature extractor.

5. The image forgery detection method based on self-supervised contrastive learning according to any one of claims 1 to 3, characterized in that: The self-supervised contrastive learning pre-training includes: The real image set is used as positive samples, and the real image set and the forged image set are used as negative samples; Extracting image features of an image in a positive sample by a feature extractor, and extracting image features of another image in the positive sample by another feature extractor; Extracting image features of real images in negative samples through a feature extractor, and extracting image features of forged images in negative samples through another feature extractor; Based on the image features extracted by the two feature extractors, the contrast loss is calculated to reduce the distance between positive samples and increase the distance between negative samples.

6. The image forgery detection method based on self-supervised contrastive learning according to claim 5, characterized in that: The self-supervised contrastive learning pre-training is performed using a self-distillation method.

7. The image forgery detection method based on self-supervised contrastive learning according to claim 3, characterized in that: The training of the image forgery detection model includes: Freeze the parameters of the feature extractor pre-trained by self-supervised contrastive learning in the image forgery detection model; Inputting the training sample set into the image forgery detection model for training to obtain a trained image forgery detection model; The samples in the training sample set include real images or forged images, image classification labels and forged region binary mask images corresponding to the forged images; the image classification labels include real image labels or forged image labels of various preset types.

8. The image forgery detection method based on self-supervised contrastive learning according to claim 7, characterized in that: The training of the image forgery detection model includes: For the feature extractor, LoRA is used to fine-tune a small number of incremental parameters.

9. The image forgery detection method based on self-supervised contrastive learning according to claim 7, characterized in that: The training of the image forgery detection model includes: Multi-label classification loss: ， ， in, is the true value of label i, which can be 0 or 1; is the probability value of the model predicting label i; C is the number of categories.

10. The image forgery detection method based on self-supervised contrastive learning according to claim 7, characterized in that: The training of the image forgery detection model includes: Localization region binary mask loss: ， ， in, is the true label of the i-th pixel, which takes a value of 0 or 1; is the predicted probability of the i-th pixel, indicating the probability that the pixel belongs to the forged area; N is the total number of pixels in the image; ， in, Represents the number of intersection pixels between the real mask and the predicted mask; represents the number of non-zero pixels in the true mask; Represents the number of non-zero pixels in the predicted mask.

11. A storage medium having stored thereon a computer program executable by a processor, characterized in that: When the computer program is executed, the steps of the image forgery detection method based on self-supervised contrast learning according to any one of claims 1 to 10 are implemented.

12. An image forgery detection device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the computer program is executed, the steps of the image forgery detection method based on self-supervised contrast learning according to any one of claims 1 to 10 are implemented.

Images