Model training method, forged image detection method and device

Abstract

The embodiment of the invention provides a model training method and device and a forged image detection method and device, and relates to the technical fields of computer technology, artificial intelligence, computer vision and the like. The method comprises the steps of obtaining a training sample set; the training sample set comprises a plurality of sample images and a sample label of each sample image; the sample label of each sample image comprises a real counterfeit identifier corresponding to each pixel point in the sample image; and performing at least one training operation on the initial counterfeit detection model based on the training sample set, and taking the initial counterfeit detection model meeting a preset training ending condition as a trained counterfeit detection model. According to the embodiment of the invention, the difference between the prediction feature of the real area and the prediction feature of the counterfeit area can be forcibly increased, the distinguishing effect in the high-similarity image is effectively improved, and the detection effect in the counterfeit detection scene of the high-similarity academic image is improved.

Description

Technical Field

[0001] This disclosure relates to the fields of computer technology, artificial intelligence, computer vision, etc. Specifically, this disclosure relates to a model training method, a forged image detection method and apparatus. Background Technology

[0002] Digital images, as a key carrier of information transmission and knowledge presentation, have deeply penetrated into various aspects such as scientific research, media, and social interaction. However, with the leapfrog development of digital image processing technology and computing power, the authenticity and integrity of image content are facing unprecedented challenges. Image forgery has evolved from traditional manual editing to highly automated, high-fidelity generation and alteration driven by artificial intelligence.

[0003] Digital images, as critical carriers of information, are facing a severe challenge to their authenticity from AI-driven forgery techniques. Image forgery has evolved from operations relying on traditional editing software and easily leaving local inconsistencies (such as boundary, lighting, and noise anomalies) to advanced generation and manipulation dominated by deep learning models. In the scientific research field, image forgery can be used to "beautify" or even falsify experimental results, affecting research integrity. Therefore, how to accurately identify forged images has become an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a model training method, a forged image detection method, and an apparatus, which can solve the problem of poor forged image detection performance in the prior art. The technical solution provided by this disclosure is as follows: According to one aspect of the present disclosure, a model training method is provided, the method comprising: Obtain a training sample set; the training sample set includes multiple sample images and a sample label for each sample image; the sample label for each sample image includes a real / fake identifier corresponding to each pixel in the sample image; Based on the training sample set, perform at least one training operation on the initial forgery detection model, and take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operation includes: For each sample image, a predicted forgery detection result is obtained through an initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image. For each reference pixel in each sample image, a first loss is determined based on the differences between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

[0005] According to another aspect of the present disclosure, a method for detecting forged images is provided, the method comprising: Acquire the image to be detected; The image to be detected is input into the trained forgery detection model to obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model; The forgery detection model is trained based on the model training method provided in any of the above embodiments.

[0006] According to another aspect of the present disclosure, a model training apparatus is provided, the apparatus comprising: The training sample acquisition module is used to acquire a training sample set; the training sample set includes multiple sample images and a sample label for each sample image; the sample label for each sample image includes a real / fake identifier corresponding to each pixel in the sample image; The model training module is used to perform at least one training operation on the initial forgery detection model based on the training sample set, and to take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operation includes: For each sample image, a predicted forgery detection result is obtained through an initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image. For each reference pixel in each sample image, a first loss is determined based on the differences between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

[0007] According to another aspect of the present disclosure, a forged image detection apparatus is provided, the apparatus comprising: The image acquisition module is used to acquire the image to be detected. The forgery detection module is used to input the image to be detected into a trained forgery detection model and obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model. The forgery detection model is trained based on the model training method provided in any of the above embodiments.

[0008] According to another aspect of the present disclosure, an electronic device is provided, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the above-described model training methods or forged image detection methods.

[0009] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the model training methods or forged image detection methods described above.

[0010] According to one aspect of the present disclosure, a computer program product is provided, which includes a computer program that, when executed by a processor, implements the steps of any of the model training methods or forged image detection methods described above.

[0011] The beneficial effects of the technical solutions provided in this disclosure are: Based on the differences between the reference pixel and the first and second pixels, the first loss corresponding to the reference pixel is determined. Based on the first loss corresponding to each reference pixel in each sample image, the total training loss is determined. The training of the initial forgery detection model is constrained by the total training loss, which can force an increase in the difference between the predicted features of the real region (i.e., the unaltered region) and the predicted features of the forgery region, effectively improving the differentiation effect in high similarity images. This is beneficial for improving the detection effect in forgery detection scenarios of high similarity academic images. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0013] Figure 1A schematic flowchart of a model training method provided in an embodiment of this disclosure; Figure 2 A schematic diagram of a feature space provided for an embodiment of this disclosure; Figure 3 A schematic diagram illustrating a counterfeit detection method provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of a forged image detection device provided in an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0014] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0015] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term, for example, “A and / or B” or “A, B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0016] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0017] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0018] Figure 1 This is a flowchart illustrating a model training method provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, the method includes: Step S110: Obtain the training sample set; the training sample set includes multiple sample images and sample labels for each sample image; the sample label for each sample image includes the real / fake identifier corresponding to each pixel in the sample image.

[0019] Specifically, the training sample set can include multiple sample images, which can include real sample images (i.e., unspoofed sample images) or spoofed sample images. For each sample image, the sample label can be the real / spoof identifier corresponding to each pixel in the sample image. The real / spoof identifier can be used to characterize whether the pixel is located in the spoofed region. For example, a label (i.e., the real / spoof identifier) ​​can be set. When the label of a pixel is 0, it means that the pixel is located in the real region, and the pixel is called a real pixel. When the label of a pixel is 1, it means that the pixel is located in the spoofed region, and the pixel is called a spoofed pixel.

[0020] Optionally, for each sample image, the sample label may also include the image identifier of the sample image. For example, when the image identifier of the sample image is 0, it indicates that the sample image is a real sample image, and when the image identifier of the sample image is 1, it indicates that the sample image is a fake sample image.

[0021] Step S120: Perform at least one training operation on the initial forgery detection model based on the training sample set, and take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operations include: For each sample image, the predicted forgery detection result is obtained through the initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image; For each reference pixel in each sample image, a first loss is determined based on the difference between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

[0022] Specifically, after obtaining the training sample set, multiple sample images can be input into the initial forgery detection model. The initial forgery detection model identifies the forgery regions in each sample image, obtaining the predicted forgery detection result for each sample image. For each sample image, the predicted forgery detection result can include the predicted forgery identifier for each pixel in the sample image.

[0023] For each pixel, the predicted forgery identifier can be used to characterize whether the pixel is located in a forgery area.

[0024] Optionally, the initial forgery detection model can be built based on a diffusion model.

[0025] Optionally, for each sample image, the initial forgery detection model can output the predicted forgery probability of each pixel in the sample image being located in the forgery region. For each pixel, if the predicted forgery probability of the pixel is greater than a preset probability threshold, the predicted forgery flag of the pixel is determined to be 1 (i.e., located in the forgery region); otherwise, the predicted forgery flag of the pixel is determined to be 0 (i.e., located in the real region).

[0026] For each sample image, any pixel in the sample image can be used as a reference pixel. If the real forgery identifier of the reference pixel is represented as a real pixel, then the pixel in the sample image whose predicted forgery identifier is represented as a real pixel is used as the first pixel, and the pixel in the sample image whose predicted forgery identifier is represented as a forgery pixel is used as the second pixel.

[0027] If the real forgery identifier of the reference pixel is represented as a forgery pixel, then the pixels in the sample image whose predicted forgery identifier is a forgery pixel are taken as the first pixel, and the pixels in the sample image whose predicted forgery identifier is a real pixel are taken as the second pixel.

[0028] In other words, the first pixel is the pixel whose predicted forgery identifier matches the real forgery identifier of the reference pixel, and the second pixel is the pixel whose predicted forgery identifier does not match the real forgery identifier of the reference pixel.

[0029] For each reference pixel in each sample image, a first loss is determined based on the difference between the reference pixel and the first pixel and the second pixel, respectively.

[0030] Optionally, based on the differences between the reference pixel and the first pixel and the second pixel, a first loss corresponding to the reference pixel is determined, including: Determine the first difference between the reference pixel and each first pixel; Determine the second difference between the reference pixel and each second pixel; The first loss corresponding to the reference pixel is determined based on the difference between the first and second differences.

[0031] Specifically, for each reference pixel, the difference between the reference pixel and each first pixel is taken as the first difference, and the difference between the reference pixel and each second pixel is taken as the second difference.

[0032] The difference between the reference pixel and the first pixel can be the difference between the real forgery identifier of the reference pixel and the predicted forgery identifier of the first pixel, or the difference between the real forgery identifier of the reference pixel and the predicted forgery probability of the first pixel.

[0033] After obtaining the first difference and the second difference, the difference between the first difference and the second difference can be used as the first loss corresponding to the reference pixel.

[0034] For each sample image, a second loss is determined based on the first loss corresponding to each reference pixel in the sample image. For example, the sum of the first losses corresponding to each reference pixel in the sample image can be used as the second loss.

[0035] The total training loss is determined based on the second loss corresponding to each sample image. For example, the sum of the second losses corresponding to each sample image can be used as the total training loss.

[0036] After obtaining the total training loss, the parameters of the initial forgery detection model corresponding to the current training operation can be adjusted based on the total training loss. The adjusted initial forgery detection model is then used in the next training operation, and the above training operation is repeated until a preset training termination condition is met. The preset training termination condition may include the convergence of the total training loss, for example, the total training loss being less than a set value or the total training loss being less than a set value after a set number of consecutive calculations; the preset training termination condition may also be that the number of training iterations reaches a preset number, and this embodiment does not limit this.

[0037] Optionally, the first loss The form can be expressed as a triplet loss function, with the specific formula as follows:

[0038] in, This represents the reference pixel (i.e., the anchor point). This represents the first pixel (i.e., the positive sample). This represents the second pixel (i.e., the negative sample). The hyperparameter representing the safety margin. Represents the feature distance. This indicates a truncation function, meaning that if the value inside the parentheses is negative, it outputs zero; otherwise, it outputs the original value.

[0039] Among them, hyperparameters The minimum distance between negative and positive sample features is specified to ensure clear separation between different classes of features in the feature space, thereby enhancing the model's ability to distinguish highly similar pixels or regions. Even if the distance between positive and negative samples already satisfies the basic order, this can be further enhanced through hyperparameters. This still effectively separates these two types of features, improving robustness and discriminative power. In other words, the aforementioned loss function can shorten the feature distance between the anchor point and positive samples while simultaneously widening the feature distance between the anchor point and negative samples, minimizing the difference between them. However, when both the feature distance between the anchor point and positive samples and the feature distance between the anchor point and negative samples are small, the difference between them will also be small, but it doesn't actually widen the gap between them. Introducing hyperparameters... This forces the model to maintain, under stricter constraints, at least a certain distance between positive and negative pixel features. The distance difference is used to truly separate the two types of features into spatial intervals.

[0040] Figure 2 A schematic diagram of a feature space provided for an embodiment of this disclosure, such as... Figure 2 As shown, circle 1 represents the feature representation of the pixel representing the real pixel by the predicted forgery identifier, circle 2 represents the feature representation of the pixel representing the forgery pixel by the real forgery identifier, circle 3 represents the feature representation of the pixel representing the forgery pixel by the predicted forgery identifier, and circle 4 represents the feature representation of the pixel representing the real pixel by the real forgery identifier. Figure 2 The dashed line in the middle represents the boundary between the unforged / tampered region and the forged / tampered region in the feature space. Figure 2 The arrows in each circle within the left-hand box indicate the direction of change in the mapping of each pixel's feature representation in the feature space during the training process constrained by the total loss. Figure 2 It can be seen that, based on the training total loss obtained from the first loss, which constrains the training of the initial forgery detection model, pixels of the same type represented by forgery identifiers (e.g., circles 1 and 4, circles 2 and 3) will be driven to move closer to each other, while pixels of different types represented by forgery identifiers (e.g., circles 2 and 4) will be forced to move further apart from each other in the feature space (e.g., ... Figure 2The double-headed arrows in the right-hand box indicate how far apart pixels representing different types of forgery identifiers are in the feature space. Figure 2 (Pixels of the same type represented by the forgery identifier in the right box are located in a circle), thereby "amplifying" the model's discrimination error signal for these two types of regions. By mapping the loss distance between forged and non-forged regions to a more distant space, the contrastive learning strategy amplifies the error signal. When the total training loss is smaller, it can forcibly increase the difference between the predicted features of the real region (i.e., the untampered region) and the predicted features of the forged region, thus giving them a clear separation boundary in the feature space.

[0041] In this embodiment, a first loss corresponding to a reference pixel is determined based on the difference between the reference pixel and the first pixel and the second pixel, respectively. Based on the first loss corresponding to each reference pixel in each sample image, a total training loss is determined. The training of the initial forgery detection model is constrained by the total training loss, which can force an increase in the difference between the predicted features of the real region (i.e., the unaltered region) and the predicted features of the forgery region, effectively improving the differentiation effect in high similarity images. This is beneficial for improving the detection effect in forgery detection scenarios of high similarity academic images.

[0042] As an optional embodiment, a second loss corresponding to the sample image is determined based on a first loss corresponding to each reference pixel in the sample image, including: Based on the difference between the predicted fake identifier and the real fake identifier of each pixel in the sample image, the third loss corresponding to the sample image is determined. The second loss is determined based on the third loss and the first loss corresponding to each reference pixel.

[0043] Specifically, for each sample image, the third loss corresponding to the sample image can be determined based on the difference between the predicted fake identifier and the real fake identifier of each pixel in the sample image.

[0044] Based on the first loss and the third loss corresponding to each reference pixel in the sample image, a second loss is determined for the sample image. For example, the sum of the first loss and the third loss corresponding to each reference pixel in the sample image is used as the second loss.

[0045] In this embodiment of the disclosure, during the model training process, by minimizing the training loss, multiple training objectives can be achieved, including the feature distance between the predicted fake identifier of a real pixel and the real fake identifier becoming closer, the feature distance between the predicted fake identifier of a fake pixel and the real fake identifier becoming closer, and the feature distance between the predicted fake identifier of a real pixel and the predicted fake identifier of a fake pixel becoming farther. This enables the model to not only learn pixel-level classification correctness, but also maximize the difference between the real region and the fake region in the overall feature representation, thereby improving the robustness and generalization of detection.

[0046] As an optional embodiment, the multiple sample images include multiple sample forged images; The sample fake images are generated based on the following method: Acquire multiple reference images; the reference images are unforged images. For any reference image, the reference image is processed based on at least one of the following forgery methods, and the processed image is used as a sample forgery image; Forgery methods include: Obtain the description text corresponding to the reference image, rewrite the description text to obtain at least one rewritten text; each rewritten text has the same semantics as the corresponding description text; for each rewritten text, generate an image that matches the rewritten text. The reference image is input into a preset image processing model to obtain a derived image corresponding to the reference image output by the image processing model; the derived image maintains visual consistency with the reference image; A portion of the reference image is used as the sample forgery region. The original image patch corresponding to the sample forgery region in the reference image is adjusted to obtain the adjusted reference image.

[0047] Specifically, multiple sample images may include multiple sample forged images, which can be obtained by processing an unforged reference image.

[0048] The process of generating the sample fake image is as follows: Acquire multiple reference images, where the reference images can be unforged images.

[0049] Optionally, multiple reference images may be acquired, including: Identify multiple different image types; For each image type, multiple images belonging to that image type are obtained from multiple published scientific and technological documents; Multiple reference images are used based on various image types.

[0050] Specifically, various image types can be preset. For example, in the scenario of academic image forgery detection, various image types can include charts, medical imaging (such as CT, color ultrasound, etc.), statistical graphs (such as line graphs, bar graphs, etc.), micrographs, stained micrographs, etc.

[0051] After identifying various image types, for each image type, multiple academic images corresponding to that type can be extracted from a large collection of scientific and technological documents, including academic papers, reports, and dissertations. These multiple images corresponding to each image type are then used as multiple reference images.

[0052] Optionally, scientific and technological literature from different academic fields can be collected, so that the extracted academic images can cover different academic fields, thereby increasing the diversity of samples and helping to improve the generalization of the model.

[0053] Optionally, to ensure the reliability of the extracted reference images, scientific literature can be collected from journals with high reliability.

[0054] In this embodiment of the disclosure, during the training process of the forgery detection model, the training sample set is formed by systematically collecting high-quality real images (i.e., reference images) from a large number of reliable scientific and technological documents, and constructing multiple types of forgery samples based on various forgery methods, thus forming a high-quality training sample set covering multiple categories of academic images and spanning multiple forgery strategies. This training sample set can characterize the feature distribution of real and forged images in academic scenarios, enabling the trained forgery detection model to be applicable to scenarios of academic image forgery detection.

[0055] After obtaining multiple reference images, for any one of the reference images, at least one of the following forgery methods can be used to process the reference image, and the processed image can be used as a sample forgery image.

[0056] The following is a detailed explanation of each method of forgery: Text-constrained forgery: For any reference image, its corresponding descriptive text can be obtained. This descriptive text can explain information within the image, such as annotations in academic papers. After obtaining the descriptive text, it can be rewritten (e.g., based on a large language model) to obtain at least one rewritten text. The rewriting of the descriptive text does not change its semantics; that is, the semantics of the rewritten text and the descriptive text are consistent. For each rewritten text, an image matching the description in the rewritten text can be generated using a pre-defined text-generated image model. This generated image is then used as a sample forgery image.

[0057] Image-based forgery: For any reference image, it can be input into a pre-defined image processing model (e.g., an image-to-image model, including a multimodal large model or a diffusion model) to obtain a derived image. This derived image is then used as a sample forgery. The derived image maintains visual consistency with the corresponding reference image; that is, while the semantics of the derived image and the corresponding reference image may differ, they remain visually consistent, meaning their texture, style, and other visual elements are identical. This forgery method results in a relatively continuous frequency domain between the original and altered images, and the boundaries between the forged and unforged regions are not clearly defined.

[0058] The two forgery methods mentioned above can be understood as global forgery methods. It is also possible to generate sample forged images by forging local parts of the image.

[0059] For any reference image, a portion of the image region in the reference image can be used as a sample forgery region, thereby obtaining the original image block corresponding to the sample forgery region in the reference image. The original image block is then adjusted, for example, by deleting, reconstructing, or modifying the original image block in at least one way. The adjusted reference image is then used as the sample forgery image.

[0060] Optionally, the original image in the reference image corresponding to the sample forgery region is adjusted, including at least one of the following: The original image block corresponding to the sample forgery region is removed from the reference image, and the sample forgery region is reconstructed based on the removed reference image to obtain the reconstructed image block corresponding to the sample forgery region. The original image blocks are modified while retaining the original image blocks corresponding to the sample forgery areas.

[0061] Specifically, after obtaining the original image block corresponding to the sample forgery region, the original image block can be removed from the reference image. For example, it can be removed by adding a mask to the sample forgery region. Based on the reference image with the original image block removed, the sample forgery region is reconstructed to obtain the reconstructed image block. The reference image including the reconstructed image block is used as the sample forgery image.

[0062] It is also possible to modify the original image blocks while retaining them. For example, the original image blocks can be deleted or inserted using graphic design software (such as Photoshop, Paint, etc.), and the reference image including the modified original image blocks can be used as a sample to forge images.

[0063] Optionally, for multiple reference images, the multiple reference images can be divided into different subsets according to a preset ratio. Each subset corresponds to a forgery method. By using multiple different forgery methods to generate multiple sample forgery images, the constructed training sample set can cover forgery images generated by different forgery methods, further improving the richness of training samples. This helps to improve the generalization of model training, enabling the trained forgery detection model to effectively detect forgery images generated based on different forgery methods, thereby improving the image forgery detection effect.

[0064] Optionally, after generating multiple sample forged images based on the reference image, the quality of each sample forged image can be evaluated, and a training sample set can be constructed based on the sample forged images that have passed the quality evaluation to ensure the quality of the training samples.

[0065] For example, each generated sample forged image can be evaluated based on a quality assessment strategy. When a sample forged image meets the quality assessment strategy, it is determined that the sample forged image has passed the quality assessment. For academic image forgery scenarios, the quality assessment strategy may include: the generated image must not contain portraits (academic images typically do not include such content); the generated image should not contain abrupt objects such as people, animals, or landscapes; if the generated image involves background modification, the modified background must blend naturally with the original background; and the text in the generated image must not contain special symbols (such as # or ...). For images generated using local forgery methods, the masked area must not exceed 80% of the entire image; the proportion of the masked area should be controlled between 10% and 70%; for images forged using mask-based region reconstruction methods, there may be edge remnants in the forged area, but the background should be naturally blended in, and such remnants are allowed to be retained.

[0066] Optionally, for sample forged images generated using a global forgery method, the true forgery identifier of a pixel can be determined based on the difference in pixel values ​​between the sample forged image and the reference image corresponding to the generated sample forged image at the same location. When the difference in pixel values ​​is greater than a preset threshold, the true forgery identifier of the pixel is marked as a forged pixel; otherwise, the true forgery identifier of the pixel is marked as a real pixel, thereby obtaining the sample label corresponding to the sample forged image.

[0067] Optionally, for sample forged images generated based on local forgery methods, the real forgery identifiers of each pixel in the sample forgery area can be labeled as forged pixels, and the real forgery identifiers of each pixel in other areas outside the sample forgery area can be labeled as real pixels, thereby obtaining the sample label corresponding to the sample forgery image.

[0068] In this embodiment, by constructing multi-level positive and negative sample pairs between real and fake, different types of fakes, and across fake methods, the model can automatically learn stable and robust fake-sensitive features. This not only enhances the model's ability to perceive fine-grained differences between different fake modes, but also significantly improves the discrimination effect in highly similar images. It can better adapt to fake detection scenarios of highly similar academic images, further enhancing the model's adaptability in scenarios with low annotation and class imbalance. It also enables the model to maintain excellent generalization performance when facing unseen fake types, effectively improving the model's discrimination ability and generalization performance on fake samples across types and sources.

[0069] As an optional embodiment, the sample label corresponding to each sample forged image includes the real forgery method; the predicted forgery detection result of each sample image includes the predicted forgery method corresponding to the sample image; Based on the first loss corresponding to each reference pixel in the sample image, a second loss corresponding to the sample image is determined, including: Based on the difference between the predicted forgery method and the real forgery method of the sample image, the fourth loss corresponding to the sample image is determined; The second loss is determined based on the fourth loss and the first loss corresponding to each reference pixel.

[0070] Specifically, for each sample forged image, the forgery method that generated the sample forged image can be used to construct the sample label of the sample forged image, and the forgery method that generated the sample forged image can be used as the real forgery method in the sample label of the sample forged image.

[0071] During model training, for each sample image, the sample image is input into the initial forgery detection model. The initial forgery detection model detects whether the sample image is a forgery image. Based on the detection of forgery images, the forgery regions in the sample image are identified to obtain the predicted forgery identifier of each pixel in the sample image, as well as the predicted forgery method corresponding to the sample forgery image.

[0072] Based on the difference between the predicted forgery method and the actual forgery method of the sample image, the fourth loss corresponding to the sample image is determined.

[0073] Any pixel in the sample image can be used as a reference pixel. Based on the differences between the reference pixel and the first pixel and the second pixel, the first loss corresponding to the reference pixel is determined. The specific method for determining the first loss is described in the corresponding embodiment above, and will not be repeated here.

[0074] Based on the fourth loss corresponding to the sample image and the first loss corresponding to each reference pixel in the sample image, the second loss corresponding to the sample image is determined. Based on the second loss corresponding to each sample image, the total training loss is determined.

[0075] Optionally, for each sample image, a third loss can be determined based on the difference between the predicted forged identifier and the real forged identifier at each pixel in the sample image. A second loss is then determined based on the third loss, the fourth loss, and the first loss corresponding to each reference pixel in the sample image.

[0076] In this embodiment of the disclosure, by adding the forgery method to the sample label of the sample image during the model training process, the initial forgery detection model learns not only the ability to identify forgery regions during training, but also the ability to identify forgery methods. As a result, the trained forgery detection model can output not only information about forgery regions, but also information about forgery methods in practical applications, providing more comprehensive image forgery information and better meeting the needs of practical applications.

[0077] Figure 3 This is a schematic diagram of a counterfeit detection method provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the method includes: Step S210: Obtain the image to be detected; Step S220: Input the image to be detected into the trained forgery detection model to obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model; The forgery detection model is trained using the model training method provided in any of the above embodiments.

[0078] Specifically, the image to be detected can be an image that needs to be forged. For example, in the scenario of academic image forgery detection, the image to be detected can be an appendix in a submitted paper.

[0079] Optionally, when a document image is acquired, sub-image recognition can be performed on the document image (e.g., a PDF document) to extract illustrations from the document image, and these extracted illustrations can be used as the images to be detected. For example, an object detection model (e.g., YOLOv7) can be fine-tuned and trained, treating each illustration (i.e., sub-image) in the paper as the "target" for object detection, and the sub-image label as the category. Seven categories can be set, including statistical graphs, bar graphs, color graphs, schematic diagrams, contrast graphs, object images, and others, to obtain a sub-image recognition model. The document image is then input into the sub-image recognition model to obtain the category of each sub-image and the location information (e.g., coordinates) of each sub-image output by the sub-image recognition model.

[0080] The image to be detected is input into a trained forgery detection model. The model performs forgery detection on the image, and the forgery detection result corresponding to the image to be detected is obtained from the output of the forgery detection model. The forgery detection result corresponding to the image to be detected can include whether the image to be detected is a forgery image; if the image to be detected is a forgery image, the forgery detection result can also include at least one of the following: forgery region and / or forgery method in the image to be detected.

[0081] Optionally, when the forgery detection model is built based on a diffusion model, the image to be detected is converted into a DCT (Discrete Cosine Transform) frequency domain image. The preprocessed RGB image and the DCT frequency domain image are input into the feature extraction module (High-Resolution Network, HRNet) of the forgery detection model to extract multi-scale features. The preprocessed RGB image is input into the diffusion reconstruction module (DenoisingDiffusion Implicit Model, DDIM) of the forgery detection model for inverse diffusion and forward reconstruction. The diffusion reconstruction error map is calculated. The multi-scale features extracted by HRNet are fused with the diffusion reconstruction error map and input into the fusion and classification module of the forgery detection model for forgery judgment and forgery region identification, resulting in a pixel-level mask of the forgery region to identify the tampered area in the image.

[0082] In this embodiment, the diffusion model is innovatively defined as a novel and efficient forgery clue by utilizing its different reconstruction capabilities for deepfake and real images. By analyzing the diffusion reconstruction error of an image, it is possible to effectively distinguish between deepfake and real images at the generation mechanism level, especially for forgery content generated by the diffusion model itself, demonstrating excellent detection capabilities.

[0083] Furthermore, a multi-level dual-domain feature fusion strategy was adopted to construct a deep forgery detection model that integrates RGB spatial domain features and DCT frequency domain features. This fully utilizes the complementary information of images in the spatial and frequency domains, designing a high-resolution feature extraction branch in the RGB spatial domain and an adaptive feature extraction branch in the DCT frequency domain to capture visual features such as texture, edges, and color, as well as frequency domain features such as compression artifacts. Through multi-level fusion units, the dual-domain features are effectively integrated, significantly improving the model's ability to perceive various forgery traces and its robustness.

[0084] Furthermore, a high-resolution network (HRNet) architecture was adopted as the backbone network for the RGB spatial domain feature extraction branch, and combined with a multi-resolution fusion strategy, to achieve efficient extraction and utilization of multi-scale image features. The HRNet architecture can maintain high-resolution feature representation at all times, avoiding the information loss problem in traditional convolutional neural networks. Combined with the multi-resolution fusion strategy, it can effectively integrate feature information at different scales, improving the model's accuracy in locating tampered regions and its ability to capture details.

[0085] Figure 4 This is a schematic diagram of the structure of a model training device provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, the apparatus of this embodiment may include: The training sample acquisition module 310 is used to acquire a training sample set; the training sample set includes multiple sample images and a sample label for each sample image; the sample label for each sample image includes a real / fake identifier corresponding to each pixel in the sample image; Model training module 320 is used to perform at least one training operation on the initial forgery detection model based on the training sample set, and take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operation includes: For each sample image, a predicted forgery detection result is obtained through an initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image. For each reference pixel in each sample image, a first loss is determined based on the differences between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

[0086] As an optional embodiment, when the model training module determines the first loss corresponding to the reference pixel based on the differences between the reference pixel and the first pixel and the second pixel, it is used to: Determine the first difference between the reference pixel and each first pixel; Determine the second difference between the reference pixel and each of the second pixels; Based on the difference between the first difference and the second difference, the first loss corresponding to the reference pixel is determined.

[0087] As an optional embodiment, when determining the total training loss based on the first loss corresponding to each reference pixel in each sample image, the model training module is used to: For each of the sample images, a second loss corresponding to the sample image is determined based on a first loss corresponding to each reference pixel in the sample image; The total training loss is determined based on the second loss corresponding to each sample image.

[0088] As an optional embodiment, the plurality of sample images includes a plurality of sample forged images; The device also includes a sample forgery image generation module, used for: Acquire multiple reference images; the reference images are unforged images; For any one of the reference images, the reference image is processed based on at least one of the following forgery methods, and the processed image is used as the sample forgery image; The forgery methods include: Obtain the descriptive text corresponding to the reference image, rewrite the descriptive text to obtain at least one rewritten text; each rewritten text has semantic consistency with the corresponding descriptive text; for each rewritten text, generate an image that matches the rewritten text. The reference image is input into a preset image processing model to obtain a derived image corresponding to the reference image output by the image processing model; the derived image maintains visual consistency with the reference image. A portion of the reference image is used as a sample forgery region. The original image block in the reference image corresponding to the sample forgery region is adjusted to obtain the adjusted reference image.

[0089] As an optional embodiment, the sample forgery image generation module, when adjusting the original image corresponding to the sample forgery region in the reference image, is used for at least one of the following: The original image block corresponding to the sample forgery region is removed from the reference image, and the sample forgery region is reconstructed based on the removed reference image to obtain the reconstructed image block corresponding to the sample forgery region. While retaining the original image block corresponding to the sample forgery region, the original image block is modified.

[0090] As an optional embodiment, when the model training module determines the second loss corresponding to the sample image based on the first loss corresponding to each reference pixel in the sample image, it is used to: Based on the difference between the predicted fake identifier and the real fake identifier of each pixel in the sample image, the third loss corresponding to the sample image is determined. The second loss is determined based on the third loss and the first loss corresponding to each reference pixel.

[0091] As an optional embodiment, the sample label corresponding to each sample forged image includes the real forgery method; the predicted forgery detection result of each sample image includes the predicted forgery method corresponding to the sample image; When determining the second loss corresponding to the sample image based on the first loss corresponding to each reference pixel in the sample image, the model training module is used to: Based on the difference between the predicted forgery method and the actual forgery method of the sample image, a fourth loss corresponding to the sample image is determined; The second loss is determined based on the fourth loss and the first loss corresponding to each reference pixel.

[0092] Figure 5 This is a schematic diagram of the structure of a forged image detection device provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the apparatus of this embodiment may include: Image acquisition module 410 is used to acquire the image to be detected; The forgery detection module 420 is used to input the image to be detected into a trained forgery detection model and obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model. The forgery detection model is trained based on the model training method provided in any of the above embodiments.

[0093] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this disclosure correspond to the steps in the method of each embodiment of this disclosure. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, and it will not be repeated here.

[0094] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0095] This disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure. Compared with the prior art, it can achieve: determining a first loss corresponding to a reference pixel based on the difference between a reference pixel and a first pixel and a second pixel, and determining a total training loss based on the first loss corresponding to each reference pixel in each sample image. Constraining the training of the initial forgery detection model based on the total training loss can forcibly increase the difference between the predicted features of the real region (i.e., the unaltered region) and the predicted features of the forged region, effectively improving the differentiation effect in highly similar images, and thus improving the detection effect in forgery detection scenarios of highly similar academic images.

[0096] In one alternative embodiment, an electronic device is provided, such as Figure 6 As shown, Figure 6 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0097] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0098] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0099] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0100] The memory 4003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 4001 to execute them. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0101] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), wearable devices, etc., as well as fixed terminals such as digital TVs, desktop computers, etc.

[0102] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0103] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0104] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0105] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A model training method, characterized in that, include: Obtain a training sample set; the training sample set includes multiple sample images and a sample label for each sample image; the sample label for each sample image includes a real / fake identifier corresponding to each pixel in the sample image; Based on the training sample set, perform at least one training operation on the initial forgery detection model, and take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operation includes: For each sample image, a predicted forgery detection result is obtained through an initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image. For each reference pixel in each sample image, a first loss is determined based on the differences between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

2. The method according to claim 1, characterized in that, The step of determining the first loss corresponding to the reference pixel based on the differences between the reference pixel and the first pixel and the second pixel includes: Determine the first difference between the reference pixel and each first pixel; Determine the second difference between the reference pixel and each of the second pixels; Based on the difference between the first difference and the second difference, the first loss corresponding to the reference pixel is determined.

3. The method according to claim 1, characterized in that, The determination of the total training loss based on the first loss corresponding to each reference pixel in each sample image includes: For each of the sample images, a second loss corresponding to the sample image is determined based on a first loss corresponding to each reference pixel in the sample image; The total training loss is determined based on the second loss corresponding to each sample image.

4. The method according to any one of claims 1 to 3, characterized in that, The multiple sample images include multiple sample forged images; The forged sample image was generated based on the following method: Acquire multiple reference images; the reference images are unforged images; For any one of the reference images, the reference image is processed based on at least one of the following forgery methods, and the processed image is used as the sample forgery image; The forgery methods include: Obtain the descriptive text corresponding to the reference image, rewrite the descriptive text to obtain at least one rewritten text; each rewritten text has semantic consistency with the corresponding descriptive text; for each rewritten text, generate an image that matches the rewritten text. The reference image is input into a preset image processing model to obtain a derived image corresponding to the reference image output by the image processing model; the derived image maintains visual consistency with the reference image. A portion of the reference image is used as a sample forgery region. The original image block in the reference image corresponding to the sample forgery region is adjusted to obtain the adjusted reference image.

5. The method according to claim 4, characterized in that, The adjustment of the original image in the reference image corresponding to the sample forgery region includes at least one of the following: The original image block corresponding to the sample forgery region is removed from the reference image, and the sample forgery region is reconstructed based on the removed reference image to obtain the reconstructed image block corresponding to the sample forgery region. While retaining the original image block corresponding to the sample forgery region, the original image block is modified.

6. The method according to claim 3, characterized in that, The step of determining the second loss corresponding to the sample image based on the first loss corresponding to each reference pixel in the sample image includes: Based on the difference between the predicted fake identifier and the real fake identifier of each pixel in the sample image, the third loss corresponding to the sample image is determined. The second loss is determined based on the third loss and the first loss corresponding to each reference pixel.

7. The method according to claim 4, characterized in that, Each of the forged sample images corresponds to a sample label that includes the real forgery method; The predicted forgery detection result for each sample image includes the predicted forgery method corresponding to the sample image; The step of determining the second loss corresponding to the sample image based on the first loss corresponding to each reference pixel in the sample image includes: Based on the difference between the predicted forgery method and the actual forgery method of the sample image, a fourth loss corresponding to the sample image is determined; The second loss is determined based on the fourth loss and the first loss corresponding to each reference pixel.

8. A method for detecting forged images, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the trained forgery detection model to obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model; The forgery detection model is trained based on the model training method provided in any one of claims 1 to 7.

9. A model training device, characterized in that, include: The training sample acquisition module is used to acquire a training sample set; the training sample set includes multiple sample images and a sample label for each sample image; the sample label for each sample image includes a real / fake identifier corresponding to each pixel in the sample image; The model training module is used to perform at least one training operation on the initial forgery detection model based on the training sample set, and to take the initial forgery detection model that meets the preset training termination condition as the trained forgery detection model. The training operation includes: For each sample image, a predicted forgery detection result is obtained through an initial forgery detection model; the predicted forgery detection result includes the predicted forgery identifier corresponding to each pixel in the sample image. For each reference pixel in each sample image, a first loss is determined based on the differences between the reference pixel and the first pixel and the second pixel, respectively; the predicted forgery identifier of the first pixel is consistent with the real forgery identifier of the reference pixel; the predicted forgery identifier of the second pixel is inconsistent with the real forgery identifier of the reference pixel. The total training loss is determined based on the first loss corresponding to each reference pixel in each sample image. The parameters of the initial forgery detection model are adjusted based on the total training loss, and the adjusted initial forgery detection model is used as the initial forgery detection model for the next training operation.

10. A forged image detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected. The forgery detection module is used to input the image to be detected into a trained forgery detection model and obtain the forgery detection result corresponding to the image to be detected output by the forgery detection model. The forgery detection model is trained based on the model training method provided in any one of claims 1 to 7.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

Images