Image forgery positioning method and device, electronic equipment and readable storage medium

Abstract

Embodiments of the present application disclose an image forgery positioning method and device, electronic equipment and readable storage medium, the method extracts, fuses and maps texture features, semantic features and general visual features in the to-be-processed image based on three different branches in the forgery positioning network, determines the corresponding output mask, can fully capture the texture features and semantic features in the image due to forgery to determine the pixel type of each pixel in the image, thereby realizing accurate forgery positioning based on the pixel type of each pixel in the image; and since the pre-training models of different branches adopt the same structure and the same initialization parameters, and the general visual features in the to-be-processed image are extracted based on the initialization parameters, the general visual features in the image can be retained for forgery positioning, preventing forgetting the inherent attributes of the image in the forgery positioning process, and improving the robustness of forgery positioning.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to an image forgery detection method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, AI-based image editing technology has made breakthrough progress. Prompt-based image editing (i.e., AI-driven image editing) is becoming a new mainstream paradigm. In this paradigm, users only need to input a natural language command, and the model can automatically understand the semantics and generate a modified image, greatly lowering the barrier to image creation. However, at the same time, some malicious users can easily use this technology to forge images, and the forged images generated are extremely realistic and difficult for the human eye to distinguish. Therefore, the development of efficient and accurate AI image forgery detection technology is particularly urgent.

[0003] While existing methods for locating image forgery regions exist, they primarily rely on identifying physical inconsistencies in the forged image (such as mismatches in sensor noise fingerprints (PRUU), interpolation anomalies in color filter arrays (CFA), and double quantization traces from JPEG compression artifacts) or high-frequency forgery features. However, imperative AI image editing typically involves global image reconstruction, which significantly weakens or even completely eliminates the original physical modifications, and also diminishes existing high-frequency features. Therefore, existing image forgery region location methods struggle to cope with the global reconstruction characteristics of imperative AI image editing and cannot accurately identify forged images generated by it. Summary of the Invention

[0004] In view of this, the present invention aims to provide an image forgery localization method, apparatus, electronic device, and readable storage medium to accurately identify forged images generated by instructional AI image editing.

[0005] In a first aspect, embodiments of the present invention aim to provide an image forgery localization method, the method comprising: Obtain the image to be processed; Based on the first branch of the fake location network, the texture features in the image to be processed are extracted to determine the texture feature map; Based on the second branch of the fake location network, semantic features are extracted from the image to be processed to determine the semantic feature map; The third branch in the fake location network extracts common visual features from the image to be processed to determine a visual feature map; the first branch, the second branch and the third branch use the same pre-trained model with the same structure and the same initialization parameters, and the third branch extracts common visual features from the image to be processed based on the initialization parameters; The texture feature map, the semantic feature map, and the visual feature map are fused to determine the target feature map; Based on the classification head in the fake localization network, feature mapping is performed on the target feature map to determine the corresponding output mask, so as to determine the fake localization result of the image to be processed based on the pixel distribution in the output mask; the output mask is used to characterize the pixel type of each pixel in the image to be processed, the pixel type is real pixel or fake pixel, the real pixel is used to characterize the corresponding pixel is located in the real area, and the fake pixel is used to characterize the corresponding pixel is located in the fake area.

[0006] Secondly, embodiments of the present invention aim to provide an image forgery detection device, the device comprising: An image acquisition unit is used to acquire the image to be processed. The feature extraction unit is used to extract texture features from the image to be processed based on the first branch of the fake localization network to determine a texture feature map; extract semantic features from the image to be processed based on the second branch of the fake localization network to determine a semantic feature map; extract general visual features from the image to be processed based on the third branch of the fake localization network to determine a visual feature map; and perform feature fusion on the texture feature map, the semantic feature map, and the visual feature map to determine a target feature map. The forgery localization unit is used to perform feature mapping on the target feature map based on the classification head in the forgery localization network, determine the corresponding output mask, and determine the forgery localization result of the image to be processed based on the pixel distribution in the output mask; the output mask is used to characterize the pixel type of each pixel in the image to be processed, the pixel type is a real pixel or a forged pixel, the real pixel is used to characterize that the corresponding pixel is located in the real area, and the forged pixel is used to characterize that the corresponding pixel is located in the forged area.

[0007] Thirdly, embodiments of the present invention aim to provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any of the preceding claims.

[0008] Fourthly, embodiments of the present invention aim to provide a computer program product, the computer program product including a computer program / instruction, which, when executed by a processor, implements the method described in any of the preceding claims.

[0009] Fifthly, embodiments of the present invention aim to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0010] The technical solution of this invention constructs a forgery localization network containing multiple branches. Based on the texture features, semantic features, and general visual features in the image to be processed by the three different branches of the forgery localization network, feature extraction, feature fusion, and feature mapping are performed to determine the corresponding output mask. This can fully capture the texture features and semantic features generated by forgery in the image to determine the pixel type of each pixel in the image, thereby achieving accurate forgery localization based on the pixel type of each pixel in the image. Furthermore, since different branches use the same pre-trained model with the same structure and the same initialization parameters and extract the general visual features in the image to be processed based on the initialization parameters, the general visual features in the image can be retained for forgery localization, preventing the inherent attributes of the image from being forgotten during the forgery localization process and improving the robustness of forgery localization. Attached Figure Description

[0011] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which: Figure 1 This is a flowchart of the image forgery localization method according to an embodiment of the present invention; Figure 2 This is an architecture diagram of a fake location network according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the determination of texture feature maps according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the process of determining a fake location network according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the determination of a fake location network according to an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the acquisition of the training dataset according to an embodiment of the present invention; Figure 7 This is a flowchart of the iterative training process according to an embodiment of the present invention; Figure 8 This is a schematic diagram of an image forgery location device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0012] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0013] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0014] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0015] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0016] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0017] Existing image forgery region localization methods include traditional detection methods based on physical features and statistical laws (such as Mersorch and PIM-Net) and deep learning detection methods based on noise domain features (such as MVSS-Net, NFA-ViT, and TruFor).

[0018] Traditional detection methods rely on one or more inherent physical properties of imaging devices or statistical regularities in the image encoding and decoding process to determine forged regions in an image. For example, methods based on sensor fingerprints (PRUU) utilize non-uniform photoelectric response noise (photoresponse non-uniformity) generated during camera sensor manufacturing as a "fingerprint" to detect regions with missing or mismatched PRUU values. Methods based on CFA interpolation traces detect whether the periodic statistical features introduced during demosaicing are disrupted. Methods based on JPEG compression artifacts analyze the discontinuities of "double quantization" or block effects generated during multiple image saving processes. These methods are suitable for image editing operations that modify only the target editing area (i.e., the forged region) while leaving the non-edited area (i.e., the real region) unchanged (such as traditional Photoshop editing and masked AI editing). However, for instruction-based AI image editing that reconstructs the entire image, the performance of these methods degrades significantly because the non-edited areas also change at the pixel level, rendering them unsuitable. Furthermore, because the microscopic statistical features in images are very fragile, once they are subjected to lossy compression, scaling, or filtering during transmission on social media, the faint traces can be masked, resulting in a high rate of missed detection in real-world applications.

[0019] Deep learning detection methods heavily rely on high-frequency forgery features. They typically assume that forgery traces exist in the high-frequency components of an image and employ specific signal processing techniques to suppress the semantic content of the image, forcing the network to focus on extracting high-frequency noise such as edges and textures. They attempt to locate forged regions by finding abrupt boundaries in the noise distribution. When traditional "stitching" or "copy-and-move" methods are used to tamper with images, significant noise discontinuities are left at the boundaries, and deep learning detection methods perform well in such scenarios. However, because AI-generated images are usually reconstructed by the model, high-frequency texture information is weak; existing deep learning detection methods struggle to extract effective forgery traces, thus failing to effectively locate forged regions. Furthermore, images in real-world scenarios often undergo post-processing operations (such as JPEG compression, cropping, scaling, etc.) or lossy compression, scaling, or filtering during social media transmission; however, the feature extraction capabilities of deep learning detection methods rapidly degrade when faced with degraded images, failing to meet the needs of practical applications.

[0020] Furthermore, the inventors discovered that in AI image editing driven by prompts, the unedited area (i.e., the real area) retains the original image content in terms of structure and semantics, while the edited area (i.e., the fake area) is synthesized entirely based on editing prompts and context. This difference leads to subtle inconsistencies in texture details and semantic distribution between the real and fake areas. Therefore, this invention provides an image forgery localization method. By utilizing texture and semantic features in the image, combined with general visual features, it achieves image forgery localization, enabling accurate identification of forged images generated by instruction-based AI image editing.

[0021] Figure 1 This is a flowchart of an image forgery localization method according to an embodiment of the present invention. Figure 1 As shown, the image forgery localization method in this embodiment includes the following processing steps.

[0022] In step S110, the image to be processed is acquired.

[0023] In this embodiment, the image to be identified is the image that needs to be localized for image forgery. After acquiring the image to be processed, it is input into a pre-trained forgery localization network. The forgery localization network identifies inconsistencies in texture details and semantic distribution between the real and forged regions in the image to be processed generated by image editing, thereby achieving image forgery localization.

[0024] Figure 2 This is an architecture diagram of a fake location network according to an embodiment of the present invention. Figure 2 As shown, the forgery localization network includes a first branch, a second branch, and a third branch. These three branches employ a pre-trained model with the same structure and initialization parameters. Here, the pre-trained model refers to an image segmentation model that can learn general features after training the network using a general dataset. Furthermore, to overcome the problem of weakened forgery traces and inaccurate forgery localization caused by pixel reconstruction in instructional AI image editing, the forgery localization network in this embodiment uses a contrastive learning mechanism for iterative learning to determine the forgery location. This allows the forgery localization network to learn the relative distance between the features of the real and forged regions within the same image in the feature space. Thus, even when AI reconstruction makes the forged region highly realistic in visual and statistical features, the contrastive loss mechanism still enables the forgery network to accurately locate the forged region, significantly improving the detection capability for high-fidelity AI-forged images.

[0025] Meanwhile, to prevent the feature extraction capability of the forgery localization network from degrading during the image forgery localization process, the third branch in this embodiment always uses fixed initialization parameters. During the training process of training the forgery localization network, only the first and second branches are iteratively trained. This allows the forgery localization network to retain a solid understanding of the inherent attributes (such as edges, textures, object categories, and other visual features) in the image during the forgery localization process, preventing overfitting on specific forgery data. It also enables the forgery localization network to maintain extremely high detection stability when facing destructive post-processing such as JPEG compression and cropping, thereby improving the robustness of forgery localization.

[0026] In step S120, texture features in the image to be processed are extracted based on the first branch in the fake localization network to determine the texture feature map.

[0027] Optionally, considering that image texture features are usually hidden in high-frequency features (i.e., areas in the image where pixel values ​​(color or brightness) change drastically), this embodiment sets a high-pass filter before the first branch. The high-pass filter filters out high-frequency features as the feature basis for extracting texture features, and then extracts texture features through the first branch to better extract texture features in the image and provide a reliable basis for subsequent image forgery location.

[0028] Figure 3 This is a flowchart illustrating the determination of texture feature maps according to an embodiment of the present invention. For example... Figure 3 As shown, in this embodiment, the extraction of texture features from the image to be processed and the determination of the corresponding texture feature map are achieved through the following method.

[0029] In step S310, the image to be processed is input into a high-pass filter for filtering to determine the preprocessed image.

[0030] In this embodiment, a high-pass filter is used to filter the image to be processed, which can effectively suppress smooth, slowly changing low-frequency features (such as large areas of background) in the image, while retaining and highlighting high-frequency features (such as edges, fine textures and details) in the image, thereby obtaining a preprocessed image with enhanced high-frequency features.

[0031] In step S320, the preprocessed image is input into the first branch of the fake localization network for feature extraction to determine the texture feature map.

[0032] In this embodiment, after determining the preprocessed image, the texture features in the preprocessed image are extracted by the first branch of the forgery localization network to capture micro-texture anomalies caused by image editing and determine the corresponding texture feature map, so that forgery localization can be performed based on the inconsistency in texture detail distribution between the real area and the forged area caused by image editing.

[0033] In step S130, semantic features in the image to be processed are extracted based on the second branch in the fake localization network to determine the semantic feature map.

[0034] In this embodiment, the second branch extracts semantic features from the image to be processed to capture macroscopic semantic anomalies caused by image editing and determine the corresponding semantic feature map. This enables forgery localization based on the inconsistency in semantic distribution between real and fake regions caused by image editing. Furthermore, in cases where texture features are lost due to compression or other operations, semantic features can be used as an effective supplement to texture features, thereby achieving accurate forgery localization.

[0035] In step S140, general visual features in the image to be processed are extracted based on the third branch in the fake location network to determine the visual feature map.

[0036] In this embodiment, the first, second, and third branches use a pre-trained model with the same structure and initialization parameters, and the third branch extracts general visual features from the image to be processed based on the initialization parameters. General visual features include edge features, texture features, object category features, and other general image features. By extracting general visual features from the image to be processed through the third branch, the forgery localization network can maintain a robust understanding of the inherent attributes of the image during the image forgery localization process, preventing overfitting on specific forged data. Furthermore, it enables the forgery localization network to maintain extremely high detection stability even when faced with destructive post-processing such as JPEG compression and cropping, thereby improving the robustness of image forgery localization.

[0037] Optionally, steps S120, S130 and S140 in this embodiment are executed in parallel. When extracting the corresponding image features in parallel through the three branches in the forgery localization network, the order of steps given here is only for ease of introduction.

[0038] In step S150, feature fusion is performed on the texture feature map, semantic feature map, and visual feature map to determine the target feature map.

[0039] In this embodiment, after determining feature maps of different dimensions in the image to be processed, namely texture feature maps, semantic feature maps, and visual feature maps, through different branches in the forgery localization network, feature fusion of feature maps of different dimensions can achieve complementarity between multi-domain information. This allows for comprehensive image forgery localization by combining the abnormal features generated by image forgery in different dimensions, thereby improving the accuracy of image forgery localization. Furthermore, feature fusion can enhance the forgery localization network's ability to perceive the boundaries of forgery regions, alleviate semantic degradation problems, and further achieve precise localization of the edges of forgery regions.

[0040] Optionally, the first, second, and third branches in this embodiment can adopt Seg-Former, U-Net, DINO series models (such as Mask DINO), or other available network architectures. Taking the first, second, and third branches using the Seg-Former model as an example, the Seg-Former model includes an encoder and a decoder. In this embodiment, the encoder in the first, second, or third branch is used to determine the texture feature map, semantic feature map, and visual feature map corresponding to the image to be processed, respectively. Then, any decoder in the first, second, or third branch is used as a shared decoder to perform feature fusion on the texture feature map, semantic feature map, and visual feature map corresponding to the image to be processed to determine the target feature map.

[0041] Furthermore, in this embodiment, during feature fusion, the texture feature map, semantic feature map, and visual feature map are concatenated along the channel dimension to determine the target feature map. Feature concatenation refers to directly splicing multiple feature vectors along a certain dimension (such as a channel or feature axis) to form a longer joint feature vector, thereby fusing feature information from different sources or types.

[0042] In step S160, feature mapping is performed on the target feature map based on the classification head in the fake localization network to determine the corresponding output mask. The fake localization result of the image to be processed is then determined based on the pixel distribution in the output mask. The output mask is used to characterize the pixel type of each pixel in the image to be processed, which is either a real pixel or a fake pixel. Real pixels are used to indicate that the corresponding pixel is located in the real region, and fake pixels are used to indicate that the corresponding pixel is located in the fake region.

[0043] In this embodiment, the output mask is a two-dimensional matrix with the same size as the image to be processed, containing information about the fake and real regions. Specifically, each element in the output mask (i.e., the two-dimensional matrix) represents a pixel, and the position of each element in the two-dimensional matrix corresponds to the position of the corresponding pixel in the image. Each pixel value (i.e., the element value in the two-dimensional matrix) is used to characterize whether the corresponding pixel is a real pixel or a fake pixel. By determining the pixel type, it is possible to identify whether the pixel belongs to a real or fake region, thereby determining whether a fake region exists in the image to be processed and the location of the fake region, thus realizing image forgery localization in instruction-based AI image editing.

[0044] Optionally, the output mask in this embodiment can be a binary mask or a probabilistic mask. When using a binary mask, the mask image contains only two pixel values ​​(usually 0 and 255, or 0 and 1); real pixels are usually represented by black (pixel value 0), indicating that the pixel at that location is an unedited background or real content pixel in the original image; fake pixels are usually represented by white (pixel value 255 or 1), indicating that the pixel at that location is a fake pixel that has been spliced, smeared, or generated by AI in the original image. When using a probabilistic mask, the pixel values ​​of the mask are continuous values ​​between 0 and 1. The closer a pixel is to 0, the more certain it is that it is a real pixel; the closer a pixel is to 1, the more certain it is that it is a fake pixel. Intermediate values ​​may represent edges or transitional pixels with uncertain annotations. Based on this, this embodiment can set a pixel threshold (e.g., 0.5), and determine pixels with pixel values ​​less than the pixel threshold as fake pixels, while determining pixels with pixel values ​​reaching the pixel threshold as real pixels.

[0045] Optionally, when determining the output mask corresponding to the image to be processed by performing feature mapping on the target feature map through the classification head in the forgery localization network, the classification head in this embodiment includes multiple sets of convolutional layers and upsampling layers. Multiple sets of convolutional layers are used to perform multi-level feature mapping on the target feature map to determine the feature map. The upsampling layer is used to reduce the dimensionality of the feature map in the channel dimension to generate the output mask. Therefore, by determining the output mask of the image in this embodiment, pixel-level forgery localization can be achieved, improving the detection accuracy of image forgery localization and thus meeting the stringent requirements for image forgery localization in highly sensitive scenarios such as judicial evidence collection, news verification, and insurance claims.

[0046] Optionally, after determining the output mask of the image to be processed, this embodiment can determine the global statistical results of the forged pixels in the output mask by traversing each pixel in the output mask, and output the image forgery positioning result by judging whether the global statistical results meet the judgment conditions.

[0047] Furthermore, in this embodiment, the global statistical result can be the total number of forged pixels in the output mask or their proportion in all pixels. The corresponding judgment condition is that the total number of forged pixels reaches a preset number or the proportion of forged pixels in all pixels reaches a preset ratio. In addition, in this embodiment, a classifier can be set to determine the statistical result of forged pixels in the output mask and determine whether the global statistical result meets the judgment condition. When the global statistical result meets the judgment condition, an image forgery location result of "no forgery exists" or "the image is a real image" is output; while when the global statistical result of forged pixels in the output mask does not meet the judgment condition, an image forgery location result of "forgery exists" is output.

[0048] Optionally, this embodiment can also include a connected component analysis module. Using algorithms such as connected component analysis, the output mask can be cleaned before or simultaneously with determining the global statistical results of the forged pixel information. This cleans the mask by removing isolated pixels with excessively small areas and statistically analyzing key geometric information such as the number of connected components formed by the forged region, the area of ​​the maximum connected component, and the coordinates of the smallest bounding rectangle. Then, based on the statistically obtained key set information, the corresponding image forgery localization result is output. For example, when the number of connected components in the statistically obtained key geometric information is 0 or the area of ​​the maximum connected component is less than the area threshold, it is determined that the image to be processed only contains minor noise or insignificant flaws. In this case, the image forgery localization result of "no forgery exists" or "the image is a real image" is output. Conversely, when the area of ​​the maximum connected component in the key geometric information reaches the area threshold, the image forgery localization result of "forgery exists" is output.

[0049] Furthermore, this embodiment can also set up a visualization output module to convert the statistical information (such as global statistical results and key geometric statistical information) corresponding to the output mask into structured data for visualization when outputting the image forgery location result of "forgery exists". For example, the boundary outline is drawn on the image to be processed according to the coordinates of the statistically determined forgery area, or the mask is superimposed on the image to be processed with a semi-transparent color, thereby generating the final image forgery location result. Thus, this embodiment breaks the "black box" of AI-edited image detection by visually displaying the image forgery location result. It can not only intuitively display the forgery area in the image, improving the interpretability and transparency of the image forgery location result; but also provide clear and reliable evidence support in actual business scenarios (such as judicial evidence collection, news verification, and insurance claims), improving the efficiency of manual review and judgment; and it can also assist in the debugging and optimization of the forgery location network, further enhancing the generalization ability and usability of image forgery location.

[0050] The technical solution of this embodiment extracts, fuses, and maps features from texture features, semantic features, and general visual features in the image to be processed through three different branches in the forgery localization network, and determines the corresponding output mask. This can fully capture the texture features and semantic features generated by forgery in the image to determine the pixel type of each pixel in the image, thereby achieving accurate forgery localization based on the pixel type of each pixel in the image. Furthermore, since different branches use the same pre-trained model with the same structure and the same initialization parameters and extract general visual features in the image to be processed based on the initialization parameters, the general visual features in the image can be retained for forgery localization, preventing the inherent attributes of the image from being forgotten during the forgery localization process and improving the robustness of forgery localization.

[0051] Furthermore, this embodiment will introduce a method for determining a fake location network.

[0052] Figure 4 This is a schematic diagram illustrating the process of determining a fake location network according to an embodiment of the present invention. Figure 4 As shown, in this embodiment, an initial localization network is first constructed, and then the fake localization network is determined by training the initial localization network. The initial localization network has the same structure as the aforementioned fake localization network, including three parallel branches (first branch, second branch, and third branch) and a classification head. The first branch, acting as a "texture branch," processes high-frequency features in the input image, capturing fine texture features related to fakery and outputting a texture feature map. The second branch, acting as a "semantic branch," extracts fake features at the semantic level of the input image and outputs a semantic feature map. The third branch, acting as a "forget-proof branch," extracts general visual features in the input image and outputs a visual feature map to maintain the general knowledge learned from the pre-training stage and mitigate catastrophic forgetting, making the fake region mask output by the network coherent and improving the accuracy of fake localization. Furthermore, the first, second, and third branches are all initialized with the same pre-trained model and share the same initialization parameters, but the third branch keeps its parameters unchanged during training, using frozen initialization parameters.

[0053] Optionally, the pre-trained model in this embodiment can be a network model with Seg-Former, U-Net, DINO series models (such as Mask DINO) or other available architectures.

[0054] Meanwhile, in this embodiment, the classification head is used to perform feature mapping on the feature maps obtained by feature fusion of the texture feature maps, semantic feature maps, and visual feature maps output from the first, second, and third branches, and output a prediction mask for the input image. Optionally, the classification head in this embodiment includes multiple sets of convolutional layers and upsampling layers. The convolutional layers are used for feature dimension mapping and refinement—including extraction, transformation, and refinement—using convolutional kernels to perform nonlinear transformations and spatial feature fusion on the magnified features, thereby achieving feature mapping. The upsampling layers are used to restore the dimensions of the feature maps after feature mapping, thereby outputting a prediction mask.

[0055] Furthermore, to enable the trained fake location network to overcome the problem of weakened fake traces and inaccurate fake location due to pixel reconstruction in instructional AI image editing, this embodiment proposes a training strategy based on intra-image contrastive learning when training the initial location network to determine the fake location network. Based on the contrast loss between the features of the real region and the fake region within the image, the first and second branches in the initial location network are iteratively trained and optimized. This can guide the initial location network to distinguish between fake and real regions in the image, ensuring that the initial location network can learn discriminative features after training, thereby achieving accurate location of the fake region.

[0056] Specifically, in this embodiment, the contrast loss function is applied to the iterative training of the initial localization network to realize the intra-image contrast learning mechanism, so as to force the initial localization network to learn the relative distance between the real region features and the fake region features in the feature space within the same image. In this way, even when the AI ​​reconstruction makes the fake region highly realistic in visual and statistical features, the trained fake localization network can still accurately locate the fake region based on the contrast loss, which significantly improves the detection capability of high-fidelity AI fake images.

[0057] Optionally, considering that the forged region usually occupies only a small portion of the total image area (e.g., less than 20%), the forged region localization task inherently suffers from class imbalance (i.e., background pixels far outnumber the tampered pixels) and difficulty in boundary definition. Therefore, this embodiment also introduces a segmentation loss function into the loss function. Furthermore, since relying solely on the binary cross-entropy (BCE) loss commonly used in semantic segmentation may lead to suboptimal training, the segmentation loss function in this embodiment employs a hybrid segmentation loss combining Focal Loss and Dice Loss to provide supervision signals to the forged localization network. This effectively alleviates the class imbalance problem and forces the network to focus on difficult-to-classify categories, thereby improving training stability.

[0058] Figure 5 This is a flowchart illustrating the determination of a spoofed location network according to an embodiment of the present invention. Figure 5 As shown, the fake location network is identified in this embodiment using the following method.

[0059] In step S510, a training dataset is obtained. The training dataset includes multiple sample images and the real mask corresponding to each sample image. The real mask includes real pixels and / or fake pixels.

[0060] In this embodiment, the ground truth mask of the image is a two-dimensional matrix with the same size as the sample image. Each element value in the ground truth mask represents the pixel type of the corresponding pixel in the sample image, and the actual real area and fake area in the sample image are represented by the pixel type of each pixel.

[0061] Optionally, considering that AI image editing usually only edits local areas of an image, in order for the determined forgery localization network to effectively identify forged regions in an image, the training dataset in this embodiment includes not only sample images with both real and forged regions (the corresponding real mask includes both real and forged pixels), but also sample images with only real regions (the corresponding real mask includes only real pixels). This can reduce the amount of network training data processing while ensuring the integrity of the sample data, which is beneficial to improving the network training efficiency.

[0062] Optionally, in this embodiment, an existing image dataset can be directly used to obtain the training dataset, thereby improving the efficiency of obtaining the training dataset. Alternatively, this embodiment can use image data from image datasets with built-in masks (such as DEAL-300K, PromptForge-350K, FragFake, etc.) as the training dataset; or first obtain image data from an existing image dataset without native masks (such as the X2Edit original dataset), and then mask these data to obtain the training dataset.

[0063] Furthermore, since the existing image datasets may have limited coverage of instructional AI image editing methods, this embodiment uses an image dataset without a native mask to obtain the training dataset.

[0064] Figure 6 This is a schematic diagram illustrating the acquisition of the training dataset according to an embodiment of the present invention. Figure 6 As shown, the training dataset is obtained in this embodiment using the following method.

[0065] In step S610, the original dataset is obtained. The original dataset includes multiple pairs of original images. Each pair of original images includes an original image and an edited image. The edited image is an image generated after editing the original image based on the instruction-based AI image editing method.

[0066] Optionally, the original dataset in this embodiment uses images from the X2Edit dataset and the Pico-Banana dataset. The edited images in the X2Edit dataset are generated by the BAGEL, Kontext, and Step1X editing methods, and the images in the Pico-Banana dataset are generated by the Nano-Banana editing method. BAGEL, Kontext, Step1X, and Pico-Banana are all instruction-based AI image editing methods.

[0067] Optionally, considering that imperative AI image editing focuses on local editing (i.e., modifying only specific regions or objects in an image), this embodiment filters images in the X2Edit and Pico-Banana datasets based on data editing tasks. Images with global editing tasks such as style transfer, color correction, and perspective transformation are filtered out, while images with preset editing tasks, such as object replacement, addition, removal, material conversion, pose change, text editing, portrait editing, and object state transformation, are retained. Therefore, this embodiment determines the edited images under preset editing tasks by filtering, and uses the edited images under preset editing tasks and their corresponding original images as original image pairs to obtain the original dataset. This reduces the amount of network training data processing while ensuring the integrity of the sample data, thus improving network training efficiency.

[0068] Furthermore, in this embodiment, after obtaining the original dataset, the images in the original dataset will also be preprocessed. For example, the images in the original dataset will be uniformly converted to images of a preset size (such as 512×512). The original images include the initial images, and the images in the original dataset will be augmented through random JPEG compression and cropping operations.

[0069] In step S620, pixel alignment is performed on the original image and the edited image in each original image pair.

[0070] In this embodiment, since some image editing models (such as Nano-Banana and Kontext) perform pixel cropping operations on the input image internally, resulting in spatial misalignment and disrupting the subsequent mask annotation process, the original image and the edited image in the original image pair are first aligned at the pixel level when constructing the dataset. The pixel alignment process includes: key point extraction and matching, affine matrix estimation, transformation and cropping.

[0071] Taking pixel alignment of a pair of original images as an example, this embodiment can optionally employ ORB, FAST, or other available image feature extraction and matching algorithms for keypoint extraction and matching. Taking the ORB algorithm as an example, keypoints are extracted from both the original and edited images. Each keypoint is represented by spatial coordinates and a (32-byte binary) descriptor (used to encode local features). Then, a brute-force matcher based on Hamming distance (BFMatcher) is used to match the two sets of keypoints. Furthermore, during the feature matching stage, this embodiment also applies a Low's ratio test to eliminate matching results where the distance ratio (i.e., the ratio between the nearest match and the second nearest match) exceeds a preset value (e.g., 0.75), to ensure the reliability of the matching results.

[0072] Optionally, this embodiment can employ the RANSAC algorithm, LMDS (Least Median Method), or other available algorithms for affine matrix estimation. Further, this embodiment uses the RANSAC algorithm, which can handle potential noise and outliers in the matching keypoints, thereby robustly estimating the model parameters supported by the maximum number of inliers, i.e., the affine transformation matrix. Specifically, by applying the RANSAC algorithm to the keypoint pairs determined after keypoint extraction and matching, and through iterative sampling and error threshold determination, a coarse transformation matrix A and a set of inlier indicators, denoted as idx, can be obtained. inliers idx inliers This is a Boolean list used to characterize whether the corresponding keypoint pair is consistent with the coarse transformation matrix A (i.e., whether it is an interior point). Then, using idx... inliers By selecting interior point pairs of key points and solving a least-squares problem over this subset of interior points, a refined affine transformation matrix A is obtained. refined The affine matrix defines the coordinate mapping relationship between the original image and the edited image.

[0073] Furthermore, after determining the affine transformation matrix, this embodiment performs transformation and cropping. The original image is deformed according to the affine transformation matrix to align the coordinate systems of the original and edited images. Additionally, since affine transformations may introduce black borders at image boundaries, the same cropping operation is performed on both the transformed original and edited images to remove the black borders.

[0074] Optionally, to ensure the quality of sample data in the training dataset, this embodiment also removes low-quality image pairs from the original dataset after pixel alignment. Low-quality image pairs include original image pairs containing original or edited images with fewer than a preset number of extracted keypoints (e.g., 10), original image pairs with fewer than a preset number of matched keypoint pairs (e.g., 10), and original image pairs with an inlier rate lower than a preset value (e.g., 60%). Further, this embodiment performs masking annotation on the remaining original image pairs after removal to determine the training dataset.

[0075] In step S630, the semantic features in the original image and the edited image in each aligned original image pair are masked to determine the corresponding sample image. The original image and the edited image in the aligned original image pair are masked based on semantic features to determine the corresponding sample image.

[0076] In this embodiment, by masking the original image and the edited image in the aligned original image pair based on the semantic features in the image, the corresponding sample image is determined. This enables the differentiation and labeling of fake and real regions in the image, with pixels in the real region labeled as real pixels and pixels in the fake region labeled as fake pixels.

[0077] Optionally, since AI image editing based on prompt words often leads to global pixel-level changes, simple pixel-level difference calculations become ineffective for accurate mask estimation. Therefore, by using DINO v3, dense semantic features are first extracted from the aligned original and edited images, and then the cosine similarity at the corresponding spatial locations is calculated. This effectively avoids the pixel-level differences introduced into the real area by AI image editing based on prompt words, thereby achieving robust mask annotation.

[0078] Optionally, in order to automatically generate the mask, this embodiment uses the Otsu method to operate on the similarity map, determine the similarity threshold, and determine the regions with similarity below the similarity threshold as fake regions, and the regions with similarity reaching the similarity threshold as real regions.

[0079] It should be noted that other available masking methods can also be used in this embodiment to mask the original image and the edited image in the original image pair to determine the sample image; the example given here is only an example.

[0080] In step S640, the training dataset is obtained from the corresponding sample images based on each original image.

[0081] In this embodiment, after determining the sample images corresponding to each original image in the original dataset using the aforementioned method, these sample images are added to the training dataset so that the training dataset can be used for subsequent network training.

[0082] In step S520, the initial localization network is iteratively trained based on the training dataset and the preset loss function.

[0083] In this embodiment, after obtaining the training dataset, the initial localization network is iteratively trained based on the training dataset and a preset loss function.

[0084] Optionally, the preset loss function in this embodiment includes a contrastive loss function and a segmentation loss function. The contrastive loss function is used to maximize the distance between features in the real region and the fake region in the feature space and minimize the distance between features in the fake region. The segmentation loss function is used to provide supervision signals for training the fake localization network. Thus, in this embodiment, the optimization process of the initial localization network is guided by summing the contrastive loss corresponding to the contrastive loss function and the segmentation loss corresponding to the segmentation loss function into a total loss, thereby iteratively updating the initial localization network to complete the training.

[0085] Meanwhile, the preset loss function is expressed by the following formula:

[0086] in, Indicates the total loss; Indicates comparative loss; This represents the dice loss, used to guide the network to prioritize the boundary between fake and real areas, thereby improving the accuracy of the positioning contour. This represents focus loss, which automatically reduces the weights of "simple samples" (large amounts of real background areas) that are easily classified correctly, so that the network focuses its training efforts on "hard samples" (pixels in fake regions) that are difficult to classify, in order to prevent the network from being disturbed by background pixels and ensure that the network can keenly capture small or hidden fake traces. , These represent the predicted mask and the actual mask, respectively. , , These are the weighting coefficients for contrast loss, dice loss, and focus loss, which can be set based on experience.

[0087] Optionally, the contrastive loss function in this embodiment is expressed by the following calculation formula:

[0088] in, Represents comparative loss; , These represent the feature vector sets of the fake region and the real region, respectively. , These represent the number of pixels in the fake area and the real area, respectively; Cosine similarity function; This represents the temperature parameter, used to control the model's sensitivity to differences in similarity between samples.

[0089] Alternatively, other forms of contrastive loss functions can be used in this embodiment, as long as they can achieve the purpose of narrowing the intra-class distance and widening the inter-class distance, the same objective of this invention can be achieved. Meanwhile, to balance computational cost and training efficiency, when the number of pixels in the sample images used during training exceeds a preset threshold, a portion of pixels will be randomly sampled.

[0090] Furthermore, in this embodiment, when iteratively training the initial localization network based on the training dataset and a preset loss function, the sample data in the training dataset is divided into a training set and a test set according to a preset ratio (e.g., 95:5). The training set is then divided into multiple batches and input into the initial localization network, with each batch corresponding to one iteration. In each iteration, the initial localization network processes the sample images in each batch to obtain the predicted mask of the sample image. Then, the preset loss function is used to calculate the total loss between the predicted mask and the true mask of the sample image. Subsequently, the gradient is calculated using the backpropagation algorithm, and the model parameters in the initial localization network are iteratively updated according to the optimization algorithm (the updated model parameters are the initialization parameters or the model parameters from the previous iteration) until the training completion condition is met.

[0091] Figure 7 This is a flowchart of the iterative training process according to an embodiment of the present invention. This embodiment uses one iteration as an example, combined with... Figure 4 and Figure 7 As shown, the specific steps involved in iteratively training the initial localization network based on the training dataset and a preset loss function are as follows.

[0092] In step S710, feature extraction is performed on the sample image based on different branches in the initial localization network to determine the first feature map, the second feature map, and the third feature map.

[0093] In this embodiment, a first feature map is determined by extracting texture features from the sample image based on the first branch of the initial localization network; a second feature map is determined by extracting semantic features from the sample image based on the second branch of the initial localization network; and a third feature map is determined by extracting general visual features from the sample image based on the third branch of the initial localization network.

[0094] In step S720, the first feature map and the second feature map are fused to determine the first fused feature map.

[0095] In this embodiment, feature fusion is achieved by splicing the first feature map and the second feature map along the channel dimension, and fused features are generated.

[0096] In step S730, the first fused feature map is divided into a set of real features and a set of fake features according to the real mask corresponding to the sample image.

[0097] In this embodiment, the first fused feature map is divided into a set of real features and a set of fake features, using the real mask corresponding to the sample image as the spatial index.

[0098] In step S740, the corresponding contrast loss is determined based on the real feature set, the fake feature set, and the contrast loss function.

[0099] In this embodiment, the contrast loss between the real feature set and the fake feature set is determined by the contrast loss function, so as to optimize the first branch network and the second branch network based on the contrast loss.

[0100] In step S750, the first feature map, the second feature map, and the third feature map are fused to determine the second fused feature map.

[0101] In this embodiment, the feature fusion of the first feature map, the second feature map, and the third feature map is achieved by performing feature stitching along the channel dimension, thereby determining the second fusion feature map.

[0102] In step S760, feature mapping is performed on the second fused feature map based on the classification head in the initial localization network to determine the corresponding prediction mask.

[0103] In step S770, the corresponding segmentation loss is determined based on the real mask and predicted mask corresponding to the sample image and the segmentation loss function.

[0104] In step S780, the total loss is determined based on the segmentation loss and the comparison loss.

[0105] In this embodiment, the segmentation loss and the comparison loss are weighted and summed using the aforementioned preset function to determine the total loss.

[0106] In step S790, the model parameters in the first branch and the second branch are iteratively updated based on the total loss.

[0107] Optionally, in this embodiment, the AdamW optimizer is used to iteratively update the model parameters in the initial localization network, with an initial learning rate of 1e. -4Furthermore, to ensure smooth model convergence (i.e., the total loss of model parameters / model output), in this embodiment, when the validation performance tends to plateau after multiple consecutive evaluations, the learning rate can be halved, with a minimum lower bound of 1e. -8 .

[0108] Optionally, after each iteration of training, this embodiment uses pixel-level F1 score, IoU (intersection and union), precision and recall as evaluation metrics to evaluate the network training process. When the model's evaluation metrics reach the corresponding metric threshold (e.g., pixel-level F1 score reaches 0.9) and / or the number of training epochs reaches a preset number (e.g., 300), it is determined that the network training meets the training completion conditions, and at this time the iterative training of the initial localization model ends.

[0109] In step S530, the initial localization network determined after training is identified as a fake localization network.

[0110] In this embodiment, after the initial localization model finishes iterative training, the initial localization model at the end of iterative training is determined as the fake localization network for image fake localization.

[0111] Furthermore, after training and determining the spoofing localization network, this embodiment also verifies the performance of the spoofing localization network using the test set in the training dataset. Specifically, in this embodiment, sample images from the test set are respectively input into the spoofing localization network determined after the training of this scheme, as well as other detection networks that can be used for image spoofing localization, including MVSS-Net, TruFor, FOCAL, Mesorch, and NFA-ViT, for processing. Pixel-level F1 scoring and IoU are used to evaluate the output results of each network, resulting in the following... Figure 9 The performance comparison table is shown below. The comparison reveals that the forgery localization network in this embodiment significantly outperforms other detection networks in pixel-level F1 score and IoU, enabling it to more accurately identify forged regions in images. Furthermore, it demonstrates strong robustness in sample images after compression and random cropping, thus showing broad prospects for practical application.

[0112] The technical solution of this embodiment constructs a forgery localization network containing multiple branches. It can extract features from the texture features, semantic features, and general visual features of the image to be processed based on three different branches in the forgery localization network. This solves the problem that single-modal features are insufficient in terms of expressive power when facing instructional AI editing. Furthermore, by introducing an "anti-forgetting branch" and its channel attention fusion mechanism with other branches, the network can learn forgery features in the image during training without "forgetting" the inherent attributes of the image. This significantly improves the accuracy of the forgery localization network in identifying forgery features and its robustness in post-processing scenarios such as compression and cropping. Furthermore, this embodiment also addresses the issues of weakened forgery traces and blurred boundaries caused by instructional AI editing by innovatively introducing an intra-image contrast loss function. Based on the intra-image contrast learning mechanism, the pixel features within the image are divided into a forgery feature set and a real feature set. By optimizing the loss, the cosine distance between these two feature sets in the feature space is maximized (i.e., widening the difference between real and fake features), while minimizing the distance within the forgery feature set (i.e., aggregating forgery features). This supervisory method enables the network to keenly capture the relative distribution differences within the image, breaking through the technical bottleneck of traditional methods that struggle to detect globally reconstructed forgery images.

[0113] Figure 8 This is a schematic diagram of an image forgery detection device according to an embodiment of the present invention. Figure 8 As shown, the image forgery localization device in this embodiment includes an image acquisition unit 81, a feature extraction unit 82, and a forgery localization unit 83. The image acquisition unit 81 is used to acquire the image to be processed. The feature extraction unit 82 is used to extract texture features from the image to be processed based on a first branch of the forgery localization network to determine a texture feature map; extract semantic features from the image to be processed based on a second branch of the forgery localization network to determine a semantic feature map; extract general visual features from the image to be processed based on a third branch of the forgery localization network to determine a visual feature map; and perform feature fusion on the texture feature map, semantic feature map, and visual feature map to determine a target feature map. The forgery localization unit 83 is used to perform feature mapping on the target feature map based on the classification head in the forgery localization network to determine the corresponding output mask, and to determine the forgery localization result of the image to be processed based on the pixel distribution in the output mask; the output mask is used to characterize the pixel type of each pixel in the image to be processed, where the pixel type is a real pixel or a forged pixel, real pixels are used to characterize that the corresponding pixel is located in a real region, and forged pixels are used to characterize that the corresponding pixel is located in a forged region.

[0114] Furthermore, the image forgery localization device in this embodiment also includes a training unit. The training unit is used to acquire a training dataset, which includes multiple sample images and corresponding real masks for each sample image. The real mask includes real pixels and / or forged pixels. Iterative training is performed on the initial localization network based on the training dataset and a preset loss function. The initial localization network determined after training is then identified as the forgery localization network. The initial localization network and the forgery localization network have the same structure. The preset loss function includes a contrast loss function and a segmentation loss function. The contrast loss function maximizes the distance between features in the real and forged regions of the image in the feature space and minimizes the distance between features in the forged region. The segmentation loss function provides a supervision signal for the training of the forgery localization network.

[0115] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of the present invention. In this embodiment, the electronic device includes a server, a terminal, etc. Figure 9 As shown, the electronic device includes at least one processor 91; and a memory 92 communicatively connected to at least one processor 91; and a communication component 93 communicatively connected to a scanning device, the communication component 93 receiving and transmitting data under the control of the processor 91; wherein the memory 92 stores instructions executable by at least one processor 91, the instructions being executed by at least one processor 91 to implement the above-described image forgery location method.

[0116] Specifically, the electronic device includes: one or more processors 91 and a memory 92. Figure 9 Taking a processor 91 as an example, the processor 91 and the memory 92 can be connected via a bus or other means. Figure 9 Taking a bus connection as an example, memory 92, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Processor 91 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in memory 92, thereby realizing the image forgery location method.

[0117] The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store an option list, etc. Furthermore, the memory 92 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 92 may optionally include memory remotely located relative to the processor 91, and these remote memories may be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0118] One or more modules are stored in memory 92 and, when executed by one or more processors 91, perform the image forgery localization method in any of the above method embodiments.

[0119] The above-mentioned products can perform the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects of performing the methods. For technical details not described in detail in this embodiment, please refer to the methods provided in the embodiments of this application.

[0120] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0121] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for locating image forgery, characterized in that, The method includes: Obtain the image to be processed; Based on the first branch of the fake location network, the texture features in the image to be processed are extracted to determine the texture feature map; Based on the second branch of the fake location network, semantic features are extracted from the image to be processed to determine the semantic feature map; The third branch in the fake location network extracts common visual features from the image to be processed to determine a visual feature map; the first branch, the second branch and the third branch use the same pre-trained model with the same structure and the same initialization parameters, and the third branch extracts common visual features from the image to be processed based on the initialization parameters; The texture feature map, the semantic feature map, and the visual feature map are fused to determine the target feature map; Based on the classification head in the fake localization network, feature mapping is performed on the target feature map to determine the corresponding output mask, so as to determine the fake localization result of the image to be processed based on the pixel distribution in the output mask; the output mask is used to characterize the pixel type of each pixel in the image to be processed, the pixel type is real pixel or fake pixel, the real pixel is used to characterize the corresponding pixel is located in the real area, and the fake pixel is used to characterize the corresponding pixel is located in the fake area.

2. The method according to claim 1, characterized in that, The fake location network is determined based on the following method: Obtain a training dataset, which includes multiple sample images and a real mask corresponding to each sample image, wherein the real mask includes real pixels and / or fake pixels; The initial localization network is iteratively trained based on the training dataset and the preset loss function; The initial positioning network determined after training is identified as the fake positioning network; The initial localization network has the same structure as the fake localization network; the preset loss function includes a contrast loss function and a segmentation loss function. The contrast loss function is used to maximize the distance between features in the real region and the fake region in the feature space and minimize the distance between features in the fake region; the segmentation loss function is used to provide supervision signals for training the fake localization network.

3. The method according to claim 2, characterized in that, The iterative training of the initial localization network based on the training dataset and the preset loss function includes: Based on the first branch in the initial localization network, texture features in the sample image are extracted to determine the first feature map; Based on the second branch in the initial localization network, semantic features are extracted from the sample image to determine the second feature map; The first feature map and the second feature map are fused to determine the first fused feature map; The first fused feature map is divided into a set of real features and a set of fake features based on the real mask corresponding to the sample image; The corresponding contrast loss is determined based on the real feature set, the fake feature set, and the contrast loss function, so as to optimize the first branch network and the second branch network based on the contrast loss.

4. The method according to claim 3, characterized in that, The iterative training of the initial localization network based on the training dataset and the preset loss function also includes: Based on the third branch in the initial localization network, general visual features are extracted from the sample images to determine the third feature map; The first feature map, the second feature map, and the third feature map are fused to determine the second fused feature map; Based on the classification head in the initial localization network, feature mapping is performed on the second fused feature map to determine the corresponding prediction mask; The corresponding segmentation loss is determined based on the real and predicted masks of the sample images and the segmentation loss function. The total loss is determined based on the segmentation loss and the comparison loss; The model parameters in the first and second branches are iteratively updated based on the total loss.

5. The method according to claim 2, characterized in that, The acquisition of the training dataset includes: Obtain the original dataset, which includes multiple pairs of original images. Each pair of original images includes an original image and an edited image. The edited image is an image generated after editing the original image based on an instruction-based AI image editing method. Perform pixel alignment on the original image and the edited image in each of the original image pairs respectively; Based on the semantic features of the original and edited images after alignment, the corresponding sample images are determined by masking the semantic features of the original and edited images. The training dataset is obtained from the corresponding sample images based on each of the original images.

6. The method according to claim 1, characterized in that, The general visual features include edge features, texture features, and object category features.

7. The method according to claim 1, characterized in that, The step of extracting texture features from the image to be processed based on the first branch of the fake location network, and determining the texture feature map, includes: The image to be processed is input into a high-pass filter for filtering to determine the preprocessed image; The preprocessed image is input into the first branch of the fake localization network for feature extraction to determine the texture feature map.

8. The method according to claim 1, characterized in that, The step of fusing the texture feature map, the semantic feature map, and the visual feature map to determine the target feature map includes: The texture feature map, the semantic feature map, and the visual feature map are concatenated along the channel dimension to determine the target feature map.

9. The method according to claim 1, characterized in that, The method further includes: the classification head comprising: Multiple convolutional layers are used to perform multi-level feature mapping on the target feature map to determine the feature mapping map; An upsampling layer is used to reduce the dimensionality of the feature map in the channel dimension to generate the output mask.

10. The method according to claim 1, characterized in that, The first branch adopts the Seg-Former, U-Net, or DINO series model architecture.

11. An image forgery location device, characterized in that, The device includes: An image acquisition unit is used to acquire the image to be processed. The feature extraction unit is used to extract texture features from the image to be processed based on the first branch of the fake localization network to determine a texture feature map; extract semantic features from the image to be processed based on the second branch of the fake localization network to determine a semantic feature map; extract general visual features from the image to be processed based on the third branch of the fake localization network to determine a visual feature map; and perform feature fusion on the texture feature map, the semantic feature map, and the visual feature map to determine a target feature map. The forgery localization unit is used to perform feature mapping on the target feature map based on the classification head in the forgery localization network, determine the corresponding output mask, and determine the forgery localization result of the image to be processed based on the pixel distribution in the output mask; the output mask is used to characterize the pixel type of each pixel in the image to be processed, the pixel type is a real pixel or a forged pixel, the real pixel is used to characterize that the corresponding pixel is located in the real area, and the forged pixel is used to characterize that the corresponding pixel is located in the forged area.

12. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any one of claims 1-10.

13. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the method of any one of claims 1-10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-10.

Images