Methods, apparatus, equipment, media, and software products for detecting synthetic images.

Abstract

This invention provides a method, apparatus, device, medium, and program product for detecting synthetic images, relating to the fields of artificial intelligence and information security, and applicable to the fintech field. The method includes: acquiring an image to be identified; and using a trained detector to detect whether the image to be identified is a real image or a synthetic image. The detector is trained using at least one sample image. In at least some rounds of training the detector, pixel-adversarial erasure and feature-adversarial erasure operations are sequentially performed on the sample image, and then a final prediction is made based on the erased image data. Specifically, the pixel-adversarial erasure operation erases high-heat regions determined based on the heatmap of the sample image, obtaining first erased image data; the feature-adversarial erasure operation erases key attention locations in a first feature map corresponding to the first erased image data identified by spatial attention weights, obtaining second erased image data.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and information security, and can be used in the field of financial technology. More specifically, it relates to a method, apparatus, device, medium, and program product for detecting synthetic images. Background Technology

[0002] With the continuous iteration and innovation of image synthesis technology (such as various AI image generation technologies), many detection methods often struggle to detect synthesized images when faced with new image generation techniques, proving inadequate. The insufficient generalization of detection algorithms has become a critical problem that urgently needs to be solved. Summary of the Invention

[0003] This invention provides a method, apparatus, device, medium, and program product for detecting synthetic images that can effectively identify even subtle artifacts.

[0004] A first aspect of this invention provides a method for detecting synthetic images. The method includes: acquiring an image to be identified; and using a trained detector to detect whether the image to be identified is a real image or a synthetic image. The detector is trained using at least one sample image. In at least a portion of the training rounds of the detector, pixel-adversarial erasure and feature-adversarial erasure operations are sequentially performed on the sample image, and then a final prediction is made based on the erased image data. Specifically, the pixel-adversarial erasure operation erases high-heat regions determined based on a heatmap of the sample image to obtain first erased image data; the feature-adversarial erasure operation erases key attention locations in a first feature map corresponding to the first erased image data, identified through spatial attention weights, to obtain second erased image data.

[0005] According to an embodiment of the present invention, the detector includes a feature extraction network and a result prediction network. The training process of the detector includes: selecting K pixel positions with the largest heat values ​​based on the heat map of the sample image to obtain the high-heat region, where K is an integer greater than or equal to 1; setting the pixel values ​​of the high-heat region in the sample image to zero to obtain the first erased image data; using the first erased image data as input to the detector to obtain the first feature map extracted by the feature extraction network of the detector from the first erased image data; identifying the spatial attention weights of the first feature map based on a spatial attention mechanism; determining the M positions with the largest weight values ​​in the spatial attention weights of the first feature map as the key attention positions, where M is an integer greater than or equal to 1; changing the weight values ​​of the key attention positions in the first feature map to the minimum weight values ​​to obtain the second erased image data; and at least inputting the second erased image data into the result prediction network to obtain the prediction result of the result prediction network for the sample image.

[0006] According to an embodiment of the present invention, the step of inputting at least the second erased image data into the result prediction network to obtain the prediction result of the result prediction network for the sample image includes: obtaining the original feature map extracted by the feature extraction network of the detector for the sample image; fusing the original feature map, the first feature map and the second erased image data to obtain the hierarchically fused features; and inputting the hierarchically fused features into the result prediction network to obtain the prediction result output by the result prediction network.

[0007] According to an embodiment of the present invention, setting the pixel values ​​of the high-heat region in the sample image to zero to obtain the first erased image data includes: creating an initial matrix with the same size as the sample image, wherein each element in the initial matrix has a value of 1; changing the value of the element in the initial matrix corresponding to the high-heat region to zero to obtain a mask matrix; and obtaining the first erased image data by multiplying the sample image with the corresponding element of the mask matrix.

[0008] According to an embodiment of the present invention, changing the weight value of the key attention position in the first feature map to the minimum weight value to obtain the second erased image data includes: changing the weight value of the key attention position in the spatial attention weight of the first feature map to the minimum weight value to obtain the masking spatial attention weight; multiplying the first feature map with the corresponding position element in the masking spatial attention weight to obtain the second erased image data.

[0009] A second aspect of this invention provides a method for training a detector, comprising: performing a pixel-adversarial erasure operation and a feature-adversarial erasure operation sequentially on a sample image to obtain erased image data; obtaining a detection result based on the erased image data, the detection result characterizing whether the sample image is a real image or a synthetic image; obtaining a loss function based on the difference between the detection result and the real label of the sample image; and updating the detector based on the loss function; wherein the pixel-adversarial erasure operation erases high-heat regions determined based on the heatmap of the sample image to obtain first erased image data; the feature-adversarial erasure operation erases key attention positions in a first feature map corresponding to the first erased image identified by spatial attention weights to obtain second erased image data, wherein the erased image data includes at least the second erased image data.

[0010] According to an embodiment of the present invention, the training method specifically includes: selecting the K pixel positions with the largest heat values ​​based on the heat map of the sample image to obtain the high-heat region, where K is an integer greater than or equal to 1; setting the pixel values ​​of the high-heat region in the sample image to zero to obtain the first erased image data; using the first erased image data as input to the detector to obtain the first feature map extracted by the feature extraction network of the detector from the first erased image data; identifying the spatial attention weights of the first feature map based on a spatial attention mechanism; determining the M positions with the largest weight values ​​in the spatial attention weights of the first feature map as the key attention positions, where M is an integer greater than or equal to 1; changing the weight values ​​of the key attention positions in the first feature map to the minimum weight values ​​to obtain the second erased image data; and at least inputting the second erased image data into the result prediction network to obtain the prediction result of the result prediction network for the sample image.

[0011] A third aspect of this invention provides a device for detecting synthetic images. The detection device includes an acquisition unit and a detection unit. The acquisition unit acquires an image to be identified. The detection unit uses a trained detector to detect whether the image to be identified is a real image or a synthetic image. The detector is trained using at least one sample image. In at least a portion of the training rounds of the detector, pixel-adversarial erasure and feature-adversarial erasure operations are sequentially performed on the sample image, and then a final prediction is made based on the erased image data. The pixel-adversarial erasure operation erases high-heat regions determined based on the heatmap of the sample image to obtain first erased image data; the feature-adversarial erasure operation erases key attention positions in a first feature map corresponding to the first erased image, identified by spatial attention weights, to obtain second erased image data.

[0012] A fourth aspect of this invention provides a training apparatus for a detector. The training apparatus includes an erasing unit, a prediction unit, and a model update unit. The erasing unit includes a pixel adversarial erasing module and a feature adversarial erasing module.

[0013] The erasure unit performs pixel-wise adversarial erasure and feature-wise erasure operations sequentially on the sample image to obtain erased image data. Specifically, the pixel-wise adversarial erasure module performs the pixel-wise adversarial erasure operation, including erasing high-heat regions identified based on the heatmap of the sample image to obtain first erased image data. The feature-wise adversarial erasure module performs the feature-wise adversarial erasure operation, including erasing key attention locations in the first feature map corresponding to the first erased image, identified by spatial attention weights, to obtain second erased image data. The erased image data includes at least the second erased image data.

[0014] The prediction unit is used to obtain a detection result based on the erased image data, and the detection result indicates whether the sample image is a real image or a synthetic image.

[0015] The model update unit is used to obtain a loss function based on the difference between the detection result and the true label of the sample image; and to update the detector based on the loss function.

[0016] A fifth aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0017] A sixth aspect of the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions, when executed by a processor, implement the steps of the above-described method.

[0018] A seventh aspect of the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-described method. Attached Figure Description

[0019] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0020] Figure 1 The illustration schematically depicts an application scenario of a method, apparatus, device, medium, and program product for detecting synthetic images according to embodiments of the present invention;

[0021] Figure 2 A flowchart illustrating a method for detecting synthetic images according to an embodiment of the present invention is shown schematically;

[0022] Figure 3 The diagram illustrates the training architecture of the detector used in a synthetic image detection method according to an embodiment of the present invention.

[0023] Figure 4 The diagram illustrates the training flowchart of the detector used in a synthetic image detection method according to an embodiment of the present invention.

[0024] Figure 5 A block diagram of a synthetic image detection apparatus according to an embodiment of the present invention is shown schematically;

[0025] Figure 6 A block diagram schematically illustrates a training apparatus for a detector according to an embodiment of the present invention; and

[0026] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a method for detecting synthetic images according to embodiments of the present invention. Detailed Implementation

[0027] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0031] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0032] Ideally, an excellent synthetic image detector should be able to identify and handle various types of artifacts, especially subtle artifacts generated by advanced digital synthesis techniques. This requires the model (i.e., the detector) to pay balanced attention to various artifact features during training, rather than focusing only on easily identifiable salient artifacts. Optimal detection performance and application value can be achieved by comprehensively mining and utilizing all available artifact information.

[0033] In view of this, embodiments of the present invention provide a method, apparatus, device, medium, and program product for detecting synthetic images capable of uncovering subtle artifacts. Accordingly, other embodiments of the present invention also provide a method, apparatus, device, medium, and program product for training a detector for detecting synthetic images.

[0034] The core of the synthetic image detection method in this invention is to encourage the detector to learn as much artifact information as possible from existing data, even subtle artifacts that are difficult to detect, thereby maintaining the detector's effectiveness when faced with synthetic images of good quality. Analysis of heatmaps, which provide information for model decision-making, reveals that the regions of focus for the network in the detector are often incomplete or inaccurate. Furthermore, both heatmaps and spatial attention mechanisms can reflect the current network's focus regions. Therefore, this invention proposes a synthetic image detection method based on adversarial erasure. Utilizing the idea of ​​adversarial erasure, after obtaining the current network's focus regions at the pixel and feature levels through heatmaps and spatial attention mechanisms, the focus regions are erased to encourage the detector to further uncover subtle artifacts. Specifically, according to embodiments of this invention...

[0035] The key terms used in this article are defined as follows:

[0036] Artifacts: In the field of synthetic image detection, "artifacts" refer to detectable visual, auditory, or physical inconsistencies that are unintentionally introduced into synthetic images due to technical limitations or algorithmic defects during the image generation process using digital technologies (such as artificial intelligence). Artifacts can be understood as key evidentiary signals revealing the synthetic nature of images.

[0037] Adversarial erasure: A deep learning technique based on adversarial principles. By erasing salient regions of interest to the network during iterative training and retraining the model, its ability to recognize global features is improved.

[0038] Heatmap: The heatmap mentioned in this article is a visualization used to interpret, after the fact, the image regions on which a deep learning model makes a specific prediction.

[0039] Attention mechanism: It is a computational technique that simulates the selective focusing ability in human cognition. It can dynamically assign different importance weights (attention weights) to different parts of the input data, so that the model can selectively focus on the most relevant and information-rich parts of the input data.

[0040] Figure 1 The illustration schematically depicts an application scenario of a method, apparatus, device, medium, and program product for detecting synthetic images according to embodiments of the present invention.

[0041] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0042] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0043] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0044] Server 105 can be a server that provides various services. Server 105 may be equipped with a detector according to the present invention, which can perform synthetic image recognition and detection on images uploaded by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103, determine whether the image is a real image or a synthetic image, and feed back the processing result to the terminal device.

[0045] It should be noted that the synthetic image detection method provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the synthetic image detection device provided in this embodiment of the invention can generally be located in server 105. The synthetic image detection method provided in this embodiment of the invention can be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the synthetic image detection device provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0046] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0047] It should be noted that the synthetic image detection method and apparatus determined in the embodiments of the present invention can be used in the field of fintech, or in any field other than fintech. The present invention does not limit the application field.

[0048] The following will be based on Figure 1 The described scene, through Figures 2-4 A method for detecting synthetic images according to embodiments of the present invention will be described in detail.

[0049] Figure 2 A flowchart illustrating a method for detecting synthetic images according to an embodiment of the present invention is shown.

[0050] like Figure 2 As shown, the detection method includes operations S210 to S220.

[0051] In operation S210, the image to be recognized is acquired.

[0052] In operation S220, the trained detector is used to detect whether the image to be identified is a real image or a synthetic image.

[0053] The detector is trained using at least one sample image, and can be referenced in at least some rounds of training the detector. Figure 3 The training architecture shown is used to train the detector.

[0054] Specifically, Figure 3 The detector shown consists of two parts: a feature extraction network 31 and a result prediction network 32. (Reference) Figure 3 In at least some rounds of the detector, the sample image X can be subjected to pixel adversarial erasure operation by pixel adversarial erasure module 611, and then to feature adversarial erasure operation by feature adversarial erasure module 612. Finally, the final prediction is made based on the erased image data.

[0055] The pixel-adversarial erasure operation is performed by the pixel-adversarial erasure module 611 to erase the high-heat areas determined based on the heat map of the sample image, thereby obtaining the first erased image data X1.

[0056] The feature adversarial erasure operation is performed by the feature adversarial erasure module 612 to erase the key attention positions in the first feature map T1 corresponding to the first erased image data identified by spatial attention weight, so as to obtain the second erased image data T2.

[0057] When making a final prediction based on the erased image data, the erased image data includes at least the second erased image data T2.

[0058] In some embodiments, the original feature map T extracted from the sample image X, the first feature map T1 mentioned above, and the second erased image data T2 can be fused to obtain a fused image, and then the final prediction can be performed based on the fused image. Fusing the features extracted from each erasure for the final prediction can avoid the forgetting phenomenon that occurs during network training.

[0059] In related technologies, the approach to improving the generalization ability of detectors can be divided into two directions. The first is to mine common, hard-to-eradicate "common defects" prevalent in synthesized image generation content as generalization cues. The core of this approach is to inject the limitations of image generation technology as prior knowledge into the model. The second is to enhance the model's adaptability to new technologies by expanding the diversity and coverage of training samples. However, while mining generalization cues can improve the model's generalization ability to some extent, this method is vulnerable to targeted fixes by attackers, resulting in limited generalization ability when facing entirely new image generation technologies. Expanding training samples to improve model performance also has significant limitations. First, this method has limited flexibility, still relying on researchers' pre-set data collection strategies (i.e., prior knowledge). This reliance may lead to limited data coverage, failing to fully encompass potentially emerging new image synthesis technologies. Second, expanding training samples through sample generation schemes requires substantial computational resources, which may result in high costs in practical applications.

[0060] Unlike the two main approaches mentioned above, the method in this invention focuses on fully utilizing the potential of existing datasets during the construction of a high-performance detector. This maximizes the detector's generalization ability while maintaining flexibility and saving computational resources. Furthermore, the method proposed in this invention has good scalability and compatibility. Not only can it operate independently, but the pixel adversarial erasure module 611 and feature adversarial erasure module 612 can also be used as plug-and-play modules, seamlessly adapting to other types of synthetic image detectors. It can even be combined with methods for mining generalization cues to further enhance the detector's generalization ability.

[0061] Figure 4 The diagram illustrates a flowchart of the training process for the detector used in a synthetic image detection method according to an embodiment of the present invention.

[0062] like Figure 4 As shown, the training process may include operations S401 to S407. Combined with... Figure 3 The explanation is as follows.

[0063] In operation S401, the K pixel locations with the highest heat values ​​are selected based on the heat map of the sample image to obtain the high heat region, where K is an integer greater than or equal to 1.

[0064] In operation S402, the pixel values ​​of the high-heat areas in the sample image are set to zero to obtain the first erased image data X1.

[0065] For example, an initial matrix of the same size as the sample image X can be created, where each element is 1. Then, the elements in the initial matrix corresponding to the high-hotspot regions are changed to zero to obtain a masking matrix. Next, the first erased image data is obtained by multiplying the sample image with the corresponding elements of the masking matrix. This method can remove high-hotspot regions relatively efficiently and accurately.

[0066] Operations S401 and S402 are specific embodiments of pixel-resistant erasure operations that can be performed by the pixel-resistant erasure module 611.

[0067] In operation S403, the first erased image data X1 is used as the input of the detector to obtain the first feature map T1 extracted by the feature extraction network 31 of the detector from the first erased image data X1.

[0068] In operation S404, based on the spatial attention mechanism, the spatial attention weights of the first feature map T1 are identified.

[0069] In operation S405, the M positions with the largest weight values ​​in the spatial attention weights of the first feature map T1 are determined as the key attention positions, where M is an integer greater than or equal to 1.

[0070] In operation S406, the weight values ​​of the key focus positions in the first feature map T1 are changed to the minimum weight values ​​to obtain the second erased image data T2.

[0071] For example, the weight values ​​of the key attention positions in the spatial attention weights of the first feature map T1 can be changed to the minimum weight values ​​to obtain the spatial attention weights for masking. Then, the first feature map T1 is multiplied by the corresponding position elements in the spatial attention weights for masking to obtain the second erased image data T2, thereby efficiently and accurately suppressing the features with higher weights in the first feature map T1.

[0072] Operations S403 to S406 are specific embodiments of feature-counterfeiting operations that can be performed by the feature counterfeiting module 612.

[0073] In operation S407, at least the second erased image data T2 is input to the result prediction network 32 to obtain the prediction result of the result prediction network 32 for the sample image.

[0074] According to the detection method of the present invention, after the pixel adversarial erasure operation is performed, the detector used will also perform a feature adversarial erasure operation on the first feature map extracted from the first erased image data during training. This can avoid the performance instability problem that exists in the training process that only performs pixel adversarial erasure operation. Specifically, the performance instability is that if only pixel adversarial operation is performed in each training round, the feature extraction network 31 may assign high weights to the high-hot areas of the previous erasure in later learning rounds, which will prevent the detector from learning subtle information.

[0075] As can be seen, the method of this invention can train the detector based on an adversarial erasure strategy. This strategy gradually encourages the detector to delve deeper into subtle artifacts by erasing the current network's key areas of focus, thereby enhancing the detector's comprehensive understanding of the synthesized image. Specifically, this invention performs region erasure operations at both the pixel and feature levels. At the pixel level, by analyzing the heatmap reflecting the network's classification criteria, the network's key areas of focus are identified and erased; at the feature level, based on spatial attention weights, key areas of focus in the feature map are identified and erased. Furthermore, to avoid forgetting during network training, the features extracted from each erasure can be fused for the final prediction.

[0076] See below for further reference. Figure 3 A training example of the detector is illustrated below. The sample image input to the detector is... Here, H and W represent the height and width of the sampled image, respectively. The training process for the detector mainly includes two parts: pixel-level adversarial erasure and feature-level adversarial erasure. A network specifically designed for image feature extraction can be used as the feature extraction network. In some embodiments, since the network performance is unstable in the early stages of training, the adversarial erasure strategy can be introduced after several training rounds.

[0077] In one embodiment, the processing procedure of the pixel adversarial erasure module 611 is detailed as follows. First, the input sample image X is processed to generate a heatmap. The size and channel parameters of the heatmap are consistent with those of the sample image X. Then, the K positions with the largest values ​​in the heatmap (the positions that play the most critical role in the prediction result of the current training round) are selected. Next, an initial matrix with all elements set to 1 and the same size as the sample image X is created, and the aforementioned K positions are set to 0 to obtain a mask matrix. Then, by multiplying the sample image X element-wise with the mask matrix, the first erased image data X1 after pixel adversarial erasure of the sample image is obtained. Finally, the feature extraction network 31 is used to extract the features of the sample image X and the first erased image data X1 respectively to obtain the original feature T and the first feature map T1.

[0078] In one embodiment, the processing procedure of the feature adversarial erasure module 612 is detailed below. First, a spatial attention mechanism is introduced. Specifically, the max-pooled data and average-pooled data after channel compression are concatenated, and then a convolutional layer is used to learn the spatial attention weights W of the first feature map T1. Second, the M positions with the largest weight values ​​(the M positions that the current network is most concerned with) are selected from the spatial attention weights W, and the weight values ​​of these positions are set to the minimum value (e.g., 1). The weight values ​​of the remaining regions remain unchanged, thus obtaining the spatial attention weights W' for masking. Finally, the first feature map T1 and the spatial attention weights W' for masking are multiplied element-wise to obtain the second erased image data T2.

[0079] In one embodiment, at the end of the network, that is, before the input result prediction network 32, the original feature map T1, the first feature map T1 extracted from the first erased image data, and the aforementioned second erased image data T2 are first fused to obtain the hierarchically fused features.

[0080] In one embodiment, as shown in Equation (1), when calculating the features after hierarchical fusion, the original feature map, the first feature map, and the second erased image data can be weighted and summed.

[0081] (1)

[0082] Next, Equation (2) is used to calculate the cross-entropy loss based on the difference between the detection result and the true label of the sample image X. Then, the model parameters of the detector are updated based on the cross-entropy loss, and iterative training is performed.

[0083] (2)

[0084] Among them, λ in equations (1) and (2) , All of these are hyperparameters. It is the network's predicted value. It is the label value of the input data (the label value has only two types: 0 and 1, where 0 represents the real image and 1 represents the synthetic image).

[0085] In the training process of the detector, this invention employs cross-dataset validation. Specifically, cross-dataset validation involves training the model on one specific dataset and then testing it on a different dataset. The different datasets use different standards or methods to acquire image data, thus evaluating the model's performance on the unseen dataset. This experimental design effectively verifies the model's generalization ability. Experimental results show that the method proposed in this invention significantly improves the detector's generalization performance. This result fully demonstrates the effectiveness of the method in improving the detector's generalization ability.

[0086] Figure 5 A block diagram of a synthetic image detection apparatus 500 according to an embodiment of the present invention is shown schematically.

[0087] like Figure 5 As shown, the detection device 500 includes an acquisition unit 510 and a detection unit 520.

[0088] The acquisition unit 510 is used to acquire the image to be recognized. In one embodiment, the acquisition unit may perform S210 as described above.

[0089] The detection unit 520 is used to detect whether the image to be identified is a real image or a synthetic image using a trained detector. The detector is trained using a training method according to an embodiment of the present invention. In one embodiment, the detection unit 520 may perform the operation S220 described above.

[0090] The detection device 500 can perform reference Figure 2 The detection methods described above are detailed in the previous text and will not be repeated here.

[0091] Figure 6 A block diagram of a detector training apparatus 600 according to an embodiment of the present invention is shown schematically.

[0092] like Figure 6 As shown, the training device 600 may include an erasure unit 610, a prediction unit 620, and a model update unit 630. The erasure unit 610 includes a pixel adversarial erasure module 611 and a feature adversarial erasure module 612.

[0093] The erasure unit 610 is used to perform pixel-adversarial erasure operation and feature-adversarial erasure operation on the sample image in sequence to obtain the erased image data.

[0094] The pixel-adversarial erasure module 611 is used to perform the pixel-adversarial erasure operation, including erasing the high-heat areas determined based on the heatmap of the sample image to obtain the first erasure image data.

[0095] The feature adversarial erasure module 612 is used to perform the feature adversarial erasure operation, including erasing the key attention positions in the first feature map corresponding to the first erasure image identified by spatial attention weight, to obtain the second erasure image data.

[0096] The prediction unit 620 is used to obtain a detection result based on the erased image data, the detection result indicating whether the sample image is a real image or a synthetic image.

[0097] The model update unit 630 is used to obtain a loss function based on the difference between the detection result and the true label of the sample image; and to update the detector based on the loss function.

[0098] In one embodiment, the training device 600 can perform a reference... Figures 3-4 The training method for the detector is described. In another embodiment, the training device 600 can be integrated into the detection device 500.

[0099] According to embodiments of the present invention, any plurality of modules among the acquisition unit 510, detection unit 520, erasure unit 610, prediction unit 620, model update unit 630, pixel adversarial erasure module 611, and feature adversarial erasure module 612 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the acquisition unit 510, detection unit 520, erasure unit 610, prediction unit 620, model update unit 630, pixel adversarial erasure module 611, and feature adversarial erasure module 612 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition unit 510, detection unit 520, erasure unit 610, prediction unit 620, model update unit 630, pixel adversarial erasure module 611, and feature adversarial erasure module 612 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0100] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a method for detecting synthetic images according to embodiments of the present invention.

[0101] like Figure 7 As shown, an electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0102] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0103] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0104] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0105] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0106] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of the present invention.

[0107] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0108] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0109] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this embodiment of the invention. According to embodiments of the invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0110] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0112] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for detecting synthetic images, wherein, The method includes: Acquire the image to be recognized; The trained detector is used to detect whether the image to be identified is a real image or a synthetic image; The detector is trained using at least one sample image. In at least some rounds of training the detector, pixel adversarial erasure and feature adversarial erasure are performed on the sample image sequentially, and then the final prediction is made based on the erased image data. The pixel-adversarial erasure operation involves erasing high-heat areas identified based on the heatmap of the sample image to obtain first erased image data. The feature adversarial erasure operation erases the key attention positions in the first feature map corresponding to the first erased image data identified by spatial attention weights, to obtain the second erased image data.

2. The detection method according to claim 1, wherein, The detector includes a feature extraction network and a result prediction network, and the training process of the detector includes: Based on the heat map of the sample image, the K pixel locations with the highest heat values ​​are selected to obtain the high heat region, where K is an integer greater than or equal to 1; Set the pixel values ​​of the high-heat areas in the sample image to zero to obtain the first erased image data; Using the first erased image data as input to the detector, the first feature map extracted by the detector's feature extraction network from the first erased image data is obtained; Based on the spatial attention mechanism, the spatial attention weights of the first feature map are identified. The M positions with the largest weight values ​​in the spatial attention weights of the first feature map are determined as the key attention positions, where M is an integer greater than or equal to 1; The weight values ​​of the key focus locations in the first feature map are changed to the minimum weight values ​​to obtain the second erased image data; and The second erased image data is input into the result prediction network at least to obtain the prediction result of the result prediction network for the sample image.

3. The detection method according to claim 2, wherein, The step of inputting at least the second erased image data into the result prediction network to obtain the prediction result of the result prediction network for the sample image includes: Obtain the original feature map extracted from the sample image by the feature extraction network of the detector; By fusing the original feature map, the first feature map, and the second erased image data, a hierarchically fused feature is obtained; The features after the hierarchical fusion are input into the result prediction network, and the prediction result output by the result prediction network is obtained.

4. The detection method according to claim 2, wherein, Setting the pixel values ​​of the high-heat areas in the sample image to zero to obtain the first erased image data includes: Create an initial matrix of the same size as the sample image, wherein each element of the initial matrix has a value of 1; The elements in the initial matrix corresponding to the high-heat regions are changed to zero to obtain the masking matrix. The first erased image data is obtained by multiplying the sample image with the corresponding element of the mask matrix.

5. The detection method according to claim 2, wherein, The step of changing the weight values ​​of the key focus locations in the first feature map to the minimum weight values ​​to obtain the second erased image data includes: The weight values ​​of the key attention positions in the spatial attention weights of the first feature map are changed to the minimum weight values ​​to obtain the spatial attention weights for masking. The first feature map is multiplied by the corresponding element in the spatial attention weights for masking to obtain the second erased image data.

6. A method for training a detector, wherein, The method includes: Pixel-wise adversarial erasure and feature-wise adversarial erasure are performed sequentially on the sample image to obtain the erased image data; The detection result is obtained based on the erased image data, and the detection result indicates whether the sample image is a real image or a synthetic image; Based on the difference between the detection result and the true label of the sample image, a loss function is obtained; and Update the detector based on the loss function; The pixel-adversarial erasure operation involves erasing high-heat areas identified based on the heatmap of the sample image to obtain first erased image data. The feature adversarial erasure operation is to erase the key attention positions in the first feature map corresponding to the first erasure image identified by spatial attention weights, to obtain the second erasure image data, wherein the erased image data includes at least the second erasure image data.

7. A device for detecting synthetic images, wherein, The device includes: The acquisition unit is used to acquire the image to be recognized; The detection unit is used to detect whether the image to be identified is a real image or a synthetic image using a trained detector; The detector is trained using at least one sample image. In at least some rounds of training the detector, pixel adversarial erasure and feature adversarial erasure are performed on the sample image sequentially, and then the final prediction is made based on the erased image data. The pixel-adversarial erasure operation involves erasing high-heat areas identified based on the heatmap of the sample image to obtain first erased image data. The feature adversarial erasure operation erases the key attention positions in the first feature map corresponding to the first erasure image identified by spatial attention weights, to obtain the second erasure image data.

8. A training apparatus for a detector, wherein, The training device includes: The erasure unit is used to perform pixel-adversarial erasure and feature-adversarial erasure operations on the sample image sequentially to obtain the erased image data; A prediction unit is used to obtain a detection result based on the erased image data, wherein the detection result characterizes whether the sample image is a real image or a synthetic image; The model update unit is used to obtain a loss function based on the difference between the detection result and the true label of the sample image; and to update the detector based on the loss function. The erasure unit includes a pixel-adversarial erasure module and a feature-adversarial erasure module; The pixel-adversarial erasure module is used to perform the pixel-adversarial erasure operation, including erasing the high-heat areas determined based on the heatmap of the sample image to obtain first erasure image data; The feature adversarial erasure module is used to perform the feature adversarial erasure operation, including erasing the key attention positions in the first feature map corresponding to the first erasure image identified by spatial attention weight, to obtain second erasure image data, wherein the erasure image data includes at least the second erasure image data.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program or instructions stored thereon, wherein, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

11. A computer program product comprising a computer program or instructions, wherein, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images