A method and apparatus for locating forged images using a diffusion model that enhances diffusion reconstruction error.

Abstract

This application discloses a method and apparatus for locating forged images using a diffusion model with enhanced diffusion reconstruction error. The method includes: inputting image samples into a pre-trained diffusion model, calculating the diffusion reconstruction error of the image samples through diffusion and denoising secondary reconstruction; inputting the diffusion reconstruction error and the image samples into a forged image detection model to obtain a forged region detection result. The forged image detection model is a two-stream architecture. The processing steps of the forged image detection model include: extracting multi-scale features from the diffusion reconstruction error and the image samples respectively to obtain multi-scale reconstruction error stream features and image stream features; fusing the two-stream features based on a cross-attention mechanism to obtain fused features; and obtaining the forged region detection result based on the fused features. This application can effectively improve the accuracy and generalization of forged region localization, and enhance the robustness of forged region localization.

Description

Technical Field

[0001] This application relates to the field of computer image processing technology, and in particular to a method, apparatus, storage medium and electronic device for locating diffusion model forged images with diffusion reconstruction error enhancement. Background Technology

[0002] The diffusion model has been widely used in the field of image generation due to its excellent generation capabilities and ease of use. It can be used not only to generate complete images but also to modify parts of existing images, such as adding, removing, replacing, or modifying backgrounds.

[0003] Currently, numerous methods exist for detecting forged images generated by diffusion models, but most focus on image-level authentication, with limited research on detecting and locating partial image tampering. The main challenge in detecting and locating partial image tampering lies in the fact that diffusion models can significantly alter image content without causing obvious frequency or structural inconsistencies, leaving fewer traces across multiple image domains, making the location of forged regions more difficult. Traditional forgery detection methods lack an understanding of deep-generated forgery traces, resulting in poor performance in practical applications. Summary of the Invention

[0004] This application provides a diffusion model forgery image localization method, apparatus, storage medium, and electronic device with diffusion reconstruction error enhancement, which can effectively improve the accuracy and generalization of forgery region localization and enhance the robustness of forgery region localization.

[0005] This application provides a diffusion model-based method for locating forged images with enhanced diffusion reconstruction error, including:

[0006] Image samples are input into a pre-trained diffusion model, and the diffusion reconstruction error of the image samples is calculated through diffusion and denoising secondary reconstruction.

[0007] The diffusion reconstruction error and the image sample are input into the forgery image detection model to obtain the forgery region detection result;

[0008] The forged image detection model is a two-stream architecture, and the processing steps of the forged image detection model include:

[0009] Multi-scale feature extraction is performed on the diffusion reconstruction error and the image sample to obtain diffusion reconstruction stream features and image stream features, respectively.

[0010] The diffusion reconstruction stream features and the image stream features are fused based on the cross-attention mechanism to obtain fused features;

[0011] The results of fake region detection are obtained based on the fusion features.

[0012] Furthermore, in the above-mentioned diffusion model forged image localization method with enhanced diffusion reconstruction error, the step of inputting the image sample into a pre-trained diffusion model and obtaining the diffusion reconstruction error of the image sample through secondary reconstruction includes:

[0013] The image samples are input into a pre-trained DDIM diffusion model to perform diffusion and denoising on the image samples, thereby obtaining a reconstructed image;

[0014] The difference between the image sample and the reconstructed image is calculated to obtain the diffusion reconstruction error.

[0015] Furthermore, in the above-mentioned diffusion model forgery image localization method with enhanced diffusion reconstruction error, the forgery image detection model includes a parallel neural network stream, a cross-attention feature fusion module, and a multi-scale forgery region localization module.

[0016] The neural network stream includes multiple cascaded Transformer blocks, each used to extract dual-stream features at different scales;

[0017] The cross-attention feature fusion module includes multiple feature fusion modules. The feature fusion module is used to receive the diffusion reconstruction stream features and image stream features output by the corresponding Transformer block, and fuse the diffusion reconstruction stream features and image stream features to obtain fused features.

[0018] The multi-scale fake region localization module includes a multilayer perceptron, which predicts the fused features to obtain the fake region detection result.

[0019] Furthermore, in the above-mentioned diffusion model forgery image localization method with enhanced diffusion reconstruction error, the processing procedure of the feature fusion module includes:

[0020] Channel weights for diffusion-reconstructed flow features and image flow features are extracted using a channel attention mechanism.

[0021] The cross-attention matrix is ​​calculated based on two sets of channel weights. The cross-attention matrix is ​​used to enhance the diffusion reconstruction stream features and the image stream features respectively, resulting in enhanced diffusion reconstruction stream features and enhanced image stream features. The enhanced diffusion reconstruction stream features and enhanced image stream features are then fused to obtain the initial fused features.

[0022] The initial fusion features are processed using a spatial attention mechanism to obtain the final fusion features.

[0023] Furthermore, in the above-mentioned diffusion model forged image localization method with enhanced diffusion reconstruction error, the step of extracting channel weights of diffusion reconstruction stream features and image stream features through a channel attention mechanism includes:

[0024] Global average pooling is performed on the diffusion reconstruction stream features and image stream features to aggregate the global spatial information of each channel;

[0025] The dependencies between channels are learned through a preset method to obtain channel weights, which are then normalized.

[0026] Furthermore, in the above-mentioned diffusion model forged image localization method with enhanced diffusion reconstruction error, the step of processing the initial fusion features based on the spatial attention mechanism to obtain the final fusion features includes:

[0027] The initial fused features are subjected to average pooling to obtain the first pooling result, and the initial fused features are subjected to max pooling to obtain the second pooling result. The first pooling result and the second pooling result are then fused to obtain the fused pooling result.

[0028] The fusion pooling result is passed through a convolutional layer and a sigmoid activation function to generate spatial attention weights. These spatial attention weights are then used to further enhance the fusion pooling result to obtain the final fusion feature.

[0029] Furthermore, the above-mentioned diffusion model forgery image localization method with enhanced diffusion reconstruction error includes, in which the forgery region detection result is obtained by predicting the fused features using a multilayer perceptron, including:

[0030] Flatten the fused features into a one-dimensional vector;

[0031] A one-dimensional vector is passed through a fully connected layer and an activation function to obtain a prediction mask, which is the result of forgery region detection.

[0032] This application also provides a diffusion model forgery image localization device with diffusion reconstruction error enhancement, including:

[0033] The reconstruction error generation module is used to input image samples into a pre-trained diffusion model and obtain the diffusion reconstruction error of the image samples through secondary reconstruction.

[0034] The forgery region detection module is used to input the diffusion reconstruction error and the image sample into the forgery image detection model to obtain the forgery region detection result;

[0035] The forged image detection model is a two-stream architecture, and the processing steps of the forged image detection model include:

[0036] Multi-scale feature extraction is performed on the diffusion reconstruction error and the image sample to obtain diffusion reconstruction stream features and image stream features, respectively.

[0037] The diffusion reconstruction stream features and the image stream features are fused based on the cross-attention mechanism to obtain fused features;

[0038] The results of fake region detection are obtained based on the fusion features.

[0039] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above-described diffusion model forged image localization methods for enhancing diffusion reconstruction error.

[0040] This application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in the steps of the diffusion model forged image localization method for diffusion reconstruction error enhancement described in any of the above claims.

[0041] This application provides a diffusion model for locating forged images with enhanced diffusion reconstruction error. The method, apparatus, storage medium, and electronic device incorporate the reconstruction error generated during the secondary diffusion process in the image feature extraction section. Simultaneously, it extracts RGB domain features to compensate for the shortcomings of single-domain features, effectively improving the accuracy and generalization of forged region localization. Furthermore, this invention employs a dual-stream network architecture to process two types of features in parallel, simultaneously extracting multi-scale image features. A cross-attention mechanism is used to aggregate the dual-stream features, fusing and enhancing them across multiple dimensions to further improve the model's generalization ability and robustness in locating forged regions. Attached Figure Description

[0042] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0043] Figure 1 A flowchart of a diffusion model-based method for locating forged images with enhanced diffusion reconstruction error, provided in an embodiment of this application.

[0044] Figure 2 A flowchart of the processing procedure of the feature fusion module provided in the embodiments of this application is shown below. Figure 2 As shown.

[0045] Figure 3 A schematic diagram of the structure of the diffusion model forged image localization device for enhancing diffusion reconstruction error provided in an embodiment of this application.

[0046] Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] This application provides a method, apparatus, storage medium, and electronic device for locating diffusion model-forged images with enhanced diffusion reconstruction error. The diffusion model-forged image location apparatus provided in this application can be integrated into an electronic device, which may be a terminal, server, or other device. The terminal may include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0049] Please see Figure 1 , Figure 1 A flowchart of a diffusion model-based method for locating forged images with enhanced diffusion reconstruction error provided in this application embodiment, which is applied in an electronic device, includes the following steps:

[0050] S1, Obtain image samples.

[0051] S2, input the image samples into the pre-trained diffusion model, and obtain the diffusion reconstruction error of the image samples through secondary reconstruction.

[0052] In one embodiment, step S2 includes the following steps:

[0053] S21, input the image samples into the pre-trained DDIM (Denoising Diffusion Implicit Models) to diffuse and denoise the image samples, and obtain the reconstructed image;

[0054] S22, calculate the difference between the image sample and the reconstructed image to obtain the diffusion reconstruction error.

[0055] For example, the difference between the pixel values ​​of the image sample and the reconstructed image can be calculated pixel by pixel and the absolute value can be taken to obtain the diffusion reconstruction error.

[0056] S3 inputs the diffusion reconstruction error and image samples into the forgery image detection model to obtain the forgery region detection result.

[0057] The forged image detection model uses a two-stream architecture, and its processing steps include:

[0058] S31, perform multi-scale feature extraction on the diffusion reconstruction error and image samples respectively to obtain diffusion reconstruction flow features and image flow features respectively.

[0059] S32, based on the cross-attention mechanism, fuses the diffusion reconstruction stream features and the image stream features to obtain the fused features.

[0060] S33, based on fusion features, obtains the results of fake region detection.

[0061] In one embodiment, such as Figure 1 As shown, the forged image detection model includes a parallel neural network stream, a cross-attention feature fusion module, and a multi-scale forged region localization module.

[0062] The neural network stream consists of multiple cascaded Transformer blocks, each used to extract dual-stream features at different scales.

[0063] Specifically, image samples and their diffusion reconstruction errors are simultaneously used as inputs to the neural network stream. The neural network stream uses a SegFormer encoder as its backbone network and employs two sets of Transformer blocks (each set containing four Transformer blocks) to construct a multi-scale feature extraction stream. Local-global features of the image are extracted at four scales. A lightweight self-attention mechanism is used to achieve long-distance dependency modeling with linear complexity, and skip connections are used to achieve cross-stage feature fusion, providing multi-scale image information for the subsequent decoding stage. Multi-scale extraction of the two types of features is performed separately to obtain multi-scale dual-stream image features.

[0064] The cross-attention feature fusion module includes multiple feature fusion modules. The feature fusion module is used to receive the diffusion reconstruction stream features and image stream features output by the corresponding Transformer block, and fuse the diffusion reconstruction stream features and image stream features to obtain the fused features.

[0065] Figure 2 A flowchart of the processing procedure of the feature fusion module provided in the embodiments of this application is shown below. Figure 2 As shown, after receiving the dual-stream image features, the channel weights of the two sets of features are first extracted through the channel attention mechanism, and the global spatial information of each channel is aggregated. Then, the cross-attention matrix between the two sets of features is calculated, and the cross-attention matrix is ​​used to enhance the two sets of input features. The two sets of enhanced features are fused through the concatenation operation. Then, for the concatenated enhanced features, a spatial attention mechanism is further introduced to guide the model to focus on important spatial regions and capture spatial information more accurately. Finally, the final fused features are output through the residual connection mechanism.

[0066] The feature fusion module's processing steps include:

[0067] S41, channel weights of diffusion reconstruction stream features and image stream features are extracted respectively through channel attention mechanism.

[0068] Specifically, step S41 includes the following steps:

[0069] S411 performs global average pooling on the diffusion reconstruction stream features and image stream features to aggregate the global spatial information of each channel;

[0070] S412 learns the dependencies between channels in a preset way, obtains channel weights, and normalizes the channel weights.

[0071] Channel attention mechanism: receiving dual-stream image features First, channel weights for the two sets of features are extracted using a channel attention mechanism. The channel attention mechanism adopts a similar implementation to Squeeze-and-Excitation (SE) attention. First, global average pooling is performed on the input feature map to compress the spatial dimension of the features and aggregate the global spatial information of each channel. Then, the dependencies between channels are learned through a “fully connected layer-activation function-fully connected layer” approach to highlight important channel features. Finally, the weights are normalized using the Sigmoid activation function.

[0072] S42, calculate the cross-attention matrix based on two sets of channel weights, and enhance the diffusion reconstruction stream features and image stream features respectively through the cross-attention matrix to obtain enhanced diffusion reconstruction stream features and enhanced image stream features. Then, fuse the enhanced diffusion reconstruction stream features and enhanced image stream features to obtain the initial fused features.

[0073] Cross-attention mechanism: Obtain two sets of channel weights Next, a cross-attention feature fusion strategy is used to interactively enhance the two sets of features. First, the cross-attention matrix between the two sets of features is calculated. Then, the cross-attention matrix is ​​used to enhance the two sets of input features. Finally, the two sets of enhanced features are fused through a concatenation operation.

[0074] S43, the initial fusion features are processed based on the spatial attention mechanism to obtain the final fusion features.

[0075] Specifically, step S43 includes the following steps:

[0076] S431, average pooling is performed on the initial fused features to obtain the first pooling result, max pooling is performed on the initial fused features to obtain the second pooling result, and the first pooling result and the second pooling result are merged to obtain the fused pooling result;

[0077] S432, the fusion pooling result is passed through a convolutional layer and a sigmoid activation function to generate spatial attention weights, and the fusion pooling result is further enhanced by the spatial attention weights to obtain the final fusion feature.

[0078] The multi-scale fake region localization module includes a multilayer perceptron, which predicts the fake region detection result by using the fused features.

[0079] Specifically, the forgery region detection result is obtained by predicting the fused features using a multilayer perceptron, including:

[0080] Flatten the fused features into a one-dimensional vector;

[0081] A one-dimensional vector is passed through a fully connected layer and an activation function to obtain a prediction mask, which is the result of the forgery region detection.

[0082] Based on the method described in the above embodiments, this embodiment will further describe the diffusion model forged image localization device with diffusion reconstruction error enhancement. The device can be implemented as an independent entity or integrated into an electronic device, such as a terminal, server, etc. The terminal may include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0083] Please see Figure 3 , Figure 3 This application provides a diffusion model forgery image localization device with enhanced diffusion reconstruction error, which is applied in an electronic device. The device may include:

[0084] The acquisition module is used to acquire image samples;

[0085] The reconstruction error generation module is used to input the image samples into a pre-trained diffusion model and obtain the diffusion reconstruction error of the image samples through secondary reconstruction.

[0086] The forgery region detection module is used to input the diffusion reconstruction error and the image sample into the forgery image detection model to obtain the forgery region detection result;

[0087] The forged image detection model is a two-stream architecture, and the processing steps of the forged image detection model include:

[0088] Multi-scale feature extraction is performed on the diffusion reconstruction error and the image sample to obtain diffusion reconstruction stream features and image stream features, respectively.

[0089] The diffusion reconstruction stream features and the image stream features are fused based on the cross-attention mechanism to obtain fused features;

[0090] The results of fake region detection are obtained based on the fusion features.

[0091] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0092] In addition, this application also provides an electronic device, which may be a computer, tablet computer, or other similar device. This electronic device can implement the steps in any embodiment of the diffusion model forged image localization method with enhanced diffusion reconstruction error provided in this application. Therefore, it can achieve the beneficial effects that any diffusion model forged image localization method with enhanced diffusion reconstruction error provided in this invention can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0093] Figure 4 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the diffusion model forged image localization method with diffusion reconstruction error enhancement provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0094] RF circuit 510 is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals and vice versa, thereby enabling communication with communication networks or other devices. RF circuit 510 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity modules (SIM cards), memory, etc. RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0095] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, such as taking pictures with the front-facing camera, processing the captured images, and switching the display colors of the content displayed on the screen. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0096] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control.

[0097] Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0098] Audio circuitry 560, speaker 561, and microphone 562 provide an audio interface between the user and electronic device 500. Audio circuitry 560 converts received audio data into electrical signals and transmits them to speaker 561, where speaker 561 converts them into sound signals for output. Conversely, microphone 562 converts collected sound signals into electrical signals, which are then received by audio circuitry 560, converted back into audio data, and processed by processor 580. The audio data is then transmitted via RF circuitry 510 to, for example, another terminal, or output to memory 520 for further processing. Audio circuitry 560 may also include an earphone jack to facilitate communication between external headphones and electronic device 500.

[0099] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0100] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0101] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0102] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0103] Obtain image samples;

[0104] The image samples are input into a pre-trained diffusion model, and the diffusion reconstruction error of the image samples is obtained through secondary reconstruction.

[0105] The diffusion reconstruction error and the image sample are input into the forgery image detection model to obtain the forgery region detection result;

[0106] The forged image detection model is a two-stream architecture, and the processing steps of the forged image detection model include:

[0107] Multi-scale feature extraction is performed on the diffusion reconstruction error and the image sample to obtain diffusion reconstruction stream features and image stream features, respectively.

[0108] The diffusion reconstruction stream features and the image stream features are fused based on the cross-attention mechanism to obtain fused features;

[0109] The results of fake region detection are obtained based on the fusion features.

[0110] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0111] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any embodiment of the diffusion model forged image localization method with enhanced diffusion reconstruction error provided by the present invention.

[0112] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0113] Since the instructions stored in the storage medium can execute the steps in any embodiment of the diffusion model forged image localization method with enhanced diffusion reconstruction error provided in the embodiments of the present invention, the beneficial effects that the diffusion model forged image localization method with enhanced diffusion reconstruction error provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0114] The foregoing has provided a detailed description of a diffusion model forgery image localization method, apparatus, storage medium, and electronic device with diffusion reconstruction error enhancement provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A diffusion model forgery image localization method that diffuses reconstruction error enhancement, characterized by, The method comprises: inputting the image sample into a pre-trained diffusion model, calculating a diffusion reconstruction error of the image sample through diffusion and denoising twice reconstruction; inputting the diffusion reconstruction error and the image sample into a fake image detection model to obtain a fake region detection result; The fake image detection model is a dual-flow architecture, and the fake image detection model comprises parallel neural network flows, a cross-attention feature fusion module and a multi-scale fake region positioning module; the neural network flows comprise a plurality of serially connected Transformer blocks, which are respectively used to extract dual-flow features of different scales; the cross-attention feature fusion module comprises a plurality of feature fusion modules, the feature fusion modules are used to receive diffusion reconstruction flow features and image flow features output by corresponding Transformer blocks, and the diffusion reconstruction flow features and the image flow features are fused to obtain fused features; the multi-scale fake region positioning module comprises a plurality of multilayer perceptrons, and the fused features are predicted through the multilayer perceptrons to obtain the fake region detection result; The processing process of the fake image detection model comprises: respectively performing multi-scale feature extraction on the diffusion reconstruction error and the image sample to obtain diffusion reconstruction flow features and image flow features; the diffusion reconstruction flow features and the image flow features are fused based on a cross-attention mechanism to obtain fused features; the fake region detection result is obtained based on the fused features.

2. The diffusion model forgery image localization method of claim 1, wherein, The method comprises: inputting the image sample into a pre-trained DDIM diffusion model, diffusing and denoising the image sample to obtain a reconstructed image; calculating the difference between the image sample and the reconstructed image to obtain a diffusion reconstruction error.

3. The diffusion model forgery image localization method of claim 1, wherein, The processing process of the feature fusion module comprises: extracting channel weights of the diffusion reconstruction flow features and the image flow features through a channel attention mechanism; calculating a cross-attention matrix based on the two groups of channel weights, enhancing the diffusion reconstruction flow features and the image flow features through the cross-attention matrix to obtain enhanced diffusion reconstruction flow features and enhanced image flow features, and fusing the enhanced diffusion reconstruction flow features and the enhanced image flow features to obtain initial fused features; processing the initial fused features based on a spatial attention mechanism to obtain final fused features.

4. The diffusion model forgery image localization method of claim 3, wherein, The method comprises: performing global average pooling on the diffusion reconstruction flow features and the image flow features to aggregate global spatial information of each channel; learning the dependency relationship between channels through a preset manner to obtain channel weights, and normalizing the channel weights.

5. The diffusion model forgery image localization method of claim 3, wherein, The method comprises: performing average pooling on the initial fused features to obtain a first pooling result, performing maximum pooling on the initial fused features to obtain a second pooling result, fusing the first pooling result and the second pooling result to obtain a fused pooling result; The fusion pooling result is generated through a convolution layer and a Sigmoid activation function to generate spatial attention weights, and the fusion pooling result is enhanced again through the spatial attention weights to obtain final fusion features.

6. The diffusion model forgery image localization method of claim 1, wherein, The fusion features are predicted through a multi-layer perception to obtain a forgery region detection result, including: The fusion features are flattened into one-dimensional vectors; The one-dimensional vectors are input into a fully connected layer and an activation function to obtain a prediction mask, which is the forgery region detection result.

7. A diffusion reconstruction error augmented diffusion model fake image positioning apparatus for implementing the diffusion reconstruction error augmented diffusion model fake image positioning method according to claim 1, characterized by, including: a reconstruction error generation module configured to input an image sample into a pre-trained diffusion model to obtain a diffusion reconstruction error of the image sample through secondary reconstruction; a forgery region detection module configured to input the diffusion reconstruction error and the image sample into a forgery image detection model to obtain a forgery region detection result; The forgery image detection model is a double-flow architecture, and the processing process of the forgery image detection model includes: multi-scale feature extraction is performed on the diffusion reconstruction error and the image sample respectively to obtain diffusion reconstruction flow features and image flow features respectively; the diffusion reconstruction flow features and the image flow features are fused based on a cross-attention mechanism to obtain fusion features; the forgery region detection result is obtained based on the fusion features.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor to execute the diffusion reconstruction error enhanced diffusion model forgery image positioning method of any one of claims 1 to 6.

9. An electronic device, comprising: including a processor and a memory, the processor being electrically connected with the memory, the memory being used for storing instructions and data, and the processor being used for executing the steps in the diffusion reconstruction error enhanced diffusion model forgery image positioning method of any one of claims 1 to 6.

Images