Potential spatial feature-based local counterfeit image detection method

Abstract

The invention relates to a local counterfeit image detection method based on potential spatial features. The method comprises the following steps: extracting global features and potential spatial features of an image to be detected; performing spatial alignment on the potential spatial features and the global features to respectively obtain spatial alignment potential features and spatial alignment global features; performing global average pooling on the spatial alignment potential features to obtain potential feature vectors; performing channel compression on the complete local features of the space pair to obtain a global feature vector; performing feature alignment on the potential feature vector and the global feature vector through a multi-head attention mechanism to obtain an aligned feature; carrying out edge feature enhancement on the aligned features to obtain an enhanced feature map; calculating an edge prediction probability graph based on the alignment features and the enhanced feature graph; and finally, performing up-sampling on the edge prediction probability graph to the size of the image to be detected to obtain a final edge prediction graph.

Description

Technical Field

[0001] The present application relates to the technical field of forged image detection, and in particular to a local forged image detection method based on latent space features. Background Art

[0002] Some progress has been made in detecting locally manipulated images generated by diffusion models. Some methods distinguish between generated and real images by measuring the error between the input image and the image reconstructed by the diffusion model. Other detection methods utilize diffusion model priors, combining macroscopic and microscopic features to significantly improve detection and localization capabilities. Furthermore, feature optimization-based methods enhance the model's sensitivity to manipulation details and generalization capabilities by optimizing the image feature extraction process. However, these detection methods suffer from blurred edge traces when locating locally manipulated areas, making it difficult to accurately identify and locate the forged parts in the image. Summary of the Invention

[0003] Based on this, it is necessary to provide a local forged image detection method based on latent space features, which includes: S1: Obtain the image to be detected, extract the global features of the image to be detected, and use the variational autoencoder to encode the image to be detected, and extract the latent space features of the image to be detected based on the encoded image; S2: Spatial alignment is performed on the latent spatial features and the global features to obtain spatially aligned latent features and spatially aligned global features respectively; global average pooling is performed on the spatially aligned latent features to obtain a latent feature vector; channel compression is performed on the spatially aligned global features to obtain a global feature vector; S3: Align the latent feature vector with the global feature vector through a multi-head attention mechanism to obtain the aligned features; S4: performing edge feature enhancement for the aligned features to obtain an enhanced feature map; calculating an edge prediction probability map based on the aligned features and the enhanced feature map; S5: Upsample the edge prediction probability map to the size of the image to be detected to obtain the final edge prediction map.

[0004] Preferably, the method of extracting the latent spatial features of the image based on the encoded image includes: ; ; in, Represents noise-related features; represents the random noise term; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step The associated attenuation coefficient; Represents the encoded image; Represent the latent spatial features of the image; express expectations; Represents an element-wise dot product operation.

[0005] Preferably, the method of extracting the latent spatial features of the image based on the encoded image also includes: In any denoising operation, ; ; in, Indicates the Noise-related features in the denoising operation; represents the random noise term; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step The associated attenuation coefficient; Represents the encoded image; Represent the latent spatial features of the image; Represents element-wise dot product operation; Indicates the number of denoising operations.

[0006] Preferably, the spatial alignment comprises: spatially aligning the latent spatial features and the global features through an adaptive average pooling layer.

[0007] Preferably, S3 includes: Project the latent feature vector into query, key and value respectively, and project the global feature vector into the global information bias term; In any attention head, multiply the query by the transpose of the key and divide the product by the square root of the key's feature dimension to get the first quotient; Add the first quotient to the transpose of the global information bias term, and pass the result through the softmax activation function to obtain the mapping result; Multiply the mapping result by the value to obtain the fused features of the corresponding attention head; The fused features of all attention heads are concatenated, and the concatenated results are mapped into aligned features through a linear layer.

[0008] Preferably, the potential eigenvectors are projected into queries, keys and values respectively through the query projection matrix, the key projection matrix and the value projection matrix; and the global eigenvectors are projected into global information bias items through the bias item projection matrix.

[0009] Preferably, performing edge feature enhancement for alignment features includes: Perform convolution operation on the horizontal edge detection operator and the alignment feature, and take the absolute value of the operation result; The absolute value of the operation result is convolved again with the edge detection operator in the vertical direction, and the absolute value of the result of the convolution operation is taken to obtain an enhanced feature map.

[0010] Preferably, calculating the edge prediction probability map based on the aligned features and the enhanced feature map includes: Calculate the difference between the aligned features and the enhanced feature map; The convolution operation is used to reduce the dimension of the difference, and the dimension reduction result is passed through the sigmoid activation function to obtain the edge prediction probability map.

[0011] Preferably, the convolution operation for performing dimensionality reduction on the difference uses a 3×3 convolution.

[0012] Preferably, an image encoder is used to extract global features of the image to be detected.

[0013] Beneficial effects: This method extracts global features and latent space features from the image to be detected; spatially aligns the latent space features and global features to obtain spatially aligned latent features and spatially aligned global features, respectively; performs global average pooling on the spatially aligned latent features to obtain a latent feature vector; performs channel compression on the spatially aligned global features to obtain a global feature vector; performs feature alignment on the latent feature vector and the global feature vector through a multi-head attention mechanism to obtain an aligned feature; performs edge feature enhancement on the aligned feature to obtain an enhanced feature map; calculates an edge prediction probability map based on the aligned feature and the enhanced feature map; and finally, upsamples the edge prediction probability map to the size of the image to be detected to obtain the final edge prediction map. The extraction of latent space features makes local tampering detection more generalizable, and feature alignment through a multi-head attention mechanism helps distinguish between real and forged areas. Finally, to further improve the ability to locate forged areas, the aligned features are edge enhanced to enhance the detection of forged trace edges. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 Flowchart of a local forged image detection method based on latent space features in an embodiment of the present application. DETAILED DESCRIPTION

[0016] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0017] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0018] like Figure 1 As shown, this embodiment provides a method for detecting local forged images based on latent space features, the method comprising: S1: Obtain an image to be detected, extract the global features of the image to be detected, encode the image to be detected using a variational autoencoder, and extract the latent space features of the image to be detected based on the encoded image.

[0019] In this embodiment, an image encoder is used to extract global features of the image to be detected.

[0020] In this embodiment, a variational autoencoder (VAE) is used to encode the image to be detected. This step aims to transform the image from its original pixel space into a more compact latent space representation, thereby extracting the image's key features. This latent space representation not only preserves the image's key information but also reduces the data's dimensionality, facilitating subsequent processing. In this embodiment, alternative encoders can be used to encode the image to be detected, changing the feature extraction method.

[0021] Furthermore, methods for extracting the latent spatial features of an image based on the encoded image include: ; ; in, Represents noise-related features; represents the random noise term, which is Gaussian noise randomly generated during the diffusion process and is used to simulate the degradation process of the image under different noise levels; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step The relevant attenuation coefficient is usually used to control the degree of noise addition in the diffusion model. As the diffusion process proceeds, the attenuation coefficient gradually decreases and the noise gradually increases; Represents the encoded image; Represent the latent spatial features of the image; express expectations; Represents the element-by-element dot product operation. The significance of this expectation is that by processing the noise-related features in any possible noise situation, a more stable feature vector that can reflect the potential tampering information of the image is obtained.

[0022] Noise is added to latent space feature extraction methods because, in forged image detection tasks, forged regions often disrupt the distribution consistency of the encoded image. The addition of random noise is not intended to generate images, but rather to enhance the model's sensitivity to data distribution anomalies by simulating the characteristic variations of data under varying noise levels. The introduction of random noise allows the model to learn more robust latent space features. This latent representation not only incorporates the semantic information of the image but also captures features that remain unchanged despite noise perturbations, enabling more effective identification of feature deviations introduced by forgeries.

[0023] During the generation of forged images, the operation of creating forged regions causes the feature distribution of local areas to differ from that of the original image. By introducing random noise terms in the latent space and performing denoising, the model can learn the feature variation patterns of normal images under noise perturbations. When the image to be tested is input, the forged regions exhibit different feature distributions from the normal image. After the same noise perturbation and denoising, they will exhibit different feature behavior from the normal regions. This difference can be captured by the model, enabling the detection and location of the forged regions.

[0024] This method of extracting latent space features fully utilizes the two unique properties of the diffusion model, overcomes the shortcomings of the existing local tampering image processing methods in terms of poor generalization, and fully extracts the features of the latent space. First, the forward process of the diffusion model has a closed form, which means that the encoded image can be directly Calculate the noise image at any time step t , without going through a complex iterative process. Secondly, the diffusion model is trained to be able to denoise the noisy image at any time step t and restore image information closer to the original. Therefore, given a noisy image , time step t and random noise term , it is possible to directly obtain latent space features through a single-step denoising operation without the need to fully reconstruct the image. Compared with traditional methods that fully reconstruct the image, this method is not only more computationally efficient but also more accurately captures local forgery traces in the image, because the tampered part will show different characteristics from the original image during the denoising process.

[0025] In this embodiment, in order to more efficiently estimate the latent space features, a Monte Carlo method is used to perform several independent denoising operations. Specifically, another method of extracting the latent space features of an image based on the encoded image includes: In any denoising operation, ; ; in, Indicates the Noise-related features in the denoising operation; represents the random noise term; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step The associated attenuation coefficient; Represents the encoded image; Represent the latent spatial features of the image; Represents element-wise dot product operation; Indicates the number of denoising operations.

[0026] The number of denoising operations is a predefined hyperparameter whose value directly affects the effectiveness of feature extraction and computational efficiency. By performing multiple denoising operations and averaging the results of the second dot product operation, the impact of random errors can be reduced, enabling more accurate estimation of latent space features, thus providing an efficient and reliable method for image tampering detection.

[0027] S2: Perform spatial alignment on the latent spatial features and the global features to obtain spatially aligned latent features and spatially aligned global features respectively; perform global average pooling on the spatially aligned latent features to obtain the latent feature vector; perform channel compression on the spatially aligned global features to obtain the global feature vector.

[0028] In this embodiment, the spatial alignment includes: spatially aligning the latent spatial features and the global features through an adaptive average pooling layer.

[0029] By performing spatial alignment, the generated spatially aligned latent features and spatially aligned global features have consistent sizes in the spatial dimension. The spatial alignment process is crucial for subsequent feature alignment because it ensures the consistency of the latent space feature map and the global feature map in the spatial dimension, providing a unified foundation for the subsequent multi-head attention mechanism.

[0030] After spatial alignment, in order to achieve the purpose of refining the forged trace features through feature alignment, firstly, a global average pooling operation is performed on the spatially aligned latent features to obtain the latent feature vector , Represents the dimension of the potential feature vector. Similarly, the spatially aligned global features are channel compressed to generate the global feature vector , Represents the dimension of the global feature vector. The main purpose of this operation is to extract local and global information from the latent space features and global features, providing rich context for the subsequent multi-head attention mechanism. In this process, global information strengthens the expression of local features, allowing the model to understand and model the data at different levels.

[0031] S3: Align the latent feature vector with the global feature vector through a multi-head attention mechanism to obtain aligned features.

[0032] Specifically, this step includes: Project the latent feature vector into query, key and value respectively, and project the global feature vector into the global information bias term; In any attention head, multiply the query by the transpose of the key and divide the product by the square root of the key's feature dimension to get the first quotient; Add the first quotient to the transpose of the global information bias term, and pass the result through the softmax activation function to obtain the mapping result; Multiply the mapping result by the value to obtain the fused features of the corresponding attention head; The fused features of all attention heads are concatenated, and the concatenated results are mapped into aligned features through a linear layer.

[0033] Furthermore, the query projection matrix, key projection matrix, and value projection matrix are used to project the latent feature vector into queries, keys, and values, respectively. The bias projection matrix is used to project the global feature vector into a global information bias. These matrices map the original features into a new feature space, effectively reducing computational effort while preserving the most important feature information.

[0034] By introducing a global information bias, this mechanism achieves the collaborative encoding of latent space features and global structural information, thereby enhancing the model's sensitivity to errors and its ability to utilize global information. This approach not only effectively captures local details but also enhances the model's understanding of the overall structure, enabling it to perform better on feature alignment tasks.

[0035] Furthermore, this step employs a multi-head attention mechanism to partition the latent feature vector into h subspaces, each of which independently learns different forgery traces. This decomposition allows the model to learn features from multiple perspectives, improving its expressiveness and generalization capabilities. Each subspace learns features using a different projection matrix (query projection matrix, key projection matrix, value projection matrix, and bias projection matrix). These matrices are continuously updated during training to accommodate different latent feature vectors, ensuring the model's adaptability to diverse data distributions. This allows the model to learn features from multiple perspectives and better capture different types of forgery traces.

[0036] After processing by the multi-head attention mechanism, the fused features output by all attention heads are concatenated and mapped to produce the final aligned features. This result leverages both local and global information, preserving details while enhancing understanding of the overall structure. This overcomes the inability of existing methods for detecting locally tampered images to effectively utilize both global and local information. This design enables the model to more accurately detect errors and better utilize global information, resulting in superior performance on feature alignment tasks.

[0037] The above-mentioned feature alignment realizes the fusion of latent feature vectors and global feature vectors, which can enhance the detection capability through the complementarity of macro structure and micro details: global features are used to detect overall abnormal areas, and latent features realize the precise positioning of texture details. The combination of the two can not only avoid the missed detection of local tampered textures by a single global feature, but also make up for the semantic missing traces of simple local features, and provide more feature information of forged traces for the subsequent edge trace mining process.

[0038] In this embodiment, a single attention mechanism can be used according to actual conditions, and only one fusion feature is used to map to an alignment feature through a linear layer.

[0039] S4: Perform edge feature enhancement on the aligned features to obtain an enhanced feature map; calculate an edge prediction probability map based on the aligned features and the enhanced feature map.

[0040] Specifically, the alignment feature is a four-dimensional feature vector that contains information such as the batch size, the number of channels, and the height and width of the feature map. Edge feature enhancement for the alignment feature includes: Perform a convolution operation on the horizontal edge detection operator (H=[1,−1]) and the alignment feature, and take the absolute value of the result; The absolute value of the operation result is compared with the edge detection operator in the vertical direction ( =[1,−1]) and take the absolute value of the result of the convolution operation to obtain the enhanced feature map.

[0041] The principle of edge enhancement in the horizontal and vertical directions is: calculate the difference between adjacent rows in the aligned features, and then take the absolute value to keep the gradient direction consistent. This absolute difference is redistributed to the current row, enhancing the edge clue feature map in the row direction. Next, the columns of the enhanced features are processed in the same way, and the difference between adjacent columns is calculated and its absolute value is taken to ensure the consistency of the gradient direction. By calculating the difference between adjacent pixels in the horizontal and vertical directions and taking the absolute value respectively, this method can effectively enhance the edge features in the image. This enhancement not only improves the edge recognition ability, but also helps to maintain the directional consistency of the edge gradient, which is crucial for subsequent image analysis and processing because it ensures that the model can accurately capture the structural information of the image, thereby improving the accuracy and reliability of the processing results. This method solves the problems of weak positioning of forged areas and blurred traces of forged boundaries in existing local tampering image processing methods, thereby improving detection performance.

[0042] Furthermore, calculating the edge prediction probability map based on the aligned features and the enhanced feature map includes: Calculate the difference between the aligned features and the enhanced feature map; The difference is reduced in dimension using a 3×3 convolution operation, and the reduced dimension result is passed through a sigmoid activation function to obtain an edge prediction probability map.

[0043] In the image tampering detection task, the alignment features are subtracted from the enhanced feature map, and a feature comparison mechanism is used to selectively extract edge clues in the forged area. Specifically, the alignment features contain the overall structural information and forgery information of the image, while the enhanced feature map strengthens the gradient changes in the forged area through horizontal and vertical gradient operations. When the difference is calculated, the features of the forged area are significantly different due to the enhanced gradient, while the true features of the overall structure have a difference close to zero due to the smaller changes. This process exploits the inconsistency of the features of the edge area before and after gradient enhancement, and achieves accurate identification of forgery traces by comparing the differences.

[0044] By calculating the difference between the original and enhanced features and using 3×3 convolution for dimensionality reduction, the feature space is effectively compressed while retaining key information. This dimensionality reduction process not only reduces the computational burden of the model and improves processing speed, but also ensures that the feature values are within a reasonable range through sigmoid normalization, which is important for improving the model's generalization ability and avoiding overfitting.

[0045] S5: Upsample the edge prediction probability map to the size of the image to be detected, obtaining the final edge prediction map. The upsampling step increases the resolution of the edge prediction probability map to the same size as the input image. This preserves more detailed information, making the final edge prediction map more accurate and clear. This is crucial for applications requiring high-precision edge information, such as image segmentation and object detection. Upsampling not only increases the resolution of edge predictions but also helps improve the accuracy and effectiveness of subsequent image processing tasks.

[0046] The local forged image detection method based on latent space features provided by this embodiment has the following beneficial effects: 1. We propose a feature extraction method based on latent space features, leveraging the most generalizable generated image detection features obtained in the latent space. These features not only significantly improve feature extraction efficiency but also retain key cues for distinguishing real images from fake ones. These features are unique in that they capture fine-grained differences in the image generation process, which are often crucial for distinguishing real from fake images. By leveraging the details of the latent space features, we can localize real and fake regions.

[0047] 2. We propose a latent space attention mechanism based on global information guidance. Unlike many existing methods that process local or global features separately, this method effectively aligns local spatial features with global features through a spatial attention mechanism guided by global information. This method uses spatial alignment, feature alignment, and a multi-head attention mechanism to enable the model to simultaneously focus on local details and overall structure, thereby increasing its sensitivity to latent space features. The introduction of a multi-head attention mechanism enables the model to learn latent features from multiple perspectives, improving feature expression and generalization capabilities, ultimately achieving more accurate feature alignment and object detection.

[0048] 3. We propose a feature enhancement method based on edge trace mining to enhance the detection and localization capabilities of image forgeries. This module is integrated into the model's decoder and operates at multiple scales to highlight boundary features within forged regions. By progressively emphasizing these boundary features, this method refines the localization of forged regions with greater accuracy. This enhancement is crucial for accurately identifying the boundaries between authentic and forged parts in an image, as some forgeries are very subtle. ECEM's ability to focus on these boundary features significantly improves the model's overall performance in detecting and localizing forgeries.

[0049] 4. Through the above three methods, the existing problems of poor generalization, need to enhance edge positioning effect, and inability to effectively deal with new types of local tampering and forged images on the Internet are solved.

[0050] 5. The method proposed in this embodiment achieves an average accuracy index of 98.00% and an AUC index of 99.38% on eight local tampering detection datasets.

[0051] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0052] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A local forged image detection method based on latent space features, characterized in that: include: S1: Obtain the image to be detected, extract the global features of the image to be detected, and use the variational autoencoder to encode the image to be detected, and extract the latent space features of the image to be detected based on the encoded image; S2: Spatial alignment is performed on the latent spatial features and the global features to obtain spatially aligned latent features and spatially aligned global features respectively; Perform global average pooling on the spatially aligned latent features to obtain a latent feature vector; perform channel compression on the spatially aligned global features to obtain a global feature vector; S3: Align the latent feature vector with the global feature vector through a multi-head attention mechanism to obtain the aligned features; S4: performing edge feature enhancement for the aligned features to obtain an enhanced feature map; calculating an edge prediction probability map based on the aligned features and the enhanced feature map; S5: Upsample the edge prediction probability map to the size of the image to be detected to obtain the final edge prediction map.

2. The method for detecting local forged images based on latent space features according to claim 1, wherein: Methods for extracting the latent spatial features of an image based on the encoded image include: ； ； in, Represents features related to noise; represents the random noise term; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step associated attenuation coefficients; Represents the encoded image; Represent the latent spatial features of the image; express expectations; Represents an element-wise dot product operation.

3. The method for detecting local forged images based on latent space features according to claim 1, wherein: Methods for extracting the latent spatial features of an image based on the encoded image also include: In any denoising operation, ； ； in, Indicates the Noise-related features in the denoising operation; represents the random noise term; represents the denoising function in the diffusion model; represents the time step in the diffusion process; Representation and time step associated attenuation coefficients; Represents the encoded image; Represent the latent spatial features of the image; Represents element-wise dot product operation; Indicates the number of denoising operations.

4. The method for detecting local forged images based on latent space features according to claim 1, wherein: Spatial alignment involves spatially aligning latent spatial features and global features through an adaptive average pooling layer.

5. The method for detecting local forged images based on latent space features according to claim 1, wherein S3 include: Project the latent feature vector into query, key and value respectively, and project the global feature vector into the global information bias term; In any attention head, multiply the query by the transpose of the key and divide the product by the square root of the key's feature dimension to get the first quotient; Add the first quotient to the transpose of the global information bias term, and pass the result through the softmax activation function to obtain the mapping result; Multiply the mapping result by the value to obtain the fused features of the corresponding attention head; The fused features of all attention heads are concatenated, and the concatenated results are mapped into aligned features through a linear layer.

6. The method for detecting local forged images based on latent space features according to claim 5, wherein: The latent feature vector is projected into query, key and value respectively through query projection matrix, key projection matrix and value projection matrix; the global feature vector is projected into global information bias item through bias projection matrix.

7. The method for detecting local forged images based on latent space features according to claim 1, wherein: Edge feature enhancement for alignment features includes: Perform convolution operation on the horizontal edge detection operator and the alignment feature, and take the absolute value of the operation result; The absolute value of the operation result is convolved again with the edge detection operator in the vertical direction, and the absolute value of the result of the convolution operation is taken to obtain an enhanced feature map.

8. The method for detecting local forged images based on latent space features according to claim 1, wherein: Calculating the edge prediction probability map based on the aligned features and enhanced feature maps includes: Calculate the difference between the aligned features and the enhanced feature map; The convolution operation is used to reduce the dimension of the difference, and the dimension reduction result is passed through the sigmoid activation function to obtain the edge prediction probability map.

9. The method for detecting local forged images based on latent space features according to claim 8, wherein: The convolution operation for dimensionality reduction of the difference uses 3×3 convolution.

10. The method for detecting local forged images based on latent space features according to claim 1, wherein: The image encoder is used to extract the global features of the image to be detected.

Images