Image forgery area detection method and system and readable medium
The method improves image forgery detection by using a convolutional neural network and Transformer model to cluster and encode features, addressing the limitations of existing methods in capturing subtle differences and providing precise localization.
Patent Information
- Application Number
- CN202411414319.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2024-12-27
AI Technical Summary
Existing image forgery detection methods struggle to accurately capture subtle differences between forged and real areas, particularly in high-quality images, and often fail to effectively combine global and local features, while lacking interpretability in their decision-making processes.
A method utilizing a convolutional neural network for feature extraction, followed by clustering and patch generation, combined with a Transformer model for self-attention, to enhance the detection of forged areas by integrating position encoding and feature encoding, enabling precise localization and improved accuracy.
The method significantly enhances the accuracy and interpretability of forgery detection by effectively fusing global and local features, allowing for precise identification and localization of forged areas, with strong adaptability across various forgery tasks.
Smart Images
Figure CN119206249A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an image forgery area detection method, system and readable medium, belonging to the technical field of computer vision and image processing. Background Art
[0002] In the field of digital image processing and computer vision, with the rapid development of image editing software, it is becoming easier and easier to generate fake images with higher quality. These fake images are often difficult to distinguish visually from real images, which brings great challenges to social, economic and judicial fields. For example, fake images may be used to spread false information, create false evidence, or even attack personal privacy and reputation.
[0003] Traditional methods for detecting image forgeries mainly rely on hand-crafted features (such as texture analysis, edge detection, etc.). However, these methods perform poorly when dealing with new forgery techniques, especially when the difference between the forged image and the corresponding part of the real image is small, the detection accuracy of these traditional methods will drop significantly. In addition, existing methods usually rely on the global features of the image and ignore the subtle differences between local areas in the image, which also limits their detection capabilities in complex scenes.
[0004] In recent years, the development of deep learning technology has brought new hope to image forgery detection. In particular, convolutional neural networks (CNNs) have demonstrated powerful capabilities in image feature extraction, which has significantly improved the performance of forgery detection methods based on deep learning. However, many existing deep learning methods still face several challenges: Although CNN performs well in global feature extraction, it still has certain limitations when dealing with subtle differences in local features of images. For those forged images that have been finely processed, existing methods may find it difficult to capture the subtle differences between the forged areas and the real areas.
[0005] In forged image detection, the effective combination of global information and local information is the key. Existing methods find it difficult to take both global and local information into account at the same time.
[0006] In image forgery detection, it is necessary to clearly point out which areas are forged and give reasonable explanations, but current deep learning models, especially complex network structures, are often difficult to explain their decision-making process. Summary of the invention
[0007] In view of the above problems, the purpose of the present invention is to provide an image forged area detection method, system and readable medium, which can not only improve the accuracy of forged image detection, but also enhance the interpretability of the model and provide support for the precise positioning of the forged area.
[0008] To achieve the above-mentioned purpose, the present invention proposes the following technical scheme: a method for detecting forged areas in an image, comprising the following steps: inputting a preprocessed image into a convolutional neural network model CNN for feature extraction; inputting the extracted features into a clustering algorithm model to divide the image into a number of patches with similar features; encoding each patch and adding a position code to the code of each patch; inputting each patch code with a position code added into a Transformer model to generate an output result for forged area detection.
[0009] Furthermore, the image preprocessing method includes normalization and color space conversion. The normalization is to standardize the pixel values of the image to the range of 0,1 to eliminate the influence caused by brightness difference; the color space conversion is to convert the image from RGB color space to other color spaces.
[0010] Furthermore, the other color spaces include YCbCr or HSV.
[0011] Furthermore, the preprocessed image is input into a convolutional neural network model CNN for feature extraction. The preprocessed image is input into a pretrained convolutional neural network model CNN, wherein the convolutional neural network model CNN includes multiple layers of interconnected convolutional layers and pooling layers. After the image undergoes multiple layers of convolution and pooling operations, a feature map of high feature dimension of the image is obtained; the high feature dimension means that its feature dimension is higher than the feature dimension of the input image.
[0012] Furthermore, the method of dividing the image into several patches with similar features is as follows: taking the feature vector of a certain position in the feature map as a data point, clustering the feature vector by using the K-means clustering algorithm; traversing each position of the feature map, and combining the feature vectors of each clustering area according to the clustering result to form a patch.
[0013] Furthermore, the method for encoding each of the patches is as follows: performing global average pooling or maximum pooling operations on the feature vectors within each patch to reduce the feature vectors of the patch to a fixed-dimensional vector; further reducing the dimension of the fixed-dimensional vector through a linear layer to unify it into a dimension that meets the input requirements of the Transformer model.
[0014] Furthermore, the method of adding position coding to the coding of each Patch is as follows: calculating the geometric center position of each Patch in the input image, using sine and cosine functions to generate position coding based on the geometric center position, and mapping the position coding to the same dimension as the feature vector of the Patch; combining the position coding with the feature vector of the Patch by addition or concatenation to form a Patch coding with position coding added.
[0015] Furthermore, the method for generating an output result for forged area detection is: comparing the output result of the Transformer model with a threshold, thereby classifying the output result of the Transformer model into two categories, wherein the category exceeding the threshold is a forged area, and the obtained forged area is masked or labeled.
[0016] The present invention also discloses an image forged area detection system, comprising: a feature extraction module, used to input the preprocessed image into a convolutional neural network model CNN for feature extraction; a clustering module, used to input the extracted features into a clustering algorithm model, and divide the image into a number of patches with similar features; an encoding module, used to encode each patch and add a position code to the code of each patch; a forged area output module, used to input each patch code with a position code added into a Transformer model, and generate an output result for forged area detection. The present invention also discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned image forgery area detection methods.
[0017] The technical solution of the present invention has at least the following technical effects or advantages: 1) Improving the accuracy of forgery detection: The present invention divides the image into regions with high feature consistency through feature clustering and adaptive patch generation. This method can more sensitively capture the subtle differences between forged regions and real regions, significantly improving the accuracy of forged image detection, especially when processing high-quality forged images.
[0018] 2) Enhance the interpretability of the model: By combining position encoding with feature encoding, the present invention can not only accurately identify the counterfeit area, but also provide a clear spatial location and regional explanation for the detection results, thereby enhancing the interpretability of the model and making it easier for users to understand and verify the detection results.
[0019] 3) Efficient fusion of global and local features: This paper uses the self-attention mechanism of the Transformer network to effectively fuse the local and global feature information in the image, so that the model can take into account both global structure and local details when processing complex image scenes, thereby improving the robustness of detection of forged areas.
[0020] 4) Strong adaptability: The feature clustering and patch generation method of the present invention can adaptively adjust the clustering results according to the features of different images, is applicable to various types of forgery detection tasks, and has broad application potential and strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a structural schematic diagram of an image forgery area detection method in one embodiment of the present invention. DETAILED DESCRIPTION
[0022] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or implying relative importance.
[0023] In order to solve the problems existing in the prior art that it is difficult to capture the slight difference between the forged area and the real area; it is difficult to take into account both global information and local information at the same time, and it is usually difficult to explain the decision-making process of complex networks, etc., the present invention proposes a method, system and readable medium for detecting image forgery areas, which is based on the Transformer forged image detection result generated by feature clustering and adaptive patch. It extracts features and clusters the image, divides the image into multiple patches with similar features, and combines the self-attention mechanism of the Transformer network to effectively capture and analyze the relationship between these areas, thereby identifying abnormal areas in the forged image. This method can not only improve the accuracy of forged image detection, but also enhance the interpretability of the model, and provide support for the precise positioning of the forged area. The technical solution of the present invention not only has a wide range of application prospects in the field of computer vision, but can also be extended to multiple important fields such as forensic identification, digital copyright protection, and social media content review, and has significant social and economic value. The following is a detailed description of the present invention through an embodiment.
[0024] Embodiment 1 This embodiment discloses a method for detecting an image forgery region. Figure 1 As shown, the following steps are included: S1 inputs the preprocessed image into the convolutional neural network model CNN for feature extraction.
[0025] The input original image is preprocessed as necessary to facilitate subsequent feature extraction and clustering operations. The image preprocessing methods include normalization and color space conversion. Normalization is to standardize the pixel values of the image to the range of 0,1 to eliminate the influence caused by brightness differences; color space conversion is to convert the image from the RGB color space to other color spaces to extract features that are more helpful for counterfeit detection. In this embodiment, the other color space can be YCbCr or HSV, but it is not limited thereto and can be determined according to the image characteristics to be analyzed.
[0026] S2 inputs the extracted features into the clustering algorithm model and divides the image into several patches with similar features.
[0027] The method of inputting the preprocessed image into the convolutional neural network model CNN for feature extraction is as follows: the preprocessed image is input into the pretrained convolutional neural network model CNN (such as ResNet, VGG), and the convolutional neural network model CNN includes multiple layers of interconnected convolutional layers and pooling layers. After the image undergoes multiple layers of convolution and pooling operations, a feature map with high feature dimensions of the image is obtained; high feature dimensions refer to feature dimensions that are higher than the feature dimensions of the input image. In this embodiment, for an image with an input size of 512,512,3, a feature map with a size of 16,16,2048 can be obtained after ResNet-50 extraction.
[0028] The method of dividing an image into several patches with similar features is as follows: taking the feature vector of a certain position in the feature map as a data point, clustering the feature vector by the K-means clustering algorithm; traversing each position of the feature map, and combining the feature vectors of each clustering area according to the clustering result to form a patch. In this embodiment, the number of clusters K=64 can be selected, and all positions in the feature map are divided into 64 clustering areas, each of which corresponds to a patch in the image. The simplest method is to directly splice the feature vectors belonging to the same clustering area in sequence. Assuming that a clustering area contains 10 feature vectors, each of which has a dimension of 2048, the feature vector of this patch will be a 20480-dimensional (10x2048) vector. According to the above method, the feature vector of the patch is finally obtained as a vector of a fixed size. Each patch has a high consistency in features and can better represent the local area in the image.
[0029] S3 encodes each patch and adds a position code to the code of each patch.
[0030] In order to enable the generated patches to be input into the Transformer model, the feature vector of each patch needs to be encoded into a vector of fixed size. The method for encoding each patch is as follows: the feature vector in each patch is globally averaged pooled or max pooled to reduce the feature vector of the patch to a fixed-dimensional vector; the fixed-dimensional vector is further reduced in dimension through a linear layer to unify it into a dimension that meets the input requirements of the Transformer model, such as 768 dimensions.
[0031] Since the Transformer model requires position information when processing sequence data, the present invention uses position coding to add position information (x, y) to each patch. The method of adding position coding to the coding of each patch is as follows: calculating the geometric center position of each patch in the input image, using sine and cosine functions to generate position coding based on the geometric center position, and mapping the position coding to the same dimension as the feature vector of the patch, such as 768 dimensions; combining the position coding with the feature vector of the patch by addition or splicing to form a patch coding with position coding added.
[0032] S4 inputs each patch encoding with position encoding into the Transformer model to generate output results for forged area detection.
[0033] The method for generating the output result for forged region detection is as follows: the processed patch feature vectors are arranged into a sequence in a certain order as the input of the Transformer model to generate the output result for forged region detection. The output result of the Transformer model is compared with the threshold, so as to classify the output result of the Transformer model into two categories, where the category exceeding the threshold is the forged region, and the obtained forged region is masked or marked.
[0034] Embodiment 2 Based on the same inventive concept, this embodiment discloses an image forgery area detection system, including: The feature extraction module is used to input the preprocessed image into the convolutional neural network model CNN for feature extraction; The clustering module is used to input the extracted features into the clustering algorithm model and divide the image into several patches with similar features; The encoding module is used to encode each patch and add a position code to the code of each patch; The forged area output module is used to input each patch encoding with position encoding into the Transformer model to generate an output result for forged area detection. Embodiment 3 Based on the same inventive concept, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement any of the above-mentioned video data structured sample processing methods.
[0035] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0036] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0037] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0038] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for detecting image forgery areas, characterized in that: The following steps are involved: The preprocessed image is input into the convolutional neural network model CNN for feature extraction; Input the extracted features into a clustering algorithm model to divide the image into several patches with similar features; Encode each of the patches, and add a position code to the code of each patch; Each patch encoding with position encoding is input into the Transformer model to generate output results for forged area detection.
2. The image forgery area detection method according to claim 1, characterized in that: The image preprocessing method includes normalization and color space conversion. The normalization is to standardize the pixel values of the image to the range of 0,1 to eliminate the influence caused by brightness difference; the color space conversion is to convert the image from RGB color space to other color spaces.
3. The image forgery area detection method according to claim 2, characterized in that: The other color spaces include YCbCr or HSV.
4. The image forgery area detection method according to claim 1, characterized in that: The method of inputting the preprocessed image into the convolutional neural network model CNN for feature extraction is: The preprocessed image is input into a pretrained convolutional neural network model CNN, wherein the convolutional neural network model CNN includes multiple layers of interconnected convolutional layers and pooling layers. After the image undergoes multiple layers of convolution and pooling operations, a feature map of high feature dimensions of the image is obtained; the high feature dimension refers to a feature dimension that is higher than the feature dimension of the input image.
5. The image forgery area detection method according to claim 4, characterized in that: The method of dividing an image into several patches with similar features is: Taking a feature vector at a certain position in the feature map as a data point, and clustering the feature vector using a K-means clustering algorithm; Each position of the feature map is traversed, and according to the clustering result, the feature vectors of each clustering area are combined to form a Patch.
6. The image forgery area detection method according to claim 1, characterized in that: The method for encoding each of the patches is as follows: Performing a global average pooling or maximum pooling operation on the feature vectors in each of the patches to reduce the feature vectors of the patches to a fixed-dimensional vector; The fixed-dimensional vector is further reduced in dimension through a linear layer to be unified into a dimension that meets the input requirements of the Transformer model.
7. The image forgery area detection method according to claim 6, characterized in that: The method of adding position coding to the coding of each patch is as follows: Calculate the geometric center position of each Patch in the input image, generate a position code based on the geometric center position using sine and cosine functions, and map the position code to the same dimension as the feature vector of the Patch; combine the position code with the feature vector of the Patch by addition or concatenation to form a Patch code with the position code added.
8. The image forgery area detection method according to claim 1, characterized in that: The method to generate the output for forged region detection is: The output result of the Transformer model is compared with a threshold, so as to classify the output result of the Transformer model into two categories, wherein the category exceeding the threshold is a forged area, and the obtained forged area is masked or marked.
9. An image forgery area detection system, characterized in that: include: The feature extraction module is used to input the preprocessed image into the convolutional neural network model CNN for feature extraction; A clustering module, used for inputting the extracted features into a clustering algorithm model to divide the image into a number of patches with similar features; An encoding module, used for encoding each of the patches and adding a position code to the code of each patch; The forged region output module is used to input each patch encoding with position encoding into the Transformer model, and generate output results for forged region detection.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the image forgery area detection method according to any one of claims 1 to 8.