Image tampering detection method based on amplified macro and micro inconsistency learning

Abstract

The present application relates to the technical field of image detection, and particularly relates to an image tampering detection method based on macro and micro inconsistency learning, which is specifically as follows: constructing an input image, inputting a to-be-positioned image into a backbone network to extract features; performing iterative clustering processing on the features, extracting two feature cluster centers with the most semantic differences, and performing feedback updating on the features based on the centers; performing progressive differential convolution processing on the features, including angle differential convolution and center differential convolution, to extract microstructure differences; fusing the features, and outputting an image tampering mask through a decoder; and jointly training multiple loss functions. Through the design of a feature extraction mechanism with adaptive perception capability, in combination with potential abnormalities of a forged area at a macro and micro level, the present application can realize efficient detection and accurate positioning of general tampered images.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to an image tampering detection method based on learning from magnified macroscopic and microscopic inconsistencies. Background Technology

[0002] With the rapid development of generative artificial intelligence technology, the technical threshold for image forgery has continued to decrease. More and more image content can be easily manipulated and synthesized, especially with the emergence of models like AIGC, which has transformed image forgery from simple image splicing to more deceptive and difficult-to-detect complex generated images. At the same time, traditional forgery methods coexist with emerging synthesis methods, and image tampering techniques are constantly being updated and iterated, further exacerbating concerns about the authenticity of information. As one of the core technologies for protecting the authenticity of visual media, image tampering detection faces the major problems of complex and diverse forgery types, varying forms of tampered areas, and severe interference from post-processing methods, leading to high uncertainty and generalization difficulty in the detection task. Compared to detection methods for specific forgery scenarios, general image tampering detection methods have greater practical application value. They are not limited to a specific type of tampering or a specific scenario, but rather aim to extract common tampering features across different tampering types through the model's learning ability, thereby achieving accurate location and identification of unknown or complex forgery samples. However, the biggest challenge in general detection is that these common forgery features are often masked by factors such as the image content itself, resolution, and compression noise, making them difficult to capture directly. Furthermore, different forgery methods exhibit significant heterogeneity in their interference with images, making it particularly difficult to extract uniform forgery representations from saliency, frequency domain, or edge responses. Some studies have attempted to introduce adversarial learning, graph neural networks, and frequency filtering into the feature space to uncover more universal forgery pattern features, but most methods still struggle to balance robustness and sensitivity. We believe that the essence of universal image tampering detection lies in uncovering potential commonalities in distribution among different forgeries, thereby enhancing the model's adaptability to tampering across scenes and tasks. Based on this premise, we aim to design a feature extraction mechanism with adaptive perception capabilities, combined with potential anomalies in the forged region at both macroscopic and microscopic levels, to achieve efficient detection and accurate localization of universally tampered images.

[0003] Therefore, this invention proposes an image tampering detection method based on amplified macroscopic and microscopic inconsistency learning to solve the above problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention develops an image tampering detection method based on amplified macroscopic and microscopic inconsistency learning. By designing a feature extraction mechanism with adaptive perception capabilities and combining it with potential anomalies in the forged region at both the macroscopic and microscopic levels, this invention can achieve efficient detection and accurate localization of commonly tampered images.

[0005] The technical solution of this invention to solve the technical problem is an image tampering detection method based on learning of macroscopic and microscopic inconsistencies, comprising the following steps:

[0006] S1. Construct an image dataset containing images to be detected. After preprocessing the images in the dataset, input them into the image encoder model for feature extraction. Then, divide the extracted feature map into two sub-feature maps along the channel dimension.

[0007] S2. Perform iterative clustering on the first sub-feature map to extract macro-difference features. Specifically, extract the centers of the two feature clusters with the most semantic differences, and update the features based on these centers.

[0008] The second sub-feature map is subjected to progressive differential convolution processing to extract micro-difference features. The differential convolution processing includes angular differential convolution and central differential convolution.

[0009] The extracted macroscopic and microscopic difference features are fused together, and then the fused features are input into the image decoder module to obtain a predicted tampered image mask of the same size as the input image.

[0010] S3. Perform joint training of multiple loss functions, and perform pixel-level loss supervision on the predicted tampered image mask obtained after decoding and the ground truth value.

[0011] S1 is as follows:

[0012] An image dataset is constructed based on the images to be detected, where the images to be detected are three-channel images and are labeled with actual tampering information.

[0013] The images in the dataset are sized and normalized. The processed images are then input into the image encoder model Segformer-B2. Image features are extracted by capturing the relationship between image information and spatial context to obtain feature maps. The feature maps are then divided into two sub-feature maps along the channel dimension.

[0014] The iterative clustering process for the first sub-feature map is as follows:

[0015] In each iteration, the high-dimensional features in the first sub-feature map are clustered. In the first iteration, adaptive average pooling is performed on the first sub-feature map to obtain the initial cluster centers to be updated. In the next iteration, the initial cluster centers to be updated are forward-propagated as the feature map for the next iteration. In subsequent iterations, the cluster centers from each iteration are forward-propagated and used as the feature map for the next iteration. In each iteration, the cluster centers are updated by calculating the cosine similarity between the current feature map and the cluster centers to construct an allocation matrix. The cluster centers are then dynamically updated based on the allocation matrix. Finally, two cluster centers with the greatest semantic difference are selected from the updated cluster centers as representatives. The feature information of these two cluster centers is then backpropagated to the first sub-feature map. The specific operations are as follows:

[0016] First, the high-dimensional features in the first sub-feature map are clustered, and the first sub-feature map is divided into clusters. pass Operation reshaping into a two-dimensional matrix , as the initial value for iterative clustering operations, , Represents the first sub-feature map The number of pixels in the middle. Indicates the channel dimension;

[0017] Then, we enter the iterative clustering phase, setting the number of iterations to [number]. The specific process is as follows:

[0018] (1) The input features are mapped to key and value features respectively through a 1×1 linear layer:

[0019] ,

[0020] ,

[0021] ,

[0022] in, Indicates the first Feature map before round clustering Represents the linear projection of the bond. Represents a linear projection of the value. Indicates the first Keys used for clustering similarity calculation after linear projection of the round , Indicates the first The values ​​used to update cluster centers after round linear projection ;

[0023] (2) Cluster center initialization: adaptive average pooling initialization is used. Cluster centers:

[0024] ,

[0025] ,

[0026] in, Indicates the first The round is used to calculate the cluster centers with respect to the original feature similarity matrix. Indicates the first Cluster centers awaiting updates This indicates an adaptive average pooling operation;

[0027] (3) Cosine similarity assignment: Calculate the cosine similarity between each pixel feature and the cluster center to form a similarity matrix. , Indicates the first The similarity matrix of the wheels is calculated using the following formula:

[0028] ,

[0029] in, Indicates the first Wheel Similarity Matrix The Line 1 Column position characteristics, Indicates the first The wheel in the cluster center Features of each location Indicates the first The wheel in the cluster center Features of each location This indicates the calculation of the vector length;

[0030] Then generate an assignment matrix based on the maximum similarity. :

[0031] ;

[0032] in, Indicates the first Round of iterations similarity matrix The Middle Line 1 Column position characteristics, Indicates selection The similarity matrix corresponding to the largest cluster center among the cluster centers is labeled as follows: , , , Indicates the number of rows in the similarity matrix;

[0033] (4) Cluster center update: For each cluster center, aggregate the value features to obtain the updated cluster centers. The calculation formula is as follows:

[0034] ,

[0035] in, Indicates the first The first round of updates Each cluster center serves as a feature map for the next iteration; Indicates the first The number of updates pending Cluster centers; Indicates the first The first round of the assignment matrix In the middle of the line The weight of each cluster center; Indicates the first The number of updates pending The value after linear projection of the cluster centers ;

[0036] Iterate through steps (1) to (4) until the clustering is updated to the two selected cluster center representatives, perform center back mapping and difference enhancement, and feed back the information of the two cluster center representatives to the first sub-feature map. The macroscopic differences are obtained, and the calculation formula is as follows:

[0037] ,

[0038] in, Indicates macroeconomic differences. This represents the sigmoid activation function. Indicates learnable weights, This indicates a learnable bias. These represent the two cluster centers obtained at the end of the iteration.

[0039] The specific process of performing progressive differential convolution on the second sub-feature map is as follows:

[0040] The difference between pairs of pixels arranged clockwise within a local neighborhood is calculated using angular difference convolution. Then, the structural difference between the center pixel and its neighborhood is calculated using center difference convolution, thereby extracting microscopic difference features. The specific operations are as follows:

[0041] First, for the second sub-feature map Pixels The formula for calculating the angle difference convolution operation is as follows:

[0042] ,

[0043] in, This represents the updated pixels after the angle difference convolution operation. Represents the second sub-feature map Pixels; This represents the angle difference convolution operation for updating pixels; Represents pixels The set of directional pixel pairs formed within a 3×3 neighborhood at a given location; express The first Each pixel feature express The first Each pixel feature This indicates that adjacent pixel feature pairs are obtained in a clockwise direction; Indicates the first One learnable weight;

[0044] Then, the pixels updated after the angle difference convolution operation. The formula for calculating the central difference convolution operation is as follows:

[0045] ,

[0046] in, This represents the pixels after the center difference convolution operation. This represents the central difference convolution operation. This represents a 3×3 neighborhood set of pixels centered at a given pixel. express The Middle Each pixel feature Represents pixels Features Indicates the first Pixel characteristics;

[0047] Finally, the final output is obtained through activation and linear transformation, calculated as follows:

[0048] ,

[0049] in, Indicates microscopic differences, This represents the convolution operation. This represents the activation function.

[0050] The feature fusion process is as follows:

[0051] Macroscopic and microscopic difference features are concatenated along the channel dimension, and then feature fusion and reconstruction are performed using a multilayer perceptron to obtain the optimized feature representation. The calculation formula is as follows:

[0052] ,

[0053] ,

[0054] in, Indicates fusion features, Indicates a fusion operation. This represents the input image to be detected. This represents a multilayer perceptron. This represents the optimized features;

[0055] Optimized features The image is input to the decoder module and gradually restored to the image to be detected using multi-layer deconvolution operations. Equal-scale tampering mask, i.e., the predicted tampered image mask. .

[0056] The joint training process using multiple loss functions is as follows:

[0057] Constructing total loss The calculation formula is as follows:

[0058] ,

[0059] in, This represents the segmentation loss based on Dice loss for the tampered region. This indicates the segmentation loss based on Dice loss for altering the edges of the tampered region. Indicates hyperparameters, This represents the true Ground Truth value of the image to be detected. This represents the true value of the tampered edge in the image to be detected. This indicates edge prediction manipulation of the image mask;

[0060] and The Dice loss involved in the loss is calculated as follows:

[0061] ,

[0062] in, This indicates Dice's loss. This indicates a genuinely altered image. This indicates a predicted image manipulation. This indicates the predicted tampered image in the th... 1 pixel The image that represents real tampering. 1 pixel This represents the total number of pixels in the image.

[0063] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0064] This invention proposes a hierarchical feature fusion method based on macroscopic clustering and microscopic differential convolution for image tampering detection. The joint analysis of macroscopic and microscopic information in image forgery detection needs to overcome the high similarity between the tampered and real regions, as well as the complex coupling of the feature distributions of the two regions.

[0065] At the macro level, through iterative clustering, this invention gathers image features into two highly differentiated centers and effectively maps the clustering results back to pixel space, thereby capturing the global structural differences between tampered and untampered regions. At the micro level, this invention designs a differential convolution operator based on pixel-level differences to accurately extract local texture and boundary changes in tampered regions. Then, by synergistically fusing the two, it can overcome the information loss and false detection problems caused by independent modeling of macro and micro features, and significantly improve the detection accuracy and robustness of highly concealed forged regions. Attached Figure Description

[0066] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0067] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0068] Figure 2 This is a comparison chart showing the image tampering detection performance of the method of the present invention and the classic image tampering localization algorithm. Detailed Implementation

[0069] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, the components and arrangements of specific examples are described below.

[0070] Example 1

[0071] S1. Construct an image dataset containing images to be detected. After preprocessing the images in the dataset, input them into the image encoder model for feature extraction. Then, divide the extracted feature map into two sub-feature maps along the channel dimension.

[0072] S2. Perform iterative clustering on the first sub-feature map to extract macro-difference features. Specifically, extract the centers of the two feature clusters with the most semantic differences, and update the features based on these centers.

[0073] The second sub-feature map is subjected to progressive differential convolution processing to extract micro-difference features. The differential convolution processing includes angular differential convolution and central differential convolution.

[0074] The extracted macroscopic and microscopic difference features are fused together, and then the fused features are input into the image decoder module to obtain a predicted tampered image mask of the same size as the input image.

[0075] S3. Perform joint training of multiple loss functions, and perform pixel-level loss supervision on the predicted tampered image mask obtained after decoding and the ground truth value.

[0076] In a specific implementation, S1 is as follows:

[0077] An image dataset is constructed based on the images to be detected, where the images to be detected are three-channel images and are labeled with actual tampering information.

[0078] The images in the dataset are sized and normalized. The processed images are then input into the Segformer-B2 image encoder model. Image features are extracted by capturing image information and spatial context, resulting in feature maps. The feature maps are then divided into two sub-feature maps along the channel dimension, which are used for subsequent iterative clustering and progressive differential convolution processing, respectively. This ensures that the high-level representation of the image can take into account both global and local tampering information, laying the foundation for accurate localization of tampered regions.

[0079] In a specific implementation, the iterative clustering process for the first sub-feature map is as follows:

[0080] In each iteration, the high-dimensional features in the first sub-feature map are clustered. In the first iteration, adaptive average pooling is performed on the first sub-feature map to obtain the initial cluster centers to be updated. In the next iteration, the initial cluster centers to be updated are forward-propagated as the feature map for the next iteration. In subsequent iterations, the cluster centers from each iteration are forward-propagated and used as the feature map for the next iteration. In each iteration, the cluster centers are updated by calculating the cosine similarity between the current feature map and the cluster centers to construct an allocation matrix. The cluster centers are then dynamically updated based on the allocation matrix. Finally, two cluster centers with the greatest semantic difference are selected as representatives from the updated cluster centers. The feature information of these two cluster centers is then backpropagated to the first sub-feature map, thereby amplifying the most representative regional differences in the image and enhancing the model's ability to perceive macroscopic tampering traces. The specific operations are as follows:

[0081] First, the high-dimensional features in the first sub-feature map are clustered, and the first sub-feature map is divided into clusters. pass Operation reshaping into a two-dimensional matrix , as the initial value for iterative clustering operations, , Represents the first sub-feature map The number of pixels in the middle. Indicates the channel dimension;

[0082] Then, we enter the iterative clustering phase, setting the number of iterations to [number]. The specific process is as follows:

[0083] (1) The input features are mapped to key and value features respectively through a 1×1 linear layer:

[0084] ,

[0085] ,

[0086] ,

[0087] in, Indicates the first Feature map before round clustering Represents the linear projection of the bond. Represents a linear projection of the value. Indicates the first Keys used for clustering similarity calculation after linear projection of the round , Indicates the first The values ​​used to update cluster centers after round linear projection ;

[0088] (2) Cluster center initialization: adaptive average pooling initialization is used. Cluster centers:

[0089] ,

[0090] ,

[0091] in, Indicates the first The round is used to calculate the cluster centers with respect to the original feature similarity matrix. Indicates the first Cluster centers awaiting updates This indicates an adaptive average pooling operation;

[0092] (3) Cosine similarity assignment: Calculate the cosine similarity between each pixel feature and the cluster center to form a similarity matrix. , Indicates the first The similarity matrix of the wheels is calculated using the following formula:

[0093] ,

[0094] in, Indicates the first Wheel Similarity Matrix The Line 1 Column position characteristics, Indicates the first The wheel in the cluster center Features of each location Indicates the first The wheel in the cluster center Features of each location This indicates the calculation of the vector length;

[0095] Then generate an assignment matrix based on the maximum similarity. :

[0096] ;

[0097] in, Indicates the first Round of iterations similarity matrix The Middle Line 1 Column position characteristics, Indicates selection The similarity matrix corresponding to the largest cluster center among the cluster centers is labeled as follows: , , , Indicates the number of rows in the similarity matrix;

[0098] (4) Cluster center update: For each cluster center, aggregate the value features to obtain the updated cluster centers. The calculation formula is as follows:

[0099] ,

[0100] in, Indicates the first The first round of updates Each cluster center serves as a feature map for the next iteration; Indicates the first The number of updates pending Cluster centers; Indicates the first The first round of the assignment matrix In the middle of the line The weight of each cluster center; Indicates the first The number of updates pending The value after linear projection of the cluster centers ;

[0101] Iterate through steps (1) to (4) until the clustering is updated to the two selected cluster center representatives, perform center back mapping and difference enhancement, and feed back the information of the two cluster center representatives to the first sub-feature map. The macroscopic differences are obtained, and the calculation formula is as follows:

[0102] ,

[0103] in, Indicates macroeconomic differences. This represents the sigmoid activation function. Indicates learnable weights, This indicates a learnable bias. These represent the two cluster centers obtained at the end of the iteration.

[0104] In a specific implementation, the process of performing progressive differential convolution on the second sub-feature map is as follows:

[0105] The difference between pairs of pixels arranged clockwise within a local neighborhood is calculated using angular difference convolution. Then, the structural difference between the center pixel and its neighborhood is calculated using center difference convolution, thereby extracting microscopic difference features. The specific operations are as follows:

[0106] First, for the second sub-feature map Pixels The formula for calculating the angle difference convolution operation is as follows:

[0107] ,

[0108] in, This represents the updated pixels after the angle difference convolution operation. Represents the second sub-feature map Pixels; This represents the angle difference convolution operation for updating pixels; Represents pixels The set of directional pixel pairs formed within a 3×3 neighborhood at a given location; express The first Each pixel feature express The first Each pixel feature This indicates that adjacent pixel feature pairs are obtained in a clockwise direction; Indicates the first One learnable weight;

[0109] Then, the pixels updated after the angle difference convolution operation. A central difference convolution operation is performed, which can effectively model micro-forgery features such as edge direction and consistency violations within texture blocks. The calculation formula is as follows:

[0110] ,

[0111] in, This represents the pixels after the center difference convolution operation. This represents the central difference convolution operation. This represents a 3×3 neighborhood set of pixels centered at a given pixel. express The Middle Each pixel feature Represents pixels Features Indicates the first Pixel characteristics;

[0112] Finally, the final output is obtained through activation and linear transformation, calculated as follows:

[0113] ,

[0114] in, Indicates microscopic differences, This represents the convolution operation. This represents the activation function.

[0115] In a specific implementation, the feature fusion process is as follows:

[0116] Macroscopic and microscopic difference features are concatenated along the channel dimension, and then feature fusion and reconstruction are performed using a multilayer perceptron to obtain the optimized feature representation. The calculation formula is as follows:

[0117] ,

[0118] ,

[0119] in, Indicates fusion features, Indicates a fusion operation. This represents the input image to be detected. This represents a multilayer perceptron. This represents the optimized features;

[0120] Optimized features The image is input to the decoder module and gradually restored to the image to be detected using multi-layer deconvolution operations. Equal-scale tampering mask, i.e., the predicted tampered image mask. .

[0121] In a specific implementation, the joint training process using multiple loss functions is as follows:

[0122] Constructing total loss The calculation formula is as follows:

[0123] ,

[0124] in, This represents the segmentation loss based on Dice loss for the tampered region. This indicates the segmentation loss based on Dice loss for altering the edges of the tampered region. Indicates hyperparameters, This represents the true Ground Truth value of the image to be detected. This represents the true value of the tampered edge in the image to be detected. This indicates edge prediction manipulation of the image mask;

[0125] and The Dice loss involved in the loss is calculated as follows:

[0126] ,

[0127] in, This indicates Dice's loss. This indicates a genuinely altered image. This indicates a predicted image manipulation. This indicates the predicted tampered image in the th... 1 pixel The image that represents real tampering. 1 pixel This represents the total number of pixels in the image.

[0128] Example 2

[0129] To verify the effectiveness of the image tampering detection method proposed in this invention, a systematic evaluation was conducted on several publicly available image tampering datasets, covering forgery samples of different types and complexities. The datasets used included CASIA (image tampering detection evaluation database, containing 7,491 original images and 5,123 tampered images, with tampering types including splicing and copy-move), IMD2020 (image tampering dataset, containing 2,010 images covering various tampering types such as splicing, copy-move, and removal), Coverage (copy-paste tampering dataset, containing tampered images and their corresponding original images and masks), NIST16 (image tampering detection dataset, containing 564 images covering splicing, copy-move, and removal tampering types), Columbia (image splicing detection evaluation dataset, containing 933 real images and 912 spliced ​​image patches, with an image size of 128×128 pixels), and COCOGLide (synthetic image tampering dataset, containing 512 tampered images). In addition, a self-constructed tamper set was constructed; by manually editing the image content, splicing, copying and pasting, or semantically modifying it, images with inconsistent textures, such as local region image synthesis, background region erasure or replacement, etc. were constructed. The training dataset randomly sampled 18,600 images in each epoch, and the test set was evaluated across datasets such as CASIAV1, Coverage, NIST16, Columbia, and COCOGLide. The evaluation metric was the pixel-level F1 score.

[0130] Table 1 shows a performance comparison between the classic image tampering localization algorithm and the present invention. The experiment was set for 100 epochs, using the Adam optimization method with a default learning rate of 1e-4. This optimization method employs a poly learning rate decay strategy. To enhance the model's ability to fit the target domain data, random contrast enhancement, illumination enhancement, saturation enhancement, and flipping operations were implemented. When testing the trained model, the image was restored to the original image for testing.

[0131] Table 1. Performance Comparison of Classic Image Tampering Detection Algorithms and the Invention on Different Datasets

[0132]

[0133] The classic image tampering localization algorithm is as follows:

[0134] ManTra-Net (Manipulation Tracing Network) is an end-to-end image tampering detection and localization method, consisting of two sub-networks: one for extracting image processing trajectory features and the other for local anomaly detection.

[0135] SPAN (Spatial Pyramid Attention Network) utilizes a spatial pyramid structure and attention mechanism to model the relationships between local regions at different scales in an image, thereby effectively detecting and locating various types of image tampering.

[0136] IF-OSN (Image Forgery detection via One-Stream Network) is a method for detecting image tampering transmitted on online social networks (OSNs).

[0137] MVSS-Net (Multi-View Multi-Scale Supervised Network) employs a multi-view, multi-scale feature supervision strategy. By fusing tamper boundary artifacts and image noise views, it achieves high-precision detection and localization of image tampering.

[0138] PSCC-Net (Progressive Spatial-Channel Correlation Network) introduces a progressive spatial-channel correlation module (SCCM), which uses a dual-path structure of top-down and bottom-up to progressively extract local and global features of the image, thereby achieving precise localization of the tampered area.

[0139] CAT-Net (Compression Artifact Tracing Network) focuses on analyzing compression artifacts in images, especially JPEG compression artifacts, to detect and locate tampered areas in images;

[0140] TruFor (Trustworthy Image Forgery Detection) utilizes the Transformer architecture, combining RGB images and noise-sensitive fingerprints to extract high-level and low-level features of images, enabling the detection and localization of various tampering types.

[0141] UnionFormer (Unified-Learning Transformer) is a unified learning framework based on Transformer. By constructing a BSFI-Net network, it integrates tampering features from RGB, noise, and edge views to achieve efficient detection and localization of image tampering.

[0142] As shown in Table 1, the effectiveness and advancement of the present invention are verified on multiple publicly available forgery detection datasets. The method of the present invention achieves excellent performance in F1 and obtains a very efficient localization effect. Therefore, the method of the present invention has outstanding technical promotion and practical application value.

[0143] Figure 2 To demonstrate the image tamper detection performance of this invention and classic image tamper location algorithms, [the following is provided]. Figure 2 It can be intuitively observed that the image tampering detection effect of this invention is more accurate than that of other classic image tampering localization algorithms, which further proves that the image tampering detection method of this invention is more effective.

[0144] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. An image tampering detection method based on learning from magnified macroscopic and microscopic inconsistencies, characterized in that, Includes the following steps: S1. Construct an image dataset containing images to be detected. After preprocessing the images in the dataset, input them into the image encoder model for feature extraction. Then, divide the extracted feature map into two sub-feature maps along the channel dimension. S2. Perform iterative clustering on the first sub-feature map to extract macro-difference features. Specifically, extract the centers of the two feature clusters with the most semantic differences, and update the features based on these centers. The second sub-feature map is subjected to progressive differential convolution processing to extract micro-difference features. The differential convolution processing includes angular differential convolution and central differential convolution. The extracted macroscopic and microscopic difference features are fused together, and then the fused features are input into the image decoder module to obtain a predicted tampered image mask of the same size as the input image. S3. Perform joint training of multiple loss functions, and perform pixel-level loss supervision on the predicted tampered image mask obtained after decoding and the ground truth value.

2. The image tampering detection method based on magnified macroscopic and microscopic inconsistency learning according to claim 1, characterized in that, S1 is as follows: An image dataset is constructed based on the images to be detected, where the images to be detected are three-channel images and are labeled with actual tampering information. The images in the dataset are sized and normalized. The processed images are then input into the image encoder model Segformer-B2. Image features are extracted by capturing the relationship between image information and spatial context to obtain feature maps. The feature maps are then divided into two sub-feature maps along the channel dimension.

3. The image tampering detection method based on magnified macroscopic and microscopic inconsistency learning according to claim 2, characterized in that, The iterative clustering process for the first sub-feature map is as follows: In each iteration, the high-dimensional features in the first sub-feature map are clustered. In the first iteration, adaptive average pooling is performed on the first sub-feature map to obtain the initial cluster centers to be updated. In the next iteration, the initial cluster centers to be updated are forward-propagated as the feature map for the next iteration. In subsequent iterations, the cluster centers from each iteration are forward-propagated and used as the feature map for the next iteration. In each iteration, the cluster centers are updated by calculating the cosine similarity between the current feature map and the cluster centers to construct an allocation matrix. The cluster centers are then dynamically updated based on the allocation matrix. Finally, two cluster centers with the greatest semantic difference are selected from the updated cluster centers as representatives. The feature information of these two cluster centers is then backpropagated to the first sub-feature map. The specific operations are as follows: First, the high-dimensional features in the first sub-feature map are clustered, and the first sub-feature map is divided into clusters. pass Operation reshaping into a two-dimensional matrix , as the initial value for iterative clustering operations, , Represents the first sub-feature map The number of pixels in the middle. Indicates the channel dimension; Then, we enter the iterative clustering phase, setting the number of iterations to [number]. The specific process is as follows: (1) The input features are mapped to key and value features respectively through a 1×1 linear layer: ， ， ， in, Indicates the first Feature map before round clustering Represents the linear projection of the bond. Represents a linear projection of the value. Indicates the first Keys used for clustering similarity calculation after linear projection of the round , Indicates the first The values ​​used to update cluster centers after round linear projection ; (2) Cluster center initialization: adaptive average pooling initialization is used. Cluster centers: ， ， in, Indicates the first The round is used to calculate the cluster centers with respect to the original feature similarity matrix. Indicates the first Cluster centers awaiting updates This indicates an adaptive average pooling operation; (3) Cosine similarity assignment: Calculate the cosine similarity between each pixel feature and the cluster center to form a similarity matrix. , Indicates the first The similarity matrix of the wheels is calculated using the following formula: ， in, Indicates the first Wheel Similarity Matrix The Line 1 Column position characteristics, Indicates the first The wheel in the cluster center Features of each location Indicates the first The wheel in the cluster center Features of each location This indicates the calculation of the vector length; Then generate an assignment matrix based on the maximum similarity. : ； in, Indicates the first Round of iterations similarity matrix The Middle Line 1 Column position characteristics, Indicates selection The similarity matrix corresponding to the largest cluster center among the cluster centers is labeled as follows: , , , Indicates the number of rows in the similarity matrix; (4) Cluster center update: For each cluster center, aggregate the value features to obtain the updated cluster centers. The calculation formula is as follows: ， in, Indicates the first The first round of updates Each cluster center serves as a feature map for the next iteration; Indicates the first The number of updates pending Cluster centers; Indicates the first The first round of the assignment matrix In the middle of the line The weight of each cluster center; Indicates the first The number of updates pending The value after linear projection of the cluster centers ; Iterate through steps (1) to (4) until the clustering is updated to the two selected cluster center representatives, perform center back mapping and difference enhancement, and feed back the information of the two cluster center representatives to the first sub-feature map. The macroscopic differences are obtained, and the calculation formula is as follows: ， in, Indicates macroeconomic differences. This represents the sigmoid activation function. Indicates learnable weights, This indicates a learnable bias. These represent the two cluster centers obtained at the end of the iteration.

4. The image tampering detection method based on magnified macroscopic and microscopic inconsistency learning according to claim 3, characterized in that, The specific process of performing progressive differential convolution on the second sub-feature map is as follows: The difference between pairs of pixels arranged clockwise within a local neighborhood is calculated using angular difference convolution. Then, the structural difference between the center pixel and its neighborhood is calculated using center difference convolution, thereby extracting microscopic difference features. The specific operations are as follows: First, for the second sub-feature map Pixels The formula for calculating the angle difference convolution operation is as follows: ， in, This represents the updated pixels after the angle difference convolution operation. Represents the second sub-feature map Pixels; This represents the angle difference convolution operation for updating pixels; Represents pixels The set of directional pixel pairs formed within a 3×3 neighborhood at a given location; express The first Each pixel feature express The first Each pixel feature This indicates that adjacent pixel feature pairs are obtained in a clockwise direction; Indicates the first One learnable weight; Then, the pixels updated after the angle difference convolution operation. The formula for calculating the central difference convolution operation is as follows: ， in, This represents the pixels after the center difference convolution operation. This represents the central difference convolution operation. This represents a 3×3 neighborhood set of pixels centered at a given pixel. express The Middle Each pixel feature Represents pixels Features Indicates the first Pixel characteristics; Finally, the final output is obtained through activation and linear transformation, calculated as follows: ， in, Indicates microscopic differences, This represents the convolution operation. This represents the activation function.

5. The image tampering detection method based on magnified macroscopic and microscopic inconsistency learning according to claim 4, characterized in that feature fusion... The process is as follows: Macroscopic and microscopic difference features are concatenated along the channel dimension, and then feature fusion and reconstruction are performed using a multilayer perceptron to obtain the optimized feature representation. The calculation formula is as follows: ， ， in, Indicates fusion features, Indicates a fusion operation. This represents the input image to be detected. This represents a multilayer perceptron. This represents the optimized features; Optimized features The image is input to the decoder module and gradually restored to the image to be detected using multi-layer deconvolution operations. Isoscale tampering mask, predicted tampered image mask .

6. The image tampering detection method based on magnified macroscopic and microscopic inconsistency learning according to claim 5, characterized in that, The joint training process using multiple loss functions is as follows: Constructing total loss The calculation formula is as follows: ， in, This represents the segmentation loss based on Dice loss for the tampered region. This indicates the segmentation loss based on Diceloss for altering the edges of the tampered region. Indicates hyperparameters, This represents the true Ground Truth value of the image to be detected. This represents the true value of the tampered edge in the image to be detected. This indicates edge prediction manipulation of the image mask; and The Dice loss involved in the loss is calculated as follows: ， in, This indicates Dice's loss. This indicates a genuinely altered image. This indicates a predicted image manipulation. This indicates the predicted tampered image in the th... 1 pixel The image that represents real tampering. 1 pixel This represents the total number of pixels in the image.

Images