A cross-domain contrastive learning network for image tampering localization

Abstract

The application provides a cross-domain contrast learning network for image tampering positioning. The network is composed of an image domain branch, a frequency domain branch, a feature fusion module, an attention enhancement module, and a cross-domain contrast loss module. The image domain branch takes ViT as the backbone network to model the spatial position relationship between pixels; the frequency domain branch describes compression artifacts based on discrete cosine transform (DCT) to extract more subtle tampering traces. The dual-domain features are fused to form fusion domain features, which are input into the channel-spatial multi-scale attention module enhanced by the median to capture local details and global context information simultaneously. Then, the enhanced multi-domain features are sent to the cross-domain contrast learning module to effectively distinguish the tampering area and the real area through intra-domain contrast learning, enhance the aggregation of positive samples, and separate the distance of negative samples; at the same time, the feature representation consistency is realized through cross-domain pair alignment to reduce the distribution difference between domains.

Description

Technical Field

[0001] This invention relates to computer vision technology and image segmentation, and more specifically, to a cross-domain contrastive learning network for image tampering localization. Background Technology

[0002] In the context of today's booming development of artificial intelligence, the widespread application of advanced image editing tools and complex generative models has significantly lowered the barrier to image forgery. This has not only led to a surge in the number of forged images but also significantly increased the difficulty of detection, posing a serious challenge to fields that rely on visual authenticity. For example, in news dissemination, forged images may mislead the public; in law enforcement, fabricated evidence may disrupt judicial fairness; and in biomedical research, tampered microscope images may lead to erroneous research conclusions. Therefore, developing technologies for efficiently detecting and accurately locating traces of image forgery is of paramount importance for ensuring the credibility of digital media.

[0003] Early image tampering forensics techniques primarily relied on low-level statistical features of images to reveal forgery traces, such as point alignment, self-consistency, and Markov features. These techniques generally assumed that these low-level features possess a certain stability and consistency in untampered images, and that tampering operations would disrupt this natural property, resulting in obvious discontinuities or anomalies. When faced with complex or unknown tampering methods, these predefined rule detectors often struggle to adapt, leading to insufficient generalization ability.

[0004] In recent years, deep learning-based image manipulation detection algorithms have shown great potential in locating manipulated regions. Faster R-CNN integrates image and noise domain features and utilizes a region recommendation network to locate manipulated regions. PSCC-Net proposes a progressive spatial channel correlation network to detect and locate image manipulations, generating manipulation masks of different scales in a coarse-to-fine manner. ObjectFormer employs an attention mechanism to extract high-frequency features from the image and combines them with image domain features as a multimodal block embedding. It uses a set of learnable object prototypes as intermediate layer representations to model object-level consistency between different regions, thereby capturing subtle manipulation traces that are not visible in the image.

[0005] However, existing models still face two major challenges: 1) Feature redundancy: The representation of noise domain and frequency domain information differs significantly across different modalities, potentially leading to the introduction of redundant features. For example, the Discrete Cosine Transform (DCT) decomposes an image into a set of frequency domain coefficients, reflecting the energy distribution of the image across different frequency components. The smallest unit of the noise domain is the noise value of a single pixel, typically defined as the difference between the original pixel value and the denoised or estimated "clean" pixel value. Existing methods often employ convolution or attention mechanisms to directly fuse features from different modalities to generate the final fused feature. However, this direct fusion approach often fails to adequately consider the information redundancy between different modalities, resulting in the fused feature still containing a large amount of irrelevant or repetitive information, thus interfering with the accurate localization of the tampered region. 2) Scale limitations: Most existing models focus only on information at a single scale, which may lead to the loss of local details or insensitivity to large-scale tampered regions, thereby reducing the accuracy of tampered region localization. In reality, image tampering operations may involve regions of different scales: some tampering may only leave small traces in a local area, while others may affect large areas. Therefore, relying solely on a single scale cannot comprehensively address tampered regions of different sizes, while multi-scale feature extraction and fusion are important directions for improving detection robustness. Summary of the Invention

[0006] Purpose of the Invention: As mentioned earlier, current mainstream image tampering localization methods use supplementary domains to construct branch networks to extract subtle tampering features. Feature fusion is typically achieved through direct concatenation or addition. This direct fusion method often fails to adequately consider information redundancy between different modalities, resulting in the fused features still containing a large amount of irrelevant or repetitive information, thus interfering with the accurate localization of the tampered region. Furthermore, most existing models only focus on information at a single scale, which may lead to the loss of local details or insensitivity to large-scale tampering regions, thereby reducing the accuracy of tampering region localization. To alleviate this problem, this invention proposes a cross-domain contrastive learning network (LCDD-Net) for image tampering localization. It consists of two branches: the image domain models the spatial features between pixels, and the frequency domain fully mines subtle tampering traces such as JPEG compression. The two domains extract tampering features at four different scales respectively, and the channels are concatenated to form the features of the fusion domain. Subsequently, the features from the three domains first enter a median-enhanced channel-multi-scale spatial attention module for preliminary feature filtering, extracting tampering-related channels and multi-scale information. The enhanced features then enter the cross-domain contrastive learning module. Intra-domain loss ensures good discriminative power of feature representations by increasing the gap between positive and negative samples within each domain. Cross-domain contrastive loss reduces distributional differences between different domains by performing pairwise alignment between the image domain, fusion domain, and frequency domain, thereby achieving superior localization performance.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A cross-domain contrastive learning network for image tampering localization includes the following steps:

[0009] Step 1: Construct a dual-branch network structure consisting of an image domain branch and a frequency domain branch. An adaptive feature fusion module fuses multi-scale features from the image and frequency domain branches. The fused features are then input into a median-enhanced channel-multi-scale spatial attention module and a cross-domain contrastive learning module to enhance features across different dimensions. Finally, the fused domain features are input into the decoder to obtain the final prediction mask. The model is defined as follows: (1); in, Indicates the input image. and These represent the image encoder and the frequency domain encoder, respectively. This indicates the adaptive feature fusion module. Median-enhanced channel-multiscale spatial attention module This indicates a cross-domain comparative learning module. Indicates decoder, This represents the predicted tampering mask.

[0010] Step 2: Input the tampered image to be detected into the image domain branch and the frequency domain branch respectively, and extract four tampering features at different scales from the two branches; concatenate the image domain features and the frequency domain features at the corresponding scale dimensions to construct the fused domain features: Where Concat() represents the channel concatenation operation, Conv represents the convolution operation, and ReLU is a non-linear activation function. Therefore, we obtain four feature graph domains at four different scales across three domains: Frequency domain: Fusion Domain: To fully utilize multi-scale features from different domains, we designed an adaptive feature fusion module (AFFM), which first uses... Convolution resizes the number of channels in the features across the four scales to a uniform value. Then, the features are... arrive Perform interpolation to adjust the size to the sum The same applies, and then each feature is multiplied by a learnable parameter. This parameter is initialized to a very small value, such as Finally, sum all the features and use... Convolution adjusts the channels, ultimately yielding features representing the three domains. .

[0011] Step 3: Based on Step 2. Although the AFFM module initially achieves cross-level feature interaction and integration, the accuracy of tampering region localization remains limited due to redundant responses between feature channels and insufficient spatial context association. Addressing the core challenges of image tampering detection, such as the easy obscuring of subtle local features and insufficient feature robustness in complex interference environments, we propose a median-enhanced channel multi-scale spatial attention. This module employs a cascaded structure. First, it dynamically calibrates the importance of feature channels using channel attention incorporating multiple statistical features. Then, it enhances feature responses at different scales through a series of multi-scale dilated convolutions (dilation rate r={1,2,3}), thereby achieving multi-granular modeling of local details and global context. Specifically, three complementary channel statistics are extracted in parallel during channel attention computation: Global Average Pooling (GAP) characterizes the overall channel activation intensity, Global Max Pooling (GMP) enhances the significant response to minor tampering regions, and Global Median Pooling (GMEP) suppresses impulse noise to improve robustness to compression artifacts. These three statistical features are input into a shared two-layer MLP to learn dynamic weights, thereby adaptively strengthening tamper-related channels and suppressing noise interference. in .here, Represents the ReLU function. This represents the Sigmoid function. The dependencies between channels are modeled using a two-layer MLP, with weights of [weights to be specified]. , This indicates multiplication by channel. To further explore multi-level information within multi-domain features, we introduce multi-scale dilated convolutions into spatial attention to construct a hierarchical feature-aware network. This aims to address the limited receptive field issue inherent in traditional convolutional operations for image tampering detection. Through dilated convolutional layers with a step-like dilation rate (r={1,2,3}), the module can model the correlation between local details and global context at different scales while maintaining lightweight computational characteristics, providing multi-granularity feature support for the accurate localization of complex tampering traces. Specifically: By linearly fusing multi-scale features, the module can adaptively enhance the tamper response at different spatial granularities, constructing a multi-level feature association system from microscopic anomalies to macroscopic structural inconsistencies. Finally, features are adjusted using spatial attention weights:

[0012] Step 4: Based on Step 3. To enhance the distinction between tampered and real regions and alleviate the differences in multi-domain feature distributions, a cross-domain contrastive loss is proposed. This method constructs a hybrid feature space in the image domain, frequency domain, and fusion domain, pulling closer the anchor points and positive samples and pushing away negative samples within the domain; at the cross-domain level, a circular contrastive strategy is adopted to alternately align the features in the three domains, further aggregating samples that are the same tampered regions in different modalities and separating negative samples, thereby expanding the distance between positive and negative samples and enhancing the complementarity of cross-domain features. To reduce the memory overhead of calculations, we first perform sampling operations on the features and real masks in different domains to obtain feature embeddings and their corresponding mask embeddings Intra-domain contrast: First, we analyze the structural relationship of the intra-domain embeddings. Given the feature embedding of each pixel and its corresponding mask embedding , we define the intra-domain supervised contrastive loss as follows: where represents the set of positive samples among all feature embeddings , while represents the set of all negative samples. denotes the category of the domain, including the image domain, frequency domain, and fusion domain. In this loss function, we use normalization to standardize the features to ensure numerical stability. For a single image sample, we calculate the average of all embedded features to obtain the final contrastive loss: Cross-domain contrast: Although MCSA attention suppresses irrelevant information and enhances tampered features to a certain extent, there is still the problem of feature redundancy. To further optimize the discriminative ability of cross-modal features, we design a cross-domain contrastive loss. We propose a circular contrastive strategy, alternately comparing pairwise in the image domain, frequency domain, and fusion domain to achieve cross-domain contrastive learning: where represents the set of positive samples in different modalities, represents the negative samples in different modalities. This refers to specific contrast types, including RGB-Fusion, Fusion-DCT, and DCT-RGB. This design not only effectively brings positive sample representations closer together across different modalities but also pushes negative samples further away, thereby reducing information redundancy and improving the model's generalization ability and robustness. In cross-domain contrast, we also take the average of all embeddings as the final cross-domain loss. Finally, we average all losses to obtain the final ternary contrast learning loss: Finally, we define our loss as: For the binary cross-entropy loss, λ is set to 1 in the formula to balance the impact of... and The degree of contribution to the loss.

[0013] In step two, the tampered image to be detected is input into both the image domain branch and the frequency domain branch, and four tampering features at different scales are extracted from both branches. The frequency domain branch first performs a forward discrete cosine transform on the input image, mapping spatial domain pixels to a frequency domain representation; then, an inverse discrete cosine transform is performed on the obtained frequency coefficients to restore them back to the spatial domain, thus completing the overall transformation process from the spatial domain to the frequency domain and back to the spatial domain. The obtained frequency domain features are then input into the frequency domain encoder for learning. in This represents the discrete cosine transform operation. This is an inverse discrete cosine transform operation, which obtains high-frequency components through high-pass filtering and transforms them back to the RGB domain to maintain the translation invariance and local consistency of the natural image. Therefore, according to formulas (2), (3), and (4), we obtain four feature image domains of different scales in the three domains: Frequency domain: Fusion Domain: These features are then fed into the adaptive feature fusion module to obtain features representing the three domains. .

[0014] In step three, to filter redundant features in the channels and space of the three domain features, we apply median-enhanced channel multi-scale spatial attention. Regarding channel attention, we obtain the features enhanced using three pooling methods through formula (5). We then learn multi-scale features through dilated convolution and finally multiply them with channel enhancement features to obtain the final feature. .

[0015] In step four, we first sample the positive and negative samples: our goal is to distinguish between positive and negative samples in the features. The most basic idea is based on pixel embedding design, but the contrast loss based on single-pixel embedding does not consider the contextual information between adjacent embeddings. Furthermore, calculating this loss requires storing a series of... The dot product matrix of this size is very memory-intensive. Therefore, we will... Divided into block, thus Get what belongs to .in Furthermore, we perform average pooling on each block, thus obtaining each... All correspond In The region. For the mask, we perform a similar operation... Divided into Block, get get belong The mask contains only 0s and 1s to indicate whether a region has been tampered with. A mean pooling operation is performed relative to the features, and the maximum value of each block is selected as the [value]. The tag. We have obtained the feature embeddings and mask embedding Subsequently, we obtain the loss in this domain using formulas (8) and (9). The cross-domain loss is obtained through formulas (10) and (11). Finally, the average of these values ​​is obtained using formula (12) to obtain the final cross-domain loss. Finally, the final loss is obtained by combining it with the binary cross-entropy loss to supervise the segmentation mask.

[0016] The beneficial effects of this invention are as follows: This invention constructs a cross-domain contrastive learning network for image tampering localization to effectively extract features of tampered images, and introduces different complementary domains to uncover subtle tampering traces. Furthermore, it designs median-enhanced multi-scale spatial attention and addresses the feature redundancy problem caused by differences in the distribution of different domains in the cross-domain contrastive learning module, thereby achieving better localization accuracy. Attached Figure Description

[0017] Figure 1 This is a flowchart of the tamper detection process of the present invention;

[0018] Figure 2 This is the overall network model diagram of the present invention;

[0019] Figure 3 This is a detailed structural diagram of the median-enhanced channel-multiscale spatial attention of the present invention;

[0020] Figure 4 This is a diagram of the cross-domain contrastive learning structure of the present invention; Detailed Implementation

[0021] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] The cross-domain contrastive learning network for image tampering localization described in this invention has the following feature extraction and localization process: Figure 1 As shown, the detailed structure of the cross-domain contrastive learning network for image tampering localization is as follows: Figure 2 As shown, it includes the following steps:

[0023] Step 1: Selecting the Training Dataset: In the experimental verification phase, this invention selected the CAT-Net dataset, used for image tampering localization tasks, as the training dataset. This dataset is built based on the COCO 2017 public dataset and incorporates several representative tampering data sources such as IMD2020, Fantastic Reality, and Tampered COCO, exhibiting good diversity and complexity. The CAT-Net dataset contains various real and synthetic image tampering scenarios, comprehensively simulating common image forgery behaviors in real-world environments. Among them, the synthetic tampering data mainly covers three typical tampering types: copy-move, cut-and-paste, and repair, corresponding to different tampering mechanisms and visual feature distributions, effectively improving the dataset's coverage of tampering patterns.

[0024] Step 2, Measurement Metrics: We use two of the most widely used metrics in image tampering localization as the detection basis: AUC and F1. AUC is the area under the ROC curve, used to measure the model's overall ability to distinguish between positive and negative samples at different discrimination thresholds. This metric comprehensively reflects the model's ranking performance for positive and negative samples and is independent of specific threshold settings. The AUC value ranges from [0,1], with a larger value indicating a stronger ability to distinguish between different categories. When the AUC is close to 1, it indicates that the model has high discriminative power and stability. The F1 score is the harmonic mean of precision and recall, used to comprehensively evaluate the model's performance in positive sample detection tasks. This metric is representative when the distribution of positive and negative samples is unbalanced, and can simultaneously reflect the model's accuracy and completeness in detecting target samples. The F1 score ranges from [0,1], with a higher value indicating better performance in balancing false positives and false negatives.

[0025] Step 3, Implementation Details: For the input image, we use techniques such as padding and cropping to fix the input image to its correct position. LCDD-Net uses Adam as the optimizer, with an initial learning rate set to... The batch size was 8, and the training epochs were 100. We used a cosine decay strategy to change the learning rate during training. To prevent overfitting, we used an early stopping mechanism during training. All experiments were performed on an NVIDIA GeForce RTX 4090 graphics card; the experimental configuration is shown in the table below.

[0026] Step 4, Training Data Preprocessing: This invention scales the training data to... Then, the images are randomly horizontally flipped. Finally, they are normalized using the mean and standard deviation from ImageNet. .

[0027] Step 5, Model Loading: Initialize the image domain and frequency domain backbone networks using a pre-trained model trained from the Image-Net dataset.

[0028] Step six, build the dual-branch network, the specific steps are as follows:

[0029] (6-1) Initialize the dual-branch network. The image domain branch uses Vision Transformer (ViT) as the backbone network, and the frequency domain branch uses HR-Net as the backbone network. Load the pre-trained model to initialize the network. Input the tampered images to be detected into the dual-branch network respectively. Obtain features at four different scales according to formula (2). According to formulas (14), (15), and (16), the tampered image is first subjected to DCT transformation, then to inverse DCT transformation, and finally input into HR-Net to obtain the frequency domain features of the corresponding four different scales. .

[0030] (6-2) In order to combine the complementary information in these two domains, according to formula (4), the different features in the two domains are spliced ​​together at the same scale according to the channel dimension, so as to construct the feature of the fusion domain. .

[0031] (6-3) To fully utilize the multi-scale features of different domains, we designed an adaptive feature fusion module (AFFM). First, we used... Convolution resizes the number of channels in the features across the four scales to a uniform value. Then, the features are... arrive Perform interpolation to adjust the size to the sum The same applies, and then each feature is multiplied by a learnable parameter. This parameter is initialized to a very small value, such as Finally, sum all the features and use... Convolution adjusts the channels, ultimately yielding features representing the three domains. .

[0032] (6-4) The multi-scale features were initially processed across layers by the AFFM module, but due to redundant responses between feature channels and insufficient spatial context association, the localization accuracy of the tampered region was still limited. Therefore, we further input them into the MCSA module to enhance both the channel and spatial dimensions. In the channel dimension, we set three complementary statistical information to obtain comprehensive channel statistical features. According to formula (5), these three statistical features were passed through a shared MLP layer and summed and activated to finally obtain the channel-enhanced features. Because the tampered regions have different scales, we then designed a multi-scale spatial attention mechanism that utilizes cascaded dilated convolution operations to enhance the different scale dimensions of the features. Finally, according to equations (6) and (7), we obtained the features after double enhancement. .

[0033] (6-5) Because feature fusion typically utilizes addition or concatenation operations, attention or feature fusion can only alleviate feature redundancy to a certain extent. To further improve the separation between the tampered region and the real region, and to reduce the feature distribution differences between different domains, we input the attention-enhanced features into the cross-domain contrast learning module for intra-domain and inter-domain dual contrast. First, we perform block operations on the features of the three domains to reduce memory overhead. Then, according to formulas (8) and (9), we apply intra-domain loss to each of the three domains to optimize the distance between positive and negative samples within the domain.

[0034] (6-6) To mitigate the distribution differences between different domains, we perform pairwise comparisons of the three domains. This reduces the distribution differences between different domains during the stepwise training process. According to formulas (10) and (11), we obtain the cross-domain loss comparison.

[0035] (6-7) Finally, we sum and average the six losses to obtain the final cross-domain comparison loss. Furthermore, it is combined with binary cross-entropy loss for final supervised prediction.

[0036] The above embodiments provide a detailed description of the specific implementation of a cross-domain contrastive learning network for image tampering localization proposed in this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the proposed method and core ideas of this invention; based on the ideas of this invention, there may be some differences in specific implementations. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A cross-domain contrastive learning network for image tampering localization. Its characteristics are: Including the following steps: Step 1: Construct a dual-branch network structure consisting of an image domain branch and a frequency domain branch. An adaptive feature fusion module fuses multi-scale features from the image and frequency domain branches. The fused features are then input into a median-enhanced channel-multi-scale spatial attention module and a cross-domain contrastive learning module to enhance features across different dimensions. Finally, the fused domain features are input into the decoder to obtain the final prediction mask. The model is defined as follows: (1); in, Indicates the input image. and These represent the image encoder and the frequency domain encoder, respectively. This indicates the adaptive feature fusion module. Median-enhanced channel-multiscale spatial attention module This indicates a cross-domain comparative learning module. Indicates decoder, This represents the predicted tampering mask. Step 2: (1) Input the tampered image to be detected into the image domain branch and the frequency domain branch respectively, and extract four tampering features at different scales in the two branches; concatenate the image domain features and the frequency domain features at the corresponding scale dimensions to construct the fusion domain features: Where Concat() represents the channel concatenation operation, Conv represents the convolution operation, and ReLU is a non-linear activation function. Therefore, we obtain four feature graph domains at four different scales across three domains: Frequency domain: Fusion Domain: To fully utilize multi-scale features from different domains, we designed an adaptive feature fusion module (AFFM), which first uses... Convolution resizes the number of channels in the features across the four scales to a uniform value. Then, the features are... arrive Perform interpolation to adjust the size to the sum The same applies, and then each feature is multiplied by a learnable parameter. This parameter is initialized to a very small value, such as Finally, sum all the features and use... Convolution adjusts the channels, ultimately yielding features representing the three domains. . (2) The AFFM module initially realizes cross-level feature interaction and integration of features from different modalities. However, due to the redundant response between feature channels and the lack of spatial context association, the localization accuracy of the tampered area is still limited. To address the core challenges of local subtle features being easily submerged and insufficient feature robustness in complex interference environments in image tampering detection, we propose a median-enhanced channel multi-scale spatial attention. This module adopts a cascaded structure. First, it uses channel attention combined with multiple statistical features to dynamically calibrate the importance of feature channels. Then, it enhances the feature response at different scales through a series of multi-scale dilated convolutions (dilation rate r={1,2,3}), thereby achieving multi-granular modeling of local details and global context. Specifically, three complementary channel statistics are extracted in parallel during channel attention computation: Global Average Pooling (GAP) characterizes the overall channel activation intensity, Global Max Pooling (GMP) enhances the significant response to minor tampering regions, and Global Median Pooling (GMEP) suppresses impulse noise to improve robustness to compression artifacts. These three statistical features are input into a shared two-layer MLP to learn dynamic weights, thereby adaptively strengthening tamper-related channels and suppressing noise interference. in .here, Represents the ReLU function. This represents the Sigmoid function. The dependencies between channels are modeled using a two-layer MLP, with weights respectively... , This indicates multiplication by channel. To further explore multi-level information within multi-domain features, we introduce multi-scale dilated convolutions into spatial attention to construct a hierarchical feature-aware network. This aims to address the limited receptive field issue inherent in traditional convolutional operations for image tampering detection. Through dilated convolutional layers with a step-like dilation rate (r={1,2,3}), the module can model the correlation between local details and global context at different scales while maintaining lightweight computational characteristics, providing multi-granularity feature support for the accurate localization of complex tampering traces. Specifically: By linearly fusing multi-scale features, the module can adaptively enhance the tamper response at different spatial granularities, constructing a multi-level feature association system from microscopic anomalies to macroscopic structural inconsistencies. Finally, features are adjusted using spatial attention weights: Step 3: 1) To enhance the distinguishability between tampered and real regions and alleviate the differences in feature distribution across multiple domains, a cross-domain contrast loss is proposed. This method constructs a hybrid feature space in the image, frequency, and fusion domains. Within each domain, it brings anchor points closer to positive samples and pushes negative samples further away. At the cross-domain level, a ring contrast strategy is adopted to alternately align features from the three domains, further aggregating samples from different modalities that are both tampered regions and separating negative samples, thereby expanding the distance between positive and negative samples and improving the complementarity of cross-domain features. To reduce computational memory overhead, we first perform sampling operations on features from different domains and the real mask to obtain feature embeddings. and its corresponding mask embedding . 2) Intra-domain comparison: First, we analyze the structural relationships of the embeddings within this domain. The feature embeddings of each pixel are known. and its corresponding mask embedding Based on this, we define the domain-specific supervised contrast loss as follows: Among them, represents the set of all feature embeddings that belong to the positive samples in while represents the set of all negative samples. represents the category of the domain, including the image domain, the frequency domain, and the fusion domain. In this loss function, we use normalization to standardize the features to ensure numerical stability. For a single image sample, we calculate the average of all the embedded features to obtain the final contrastive loss: (3) Cross-domain contrast: Although MCSA attention suppresses irrelevant information and enhances tampered features to some extent, feature redundancy still exists. To further optimize the discriminative ability of cross-modal features, we designed a cross-domain contrast loss. We propose a ring contrast strategy that alternates between pairwise contrasts in the image domain, frequency domain, and fusion domain to achieve cross-domain contrast learning: in, This represents the set of positive samples in different modalities. These represent negative samples in different modalities. This refers to specific contrast types, including RGB-Fusion, Fusion-DCT, and DCT-RGB. This design not only effectively brings positive sample representations closer together across different modalities but also pushes negative samples further away, thereby reducing information redundancy and improving the model's generalization ability and robustness. In cross-domain contrast, we also take the average of all embeddings as the final cross-domain loss. Finally, we average all losses to obtain the final ternary contrast learning loss: Finally, we define our loss as: For the binary cross-entropy loss, λ is set to 1 in the formula to balance the impact of... and The degree of contribution to the loss.

2. The cross-domain contrastive learning network for image tampering localization according to claim 1, characterized in that: (1) In step (2), the tampered image to be detected is input into the image domain branch and the frequency domain branch respectively, and four tampering features of different scales are extracted in the two branches. The frequency domain first performs a forward discrete cosine transform on the input image to map the spatial domain pixels to the frequency domain representation; then, the obtained frequency coefficients are subjected to an inverse discrete cosine transform to restore them back to the spatial domain, thus completing the overall transformation process from the spatial domain to the frequency domain and back to the spatial domain. The obtained frequency domain features are then input into the frequency domain encoder for learning. in This represents the discrete cosine transform operation. This is an inverse discrete cosine transform operation, which obtains high-frequency components through high-pass filtering and transforms them back to the RGB domain to maintain the translation invariance and local consistency of the natural image. Therefore, according to formulas (2), (3), and (4), we obtain four feature image domains of different scales in the three domains: Frequency domain: Fusion Domain: These features are then fed into the adaptive feature fusion module to obtain features representing the three domains. . (2) To filter redundant features in the channels and space of the three domain features, we apply median-enhanced channel multi-scale spatial attention. Regarding channel attention, we obtain the enhanced features using three pooling methods via formula (5). We then learn multi-scale features through dilated convolution and finally multiply them with channel enhancement features to obtain the final feature. .

3. The cross-domain contrastive learning network for image tampering localization according to claim 1, characterized in that: (1) In step three, we first sample the positive and negative samples: our goal is to distinguish between positive and negative samples in the features. The most basic idea is based on pixel embedding design, but the contrast loss based on single pixel embedding does not consider the contextual information between adjacent embeddings. In addition, calculating this loss requires storing a series of The dot product matrix of this size is very memory-intensive. Therefore, we will... Divided into block, thus Get what belongs to .in Furthermore, we perform average pooling on each block, thus obtaining each... All correspond In The region. For the mask, we perform a similar operation... Divided into Block, get get belong The mask contains only 0s and 1s to indicate whether a region has been tampered with. A mean pooling operation is performed relative to the features, and the maximum value of each block is selected as the [value]. The tag. (2) We have obtained the feature embedding and mask embedding Subsequently, we obtain the loss in this domain using formulas (8) and (9). The cross-domain loss is obtained through formulas (10) and (11). Finally, the average of these values ​​is obtained using formula (12) to obtain the final cross-domain loss. Finally, the final loss is obtained by combining it with the binary cross-entropy loss to supervise the segmentation mask.

Images