A JPEG compression artifact learning module and image tamper detection system

Abstract

The application provides a JPEG compression artifact learning module and an image tamper detection system, and the JPEG compression artifact learning module comprises a DCT coefficient processing unit, a quantization table processing unit, a feature splicing unit and a neural convolution processing unit, wherein the DCT coefficient processing unit is used for encoding a DCT coefficient matrix into a DCT plane binary volume representation by using threshold truncation and One-Hot coding, and then performing feature extraction and processing to obtain a first feature map; the quantization table processing unit is used for expanding and processing a quantization table to obtain a second feature map; the feature splicing unit is used for splicing the first feature map and the second feature map; and the neural convolution processing unit is used for performing channel compression, normalization and activation on the spliced feature vector to obtain a final DCT feature. The application enables the CNN to directly learn local JPEG compression artifact patterns, and improves tamper detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a JPEG compression artifact learning module and an image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction. Background Technology

[0002] In the field of digital image processing, image tampering is extremely common and takes many forms. Common types of tampering include splicing, copy-paste, and content restoration forgery. In real-world scenarios, it is often difficult to know in advance which type of tampering, or even multiple tampering methods, have been used on an image. Currently, most image tampering detection methods have significant limitations; they are typically only effective against certain specific types of tampered images. Furthermore, JPEG compression, as the most widely used image compression method, also presents additional challenges to image tampering detection.

[0003] JPEG compression typically involves the following key steps: First, the input image is divided into multiple 8×8 blocks. Each block undergoes Discrete Cosine Transform (DCT) processing to obtain a corresponding 8×8 DCT coefficient matrix. The DCT coefficients contain the frequency information of the image block, including both low-frequency and high-frequency components. Next, the DCT coefficients undergo quantization. Quantization is a crucial step in JPEG compression that leads to information loss. It works by using a predefined quantization table (qtable) to perform division on the DCT coefficients and rounding the result to an integer. This process significantly reduces high-frequency coefficients with smaller amplitudes while preserving low-frequency information, thereby reducing data redundancy and compressing the image. The quantized DCT coefficients are then entropy encoded to further compress the data and reduce redundancy. Finally, the JPEG file stores both the quantized DCT coefficients and the quantization table used, allowing for the most accurate possible image reconstruction during decompression.

[0004] Double JPEG compression leaves specific periodic artifacts in the DCT spectrum, which can be used to effectively detect tampered regions in images. Due to the sparsity, redundancy, and frequency domain distribution characteristics of DCT coefficients, it is difficult for Convolutional Neural Networks (CNNs) to learn effective information directly from the original DCT coefficients. Feature extraction and analysis using DCT histograms or other statistical methods are usually required for effective tamper detection. Therefore, DCT histograms, as a global statistical feature, have significant application value in double JPEG compression detection tasks. However, DCT histograms only reflect the statistical distribution of DCT coefficients, ignoring their spatial organization. This makes it difficult for CNNs to effectively utilize local correlations in such data, resulting in their inability to handle complex scenarios in image tamper detection and limiting them to image-level classification, failing to meet the needs of pixel-level tamper localization. Therefore, effectively preserving the spatial location information of DCT coefficients when extracting JPEG compression artifacts is a crucial problem that urgently needs to be solved. Summary of the Invention

[0005] To address the problems in the prior art, this invention provides a JPEG compression artifact learning module that can effectively preserve the spatial location information of DCT coefficients. It also provides an image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction using the JPEG compression artifact learning module, which effectively improves the detection efficiency and accuracy of the system.

[0006] This invention discloses a JPEG compression artifact learning module, comprising a DCT coefficient processing unit, a quantization table processing unit, a feature concatenation unit, and a neural convolution processing unit. The DCT coefficient processing unit encodes the DCT coefficient matrix into a DCT planar binary volume representation using threshold truncation and One-Hot encoding, then performs feature extraction and processing to obtain a quantized DCT feature map. The DCT feature map is then reshaped into a first feature map that the neural convolution processing unit can learn. DCT stands for Discrete Cosine Transform. The quantization table processing unit expands the quantization table to the same size as the DCT coefficient matrix using repeated array operations. The expanded quantization table is then multiplied element-wise with the DCT coefficient feature map to obtain an inversely quantized feature map. This inversely quantized feature map is then reshaped into a second feature map that the neural convolution processing unit can learn. The feature concatenation unit concatenates the first and second feature maps along the channel dimension to form a concatenated feature vector. The neural convolution processing unit performs channel compression, normalization, and activation on the concatenated feature vector to obtain the final DCT features.

[0007] This invention relates to an image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction, comprising: a spatial feature encoder (SFE) for extracting RGB features from the spatial domain to obtain the spatial features of the image; a frequency domain feature encoder (FFE) including a JPEG compression artifact learning module for extracting JPEG compression artifact features from the frequency domain based on the DCT features output by the JPEG compression artifact learning module to obtain the frequency features of the image; and a convergent reconstruction decoder (PRD) for acquiring the joint features of the spatial and frequency domain features, eliminating spatial and channel redundancy through separation and reconstruction strategies, and outputting a tampering probability map.

[0008] Compared with existing technologies, the advantages of this invention are as follows: When processing DCT coefficients, this invention first uses threshold truncation and One-Hot encoding to encode the DCT coefficient matrix into a binary volume representation of the DCT plane, and then inputs it into the CNN for learning. This approach can well preserve the spatial arrangement information in the DCT spectrum, enabling the CNN to directly learn local JPEG compression artifact patterns based on DCT features.

[0009] This invention's image tampering detection system utilizes JPEG compression artifacts to detect various types of image tampering. It replaces the traditional DCT histogram method with DCT volume representation, effectively preserving the spatial location information of DCT coefficients. RGB features in the spatial domain are introduced as a supplementary detection method, constructing a joint model for dual-branch feature extraction. Simultaneously, spatial and channel reconstruction convolutional modules are introduced to eliminate redundant information in the joint features at the ends of the dual branches, making the network model more focused on the tampered area, thereby effectively improving the system's detection efficiency and accuracy. Attached Figure Description

[0010] To more clearly illustrate the solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the image tampering detection system of the present invention;

[0012] Figure 2 This is a schematic diagram of the JPEG compression artifact learning module structure of the present invention;

[0013] Figure 3 This is a schematic diagram of an embodiment of the present invention that encodes the DCT coefficient matrix into a binary volume representation of the DCT plane;

[0014] Figure 4 This is a schematic diagram of the aggregation reconstruction decoder structure of the present invention;

[0015] Figure 5 This is a schematic diagram of the spatial and channel reconstruction convolution module structure of the present invention;

[0016] Figure 6 This is a flowchart of the spatial reconstruction unit processing method of the present invention;

[0017] Figure 7 This is a schematic diagram of the image tampering detection system of the present invention, showing the results of the detection. Detailed Implementation

[0018] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings are used to distinguish different objects, not to describe a particular order.

[0019] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment to other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention can be combined with other embodiments.

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0021] A universal image tampering detection system refers to a system model that can achieve good detection performance and strong generalization ability when faced with different tampering types and different image formats. To meet the demand for a universal tampering detection method in real-world scenarios, this invention utilizes JPEG compression artifacts for image tampering detection of various tampering types. Furthermore, it introduces RGB features in the spatial domain as a supplementary detection method, constructing a joint model with dual-branch feature extraction to achieve universal tampering detection for both JPEG and non-JPEG format images.

[0022] Existing image tampering detection methods based on JPEG compression artifacts and dual-branch network frameworks mainly suffer from the following problems:

[0023] (1) Due to the decorrelation and local heterogeneity of DCT coefficients, CNNs cannot directly extract effective features from the original DCT coefficients and must use the DCT histogram as an intermediate step. However, the DCT histogram is essentially a statistical distribution and lacks the spatial location information of the original image, making it difficult for the CNN convolution kernel to effectively utilize local correlation on such data. This results in the CNN not being able to handle the complex scenarios in image tampering detection well and being limited to image-level classification, which is difficult to meet the needs of pixel-level tampering localization.

[0024] (2) While using a dual-branch network framework for image tampering detection can improve the model's detection capabilities in various complex tampering scenarios, the dual-branch features overlap in the captured color, texture, and frequency domain artifacts, resulting in the stitched feature map potentially containing a large amount of redundant information. If these redundant features are not effectively removed, the model will face significant computational overhead and may be affected by irrelevant information, thereby affecting its ability to distinguish between tampered and untampered images and ultimately reducing detection performance.

[0025] To address the aforementioned technical problems, this invention proposes an image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction, which is named DVAR-Net (Dual Branch Evidence Network) or simply DVAR-Net, for image tampering detection and localization.

[0026] like Figure 1 As shown, the DVAR-Net of this invention includes a spatial domain feature encoder (SFE), a frequency domain feature encoder (FFE), and a convergent reconstruction decoder (PRD). The two parallel feature extractors, SFE and FFE, form a dual-branch feature extraction framework based on HRNet, extracting multi-scale information from different feature layers in the spatial and frequency domains of the image. In FFE, the image is first input into the JPEG Compression Artifact Learning Module (CALM) to extract various DCT frequency domain features. These artifact features are then input into the backbone network for tamper detection. Subsequently, in PRD, the joint features at the ends of RGB and DCT features are optimized by introducing a Spatial and Channel Reconstruction Convolution (SCConv) module. Through separation and reconstruction strategies, spatial and channel redundancy are effectively suppressed, enhancing the expressive power of the joint features and enabling the network to focus more on key detail features.

[0027] This invention is based on a dual-branch network architecture, extracting RGB features from the spatial domain and JPEG compression artifact features from the frequency domain, and then jointly modeling them. To effectively improve the system's ability to detect JPEG compression artifacts, this invention designs a JPEG compression artifact learning module, using threshold truncation and One-Hot encoding to process the DCT coefficients, effectively preserving the spatial location information of the DCT coefficients. Simultaneously, by introducing SCConv convolutions to eliminate redundant information in the dual-branch joint features, the interference of redundant features on the extraction of key tampering features is reduced, allowing the network model to focus more on the tampered region, thereby effectively improving the model's detection efficiency and accuracy. Finally, a high-precision pixel-level tampering probability map is output.

[0028] The DVAR-Net of this invention exhibits excellent performance in detecting various types of tampering and maintains strong robustness regardless of whether the image is in JPEG format.

[0029] I. Spatial Feature Encoder (SFE)

[0030] RGB features in the spatial domain are crucial clues for detecting image tampering, directly reflecting the visual content and pixel-level details of an image. When an image undergoes tampering operations (such as splicing or deletion), it often introduces human modifications from pixels under different lighting conditions or from different devices, disrupting the natural correlation between RGB channels and causing anomalies such as color inconsistencies, abrupt texture changes, or blurred edges. Analysis of RGB features can effectively detect these localized tampering traces. Furthermore, the RGB color mode is one of the most commonly used color modes in image processing; almost all images, whether original and uncompressed or processed, can be converted to RGB format. This makes RGB features widely applicable. Therefore, this invention selects RGB features in the spatial domain as a supplementary detection clue to adapt to images of different types and sources, and to assist JPEG compression artifact detection methods in effectively detecting non-JPEG images.

[0031] For image tampering detection based on JPEG compression artifacts, compression artifact features are low-level, fine-grained signals that are crucial to the task. HRNet maintains high-resolution feature maps throughout the network, avoiding premature loss of fine-grained information through pooling or large strides. Furthermore, HRNet's multi-resolution branches can simultaneously process low, medium, and high-resolution image features, enhancing feature representation capabilities at each stage through parallel multi-scale feature fusion, making it ideal for extracting complex image features (such as color distribution anomalies and texture mutations). In addition, the feature map size generated by HRNet is highly compatible with the requirements of JPEG compression artifact analysis. Discrete Cosine Transform (DCT), a key component of JPEG compression, transforms image blocks (typically 8×8 pixels) to process image features. Therefore, the smallest processing unit or resolution of DCT is 8×8. This means that in the DCT stream, the lowest frequency features are typically represented within 8×8 pixel blocks. In contrast, HRNet's second-level resolution is typically at a lower scale (e.g., 1 / 4 or 1 / 8 of the original image), which allows HRNet to efficiently handle JPEG compression artifacts. This scale matching not only helps preserve detailed information in high-resolution feature maps but also effectively reduces computational complexity.

[0032] The SFE of this invention adopts the HRNet network architecture. As an embodiment of this invention, this example includes four stages. First, an RGB image is taken as input. Before being input into the backbone, the image undergoes two 3×3 convolutions with a stride of 2, which reduces the image resolution to 1 / 4 of its original value. The image then enters the first stage of the network. In the first stage, the network maintains... The resolution of the feature maps is then determined. Subsequently, in the second, third, and fourth stages, the resolutions of the feature maps are as follows: , and After convolution at these four resolutions, the feature channels extracted from the image at the four scales are C1=48, C2=96, C3=192, and C4=384, respectively. This multi-resolution structure design enables the network to acquire editing information related to image manipulation from multiple scales, thereby effectively capturing brightness anomalies, correlation changes, and spatial domain feature anomalies caused by other image operations between RGB channels.

[0033] II. Temporal Feature Encoder (FFE)

[0034] While the frequency domain feature encoder also has four stages, unlike the spatial domain feature encoder, it employs three different resolution configurations of HRNet. Specifically, the second, third, and fourth stages of the frequency domain feature encoder correspond to... , and The resolution is consistent with the resolution setting in the spatial domain feature encoder. The first stage is replaced by the JPEG compression artifact learning module (CALM).

[0035] The workflow of a frequency domain feature encoder is as follows:

[0036] First, the input image is fed into CALM to extract compression artifact features. These features are then fed into the backbone network, utilizing repetitive modular multi-resolution blocks for subsequent tamper detection. The extracted DCT features can be directly input into the second stage for further processing because in image processing, DCT transformation is performed on multiple 8×8 image blocks, a process that perfectly matches the 1 / 8 resolution branch in the second stage. Through the design of the multi-resolution module, the image's frequency domain features are extracted and mapped to three scales, corresponding to channel numbers C2=96, C3=192, and C4=384. Finally, the module outputs three resolutions: × , × , × The feature maps are shown. These three resolutions correspond to the resolutions of the last three stages in the spatial domain feature encoder. In this way, the frequency domain feature encoder maintains a certain structural consistency with the spatial domain feature encoder in terms of spatial feature extraction when processing frequency domain features, providing a good foundation for subsequent bi-branch multi-resolution feature fusion.

[0037] In summary, in order to fully capture tampering traces in both the spatial and frequency domains of an image, the present invention employs a dual-branch feature encoder (such as...) Figure 1 As shown, HRNet is used as the basic framework to extract features from the spatial domain and frequency domain, thereby realizing fine-grained detection of tampered regions in JPEG and non-JPEG compressed images.

[0038] like Figure 2 and Figure 3 As shown, the JPEG compression artifact learning module of this invention includes a DCT coefficient processing unit, a quantization table processing unit, a feature splicing unit, and a neural convolution processing unit, wherein...

[0039] The DCT coefficient processing unit is used to encode the DCT coefficient matrix into a binary volume representation of the DCT plane using threshold truncation and One-Hot encoding, and then perform feature extraction and processing to obtain a quantized DCT feature map. The DCT feature map is then reshaped and transformed into a first feature map that the neural convolution processing unit can learn. DCT stands for Discrete Cosine Transform.

[0040] The quantization table processing unit is used to expand the quantization table to the same size as the DCT coefficient matrix using repeated array operations, and then multiply the expanded quantization table element-wise with the DCT coefficient feature map to obtain the dequantized feature map. Then, the dequantized feature map is reshaped and transformed into a second feature map that the neural convolution processing unit can learn.

[0041] The feature splicing unit is used to splice the first feature map and the second feature map in the channel dimension to form a spliced ​​feature vector.

[0042] The neural convolution processing unit is used to perform channel compression, normalization, and activation on the spliced ​​feature vector to obtain the final DCT features.

[0043] 1. DCT coefficient processing unit.

[0044] In this invention, the DCT coefficient processing unit first uses threshold truncation and One-Hot encoding to encode the DCT coefficient matrix into a binary volume representation of the DCT plane before inputting it into the CNN for learning. This approach effectively preserves the spatial arrangement information in the DCT spectrum, enabling the CNN to directly learn local JPEG compression artifact patterns based on DCT features.

[0045] Specifically, for an input image I∈R 3×H×W H represents the image height, and W represents the image width. In the DCT coefficient processing stage, the quantized DCT coefficients and corresponding quantization tables are first obtained from the JPEG header file. The quantized DCT coefficients include coefficients from three channels: Y (luminance channel), Cb, and Cr (chrominance channels). Since JPEG compression leverages the theory that the human eye's perception of image luminance is far stronger than its perception of chrominance, it fully preserves the luminance component (Y channel) while compressing the chrominance components (Cb and Cr channels) more. Therefore, in this invention, only the pair of DCT coefficient matrices D∈Z from the Y channel component, which are most helpful for evidence collection, are extracted. H×W Perform the analysis.

[0046] The coefficients after DCT transformation exhibit a symmetrical distribution in the frequency domain. To extract features more effectively, the absolute value of the DCT coefficient matrix is ​​taken before further processing to ignore sign information and focus on the amplitude features in the frequency domain. Since the range of the quantized DCT coefficients is uncertain, their absolute values ​​may fluctuate significantly. If one-hot encoding is directly applied to the DCT coefficient matrix, the dimension of the generated binary volume channels will increase linearly with the dynamic range of the DCT coefficients, leading to uncontrollable memory consumption and an explosion in model parameters. Furthermore, the sparse representation of a large number of high-frequency noise channels will reduce the weights of key features related to tampering, thereby reducing the model's detection efficiency and accuracy. Therefore, this invention first truncates the DCT coefficient matrix by a threshold T based on its absolute value to limit its dynamic range. Then, one-hot encoding is used to convert it into a binary volume tensor. .

[0047] The specific processing formula for encoding the DCT coefficient matrix into a binary volume representation of the DCT plane using threshold truncation and One-Hot encoding in this invention is as follows:

[0048]

[0049] in, This is an indicator function that returns 0 or 1, where T is the first threshold, D is the DCT coefficient matrix, and D' is the binarized coefficient matrix of the binary volume representation of the DCT plane. This means that if the binarized coefficient matrix is ​​equal to t, then output 1; otherwise, output 0. This means that if the value of the binarized coefficient matrix is ​​greater than or equal to t, the output is 1; otherwise, the output is 0.

[0050] In the processing of this invention, a larger T value can preserve a wider range of DCT coefficient distributions, thereby capturing richer frequency domain features. However, this also increases computational and memory requirements. Conversely, a smaller T value may not adequately preserve low- and mid-frequency components, thus affecting the model's ability to capture compression artifacts. Therefore, choosing a reasonable T value is crucial for balancing frequency domain feature extraction and computational resources. In this embodiment, to extract as many frequency domain features as possible, the final T value is set to 20 without exceeding GPU memory limitations.

[0051] Figure 3 The diagram shows the state changes of DCT coefficients after threshold truncation and One-Hot encoding. After this DCT volume representation operation, the converted binary volume tensor is obtained. , It has 21 (T+1) channels, where each channel is a binary matrix of size H×W, corresponding to the quantization range of a DCT coefficient. Subsequently, The features will be extracted and processed through two convolutional modules, such as... Figure 2 As shown.

[0052] First, the model enters the first convolutional module, which employs a two-branch structure to extract JPEG artifact features at different scales. The first branch uses a 3×3 dilated convolution (dilation ratio of 8) to expand the receptive field, thereby extracting large-scale artifact information and enhancing the model's ability to perceive global artifact patterns. The second branch uses a standard 3×3 convolution (3×3 Conv), focusing on local artifact features to accurately capture small-scale tampering regions. Both branches undergo batch normalization to ensure stable feature distribution and are then fused through channel concatenation, followed by ReLU activation to enhance non-linear representation capabilities. Next, the features enter the second convolutional module, which contains a 1×1 convolutional layer (1×1 Conv) to further integrate cross-channel information and reduce computational complexity. It also undergoes batch normalization and ReLU activation. Finally, the processed quantized DCT feature map is obtained. The overall processing formula is:

[0053]

[0054] in, These represent the ReLU activation function, batch normalization, 3×3 dilated convolution, standard 3×3 convolution, and 1×1 convolution, respectively, resulting in the final output. .

[0055] 2. Quantitative Table Processing Unit

[0056] In the quantization table processing unit, a Y-channel quantization table (Y-QuantizationTable, Q) is input. y The quantization table is an 8×8 matrix. First, to ensure consistency of quantization information across different regions in subsequent processing, repeated array operations are used to expand the quantization table to the same size as the DCT coefficient matrix, i.e. Then, the expanded quantization table is multiplied element-wise by the DCT coefficient feature map, and the calculation formula is as follows:

[0057]

[0058] The feature map after "inverse quantization" is obtained. Inverse quantization helps the network recover the true information of the image from these features, thereby detecting whether the image has been tampered with.

[0059] In an 8×8 DCT coefficient block, each coefficient position represents a specific frequency component. However, the convolutional operations in the backbone network are designed for RGB color images, relying primarily on the spatial neighborhood information of pixels for feature extraction. Since different coefficient positions in the DCT spectrum represent frequency information, not direct spatial locations, CNNs cannot directly learn the correlation information between DCT frequencies. To fully utilize the spatial and frequency information of the DCT coefficients, a reshaping operation is needed to enable the CNN to correctly learn frequency features.

[0060] This example reshapes each DCT block of size (8×8×1) into a (1×1×64) vector, achieving the transformation from... arrive The index transformation mapping is performed. In this (1×1×64) vector, the first and second dimensions represent the spatial location information of the DCT block, and the third dimension represents the 64 DCT frequency components (8×8), i.e., feature vectors of different frequencies. At this point, the CNN can learn the 64-dimensional DCT frequency relationships without mistakenly identifying them as neighborhood relationships of spatial pixels. Therefore, for the quantized DCT features... DCT features with "inverse quantization" All of these undergo reshaping operations to convert the DCT frequency components into a format suitable for CNN processing. At this point, after reshaping...

[0061] The feature maps obtained after the shaping operation are as follows: , .

[0062] 3. Feature splicing unit and neural convolution processing unit.

[0063] The feature maps obtained from the two branches are concatenated along the channel dimension to form a feature vector of size 512, which is then input into a 1×1 convolutional layer for channel compression, normalization, and activation to obtain the final DCT features. And output the calculation formula:

[0064]

[0065] At this point, the size of the feature map exactly matches the resolution of the second stage of HRNet, ensuring... It can be directly input into the backbone network of the frequency domain feature encoder to continue performing subsequent detection tasks.

[0066] III. Aggregated Reconstruction Decoder PRD.

[0067] like Figure 4As described above, the aggregation reconstruction decoder PRD consists of a multi-resolution convolution and a spatial and channel reconstruction convolution (SCConv) module. In the PRD, the output features in the SFE and FFE are fused to make full use of the RGB spatial domain information and the DCT frequency domain artifact features, thereby enhancing the detection accuracy and localization ability of tampering artifacts. At the same time, a spatial and channel reconstruction mechanism is introduced. By eliminating the redundant information of the end features, correcting the distortion of spatial information, and optimizing the channel allocation, it is ensured that the network can more accurately learn the key features for distinguishing tampered and untampered images. The following is a specific description.

[0068] In this example, the spatial domain feature encoder SFE adopts a four-stage HRNet network structure to generate four types of feature maps with different resolutions. The frequency domain feature encoder FFE adopts a three-stage HRNet network structure to generate three types of feature maps with different resolutions. The joint splicing module is for the spatial domain features extracted by the spatial domain feature encoder SFE and the frequency domain features extracted by the frequency domain feature encoder FFE . First, they are spliced in the channel dimension according to the corresponding resolutions to obtain joint features 、 、 . The features and 、 、 are input into the multi-resolution fusion module together.

[0069] In the multi-resolution fusion module, the transform function is used to process the multi-resolution feature R to obtain the end joint features: 、 、 、 . The specific calculation method is:

[0070]

[0071]

[0072] where x represents the input resolution index, r represents the output resolution index. When x < r, downsampling operations are performed using (r - x) 3×3 convolutions with a stride of 2; when x = r, a flat-level operation is adopted; when x > r, upsampling is performed using bilinear interpolation, and then a 1×1 convolution is used to align the number of channels.

[0073] So far, the present invention obtains four joint feature output representations after multi-resolution fusion 、 、 、 Subsequently, the terminal features are fused and redundancy is eliminated through upsampling and spatial and channel reconstruction convolution (SCConv) modules. Finally, after convolution, normalization, and activation operations, a tampering probability map is generated.

[0074] In the HRNet-based dual-branch network structure, the SFE and FFE branches extract feature information from the image in the spatial and frequency domains, respectively. However, there is overlap between the color and texture information captured by RGB features and the frequency components and compression artifacts in DCT features, resulting in a large amount of redundant information in the final stitched feature map. If these redundant features are not effectively filtered, the model not only needs to handle greater computational overhead but may also be affected by irrelevant information, thus failing to effectively learn the key features that distinguish tampered from tampered images, ultimately affecting the detection performance.

[0075] Furthermore, in the upsampling module of this invention, bilinear interpolation upsampling is used to adjust feature maps of different scales to a uniform size for stitching. Figure 4 (As shown by the red dashed line in the image), a convolutional layer is then used for final feature fusion or prediction. However, in this process, bilinear upsampling blurs the precise information learned in the feature map, failing to accurately preserve the original spatial information structure, thus leading to inaccurate spatial information. Image tampering detection highly depends on the detailed information in the image, especially features such as the texture, edges, and color distribution of regions. If the precise information of the tampered region is lost, the detection algorithm will have difficulty identifying anomalies or forgery traces in that region. Therefore, this invention uses the Spatial and Channel Reconstruction Convolution (SCConv) module to process the terminal features.

[0076] like Figure 5 As shown, the present invention SCConv includes a Spatial Reconstruction Unit (SRU), a Channel Reconstruction Unit (CRU), and a Residual Connection Unit. The SRU suppresses spatial redundancy through separation and reconstruction. Simultaneously, by dynamically weighting each spatial location in the feature map, it corrects spatial information distortion caused by upsampling, thereby avoiding the blurring effect caused by simple bilinear interpolation. The CRU adaptively weights the importance of different channels, dynamically adjusting channel allocation and reducing channel redundancy.

[0077] The specific processing procedure of SCConv in this invention is as follows:

[0078] Input features First, the spatial location is dynamically weighted using a SRU process to obtain refined spatial features. Subsequently, through CRU, the importance of different channels is adaptively weighted, and the channel allocation is dynamically adjusted to obtain channel refinement features. While SRU and CRU can optimize feature distribution in both spatial and channel dimensions and eliminate redundant information, residual connection units are introduced in SCConv to avoid losing important discriminative features or altering the semantic information of the ground truth region during the optimization process. These residual connection units convert the original input features... With optimized channel refinement features Add them together to obtain the final output features. .

[0079] like Figure 6 As shown, SRU is mainly based on two core steps: separation and cross-reconstruction, aiming to optimize feature representation and reduce unnecessary redundancy. The core objective of the separation step is to distinguish the parts of the feature map containing rich information from those containing less spatial information. To achieve this goal, firstly, trainable parameters in the GN (Group Normalization) layer are used to evaluate the spatial information content of different feature maps. GN groups channels (the number of groups is set to 4 in this invention) and calculates the mean and variance within each group for normalization. Using GN better reflects the spatial information content of channels within each group. Secondly, a gating mechanism is used to distinguish between information-rich features and information-poor features.

[0080] The specific processing method for this example is as follows:

[0081] For input features Where B is the batch size and C is the number of channels, firstly, for the input features... Standardization is performed by subtracting the mean and dividing by the standard deviation. This is done to account for cases where the standard deviation might be zero if some feature maps have no channel variation. To avoid zero errors, the standard deviation is... Replace the value with ,in, For a very small positive number, the specific calculation formula is:

[0082]

[0083] in, β and μ are trainable affine transformation parameters, μ is the mean of the feature map, and i and j are specific channels. The trainable parameters in the GN layer are utilized. This serves as a metric for measuring the spatial pixel variance within each batch and channel. Generally, richer spatial information means greater variation in spatial pixels, which in turn leads to… The larger the value, the higher the normalized correlation weights can be obtained using this formula. This weight reflects the importance of different feature maps. Next, we will proceed through... The weighted feature map weights are mapped to the range (0,1) using the sigmoid function, and a gate operation is performed using a threshold (set to 0.5 in this example) to obtain the information weight W. Weights greater than the threshold are set to 1 to obtain the information weight W. For weights less than or equal to the threshold, they are set to 0 to obtain the information weights. Subsequently, the input feature X is compared with... and Multiplying them together yields two weighted features: and . It contains spatial content that is both informative and expressive, and Features containing little or no information are considered redundant. To further reduce spatial redundancy, a cross-reconstruction operation is used to combine information-rich features with information-poor features to enhance their expressive power. Subsequently, the cross-reconstructed features are... and By stitching the images together, a spatially refined feature map is obtained. The specific calculation formula is as follows:

[0084]

[0085] Here, ⊕ represents element-wise summation, and ∪ represents concatenation. By processing the input features in the SRU, not only is the goal of separating features with high information content from those with low information content achieved, but also detailed features are enhanced through reconstruction operations, and redundant features in the spatial dimension are suppressed. However, some redundancy still exists in the channel dimension. Therefore, spatial refinement of features... It is then fed into the CRU for further optimization.

[0086] This CRU example includes three steps: splitting, conversion, and fusion. The specific processing method is as follows:

[0087] (1) Splitting step: refine the spatial features of the input Divided into two parts according to the passage. Channels and The passage, in which By using 1×1 convolutional units to compress the channels of the feature map, spatial features are refined. Divided into first-space refinement features Second space refinement features .

[0088] (2) Transformation steps: First spatial refinement features The data is fed into an upper-level extractor for processing, where rich and representative features are extracted to obtain the output features. Second space refinement features The data is fed into a lower-level extractor for processing, generating a feature map with shallow details, resulting in the output feature map. .

[0089] In the upper-layer extractor, a combination of grouped convolution (GConv) and pointwise convolution (PConv) is used to replace the traditional standard convolution, extracting rich and representative features with less computational cost. Grouped convolution reduces the number of parameters and computational complexity through sparse convolutional connections, but it also limits the information transfer between channel groups. To compensate for this deficiency, pointwise convolution effectively promotes the flow of information between functional channels, ensuring the integrity of features.

[0090] In the lower-level extractor, only a single 1×1 convolution is used to generate a feature map with shallow details, supplementing the upper-level feature extractor. Simultaneously, the input second-space refinement features are... The features are then concatenated with the features obtained after pointwise convolution to obtain the output features. .

[0091] (3) Fusion step: Using a simplified SKNet method, the output features from the upper and lower extractors are adaptively fused. and The specific processing method is as follows:

[0092] First, global spatial information is extracted using global average pooling to obtain channel-wise descriptors. Next, the channel descriptors S1 from the upper-layer extractor and S2 from the lower-layer extractor are stacked, and feature importance vectors β1 and β2 are calculated using a channel-level soft attention mechanism. Finally, guided by the feature importance vectors β1 and β2, the output features of the upper layer are processed along the channel dimension. and the output features of the lower layer Weighted fusion is performed to obtain channel refinement features. .

[0093] Although JPEG is one of the most widely used formats for storing image data, many other image formats exist in real-world scenarios. Non-JPEG images may not contain the DCT coefficients or quantization tables required in the Frequency Domain Feature Encoder (FFE) of this invention. Therefore, this invention processes non-JPEG images in the following manner, enabling DVAR-Net to correctly detect tampering in non-JPEG images.

[0094] Before detection, the image tampering detection system of this invention also includes an image judgment module, used to determine whether the image is in JPEG compressed image format, uncompressed image format, or other lossy compressed image format besides JPEG compressed image format. Then, each format is processed accordingly.

[0095] (1) Uncompressed format and lossless compression format.

[0096] Uncompressed images (such as RAW and BMP) are, as the name suggests, images that have not undergone any compression, retaining the original pixel values. Lossless compressed images (such as PNG, TIFF, and GIF), while compressed, still retain complete image data and can be perfectly restored to the original image after decompression. Since uncompressed and lossless compressed images have similar characteristics in preserving frequency domain information, both can retain the original information and details of the image. Therefore, the same method can be used in image tampering detection tasks. For the convenience of the following description of this invention, both will be collectively referred to as uncompressed images.

[0097] In JPEG compression, the image is first converted from the RGB color space to the YCbCr color space. Then, the Y (luminance) channel undergoes a DCT transform, while the Cb and Cr (chrominance) channels are typically downsampled at 4:2:2 or 4:2:0 to reduce the resolution of chrominance information and thus reduce the amount of data. Next, the DCT coefficients are quantized. However, uncompressed images directly store RGB values ​​without undergoing DCT transformation, making it impossible to directly obtain the DCT coefficients. To process these images, DVAR-Net employs a strategy that simulates JPEG compression. The specific processing method is as follows:

[0098] 1. DCT Processing: DVAR-Net first converts the original image from RGB to YCbCr color space, then divides the image into multiple 8x8 blocks, performing DCT transformation on each block to obtain DCT coefficients. During this process, chroma downsampling is not performed to preserve color information to the maximum extent possible.

[0099] 2. Quantization Processing: In actual JPEG compression, the quantization table reduces and rounds the DCT coefficients, thus losing some information to achieve compression. To simulate compression, we assume all elements in the quantization table are 1, and then quantize the DCT coefficients. This way, the accuracy of the DCT coefficients is almost unaffected, closely approximating the original data of the uncompressed image, and yielding the corresponding quantization table and the quantized DCT coefficients.

[0100] To simulate this compression process, this invention performs JPEG compression on the original image with a quality factor (QF) of 100 and a chroma sampling ratio of 4:4:4. When QF=100, the elements in the quantization table are close to 1, with almost no rounding. The 4:4:4 chroma sampling setting ensures consistent resolution for the Y, Cb, and Cr channels, avoiding chroma downsampling. In this way, the uncompressed state is successfully simulated, while preserving the frequency domain information of the original image to the maximum extent.

[0101] It is important to note that when processing uncompressed images, this invention does not rely solely on the spatial encoder for anomaly feature extraction; instead, it performs DCT coefficient analysis on both images. This is because an uncompressed tampered image only indicates that it was not compressed during final saving, but JPEG compression artifacts may still exist during the image acquisition stage. Extracting and analyzing the DCT coefficients of the uncompressed image helps improve detection accuracy.

[0102] (2) Other lossy compression formats.

[0103] Due to the use of DCT coefficients and quantization tables, the frequency domain feature encoder is not suitable for analyzing tampering traces in images using other lossy compression formats (such as HEIC, WebP, etc.). However, in this case, DVAR-Net can still rely on the spatial domain feature encoder to extract RGB anomaly features for image tampering detection.

[0104] To objectively evaluate model performance, this invention used nine standard image forgery detection datasets throughout the experiment. In most image tampering detection methods, the same dataset is typically divided proportionally into training and testing sets. However, this division may lead to potential similarities between the training and testing data (e.g., the same image source, similar tampering methods, or noise distribution). To more comprehensively evaluate the model's generalization ability and to better reflect real-world application scenarios, this invention used completely different datasets for the training and testing phases. Five datasets with smaller data volumes were selected for model testing, while the remaining datasets were used for model training. Detailed information on the training and testing sets is shown in Table 1.

[0105] Table 1 Experimental Dataset

[0106]

[0107] In Table 1, the `auth.` and `tamp.` fields represent the number of real images and the number of forged images, respectively. The tampering types in the training dataset include splicing forgery and copy-and-move forgery; the tampering types in the test dataset include splicing forgery, copy-and-move forgery, and repair forgery.

[0108] I. Constructing the dataset.

[0109] 1. Training Dataset: This invention uses four standard image detection datasets for training. The four standard image detection datasets are CASIAv2, FantaticReality, tamperedCOCO, and tamperedRAISE.

[0110] CASIAv2: CASIAv2 is a dataset specifically designed for image tampering detection research, containing 7491 real images and 5105 forged images. The real images come from multiple individuals, while the forged images are generated in different ways, including splicing forgeries and copy-and-move forgeries.

[0111] FantasticReality: This dataset focuses on high-quality image stitching forgery detection. Its forgery methods are sophisticated, and the image scenes closely resemble real-life and natural scenes. The dataset contains 16,592 real images and 19,423 forged images. The real images use 153 different quantization tables, but all the forged images use only the same quantization table.

[0112] TamperedCOCO: Common training sets for fake images contain a limited number of images and a limited number of quantization tables, insufficient to represent the real-world image distribution and compression artifacts. Therefore, five custom datasets were created for training: SPCOCO, CMCOCO, CMRAISE, CM-JPEGRAISE, and JPEGRAISE. The tamperedCOCO dataset consists of two fake datasets: SPCOCO and CMCOCO. The SPCOCO dataset is a splicing fake detection dataset built from the COCO2017 dataset, containing 200,000 fake images. In the COCO2017 dataset, one or more objects from one image are pasted onto another image, and random rotation and scaling are used to automatically create tampered images. Finally, these images are subjected to JPEG compression within a random quality factor range of 60-100 to obtain the SPCOCO dataset. The CMCOCO dataset is also built from the COCO2017 dataset and contains 200,000 fake images. Unlike the SPCOCO dataset, the CMCOCO dataset is specifically designed for detecting copy-and-move forgeries. Specifically, it involves randomly selecting an object within the same image and copying it one or more times. Subsequently, it undergoes JPEG compression within a random quality factor range of 60-100 to obtain the CMCOCO dataset.

[0113] The tamperedRAISE dataset consists of three forged datasets: CMRAISE, CM-JPEGRAISE, and JPEGRAISE. These are all forged image datasets created from the RAISE dataset. The CMRAISE dataset is specifically designed for copy-and-paste tampering and contains 200,000 forged images. The CMRAISE dataset is constructed by first compressing the RAISE dataset images, then using random, irrelevant polygons to outline arbitrary regions of images from the COCO dataset, pasting these regions onto the compressed RAISE dataset image, and finally compressing the entire image. It's important to note that when a background region in a COCO dataset image is selected for copy-paste, this tampering is similar to a deletion tampering operation. The CM-JPEGRAISE dataset is obtained by further compressing the CMRAISE dataset, also containing 200,000 forged images. This approach simulates the further compression often required due to upload or bandwidth limitations when sending tampered images via social networks. JPEGRAISE is a real dataset created by simply compressing the RAISE dataset, containing 24,462 real images.

[0114] 1. Test Datasets: This invention evaluated the manipulation localization on five standard test datasets: Columbia, Coverage, CoMoFoD, GRIP, and IMD2020.

[0115] Columbia: This dataset contains 180 stitched images for stitch forgery detection. In this dataset, the source images were taken with different cameras, while the forged images are high-quality, uncompressed images, thus leaving no compression artifacts.

[0116] Coverage: This is a dataset specifically designed for image copying and moving forgery detection. It aims to highlight and address the ambiguities in tamper detection caused by the self-similarity of natural images in popular methods. It contains 100 pairs of images, each pair consisting of an original image and a corresponding forged image, for a total of 200 images. The forged images are overlaid with objects similar to those in the original images, with extremely fine forgery details.

[0117] CoMoFoD: The CoMoFoD dataset for detecting copied moving forgeries consists of 200 real images and 200 forged images. Based on the application's operation, the forged images are divided into 5 groups: translation, rotation, scaling, combination, and distortion. This dataset features different types of post-processing methods, such as JPEG compression, blurring, adding noise, color reduction, etc., applicable to all forged and original images, and presents a certain level of challenge.

[0118] GRIP: This dataset contains 80 replicated motion-forged images and their corresponding real-world masks. Its real-world masks include not only the forged object region but also the source object region. Therefore, the source object region needs to be manually removed to ensure the mask is consistent with masks in other digital image forgery datasets. Forged objects include animals, buildings, and natural landscapes.

[0119] IMD2020: This is a large-scale labeled dataset specifically designed for image tampering detection. It contains 2010 forged images and manually created binary masks for each forged image to indicate the tampered areas. Tampering types include copy-and-move forgery, splicing forgery, and repair forgery. These 2010 forged images are real-life tampered images collected from the internet, reflecting real-world image compression schemes at the media level.

[0120] II. Experimental Evaluation Indicators

[0121] In digital image tampering detection tasks, model performance evaluation mainly relies on two core metrics: pixel-level localization accuracy and image-level classification accuracy. The former measures the model's ability to detect and segment tampered regions at the pixel level, while the latter reflects the model's tampering recognition performance at the overall image level. In the experiments of this invention, pixel-level F1 scores are used to evaluate the localization results. For image-level detection, image-level accuracy (ACC) is used to measure the overall detection performance. Furthermore, to avoid the influence of thresholds on model performance, we also use the threshold-free evaluation metric AUC. AUC can overcome the bias that may arise from relying solely on a single threshold (such as the F1 score), providing a comprehensive evaluation method for model performance across all thresholds. The relevant definitions are as follows:

[0122]

[0123] In the formula, Precision represents accuracy, Recall represents recall, TP, TN, FP, and FN refer to the tampered pixels correctly identified by the model, the real pixels correctly identified by the model, the real pixels falsely detected by the model, and the tampered pixels falsely detected by the model, respectively; G and P represent the ground truth mask and the model prediction output, respectively.

[0124] III. Experimental Comparison Methods.

[0125] In this invention's experiments, several mainstream and state-of-the-art methods were selected from both traditional feature extraction-based and deep learning-based methods as comparison objects, and uniform evaluations were performed on the same benchmark dataset to ensure fairness in the comparison. Traditional feature extraction-based comparison methods included: DCT, ADQ, CFA, and EXIF. Deep learning-based comparison methods included: MFCN, Noiseprint, ManTra-Net, CAT-Net, and TruFor.

[0126] During training, RGB images and JPEG-encoded DCT coefficients and quantization tables were used as inputs to the RGB and DCT streams, respectively. Image sizes were uniformly set to 512×512, and RGB images were normalized to the [0,1] range. The model was trained using the SGD optimizer with an initial learning rate of 0.005, momentum of 0.9, and weight decay of 0.0005. The learning rate was decayed to 50% every 10 epochs. The batch size was 9 (3 GPUs trained in parallel, each GPU executing 3 samples), for a total of 200 epochs. The loss function was binary cross-entropy loss, with a five-fold increase in weight on the tampered class to address class imbalance.

[0127] IV. Experimental Results and Analysis.

[0128] This invention conducted comparative experiments on publicly available image tampering localization benchmark datasets, selecting mainstream and state-of-the-art methods from recent years and performing unified evaluations on the same benchmark dataset to ensure fairness in the comparison. Finally, the performance metrics of each method on the test set were statistically analyzed to evaluate their detection effectiveness. In the experimental results table, "-" indicates that the dataset is not suitable for the method and cannot provide reliable test values; bold black values ​​represent the best performance in that column of the dataset; underlined values ​​represent the second-best performance, highlighting the detection effect that is just below the best, making the data comparison more intuitive and clear.

[0129] (1) Pixel-level tampering detection.

[0130] In tampering detection tasks, pixel-level evaluation focuses on the model's accuracy in identifying tampered regions, i.e., whether the detection model can correctly segment the tampered areas. Table 2 shows the experimental results of the proposed method DVAR-Net compared with nine other algorithms in pixel-level tampering detection tasks, with the F1 score used as the evaluation metric. The comparison of the pixel-level performance metrics (F1 scores) for various methods is shown in Table 2.

[0131] Table 2 Comparison of pixel-level performance metrics (F1 scores) between DVAR-Net and other methods.

[0132]

[0133] Overall, the experimental results show that all deep learning-based methods generally outperform traditional feature extraction-based methods, demonstrating the advantages of deep learning in pixel-level detection tasks. The proposed method, DVAR-Net, achieves state-of-the-art performance on the Columbia, CoMoFoD, Coverage, and IMD2020 datasets. Although DVAR-Net's F1 score is slightly lower than CAT-Net and TruFor on the GRIP dataset, the difference is within an acceptable range. This indicates that DVAR-Net has strong generalization ability and performs consistently and superiorly on different datasets. On the IMD2020 dataset, DVAR-Net achieves an F1 score of 0.690, an improvement of 0.027 compared to TruFor (the best among deep learning methods) and a significant improvement of 0.363 compared to EXIF ​​(the best among traditional methods), indicating that DVAR-Net has superior pixel-level manipulation detection capabilities on this dataset. This result may be attributed to DVAR-Net's detection mechanism based on JPEG compression artifact analysis. The IMD2020 dataset contains a large number of tampered JPEG images, allowing the method to fully utilize compression artifact features and accurately detect tampered regions. Traditional feature extraction-based DCT methods performed the worst on the five datasets, primarily because they rely on DCT histograms to identify inconsistencies in JPEG images. Detection performance deteriorates significantly when the DCT statistical features of the tampered region are similar to those of the untampered region, or when the tampering method does not significantly alter the DCT coefficient distribution.

[0134] (2) Image-level tampering detection.

[0135] To further verify the effectiveness of DVAR-Net, ACC and AUC were used as performance evaluation metrics to analyze its performance in image-level tamper detection. The test results are shown in Table 3.

[0136] Table 3 Comparison of image-level performance metrics ACC & AUC results between DVAR-Net and other methods.

[0137]

[0138] Experimental results show that the detection method based on deep learning generally outperforms traditional feature extraction-based methods. On the Columbia, CoMo-FoD, and IMD2020 datasets, the system of this invention achieves the best performance in terms of ACC / AUC, reaching 0.843 / 0.859, 0.615 / 0.622, and 0.722 / 0.741, respectively.

[0139] (3) Visualization of DVAR-Net detection results.

[0140] like Figure 7 As shown, this invention visualizes the detection results of DVAR-Net on five test datasets, with two test images selected from each dataset for display. Figure 7 The first to fifth rows correspond to the detection results on the Columbia, Coverage, Co-MoFoD, GRIP, and IMD2020 test datasets, respectively. Figure 7 As can be seen, the image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction proposed in this invention can accurately locate the tampered region, regardless of the type of tampering, image format, or post-processing. Notably, DVAR-Net performs well even on the challenging Coverage and CoMoFoD datasets.

[0141] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the present invention are within the protection scope of the present invention.

Claims

1. A method for learning JPEG compression artifacts, characterized in that, It includes a DCT coefficient processing unit, a quantization table processing unit, a feature splicing unit, and a neural convolution processing unit, among which, The DCT coefficient processing unit is used to encode the DCT coefficient matrix into a binary volume representation of the DCT plane using threshold truncation and One-Hot encoding, and then perform feature extraction and processing to obtain a quantized DCT feature map. The DCT feature map is then reshaped and transformed into a first feature map that the neural convolution processing unit can learn. DCT stands for Discrete Cosine Transform. The quantization table processing unit is used to expand the quantization table to the same size as the DCT coefficient matrix using repeated array operations, and then multiply the expanded quantization table element-wise with the DCT coefficient feature map to obtain the dequantized feature map. Then, the dequantized feature map is reshaped and transformed into a second feature map that the neural convolution processing unit can learn. The feature splicing unit is used to splice the first feature map and the second feature map in the channel dimension to form a spliced ​​feature vector. The neural convolution processing unit is used to perform channel compression, normalization, and activation on the spliced ​​feature vector to obtain the final DCT features.

2. The JPEG compression artifact learning method according to claim 1, characterized in that, Also includes: The image determination module is used to determine whether an image is a JPEG compressed image format, an uncompressed image format, or a lossy compressed image format other than JPEG compressed image format. Uncompressed image format processing unit: used to perform JPEG compression processing on the raw image in uncompressed image format to obtain the quantization table and the quantized DCT coefficients.

3. The JPEG compression artifact learning method according to claim 1, characterized in that, The specific processing method for encoding the DCT coefficient matrix into a binary volume representation of the DCT plane using threshold truncation and One-Hot encoding is as follows: in, Let T be the indicator function, T be the first threshold, D be the DCT coefficient matrix, D' be the binarized coefficient matrix of the binary volume representation of the DCT plane, and t be the comparison parameter for the values ​​of the binarized coefficient matrix.

4. An image tampering detection system based on DCT volume representation and spatial channel adaptive reconstruction, characterized in that, include: Spatial feature encoder: used to extract RGB features from the spatial domain to obtain the spatial features of the image; Frequency domain feature encoder: includes a JPEG compression artifact learning module, the JPEG compression artifact learning module is used to execute the JPEG compression artifact learning method according to any one of claims 1-3, and the frequency domain feature encoder is used to extract JPEG compression artifact features from the frequency domain based on the DCT features output by the JPEG compression artifact learning module to obtain the frequency domain features of the image. Aggregation Reconstruction Decoder: Acquires joint features of spatial and frequency domain features, eliminates spatial and channel redundancy through separation and reconstruction strategies, and outputs a tampering probability map.

5. The image tampering detection system according to claim 4, characterized in that, The spatial domain feature encoder adopts an N-stage HRNet network structure, and the frequency domain feature encoder FFE adopts an N-1-stage HRNet network structure. The DCT feature resolution output by the JPEG compression artifact learning module is consistent with the resolution output by the first stage of the spatial domain feature encoder SFE, and the image resolution of the N-1 stage of the frequency domain feature encoder FFE is consistent with the resolution output by the Nth stage of the spatial domain feature encoder SFE, where N is a positive integer greater than 2.

6. The image tampering detection system according to claim 5, characterized in that, The aggregation reconstruction decoder includes: Joint splicing module: used to splice the obtained spatial features and frequency domain features in the channel dimension to obtain N - 1 joint features; Multi-resolution fusion module: obtains N - 1 joint features and the spatial features output by the first stage of the spatial feature encoder SFE, and uses a transformation function to process the multi-resolution features to obtain the final joint features; Upsampling module: used to obtain each final joint feature, align the sizes of the N - 1 joint features and the spatial features output by the first stage of the spatial feature encoder SFE, and output to the spatial and channel reconstruction convolutional module; Spatial and channel reconstruction convolutional module: used to obtain the features input by the upsampling module, perform dynamic weighting processing on each spatial position, adaptively weight the importance of different channels, dynamically adjust the channel allocation, and then obtain the final output features.

7. The image tampering detection system according to claim 6, characterized in that, The spatial and channel reconstruction convolutional module includes: Spatial reconstruction unit: used to convert input features Each spatial location in the data is dynamically weighted to obtain spatial refinement features. ; Channel reconstruction unit: Acquiring spatial refinement features By adaptively weighting the importance of different channels and dynamically adjusting the channel allocation, channel refinement features are obtained. ; Residual connection unit: used to process the original input features With optimized channel refinement features Add them together to obtain the final output features. .

8. The image tampering detection system according to claim 7, characterized in that, The operation method of the spatial reconstruction unit includes separation and cross-reconstruction steps. The processing method of the separation step is: Use the trainable parameters in the group normalization layer to evaluate the spatial information content of different feature maps. Group normalization groups the channels and calculates the mean and variance within each group for normalization to obtain the normalized correlation weights; Distinguish information-rich features from the rest through a gating mechanism, map the normalized correlation weights to the range (0, 1), set a threshold for gating operation, assign weights greater than the threshold to 1, and other weights to 0, obtain the first information weight with a value of 1 and the second information weight with a value of 0, and then multiply the input features by the first information weight and the second information weight respectively to obtain two weighted features; The cross-reconstruction step is performed by combining information-rich features with other features, and then concatenating the two cross-reconstructed features to obtain spatially refined features. .

9. The image tampering detection system according to claim 7, characterized in that, The processing method of the channel reconstruction unit is: (1) Splitting step: refine the spatial features of the input Divided into two parts according to the passage. Channels and The passage, in which By using 1×1 convolutional units to compress the channels of the feature map, spatial features are refined. Divided into first-space refinement features Second space refinement features ; (2) Transformation steps: First spatial refinement features The data is fed into an upper-level extractor for processing, where rich and representative features are extracted to obtain the output features. Second space refinement features The data is fed into a lower-level extractor for processing, generating a feature map with shallow details, resulting in the output feature map. ; (3) Fusion step: Using the SKNet method, the output features from the upper and lower layer extractors are adaptively fused. and .

10. The image tampering detection system according to claim 6, characterized in that, The spatial domain feature encoder (SFE) employs a four-stage HRNet network structure, while the frequency domain feature encoder (FFE) employs a three-stage HRNet network structure. The joint stitching module targets the spatial features extracted from the spatial domain feature encoder (SFE). Frequency domain features extracted from the frequency domain feature encoder FFE First, the joint features are obtained by concatenating the channels according to the corresponding resolution. , , , In the multi-resolution fusion module, a transformation function is used. Processing the multi-resolution feature R yields the joint terminal feature: , , , The specific calculation method is as follows: Where x represents the input resolution index, r represents the output resolution index. When x < r, use (r - x) 3×3 convolutions with a stride of 2 for downsampling operations; when x = r, use a peer operation; when x > r, use bilinear interpolation for upsampling, and then use a 1×1 convolution to align the number of channels.

Images