A method for locating tampering of a document image based on halftone noise and artifact enhancement

Abstract

The present application relates to a kind of based on halftone noise and artifact enhancement document image tampering positioning method.The method constructs halftone noise perception stream, utilizes denoising residual mechanism to extract the micro halftone statistical law damaged by tampering;Frequency domain perception stream is constructed, decouples luminance and chroma in YCrCb color space, removes low-frequency background by combining adaptive frequency domain enhancement and adaptive high-pass filter, and accurately captures high-frequency quantization artifact;Covariance-based feature fusion mechanism is constructed, spatial domain, frequency domain and halftone noise domain features are cascaded and fused, and the conflict of multi-modal heterogeneity is relieved to accurately locate tampering area.The method is for the authenticity verification scene of document image disturbed by complex post-processing such as JPEG compression, scaling, and good results are obtained in positioning accuracy and robustness.

Description

Technical Field

[0001] This invention belongs to the field of document image forensics and multimedia security technology, and in particular relates to a method for locating document image tampering based on halftone noise and artifact enhancement. Background Technology

[0002] In recent years, digital document images (such as contracts, invoices, and certificates) have become a crucial carrier of information flow in social, economic, and judicial activities. However, with the widespread use of image editing software, malicious users can easily forge key semantic content without leaving obvious visual traces, posing a serious threat to information security. Furthermore, document images typically undergo lossy compression during transmission and storage. This compression often erases subtle traces of tampering, complicating forensic analysis. Therefore, developing robust document image tampering detection technologies is of great significance for maintaining judicial fairness and financial security.

[0003] Significant progress has been made in deep learning-based natural image forensics research, but directly applying it to document image forensics remains a formidable challenge. This is because document images have sparse background textures and semantically rich foreground text, and forgeries often focus on text characters, with the tampered area being much smaller than the tampered object in a natural image. Most existing document image forensics methods rely on convolutional neural networks to directly extract features in the RGB domain. However, in document images, the high-contrast structure of text often masks subtle tampering artifacts (such as edge discontinuities or recompressed noise), limiting the performance of models relying solely on RGB information when faced with complex forgeries. Methods have been developed to mine deeper physical artifacts and multi-scale features to enhance forensic clues. For example, the paper "Weixiang Li, Bin Li, et al. Document imageforgery detection and localization in desensitization scenarios. SignalProcessing, 2025" discloses a document image tampering localization method that extracts multi-scale convolutional features in the RGB spatial domain and combines them with contrastive learning to enhance the discriminability of forged regions. The paper "AV Chuiko, KB Bulatov, et al. Copy-move document image forgery detection and localization based on JPEGclues. International Conference on Machine Vision (ICMV), 2023" solves the problem of copy-move forgery localization in document images by analyzing the differences between DCT coefficients and quantization matrices. The paper "HuiruShao, Kaizhu Huang, et al. Progressive supervision for tampering localization in document images. International Conference on Neural Information Processing (IOCNIP), 2023" introduces noise domain features and combines them with a layer-by-layer enhancement supervision strategy to achieve the localization of weakly tampered regions.

[0004] However, existing multi-stream architectures face three main limitations. First, current noise-based distortion models primarily capture macroscopic geometric inconsistencies from media transfer, neglecting microscopic halftone dot anomalies caused by fine-grained digital tampering. Second, frequency preprocessing typically relies on fixed-parameter filters lacking texture adaptability, introducing noise or losing high-frequency edge cues in flat regions. Third, multimodal fusion often employs simple linear concatenation or addition, ignoring feature heterogeneity and redundancy, making weak physical tampering signals easily suppressed by strong semantic features.

[0005] In view of this, this invention proposes a document image tampering localization method based on halftone noise and artifact enhancement. This method explicitly extracts the microscopic halftone dot distribution patterns disrupted by tampering operations based on a denoising residual mechanism, constructs a frequency-domain perceptual flow in the YCrCb color space, and combines an Adaptive Frequency Enhancement Module (AFEM) to accurately capture high-frequency quantization artifacts and dynamically suppress low-frequency background redundancy. Finally, a covariance-based feature fusion mechanism (CFFM) is introduced to resolve cross-modal feature heterogeneity conflicts, thus enabling accurate localization of subtle tampered areas in document images even when faced with complex post-processing interference. Summary of the Invention

[0006] The purpose of this invention is to provide a document image tampering location method based on halftone noise and artifact enhancement, so as to achieve accurate location of forged document images.

[0007] Existing document image forensics methods primarily rely on the RGB color space, making them highly susceptible to semantic masking by high-contrast text. Furthermore, document images often undergo lossy post-processing such as JPEG compression during storage and social media transmission, making it difficult for traditional methods to capture these subtle low-level artifacts. Halftone technology is a core process in digital document imaging, simulating the visual variations in brightness and darkness of continuous-tone images by adjusting the size or frequency of halftone dots. When forgers tamper with document images, the original halftone pattern is destroyed, leaving inconsistent physical features at the noise level. Moreover, the standard coding space for JPEG compression is YCrCb, which, compared to the highly coupled RGB space, retains the most original and significant compression fingerprint in its chromaticity components. Therefore, extracting halftone noise artifacts and high-frequency quantization artifacts from document images is crucial for solving the problem of document image tampering localization in complex post-processing scenarios. This invention considers the extraction of halftone noise domain information unique to document images, and combines the YCrCb color space with adaptive frequency perception to extract high-frequency quantization artifacts. Finally, a covariance-based feature fusion mechanism maps and fuses multimodal features into a single feature space, ensuring that weak physical tampering fingerprints are not suppressed or overwhelmed when fused with strong semantic features. The method includes:

[0008] (1) Construct a halftone noise perception stream, input the original document image containing halftone texture into the denoising module, separate the high-frequency halftone signal to obtain a smooth denoised image; obtain halftone noise artifact information by calculating the residual between the original tampered image and the denoised image; input the halftone noise artifact information into the noise stream composed of multiple convolutional layers and max pooling layers to extract depth representation features, thereby extracting the microscopic halftone statistical regularity damaged by tampering.

[0009] (2) Construct a frequency domain perceptual flow and use linear projection to convert RGB images to YCrCb color space to explicitly decouple luminance and chrominance components; use the adaptive high-pass filter (AHFP) embedded in AFEM to analyze the discrete cosine transform spectrum to dynamically predict the optimal cutoff frequency and construct a high-frequency prior mask; then, through the adaptive frequency domain enhancement module combined with inverse transform and dual-branch interaction mechanism, adaptively recalibrate feature importance, generate multi-dimensional enhanced evidence features, realize luminance and chrominance decoupling, and highlight high-frequency quantization artifacts related to tampering.

[0010] (3) A feature fusion mechanism based on covariance is constructed, which performs feature fusion in a two-stage cascade manner: the first stage fuses the input RGB image features with the frequency enhancement features output by the frequency domain sensing stream; the second stage fuses the fused features from the first stage with the halftone noise features obtained by the halftone noise sensing stream. This mechanism uses a correlation-aware covariance weighting strategy to model the nonlinear dependency relationship between cross-modal feature channels, identifies and suppresses background redundant channels through intra-feature weighting, and generates global complementary weights by modeling cross-domain correlation through cross-feature weighting, thereby resolving cross-domain heterogeneity conflicts, realizing multimodal information complementarity, and improving the ability to capture subtle tampering traces.

[0011] The details are as follows:

[0012] (1) Construct a halftone noise sensing flow, the schematic diagram of which is shown below. Figure 1 As shown, halftone technology utilizes variations in the density and size of discrete dots to achieve tonal transitions in document images. Since tampering operations cannot perfectly replicate this underlying dot generation logic, the statistical characteristics of the forged region deviate from the global distribution, leaving significant discontinuities at the noise level. These microscopic physical traces are usually independent of macroscopic visual semantics; therefore, halftone noise artifacts can be used to locate document image tampering. To effectively capture halftone noise artifacts and overcome the limitations of traditional fixed filters, which are restricted to specific noise types and susceptible to strong semantic interference, this invention employs a noise extraction mechanism based on the concept of "denoising-residual." The core of this mechanism lies in using a denoising module to explicitly reconstruct smooth visual content without halftone textures, and then using residual operations to eliminate interference from high-contrast text information, thereby accurately separating microscopic halftone noise and its statistical anomalies. A schematic diagram of the denoising module is shown below. Figure 2 As shown.

[0013] Specifically, first, the document image The input is fed into a denoising module consisting of stacked convolutional layers and ResNet blocks, both using a 3×3 kernel size. This module maps the original image, which contains halftone textures, to a smoothed, denoised image. This process separates the original high-frequency halftone signal. Then, the final halftone noise artifact information is obtained by calculating the residual between the original image and the denoised image. To more accurately characterize the noise intensity and maintain consistency with human visual characteristics, a weighted transformation of the RGB channels is introduced in the residual calculation, as shown below:

[0014]

[0015] in, This represents the halftone noise intensity at coordinates (x, y). and These represent the RGB values ​​at the corresponding pixels in the original tampered image and the denoised image, respectively. It is a preset weight vector with values... This vector is used to convert RGB channels to the grayscale domain, explicitly decoupling luminance and chrominance, forcing the network to focus on the morphological density of halftone dots rather than semantic color interference.

[0016] at last, The noise stream is fed into the noise stream to extract depth representation features. A schematic diagram of the noise stream is shown below. Figure 3 As shown, this stream adopts a DenseNet-like structure, consisting of cascaded convolutional layers and max pooling layers, and incorporates dilated convolutions in deeper network layers to expand the receptive field and capture multi-scale halftone inconsistency patterns.

[0017] (2) Construct frequency domain sensing flow, as shown in the schematic diagram. Figure 1 As shown. To overcome the limitation of relying solely on RGB information in capturing subtle compression artifacts, this invention constructs a frequency-domain perceptual stream based on the YCrCb color space and AFEM. JPEG is the most prevalent encoding standard for document image storage and transmission; therefore, most document images undergo JPEG compression, and the nature of tampering usually involves the destruction of the original JPEG block artifacts and quantization statistics. Since the standard color space used in JPEG encoding is YCrCb, which typically involves subsampling of the chrominance components, the YCrCb space retains the most original and significant compression fingerprints compared to the highly coupled RGB space, making potential tampering traces easier to expose. Furthermore, the YCrCb color space is more consistent with human color perception, making it particularly effective in highlighting color differences. Converting RGB images to YCrCb not only improves the model's sensitivity to tamper detection by decoupling luminance and chrominance information, enabling it to better capture subtle detail loss and color anomalies, but also significantly improves the quality of subsequent frequency-domain feature extraction. Specifically, the luminance and chrominance components are explicitly decoupled using the following linear projection:

[0018]

[0019] By utilizing the YCrCb color space, the model can independently handle structural texture and color residuals. This transformation aligns the feature extraction process with the physical mechanisms of image compression standards, thereby significantly amplifying subtle color inconsistencies that are typically masked in the RGB domain.

[0020] The document images exhibit significant frequency domain distribution characteristics: on the one hand, they contain large areas of flat background composed of low-frequency components; on the other hand, the foreground text and table edges have extremely high frequency responses. Traditional frequency domain forensics methods rely on high-pass filters with fixed parameters to extract residuals. However, this fixed-parameter strategy is difficult to perceive local texture changes in document content, often introducing redundant noise in flat background areas or causing the loss of key edge details in dense text areas. Furthermore, tampering artifacts are often unevenly distributed across specific feature channels and spatial locations. To address these issues, this invention designs an adaptive frequency enhancement module (AFEM), such as... Figure 4 As shown, a dual-branch interaction strategy is adopted to dynamically construct high-frequency priors and perform multi-dimensional adaptive weighting of the original features based on the local texture complexity of the document content.

[0021] The core of AFEM lies in dynamically constructing high-frequency priors through embedded AHPF. To adapt the filter to different document texture complexities, AHPF is combined with a lightweight parameter prediction subnetwork, which is responsible for analyzing the input. DCT spectrum Specifically, this sub-network utilizes Global Average Pooling (GAP) to compress spatial information and extracts a global context descriptor reflecting the overall texture complexity of the image. Subsequently, it employs a "squeeze-excitation" style Multi-Layer Perception (MLP) structure to establish a non-linear mapping from texture features to the optimal cutoff frequency: the network first... Convolutional layers form the bottleneck structure to reduce the number of parameters and eliminate global statistical redundancy. Next, a ReLU activation function is introduced to enhance non-linear expressiveness, fitting the complex dependency between document texture complexity and the optimal frequency threshold. This is followed by... The convolutional layer projects features into independent control parameters for height and width dimensions. Finally, the output value is strictly constrained to the (0,1) interval using the sigmoid function and mean calculation. The computation process of this parameter prediction sub-network is defined as follows:

[0022]

[0023] in and the preset benchmark ratio Construct a binary frequency domain mask In this mask, and These represent the coordinate indices of the frequency domain feature map in the height and width directions, respectively (where...) , and (The size of the spectrum corresponding to the input feature map). To achieve a high-pass filtering effect, the low-frequency region in the mask is determined by... Dynamic determination sets the value to 0, while the remaining high-frequency regions are set to 1. The formula is as follows:

[0024]

[0025] Subsequently, through frequency masking and inverse DCT transformation, the enhanced high-frequency information is mapped back to the spatial domain, generating features. This feature recovers the spatial structure while containing only enhanced high-frequency information. To fully utilize... To recalibrate the original features using forensic clues, this invention introduces parallel channel interaction and spatial interaction mechanisms. The channel interaction branch focuses on identifying feature channels rich in forensic clues, capturing global background responses and local anomalous activations by performing GAP and Global Maximum Pooling (GMP) in parallel. The outputs of the two paths are then superimposed and processed... Convolution is used to reduce dimensionality and generate channel attention weights. Meanwhile, the spatial interaction branch utilizes... Convolutional layers nonlinearly map high-frequency features into spatial attention weights. This allows the model to pinpoint the spatial distribution of tampering artifacts. The mask adaptively highlights text edges and splice boundaries with high-frequency inconsistencies while suppressing flat background areas. Finally, the outputs of these two branches are smoothed and fused to generate forensic features with multi-dimensional enhancements. This process achieves deep feature recalibration, enabling the model to suppress redundant background noise while significantly improving its sensitivity to subtle high-frequency tampering artifacts.

[0026] (3) Construct a feature fusion mechanism based on covariance, as shown in the diagram. Figure 1As shown, in a multi-stream forensics framework, features extracted from different branches exhibit significant distributional differences: the RGB stream is rich in high-level semantic information, while the halftone and frequency domain streams emphasize subtle low-level statistical anomalies. Existing linear fusion strategies often lead to strong semantic features dominating gradient propagation, thereby suppressing crucial but weak frequency domain tampering clues and causing subtle physical artifacts to be masked by semantic information. To address the limitations posed by this heterogeneity and to model the deep correlations between multi-domain features, this invention introduces a Correlation-Aware Covariance Weighting (CACW) unit to construct CFFM. The CACW unit explicitly models the nonlinear dependencies between channels by calculating the second-order covariance matrix of the input features. Specifically, CACW reshapes and centers the feature map to calculate the channel covariance matrix, and then maps it to attention weights through an MLP. This process effectively characterizes the redundancy and complementarity of information, dynamically selecting the most discriminative feature channels to achieve accurate document image forgery localization. Based on the CACW unit, CFFM adopts a hierarchical two-level weighting strategy that includes Intra-Feature Weighting (IFW) and Cross-Feature Weighting (CFW) to achieve deep integration of RGB domain features, halftone noise domain features and frequency domain features.

[0027] First, IFW: Each data stream typically contains a large amount of non-discriminatory data, such as dominant text strokes in an RGB stream or consistent background noise in a halftone stream. To address the information redundancy problem within each domain, such as... Figure 5 As shown, by applying CACW units to the input features The IFW module automatically identifies and suppresses these redundant background channels while highlighting channels containing unique forensic fingerprints, thus outputting calibrated multi-domain features. Secondly, CFW: In document images, the response intensity of RGB semantic features is often much higher than that of frequency or noise features, which can easily mask subtle tampering traces. To address the challenges of cross-domain feature heterogeneity and feature amplitude differences, such as... Figure 6 As shown, the CFW module models the cross-domain correlations between the original features to generate normalized global complementary weights. This process effectively resolves the heterogeneity conflict of multi-domain features, ensuring that weak physical tampering fingerprints are reflected in the fused features. It has been effectively preserved and enhanced.

[0028] Through this cascaded processing, from "eliminating intra-feature redundancy" to "complementary enhancement between features," CFFM effectively resolves the heterogeneity conflict between RGB domain, halftone noise domain, and frequency domain features. Finally, the fused features are input into the decoder to restore spatial resolution and generate a localization mask. The decoder employs a cascaded upsampling strategy including transpose layers and dense blocks. Specifically, the transpose layer is used to progressively restore the feature map resolution, while the inserted dense blocks refine local tampering details and enhance feature reuse. Finally, the refined features are processed by the Softmax function to output the prediction mask.

[0029] Compared with existing technologies, the above technical solution has at least the following beneficial effects:

[0030] 1. This invention provides a frequency-domain aware streaming approach. By converting images to the JPEG standard coding space YCrCb, the luminance and chrominance components are explicitly decoupled, fully exposing subtle color anomalies and original JPEG block artifacts that are difficult to capture in the traditional RGB space. Simultaneously, by utilizing the AHPF analysis spectrum embedded in AFEM to dynamically predict the optimal cutoff frequency, adaptive adjustments are achieved for different local texture complexities in documents. This design accurately highlights high-frequency tampering traces while suppressing low-frequency background redundancy noise, significantly improving the model's localization robustness in complex post-processing scenarios such as JPEG recompression and image scaling.

[0031] 2. This invention provides a covariance-based feature fusion mechanism. This mechanism employs a two-stage cascade approach, utilizing a correlation-aware covariance weighting strategy to model the nonlinear dependencies between cross-modal feature channels. By suppressing background redundancy through intra-feature weighting and generating globally complementary weights through cross-feature weighting, it effectively resolves the heterogeneity conflict between the spatial domain, frequency domain, and halftone noise domain, thereby more comprehensively capturing subtle traces of tampering and improving the accuracy of document image tampering detection. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the model of the present invention, "A document image tampering localization method based on halftone noise and artifact enhancement";

[0033] Figure 2 This is a schematic diagram of the denoising module of the halftone noise sensing stream in the present invention "A document image tampering localization method based on halftone noise and artifact enhancement";

[0034] Figure 3 This is a schematic diagram of the noise flow of the halftone noise perception flow in the present invention, "A document image tampering localization method based on halftone noise and artifact enhancement";

[0035] Figure 4This is a schematic diagram of the adaptive frequency domain enhancement module of the frequency domain sensing flow in the present invention "A document image tampering localization method based on halftone noise and artifact enhancement";

[0036] Figure 5 This is a schematic diagram of the feature weighting module based on the covariance-based feature fusion mechanism in the present invention "A document image tampering localization method based on halftone noise and artifact enhancement";

[0037] Figure 6 This is a schematic diagram of the cross-feature weighting module of the feature fusion mechanism based on covariance in the present invention "A document image tampering localization method based on halftone noise and artifact enhancement";

[0038] Figure 7 This is a schematic diagram of halftone noise artifacts in an embodiment of the present invention, "A document image tampering location method based on halftone noise and artifact enhancement";

[0039] Figure 8 This is a schematic diagram of the tampered region mask in an embodiment of the present invention, "A document image tampering location method based on halftone noise and artifact enhancement";

[0040] Figure 9 This is a schematic diagram of the model predicting the tampered area in an embodiment of the present invention, "A document image tampering location method based on halftone noise and artifact enhancement". Detailed Implementation

[0041] This invention is based on document image tampering localization using halftone noise and artifact enhancement. For ease of explanation, this embodiment focuses on the FCTM dataset. This dataset contains 4000 tampered trademark registration certificate images and corresponding labeled mask images. The tampering types include copying, splicing, and removal. Each tampered image undergoes three post-processing operations: adding noise, random cropping, and JPEG compression. The image resolution is approximately 750×820. This invention uses an 8:1:1 ratio to split the training, validation, and test sets for initial training and testing. The specific steps are as follows:

[0042] During training and validation, halftone noise needs to be generated. Specifically, the original document images from the training and validation sets are input into the network, and the constructed halftone noise perceptual flow branch is used to denoise the images. The halftone noise is then obtained through residual processing. The halftone noise image is shown below. Figure 7As shown; simultaneously, combining RGB spatial domain features and high-frequency quantized artifact features extracted from frequency domain perceptual flow, a two-stage cascaded fusion mechanism based on covariance is used for feature fusion and input into the decoder. The model uses a pixel-wise cross-entropy loss function to calculate the difference between the predicted tampered region mask image and the real tampered region mask image. The training settings are a batch size of 8, a maximum number of iterations of 400, an initial learning rate of 0.0001, and a learning rate that remains constant throughout the iterative training process. The SGD optimizer is used to iteratively update the model parameter weights, with the momentum parameter set to 0.9 and the weight decay parameter set to 0.0001. Finally, in the model testing phase, the test set does not need to generate halftone noise; instead, the test set images are directly input into the trained network, and the Softmax function is used to output a schematic diagram of the model's predicted tampered region. The real tampered region mask image is shown below. Figure 8 As shown, the predicted region mask image is as follows: Figure 9 As shown.

Claims

1. A document image tampering localization method based on halftone noise and artifact enhancement, characterized in that, The method includes: A halftone noise sensing stream is constructed to extract the microscopic halftone statistical regularities damaged by tampering operations; a frequency domain sensing stream is constructed to decouple the luminance and chrominance components and highlight tamper-related high-frequency artifacts; a covariance-based feature fusion mechanism is constructed to fuse spatial domain features, frequency domain features and halftone noise domain features through a two-stage cascade approach to achieve multimodal information complementarity.

2. The document image tampering localization method based on halftone noise and artifact enhancement according to claim 1, characterized in that, The construction of a halftone noise-sensing stream and the extraction of microscopic halftone statistical regularities disrupted by tampering operations specifically include: The original document image is input into the denoising module, and the high-frequency halftone signal is separated to obtain a smooth denoised image. Halftone noise artifacts are obtained by calculating the residual between the original tampered image and the denoised image. The halftone noise artifact information is input into a noise stream composed of multiple convolutional layers and max pooling layers to extract depth representation features.

3. The document image tampering localization method based on halftone noise and artifact enhancement according to claim 1, characterized in that, The construction of a frequency-domain sensing stream, decoupling the luminance and chrominance components, and highlighting high-frequency artifacts related to tampering specifically includes: The RGB image is converted to the YCrCb color space to decouple the luminance and chrominance components. The adaptive high-pass filter embedded in the adaptive frequency domain enhancement module is used to analyze the discrete cosine transform spectrum to dynamically predict the optimal cutoff frequency and construct a high-frequency prior mask. The enhanced high-frequency information is mapped back to the spatial domain using the inverse discrete cosine transform. The feature importance is recalibrated through parallel channel interaction branches and spatial interaction branches to generate forensic features with multi-dimensional enhancement effects.

4. The document image tampering localization method based on halftone noise and artifact enhancement according to claim 1, characterized in that, The aforementioned construction of a covariance-based feature fusion mechanism, which fuses spatial domain features, frequency domain features, and halftone noise domain features through a two-stage cascade approach, achieves multimodal information complementarity, specifically including: The first stage of fusion inputs the spatial domain features and the frequency-enhanced frequency domain features output by the frequency domain sensing stream into a covariance-based feature fusion mechanism. The second stage of fusion inputs the feature map fused in the first stage and the halftone noise features obtained by the halftone noise sensing stream into the covariance-based feature fusion mechanism. The feature fusion mechanism uses a covariance weighting strategy to model the nonlinear dependencies between cross-modal feature channels.

Images