A document image tampering detection method based on text aggregation and multi-frequency enhancement

Abstract

This invention discloses a document image tampering detection method based on text aggregation and multi-frequency enhancement, relating to the fields of computer vision, image forensics, and deep learning. The method includes: acquiring the original RGB image to be detected; performing multimodal decomposition on the image to be detected to obtain a DCT coefficient map and corresponding quantization table, high-frequency view, and low-frequency view; inputting the original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view, and low-frequency view into a document image tampering detection model for processing, and outputting the final detection result; wherein, the document image tampering detection model performs feature dimensionality reduction and preliminary text aggregation through a vision-frequency fusion module, uses a multi-frequency feature extractor for encoding and fusion to generate comprehensive frequency features, performs wavelet transform decoupling through a direction-aware frequency decoupling enhancement module, and outputs a tampered region mask based on a decoding prediction module. This invention can more comprehensively reveal hidden tampering artifacts.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, image forensics and deep learning, and in particular to a document image tampering detection method based on text aggregation and multi-frequency enhancement. Background Technology

[0002] With the widespread adoption of smartphones, digital cameras, and various portable photography devices, human society has entered a "visual age." Digital images are no longer merely tools for recording life; they have deeply permeated various fields such as news dissemination, judicial evidence collection, medical diagnosis, scientific research, and social entertainment. Statistics show that billions of images are uploaded to the internet every day. Due to their intuitiveness, ease of dissemination, and large information capacity, images have become the primary carrier of information exchange in modern society. This high degree of reliance means that once the authenticity of an image's content is challenged, its impact will be global and far-reaching. In the past, modifying a photograph required professional darkroom techniques or advanced Photoshop skills. However, with the rapid development of computer vision and artificial intelligence, the threshold for image tampering has been significantly lowered, while the realism has increased exponentially. When "seeing is believing" no longer holds true, a crisis of trust ensues. Realistically altered images can cause multifaceted harm; for example, criminals can use altered ID photos, scanned contracts, or forged scene images to commit online fraud and insurance fraud, causing huge economic losses to individuals and businesses. Therefore, document image tampering detection technology has become an important scientific research field in today's society, with wide application value.

[0003] The core challenge of document image tampering detection lies in its "high concealment" and "weak feature" characteristics: tampering often involves only minor stroke-level changes to key values ​​or characters, accounting for a very small percentage of the overall image, making it highly susceptible to signal loss during downsampling in deep learning models; simultaneously, thanks to the uniformity of the document background and the standardization of its structure, the tampered content blends perfectly with the background in terms of font and texture, lacking the light and shadow or perspective anomalies commonly found in natural images. This makes it difficult for existing algorithms based on general visual features to effectively distinguish between normal printing and scanning defects and malicious tampering, resulting in a high false negative rate and a high false positive rate. Although existing document image tampering detection methods attempt to utilize frequency domain information (such as JPEG DCT coefficients, wavelet transforms, etc.) to enhance visual features, the following major drawbacks remain: 1. Insufficient utilization of frequency information: Most existing methods only utilize limited frequency components (such as using only DCT coefficients or only high / low frequency separation), failing to fully explore the rich tampering traces in the image spectrum and making it difficult to capture subtle tampering artifacts.

[0004] 2. Lack of aggregation mechanism for scattered tampering traces: Text in document images often has a large aspect ratio (e.g., long text lines). When text is tampered with, the tampering traces are usually scattered, existing only in local areas of the tampered text. Existing methods ignore this problem, making it difficult to capture features and textures in specific directions (vertical or horizontal), resulting in the inability to detect complete tampering content when faced with slender text tampering, leading to detection gaps or missed detections.

[0005] Therefore, there is an urgent need for a document image tampering detection method that can comprehensively utilize multiple frequency information and effectively aggregate text features (especially for long and thin text). Summary of the Invention

[0006] This invention addresses the problems of missed detection and incomplete detection caused by the weak and concealed tampering traces and their discrete distribution in long and thin text regions in existing document image tampering detection technologies. It proposes a document image tampering detection method based on text aggregation and multi-frequency enhancement, which achieves high-precision and complete localization of subtle and long and thin tampered text regions in document images.

[0007] On the one hand, to achieve the above objectives, the present invention provides a document image tampering detection method based on text aggregation and multi-frequency enhancement, comprising: The original RGB image to be detected is obtained, and the image to be detected is decomposed into multiple modes to obtain a DCT coefficient map and the corresponding quantization table, a high-frequency view containing edge texture, and a low-frequency view containing smooth background and semantic structure. The original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view are input into the document image tampering detection model for processing, and the final detection result is output. The document image tampering detection model performs feature dimensionality reduction and preliminary text aggregation through a vision-frequency fusion module, encodes and fuses using a multi-frequency feature extractor to generate comprehensive frequency features, performs wavelet transform decoupling and amplifies frequency information in specific directions through a direction-aware frequency decoupling enhancement module, and outputs a tampered region mask based on a decoding prediction module.

[0008] Preferably, multimodal decomposition is performed on the image to be detected, including: The original RGB image is converted to the YCrCb color space, the Y channel is extracted and a block DCT transform is performed to obtain the DCT coefficient map and the corresponding quantization table. The original RGB image is separated into a high-frequency view and a low-frequency view using DCT transformation; The high-frequency view is stitched together with the original RGB image to form a high-frequency enhanced input, and the low-frequency view is stitched together with the original RGB image to form a low-frequency enhanced input.

[0009] Preferably, the vision-frequency fusion module includes: Direction-sensitive branch: Parallel multi-layer asymmetric convolution kernels are used to process the high-frequency enhanced input and the low-frequency enhanced input respectively, aggregating text features in the horizontal and vertical directions; Orientation-independent branch: Use depthwise separable convolution to extract general tampering features that do not have a specific orientation; Feature dimensionality reduction and fusion unit: used to concatenate and reduce the dimensionality of the aggregated horizontal and vertical text features with the general tampering features to generate enhanced high-frequency representation and enhanced low-frequency representation.

[0010] Preferably, the multi-frequency feature extractor includes: Frequency sensing head: used to encode the DCT coefficient map and the corresponding quantization table, and extract DCT frequency features; CNN encoder: used to encode the enhanced high-frequency representation and extract high-frequency features; Transformer encoder: used to encode the enhanced low-frequency representation and extract low-frequency features; Fusion module: used to perform attention-weighted fusion of the high-frequency features, the low-frequency features and the DCT frequency features at several scales to generate the comprehensive frequency features.

[0011] Preferably, the document image tampering detection model uses SegFormer as the backbone network, which is used to inject the comprehensive frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps.

[0012] Preferably, the wavelet transform decoupling and amplification of frequency information in a specific direction via the direction-aware frequency decoupling enhancement module includes: The multi-scale feature maps are subjected to discrete wavelet transform to decouple them into low-frequency approximate sub-bands, vertical high-frequency sub-bands, horizontal high-frequency sub-bands, and diagonal high-frequency sub-bands. The vertical high-frequency subband is enhanced by horizontal asymmetric convolution, wherein the vertical high-frequency subband includes horizontal edge information; The horizontal high-frequency subband is enhanced by vertical asymmetric convolution, wherein the horizontal high-frequency subband includes vertical edge information; The low-frequency approximate subband and the diagonal high-frequency subband are processed using standard square convolution; The enhanced subbands are recombined and fused with the shallow and deep features of the backbone network to obtain optimized features.

[0013] Preferably, the discrete wavelet transform uses the Daubechies-4 wavelet basis function.

[0014] Preferably, the decoding prediction module uses the SegFormer decoding head and jointly optimizes the document image tampering detection model using cross-entropy loss and Lovasz-Softmax loss.

[0015] On the other hand, to achieve the above objectives, the present invention also provides a document image tampering detection system based on text aggregation and multi-frequency enhancement, comprising: Image acquisition unit: used to acquire the original RGB image to be detected, perform multimodal decomposition on the image to be detected, and obtain DCT coefficient map and corresponding quantization table, high-frequency view containing edge texture and low-frequency view containing smooth background and semantic structure; Image processing unit: used to input the original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view into the document image tampering detection model for processing, and output the final detection result.

[0016] Preferably, the document image tampering detection model includes: The visual-frequency fusion module is used to perform feature dimensionality reduction and coarse-grained text aggregation on high-frequency and low-frequency enhanced inputs, and output the enhanced high-frequency and low-frequency representations. Multi-frequency feature extractor: used to encode and fuse the DCT coefficient map and its corresponding quantization table, the enhanced high-frequency representation and low-frequency representation to generate comprehensive frequency features; Backbone network module: used to inject the integrated frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps; Direction-aware frequency decoupling enhancement module: used to decouple the multi-scale feature map by wavelet transform and amplify the frequency information in a specific direction, perform differential convolution enhancement on sub-bands in different directions, and obtain optimized features after recombination; Decoding and prediction module: Used to output the mask of the tampered area.

[0017] Compared with the prior art, the present invention has the following advantages and technical effects: (1) Strong ability to capture minute traces: This invention integrates the high-frequency / low-frequency components of JPEG DCT coefficients and RGB images through a multi-frequency feature extractor. Compared with the existing technology that only uses single frequency information, it can more comprehensively reveal hidden tampering artifacts. (2) Solving the problem of detecting long and thin text: This invention uses a visual-frequency fusion module and a direction-aware frequency decoupling enhancement module to perform text aggregation. Both of them use asymmetric convolution to specifically aggregate features in the horizontal and vertical directions. The visual-frequency fusion module performs coarse-grained aggregation at the input end, and the direction-aware frequency decoupling enhancement module performs fine-grained enhancement at the feature end, which effectively solves the problem of incomplete detection of long and thin text caused by scattered tampering traces. (3) High robustness: The method of this invention has achieved the best results on both DocTamper and RTM datasets, and still maintains superior robustness when facing attacks such as Gaussian noise, blurring, scaling and JPEG compression. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a document image tampering detection method based on text aggregation and multi-frequency enhancement according to an embodiment of the present invention. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0021] This embodiment proposes a document image tampering detection method based on text aggregation and multi-frequency enhancement, such as... Figure 1 ,include: The original RGB image to be detected is obtained, and the image to be detected is decomposed into multiple modes to obtain a DCT coefficient map and the corresponding quantization table, a high-frequency view containing edge texture, and a low-frequency view containing smooth background and semantic structure. The original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view are input into the document image tampering detection model for processing, and the final detection result is output. The document image tampering detection model performs feature dimensionality reduction and preliminary text aggregation through a vision-frequency fusion module, encodes and fuses using a multi-frequency feature extractor to generate comprehensive frequency features, performs wavelet transform decoupling and amplifies frequency information in specific directions through a direction-aware frequency decoupling enhancement module, and outputs a tampered region mask based on a decoding prediction module.

[0022] Further, multimodal decomposition is performed on the image to be detected, including: The original RGB image is converted to the YCrCb color space, the Y channel is extracted and a block DCT transform is performed to obtain the DCT coefficient map and the corresponding quantization table. The original RGB image is separated into a high-frequency view and a low-frequency view using DCT transformation; The high-frequency view is stitched together with the original RGB image to form a high-frequency enhanced input, and the low-frequency view is stitched together with the original RGB image to form a low-frequency enhanced input.

[0023] Specifically, in order to comprehensively capture the traces left by the tampering operation at both the visual and frequency layers, the original input RGB document image is first subjected to multimodal decomposition: JPEG frequency domain feature extraction: The original RGB image is converted to the YCrCb color space, and the Y channel containing the main brightness and structural information is extracted. The Y channel is then divided into blocks (usually 8x8 pixel blocks) and subjected to Discrete Cosine Transform (DCT) to obtain the DCT coefficient map and the corresponding quantization table, which serves as the JPEG frequency domain input reflecting the image compression history and potential inconsistencies.

[0024] High- and low-frequency view separation: The RGB image is converted to the frequency domain using Discrete Cosine Transform (DCT), the spectrum is separated by high-pass and low-pass filters, and then reconstructed by inverse transform (IDCT) to obtain a high-frequency view containing only edge texture and a low-frequency view containing smooth background and semantic structure.

[0025] Multi-view stitching: The separated high-frequency view is stitched with the original RGB image in the channel dimension to form a high-frequency input, which aims to enhance edge artifacts; the low-frequency view is stitched with the original RGB image to form a low-frequency input, which aims to maintain semantic coherence.

[0026] Furthermore, the vision-frequency fusion module includes: Direction-sensitive branch: Parallel multi-layer asymmetric convolution kernels are used to process the high-frequency enhanced input and the low-frequency enhanced input respectively, aggregating text features in the horizontal and vertical directions; Orientation-independent branch: Use depthwise separable convolution to extract general tampering features that do not have a specific orientation; Feature dimensionality reduction and fusion unit: used to concatenate and reduce the dimensionality of the aggregated horizontal and vertical text features with the general tampering features to generate enhanced high-frequency representation and enhanced low-frequency representation.

[0027] Specifically, the obtained high-frequency and low-frequency inputs are fed into the vision-frequency fusion module. This module aims to introduce frequency information in the shallow stages of the model and perform coarse-grained aggregation on the slender geometric features of the text. Direction-sensitive branch: Considering that document text is usually distributed horizontally or vertically linearly, this branch uses an asymmetric convolution kernel (e.g., and The input features are processed separately. Through this elongated receptive field, the network can aggregate features along the text line direction, effectively capturing subtle tampering traces such as inconsistent fonts and abnormal character spacing, while suppressing background noise.

[0028] Direction-independent branches: Depth-separable convolutions using standard square kernels (e.g., Extract general tampering features that do not have a specific direction, such as local blurring and block effects.

[0029] Feature dimensionality reduction and fusion: The features extracted from the two branches above are concatenated and then... The convolutional layer reduces the number of channels to 3, generating enhanced high-frequency and low-frequency representations, which serve as the input foundation for subsequent networks.

[0030] Furthermore, the multi-frequency feature extractor includes: Frequency sensing head: used to encode the DCT coefficient map and the corresponding quantization table, and extract DCT frequency features; CNN encoder: used to encode the enhanced high-frequency representation and extract high-frequency features; Transformer encoder: used to encode the enhanced low-frequency representation and extract low-frequency features; Fusion module: used to perform attention-weighted fusion of the high-frequency features, the low-frequency features and the DCT frequency features at several scales to generate the comprehensive frequency features.

[0031] Specifically, the multi-frequency feature extractor is used to perform deep encoding and fusion of frequency information from different sources: Frequency coding: Using a specially designed frequency sensing head, the extracted DCT coefficient map and the corresponding quantization table are convolutionally encoded to extract statistical anomalies hidden in the compressed domain.

[0032] Dual-stream backbone coding: CNN encoder: Uses a convolutional neural network (CNN) as the backbone to process and enhance the high-frequency representation. It leverages the sensitivity of CNN to local textures to accurately capture edge tampering artifacts.

[0033] Transformer encoder: Uses Transformer as the backbone to process enhanced low-frequency representations, leverages its self-attention mechanism to capture long-distance dependencies, and analyzes the global semantic consistency of the document.

[0034] The fusion module concatenates the high-frequency features extracted by the CNN encoder and the low-frequency features extracted by the Transformer encoder at their corresponding scales, and then performs weighted fusion through an attention mechanism. Simultaneously, DCT features extracted by the frequency-aware head are injected into this module, ultimately generating a comprehensive frequency feature containing multi-level information.

[0035] Furthermore, the document image tampering detection model uses SegFormer as the backbone network, which is used to inject the comprehensive frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps.

[0036] Specifically, in this embodiment, the document image tampering detection model uses SegFormer as the segmentation backbone network. The comprehensive frequency features are injected step-by-step into the four stages of the SegFormer encoder via a feature injection module, thereby obtaining a multi-scale feature map. , , , These feature maps contain both hierarchical semantic information extracted by SegFormer and multimodal frequency domain anomalous signals, which significantly enhances the discriminative power of the features.

[0037] Furthermore, the frequency information in a specific direction is decoupled and amplified by wavelet transform through the direction-aware frequency decoupling enhancement module, including: The multi-scale feature maps are subjected to discrete wavelet transform to decouple them into low-frequency approximate sub-bands, vertical high-frequency sub-bands, horizontal high-frequency sub-bands, and diagonal high-frequency sub-bands. The vertical high-frequency subband is enhanced by horizontal asymmetric convolution, wherein the vertical high-frequency subband includes horizontal edge information; The horizontal high-frequency subband is enhanced by vertical asymmetric convolution, wherein the horizontal high-frequency subband includes vertical edge information; The low-frequency approximate subband and the diagonal high-frequency subband are processed using standard square convolution; The enhanced subbands are recombined and fused with the shallow and deep features of the backbone network to obtain optimized features.

[0038] Specifically, in the decoding stage, to address the feature degradation problem caused by multiple compressions or post-processing of document images, a direction-aware frequency decoupling enhancement module is introduced. This module utilizes the characteristics of wavelet transform for refined processing. Frequency decoupling: The input feature map is subjected to Daubechies-4 (DB4) discrete wavelet transform to decouple it spatially into four sub-bands without loss: LL (Low-Frequency Approximation): Contains the main low-frequency structure of the image; LH (Vertical High Frequency): Contains horizontal edge information; HL (Horizontal High Frequency): Contains vertical edge information; HH (Diagonal High Frequency): Contains high-frequency details of the image.

[0039] Enhanced orientation awareness: Different processing strategies are designed based on the physical characteristics of different subbands: For the LL and HH subbands, standard square convolution is used to process them, preserving the basic structure and diagonal texture; For the HL (horizontal high frequency, vertical edge) subband, a vertical rectangular asymmetric convolution (e.g., k×1) is used to enhance the text edge features and artifacts in the vertical direction, making it more suitable for detecting vertical layouts or table lines. For the LH (vertical high frequency, horizontal edge) subband, a horizontal rectangular asymmetric convolution (e.g., 1×k) is used to enhance the text line features in the horizontal direction and improve the sensitivity to in-line tampering.

[0040] Feature Recombination and Interaction: The enhanced three high-frequency subbands (LH, HL, HH) are spliced ​​and fused to reorganize into the final high-frequency features, and then fused with the low-level visual features of the shallow layers of the network to supplement the lost spatial details. The enhanced LL subbands are used as the final low-frequency features and fused with the high-level visual features of the deep layers of the network to enhance semantic robustness and prevent false detections.

[0041] Furthermore, the decoding prediction module uses the SegFormer decoding head and jointly optimizes the document image tampering detection model through cross-entropy loss and Lovasz-Softmax loss.

[0042] Specifically, the feature map, enhanced and fused with multi-scale information by the direction-aware frequency decoupling enhancement module, is input into the SegFormer decoding head. The decoding head uses a multilayer perceptron (MLP) to unify and fuse the feature maps, and finally restores them to the original image resolution through an upsampling layer, outputting a binary tamper mask, where a pixel value of 1 represents the tampered area and 0 represents the real area, thus achieving high-precision detection of document image tampering.

[0043] This embodiment also provides a document image tampering detection system based on text aggregation and multi-frequency enhancement, including: Image acquisition unit: used to acquire the original RGB image to be detected, perform multimodal decomposition on the image to be detected, and obtain DCT coefficient map and corresponding quantization table, high-frequency view containing edge texture and low-frequency view containing smooth background and semantic structure; Image processing unit: used to input the original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view into the document image tampering detection model for processing, and output the final detection result.

[0044] Furthermore, the document image tampering detection model includes: The visual-frequency fusion module is used to perform feature dimensionality reduction and coarse-grained text aggregation on high-frequency and low-frequency enhanced inputs, and output the enhanced high-frequency and low-frequency representations. Multi-frequency feature extractor: used to encode and fuse the DCT coefficient map and its corresponding quantization table, the enhanced high-frequency representation and low-frequency representation to generate comprehensive frequency features; Backbone network module: used to inject the integrated frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps; Direction-aware frequency decoupling enhancement module: used to perform wavelet transform decoupling on the multi-scale feature map, and perform differentiated convolution enhancement on sub-bands in different directions, and obtain optimized features after recombination; Decoding and prediction module: Used to output the mask of the tampered area.

[0045] To more clearly illustrate the technical solution of the present invention, specific embodiments are provided below for description: The method in this embodiment specifically includes the following steps: Step 1: Data Preprocessing and Multi-View Input Construction In this embodiment, in order to capture subtle and scattered tampering traces in a document image, a multi-view input containing visual and frequency information is first constructed, specifically including the following steps: 1. Obtain the original input RGB image ,in, For the height of the image, The width of the image.

[0046] 2. Convert the original RGB image Convert from RGB color space to YCrCb color space, and calculate the JPEG Discrete Cosine Transform (DCT) coefficient map D and quantization table T from only the Y channel. , .

[0047] 3. Use Discrete Cosine Transform (DCT) to transform RGB document images Separate into high-frequency view and low-frequency view Both are the same size .

[0048] 4. Perform splicing operations at the channel level to construct high-frequency enhanced input. and low-frequency enhanced input ;Specifically, , , To perform the splicing operation based on the channel dimension; at this time, and All dimensions are .

[0049] Step 2: Perform feature dimensionality reduction and preliminary text aggregation using the vision-frequency fusion module: Since pre-trained backbone networks typically only accept 3-channel input, and the text in the document has an extreme aspect ratio, this step uses the visual-frequency fusion module to convert the 6-channel input into 3-channel features and initially aggregates the text features.

[0050] For high frequency enhanced input and low-frequency enhanced input The processing structure is the same, with high-frequency enhanced input. For example, it specifically includes two parallel branches: Orientation-sensitive branch: designed to aggregate coarse-grained text features in both horizontal and vertical directions; First, high-frequency enhanced input After a Convolutional blocks (containing convolutional layers, batch normalization layers BatchNorm, and ReLU activation functions) reduce the number of channels from 6 to an intermediate dimension C. Subsequently, the feature map is passed through four parallel asymmetric convolutional layers: (1) Convolutional layers: aggregate short-range horizontal features; (2) Convolutional layers: aggregate short-range vertical features; (3) Convolutional layers: aggregate long-range horizontal features (simulating long text lines); (4) Convolutional layers: aggregate long-range vertical features (simulating text columns); Finally, through a Convolutional blocks integrate features and output orientation-sensitive feature maps.

[0051] Direction-independent branch: designed to extract local fuzziness and non-directional frequency anomalies. High-frequency enhanced input Firstly, after Dimensionality reduction using convolutional blocks; Subsequently, through a Depthwise separable convolutional blocks extract local spatial features while reducing the number of parameters; Next, a feedforward neural network (FFN) is used to enhance the nonlinear representation of the features.

[0052] Branch fusion: The outputs of direction-sensitive branches and direction-independent branches are concatenated in the channel dimension; Through a The convolutional block fuses and adjusts the channels of the concatenated features, ultimately outputting a fused high-frequency view representation. ; Similarly, for low-frequency enhanced input The same processing is performed to obtain a low-frequency view representation. .

[0053] Step 3: Construct comprehensive frequency features using a multi-frequency feature extractor: This step utilizes three different encoders to process DCT coefficients, high-frequency view representations, and low-frequency view representations respectively, to uncover complementary evidence of tampering.

[0054] 1. DCT frequency coding: Using a frequency-sensing head to process DCT coefficients D and quantization tables T, the frequency-sensing head identifies [data / values] through an embedding layer and a self-attention mechanism. The spectral distribution within the block is abnormal, and the output DCT frequency characteristics are abnormal. Its spatial resolution is the original Figure 1 / 8, meaning the size is ,in, DCT frequency characteristics The number of channels.

[0055] 2. High-frequency / low-frequency feature encoding: High-frequency encoding: Utilizing convolutional neural networks (CNN Blocks) as encoders to process high-frequency view representations. Because CNNs are sensitive to texture details, they are suitable for extracting high-frequency noise traces. Features at two scales are extracted: the first scale feature... (size ) and second-scale features (size ).

[0056] Low-frequency encoding: Using Transformer Blocks as encoders to process low-frequency view representations Leveraging the long-range dependency modeling capabilities of the Transformer, low-frequency illumination and semantic consistency are captured. Features at the corresponding scale are extracted: third-scale features. (size ) and fourth-scale features (size ).

[0057] 3. Multi-source frequency fusion: First-level fusion: Integrating first-scale features With third-scale features The input space-channel attention module is concatenated along the channel dimension, and the first fused feature is output. .

[0058] Second-level fusion: incorporating second-scale features Features of the fourth scale By splicing and merging, the second fusion feature is obtained. .

[0059] Comprehensive fusion: Integrating DCT frequency characteristics With the second fusion feature The data is concatenated along the channel dimension and then fused again through a convolutional layer to obtain a comprehensive frequency feature containing full-spectrum information. The size is ,in, For comprehensive frequency characteristics The number of channels.

[0060] Step 4: Backbone Network Feature Extraction and Injection SegFormer is used as the backbone network, which consists of four stages (Stage 1-4), and frequency features extracted by a multi-frequency feature extractor are injected into the first two stages.

[0061] Phase 1: Input the raw RGB image After passing through SegFormer's Patch Embedding and Transformer Block, the output features are... (size ),Will With the first fusion feature Element-wise addition yields the first-level features. .

[0062] Phase Two: Based on Level One Features The input is downsampled and processed by a Transformer Block, and the output feature is... (size ),Will Combined frequency characteristics Element-wise addition yields the second-level feature. .

[0063] Subsequent stage: Second-level features Continue inputting the third and fourth stages, and output the third-level features respectively. (size ) and fourth-level features (size ).

[0064] Step 5: Optimize features through the direction-aware frequency decoupling enhancement module: To address the issue of scattered text tampering traces in documents, this step enhances the directionality of multi-scale features based on wavelet transform before decoding.

[0065] 1. Wavelet transform decoupling: Features , , , Apply DB4 (Daubechies-4) wavelet transform respectively.

[0066] Each feature is decomposed into four sub-bands: low-frequency approximate sub-band Vertical high-frequency subband Horizontal high-frequency subband Diagonal high-frequency subband In this case, the length and width of all sub-bands are half of the original feature.

[0067] 2. Cross-scale subband aggregation: The frequency fusion module FFM is used to process the four types of subbands respectively.

[0068] The frequency fusion module receives vertical high-frequency subbands at four scales (i=1,2,3,4) after specific convolution. As input, cascaded guided feature aggregation units fuse vertical high-frequency information of different resolutions into a unified feature representation.

[0069] 3. Orientation-aware convolution enhancement: Specific convolutional kernel sequences are designed to enhance features based on the physical characteristics of different subbands: Low-frequency approximate subband and diagonal high-frequency subband: using a standard square convolution kernel, specifically, first passing through... Convolution expands the receptive field, then it passes through the frequency fusion module FFM, and then... Convolution Refinement Features: ; .

[0070] Vertical high-frequency subband (including horizontal edge texture): Employs horizontal asymmetric convolution, first processed... Convolution, then through the frequency fusion module FFM, and then through... Convolution. This elongated convolution kernel is highly matched to the orientation of the horizontal texture. .

[0071] Horizontal high-frequency subband (containing vertical edge texture): employs vertical asymmetric convolution. First, it undergoes... Convolution, then through the frequency fusion module FFM, and then through... convolution: .

[0072] 4. Feature recombination and injection: High-frequency injection: enhancing the vertical high-frequency subband feature Horizontal high-frequency subband aggregation characteristics Diagonal high-frequency subband aggregation characteristics splicing along the channel dimension, through Convolutional fusion yields total high-frequency features Total high-frequency characteristics Compared with first-level features splicing, and through Convolutional dimensionality reduction and fusion yield enhanced shallow features. This operation explicitly restores the lost edge and texture details.

[0073] Low-frequency injection: Aggregates enhanced low-frequency approximate subband features With fourth-level features splicing, and through Convolutional fusion yields enhanced deep features. This operation enhances the model's global robustness against interference.

[0074] Step Six: Decoding Prediction and Loss Function Optimization: 1. Predicted output: The enhanced and processed multi-scale feature set { , , The input is the SegFormer decoding head (SegFormerHead). The decoding head is unified through an MLP layer, upsampled to the same resolution, and then concatenated. Finally, a prediction convolutional layer outputs a probability map of the tampered region. .

[0075] 2. Model Training: The loss function formula is: ; In the formula, Cross-entropy loss is used for pixel-level classification supervision. Lovasz-Softmax loss is used to directly optimize the IoU (Intersection over Union) metric, addressing the issue of sample imbalance between the tampered region and the background region. This is to verify the actual tampering with the region's true label.

[0076] The AdamW optimizer was used to update the parameters, with the initial learning rate set to 0.0001 and a Poly learning rate decay strategy employed.

[0077] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A document image tampering detection method based on text aggregation and multi-frequency enhancement, characterized in that, include: The original RGB image to be detected is obtained, and the image to be detected is decomposed into multiple modes to obtain a DCT coefficient map and the corresponding quantization table, a high-frequency view containing edge texture, and a low-frequency view containing smooth background and semantic structure. The original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view are input into the document image tampering detection model for processing, and the final detection result is output. The document image tampering detection model performs feature dimensionality reduction and preliminary text aggregation through a vision-frequency fusion module, encodes and fuses using a multi-frequency feature extractor to generate comprehensive frequency features, performs wavelet transform decoupling and amplifies frequency information through a direction-aware frequency decoupling enhancement module, and outputs a tampered area mask based on a decoding prediction module. The vision-frequency fusion module includes: Orientation-sensitive branch: Parallel multi-layer asymmetric convolution kernels are used to process high-frequency enhanced input and low-frequency enhanced input separately, aggregating text features in the horizontal and vertical directions; Orientation-independent branch: Depthwise separable convolution is used to extract general tampering features; Feature dimensionality reduction and fusion unit: used to concatenate and reduce the dimensionality of the aggregated horizontal and vertical text features with the general tampering features to generate enhanced high-frequency representation and enhanced low-frequency representation; The multi-frequency feature extractor includes: Frequency sensing head: used to encode the DCT coefficient map and the corresponding quantization table, and extract DCT frequency features; CNN encoder: used to encode the enhanced high-frequency representation and extract high-frequency features; Transformer encoder: used to encode the enhanced low-frequency representation and extract low-frequency features; Fusion module: used to perform attention-weighted fusion of the high-frequency features, the low-frequency features and the DCT frequency features at several scales to generate the comprehensive frequency features; The frequency information is decoupled and amplified by wavelet transform through the direction-aware frequency decoupling enhancement module, including: Discrete wavelet transform is performed on the multi-scale feature maps to decouple them into low-frequency approximate sub-bands, vertical high-frequency sub-bands, horizontal high-frequency sub-bands, and diagonal high-frequency sub-bands. The vertical high-frequency subband is enhanced by horizontal asymmetric convolution, wherein the vertical high-frequency subband includes horizontal edge information; The horizontal high-frequency subband is enhanced by vertical asymmetric convolution, wherein the horizontal high-frequency subband includes vertical edge information; The low-frequency approximate subband and the diagonal high-frequency subband are processed using standard square convolution; The enhanced subbands are recombined and fused with the shallow and deep features of the backbone network to obtain optimized features.

2. The document image tampering detection method based on text aggregation and multi-frequency enhancement according to claim 1, characterized in that, Multimodal decomposition of the image to be detected includes: The original RGB image is converted to the YCrCb color space, the Y channel is extracted and a block DCT transform is performed to obtain the DCT coefficient map and the corresponding quantization table. The original RGB image is separated into a high-frequency view and a low-frequency view using DCT transformation; The high-frequency view is stitched together with the original RGB image to form a high-frequency enhanced input, and the low-frequency view is stitched together with the original RGB image to form a low-frequency enhanced input.

3. The document image tampering detection method based on text aggregation and multi-frequency enhancement according to claim 1, characterized in that, The document image tampering detection model uses SegFormer as the backbone network. The backbone network is used to inject the comprehensive frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps.

4. The document image tampering detection method based on text aggregation and multi-frequency enhancement according to claim 1, characterized in that, The discrete wavelet transform uses the Daubechies-4 wavelet basis functions.

5. The document image tampering detection method based on text aggregation and multi-frequency enhancement according to claim 1, characterized in that, The decoding prediction module uses the SegFormer decoding head and jointly optimizes the document image tampering detection model through cross-entropy loss and Lovasz-Softmax loss.

6. A document image tampering detection system based on text aggregation and multi-frequency enhancement for implementing the document image tampering detection method based on text aggregation and multi-frequency enhancement as described in any one of claims 1-5, characterized in that, include: Image acquisition unit: used to acquire the original RGB image to be detected, perform multimodal decomposition on the image to be detected, and obtain DCT coefficient map and corresponding quantization table, high-frequency view containing edge texture and low-frequency view containing smooth background and semantic structure; The detection image processing unit is used to input the original RGB image, DCT coefficient map and corresponding quantization table, high-frequency view and low-frequency view into the document image tampering detection model for processing, and output the final detection result. The document image tampering detection model includes: The visual-frequency fusion module is used to perform feature dimensionality reduction and coarse-grained text aggregation on high-frequency and low-frequency enhanced inputs, and output the enhanced high-frequency and low-frequency representations. Multi-frequency feature extractor: used to encode and fuse the DCT coefficient map and its corresponding quantization table, the enhanced high-frequency representation and low-frequency representation to generate comprehensive frequency features; Backbone network module: used to inject the integrated frequency features into the SegFormer encoder in stages to obtain multi-scale feature maps; Direction-aware frequency decoupling enhancement module: used to perform wavelet transform decoupling on the multi-scale feature map and amplify the frequency information, perform differential convolution enhancement on sub-bands in different directions, and obtain optimized features after recombination; Decoding and prediction module: used to output the mask of the tampered area; The vision-frequency fusion module includes: Direction-sensitive branch: Parallel multi-layer asymmetric convolution kernels are used to process the high-frequency enhanced input and the low-frequency enhanced input respectively, aggregating text features in the horizontal and vertical directions; Orientation-independent branch: Depthwise separable convolution is used to extract general tampering features; Feature dimensionality reduction and fusion unit: used to concatenate and reduce the dimensionality of the aggregated horizontal and vertical text features with the general tampering features to generate enhanced high-frequency representation and enhanced low-frequency representation; The multi-frequency feature extractor includes: Frequency sensing head: used to encode the DCT coefficient map and the corresponding quantization table, and extract DCT frequency features; CNN encoder: used to encode the enhanced high-frequency representation and extract high-frequency features; Transformer encoder: used to encode the enhanced low-frequency representation and extract low-frequency features; Fusion module: used to perform attention-weighted fusion of the high-frequency features, the low-frequency features and the DCT frequency features at several scales to generate the comprehensive frequency features; The frequency information is decoupled and amplified by wavelet transform through the direction-aware frequency decoupling enhancement module, including: The multi-scale feature maps are subjected to discrete wavelet transform to decouple them into low-frequency approximate sub-bands, vertical high-frequency sub-bands, horizontal high-frequency sub-bands, and diagonal high-frequency sub-bands. The vertical high-frequency subband is enhanced by horizontal asymmetric convolution, wherein the vertical high-frequency subband includes horizontal edge information; The horizontal high-frequency subband is enhanced by vertical asymmetric convolution, wherein the horizontal high-frequency subband includes vertical edge information; The low-frequency approximate subband and the diagonal high-frequency subband are processed using standard square convolution; The enhanced subbands are recombined and fused with the shallow and deep features of the backbone network to obtain optimized features.

Images