Text image tampering detection method and device

By adopting high-frequency feature aggregation and multi-scale feature fusion methods in text image tamper detection, combined with semantic information and attention mechanism, the problems of poor detection generalization and low accuracy in the prior art are solved, and more efficient Tamper detection effect.

CN119027788APending Publication Date: 2024-11-26INST OF AUTOMATION CHINESE ACAD OF SCI +1
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202410900407.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-05
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In the prior art, in text image tamper detection, the extracted frequency features have limited ability to express complex and subtle tampering, and the image features have weak ability to perceive global information, resulting in poor detection generalization and low accuracy.

Method used

Using an image tamper detection model based on RGB images, selective scanning and semantic axial attention decoupling are performed through high-frequency feature aggregation module, semantic Mamba module and multi-scale feature fusion decoder. and upsampling, using cross entropy loss function and LOVASZ loss for training, enhance the robustness and accuracy of the detection model.

Benefits of technology

It improves the generalization and accuracy of image tamper detection, can capture tamper traces more comprehensively and meticulously, and enhances the detection ability of complex and subtle tampering.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027788A_ABST
    Figure CN119027788A_ABST
Patent Text Reader

Abstract

The invention provides a text image tampering detection method and device. The method comprises the following steps: acquiring a to-be-detected text image; processing the to-be-detected text image based on the image tampering detection model to obtain a detection result; the image tampering detection model is obtained by training a semantic network model by taking a sample RGB image as a training sample, taking a black and white binary Mask graph as a training label, taking target semantic information as prior information and taking a multi-scale fusion feature as a training feature; the target semantic information is obtained by performing selective scanning, multi-stage semantic axial attention decoupling and up-sampling on the fusion features, and the multi-scale fusion features are obtained by performing selective scanning, multi-stage semantic axial attention decoupling and multi-scale fusion decoding on the fusion features. According to the method, more comprehensive and detailed target representation can be captured, and generalization and accuracy of image tampering detection are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a text image tampering detection method and device. Background Art

[0002] Text with images as carriers effectively conveys dense information and has become one of the most commonly used media for various applications such as digital finance, e-commerce, security auditing, and qualification review.

[0003] As the demand for authentication of online text content becomes increasingly common, users need to upload images containing text (such as certificates, transcripts, contracts, documents, etc.) to various websites to pass authentication. However, some image editing tools or deep learning-based technologies can directly tamper with the text content, posing a security risk to online text content authentication. Therefore, it is crucial to prevent the text content from being tampered with.

[0004] In the related art, a similarity model of frequency features is usually used to improve the performance of the model in the invisible domain, and it is assumed that the high-frequency noise of the image can remove color texture and mine tampering traces, and the image noise is used to improve the generalization ability. However, in the above-mentioned tampering detection process, only the frequency features sensitive to compression artifacts extracted by discrete cosine transform (DCT) have limited expression ability for complex and subtle tampering, and the extracted image features have weak perception ability for global information, resulting in poor generalization of image tampering detection and low tampering detection accuracy. Summary of the invention

[0005] The present invention provides a text image tampering detection method and device, which are used to solve the defects of the prior art in the tampering detection process, that is, the extracted frequency features have limited expression ability for complex and subtle tampering, and the extracted image features have weak perception ability for global information, resulting in poor generalization of image tampering detection, thereby improving the generalization and accuracy of image tampering.

[0006] The present invention provides a text image tampering detection method, comprising: Get the text image to be detected; Processing the text image to be detected based on the image tampering detection model to obtain a detection result; Among them, the image tampering detection model is based on training the semantic network model with sample RGB images as training samples, black and white binary Mask images as training labels, target semantic information as prior information, and multi-scale fusion features as training features; the target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on the image features and high-frequency features extracted from the sample RGB images.

[0007] The present invention also provides a text image tampering detection method, wherein the image tampering detection model includes a high-frequency feature aggregation module, at least two semantic Mamba modules and a multi-scale feature fusion decoder; the at least two semantic Mamba modules are connected in series in descending order according to the corresponding spatial resolution, and each semantic Mamba module includes a VSS layer and a semantic layer; The image tampering detection model is trained through the following steps: The RGB image is tampered and post-processed to obtain a sample RGB image; Extracting RGB features from the sample RGB image based on the high-frequency feature aggregation module, performing DCT transform on the sample RGB image, performing SRM convolution calculation on the transformed image to obtain SRM features, performing BayarConv filtering on the transformed image to obtain BayarConv features, and performing attention calculation on the transformed image, the SRM features and the BayarConv features to obtain the high-frequency features; For each semantic Mamba module, selectively scan the fusion features corresponding to the RGB features and the high-frequency features based on the VSS layer of the semantic Mamba module to obtain a scanning result; perform semantic axial attention decoupling on the scanning result based on the semantic layer of the semantic Mamba module to obtain semantic information output by each semantic Mamba module, and upsample the semantic information output by each semantic Mamba module to obtain the target semantic information; Based on the multi-scale feature fusion decoder, pixel-level segmentation detection is performed on the semantic information output by each semantic Mamba module to obtain the multi-scale fusion feature; The semantic network model is trained according to the sample RGB image, the target semantic information, and the multi-scale fusion features to obtain the image tampering detection model.

[0008] The present invention also provides a text image tampering detection method, wherein the image tampering detection model further includes a dynamic fusion module; The fusion features are obtained through the following steps: Based on the dynamic fusion module, the RGB features and the high-frequency features are fused through spatial attention and channel attention to obtain the fused features.

[0009] The present invention also provides a text image tampering detection method, wherein the image tampering detection model is obtained by training with a combined loss function corresponding to a cross entropy loss function and a LOVASZ loss as a training loss function.

[0010] The present invention also provides a text image tampering detection method, wherein the tampering and post-processing of the RGB image to obtain a sample RGB image comprises: Performing in-frame semantic modification on the RGB image, and adjusting the quality factor of the semantically modified image according to the compression quality factor of the RGB image to obtain a tampered image; The area corresponding to the target size at each position in the tampered image remains unchanged, and the pixel values ​​of other areas except the area of ​​the target size at each position in the tampered image are replaced with the pixel values ​​of the corresponding area of ​​the RGB image to obtain a new tampered image; The new tampered image is compressed using the new quality factor to obtain the sample RGB image.

[0011] The present invention also provides a text image tampering detection method. After obtaining the image tampering detection model, the method further includes: When the sample RGB image is input to the high-frequency feature aggregation module, randomly disturbed additive Gaussian noise and affine transformation are applied to the high-frequency feature aggregation module to obtain disturbance information output by the high-frequency feature aggregation module, and the network parameters of the image tampering detection model are fine-tuned according to the disturbance information to obtain an updated image tampering detection model.

[0012] The present invention also provides a text image tampering detection method, wherein the VSS layer includes a plurality of VSS blocks and a Patch merging structure; The multiple VSS blocks are used to extract global feature representation from the fused features, and the Patch merging structure is used to reduce the height and width of the global feature representation output by the target VSS block and increase the number of channels of the global feature representation output by the target VSS block; the target VSS block belongs to the multiple VSS blocks.

[0013] The present invention also provides a text image tampering detection device, comprising: An image acquisition module is used to acquire the text image to be detected; A detection module processes the text image to be detected based on an image tampering detection model to obtain a detection result; Among them, the image tampering detection model is based on training the semantic network model with sample RGB images as training samples, black and white binary Mask images as training labels, target semantic information as prior information, and multi-scale fusion features as training features; the target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on the image features and high-frequency features extracted from the sample RGB images.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the text image tampering detection method described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the text image tampering detection method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the text image tampering detection method as described above is implemented.

[0017] The text image tampering detection method and device provided by the present invention processes the text image to be detected by obtaining an image tampering detection model through target semantic information and multi-scale fusion feature training to obtain a detection result, and can further mine the tampering traces in the frequency domain by combining high-frequency feature aggregation with the output of various complementary domain extractors. At the same time, multi-scale features are used to characterize feature information at different levels, so that a more comprehensive and detailed target representation can be captured, further improving the generalization and accuracy of image tampering detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is one of the flow charts of the text image tampering detection method provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the process of extracting high-frequency features by the high-frequency feature aggregation module provided by the present invention.

[0021] Figure 3 It is a schematic diagram of the workflow of the semantic Mamba module provided by the present invention.

[0022] Figure 4 This is the second flow chart of the text image tampering detection method provided by the present invention.

[0023] Figure 5 It is a structural schematic diagram of the text image tampering detection device provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Combine the following Figure 1-Figure 6 The text image tampering detection method and device of the present invention are described.

[0027] Figure 1 is one of the flow charts of the text image tampering detection method provided by the present invention, such as Figure 1 As shown, the text image tampering detection method includes the following steps: Step 110: Obtain a text image to be detected.

[0028] Step 120, processing the text image to be detected based on the image tampering detection model to obtain a detection result; wherein the image tampering detection model is based on training the semantic network model with the sample RGB image as the training sample, the black and white binary Mask image as the training label, the target semantic information as the prior information, and the multi-scale fusion feature as the training feature; the target semantic information is obtained by selectively scanning the fusion feature, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion feature is obtained by selectively scanning the fusion feature, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion feature is determined based on the image feature and high-frequency feature extracted from the sample RGB image.

[0029] In the above steps, the black and white binary Mask image can be obtained from the tampering positioning image detected in the history.

[0030] In this embodiment, multiple visual clues in the initial input image are captured by high-frequency features extracted from the sample RGB image; for example, a static SRM filter, a learnable BayarConv filter, and a discrete cosine transform (DCT) can be combined to simultaneously utilize manually designed filters and adaptively learn optimal filter parameters and discrete cosine transform coefficients based on data to capture tampered high-frequency texture patterns, and different types of feature descriptors such as frequency domain, rich features, and Bayesian are utilized to capture the features of text images from multiple perspectives, thereby enhancing the robustness and generalization ability of feature representation.

[0031] In this embodiment, the image tampering detection model utilizes a selectivity mechanism to enable the model to dynamically adjust its state according to the current input, selectively retaining or ignoring information in the sequence, thereby maintaining high efficiency while providing higher quality predictions.

[0032] In this embodiment, the multi-scale fusion features can be obtained through bottom-up feature extraction and top-down feature fusion.

[0033] The text image tampering detection method provided by the embodiment of the present invention processes the text image to be detected by obtaining an image tampering detection model through target semantic information and multi-scale fusion feature training to obtain a detection result. It can further mine the tampering traces in the frequency domain by combining high-frequency feature aggregation with the output of various complementary domain extractors. At the same time, it uses multi-scale features to characterize feature information at different levels, and can capture a more comprehensive and detailed target representation, further improving the generalization and accuracy of image tampering detection.

[0034] In some embodiments, the image tampering detection model includes a high-frequency feature aggregation module, at least two semantic Mamba modules and a multi-scale feature fusion decoder; at least two semantic Mamba modules are connected in series from high to low according to the corresponding spatial resolution, and each semantic Mamba module includes a VSS layer and a semantic layer; the image tampering detection model is trained by the following steps: tampering and post-processing the RGB image to obtain a sample RGB image; extracting RGB features from the sample RGB image based on the high-frequency feature aggregation module, performing DCT transformation on the sample RGB image, performing SRM convolution calculation on the transformed image to obtain SRM features, performing BayarConv filtering on the transformed image to obtain BayarConv features, and filtering the transformed image, SRM features and Attention calculation is performed on the BayarConv features to obtain high-frequency features. For each semantic Mamba module, the fusion features corresponding to the RGB features and high-frequency features are selectively scanned based on the VSS layer of the semantic Mamba module to obtain the scanning results. The scanning results are semantically decoupled based on the semantic layer of the semantic Mamba module to obtain the semantic information output by each semantic Mamba module, and the semantic information output by each semantic Mamba module is upsampled to obtain the target semantic information. Pixel-level segmentation detection is performed on the semantic information output by each semantic Mamba module based on the multi-scale feature fusion decoder to obtain multi-scale fusion features. The semantic network model is trained according to the sample RGB image, target semantic information and multi-scale fusion features to obtain the image tampering detection model.

[0035] In this embodiment, the high-frequency feature aggregation module combines static SRM filters, learnable BayarConv filters and discrete cosine transform (DCT) to simultaneously utilize hand-designed filters and adaptively learn optimal filter parameters and discrete cosine transform coefficients based on data to capture tampered high-frequency texture patterns, thereby enhancing the robustness and generalization ability of feature representation.

[0036] In this embodiment, the high-frequency feature aggregation module includes a high-frequency image acquisition unit, an RGB image acquisition unit, and an image feature enhancement processing unit; wherein the high-frequency image acquisition unit is used to acquire the high-frequency image to be processed as the initial input image, the RGB image acquisition unit is used to acquire the RGB image to be processed as the initial input image, and the image feature enhancement processing unit is used to convert the initial input image into a YCrCb color space, and extract the image DCT coefficient attention map through the frequency domain attention module, and then combine the static SRM filter and the learnable BayarConv filter to capture common high-frequency texture patterns, pass the fused features through a series of convolutional encoders, and finally combine them with the RGB initial input image and send them to the multi-stage semantic Mamba module.

[0037] Figure 2 is a schematic diagram of the process of extracting high-frequency features by the high-frequency feature aggregation module provided by the present invention. Figure 2 In the illustrated embodiment, the SRM filter (SRM block) can be configured as three static high-pass filters that suppress image content and highlight forensic traces and noise; the BayarConv filter (BayarConv block) can be configured to be pre-trained separately in a dual-branch encoder and freeze the weights, and use the tampering traces adaptively learned from the data by the constrained convolution layer as additional noise features; the convolution encoder includes a convolution layer and a non-linear activation layer, which is used to map high-frequency image inputs to a high-dimensional feature domain. Through several groups of basic modules consisting of convolution layers and non-linear activation layers, and by stacking multiple basic modules, the convolution encoder can gradually extract more abstract and advanced feature representations; the DCT block performs a DCT transform on the image to obtain a DCT coefficient, and then applies the attention weight to the DCT coefficient to generate an attention map, and each pixel value of the attention map represents the degree of attention of the model to the corresponding frequency.

[0038] In this embodiment, the high-frequency texture pattern of the high-frequency feature aggregation module is obtained by using SRM to calculate the statistical relationship between the waveform coefficient and its neighbors at each scale and direction and using the BayConv constrained convolution kernel to extract low-level to high-level features at different levels; among them, the high-frequency texture is highly correlated with the image content, and extracting and utilizing these high-frequency patterns helps to improve the performance of the model in detecting tampered images.

[0039] In this embodiment, the frequency domain attention module adopts a window-based self-attention calculation module, which can realize and utilize information interaction between frequency domain features.

[0040] In this embodiment, at least two semantic Mamba modules are used to implement multi-stage selective scanning and semantic content updating; for example, the image tampering detection model includes 4 semantic Mamba modules, each of which is used to perform one stage of selective scanning and semantic axial attention decoupling; wherein the first stage integrates RGB and high-frequency domain complementary information features, and the semantic Mamba modules of the last three stages use the VSS layer and the semantic layer to further extract the global tampering information of the image and model the semantic perception capabilities of the real and tampered categories.

[0041] In this embodiment, the semantic Mamba module is a new type of selective structured state space model. The semantic Mamba module can effectively process long sequence data while maintaining linear time complexity by introducing a selective mechanism, thereby improving the computational efficiency of the model and enhancing the performance of the model in information-intensive tasks.

[0042] In this embodiment, the global dependencies and thus the contextual information in the input sequence are captured from the image through the selective scanning mechanism of the VSS layer, and then the semantic content is decoupled from the feature representation through the semantic layer to generate a semantic prior that captures the semantic context within the feature extractor; based on the segmentation score, the semantic layer updates and guides the features from each Mamba stage, and at the same time provides a semantic prior graph, and the output of the feature extractor in the previous stage is used as the input of the feature extractor in the current stage; the semantic layer uses semantic axial attention to capture the tampered semantic features at different stages, further improving the difference in feature representation.

[0043] In this embodiment, the multi-scale feature fusion decoder obtains multi-scale fusion features through bottom-up feature extraction and top-down feature fusion; specifically, in the bottom-up feature extraction stage, features are extracted from the input image through convolution and pooling operations, and in the top-down feature fusion stage, the feature map of the bottom layer is enlarged to the original scale through upsampling operations and fused with the feature map of the previous layer; this process continues to the top layer, and finally a pyramid-shaped feature map containing features of different scales is obtained. In this way, targets of different scales are processed simultaneously, and the output feature map is decoded to obtain more accurate detection results.

[0044] The text image tampering detection method provided by the embodiment of the present invention captures the key local details of the tampered image through a high-frequency feature aggregation module, captures the global long-range dependency while maintaining the linear complexity through the state-space model Mamba, and can capture the semantic features of tampering at different stages, further enhancing the differentiation of feature representation, and finally, fully utilizes the feature information at different levels through a multi-scale feature fusion decoder to capture a more comprehensive and detailed target representation, further improving the generalization performance of the invented method.

[0045] In some embodiments, the image tampering detection model also includes a dynamic fusion module; the fusion feature is obtained through the following steps: based on the dynamic fusion module, RGB features and high-frequency features are fused through spatial attention and channel attention to obtain the fusion feature.

[0046] In this embodiment, the dynamic fusion module adaptively selects the optimal fusion mode according to different input images and task requirements to obtain fused discriminant features; for example, the dynamic fusion module can obtain the fusion features through the following formula: Z=x*X + y*Y ; in, Z To fusion features, X is the RGB feature, Y It is a high frequency feature; x and yare all weight coefficients. The dynamic fusion module can adaptively learn the fusion weights. x and y It can be obtained by training a sub-network with fused weights.

[0047] In this embodiment, the dynamic fusion module models the interdependence between RGB and high-frequency information images by utilizing spatial and channel attention, so that the model can extract image features more accurately.

[0048] In some embodiments, the image tampering detection model is trained by taking a combined loss function corresponding to a cross entropy loss function and a LOVASZ loss as a training loss function; through the combined loss function, richer global information in the image can be learned, thereby improving the model training performance.

[0049] In some embodiments, tampering and post-processing an RGB image to obtain a sample RGB image includes: performing in-frame semantic modification on the RGB image, and adjusting the quality factor of the semantically modified image according to the compression quality factor of the RGB image to obtain a tampered image; keeping the area of ​​the target size corresponding to each position in the tampered image unchanged, and replacing the pixel values ​​of other areas except the area of ​​the target size in each position in the tampered image with the pixel values ​​of the corresponding area of ​​the RGB image to obtain a new tampered image; and compressing the new tampered image using the new quality factor to obtain a sample RGB image.

[0050] In this embodiment, the new quality factor can be set according to user needs. For example, the new quality factor can be set to 98 or 99.

[0051] In this embodiment, Photoshop can be used to perform semantic modification within the frame of the image to be tampered with, and a copy-move method can be used. If there are text patterns with relatively consistent statistical characteristics in the original image, these text patterns can be used to perform semantic tampering.

[0052] In this embodiment, the post-processing specifically includes: obtaining the compression quality factor of the original image, and saving the tampered image with the same quality factor, which can keep the compression characteristics of the image consistent and reduce the visible traces introduced by the tampering; then comparing the pixel values ​​of the original image and the tampered image to find out the different positions between the two; keeping the 8x8 area corresponding to each position in the tampered image unchanged, and replacing the remaining areas with the pixel values ​​of the corresponding areas of the original image, so that part of the content of the original image can be retained and the visibility of the tampering can be reduced; using MATLAB to compress the image with a quality factor of 98, and then compressing it again with a quality factor of 99. This heavy compression process can blur the image details and reduce the visibility of the tampering traces.

[0053] In this embodiment, the above-mentioned image tampering operation involves identifying text elements in the original image that can change the semantic content after rearrangement within the frame, for example, by applying modifications of different proportions and complexities on different images to achieve the desired effect.

[0054] In this embodiment, the above-mentioned post-processing operation can reduce the visible traces introduced by tampering by maintaining the same compression quality factor, making the tampered image closer to the characteristics of the original image. Selecting a suitable processing scheme can reduce the visibility of tampering while maintaining consistency within the tampered area. Further executing the re-compression process can effectively confuse the image details, making the tampering traces more concealed.

[0055] The text image tampering detection method provided by the embodiment of the present invention performs semantic modification on an RGB image within a frame, and adjusts the quality factor of the semantically modified image according to the compression quality factor of the RGB image to obtain a tampered image; keeps the area of ​​the target size corresponding to each position in the tampered image unchanged, and replaces the pixel values ​​of other areas except the area of ​​the target size in each position in the tampered image with the pixel values ​​of the corresponding area of ​​the RGB image to obtain a new tampered image; compresses the new tampered image using the new quality factor to obtain a sample RGB image, thereby reducing the visibility of the tampering, making the tampering traces more concealed, and thereby improving the quality of the training samples.

[0056] In some embodiments, after obtaining the image tampering detection model, the method also includes: when the sample RGB image is input to the high-frequency feature aggregation module, applying randomly disturbed additive Gaussian noise and affine transformation to the high-frequency feature aggregation module to obtain disturbance information output by the high-frequency feature aggregation module, and fine-tuning the network parameters of the image tampering detection model according to the disturbance information to obtain an updated image tampering detection model.

[0057] In this embodiment, the robustness and adaptability of the model are improved by a conditional enhancement strategy for modeling and training uncertain quantization noise.

[0058] In this embodiment, the conditional enhancement strategy increases the diversity of the quantization table by adding Gaussian noise to the quantization table to simulate the quantization noise caused by actual JPEG compression, so as to expand the distribution of quantization tables seen by the model during training; and configures the random affine transformation to simulate the possible rotation of the image, corresponding to the change of the quantization step size on different frequency components, so as to improve the adaptability of the image tampering detection model to the quantization table under different image contents and different compression quality levels.

[0059] Specifically, the above conditional enhancement strategy maps the original DCT coefficients and quantization table into vector space respectively, introduces high-frequency perturbations to the quantization table, and then quantizes the DCT coefficients. After a series of convolution and downsampling operations, an enhanced frequency domain feature representation is obtained; this strategy simulates the high-frequency distortion introduced by JPEG compression and enables the model to better understand the impact of different quantization levels on image quality.

[0060] Specifically, by enhancing the quantization table with Gaussian noise and affine transformation, the distribution of quantization tables seen by the model during training is more diversified and covers more actual situations. This data enhancement method makes full use of the structural characteristics of the quantization table, so that the image tampering detection model has stronger robustness and adaptability, thereby improving the performance of the model for compressed images; in this way, this embodiment adds Gaussian noise and random affine transformation to the quantization table through a conditional enhancement strategy for modeling and training uncertain quantization noise, thereby guiding the encoder to focus on fine-grained robust tampering traces.

[0061] In this embodiment, the high-frequency feature aggregation module processes the input image by: extracting spatial domain features from the RGB image through continuous deep convolution blocks, converting the RGB image into a YCrCb color space, extracting discrete cosine transform (DCT) coefficients and corresponding quantization tables from the Y channel of the image. The discrete cosine transform regards each pixel in the image as a signal, and converts the signal into a frequency domain signal through a linear combination of cosine functions; after the DCT transform, each pixel in the image is converted into a frequency component; wherein the low-frequency component represents the basic features of the image, such as color and brightness; the high-frequency component represents the detailed features of the image, such as edges and textures; and then, through a conditional enhancement strategy, some disturbances are generated to judge the relative quality of the image, and the prediction results are dynamically selected as the model output, thereby fine-tuning the network and increasing the network's fitting ability for unknown data. The original DCT coefficients and quantization tables are respectively mapped to the vector space and can be expressed by the following formula: ; ; in, and It is an embedding function that maps DCT coefficients and quantization tables to the corresponding vector space. The enhanced quantization table embedding is obtained by adding Gaussian noise with a standard deviation of 0.1 and affine transformation perturbation. The specific calculation process is as follows: .

[0062] Specifically, a set of static SRM convolution kernels are respectively and a set of adaptive BayarConv filters Applied to the input RGB image to generate preliminary feature maps and , and gradually improve these initial features through a series of convolutional layers and activation functions to obtain more expressive feature representations; finally, the input image After passing through the high-frequency feature aggregation module, the fused feature input is obtained .

[0063] The text image tampering detection method provided by the embodiment of the present invention obtains disturbance information output by the high-frequency feature aggregation module by applying randomly disturbed additive Gaussian noise and affine transformation to the high-frequency feature aggregation module, and fine-tunes the network parameters of the image tampering detection model according to the disturbance information to obtain an updated image tampering detection model, so as to enhance the fitting ability of the image tampering detection model for unknown data and further improve the generalization ability of the image tampering detection model.

[0064] In some embodiments, the present invention also provides a text image tampering detection method, the VSS layer includes multiple VSS blocks and a Patch merging structure; the multiple VSS blocks are used to extract global feature representations from fused features, and the Patch merging structure is used to reduce the height and width of the global feature representation output by the target VSS block, and increase the number of channels of the global feature representation output by the target VSS block; the target VSS block belongs to multiple VSS blocks.

[0065] In this embodiment, the VSS layer utilizes a selectivity mechanism to enable the model to dynamically adjust its state based on the current input, selectively retaining or ignoring information in the sequence, thereby maintaining high efficiency while providing higher quality predictions.

[0066] Figure 3 is a schematic diagram of the workflow of the semantic Mamba module provided by the present invention. Figure 3 In the illustrated embodiment, the method for processing the input image by the semantic Mamba module VSS layer includes: stacking multiple VSS blocks and patch merging in a preset order; at each stage, the process generates a hierarchical feature map ,in ; These feature maps are then passed to the decoupled semantic axial attention layer at their corresponding stages; axial attention (including vertical axis attention and horizontal axis attention) is applied along the height and width directions, and the calculation process is as follows: ,

[0067] ,

[0068] ,

[0069] ,

[0070] ,

[0071] Among them, AxialAttnh and AxialAttnw represent the input feature map The axial attention operation is performed along the height and width dimensions; then, the semantic feature map is obtained by applying the channel compression layer, and the calculation process is: ; in, yes The parameters of the convolution layer; finally, the original input feature map and the calculated semantic feature map are combined through an attention gating mechanism to obtain a fused feature map , which can be expressed as: ,

[0072] ,

[0073] in, is the output of the attention gating mechanism, is the sigmoid activation function, From the extended feature map (express and connections along the channel dimension) to the gated feature map Parameters of the convolution.

[0074] The text image tampering detection method provided by an embodiment of the present invention, by setting a VSS layer including multiple VSS blocks and a Patch merging structure, the multiple VSS blocks are used to extract global feature representations from fused features, the Patch merging structure is used to reduce the height and width of the global feature representation output by the target VSS block, and increase the number of channels of the global feature representation output by the target VSS block; it is capable of selectively retaining or ignoring information in a feature sequence to provide the accuracy of the prediction results.

[0075] Figure 4 This is the second flow chart of the text image tampering detection method provided by the present invention. Figure 4In the illustrated embodiment, in the first stage, RGB features and high-frequency features are extracted from the text image (H×W×3) through a high-frequency feature aggregation module, and the RGB features and high-frequency features are fused through a dynamic fusion module to obtain fused features; in the second stage, the input fused features are selectively scanned through the VSS layer (including 2 VSS blocks and a Patch merging structure) of the semantic Mamba module, and the scanning results are axially decoupled by the semantic layer, and the semantic information obtained in the first stage is output; in the third stage, the semantic information of the second stage is selectively scanned through the VSS layer (including 27 VSS blocks and a Patch merging structure) of the semantic Mamba module , the scanning results are axially decoupled through the semantic layer, and the semantic information obtained in the second stage is output; in the fourth stage, the semantic information of the third stage is selectively scanned through the VSS layer (including 2 VSS blocks and Patch merging structure) of the semantic Mamba module, and the scanning results are axially decoupled through the semantic layer, and the semantic information obtained in the fourth stage is output; on the one hand, the semantic information output from the second to the fourth stage is upsampled to obtain the target semantic information, that is, the semantic output; on the other hand, the semantic information output from the second to the fourth stage is multi-scale fused and decoded through the multi-scale feature fusion decoder to obtain the corresponding tampering localization result.

[0076] The text image tampering detection device provided by the present invention is described below. The text image tampering detection device described below and the text image tampering detection method described above can be referenced to each other.

[0077] Figure 5 is a structural schematic diagram of a text image tampering detection device provided by the present invention. Figure 5 In the illustrated embodiment, the text image tampering detection device includes: an image acquisition module 510 and a detection module 520 .

[0078] An image acquisition module 510 is used to acquire a text image to be detected; A detection module 520 processes the text image to be detected based on the image tampering detection model to obtain a detection result; Among them, the image tampering detection model is based on the semantic network model trained with sample RGB images as training samples, black and white binary Mask images as training labels, target semantic information as prior information, and multi-scale fusion features as training features; the target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on the image features and high-frequency features extracted from the sample RGB images.

[0079] The text image tampering detection device provided by the embodiment of the present invention processes the text image to be detected by obtaining an image tampering detection model through target semantic information and multi-scale fusion feature training to obtain a detection result. It can further mine the tampering traces in the frequency domain by combining high-frequency feature aggregation with the output of various complementary domain extractors. At the same time, it uses multi-scale features to characterize feature information at different levels, and can capture a more comprehensive and detailed target representation, further improving the generalization and accuracy of image tampering detection.

[0080] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 complete mutual communication through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the text image tampering detection method, which includes: obtaining a text image to be detected; processing the text image to be detected based on the image tampering detection model to obtain a detection result; wherein the image tampering detection model is based on training the semantic network model with the sample RGB image as the training sample, the black and white binary Mask image as the training label, the target semantic information as the prior information, and the multi-scale fusion feature as the training feature; the target semantic information is obtained by selectively scanning the fusion feature, decoupling the multi-stage semantic axial attention and upsampling, and the multi-scale fusion feature is obtained by selectively scanning the fusion feature, decoupling the multi-stage semantic axial attention and multi-scale fusion decoding; the fusion feature is determined based on the image feature and high-frequency feature extracted from the sample RGB image.

[0081] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the text image tampering detection method provided by the above methods, which includes: obtaining a text image to be detected; processing the text image to be detected based on an image tampering detection model to obtain a detection result; wherein the image tampering detection model is based on training a semantic network model with a sample RGB image as a training sample, a black and white binary Mask image as a training label, target semantic information as prior information, and multi-scale fusion features as training features; the target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on image features and high-frequency features extracted from the sample RGB images.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the text image tampering detection method provided by the above-mentioned methods, the method comprising: obtaining a text image to be detected; processing the text image to be detected based on an image tampering detection model to obtain a detection result; wherein the image tampering detection model is based on training a semantic network model using a sample RGB image as a training sample, a black and white binary Mask image as a training label, target semantic information as prior information, and multi-scale fusion features as training features; the target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on image features and high-frequency features extracted from the sample RGB images.

[0084] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text image tampering detection method, characterized in that: include: Get the text image to be detected; Processing the text image to be detected based on the image tampering detection model to obtain a detection result; The image tampering detection model is obtained by training the semantic network model based on the sample RGB image as the training sample, the black and white binary Mask image as the training label, the target semantic information as the prior information, and the multi-scale fusion feature as the training feature; The target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on the image features and high-frequency features extracted from the sample RGB image.

2. The text image tampering detection method according to claim 1, characterized in that: The image tampering detection model includes a high-frequency feature aggregation module, at least two semantic Mamba modules and a multi-scale feature fusion decoder; the at least two semantic Mamba modules are connected in series from high to low according to the corresponding spatial resolution, and each semantic Mamba module includes a VSS layer and a semantic layer; The image tampering detection model is trained through the following steps: The RGB image is tampered and post-processed to obtain a sample RGB image; Extracting RGB features from the sample RGB image based on the high-frequency feature aggregation module, performing DCT transform on the sample RGB image, performing SRM convolution calculation on the transformed image to obtain SRM features, performing BayarConv filtering on the transformed image to obtain BayarConv features, and performing attention calculation on the transformed image, the SRM features and the BayarConv features to obtain the high-frequency features; For each semantic Mamba module, selectively scan the fusion features corresponding to the RGB features and the high-frequency features based on the VSS layer of the semantic Mamba module to obtain a scanning result; perform semantic axial attention decoupling on the scanning result based on the semantic layer of the semantic Mamba module to obtain semantic information output by each semantic Mamba module, and upsample the semantic information output by each semantic Mamba module to obtain the target semantic information; Based on the multi-scale feature fusion decoder, pixel-level segmentation detection is performed on the semantic information output by each semantic Mamba module to obtain the multi-scale fusion feature; The semantic network model is trained according to the sample RGB image, the target semantic information, and the multi-scale fusion features to obtain the image tampering detection model.

3. The text image tampering detection method according to any one of claims 1-2, characterized in that: The image tampering detection model also includes a dynamic fusion module; The fusion features are obtained through the following steps: Based on the dynamic fusion module, the RGB features and the high-frequency features are fused through spatial attention and channel attention to obtain the fused features.

4. The text image tampering detection method according to any one of claims 1 to 2, characterized in that: The image tampering detection model is trained by taking the combined loss function corresponding to the cross entropy loss function and the LOVASZ loss as the training loss function.

5. The text image tampering detection method according to claim 2, characterized in that: The tampering and post-processing of the RGB image to obtain a sample RGB image includes: Performing in-frame semantic modification on the RGB image, and adjusting the quality factor of the semantically modified image according to the compression quality factor of the RGB image to obtain a tampered image; The area corresponding to the target size at each position in the tampered image remains unchanged, and the pixel values ​​of other areas except the area of ​​the target size at each position in the tampered image are replaced with the pixel values ​​of the corresponding area of ​​the RGB image to obtain a new tampered image; The new tampered image is compressed using the new quality factor to obtain the sample RGB image.

6. The text image tampering detection method according to claim 2, characterized in that: After obtaining the image tampering detection model, the method further includes: When the sample RGB image is input to the high-frequency feature aggregation module, randomly disturbed additive Gaussian noise and affine transformation are applied to the high-frequency feature aggregation module to obtain disturbance information output by the high-frequency feature aggregation module, and the network parameters of the image tampering detection model are fine-tuned according to the disturbance information to obtain an updated image tampering detection model.

7. The text image tampering detection method according to claim 2, characterized in that: The VSS layer includes a plurality of VSS blocks and a Patch merging structure; The multiple VSS blocks are used to extract global feature representation from the fused features, and the Patch merging structure is used to reduce the height and width of the global feature representation output by the target VSS block and increase the number of channels of the global feature representation output by the target VSS block; the target VSS block belongs to the multiple VSS blocks.

8. A text image tampering detection device, characterized in that: include: An image acquisition module is used to acquire the text image to be detected; A detection module processes the text image to be detected based on an image tampering detection model to obtain a detection result; The image tampering detection model is obtained by training the semantic network model based on the sample RGB image as the training sample, the black and white binary Mask image as the training label, the target semantic information as the prior information, and the multi-scale fusion feature as the training feature; The target semantic information is obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and upsampling, and the multi-scale fusion features are obtained by selectively scanning the fusion features, multi-stage semantic axial attention decoupling and multi-scale fusion decoding; the fusion features are determined based on the image features and high-frequency features extracted from the sample RGB image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the text image tampering detection method as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the text image tampering detection method as claimed in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Seismic data interpolation method based on Mama

    CN119622231A

  • Gravel terrain passable area segmentation method

    CN119785025A

  • Document image tampering detection method and system based on double-branch feature fusion

    CN119904879A

  • A Document Image Tampering Detection Method and System Based on Dual-Branch Feature Fusion

    CN119904879B

  • JPEG (Joint Photographic Experts Group) image restoration method based on discrete cosine transform and Mama network

    CN120070268A