An open set visual text tampering detection method based on sparse constraint rectified flow

Abstract

The application relates to the technical field of visual text tamper detection, and specifically discloses an open set visual text tamper detection method based on sparse constraint rectified flow, which comprises the following steps: acquiring a real image and constructing training data; constructing a neural network, including a multi-modal forensic tokenizer, an F-DiT encoder and an F-DiT decoder; wherein the multi-modal forensic tokenizer extracts RGB visual block features, spatial rich model filtering features and block discrete cosine transform frequency domain features from an image state, and fuses the features to obtain a multi-modal forensic information feature vector; the neural network is trained by using the training data to obtain a forensic feature network; a query image is acquired, the forensic feature network is used to predict a recovery velocity vector field, a pixel-level tamper probability graph is calculated according to the recovery velocity vector field, and tamper area positioning is realized. The application can realize high-precision positioning of unknown open set text tampering without supervised paired data.

Description

Technical Field

[0001] This invention relates to the field of visual text tampering detection technology, and in particular to an open set visual text tampering detection method based on sparse constrained rectified flow. Background Technology

[0002] Text within images is a crucial medium for conveying semantic information. With the rapid development of generative artificial intelligence, tampering techniques targeting visual text are becoming increasingly difficult to detect with the naked eye, posing a serious threat to digital trust. Current visual text tampering detection systems typically employ a discriminative paradigm in their core processes, modeling the task as a pixel-level semantic segmentation or object detection problem. Their goal is to learn a decision boundary capable of distinguishing between "real regions" and "tampered regions."

[0003] In terms of implementation, early background techniques mainly relied on general-purpose deep learning image forgery detection networks. Subsequent implementations, in order to capture specific text tampering traces, further introduced local texture statistics, used a two-stream architecture for structural decoupling, or utilized frequency domain prior knowledge and character-level feature interactions for modeling, in order to locate tampered regions containing JPEG compression artifacts or GAN fingerprints in complex backgrounds.

[0004] The existing methods have the following problems:

[0005] 1. Poor generalization ability and difficulty in dealing with open set attacks: Existing discriminative models rely heavily on the closed set distribution in the training data, tending to overfit specific forgery patterns. When faced with unknown tampering attacks synthesized by novel generative AI, the system is prone to failure and cannot achieve generalization because its statistical characteristics exceed the decision boundaries learned by the model.

[0006] 2. Generative models suffer from the vanishing gradient problem, akin to "lazy models," in sparse anomaly scenarios: If generative optimal transmission frameworks such as stream matching are directly used for anomaly detection, the computation of the standard objective function is dominated by the vast real background region because the text tampering region is extremely sparse in space (typically occupying less than 5% of the entire image). This leads to the vanishing gradient in the anomaly region, degenerating into a direct identity mapping of the input and output, and failing to effectively recover the tampered region.

[0007] 3. Existing generative vision architectures are prone to losing microscopic evidence traces: Standard diffusion transformer (DiT) architectures typically rely on variational autoencoders (VAEs) to compress images into the latent space to reduce computational cost. This aggressive downsampling operation irreversibly destroys high-frequency microscopic artifacts such as sensor noise and compressed meshes, which are key clues in digital forensics.

[0008] 4. Lack of real-world paired supervised data: In real-world forensic tasks, only altered images are often available, lacking the corresponding original, authentic images. This makes it difficult to directly apply traditional supervised learning methods to train models for trajectory recovery. Summary of the Invention

[0009] This invention aims to address the problems of sparse spatial distribution of tampered regions, easy loss of high-frequency microscopic features, and lack of paired supervision data. To this end, this invention provides an open-set visual text tampering detection method based on sparse-constrained rectified flow. It designs a sparse-constrained rectified flow mechanism, an uncompressed pixel-level multimodal forensics architecture, and a self-supervised artifact injection strategy, thereby achieving high-precision localization of unknown open-set text tampering even without supervised paired data.

[0010] This invention provides an open-set visual text tampering detection method based on sparse-constrained rectified flow, the technical solution of which is as follows:

[0011] S1: Obtain real images and construct training data;

[0012] S2: Construct a neural network for extracting and processing micro-forensic features, the neural network including a multimodal forensic word segmenter, an F-DiT encoder, and an F-DiT decoder;

[0013] Among them, the multimodal forensic word segmenter extracts RGB visual block features, spatial rich model filtering features and block discrete cosine transform frequency domain features from the image state, and fuses them to obtain multimodal forensic information feature vectors;

[0014] Global average pooling is performed on the feature vectors of multimodal forensic information to obtain a global forensic fingerprint; the temporal embedding is combined with the global forensic fingerprint to regress the scaling factor and translation factor; the scaling factor and translation factor are applied to each Transformer block of the F-DiT encoder and F-DiT decoder.

[0015] Both the F-DiT encoder and the F-DiT decoder employ multiple stacked Transformer blocks;

[0016] S3: Train the neural network using the training data to obtain the evidence-gathering feature network;

[0017] S4: Obtain the query image, use the evidence feature network to predict the recovery velocity vector field, calculate the pixel-level tampering probability map based on the recovery velocity vector field, and realize the tampering area location.

[0018] Furthermore, the neural network adopts a U-DiT architecture without VAE compression, combining the long-range dependency modeling of Transformer with the skip connections of U-Net.

[0019] Furthermore, in step S2, the RGB visual block features, spatial rich model filtering features, and block discrete cosine transform frequency domain features are mapped to a shared latent dimension and added element-wise to obtain a multimodal forensics information feature vector.

[0020] Furthermore, the formulas for calculating the scaling factor and translation factor are as follows:

[0021]

[0022] in, Indicates the scaling factor. Indicates the translation factor. This represents a multilayer perceptron. Indicates the current time step The time embedding vector generated by the mapping, This represents a linear projection layer operation. This indicates a global fingerprint forensics.

[0023] Furthermore, the Transformer block incorporates a frequency-gated multi-head self-attention mechanism, calculated as follows:

[0024]

[0025] in, This represents the output features of the attention mechanism. express Activation function Represents the query matrix. Represents the key matrix. Represents a value matrix, Indicates the scaling dimension of the feature. express transpose, Indicates the influence weight. This is the frequency offset matrix, which is used to measure the frequency domain forensic differences between image blocks.

[0026] Furthermore, the formula for calculating the frequency offset matrix is ​​as follows:

[0027]

[0028] in, Represents the elements of the frequency offset matrix. Indicates the first Frequency domain features corresponding to each image patch Indicates the first Frequency domain features corresponding to each image patch This represents a multilayer perceptron.

[0029] Furthermore, in step S3, the network optimization objective is defined as:

[0030]

[0031] in, The loss function represents the sparsely constrained rectified flow. Represents network model parameters, Represents the time variable and image pairs The mathematical expectation, A set representing the spatial pixel locations of an image. Indicates the total number of pixels. Indicates the specific spatial pixel index, This represents the spatial weight at position s. This represents the recovery velocity vector field predicted by the network at the location. The value, This represents the true recovered target vector at position s. Represents a real image. This indicates a damaged image after artifact injection;

[0032]

[0033] in, The hyperparameter representing the control of the penalty intensity. This represents the tampering mask value at position s.

[0034] Furthermore, in step S3, the model training adopts a three-stage course learning: first, pre-training is performed at an artifact ratio of 50%-80%; then, the injection ratio is linearly reduced; and finally, fine-tuning is performed under a true sparsity setting, such as less than 5%.

[0035] Furthermore, in step S1, a text region is randomly selected from the real image, microscopic artifacts are injected, and a corresponding damaged image is generated. The mask is tampered with to record the pixel region where the microscopic artifacts are injected.

[0036] The operations that inject microscopic artifacts include one or more of Gaussian blur, JPEG compression, Gaussian noise, and resampling traces.

[0037] Furthermore, in step S4, the L2 norm of the recovered velocity vector field is calculated in the channel dimension to obtain a pixel-level tampering probability map; the tampering probability map is then thresholded to obtain the tampering region location result.

[0038] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0039] 1. This invention abandons the traditional binary classification discrimination boundary learning and does not learn the features of specific forgery types. Instead, it models the "statistical manifold of real text images" and detects the tampered image by calculating the projection cost of pulling the tampered image back to the real manifold, which greatly improves the open set generalization ability in the face of unknown attacks.

[0040] 2. The Forensic-DiT architecture designed in this invention operates directly in the pixel space, avoiding the loss of high-frequency information caused by VAE compression. It also integrates RGB, spatially rich model SRM and discrete cosine transform (DCT) multimodal features, and combined with a frequency-gated attention mechanism, significantly enhancing the sensitivity to microscopic artifacts.

[0041] 3. This invention proposes a sparsely constrained rectified flow objective function. By introducing a spatially weighted measure parameterized by the tamper mask, the network is forced to predict high-amplitude recovery vectors in sparse tamper regions, effectively overcoming the gradient vanishing problem.

[0042] 4. The self-supervised artifact injection and course learning strategy of the present invention can complete training without actually tampering with image pairs, and has great engineering application value.

[0043] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart of the method provided by the present invention.

[0046] Figure 2 This is a structural diagram of the forensic feature network provided by the present invention.

[0047] Figure 3 This is a structural diagram of the Transformer block of the forensic feature network provided by the present invention.

[0048] Figure 4 This is a comparison chart of different prediction methods provided by the present invention.

[0049] Figure 5 This is a visualization diagram of the feature space manifold projection provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0051] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0052] The following is combined with Figures 1 to 5 The present invention will be further described in detail below, which describes an open set visual text tampering detection method based on sparse constrained rectified flow:

[0053] In this embodiment, as Figure 1 As shown, an open-set visual text tampering detection method based on sparse constrained rectified flow is provided, including the following steps:

[0054] S1: Obtain real images and construct training data.

[0055] This step aims to address the lack of paired supervised data in real-world scenarios. The collected set of real text images is treated as a real manifold distribution.

[0056] For real images Randomly select text regions, inject microscopic artifacts, and generate corresponding damage source distribution images. The operations that inject microscopic artifacts include one or more of Gaussian blur, JPEG compression, Gaussian noise, and resampling artifacts.

[0057] Image compositing is represented by the following formula:

[0058]

[0059] in, Represents a real image. This indicates a damaged image after artifact injection. This represents a tampering mask matrix with values ​​of 0 or 1. , Indicates the height of the image. Indicates the width of the image. This indicates a tampering operation that injects microscopic artifacts. This represents element-wise multiplication of a matrix.

[0060] Using a tampered mask matrix The pixel regions where microscopic artifacts are injected (modified) are recorded, thereby constructing paired data for model training. .

[0061] S2: Construct a neural network for extracting and processing micro-forensic features, the neural network including a multimodal forensic word segmenter, an F-DiT encoder, and an F-DiT decoder.

[0062] To accurately predict the velocity vector field for image reconstruction to the real manifold within the SC-RF (Sparse Conditional Random Field) framework, this embodiment designs a neural network (Forensic-DiT) specifically for extracting and processing microscopic (pixel-level) forensic features, with the structure as follows: Figure 2 As shown, it includes a multimodal forensic word segmenter, an F-DiT encoder, and an F-DiT decoder.

[0063] In existing technologies, standard generative vision architectures (such as standard DiT) typically rely on variational autoencoders (VAEs) to compress images into the latent space, which irreversibly destroys the microscopic features required for digital forensics. Therefore, the Forensic-DiT in this embodiment employs a U-DiT architecture without VAE compression, combining the long-range dependency modeling of Transformers with the skip connections of U-Nets, and operates directly in the pixel space.

[0064] To comprehensively capture traces of tampering, Forensic-DiT designed a multimodal forensic word segmenter at the input end, which analyzes the state of the input image at the current time step t. Extract three features and fuse them:

[0065] (1) RGB visual block features Used to capture the macroscopic semantic content and visual structure of an image; Obtained through image segmentation and linear layers .

[0066] (2) Spatial Rich Model (SRM) Filtering Features : Filter out semantic content of the image and expose inconsistencies in local noise residuals; Obtained through SRM blocks and image segmentation .

[0067] (3) Frequency domain characteristics of Block Discrete Cosine Transform (Block-DCT) : Accurately capture compression grids or spectral anomalies left in the frequency domain by generative tampering or JPEG compression; Obtained through DCT blocks and expansion .

[0068] The three types of features are mapped to a shared latent dimension and then summed element-wise to obtain a fused multimodal forensics information feature vector. The calculation formula is:

[0069]

[0070] in, Constructed using the following formula:

[0071] = .

[0072] Self-fingerprint adaptive normalization and frequency-gated self-attention computation: In the Forensic-DiT process, in order to enable the model to adapt to different background noise levels of images, the input multimodal forensic information feature vector is first processed. Perform global average pooling to obtain the global forensic fingerprint. Then, embed the time. Combined with the global forensic fingerprint, the scaling factor and translation factor are regressed to achieve self-fingerprint adaptive normalization. The calculation formula is as follows:

[0073]

[0074] in, Indicates the scaling factor. Indicates the translation factor. This represents a multilayer perceptron. Indicates the current time step The time embedding vector generated by the mapping, This represents a linear projection layer operation. This indicates a global fingerprint forensics.

[0075] Scaling and translation factors are applied to each Transformer block of the F-DiT encoder and F-DiT decoder. Both the F-DiT encoder and F-DiT decoder use multiple stacked Transformer blocks. For example, a 28-layer Transformer block structure, with the first 18 layers defined as the F-DiT encoder and the last 10 layers as the F-DiT decoder.

[0076] Existing technologies employ standard self-attention mechanisms that tend to aggregate features from semantically similar image patches. In forensic tasks, tampered regions are often semantically extremely similar to the real background, such as having the same background color. This allows the tampered region to "steal" features from the real region to disguise itself, thus smoothing out subtle tampering traces. To address this issue, the Forensic-DiT implementation in this example designs a Frequency-Gated Multi-Head Self-Attention (FG-MHSA) mechanism within the Transformer block, such as... Figure 3 Frequency-gated multi-head self-attention defines the frequency bias matrix. Its elements Measure the first The and the first Frequency domain forensic differences between image patches, where... Represents the elements of the frequency offset matrix. Indicates the first The frequency domain features corresponding to each image patch, i.e., the frequency domain feature extraction operation The extracted corresponding components, Indicates the first The frequency domain features corresponding to each image patch are used. A frequency bias matrix is ​​introduced into the self-attention calculation to forcibly sever connections between image patches with different frequency statistical features. The calculation formula is as follows:

[0077]

[0078] in, This represents the output features of the attention mechanism. express Activation function Represents the query matrix. Represents the key matrix. Represents a value matrix, Indicates the scaling dimension of the feature. express transpose, Indicates the influence weights, used to control the frequency bias matrix. Impact on attention calculation.

[0079] The working process of the frequency-gated multi-head self-attention Transformer block in this embodiment is as follows:

[0080] The input features are layer-normalized, and the layer-normalized features are scaled and shifted using scaling and translation factors. Then, frequency-gated multi-head self-attention is applied. The attention output features are scaled using scaling factors and added to the input features to obtain the residual features.

[0081] After the residual features are normalized by the layers, they are scaled and shifted using scaling and translation factors, and then input into the feedforward network. The output features of the feedforward network are scaled using the scaling factor and then added to the residual features to obtain the output features.

[0082] S3: Model training based on sparse constrained rectified flow: The neural network is trained using training data to obtain the evidence feature network.

[0083] This step trains the network to predict the recovered vector field using a spatially weighted optimal transmission objective function. Due to the extreme spatial sparsity of the tampered target vector field, this embodiment replaces the uniform integral measure of standard stream matching with a sparsely weighted measure parameterized by the tamper mask. The network optimization objective is defined as:

[0084]

[0085] in, The loss function represents the sparsely constrained rectified flow. Represents network model parameters, Represents the time variable and image pairs The mathematical expectation, A set representing the spatial pixel locations of an image. Indicates the total number of pixels. Indicates the specific spatial pixel index, This represents the spatial weight at position s. This represents the hyperparameter controlling the intensity of the penalty, and , This represents the tamper mask value at position s. This represents the recovery velocity vector field predicted by the network at the location. The value, This represents the true recovered target vector at position s.

[0086] Training phase: The neural network receives the damaged image synthesized in step S1 after the injection of artifacts. Real images And the current time step t. Image state The input Forensic-DiT algorithm, after feature extraction by a multimodal forensic segmenter, global forensic fingerprint calculation, and processing by an F-DiT encoder and decoder, outputs the network-predicted recovery velocity vector field. The output is then fed into the SC-RF objective function ( The sparse constraint loss is calculated based on the weighted true recovery trajectory using the tampered mask matrix M, in order to optimize the network parameters of Forensic-DiT. ).

[0087] To ensure stable convergence, the model training adopts a three-stage learning process: first, pre-training is performed at an artifact ratio of 50%-80%; then, the injection ratio is linearly reduced; and finally, fine-tuning is performed under a true sparsity setting, where the true sparsity is set to less than 5%.

[0088] S4: Obtain the query image, use the evidence feature network to predict the recovery velocity vector field, calculate the pixel-level tampering probability map based on the recovery velocity vector field, and realize the tampering area location.

[0089] In the actual test inference phase, given the query image to be detected... Take it as the initial state Inputting the trained forensic feature network, the predicted recovery velocity vector field is obtained. This vector field represents the gradient required to project the input image back to the real manifold. Calculating the L2 norm of the recovered velocity vector field along the channel dimension directly yields a pixel-level tamper probability map. :

[0090]

[0091] in, This represents the predicted tampering probability at position s. Represents the recovery velocity vector field at position The value, Representing vectors Norm calculation.

[0092] Tampering probability diagram After thresholding, the final result of tampering area location is output.

[0093] This embodiment conducted extensive system experiments to verify the effectiveness of the proposed method. The experiments were based on three authoritative text tampering detection benchmark datasets: Tampered-IC13 (scene text), DocTamper (large-scale document images), and OSTF (an open set benchmark containing diverse unseen generative editing patterns).

[0094] Table 1 defines two evaluation settings: Ours(z.) represents the zero-shot setting, where the model is tested directly without fine-tuning on the training set of this dataset, used to evaluate the generalization ability on open sets; Ours(f.) represents the full-shot setting, where the model is fine-tuned using the training set of this dataset, used to evaluate the upper limit of the model's performance. Existing methods such as MVSS-Net, PSCC-Net, DeepLabV3+, and SegFormer were selected, all of which were trained on full-shot datasets. The evaluation metrics selected were F1 (F1 score), IoU (Intersection over Union), and AUC (Area Under the ROC Curve).

[0095] The quantitative comparative experimental results are shown in Table 1. Under the full-sample training setting, our method (Ours (f.)) achieved an average F1 score of 0.869 and an average IoU of 0.838. Compared with the suboptimal proprietary text forensics models (such as TTDMamba and DAF), the average F1 score was improved by 3.2% and the IoU by 6.2%. In particular, in the zero-shot evaluation of the OSTF dataset, which tests the generalization ability of open sets, our method (Ours (z.)) relies on the manifold projection mechanism, and its F1 score (0.781) significantly outperforms several baseline models trained under full supervision, verifying its excellent cross-domain adaptability. A visual comparison of the prediction results of different methods is shown in [Table 1]. Figure 4 The positioning mask generated by this method is more compact and accurate.

[0096] Regarding the mechanism by which this method handles tampered samples (adversarial coordination phenomenon): During inference, this method outputs a restored velocity vector field pointing to the "realistic manifold". When this method actually applies the predicted velocity vector to the tampered image (i.e., updates the image pixels by numerical integration along the vector direction), the model physically smooths and removes microscopic artifacts and high-frequency anomalies contained in the image, forcing its statistical distribution to approach that of the real image.

[0097] Table 2 shows the relevant performance degradation data and Figure 5 The visualization of the feature space manifold projection confirms this process. Figure 5 As shown in (a), there is a significant distribution gap between the tampered samples (red) and the real samples (blue) in the original feature space; Figure 5 Figure (b) shows that after the projection processing described above, the abnormal features of the tampered sample are erased, forcibly pulled back and integrated into the real sample cluster.

[0098] This processing renders most existing detectors (such as DTD and DAF) that rely on these microscopic traces for discrimination completely ineffective (the F1 score drops significantly by more than 0.260 in Table 2). This not only exposes the vulnerability of existing discriminators that rely on shallow cues, but also verifies from a physical mechanism perspective that this method has successfully achieved a deep understanding and accurate reconstruction of the "realistic manifold" of images.

[0099] Table 1

[0100]

[0101] Table 2

[0102]

[0103] This invention proposes a novel paradigm for scene text tampering detection based on real manifold projection. It abandons the traditional approach of relying on learning true / false boundaries for classification, instead generatively modeling the statistical manifold of the real image. By redefining tampering detection as calculating the projection cost (i.e., the recovery velocity vector field) of recovering anomaly regions to the real statistical manifold, this invention completely breaks the limitation of closed-set training data. Since any unknown generative tampering method will cause the statistical features of the image to deviate from the true manifold distribution, this projection cost-based paradigm enables the model to maintain extremely high detection accuracy and robustness when facing novel open-set attacks not seen during training.

[0104] This invention designs a Sparse Constrained Rectified Flow (SC-RF) objective function. This is specifically designed for text tampering regions that are extremely sparse (typically accounting for less than half of the total text). Figure 5 Based on the characteristics of %), this invention reconstructs the optimal transmission objective function for stream matching. By introducing a spatially sparse weighted measure parameterized by the tamper mask, the network imposes a high penalty on the prediction error of small tampered regions when calculating the loss. This design completely solves the gradient vanishing problem in generative models when faced with extremely sparse anomalous signals, where they are easily dominated by a large real background and degenerate into output identity mappings. This allows the model to focus on the high-frequency detail recovery of sparse regions, significantly improving the localization and recall rate of subtle tampering traces.

[0105] This invention proposes a compression-free, frequency-biased multimodal pixel-level network (Forensic-DiT). To avoid the loss of high-frequency features caused by downsampling of variational autoencoders (VAEs) in standard diffusion architectures, this invention employs a U-DiT architecture that operates directly in the pixel space, fusing RGB visual features, noise residual features extracted by a spatial richness model (SRM), and frequency domain anomaly features captured by discrete cosine transform (DCT) at the input. More importantly, this invention designs a frequency-gated multi-head self-attention mechanism (FG-MHSA) within the network, using a frequency-domain bias matrix to forcibly sever attention connections between image patches with different statistical characteristics. This not only effectively extracts microscopic artifacts such as compressed grids and GAN fingerprints, but also prevents semantically similar tampered regions from disguising themselves by stealing features from the surrounding real background at the network structure level, significantly improving the detection rate of highly realistic tampered images.

[0106] This invention proposes a self-supervised artifact injection mechanism combining multiple micro-degradations with a curriculum learning strategy. To achieve robust training without requiring real paired tampered data, this invention randomly injects various degradation traces such as blurring, compression, and noise into the text regions of the original real image to artificially synthesize a fake data source. Based on this, this invention designs a three-stage curriculum learning strategy that transitions from large-area dense artifacts to extremely sparse micro-artifacts. This self-synthesis and "artifact removal" learning process not only solves the problem of data scarcity in the forensic field but also further enhances the model's generalization ability across data sources and tampering tools, ensuring stable convergence of the model when dealing with extremely sparse real-world scenarios.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An open-set visual text tampering detection method based on sparse-constrained rectified flow, characterized in that, include: S1: Obtain real images and construct training data; S2: Construct a neural network for extracting and processing micro-forensic features, the neural network including a multimodal forensic word segmenter, an F-DiT encoder, and an F-DiT decoder; Among them, the multimodal forensic word segmenter extracts RGB visual block features, spatial rich model filtering features and block discrete cosine transform frequency domain features from the image state, and fuses them to obtain multimodal forensic information feature vectors; The multimodal forensic word segmenter analyzes the state of the input image at the current time step t. Extract three features and fuse them: RGB visual block features Used to capture the macroscopic semantic content and visual structure of an image; Obtained through image segmentation and linear layers ; Spatial rich model filtering features : Filter out semantic content of the image and expose inconsistencies in local noise residuals; Obtained through SRM blocks and image segmentation ; Block Discrete Cosine Transform Frequency Domain Characteristics : Capture compression grids or spectral anomalies left in the frequency domain by generative tampering or JPEG compression; Obtained through DCT blocks and expansion ; The three types of features are mapped to a shared latent dimension and then summed element-wise to obtain a fused multimodal forensics information feature vector. The calculation formula is: in, Constructed using the following formula: = ； Global average pooling is performed on the feature vectors of multimodal forensic information to obtain a global forensic fingerprint; the temporal embedding is combined with the global forensic fingerprint to regress the scaling factor and translation factor; the scaling factor and translation factor are applied to each Transformer block of the F-DiT encoder and F-DiT decoder. Both the F-DiT encoder and the F-DiT decoder employ multiple stacked Transformer blocks; S3: Train the neural network using the training data to obtain the forensic feature network; In step S3, the network optimization objective is defined as: in, The loss function represents the sparsely constrained rectified flow. Represents network model parameters, Represents the time variable and image pairs The mathematical expectation, A set representing the spatial pixel locations of an image. Indicates the total number of pixels. Indicates the specific spatial pixel index, This represents the spatial weight at position s. This represents the recovery velocity vector field predicted by the network at the location. The value, This represents the true recovered target vector at position s. Represents a real image. This indicates a damaged image after artifact injection; in, The hyperparameter representing the control of the penalty intensity. This represents the tampering mask value at position s; S4: Obtain the query image, use the evidence feature network to predict the recovery velocity vector field, calculate the pixel-level tampering probability map based on the recovery velocity vector field, and realize the tampering area location.

2. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1, characterized in that, The neural network adopts a U-DiT architecture without VAE compression, combining the long-range dependency modeling of Transformer and the skip connections of U-Net.

3. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1, characterized in that, In step S2, the RGB visual block features, spatial rich model filtering features, and block discrete cosine transform frequency domain features are mapped to a shared latent dimension and added element-wise to obtain a multimodal forensics information feature vector.

4. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1 or 3, characterized in that, The formulas for calculating the scaling factor and translation factor are: in, Indicates the scaling factor. Indicates the translation factor. This represents a multilayer perceptron. Indicates the current time step The time embedding vector generated by the mapping, This represents a linear projection layer operation. This indicates a global fingerprint forensics.

5. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1, characterized in that, The Transformer block internally incorporates a frequency-gated multi-head self-attention mechanism, calculated using the following formula: in, This represents the output features of the attention mechanism. express Activation function Represents the query matrix. Represents the key matrix, Represents a value matrix, Indicates the scaling dimension of the feature. express transpose, Indicates the influence weight. This is the frequency offset matrix, which is used to measure the frequency domain forensic differences between image blocks.

6. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 5, characterized in that, The formula for calculating the frequency offset matrix is: in, Represents the elements of the frequency offset matrix. Indicates the first Frequency domain features corresponding to each image patch Indicates the first Frequency domain features corresponding to each image patch This represents a multilayer perceptron.

7. The method for detecting open-set visual text tampering based on sparse-constrained rectified flow as described in claim 1, characterized in that, In step S3, the model training adopts a three-stage course learning: first, pre-training is performed in the stage of 50%-80% artifact ratio; then the injection ratio is linearly reduced; and finally, fine-tuning is performed under the true sparsity setting.

8. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1, characterized in that, In step S1, a text region is randomly selected from the real image, microscopic artifacts are injected, and a corresponding damaged image is generated. The mask is tampered with to record the pixel region where the microscopic artifacts are injected. The operations that inject microscopic artifacts include one or more of Gaussian blur, JPEG compression, Gaussian noise, and resampling traces.

9. The open set visual text tampering detection method based on sparse constrained rectified flow as described in claim 1, characterized in that, In step S4, the L2 norm of the recovered velocity vector field is calculated in the channel dimension to obtain a pixel-level tampering probability map; the tampering probability map is then thresholded to obtain the tampering region location result.

Images